針對影片及指尖追蹤應用的立體匹配技術

Stereo Matching Techniques for Videos and Fingertip-tracking Applications

許嘉容(Chia-Jung Hsu)

指導教授 ： 王家祥

國立清華大學/電機資訊學院/資訊工程學系/碩士(2014年)

摘要

三維立體視覺的應用中，如何得到準確的深度資訊是一大關鍵議題，其中立體匹配演算法是解決此問題的方法之一。本論文提出一個利用條件式傳遞(weighted propagation)的方式綜合顏色和空間結構的資訊，並針對邊界、遮蔽和信賴度低的區域做修正，得到視差圖做為深度資訊。此方法可以針對不同的應用做彈性調整達到内容感知(content-aware)及加速的效果。第一，應用於影片，加入運動向量的考量達到時間上的連續性，並且利用內差法達到加速的效果；第二，應用於指間追蹤，導入皮膚色的特質使手部的深度資訊更趨完善，同時減少深度等級的運算降低運算量。實驗結果顯示，我們提出的方法可以快速且準確的得到視差圖，且針對不同的應用皆突顯我們的優勢。

關鍵字

區域性立體匹配 ； 視差圖 ； 深度圖 ； 時間上的一致性 ； 加權傳播

並列摘要

How to get accurate depth information is an essential issue in computer vision. Stereo matching has been proved to be the effective way to compute such dense and reliable disparity maps. In this thesis, several stereo matching algorithms are proposed to compute disparity maps for several kinds of applications, such as videos, fingertip-tracking, etc. Basically, these algorithms are local approach with the concept of weighted propagation combining both color and spatial structure simultaneously. Also, several refinement techniques are joined to improve the accuracy on border, occluded and unreliable pixels. Furthermore, for the applications of video processing, the issue of temporal consistency is considered and solved. And, for the fingertip-tracking applications, the skin-color is heavily weighted to enhance the data integrity within the hand regions. The experimental results show that the proposed algorithms provide comparable high-quality disparity maps for both images and videos and suitable for some applications, such as fingertip-tracking.

並列關鍵字

local stereo matching, ； disparity map ； temporal consistency ； weighted propagation ； fingertip-tracking

參考文獻

[1] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” International Journal of Computer Vision, vol. 47, no. 1-3, 2002, pp 7-42. http://www.middlebury.edu/stereo/

[2] Q. Yang, L. Wang, R. Yang, H. Stewénius, and D. Nistér, “Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 3, March 2009, pp.492-504.

[3] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions using Graph Cuts,” IEEE International Conference on Computer Vision (ICCV), vol.2, 2001, pp. 508 - 515.

[4] Z. Wang and Z. Zheng, “A region based stereo matching algorithm using cooperative optimization,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2008, pp. 1-8.

[5] O. Veksler, “Stereo correspondence by dynamic programming on a tree,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2005, pp. 384 -390.