利用相對應的彩色影像結構與特徵點匹配進行深度圖復原

Depth Image Recovery Based on Correlative Color Image Structure and Feature Matching

簡偉哲(Wei-Che Chien)

指導教授 ： 連豊力

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201700139

摘要

隨著三維感測器的快速發展，三維空間技術也被利用在許多方面，而在三維數據方面，深度圖的品質好壞嚴重地影響了應用方面的成效。有別於雷射僅提供空間資訊或單一攝影機僅提供色彩資訊，微軟的Kinect感測器能同時提供色彩及空間資訊，更能完整地描述周遭環境狀態。在影像數據方面，空間資訊的取得並不像硬體設備已趨成熟的彩色資訊來得完整，其中一種空間資訊取得的方法乃是藉由光線反射時間差來估測環境中的深度資訊，因遮蔽物阻擋導致待測物無法被探測光線照射到、過於光滑或反光的待測物表面都可能會造成空間資訊的取得錯誤。在本文中，我們提出一個修補深度資訊的方法，乃是藉由不同位置所截取到的四維資料集，使用後取得的影像深度資訊去填補前一位置的深度影像中缺乏深度資訊的部分。藉由影像疊合的演算法中基於彩色影像上的特徵點去尋找兩張影像相同的部分並假定存在一轉換關係使後取的影像能藉此轉換矩陣疊合到前一影像上。利用匹配特徵點與隨機抽樣演算法去計算得夠獲得最大內群數的轉換矩陣，接著再將此轉換矩陣使用在深度影像中，將後取得的深度影像藉由此轉換矩陣轉換至前一深度影像座標下，前一深度影像中的破碎點即可由後一深度影像中的深度值來填補。利用感測器的移動來取得多個不同位置的四維資料集，盡可能地建立出一完整的深度影像。

關鍵字

RGB-D感測器 ； 深度修補 ； 破碎點填補 ； 特徵點 ； 隨機抽樣演算法

並列摘要

With the quickly development of three-dimensional scanners, three-dimensional techniques are used for many applications. Focus on data in three-dimensional, the quality of depth image has a significantly influence in the implementation of 3D applications. Unlike sensors such as mono camera which can only provide color information or laser range finder that only provides spatial information, RGB-D sensor like Microsoft Kinect can provide color and spatial information simultaneously that can display the surrounding environment more complete. One of a method to acquire spatial information is estimating the surrounding depth information based on the known speed of light, measuring the time-of-flight of a light signal between the sensor and objects for each point of the image. Another method named Light Coding technique uses infrared to mark objects by speckles, and each speckle has a unique shape to represent spatial information. Due to the physical offset or interference noises like shiny surfaces, smooth surfaces or boundary of objects, the acquisition of depth information may be incorrect.
This thesis proposes a depth information recovery method that uses the latter depth image to fill holes region in the former depth image by the RGB-D data captured from different location. Based on the feature points in two color images and transfer one to the other, the algorithm can find overlapping areas of these two images. Using the matched feature points and Random Sample Consensus (RANSAC) to estimate transformation matrix with maximum number of inliers, the transformation matrix can be utilized in depth images afterwards. Then, the holes region in former depth image can be filled by depth information in transferred latter depth image.

並列關鍵字

RGB-D sensor ； Depth recovery ； Hole filling ； Feature ； RANSAC

參考文獻

[1: Yang et al. 2012]

[2: Han et al. 2013]

J. Han, L. Shao, D. Xu, and J. Shotton, “Enhanced Computer Vision with Microsoft Kinect Sensor: A Review,” IEEE Transactions on Cybernetics, vol. 43, no. 5, Oct. 2013.

[3: Henry et al. 2012]

P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox, “RGB-D mapping - Using Kinect-style depth cameras for dense 3D modeling of indoor environments,” The International Journal of Robotics Research, vol. 31, no. 5, pp. 647-663, February 10, 2012.