利用立體攝影機進行色彩與深度感測以達成三維環境重建及物體追蹤

Three-Dimensional Environment Reconstruction and Object Tracking Using RGB-D Sensing of Stereo Camera

洪中易(Chung-Yi Hung)

指導教授 ： 連豊力

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2013年)

https://doi.org/10.6342/NTU.2013.00625

摘要

三維環境重建是目前一項熱門且應用廣泛的議題，諸如室內環境導覽、虛擬實境以及微創手術之影像導覽系統。立體攝影機同時提供色彩及空間資訊，相較於雷射僅提供空間資訊或單一攝影機提供色彩資訊，更能完整描述環境狀態，提供充足的資訊於三維重建任務上。若能精確地將每一時刻攝影機的相對轉換關係估算出來，立體攝影機量測點便能夠放置在正確的世界座標上，進而建立出三為環境模型。因此首要的任務是利用連續影像上相同特徵點達成立體攝影機的定位。然而，由於立體攝影機的不確定性及錯誤特徵點匹配，不將離群匹配點剃除直接估測攝影機相對姿態將導致定位不精確或是錯誤估測。因此，隨機抽樣一致演算法在此論文中用來作為離群匹配對的剃除。另一方面，由於立體攝影機為被動式感測器，在許多情況如低紋理及光滑材質下，視差影像將產生許多破碎區域，影響三維重建所需的資訊量。因此本論文將提出一個資料前處理的方法，降低量測破碎，進而提高空間重建的品質。
　　此外，考量到動態環境下建置靜態地圖時，必須將動態物偵測出並將其濾除。因此本論文提出了一套物體偵測及追蹤演算法，以機率形式建立佔據網格地圖擷取出候選物體。接著，候選物體利用HSV色彩模型中的色相及飽和度分佈相似性對應到正確的資料庫物體，以解決資料關聯性問題。最後，物體狀態的更新以本論文所提出的更新策略搭配卡爾曼濾波器來達成。實驗結果顯示此系統能夠同時追蹤多重物體，即使物體在一段時間超出攝影機視野或是被遮擋後再被偵測，仍能夠準確追蹤。

關鍵字

立體攝影機 ； RGB-D定位 ； 三維環境重建 ； 物體追蹤 ； 基於可視度之佔據網格

並列摘要

Three-dimensional environment reconstruction is a key technology that has been widely researched over the last decade and has many applications such as indoor environment navigation, virtual reality and visual guidance system for minimal invasive surgery. Stereo camera provides color and spatial information together and therefore is more suitable in 3D environment reconstruction task than other sensors like laser range finder that only provides spatial information or mono camera that only provides color information. Once each camera relative pose is estimated precisely, measurement points provided from stereo camera can be placed at the correct position in the global coordinate to reconstruct the 3D environment model. Thus, the most important task is to achieve the goal of localizing the camera pose by using the same feature points in the consecutive frames. However, because of the uncertainty caused by the stereo camera noise and the feature point mismatching, estimating the camera pose directly without eliminating the outliers could lead to an inaccurate or wrong result. Therefore, Random Sample Consensus (RANSAC) algorithm is applied to solve the outlier problem in this thesis. On the other hand, because of the limitation of the passive type sensor like stereo camera, the disparity map has many missing data areas that occur in several situations such as measuring object in low textureness or glossy surface. This problem may affect the quality of the reconstructed 3D model. Thus, the data preprocessing method is proposed to enhance the 3D reconstruction quality by reducing the missing data areas.
In addition, considering 3D model reconstruction task in dynamic scene, moving object needs to be detected and removed. Therefore, the object detection and tracking method is proposed to detect an object by constructing the occupancy grid map in probability representation to extract object candidate. Then the distributions of hue and saturation in HSV color space are used to link the candidate to the corresponding database object correctly to solve the data association problem. Finally, the proposed update strategy with Kalman filter is used to renew object states. The experiment results demonstrate that the system can track multiple objects simultaneously and even though an object is out of the field of view for a while or is in occlusion, the object can still be tracked correctly.

並列關鍵字

Stereo camera ； RGB-D localization ； 3D environment reconstruction ； object tracking ； visibility-based occupancy grid

參考文獻

Peter Henry, Michael Krainin, Evan Herbst, Xiaofeng Ren, Dieter Fox, “RGB-D Mapping: Using Kinect-style Depth Cameras for Dense 3D Modeling of Indoor Environments,” International Journal of Robotics Research, vol. 31, no. 5, pp. 647-663, April 2012.

[2: Marcincin et al. 2012]

J.Novak-Marcincin, J. Torok, J. Barna, M. Janak, L. Novakova-Marcincinova and V. Fecova, “Realization of 3D Models for Virtual Reality by Use of Advanced Scanning Methods,” in Proceedings of IEEE International Conference on Cognitive Infocommunications, pp. 787-790, December 2-5, 2012.

[3: Park et al. 2012]

[4: Noonan et al. 2009]