以多光譜立體視覺技術提升RGB-D攝影機在室外環境下深度估測準確度

Extending Structured-light RGB-D Cameras in Outdoor Scenes via Cross-spectral Stereo

沈贇珺(Yun-Jun Shen)

指導教授 ： 王傑智

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2015年)

https://doi.org/10.6342/NTU201601331

摘要

近年來，低成本的RGB-D攝像機例如微軟的Kinect，華碩的Xtion正逐漸變為主流的研究設備。然而，絕大部分這樣的設備是為室內環境使用而設計的。當這些設備移到室外環境下工作時，它們往往會出現各種不穩定的狀況，如深度測量距離變短，深度圖像破碎等。這是由於這些設備的工作機制是依賴於檢測其所投影的紅外圖像的扭曲程度。相比於室內環境，室外環境下環境光中的紅外光強度較高。較強的環境光使得RGB-D攝影機難以捕捉到投影的紅外圖像，從而導致無法計算深度信息。
在此論文中，我們討論了如何利用RGB-D攝影機自身的紅外攝影機以及彩色攝影機進行立體視覺匹配來恢復室外環境下丟失的深度信息。對紅外圖像以及彩色圖像進行立體視覺匹配來恢復室外環境下丟失的深度信息。對紅外圖像以及彩色圖像進行立體視覺匹配是一項富有挑戰的任務。由於紅外光譜和可見光譜之間的重疊較小，使得同樣的物體在不同光譜攝影機下有著不同的色彩表現。在這篇論文中，我們提出了一種基於物體材質信息的多光譜色彩轉換方法。並通過對轉換後的彩色圖像，原始紅外圖像以及原始深度圖像進行聯合視差信息恢復的方式來提升RGB-D攝影機在室外環境下的表現。為了評估所提出的方法，我們利用RGB-D攝影機採集了一組全新的影像數據庫。對我們採集的數據庫進行試驗後，結果證明此論文所提出的方法的確有效地恢復了絕大部分RGB-D攝影機所缺失的深度信息。

關鍵字

多光谱 ； 立體視覺 ； 材質 ； RGB-D攝影機

並列摘要

Structured-light based 3D cameras such as Microsoft's Kinect or Asus's Xtion are popular low-cost RGB-D sensors recent years. However, most of these sensors are assumed to be used in indoor environments with moderate ambient light. Once these devices are taken to outdoor scenes, the bright sunlight makes the projected pattern obscure to be seen and causes the dramatic reduction of working range. While IR pattern disappears in sunlight, the background becomes bright and clear in IR image. This brings the opportunity to use stereo algorithms on RGB and IR image to recover the unmeasured depth.
In this work, we investigate the possibility of recovering the unmeasured depth information of the structured light device via stereo matching in outdoor scenes. Densely matching RGB and IR images is a challenging task since they represent the information in two almost non-overlapped spectrums. Different from other edge-based cross spectral stereo approaches, we analyze the camera imaging model and found the hidden relation between the RGB and IR spectrum in material level. Based on this relation, we propose a material-based color conversion method to make the cross-spectral problem become a general stereo problem. In addition, we also introduce a way to utilize depth information in the stereo disparity optimization stage. To evaluate our method, an outdoor dataset is collected via Xtion. The experiment results show that the proposed method works well the estimating of depth for the regions Xtion failed to compute.

並列關鍵字

Cross-spectral ； Stereo ； RGB-D Camera ； Material

參考文獻

Alhwarin, F., Ferrein, A., & Scholl, I. (2014). Ir stereo kinect: improving depth images by combining structured light with ir stereo. In PRICAI 2014: Trends in Artificial Intelligence (pp. 409–421). Springer.

Ben-Ari, R. & Sochen, N. (2010). Stereo matching with mumford-shah regularization and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 2071–2084.

Bensrhair, A., Mich´e, P., & Debrie, R. (1996). Fast and automatic stereo vision matching algorithm based on dynamic programming method. Pattern Recognition Letters, 17(5), 457–466.

Bleyer, M. & Gelautz, M. (2007). Graph-cut-based stereo matching using image segmentation with symmetrical treatment of occlusions. Signal Processing: Image Communication, 22(2), 127–143.

Chiu,W.-C., Blanke, U., &Fritz, M. (2011a). I spy with my little eye: Learning optimal filters for cross-modal stereo under projected patterns. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), (pp. 1209–1214).