利用高解析度彩色影像提升時差測距深度圖的品質

TOF-Sensor Captured Depth Map Refinement Using High-Resolution Color Images

陳敬昆(Ching-Kun Chen)

指導教授 ： 杭學鳴

國立交通大學/電機學院/電子工程系所/碩士(2014年)

https://doi.org/10.6842/NCTU.2014.00746

摘要

隨著人機互動(Human-Machine Interaction)的發展，人們可以更直覺地操作電腦。Microsoft在2010年發表的Kinect for Xbox 360可以視為人機互動發展的里程碑。玩家不需要任何穿戴裝置，就可藉由簡單的肢體動作對Xbox 360下達指令。而人們可以有如此豐富的遊戲體驗都要歸功於Kinect。Kinect是一台深度攝影機，它會依照距離偵測前方物體的深度值並轉譯成深度圖。雖然Kinect深度圖的解析度非常小，卻是Xbox 360辨識人體動作的重要資訊來源。深度圖的品質對於電腦識別的結果有很大的影響，所以如何提升深度圖的解析度已經成為一個重要議題。
在本實驗中，我們使用MESA Imaging在2008年生產的時差測距(Time-of-Flight)深度攝影機SR4000來擷取深度資訊。然而SR4000的解析度只有176×144，如此低的解析度會限制許多深度圖應用的發展，因此我們使用一台超高解析度彩色攝影機Flea3把SR4000的解析度提升到跟彩色攝影機一樣高。Flea3是Point Grey在2012年生產的超高解析度彩色攝影機，它最高的解析度可以達到4096×2160。
我們提出一個「深度精煉演算法」，它可以藉由高解析度彩色照片來提升深度圖的低解析度。只要取得同一個視角的深度與彩色影像，深度精煉演算法就可以把原本又小又模糊的深度圖轉變成高解析度深度圖。深度精煉演算法可以藉由參考彩色影像來得知正確的物體邊界，進而正確地修正深度圖。因為這個演算法只適用於小範圍的影像，所以深度圖與彩色影像會被各別拆解成好幾個小塊再做處理。最後我們拿我們提出的高解析度深度圖與傳統內插法產生的深度圖相比，實驗結果顯示，深度精煉演算法可以產生更自然的高解析度深度圖。

關鍵字

深度圖 ； 時差測距 ； 高解析度深度圖

並列摘要

Kinect for Xbox 360 made by Microsoft Company in 2010 is a milestone for Human-Machine Interaction (HMI). Without any sensor wearing on body, players can send commands to Xbox 360 directly by simple limb motion. Kinect has a depth camera for producing depth maps as object descriptor. Its depth map has a rather low resolution and for many other applications, we need high quality depth maps. Hence, how to improve the resolution of depth maps has become a major research topic.
In our experiments, we use a Time-of-Flight (ToF) camera (SR4000) to get the depth information. However, the resolution of SR4000 is only 176×144 pixels. We thus use a high resolution color camera to collocate with SR4000, and we wish to improve the depth map resolution based on the high resolution color images. Flea3 is used in our experiments which is produced by Point Grey Company in 2012. Its maximum resolution is 4096×2160.
We propose a depth refinement algorithm to enhance the low resolution depth maps using high resolution color images. Align depth maps and color images at the same view point, the depth refinement algorithm can transform the small and blurred depth map into high resolution one. Based on the color images, our depth refinement algorithm can extract the exact object edges and revise the depth maps correctly. Because this algorithm is suitable for small, local regions, the depth maps and color images are divided into several patches and are processed individually. Finally, we compare the interpolated depth maps with the proposed high resolution depth maps. Experimental results show that our depth refinement algorithm can produce a good quality, high resolution depth map.

並列關鍵字

depth map ； TOF-Sensor ； high resolution depth map

參考文獻

[4] K. Xu, J. Zhou, and Z. Wang, "A Method of Hole-filling for the Depth Map Generated by Kinect with Moving Objects Detection," presented at the 2012 Ieee International Symposium on Broadband Multimedia Systems and Broadcasting (Bmsb), June 2012.

[18] E. Reinhard, E. A. Khan, A. O. Akyüz, and G. M. Johnson, Color Imaging Fundamentals and Applications. A K Peters/CRC Press, 2008.

[1] J. H. Cho, K. H. Lee, and K. Aizawa, "Enhancement of Depth Maps With Alpha Channel Estimation for 3-D Video," Ieee Journal of Selected Topics in Signal Processing, vol. 6, pp. 483-494, Sept 2012.

[2] "3DV data: Newspaper," Gwangju Institute of Science and Technology (GIST), Republic of Korea.

[3] Kinect, http://en.wikipedia.org/wiki/Kinect