在基於Kinect的三維point cloud中進行交互遮蔽偵測及遮蔽缺陷之缺補

Occlusion Detection and Patching of Occluded Regions in 3-D Point Clouds from Kinect Sensors

王嘉暐(Chia-Wei Wang)

指導教授 ： 王才沛

國立交通大學/資訊學院/多媒體工程研究所/碩士(2016年)

摘要

隨著camera的進步，能在Camera加上IR來取得彩色影像及深度影像 來建構三維空間模型，以視覺化效果呈現於使用者面前。三維空間模型之研究，其相關發展應用於三維城市模型(3D city Model)、三維空間資訊系統(3D spatial Information System)、房屋模型等。然而，在三維建模過程中常受些物件遮蔽之影響，導致某些區域不易呈現，因此，有效地偵測及透過平面紋理修補遮蔽是一個值得研究的課題。
與現有影像修補相比，如果將影像修補技術直接用於深度、彩色影像上來修補，深度影像的遮蔽會產生不理想結果。本研究為了修補三維模型的遮蔽，希望自動偵測遮蔽區塊並且透過三維座標系與影像座標系的轉換關係，將在三維模型上的遮蔽放到影像座標系上修補。本研究藉由平面紋理上之明顯結構進行修補後，再將紋理敷貼還原到三維模型上，以讓整體三維建模的結果更寫實。

關鍵字

Kinect ； Occlusion ； Image Inpainting ； 3D Hough Transform ； Filling Hole ； Detection

並列摘要

With the advancement of cameras, it is easy to construct three dimension model
from camera with IR to simultaneously get color and depth images, enhancing visual effect for users. The study in three-dimensional space model and related development have applications in urban modeling, spatial information systems and building models. However, there are usually occlusions from unwanted object in three dimension modeling. Therefore, it is worthwhile to develop effective methods for efficiently detecting and repairing occlusions in plane texture.
Compared with existing image inpainting approaches, directly applied to RGBD image, depth image with occlusions may not produce satisfactory results. In order to repair occlusions on three-dimensional models, our approach aims to automatically
detect occluded regions. The repair is done in the image frame of the planar structure
and the result is then transformed back to the world frame. Our approach also apply texture mapping so that the plane can be rendered with better visual results.

並列關鍵字

Kinect ； Occlusion ； Image Inpainting ； 3D Hough Transform ； Filling Hole ； Detection

參考文獻

[2] Richard O. Duda and Peter E. Hart Use of the Hough Transformation To Detect Lines and Curves in Pictures, Stanford Research Institute

[3] Borrmann D, Elseberg J, Lingemann K and Nüchte A, The 3D Hough Transform for Plane Detection in Point Clouds: A Review and a new Accumulator Design, 2011.

[4] Bertalmio M, Vese L, Sapiro G, and Osher S, Simultaneous Structure and Texture Image Inpainting, IEEE TRANSACTIONS ON IMAGE PROCESSING, 2003.

[6] Bertalmio M, Bertozzi A.L, Sapiro G. Navier-Stokes, fluid dynamics, and image and video inpainting, Proc. of the IEEE Int’l Conf. on Computer Vision and Pattern Recognition, 2001

[7] Ballester C, Bertalmio M, Caselles V, Sapiro G, Verdera J, Filling–In by Joint Interpolation of Vector Fields and Gray Levels, IEEE Transactions on Signal Processing , 2001