基於Kinect實現具互動與遮蔽處理之擴增實境系統

擴增實境(Augmented Reality, AR)為利用電腦繪圖與攝影機方位計算，結合虛擬影像與真實環境的一種技術。本研究應用相機校正方法，解決擴增實境中的遮蔽問題，並提出擴增實境中使用者與虛擬物體直接互動的方法。透過擴增實境家具展示系統的建立，驗證本研究提出之方法與流程是有效且優秀的。研究方法為應用微軟(Microsoft)開發之Kinect，透過彩色與紅外線相機立體對(stereo pair)校正、深度與紅外線影像的平移校正，解決了擴增實境中的遮蔽問題。本研究遮蔽處理的方法應用於室內的動態場景、不易反光的任易物體，具有即時而準確的結果。深度與紅外線影像平移校正係利用自製的校正物校正平移誤差。應用邊緣偵測、橢圓擬合與濾波找尋校正物的橢圓中心對應點，計算最小平方誤差得到平移校正參數。結果顯示本方法可有效的修正兩張影像之平移誤差。利用相機校正結果進行深度圖的校正，解決擴增實境中的遮蔽問題，方法為使用校正過的深度圖覆寫(overwrite)繪圖函式庫中的深度緩衝(Z-buffer)，在繪製虛擬物體前進行深度之比較決定繪製與否。此方法透過逐一像素之比較可以呈現部份遮蔽的效果。在擴增實境互動方面，提出人體座標系統與虛擬物體座標系統的轉換流程，與事件的觸發方法，並設計一系列的互動方法。透過自然使用者介面(natural user interface)與虛擬物體直接互動。最後，本研究利用上述的研究方法建立一套具有遮蔽處理與互動功能的家具展示系統。使用者測試結果顯示，本研究的遮蔽處理是必須且有幫助的，家具的互動是直覺且具有樂趣的，並且系統運作是流暢的。使用者測試結果可做為未來擴增實境人機互動與介面開發的參考。

關鍵字

Kinect校正；擴增實境；遮蔽處理；擴增實境互動；自然使用者介面

並列摘要

Augmented Reality is a technology which combines virtual object with real scene captured from camera using computer graphic rendering and calculation of camera position technology. This study solved occlusion problem in augmented reality by camera calibration method and propose a method that let user directly interact with virtual object in augmented reality system. The result shows the method proposed by this study is effective. Kinect from Microsoft was used in this method. Infrared and color camera were calibrated by a stereo calibration method. Infrared and depth image’s translation parameters were calibrated by a calibration board made by this study. Occlusion problem was solved through the developed calibration procedures which can be used in dynamic indoor scenes and have accurate and real time results. Using the calibration board made in this study to correct the translation error between the depth map and infrared image. This study applied edge detection, ellipse fitting, and image filtering to search for correspondence points of the center of ellipse. Then calculated the least square solution of translation error. The result indicated the method to be effective. This study solved occlusion problem by the camera calibration parameters. The approach overwrites Z-buffer in the drawing library with the calibration depth map. Per pixel comparing method was conducted to show the partial occlusion images. This study proposes a method which transform user skeleton coordinate into model coordinate system and designs series of methods to interact with virtual object using natural user interface. Finally, this study created an augmented reality system for furniture display and conducted a user test. The result shows that users think occlusion handling is effective and must be used in the system; system runs smoothly and interaction with virtual object is enjoyable; design of interaction is intuitive. The result of the user test can be used as an example for developing future natural user interfaces in augmented reality systems.

並列關鍵字

Kinect calibration ； augmented reality ； occlusion handling ； interactive augmented reality system ； natural user interface

參考文獻

Azuma, R. T., & others. (1997). A survey of augmented reality. Presence, 6(4), 355–385.

Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6), 679–698.

Carozza, L., Tingdahl, D., Bosché, F., & Gool, L. (2014). Markerless Vision-Based Augmented Reality for Urban Planning. Computer-Aided Civil and Infrastructure Engineering, 29(1), 2–17.

Corbett-Davies, S., Dunser, A., & Clark, A. (2012). An interactive augmented reality system for exposure treatment. 2012 IEEE International Symposium on Mixed and Augmented Reality, Atlanta, Georgia, USA, 95–96.

Feng, Y., Du, W., Guan, X., Gao, F., & Chen, Y. (2006). Realization of multilayer occlusion between real and virtual scenes in augmented reality. CSCWD’06. 10th International Conference on Computer Supported Cooperative Work in Design, 2006, Nanjing, 1–5.

國際替代計量

基於Kinect實現具互動與遮蔽處理之擴增實境系統

全文下載

主題瀏覽