以RGB-D相機在書桌環境完成物品建模與搜尋

RGB-D Based Desk Object Modelling and Search

盧恭弘(Kung-Hung Lu)

指導教授 ： 王傑智

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2015年)

https://doi.org/10.6342/NTU.2015.02019

摘要

在雜亂無章的書桌上尋找物品是一件惱人且瑣碎的工作。許多先前的研究嘗試使用物品辨識技術來解決在真實世界中尋找實體物品的問題。然而，此種方法通常需要事前費時且繁重的機器學習過程，且難以移植到其他情境中使用。在此篇論文中，透過使用RGB-D 結構光距離感測器，我們提出了一個概念簡單卻效果顯著的方法，利用偵測場景穩定度來區分靜態場景與動態場景來完成書桌物品的建模與搜尋。在靜態場景中，藉由比較當前場景與資料庫中的物品模型來完成物品的分割與建模；而在動態場景中，我們使用多目標追蹤的技術來確認各物品的位置。同時，本論文實作了一個即時書桌物品搜尋系統，同時提供了物品的三維
模型與其空間中所處位置的資訊予使用者，讓使用者能夠以簡單且直觀的方式來了解書桌上各物品的位置並進而完成尋找物品的任務。最後，在實驗中我們測試了各種書桌環境中常見的物品與不同的操作方式來展示此方法的可行性與有效性。

關鍵字

RGB-D影片 ； 物體分割 ； 多物體追蹤

並列摘要

FINDING desk objects could be annoying with a messy desk. Many previous works used object recognition technique to deal with this object searching problem. However, it usually requires massive pre-training process, which has much more effort when it is transferred to another scenario. In this work, we propose a simple yet effective approach to accomplish desk object modelling and search using a static RGB-D camera. Assume that the desk could be monitored over time, the concept of scene stability
is proposed to distinguish between stable and dynamic scene. Object segmentation and modelling are done concurrently by differentiating the current stable scene state and the model of object in database so the new object; while multiple objects tracking is adopted to find the locations of objects in dynamic scene. A user interface is designed in which both locations and appearances of the modelled objects are provided. It is easy for the user to have an understanding of the objects on the desk and the minimum effort is required to find a specific object. A variety of the desk objects with different sizes and thickness are tested, even the objects with indistinguishable volume. We also test the proposed approach for various manipulations. The experimental results demonstrate the feasibility and effectiveness of the proposed desk object modelling and search system.

並列關鍵字

RGB-D video ； Segmentation ； Multi-Object Tracking

參考文獻

Butz, A., Schneider, M., and Spassova, M. (2004). Searchlight–a lightweight search function for pervasive environments. In Pervasive Computing, pages 351–356. Springer.

Koyuncu, H. and Yang, S. H. (2010). A survey of indoor positioning and object locating systems. International Journal of Computer Science and Network Security(IJCSNS), 10(5):121–128.

Kuhn, H. W. (1955). The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2:83–97.

Lai, K., Bo, L., Ren, X., and Fox, D. (2011b). Sparse distance learning for object recognition combining rgb and depth information. In IEEE International Conference on Robotics and Automation (ICRA), pages 4007–4013.

Lai, K. and Fox, D. (2010). Object recognition in 3d point clouds using web data and domain adaptation. The International Journal of Robotics Research, 29(8):1019 1037.