使用時序注意力機制之基於影像序列的物體位姿估測

Object Pose Estimation Using Image Sequence via Temporal Attention

陳竣宇(Chun-Yu Chen)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2022年)

https://doi.org/10.6342/NTU202203024

摘要

物體位姿估測是一種用於偵測圖片中感興趣物體的技術。由單張RGB影像來做6D物體位姿估測的一個常見的挑戰就是是物體在雜亂的場景中彼此的互相遮擋。除了只使用輸入影像的空間資訊外，利用影片資料中連續影像的之間的時間資訊可以進一步提升這項任務的表現。舉例來說，考慮到輸入影像中的物體在當前的相機視角下被其他物體遮擋的情況，結合鄰近影像的相機視角就有機會去回復未看到的物體的位姿。在本論文中，我們對使用了深度學習的端到端單張影像的位姿估測方法進行了充分的分析與實驗，並且提出了一種端到端方法將單張影像位姿估測擴展到多張影像的版本。實驗結果顯示，我們的方法相較於基準模型提供了更準確的結果。

關鍵字

物體位姿估測 ； 語義分割 ； 時序注意力 ； 深度學習

並列摘要

Object pose estimation is a technique used to detect objects of interest in images. A common challenge of 6D pose estimation from a single RGB image is the occlusions between objects in cluttered environment. In addition to only use the spatial information in the input frame, utilizing the temporal information between consecutive frames in the video data may further improve the performance in this task. For instance, taking account of the situation that the objects in the input frame be occluded by other objects in current camera perspective, combining the camera perspectives of neighboring frames makes it possible to recover the poses of unseen objects. In this thesis, we fully analyze and experiment on an end-to-end single-frame pose estimation method using deep learning. We also propose an end-to-end approach to extend the single-frame pose estimation to a multi-frame version. The experimental results show that our method provides more accurate results than the baseline model.

並列關鍵字

Object Pose Estimation ； Semantic Segmentation ； Temporal Attention ； Deep Learning

參考文獻

H. Bay, T. Tuytelaars, and L. V. Gool. Surf: Speeded up robust features. In European conference on computer vision, pages 404–417. Springer, 2006.

G.-J. v. d. Braak, C. Nugteren, B. Mesman, and H. Corporaal. Fast hough transform on gpus: Exploration of algorithm trade-offs. In International Conference on Advanced Concepts for Intelligent Vision Systems, pages 611–622. Springer, 2011.

Y. Bukschat and M. Vetter. Efficientpose: An efficient, accurate and scalable end- to-end 6D multi object pose estimation approach. arXiv preprint arXiv:2011.04307, 2020.

B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pages 510–517. IEEE, 2015.

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.