基於深度學習方法之相機定位

Camera Re-Localization with Deep Learning Methods

蔡侑霖(Yu-Lin Tsai)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2020年)

https://doi.org/10.6342/NTU202003304

摘要

基於影像的定位是希望透過影像資訊推測相機自我位置的問題，同時，對於自駕車、擴增實境、智慧機器人來說是一個關鍵且基礎的技術。近年來，隨著算力的提升和深度學習的發展，許多研究嘗試利用卷積網路強大的特徵描述能力來幫助相機自我定位。然而，當使用場域改變時，這些方法都必須花很多力氣和時間重新訓練其模型，同時顯示其泛化能力相當受到限制。基於圖像搜索概念的定位架構提升了在不同場景的泛化能力，但在預測相機相對位置時而會受限於場景。我們基於圖像檢索的概念提出了一個相機定位的架構，在計算相機相對位置時討論了更多傳統的空間幾何。同時，我們也嘗試用深度學習的方法預測影像深度資訊並加強了我們方法的定位精準度。實驗結果顯示我們的方法和現在最先進的方法有並駕齊驅的定位能力，此外，利用模型壓縮讓我們的定位流程能達到幾乎即時運行。因此，我們認為融合傳統相機方法和深度學習是一個相當有潛力的發展方向。

關鍵字

基於影像的定位 ； 相機定位 ； 深度學習 ； 擴增實境

並列摘要

Image-based localization is used to estimate the camera poses within a specific scene coordinate, which is a fundamental technology towards augmented reality, autonomous driving, or mobile robotics. As the advancement of deep learning, end-to-end approaches based on convolutional neural networks have been well developed. However, these methods suffer from the overhead of reconstructing models while been applied to unseen scene. Therefore, image retrieval-based localization approaches have been proposed with generalization capability. In this paper, we follow the concept of image retrieval-based methods and adopt traditional geometry calculation while performing relative pose estimation. We also use the depth information predicted from deep learning methods to enhance the localization performance. The experimental result in indoor dataset shows the state-of-the-art accuracy. Furthermore, by distilling and sharing the encoder of global and local feature, we make our system possible for real-time application. Our method shows great potential to leverage traditional geometric knowledge and deep learning methods.

並列關鍵字

Image-based localization ； Camera pose estimation ； Deep learning ； Augmented Reality

參考文獻

[1] D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, no. 2, pp. 91-110, 2004.

[2] A. Kendall, M. Grimes, and R. Cipolla, "Posenet: A convolutional network for real-time 6-dof camera relocalization," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 2938-2946.

[3] Z. Laskar, I. Melekhov, S. Kalia, and J. Kannala, "Camera relocalization by computing pairwise relative poses using convolutional neural network," in Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 929-938.

[4] P.-E. Sarlin, C. Cadena, R. Siegwart, and M. Dymczyk, "From coarse to fine: Robust hierarchical localization at large scale," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 12716-12725.

[5] T. Sattler, B. Leibe, and L. Kobbelt, "Efficient effective prioritized matching for large-scale image-based localization," IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 9, pp. 1744-1756, 2016.