以多視立體影像結合機器學習進行三維場景重建

3D Scene Reconstruction from Multi-View Stereo Images Using Machine Learning

曹雅筑(Ya-Chu Tsao)

指導教授 ： 徐百輝

國立臺灣大學/工學院/土木工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202003821

摘要

由多角度獲取的二維影像常被應用於三維場景重建，利用兩張以上不同視點的影像，模擬人類視覺系統，基於視差原理獲取影像對應點之間的位置偏差，傳統以人工立體量測方式獲得三維空間資訊，其後電腦視覺與影像處理的引入，加速影像匹配技術的發展，利用密匹配逐像元地自動化尋找立體像對中之共軛像點，並由密匹配點前方交會出物空間三維地面坐標，產製高密度點雲，然其成果仰賴人工評估，且點雲模型易有匹配錯誤所產生的雜訊，遮蔽區資料流失與同調區匹配仍為待解之題。近年來許多研究嘗試引入機器學習技術，直接從二維影像和經驗中進行學習並訓練預測模型，保留同調區特徵並學習遮蔽區與三維模型間之幾何關係，最後輸出三維空間資訊預測，更自我評估預測成功率。
　　現今三維建模廣泛應用在各領域，使用率增加，更追求其作業效率與精度，許多文獻與研究已針對小尺度物件與場景重建，在處理二維影像的領域展現了優秀的表現，然影像遮蔽區重建困難，重建場景尺度擴大仍是挑戰，因此本研究比較與分析基於多視立體影像，使用機器學習直接從多視角二維影像重建三維模型的方法，簡化數據處理作業，觀察不同方法對不同場景之適用性，以期針對不同場景提供應用建議以及預期成果。

關鍵字

多視立體 ； 機器學習 ； 三維場景重建

並列摘要

3D scene model is the basic data model in 3D GIS (Geographic Information System) which can be used for 3D geo-visualization and scene analysis. Commonly the 3D scene can be reconstructed by means of LiDAR and photogrammetry technologies, however most of the methods are time-consuming and not fully automatic. How to efficiently and automatically reconstruct the 3D scene models has become an important research issue. This paper proposes a 3D scene reconstruction method from multi-view stereo (MVS) images based on machine learning. Similar to the stereo-pair for 3D vision, the multi-view stereo mimics the human visual system (HVS) to acquire 3D information from multiple overlapping images. Because of the multiple view of an object, the problem of occlusion can be overcome. However, the complex geometric relationship between multiple view stereo images also increase the difficulty of calculation.
To make the processing of 3D reconstruction more efficient and automatic, a novel method based on machine learning was introduced. Machine learning is a subset of artificial intelligence (AI) that provides systems the ability to automatically learn from data and improve from experience without too much manual intervention. Therefore, this study intends to use the advantages of machine learning to extract and train the useful features for reconstruction, improving the problems from occlusion. Based on multi-view stereo images and the machine learning model, this study aims to reconstruct the object or even the scene directly and compare the applicability of different scene from algorisms. Make the data processing operations simplified and the entire process more efficient or fully automated.

並列關鍵字

Machine learning ； Multi-view stereo ； Scene reconstruction

參考文獻

Aanæs, H., Jensen, R. R., Vogiatzis, G., Tola, E., and Dahl, Anders Bjorholm, 2016. Large-Scale Data for Multiple-View Stereopsis, International Journal of Computer Vision, 120(2), 153-168.

Ann, N. Q., Achmad, M. S. H., Bayuaji, L., Daud, M. R., and Pebrianti, D., 2016. Study on 3D Scene Reconstruction in Robot Navigation using Stereo Vision, Proceedings of 2016 IEEE International Conference on Automatic Control and Intelligent Systems, Selangor, Malaysia, pp. 72-77.

Bülthoff, I., Bülthoff, H., and Sinha, P. J. N. n., 1998. Top-down influences on stereoscopic depth-perception, Nature neuroscience, 1(3), 254-257.

Battiato, S., Capra, A., Curti, S., and Cascia, M. L., 2004. 3D stereoscopic image pairs by depth-map generation, 2nd International Symposium on 3D Data Processing, Visualization and Transmission, Thessaloniki, Greece, pp. 124-131.

Bay, H., Tuytelaars, T., and Van Gool, L., 2006. SURF: Speeded Up Robust Features, Proceedings of European conference on computer vision, Berlin, Germany, pp. 404-417.