使用飛行模擬器進行空拍影像語義分割的領域遷移

電腦視覺中的語義分割是對圖像中每一個像素進行分類，把屬於相同類別的歸為同一類。對於機器人和其他無人系統來說，這是個理解周圍場景重要的資訊。自深度神經網絡出現以來，語義分割取得了巨大進步。由於需要大量的訓練數據，有許多大量公開的資料庫，但這些數據都是地上的視角。因此，這篇論文提出一個領域遷移到無人飛行機視角的方法。在我們的研究中，我們開發了一個工具讓使用者可以輕易地從飛行模擬器產生航空影像，在現實中，人工的去標記影像是很費時的。我們先使用Cityscape資料庫進行事前訓練，這是一個以汽車視角構成的現實影像資料庫，接著再使用我們產生的資料進行再次訓練，來進行領域遷移。根據我們的實驗結果，經過我們的資料進行領域遷移後，平均準確率從先前的8 % 上升至 51.9 %。除此之外，對於現實影像的辨識，領域遷移後的結果可以識別出更細微的特徵。總結來說，我們提出一個方法，先由現實影像建構的資料庫訓練深度神經網路模型，再使用從飛行模擬器產生虛擬環境的資料進行領域遷移。這是一個可以解決現存因為缺乏無人飛行機資料庫，難以在無人飛行機進行應用的方式。

關鍵字

電腦視覺；語義分割；無人飛行機；領域遷移

並列摘要

Semantic segmentation is the task of clustering parts of images together which belong to the same object class. It is an essential data processing step for robots and other unmanned systems to understand the surrounding scene. Since the emergence of Deep Neural Network (DNN), semantic segmentation has made a tremendous progress. Due to the requirement of large amount of training data, lots of benchmark datasets are released for researchers, but the views of those data are all from the ground. Therefore, our thesis proposed a method to do domain transfer from the view from the car to the view from the drone. In our system, we developed a tool which could easily collect the data of aerial images from drone simulator, since it is time-consuming to annotate by human in reality. We first trained the model with Cityscapes dataset, a real-world dataset, then retrain with the data we generated to transfer the domain to aerial imagery. According to our results, the neural networks after domain transfer with our data generated from virtual environment rise the accuracy from 8 % up to 51.9 % in MIou evaluation. Moreover, testing with real-world data, it will predict in more detail. In summary, the proposed method that retrain the model which pretrained by real-world dataset with the data generated form virtual environment will be a solution to overcome the problem on computer vision of the drone.