基於深度學習框架之衛星圖像人造物切割

Segmentation of Man-Made Structures in Satellite Images Using Deep Learning Approaches

陳忠揚(Chen Chung-Yang)

指導教授 ： 廖文宏

國立政治大學/理學院/資訊科學系碩士在職專班/碩士(2021年)

https://doi.org/10.6814/NCCU202101671

摘要

遙測(remote sensing)是近年來影像處理熱門領域之一，該技術被廣泛應用於水土監測、環境監測、以及軍事類活動監控等多項應用，囿於衛星資料取得成本相對較高，致使提供學術研究的公開資料與相關研究之應用起步較晚，眾多研究中可以發現，針對衛星影像的語意切割(semantic segmentation)整體表現上仍然不佳，本研究將衛星影像分為同質性與異質性兩種資料，前者的訓練與測試資料，皆來自相同衛星及成像條件的影像，後者則是訓練和測試資料集隸屬於不同區域及季節之影像，分別探討如何透過影像增強與深度學習框架的方式，提升衛星影像的物件切割表現，以及透過「無監督域適應(unsupervised domain adaptation, UDA)」的技術，使模型面對更加複雜的衛星圖資，於跨域任務的影像分割仍保有一定的適應力。
同質性衛星影像的應用，本研究透過訓練資料的前處理，使用深度學習中遷移學習之概念，載入預訓練模型，搭配模型再訓練、Mixed Pooling Module (MPM)模組應用以及相關參數調校後，找到最佳搭配組合，提升衛星影像之切割效能；前處理包括影像增強、高頻強化、邊緣銳化等方式，目標鎖定人造物體的建築與道路，提升整體影像切割校能的mIoU指標。最終，透過資料前處理、特徵強化模組、骨幹網路選擇之搭配，獲得83.5%的mIoU效能表現，與原始效能相比大約精進3%。
異質性衛星影像的應用，本研究依序驗證Source Only、現有UDA技術以及域轉換與強化網路(Domain Transfer and Enhancement Network, DTEN)架構，透過調整其中的關鍵參數設定，試圖讓模型更有效執行跨域影像分割任務，最終超越UDA最佳效能mIoU指標3.6%，達到45.3%之表現。

關鍵字

深度學習 ； 衛星圖資 ； 語意分割 ； 影像強化 ； 無監督域適應

並列摘要

Analysis of remote sensing images is one important application of computer vision. It has been widely used in land and water surveillance, environmental monitoring, and military intelligence. Due to the relative high cost of obtaining satellite images and the lack of open data, academic research in satellite imagery analysis is gaining attention only recently. Many well-developed techniques in computer vision still have to prove their validity when satellite images are concerned. In this work, we attempt to tackle semantic segmentation of man-made structures in satellite images in two aspects: homogeneous and heterogenous datasets. The former contains images from the same satellite and imaging conditions in both training and test set, while in the latter case, training and test data are captured in different locations or seasons. To gain better performance, we have experimented with different strategies, including image enhancement, backbone substitution and architecture modification. We have also employed unsupervised domain adaptation (UDA) techniques to cope with heterogenous data, hoping that the model can still maintain its capability in cross-domain segmentation tasks.
For homogeneous satellite images, our research uses transfer learning, image pre-processing, backbone replacement, mixed pooling module (MPM) and parameter tuning to find the combination that yields the best mIoU for building and road extraction. After extensive experiments, the highest mIoU is 83.5%, an improvement of 3% over existing techniques.
For heterogeneous satellite images, our research tested and compared source only model, existing UDA methods, and domain transfer and enhancement network (DTEN). Experimental results indicate that DTEN has the best performance with an mIoU 45.3%, an improvement of 3.6% over state-of-the-art UDA techniques.

並列關鍵字

Deep Learning ； Satellite Images ； Semantic Segmentation ； Image Enhancement ； Unsupervised Domain Adaptation

參考文獻

[1] 維基百科:電腦視覺定義。

https://en.wikipedia.org/wiki/Computer_vision

[2] Aditya Kulkarni, Tharun Mohandoss, Daniel Northrup, Ernest Mwebaze, Hamed Alemohammad (arXiv 2020). Semantic Segmentation of Medium-Resolution Satellite Imagery using Conditional Generative Adversarial Network

[3] Yilei Shi, Qingyu Li, Xiao Xiang Zhus (IEEE 2018). Building Footprint Generation Using Improved Generative Adversarial Networks

[4] Mingyuan Fan, Shenqi Lai , Junshi Huang , Xiaoming Wei , Zhenhua Chai, Junfeng Luo, Xiaolin Wei. (arXiv 2021). Rethinking BiSeNet For Real-time Semantic Segmentation