以生成式對抗網路與選擇性分類技術之實時語意分割錯誤偵測

Real-time Semantic Segmentation Fault Detection Using Generative Adversarial Network and Selective Classification Techniques

林宗郁(Tsung-Yu Lin)

指導教授 ： 李綱

國立臺灣大學/工學院/機械工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202102692

摘要

現今的語意分割方法因缺乏偵測錯誤以及輸出信心指數的能力，較難以實際運用到許多安全至上的應用中，例如:自動駕駛。因此本研究針對這些問題，提出了一套語意分割錯誤偵測框架，讓使用者可以透過對輸出設定一個特定的閾值，了解到此語意分割模型對於現在的環境辨識狀況，信心指數，以及哪些東西是辨識錯誤的。本研究的框架透過生成式對抗網路還原語意分割的結果，並藉由一個比較模組比較原始圖像與重新生成的圖像以預測每個像素的信心指數，最後利用選擇性分類選定一個合適的閾值以判定圖中哪些像素為辨識錯誤。其中本研究提出了一個嶄新的比較模組SiameseFPN，能夠針對原始圖像與生成圖像進行更好的分辨，達到更好的效能，最終在Cityscape資料集中達到AUC=93.61、AUPR-ERROR=58.92以及FPR95=22.39，均為現今方法中的最佳，並且能在單張NVIDIA RTX 2080Ti 顯示卡上達到30FPS以上的推論速度，可以應用在大部分的實時語意分割模型。

關鍵字

語意分割 ； 錯誤辨識 ； 異常偵測 ； 選擇性分類 ； 生成式對抗網路

並列摘要

The inability for semantic segmentation to detect fault and output confidence hinders them from being deployed in safety-critical and complex applications, such as autonomous driving. In this thesis, we systematically study failure detection for semantic segmentation and purpose a general framework, consisting of three modules, to explain what the model has recognized, how confident it is and which pixel was wrong by setting a threshold for the output error map. The first module is a segmentation module, which takes an image as the input and outputs a semantic map. The second module is an image synthesis module using a generative adversarial network, which generates a synthesized image from the semantic map output by the first module. The third module is a novel comparison module - SiameseFPN, which compares the difference between the synthesized image and the input image and outputs the confidence score. Lastly, we set a threshold on the confidence score by using selective classification to output an error map. We evaluate our framework on the Cityscape dataset and significantly outperforms all existing methods, i.e., AUC=93.61, AUPR-ERROR=58.92 and FPR95=22.39. Furthermore, our framework is able to reach 30FPS, which is referred to as real time computing, on a single NVIDIA RTX 2080 Ti GPU, and can be applied on any already trained real-time segmentation networks.

並列關鍵字

semantic segmentation ； failure detection ； anomaly detection ； selective classification ； generative adversarial network

參考文獻

[1] 花蓮分局秘書室, 經濟部標檢局, 無人駕駛時代來臨！《無人載具科技創新實驗條例》三讀通過, January 2019, https://www.bsmi.gov.tw/bsmiGIP/wSite/ct?xItem=77314 ctNode=7941 mp=1.

[2] EBC東森電視, 東森新聞, 國道車禍！特斯拉煞不住 車身「穿破」貨車頂, June 2020, https://news.ebc.net.tw/news/society/212290

[3] Hendrycks, D., Gimpel, K. “A baseline for detecting misclassified and out-ofdistribution examples in neural networks.” International Conference on Learning Representations, ICLR. 2017.

[4] Corbi`ere, C., Thome, N., Bar-Hen, A., Cord, M., P´erez, P. “Addressing failure prediction by learning model confidence.” Advances in Neural Information Processing Systems. 2019.

[5] Geifman, Y., El-Yaniv, R. “Selective classification for deep neural networks.” Advances in Neural Information Processing Systems. 2017.