基於深度學習的單張夜晚陰雨影像除雨與強化演算法

Deep Learning-based Single Nighttime Rainy Image Deraining and Enhancement Algorithm

黃紹曄(Shao-Yeh Huang)

指導教授 ： 貝蘇章

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202100915

摘要

在惡劣的天氣條件下(尤其是在下雨的夜晚) 將使圖像質量下降，並導致許多基於視覺的應用失敗，例如自動駕駛和物體檢測。為了解決該問題並從夜間陰雨圖像中獲得清晰的圖像，我們提出了一種基於深度學習的單張夜間陰雨圖像除雨與強化模型。該模型包含兩個子模型：首先，亮度增強網絡基於對抗生成網路(GAN) 架構，用於將夜間陰雨圖像的亮度調整為更加清晰與明亮。然後，雨水去除網絡使用上下文相關的擴張網絡從強化後的圖像中除去除雨水條紋。基於改良JORDER和EnlightenGAN的訓練架構，提出的方法可以同時增強亮度效果並消除夜間多雨圖像的雨紋。透過在生成數據集和真實數據集上進行廣泛的實驗，所提出的方法比現有技術有了顯著的改進。

關鍵字

夜晚影像強化 ； 夜晚下雨除雨 ； 生成對抗網路 ； 卷積神經網路

並列摘要

The severe weather condition, especially in rainy night, will degrade images and cause many vision-based tasks fail, such as autonomous driving and object detection. To solve this problem and restoring the clean image from nighttime rainy image, we proposed a deep learning-based single nighttime rainy image deraining and enhancement model. This model contains two sub-networks: First, the illumination enhancement network is based on GAN architecture for adjusting the illumination of nighttime rainy image to appealing lighting condition. Then, a rain removal network use contextualized dilated network to jointly remove the rain streaks from the enhanced image. Based on the framework of improved JORDER and EnlightenGAN, the proposed method can simultaneously enhance illumination and remove the rain streaks of the nighttime rainy image. The proposed method achieves significant improvements over state-of-the-art by extensive experiment on both synthetic and real datasets.

並列關鍵字

Nighttime Image Enhancement ； Nighttime Rainy Image Enhancement ； Generative Adversarial Network ； Convolutional Neural Network

參考文獻

References

[1] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214–223. PMLR, 2017.

[2] P. C. Barnum, S. Narasimhan, and T. Kanade. Analysis of rain and snow in frequency space. International journal of computer vision, 86(23): 256, 2010.

[3] C. Chen, Q. Chen, J. Xu, and V. Koltun. Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3291–3300, 2018.

[4] D.Y. Chen, C.C. Chen, and L.W. Kang. Visual depth guided image rain streaks removal via sparse coding. In 2012 International Symposium on Intelligent Signal Processing and Communications Systems, pages 151–156. IEEE, 2012.