使用三維 N3Net 和細節水平圖作影片去噪

Video Denoising using 3D N3Net and Detail-Level Map

翁浚瑞(Jyun-Ruei Wong)

指導教授 ： 莊永裕

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2021年)

https://doi.org/10.6342/NTU202100123

摘要

由於硬體的限制，雜訊在攝影學上是一個不可避免的問題。為了處理雜訊學術上已經有了許多去雜訊的方法。其中一個稱為影片去雜訊，也就是利用影片中其他幀來幫忙對每個幀去雜訊。本論文中提出一個使用 N3NET 作為骨幹的模型。我們將這個概念延伸到多影像的去雜訊問題。另外我們還訓練另一個子模型來學一個所謂的細節水平圖。細節水平圖之於影像的概念類似於雜訊水平圖之於雜訊。整個模型最後使用細節水平圖和原本的影像共同預測最後的結果。利用 3D 的 N3NET 可以在視覺上得到和前人成果類似的品質。並且使用接近真實的細節水平圖，我們可以得到再進一步更好的結果。

關鍵字

深度學習 ； 影像去雜訊 ； 影片去雜訊

並列摘要

Noise is an inevitable problem of photography due to hardware limitations. To tackle with it, researchers have developed various kinds of denoising methods. One of the methods use neighbor frames from video to help denoising each frames, which is so-called video denoising. In this paper, we use N3NET as backbone, which leverages neighbor patches to help denoising, and extend the concept of it to multiple images denoising problem. Furthermore, we train another sub-model to learn a so-called detail-level map of images, an analogy to noise-level map of noise from photography terms. In the end we use both detail-level map and original frames to predict the denoised result. We show that by using 3D N3Net we can have similar visual quality with state-of-the-art methods. And with close-to-ground-truth detail-level map, we can further improve the result.

並列關鍵字

Deep learning ； Image Denoising ； Video Denoising

參考文獻

[1] A. Abdelhamed, S. Lin, and M. S. Brown. A high­ quality denoising dataset for smartphone cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1692–1700, 2018.

[2] M. Aittala and F. Durand. Burst image deblurring using permutation invariant convo­lutional neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 731–747, 2018.

[3] A. Alsaiari, R. Rustagi, M. M. Thomas, A. G. Forbes, et al. Image denoising using a generative adversarial network. In2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT), pages 126–132. IEEE,2019.

[4] J. Anaya and A. Barbu. Renoir­-a dataset for real low ­light noise image reduction.arXiv preprint arXiv, 1409:6, 2014.

[5] T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, and J. T. Barron. Unprocess­ing images for learned raw denoising. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11036–11045, 2019.