利用神經網路曲線模型進行RAW照片多重曝光融合

Multi-Exposure RAW Image Fusion with the Neural Curve

闕壯維(Chuang-Wei Chueh)

指導教授 ： 莊永裕

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202200081

摘要

近年來的論文已經證明我們可以學習一個轉換RAW到sRGB的深度學習模型來取代傳統相機的訊號處理器，藉此來減少開發相機訊號處理器的時間和成本。但是近年使用智慧型手機拍照變得愈來愈普遍，智慧型手機硬體的限制使得單張照片的品質受限，常常有過曝及曝光不足的區域，難以僅藉由一個轉換RAW到sRGB的深度學習模型來完全恢復這些區域的資訊，因此我們提出一個方法來融合兩張不同曝光值的RAW影像來解決這個問題。同時使用一個神經網路曲線模型來幫助轉換RAW到sRGB影像的顏色，經過我們的實驗證實，加入曲線模型特徵可以使融合兩張RAW影像的模型在轉換顏色和合成顏色上有更好的表現。

關鍵字

多重曝光融合 ； RAW圖片到sRGB圖片 ； RAW圖片 ； 神經網路曲線模型 ； 深度學習

並列摘要

Recently works has demonstrated that we can learn a RAW-to-sRGB model to replace a traditional hand-crafted camera ISP, thereby saving much time and cost to develop complex camera ISP.

However, it has become more popular to take photo by smartphone nowadays, the limitation of hardware limits dynamic range of photo, makes recorded images that are often lost detail in under-/over-exposed region.

Therefore, we propose a method to fuse two RAW images with different exposure values to solve this problem. We also add a neural curve module to help convert the color of RAW-to-sRGB images. Our experiments show that adding the neural curve feature can make fusion perform better in color conversion of RAW-to-sRGB, and can generate more reasonable color when fusing different exposure images.

並列關鍵字

Multi-Exposure Fusion ； RAW-to-sRGB Mapping ； RAW image ； Neural Curve ； Deep Learning

參考文獻

[1] G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte. Deep burst super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9209–9218, 2021.

[2] T. Brooks, B. Mildenhall, T. Xue, J. Chen, D. Sharlet, and J. T. Barron. Unprocessing images for learned raw denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11036–11045, 2019.

[3] J. Cai, S. Gu, and L. Zhang. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing, 27(4):2049–2062, 2018.

[4] S.-Y. Chen and Y.-Y. Chuang. Deep exposure fusion with deghosting via homography estimation and attention learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1464–1468. IEEE, 2020.

[5] F. Ciurea and B. Funt. A large image database for color constancy research. In Color and Imaging Conference, volume 2003, pages 160–164. Society for Imaging Science and Technology, 2003.