摘要
連拍，一種快速拍攝多張圖像的過程，帶來了獨特的挑戰，包括幀間的大幅度移動和在圖像捕獲過程中引入的噪聲。然而，良好的圖像潛在品質為克服這些障礙提供了顯著的動力。一種有效的提高連拍圖像品質的技術是同時結合去噪和去馬賽克的聯合方法。雖然這種聯合方法顯著提高了圖像品質，但往往由於幀間的大幅度移動而遇到困難。為了應對這一挑戰，我們引入了一種兩階段對齊方案。這種粗略對齊技術熟練地處理大幅度的像素位移，降低對齊的複雜性，進而幫助後續更進階的圖像修復技術。然而，在完成粗略對齊之後，需要更精細的對齊來糾正幀間的微小差異。一種有效的解決方案是結合邊緣增強特徵對齊模塊和金字塔可變形對齊模塊。這種方式可以處理大和小的像素位移，確保了對齊的準確性，從而顯著提高圖像重建的品質。另一方面，去噪和去馬賽克技術的一個副作用可能是紋理細節的丟失。為了儘量避免此情況，可以應用非銳化濾鏡方法。這種方法通過增強邊緣的對比度來提高圖像的銳利度，有效地恢復在去噪和去馬賽克過程中可能消失的紋理。此外，基於轉換器的結構可以捕捉圖像數據內部的長距離相依性。通過理解和利用圖像數據內部的複雜關係，這種創新的結構進一步增強了圖像去噪和去馬賽克過程的性能。總體而言，我們提出了一種較為全面的方法集成了聯合去噪和去馬賽克、兩階段對齊、邊緣增強特徵對齊模塊、金字塔可變形對齊模塊、非銳化濾鏡，以及基於轉換器的結構，可以為連拍圖像修復帶來的挑戰提供一種解決方案。值得注意的是，這種方法顯著提高了合成和真實世界圖像去噪和去馬賽克的品質。
並列摘要
Burst photography, a process involving the rapid capture of multiple images, presents unique challenges, including substantial shifts between frames and noise introduced during image capture. However, the potential for superior image quality provides a significant motivation for overcoming these hurdles. An effective technique for enhancing the quality of burst images combines the processes of denoising and demosaicing. While this joint approach significantly improves image quality, it often confronts difficulties due to large shifts between frames. In response to this challenge, we introduce a two-stage alignment scheme. This coarse alignment technique adeptly manages large shifts, reducing alignment complexity and preparing the ground for more advanced image restoration techniques. However, a more refined alignment is necessary to correct minor discrepancies between frames upon completion of the coarse alignment. Combining an edge-boosting feature alignment module and a pyramid deformable alignment module can provide a powerful solution for this refined alignment. This dual-method approach ensures a comprehensive alignment process that addresses large and small shifts, thereby significantly improving image reconstruction. On the other hand, one side effect of the denoising and demosaicing techniques can be the loss of texture detail. To counter this, the unsharp masking method can be applied. This method enhances the sharpness of an image by boosting the contrast of edges, effectively restoring any textures that may have vanished during denoising and demosaicing. In addition, a transformer-based structure can capture long-range dependencies within the image data. This innovative structure further enhances the performance of the image denoising and demosaicing processes by understanding and harnessing the intricate relationships within the image data. In conclusion, a comprehensive approach integrates a joint denoising and demosaicing process, a two-stage alignment scheme, an edge-boosting feature alignment module, a pyramid deformable alignment module, unsharp masking, and a transformer-based structure can provide a holistic solution to the challenges presented by burst images. Notably, this approach has significantly improved noise reduction and demosaicking of synthetic and real-world image quality.
參考文獻
A. Dudhane, S.W. Zamir, S. Khan, F. S. Khan, and M.-H. Yang, “Burst image restoration and enhancement,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 5759–5768. v, 4, 10, 16, 18, 20, 21, 29, 39, 41
S. Guo, X. Yang, J. Ma, G. Ren, and L. Zhang, “A differentiable two-stage alignment scheme for burst image reconstruction with large shift,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 17 472–17 481. v, 5, 7, 9, 10, 22, 26, 27, 29, 39, 41
B. Gunturk, J. Glotzbach, Y. Altunbasak, R. Schafer, and R. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 44–54, 2005. 3
T. Ehret and G. Facciolo, “A Study of Two CNN Demosaicking Algorithms,” Image Processing On Line, vol. 9, pp. 220–230, 2019. 3
F. Luisier, T. Blu, and M. Unser, “Image denoising in mixed poisson–gaussian noise,” IEEE Transactions on Image Processing, vol. 20, no. 3, pp. 696–708, 2011. 4