夜晚除雨、水下及x-光的影像處理及增強

Image Processing and Enhancement for Nighttime Deraining, Underwater and X-ray Images

陳佳誼(Chia-Yi Chen)

指導教授 ： 貝蘇章

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202201022

摘要

在惡劣環境下，像是，極黑環境、夜間陰雨、X射線、水下的圖像等，通常會有能見度低及對比度降低的問題，進一步影響我們人眼視覺的判斷和電腦視覺應用(例如，自動駕駛、物體辨識，偵測和追蹤)。由於空氣中不同的懸浮粒子會吸收光的能量並散射光線，導致相機拍攝之影像能見度下降; 且在下雨的夜晚，進而造成視覺敏銳度降低且顏色偏移以及照明不足的問題。而在牙齒的X 射線圖像中，牙齦掩蓋了牙齒結構，因此牙齒上的細節不清晰。對於在水下影像而言，不同波長的光在水下傳播具有不同的衰減率，使水下影像存在著明顯的色散及色彩偏差。因此解決各種環境下造成的視覺影響並設計不同演算法來獲得清晰的圖像具有重要意義。
在本篇論文中，我們介紹了大氣散射的模型，並說明如何將此模型應用於不同的環境。首先，對於夜間陰雨圖像除雨，我們改良了現有的去除雨水的演算法，並將其結合大氣散射模型，來消除因雨產生的霧化影響 ; 而針對夜晚的場景，我們解決了因夜晚的燈光而產生的光暈效應，並提出了色彩恆定及亮度增強的演算法，以達到更好的視覺效果。我們進一步將提出基於大氣散射模型的演算法應用於牙齒的X 射線中，目的在於能有效的去除牙齦，使牙齒更加明顯。再者，我們設計應用於水下影像增強有效方法，特別的是我們擴展了大氣散射的模型，旨在於利用此模型直接將水去除。此外，我們提出將極低亮度環境下的影像的亮度增強的演算法，以Retinex理論為基礎，除了抑制雜訊和對光照分量進行強化效果，我們也有進行色彩校正的處理。不僅如此，我們基於能量曲線(Energy Curve)，提出了一種新穎的圖像對比度增強技術和能量曲線的動量守恆來決定門檻值，並將對比度增強的結果分成有無閾值限制的兩種方法。最後，我們提出了基於高斯差和深度學習的方法成功地將人臉及動物的照片素描上色，並比較了兩者的優劣。
本篇論文透過廣泛的實驗，成功地使各種惡劣的環境下的影像擁有較好的視覺感受，且能保留圖像細節及場景的自然性，且此篇論文也可以作為預處理步驟，往後應用於醫學圖像處理。

關鍵字

夜間影像除雨 ； 亮度增強 ； 水下影像增強 ； 對比度增強 ； 牙齒提取 ； 極低亮度影像增強 ； 色彩恆定性 ； 大氣散射模型 ； 人臉照片彩色素描

並列摘要

In harsh environments, such as extremely dark environments, rainy nights, X-rays, and underwater, images usually suffer from poor visibility and low contrast, which further affect our human visual judgment and computer vision applications, e.g., automatic driving, object recognition, detection and tracking. Distinct suspended particles in the air absorb and scatter the light, resulting in poor visibility of the image captured by the camera; and especially in rainy nights, it causes the problems of low visual acuity, color shift and insufficient illumination. In the X-ray image of the teeth as Orthopantomagram (OPG image), the gingivae cover the teeth structure, so the details on the teeth are not clear. For underwater images, light of different wavelengths has different attenuation rates when propagating underwater, bringing about obvious dispersion and color deviation in underwater images. Therefore, it is of great significance to solve the visual impact caused by various environments and design distinctive algorithms to obtain clear images.
In this thesis, we introduce the atmospheric scattering model and explain how to apply this model to different scenario. First of all, for the nighttime rain images, we improved the existing rain removal algorithm and combined it with the atmospheric scattering model to eliminate the veiling effect caused by rain; and for the night scene, we solved the halo effect resulting from the light sources at night and proposed an algorithm for color constant and illumination enhancement to achieve better visual effects. We further applied proposed algorithm based on the model to the OPG images with the purpose of efficiently removing the gingiva. Furthermore, we design an effective method for underwater image enhancement and especially we have extended the atmospheric scattering model, aiming to use this model to directly remove the water. In addition, for the extremely low-illumination images, we proposed an algorithm based on Retinex theory and besides suppressing the noise and enhancing the illumination component, it also perform color correction processing. Additionally, we proposed the method of the contrast enhancement and the moment preserving threshold on the basis of Energy Curve, and divide our results into two methods with or without threshold limitation. Finally, we proposed an algorithm to colorize the photo sketches of the faces of human and animals based on the difference of Gaussian (DoG) and deep learning and compare the difference of the two.
This thesis has successfully made images in various severe environments have a better visual perception, while retaining the image details and naturalness of the scene. And the thesis can also be used in medical image processing as a preprocessing step.

並列關鍵字

Nighttime rain images rain removal ； Low-illumination enhancement ； Underwater image enhancement ； Contrast enhancement ； Tooth extraction ； Extremely low-light image enhancement ； Color constancy ； Atmospheric scattering model ； Face color photo sketch

參考文獻

[1] Harald Koschmieder. Theorie der horizontalen Sichtweite. Keim Nemnich, 1925.

[2] Earl J McCartney. Optics of the atmosphere: scattering by molecules and particles. New York, 1976.

[3] Shree K Nayar and Srinivasa G Narasimhan. Vision in bad weather. In Proceedings of the seventh IEEE international conference on computer vision, volume 2, pages 820–827. IEEE, 1999.

[4] Robby T Tan. Visibility in bad weather from a single image. In 2008 IEEE conference on computer vision and pattern recognition, pages 1–8. IEEE, 2008.

[5] Srinivasa G Narasimhan and Shree K Nayar. Vision and the atmosphere. International journal of computer vision, 48(3):233–254, 2002.