不受光源影響之色彩特徵暨光源偵測之色彩校正

Color Invariant Descriptors and Color Correction Based on Illuminant Estimation

張鳳洳(Feng-Ju Chang)

指導教授 ： 貝蘇章

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2011年)

https://doi.org/10.6342/NTU.2011.02268

摘要

對於影像處理的很多運用中像是影像檢索, 影像切割, 影像分類, 物體或是景像辨識,我們都希望在擷取或是偵測影像的特徵時能夠不受到光源變化的影響。經由了解影像生成模型，並且在特定光源和物體表面假設下，我們可以得到各式各樣不受光源影響亦或是只跟物體反射率有關的全色彩特徵。另外一種色彩特徵，我們稱為半色彩特徵，它是透過色彩一次微分減去其在光影變化例如陰影或是強光變化方向的投影而得到的，它比全色彩特徵更加穩定，所以比較適合拿來偵測影像特徵，但很不幸地，根據影像生成模型的推導，我們可以發現半色彩特徵還是跟光源會有關係，因此並不適合在不同的光源變化下擷取和比較影像特徵。想要擷取影像特徵還是必須動用到全色彩特徵，我們可以利用測量到的不確定性去給予權重，之後再結合顏色張量以改善全色彩特徵的不穩定性。
假如我們可以投影一個像素至與光源垂直即不受光源影響的方向，那麼我們就可以得到一維無陰影的影像。藉由乘上一個投影矩陣，我們可以進一步顯示二維無陰影影像，至於三維無陰影影像的還原，我們提出利用比較有彈性的分類方法(Fuzzy C means)來做更有效率及精確的陰影切割並偵測陰影的邊緣,透過所切割的結果，我們提出利用非陰影區域來調整陰影區域的亮度和質地，此舉是為了使還原結果更好，接著利用離散餘弦轉換來解帕松方程式藉以得到三維無陰影影像，最後我們還提出利用彌補影像缺失的技術讓靠近陰影邊緣的還原像素看起來更自然。
光源恆定性除了找出不受光源影響的色彩特徵外，光源偵測及色彩校正也是另外一個很重要且更具挑戰性的分枝。在此論文，我們會複習一些根據影像像素或是像素差假設的光源偵測法並且提出一個新的偵測光源方法:透過對人眼均勻的色彩空間進行色偏差分析以便找出掃描累加的RGB分布時所需的影像像素比例，接著將大於此比例所對應像素值的所有像素去做一個權重平均以得到一個更具代表且比較不受脈衝雜訊影響的光源顏色。此外，不同於純粹取像素平均值以估計光源顏色的Gray World方法，我們提出利用色彩張量特徵值的和來給予較大像素變化區域更多的權重並且壓低一大塊區域都是類似顏色的影響。至於Gray Edge的方法，我們提出利用顏色張量加上陰影變量和強光變量來對不同的邊緣做權重的分配並且更進一步利用奇異值分解使得此方法可以推廣到真實世界的影像上。
既然沒有任何一個光源偵測法可以適用於所有影像，一個最新的光源偵測趨勢就是想辦法結合不同光源偵測法的結果，最常見的有資料驅動的類型例如最小平方法，另外就是根據現有的知識例如在一堆訓練用的影像中去計算最好方法出現次數的比例來給予各個估計的光源不同的權重。我們覺得只考慮最好的光源偵測法過於主觀，於是我們提出改成考慮所有要結合的光源偵測法，並且根據它們所偵測出來的光源和真實光源的角度差來給予不同的權重。此外，一個最新的光源結合技術就是整合最小平方法和最好方法出現次數的比例。 我們進一步地將最好方法出現次數的比例換成新提出來的光源結合方法，藉以用來給定調節項的初始權重值，從我們的實驗結果發現到我們所提出的光源結合法確實可以得到更精準的光源估測。

關鍵字

色彩恆常性 ； 半色彩 ； 色彩張量 ； 光源估計 ； 色彩校正

並列摘要

Photometric invariance or we say color invariance is important for many applications such as image retrieval, image segmentation, image classification, object and scene recognition, etc. Since both color based feature extraction and detection are susceptive to the varying imaging conditions including light source as well as the surface geometry, our aim is to obtain the color descriptor or feature which is invariant with respect to the photometric variation and only depends on the object reflectance.
According to the color image formation model, we can acquire numerous full color invariant descriptors under a certain illuminant as well as the surface assumption. The quasi-invariants, derived by subtracted the projected variants from the color derivative are more stable than full invariants to remove shadow-shading and highlight edges and therefore suitable for feature detection. As to the feature extraction, the full invariants are still needed since the quasi-invariants are in fact related to the illuminant from the mathematical deduction. With the weights by measured uncertainties as well as the structure (color) tensor, the full invariants become more robust in feature extraction.
If we can represent a pixel in the illuminant invariant direction, the 1D shadow free image can be obtained. The 2D illuminant invariant image can be further acquired by multiplied with the projection matrix. As to the 3D shadow free image, we propose to use a soft classification based method, fuzzy C means, to effectively and accurately segment the shadow area and then detect the shadow edges. According to the segmentation map, we can further adjust the brightness as well as the texture of the shadow area in order to get better recovered result. Finally, we utilize the modified intensity values as well as the gradients to solve the Poisson equation by discrete sine transform to obtain a 3D shadow-free image. Moreover, we propose to use the exemplar-based inpainting technique to make the reconstructed pixels which are near shadow edges look more natural.
In addition to the representation of the color invariant descriptors for reaching color constancy, illuminant estimation and color correction is another essential branch for color constancy. Considering the bad impulse noise resistance of the White Patch Retinex (WPR), susceptibility to large uniformly colored regions of the Gray world (GW), non-generality of the weighted Gray Edge (WGE) method, we propose an illuminant estimation method by adaptive scanning of accumulative color histograms based on the color cast analysis in color space for improving WPR. Also, we use the color tensor to assign more weight to regions with larger color variation for improving GW. As to WGE, the initial guess of the illuminant from original Gray Edge hypothesis, eigen-value sum of the structure tensors with shadow-shading as well as specular variants, and singular value decomposition are considered to make it more generalized. After improving all above single algorithms, we further propose an illuminant fusion scheme which is called fuzzy frequency ratio (FFR) in that it is not reasonable for best frequency ratio (BFR) to only consider the best illuminant (hard decision). That is, we take all above improved illuminant estimation measures into account and assign distinct weights to them. Finally, FFR is integrated to the regularized local regression (RLR-FFR). From our experiments, the improved single algorithms are all indeed better than original methods and the proposed combination scheme (FFR) is better than BFR and also more proper to be used for setting the initial weights in the regularized local regression.

並列關鍵字

Color invariance ； Quasi-invariants ； Color tensor ； Illuminant estimation ； Color correction

參考文獻

[3] Shafer, S. A., “Using Color to separate reflection components,” in Color Resolution Applications, vol. 10, no.4, pp. 210-218, 1985.

[4] Y. C. Chung, S. L. Chang, S. Cherng, and S.W. Chen, “The Invariance Properties of chromatic characteristics,” in PSIVT, 2006, pp.128-137.

[5] J. van de Weijer, Th. Gevers, and J. M. Geusebroek, “Edge and Corner Detection by Photometric Quasi-Invariants,” in IEEE Trans. on PAMI, 2005, pp.625-630.

[6] J. van de Weijer, Th. Gevers, and J. M. Geusebroek, “Color Edge Detection by Photometric Quasi-Invariants,” in ICCV, 2003.

[7] J. van de Weijer, Th. Gevers, and A. W. M. Smeulders, “Robust Photometric Invariant Features From the Color Tensor,” in IEEE Trans. on Image Processing, 2006, pp. 118-127.