自動適應內容之反向色調映射演算法暨硬體架構設計

隨著顯示器和攝影機的演進，高動態對比(High Dynamic Range)影像以及影片日漸風行。市售的相機、其他具拍照功能的可攜式裝置或電視不斷地把內建此種功能列為產品特色之一。在這些高動態對比相關技術裡，色調映射(Tone Mapping)是其中一個相當重要的技術，用以將高動態對比影像或影片在低動態對比(Low Dynamic Range)顯示器上播出或顯示。而反向色調映射(Inverse Tone Mapping)就是用於將低動態對比影片或影像在高動態對比顯示器上播出或顯示，並使之具備高動態對比影像或影片的視覺效果。現存的反向色調映射演算法著重於如何提昇過度曝光(Over-Exposure)區域的亮度，較忽略正常曝光(Well-Exposure)區域。我們提出一個演算法，不只能加強過曝區域，也能合理地加強正常曝光區域。這個演算法包含一個內容自動適應(content-adaptive)的反向色調映射運算單元，用以根據不同影片場景特性做出不一樣的反向色調映射反應曲線。另外，在這演算法中需要一個場景辨識器(Scene Classifier)，用以判斷不同場景以便採用不同的處理。最後，過曝加強(Over-Exposure Enhancement)部份會重建回過曝區域被截斷的資訊。就我們所知，這個演算法是第一個能夠自動依照場景變化調整的演算法。在這演算法中，計算的瓶頸在於過曝加強單元。此單元需要數個大型濾波器(filter)，這些大型濾波器將會導致演算複雜度變得非常高。為了解決這個問題，我們採用了基於色彩統計直方圖(histogram-based)的方法。然而傳統的這種方法在處理雙向濾波器(bilateral filter)時在該濾波器的空間核心(spatial kernel)將被過度簡化而產生不需要的人工雜訊(artifact)。為此，我們提出了保留空間核心(Spatial-Kernel-Preserved)的方法以解決這個問題。儘管此種基於色彩統計直方圖的方法能夠大量降低運算量，高解析度影片的運算複雜度依然是相當驚人。為此，我們設計了硬體以便能夠達成Full HD影片的即時處理。

關鍵字

高動態對比；反向色調映射；色調映射

並列摘要

With the development of the displays and photographic devices, the high dynamic range (HDR) image and video are getting popular. More and more consumer electronics products, such as cameras, mobile phones and televisions, are advertised for their unique built-in HDR functions. Among the HDR relevant techniques, tone mapping is an important technique used for displaying HDR content on low dynamic range (LDR) devices. On the other hand, inverse tone mapping enables LDR contents to appear with an HDR effect on HDR displays. The existing inverse tone mapping algorithms usually focus on enhancing the luminance in over-exposed regions with less (or even no) effort on the process of the well-exposed regions. In this thesis, an algorithm is proposed with not only enhancement in the over-exposed regions but also in the remaining well-exposed regions. The proposed algorithm contains a content-adaptive inverse tone mapping operator, which has different responses with different scene characteristics. Scene classification is included in this algorithm to select the environment parameters. Lastly, enhancement of the over-exposed regions is performed to reconstructs the truncated information. To the best of our knowledge, this work is the first inverse tone mapping which is self-adjusted depending on scenes of contents. In this algorithm, the computational bottleneck lies in the over-exposure enhancement. Several filters with large kernel size are required in this algorithm, where the convolution makes the computational complexity be very high. To resolve this problem, histogram-based method is adopted in this thesis. However, the spatial kernel of the traditional histogram-based bilateral filter is oversimplified, which results in artifacts in our application. In fact, this characteristic has a negative effect on denoising and image processing. In this thesis, a spatial-preserved histogram-based bilateral filter is proposed to resolve such problem. Although the histogram-based method reduces the computational complexity drastically, it is still difficult to achieve a real time high resolution video processing. As a result, a hardware architecture is proposed to achieve the real time processing of Full HD (1920$ imes$1080) 30fps. This design operates at 100MHz with TSMC 90nm cell library, whose gate-count is 1.3M and 28.99KB on-chip memory.