應用於高畫質液晶顯示器之畫面插補演算法與硬體架構設計

Frame Rate Up-Conversion Algorithm and Hardware Architecture Design for High Definition Liquid Crystal Display

許功炎(Kun-Yen Hsu)

指導教授 ： 簡韶逸

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2008年)

https://doi.org/10.6342/NTU.2008.01214

摘要

畫面更新率的提升根據不同的應用是將原來的具有較低畫面更新率的視訊訊號提升到較高的畫面更新率. 其中一個例子就是液晶螢幕藉由提高畫面更新率來降低其本身存在的動態模糊問題。
因為液晶螢幕本身是屬於hold-type 的螢幕，亦即每一張畫面都會停留一個畫面的時間這與傳統的CRT 螢幕而有所不同。由於人眼對於移動物體存在著追蹤的與視覺暫留的特性，因此當移動物體在液晶螢幕上顯示時，動態模糊就會被感覺出來。經由研究發現，將液晶螢幕的畫面更新率由原本的60 赫茲提升到120 赫茲可以有效的解決動態模糊的問題。
畫面更新率提升主要是將新的畫面藉由已存在的畫面資料內插出來。其中主要的挑戰是在畫面中存在著複雜的運動時，其中包含攝影機的移動以及前景物體的移動。在文獻中探討中，有三類以區塊為基礎的(block-based)演算法來內插畫面。然而這三類方法只受限於畫面中的移動物體存在區域性的移動。有些演算法亦有討論到攝影機的各類移動，但是卻無法利用簡單的方法將攝影機移動參數準確的估測出來
在這篇論文中， 我們提出了一個名為“ Frame Rate Up-Conversion with Global-to-Local Iterative Motion Compensated Interpolation＂的演算法。對於全域的補償內插，其全域參數會先被估測出來，而我們所利用的演算法稱為“Gradient Descent algorithm＂，而這種演算法可以將全域參數準確的估測出
來。在全域內插補償之後，畫面中屬於移動物體上的區塊會被區分出來。之後這些屬於移動物體的區塊會被所提出的區域性移動內插演算法給內插出來。而所提出的區域內插補償演算法主要可以分成兩個步驟。第一個步驟是將屬於移動物體內部的區塊給內插出來。然而在畫面中區塊與區塊之間是具有高度空間相關性的。因此在第二步驟就會利用已經補好的區塊將剩餘尚未內插出的區塊給內插出來。除此之外，畫面是利用多次性內插的方法給插補出來的。經由實驗發現，所提出的演算法在視覺感官上可以插補出較好的畫面。
對於畫面解析度為1920x1080 的高畫質液晶螢幕，我們所提出的演算法對於硬體實現上是過於複雜的。為了要降低其運算複雜度以及記憶體頻寬需求，在所提出的演算法中所需的記憶體頻寬首先會被分析，除此之外一些方法更被我們所提出，主要目的是希望可以設計出一個更符合硬體實現的演算法。首先在所提出的全域移動參數估測的演算法中，在“Gradient Descent algorithm＂中多餘的運算會被去除掉。而在前一章畫面所被估測到的全域移動參數亦會被用來當作預測值，除此之外在畫面中只有部分的像素會被利用來做估測。利用所提出的方法可以將記憶體頻寬的需求量降到只需要原來的1/6，而且可以達到類似的效果。除此之外，畫面中的每個區塊會被分類成是屬於背景部分還是移動物體部分。在本篇論文硬體實作部分，由於所提出的硬體架構是針對1920x1080 的畫面大小去做設計，且畫面更新率可以由原來的60 赫茲提升到120 赫茲，因此高的記憶體頻寬以及高的運算量將是個挑戰。因此在所設計的硬體中應用了高度的硬體平
行化去做加速。在全域性參數估測的部分，以4 倍的硬體來加速。而在全域移動補償的部分，我們所設計的硬體可以達到16 倍的加速。除此之外，針對移動區塊內插的部分亦包含在所提出的硬體架構內.我們所提出的硬體是以Verilog-HDL 來實現並且以SYNOPSYS Design Compiler.來做合成。在所提出的硬體架構中，全部的閘數為1,303k，且總功率消耗在0.9V的供應電壓下為64.474mW. 除此之外，我們設計了第一顆利用高準度全域參數估測並且將畫面更新率由60 赫茲提升到120 赫茲的硬體。.

關鍵字

畫面插補演算法

並列摘要

Frame rate up-conversion (FRUC) is applied for video sequences with lower frame rate and to converts the original frame rate to higher one according to different applications. One example application is motion blur reduction on liquid crystal displays (LCD).
Because LCD is a hold-type display, each frame will be displayed and hold in a period of time which is equal to one frame which is different from CRT. Since human eyes have the characteristic to pursuit moving objects with eye-tracking integration smoothly, the artifact named motion blur will be perceived by human eyes. Frame rate up conversion which converts the frame rate to 120Hz, can reduce the motion blur problem effectively.
Frame rate up conversion is to interpolate the new frames from the original video sequence. The challenge is to deal with the complex motion in a frame, including camera motion and motion of moving objects. There are three kinds of block-based motion compensated interpolating (BMCI) method for frame rate up conversion in the literature. However, the frame interpolation capabilities of these algorithms are only limited to locally translational motion. Some algorithms also consider the camera motion with rotation zoom-in/out can’t be estimated correctly with these simple methods.
In this thesis, a new algorithm “Frame Rate Up-Conversion with Global-to-Local Iterative Motion Compensated Interpolation” is proposed. For the global motion compensated interpolation (GMCI), the global motion parameters are estimated with Gradient Descent algorithm, which has high precision. In the next stage, the frame is interpolated with the estimated parameters. After GMCI, the moving blocks are masked and those masked block will be interpolated with BMCI. There are two stages in the proposed BMCI algorithm. In the first stage, those blocks in the moving object will be interpolated. After GMCI and the first stage, the number of rest moving blocks that have not been interpolated becomes small. Because of the high spatial correlation for moving blocks, the motion vectors (MVs) of these blocks are estimated with the predicted MVs belonging to the neighbored interpolated blocks in the second stage.
In addition, the frame is interpolated with iterative operations. Through experimental result, the proposed algorithm can provide better visual quality.
For high-definition of LCD with the resolution in 1920x1080, the proposed algorithm is too complex for VLSI design. In order to reduce the computation complexity and memory requirement, the memory bandwidth is first analyzed, and several techniques are the proposed to develop a more hardware friendly algorithm. First, in the proposed GME algorithm, the redundant computation of Gradient Descent Algorithm is avoided. The motion parameters of the previous frame are used to estimate global motion parameters with partial pixels and GME is done iteratively in the sub-sampled images. With these techniques, only 1/6 memory bandwidth is required to achieve the similar quality. Furthermore, moving flag masking and probability estimation technique is proposed to decide the interpolation method for each block in advance to reduce the iteration number of the proposed algorithm.
In the VLSI hardware design part of this thesis, because the target specification is for frame size 1920x1080 and from 60 fps to 120fps, high memory bandwidth and high complex computing are required. Highly parallel hardware is designed to execute the proposed algorithm in the limited number of cycles. The hardware is accelerated with four times in the proposed GME hardware architecture. In addition the size of local memory is also analyzed and the proposed GMCI hardware architecture can process 16 pixels in parallel in a cycle with the analysis of position for corresponding pixel in the reference frame. The moving block masking and motion compensated interpolation with two stages are also implemented in our proposed hardware architecture.
The hardware is implemented with Verilog-HDL and synthesized with SYNOPSYS Design Compiler. Faraday 90 nm cell library is adopted to design the hardware. The total gate count is 1,303 k and the usage of local memory size is 98.92kb. In addition, the power is 64.474mW with 0.9V supply voltage. It is the first hardware architecture for FRUC with GME based on Gradient Descent algorithm which estimates global motion parameters in high precision.

並列關鍵字

Frame rate up-conversion

參考文獻

[1] A. -M. Hung and T. Nguyen, “A Novel Multi-Stage Motion Vector Processing Method For Motion Compensated Frame Interpolation,” in Proc. of IEEE Int. Conf. on Image Processing, vol. 5, pp. V389-V392, Sept. 2007.

[2] A. K. Michiel, “Temporal Impulse Response And Bandwidth of Display in Relation to Motion Blur,” in SID Symp. Digest, vol. 36, pp. 1578-1581, May 2005.

[3] A. K. Michiel, “Comparison of LCD Motion Blur Reduction Methods Using Temporal Impulse Response and MPRT,” in SID Symp. Digest, vol.37, pp. 1700-1703, June 2006.

[4] K. Taichiro, “Moving Picture Quality Improvement for Hold-type AM-LCDs,” in SID Symp. Digest, vol. 32, pp. 986-989, June 2001.

[5] Y. Kuroki, T. Nishi, K. Seiji, O. Hideki and Y. Shinichi, “Improvement of Motion Image Quality by High Frame Rate,” in SID Symp. Digest, vol. 37, pp. 14-17, June 2006.