- 學位論文
利用感知特性及移動補償技術提升畫面禎幅率之演算法及硬體架構設計
Algorithm and Hardware Architecture Design of Perception-Aware Motion Compensated Frame Rate Up-Conversion
摘要
禎幅率，或稱畫面更新率之提升，是一藉由增加視訊訊號內容將原有之較低畫面更新率提升為較高的更新率的技術。此技術早期被廣泛的運用在視訊的壓縮技術中，而近年來，此一技術被運用在將液晶螢幕之畫面更新頻率從傳統之60赫茲提升至120赫茲或更高的顯示頻率以解決液晶螢幕動態模糊的問題。 在所有利用高顯示頻率來解決動態模糊的方法中，利用動態補償技術插補之畫面品質為最好，但是其主要的挑戰是運算量、記憶體和頻寬的需求都非常大，成本昂貴。 在文獻探討中，可將以區塊為基礎的動態補償演算法分為三個階段，第一階段是動態預估以得到移動向量，第二階段是對所得到的移動向量作修正和優化處理，最後一階段則是利用優化後的移動向量指向現有的畫面資料來將畫面內插出來。其中以第一階段的動態預估需要最大的運算量和記憶體，由於此一步驟已經在編碼器中實行並且將預估的移動向量傳送到解碼器中，因此若能直接使用解碼器中的移動向量來當初始值，再在第二階段裡進行優化，可大量降低成本。 我們的研究目標是發展出一低成本之動態補償插補畫面技術提供畫面解析度1920x1080的數位電視系統每秒120禎的畫面更新率。基於每秒60禎之顯示已經可以提供人眼視覺自然的動態感覺一事實，給我們一動機去研究是否能利用人眼視覺的特性來節省運算成本。在此篇論文中，我們進行了人眼視覺之心理物理學實驗對人眼是否能夠分辨以120赫茲以及60赫茲顯示之物件運動之差別。在實驗中，我們得到了人眼難以察覺維持時間短於100毫秒或者速度小於每秒3度視角以120赫茲和60赫茲顯示之運動的差別之結論。 利用心理物理學之實驗結果，我們提出了一參考感知特性且利於硬體實現之動態補償畫面插補演算法。首先由解碼器取得移動向量以大量減少運算量，並使用解碼器中之資訊得到畫面轉換的資訊來決定是否要重複顯示上一張畫面。接著以提出之可適性中位數濾波器以及對稱性區塊重疊動態預估技術來優化所得到知向量，並使用提出之反投式運動軌跡預估將雙向的移動向量求出，最後參考感知特性篩選出可簡化運算之區塊來做動態補償之畫面插補。感知特性之運用可降低硬體對於系統頻寬之需求並且增加運算動態預估優化之區塊百分比。 而在硬體實現上，我們將所提出之硬體架構以Verilog硬體描述語言實現，並且以SYNOPSYS Design Compiler來合成。合成立用90um製程元件庫，所得之全部閘數為212 k，運算頻率為200MHz。相較於其他使用全域移動預估以及移動補償技術的演算法來說，成本大為降低。
並列摘要
Frame rate up-conversion (FRUC) is a technique converting video sequence from lower frame rate to a higher one, which is originally widely-used in the video compression system to reconstruct frames at the decoder side that skipped by the encoder, and also applied in the high frame rate LCD system nowadays to reduce motion artifacts. Among the motion blur reduction methods, motion-compensated frame interpolation (MCFI) yields the best interpolation results by taking the motion information into consideration and no decrease in the overall brightness. However, the cost is high, since the process to estimate and compensate motions in the MCFI algorithm is computationally expensive, and with high bandwidth and memory requirements. The target application of this work is the HDTV system using LCD with 120Hz refresh rate, where a cost-effective MCFI hardware is desirable for such system, and the frame rate that is much higher than the sampling rate of human eye motivated us to seek cost reduction solution with the perceptual characteristics of human eye. In this thesis, a psychophysical experiment has been conducted and the capability of human to distinct the difference between the motions displayed with 60fps and 120fps is studied. Where the difference of motions with velocity under 3 °/sec and with duration under 100 ms in 60fps and 120fps has proved to be hard-to-detective for human eyes. Base on the psychophysical experiment results, a novel hardware-oriented perception-aware motion-compensated frame interpolation algorithm is proposed. For the VLSI hardware design, the target specification is set to 1920x1080 frame size, with throughput of 60 interpolated frames per second. The hardware is implemented with Verilog-HDL and synthesized with SYNOPSYS Design Compiler. Faraday 90um cell library is adopted to design the hardware. The total gate count is 212K.
參考文獻
延伸閱讀
- Li, C. Y. (2005). 以全域移動補償為基礎之視訊編碼方式及其在提高視訊解析度上之應用 [master's thesis, National Taiwan University]. Airiti Library. https://doi.org/10.6342/NTU.2005.02904
- Yunjhen, C. (2013). 以基要畫框及可調適細節扭曲作超解析度視訊重建 [master's thesis, National Chung Cheng University]. Airiti Library. https://www.airitilibrary.com/Article/Detail?DocID=U0033-2110201613535658
- Chien, H. Y. (2019). 採用優化的動態像素位移演算法達到解析度提升 [master's thesis, National Chiao Tung University]. Airiti Library. https://www.airitilibrary.com/Article/Detail?DocID=U0030-0306202016345972
- Tsao, Y. B. (2020). Deep Learning-based High-Resolution Frame Interpolation Algorithm and Real-Time Hardware Architecture [master's thesis, National Taiwan University]. Airiti Library. https://doi.org/10.6342/NTU202100009
- Kuo, P. H. (2012). An Algorithm and Hardware Architecture Design of Content-Adaptive Inverse Tone Mapping [master's thesis, National Taiwan University]. Airiti Library. https://doi.org/10.6342/NTU.2012.10523