依據空間與時間相關性之適應性視差估算產生中間景物的分析與硬體設計

Analysis and Hardware Design of Intermediate View Generation Using Adaptive Disparity Estimation Based on Spatial and Temporal Correlation

李俊宜(Chun-Yi Li)

指導教授 ： 陳永昌

國立清華大學/電機資訊學院/電機工程學系/碩士(2007年)

摘要

3維電視(3D-TV)將會是高畫質電視(HDTV)下一步要走的路線。許多國內外知名大學與研究機構已經開始著力於如何設計具兼容性與彈性化的3DTV廣播系統。視訊的壓縮跟網路上的強健傳輸在3D-TV更是重要的課題之一。另外，立體影像如何呈現也是另一個熱門話題。未來，3D-TV技術可透過廣告看板、電影院、家庭電視、電腦螢幕等呈現，影像內容將會變得更逼真生動。但，由於頻寬有限，將所有視角的影像全部都傳輸過來勢必是行不通的，所以必須借助中間影像的合成，來符合人類視線環繞或者移動視差的特性，讓我們的眼睛更加舒服。
要算出立體影像的中間影像首先必須先估算出兩張影像的視差(Disparity)，而估算視差的方法有很多種，較適合硬體實現的不外乎是方塊匹配(Block Matching)。目前的文獻中，方塊匹配包括下列幾種方法：沿Epipolar line做全搜尋(Full Search)；二，使用階層式搜尋(Hierarchical Search)；三，使用四元樹搜尋(Quadtree-based Search)。第三個方法，比其他方法更可以降低合成之中間影像內物體邊緣閃爍(flicker)效應，提升PSNR比。
我們將使用方塊與方塊在空間與時間相關性質。在空間相關性，可以用來判斷我們方塊的搜尋範圍，算出更精確的視差估計(Disparity Estimation)；在時間相關性，我們的方法可以用來大幅度降低閃爍效應，另外在補洞(Hole-filling)部分會留下邊緣的特性。最後，會將中間影像合成的演算法使用硬體描述語言去設計硬體架構，並在Xilinx多媒體版來實現設計的原型。

關鍵字

三維電視 ； 視差估算 ； 中間影像合成 ； 方塊匹配 ； 立體影像

並列摘要

Three dimensional television (3D-TV) will be the next step following high definition television (HDTV). Many famous universities and research organizations all over the world have concentrated on how to design a compatible and flexible broadcasting system of 3D-TV. Besides, another popular topic is the stereo image rendering. In the future, the technology of 3D-TV will be presented by advertisement boards, movie theaters, home TVs, and LCD monitors. Their contents become more photographic than before. But, it is impractical to transmit images of all view angles due to the limitation of bandwidth. In order to provide “look-around” and “motion-parallax” feeling to our eyes and let our eyes more comfortable, we use intermediate view synthesis.
First of all, we estimate the disparity of the left and right images before interpolating the intermediate view. There are many methods to estimate disparity. Block Matching is suitable for hardware implementation. Block Matching mainly includes the following methods: 1, Full Search along epipolar line. 2, Hierarchical Search. 3, Quadtree-based Search. The third method is better at reducing the flicker effect in edge part than other methods and gets higher PSNR.
We use the spatial correlation and temporal correlation between blocks. In the spatial correlation, we decide the search range adaptively to calculate the more correct disparity. In the temporal correlation, our method is good at reducing the flicker effect. In Hole-filling part, we retain the property of edge. Finally, we use Verilog HDL to develop the hardware architecture of the proposed algorithm and implement the prototype in Xilinx multimedia board.

並列關鍵字

3DTV ； Disparity Estimation ； Intermediate View Synthesis ； Block Matching ； Stereoscopic Image

參考文獻

[1] Peter Hohenstatt, Leonardo da Vinci, 1998.

[7] A. Redert, E. Hendriks, and J. Biemond, “Correspondence Estimation in Image Pairs,” IEEE Signal Processing Magazine 16, pp. 29-46, May 1999.

[9] D. Fleet, A. Jepson, and M. Jenkin, “Phase-based Disparity Measurement,” CVGIP: Image Understanding 53, pp. 198-210, 1991.

[10] S. Chien, S. Yu, L. Ding, Y. Huang, and L. Chen, “Fast Disparity Estimation Algorithm for Mesh-based Stereo Image/Video Compression with Two-stage Hybrid Approach,” Visual Communications and Image Processing 2003. Proceedings of SPIE, Vol. 5150, pp. 1521-1530, June 2003.

[11] M. Accame, F. Natale, and D. Giusto, “Hierarchical Block Matching for Disparity Estimation in Stereo Sequences,” International Conference on Image Processing, 1995, Vol. 2, pp. 374-377, Oct. 1995.