對適合建立虛擬視野的補洞演算法之分析與硬體實現

Analysis on Disocclusion Algorithms and Hardware Implementation for Virtual View Generation

劉致宏(Chih-Hung Liu)

指導教授 ： 陳永昌

國立清華大學/電機資訊學院/電機工程學系/碩士(2009年)

https://doi.org/10.6843/NTHU.2009.00714

摘要

3維電視(3D-TV)被期待是電視改進過程中的下一部。許多國內外知名大學與研究機構已經開始著力於如何設計具兼容性與彈性化的3DTV廣播系統。視訊的壓縮跟網路上的強健傳輸在3D-TV更是重要的課題之一。另外，立體影像如何呈現也是另一個熱門話題。未來，3D-TV技術可透過廣告看板、電影院、家庭電視、電腦螢幕等呈現，影像內容將會變得更逼真生動。但，由於頻寬有限，將所有視角的影像全部都傳輸過來勢必是行不通的，Depth image based rendering(DIBR)這項技術於是被提出，這個技術主要是利用一張已知視角的影像與其相對應的深度圖(depth map)來提供觀賞者一張不同視角的影像。
Disocclusion是DIBR中持續被研究解決的問題。利用位移像素(shift pixels)來建立新視角，有些區域(disoccluded regions)會暴露出來，如果沒有適當的填補這些洞，影像的品質會大大的降低。許多文獻提出不同的方法要來填補這些洞(Hole-filling)，我們將分析幾種較常用的補洞方法，包含：(a)constant color filling、(b)horizontal interpolation、(c) horizontal extrapolation using depth information、(d)variational inpainting using depth information 和(e) smoothing of depth maps，分析這些方法用來硬體實現的可行性。最後，利用軟體模擬各演算法的結果，選擇一個較適合的演算法，使用硬體描述語言去設計其硬體架構，並在Xilinx多媒體版來實現設計的原型。

關鍵字

補洞演算法 ； 硬體實現 ； 虛擬視野建立

並列摘要

Three dimensional television (3D-TV) is expected to be the next step in the advancement of television. Many famous universities and research organizations all over the world have concentrated on how to design a compatible and flexible broadcasting system of 3D-TV. Besides, another popular topic is the stereo image rendering. In the future, the technology of 3D-TV will be presented by advertisement boards, movie theaters, home TVs, and LCD monitors. Their contents become more photographic than before. But, it is impractical to transmit images of all view angles due to the limitation of bandwidth. Depth image based rendering (DIBR) is a commonly used technique to generate virtual views of a scene from a known view and an associated depth map.
However, a major problem with DIBR is that the shifted pixels now occupy new positions and leave areas that they originally occupied ”empty”. These disoccluded regions have to be filled properly, otherwise they can degrade image quality. There are many proposals to deal with the problem. We analyze different methods for filling the disoccluded regions, including (a) constant color filling, (b) horizontal interpolation, (c) horizontal extrapolation using depth information, (d) variational inpainting using depth information and (e) smoothing of depth maps. It is analyzed about the feasibility of using these methods for hardware implementation. Finally, we compare the results of software simulation for visible perspective of human and use Verilog HDL to develop the hardware architecture of the suitable algorithm and implement the prototype on Xilinx multimedia board.

並列關鍵字

disocclusion algorithms ； hardware implementation ； virtual view generation

參考文獻

[1] Peter Hohenstatt, Leonardo da Vinci, 1998.

[6] C.Fehn,“Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3DTV,”in Proc. of SPIE Conference on Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93-104, San José, CA, May 2004.

[9] M.Ziegler, L.Falkenhagen, R.Horst and D.Kalivas, “Evolution of stereoscopic and three-dimensional video,” Signal Processing: Image Communication, vol. 14, pp. 173-194, 1998.

[11] L.Lipton, Foundations of the Stereoscopic Cinema – A Study in Depth, Van Nostrand Reinhold, New York, NY, USA, 1982.

[14] W.J.Tam, A.Soung Yee, J.Ferreira, S.Tariq and F.Speranza, "Stereoscopic image rendering based on depth maps created from blur and edge information," In Proc. of SPIE: Stereoscopic Displays and Applications XII, vol. 5664, pp.104-115, 2005.