視點合成器分析與設計

Analysis and Design of a View Synthesis Engine

洪瑩蓉(Ying-Rung Horng)

指導教授 ： 張添烜

國立交通大學/電機學院/電子工程系所/碩士(2010年)

https://doi.org/10.6842/NCTU.2010.01031

摘要

自由視點電視系統（FTV)可以產生自由視點視訊作為新一代的顯示應用。在自由視點電視系統之中，視點合成(view synthesis)是其中重要的一部分，作為接收端的視點內差以及重建。MPEG-FTV發展提供的視點合成演算法（view synthesis reference software, VSRS)使用了兩個參考視點的視訊及其深度圖來合成虛擬視點的視訊。為了達到即時處理高解析度視訊的目標，我們根據VSRS演算法設計一個高規格視點合成器的積體電路。然而，尤其是高解析度的視訊，在視點之間不規則的對應關係使得硬體設計時遇到資料控制複雜、晶片記憶體需求高以及高頻寬等難題。
在這篇論文中，針對上述的挑戰我們提出一個從像素等級、列等級、到畫框等級的階層式管線化架構。在畫框等級管線化中，以兩個畫框等級分別進行深度貼圖(depth mapping)與圖像貼圖(texture mapping)，並將第一級深度貼圖所產生之虛擬視點的深度圖寫到外部記憶體中來減少內部晶片記憶體的需求。在列等級管線化中，我們利用環形FIFO(Circular FIFO)以及列層級緩衝的架構來改進對外部記憶體資料傳輸的效率。最後，所有的運算元都以像素等級的管線化來達到每秒0.5像素分別處理左、右兩個參考視點的生產效率。
在聯電90奈米的製程下，此設計的硬體消耗為268.5K個邏輯閘、單埠及雙埠晶片記憶體各需56.9K及12.5K。此外，在實驗結果的客觀評比中，我們提出的硬體化設計相比於原本的VSRS演算法可以達到無PSNR下降的品質。

關鍵字

視點合成 ； 積體電路設計

並列摘要

The free view-point television (FTV) system is a developing innovative system that can generate free view-point video for the new trend of display application. In the FTV system, the view synthesis is one of the most important components in the receiver-side for view interpolation and reconstruction. MPGE-FTV forum proposed view synthesis reference software (VSRS) that uses two-view videos and the corresponding depth maps to synthesize the virtual-view video. To achieve the demand of synthesizing high quality videos in real-time, we implemented a view synthesis engine in VLSI design based on the VSRS algorithm. However, the irregular mapped positions between views result in complicated data control, high internal storage, and high external bandwidth in hardware implementation, especially for high resolution videos.
To address the design challenges, in this thesis, a hierarchical pipelining architecture from the frame-level, column-level, to pixel-level is proposed. In the frame-level pipelining, two stages are installed for the depth mapping and texture mapping to store temporary warped maps in external memory and reduce internal memory. In the column-level pipelining, the circular FIFO buffering architecture and column-level buffering are proposed to improve the efficiency of external data access. In the pixel-level pipelining, all the computations are parallelized to achieve the throughput of 0.5 pixels per cycle.
With the UMC 90nm CMOS technology, this design consumes 268.5K gate counts, 56.9KB and 12.5KB memory space for on-port and two-ports SRAMs. In the objective evaluation, our hardware implementation can generate the virtual view without quality drop in PSNR, compared to the original results of VSRS.

並列關鍵字

view synthesis ； VLSI design

參考文獻

[1] C. Fehn, “Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV, ” in Proc. of SPIE conf. on Stereoscopic Displays and Virtual Reality Systems, vol. 5291, pp. 93-104, May 2004.

[3] P. Kauff, N. Atzpadin, C. Fehn, M. Mller, O. Schreer, A. Smolic, and R. Tanger. “Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability,” Signal Processing: Image Communication. Special Issue on Three-Dimensional Television and Applications, 22(2), pp.217-234, Feb. 2007.

[7] Y. Mori, N. Fukushima, T. Fuji, and M. Tanimoto, “View generation with 3D warping using depth information for FTV,” 3DTV-CON’08, Istanbul, Turkey, May 2008.

[8] M. Gotfryd, Krzysztof Wegner, Marek Domański, “View synthesis software and assessment of its performance, ” MPEG Doc. M15672, Hannover, Germany, July 2008.

[10] C. Vzquez, W. J. Tam, F. Speranza, Stereoscopic imaging: Filling disoccluded areas in image-based rendering, Proc. SPIE Three-Dimensional TV, Video, and Display (ITCOM), Vol. 6392, Boston, MA, 2006.