以空間-時間資訊及階層式填補作自由視角及三維立體視訊之視圖合成

Free Viewpoint TV and 3D Video View Synthesis Using Spatio-temporal Information and Hierarchical Hole-filling

楊長信(Chang-Sin Yang)

指導教授 ： 柳金章

國立中正大學/工學院/資訊工程學系/碩士(2015年)

摘要

為了降低傳輸位元流量，虛擬視圖生成是三維立體電視及自由視角電視應用中一項相當重要的技術。虛擬視角生成的主要目的是，針對攝影機所拍攝的多視角視訊加深度資訊序列，生成目標視角的虛擬視圖序列。本研究針對non-parallel攝影機所拍攝的多視角視訊加深度資訊序列，我們提出以空間-時間資訊及階層式填補作自由視角及三維立體視訊之虛擬視圖生成方法。在所提的虛擬視圖生成方法中，我們採用改良式適應性中值濾波、權重計算、3D扭曲、blending、及階層式填補等技術；同時也結合運動估測與補償、“時間”資訊融合、及“時間”資訊refinement等相關技術。根據實驗結果顯示，本研究所提方法生成的虛擬視圖比其他三種比較方法生成的虛擬視圖有更高的視訊品質。

關鍵字

虛擬視角合成

並列摘要

To reduce the transmission bitrate, virtual view synthesis is an important technique for three-dimensional television (3DTV) and free viewpoint television (FTV) applications. The main aim of virtual view synthesis is to synthesize the virtual view sequence of some viewpoint from multi-view video plus depth sequences taken by cameras. In this study, virtual view synthesis approach for multi-view video plus depth sequences taken by non-parallel cameras using spatio-temporal information and hierarchical hole-filling are proposed. The proposed approach contains: modified adaptive median filtering, weight computation, 3D warping, blending, and hierarchical inpainting, whereas using temporal information, such as motion estimation and compensation, “temporal” fusion, and “temporal” refinement. Based on the experimental results obtained in this study, the synthesis virtual view quality of the proposed approach are better than those of the three comparison approaches.

並列關鍵字

virtual view synthesis

參考文獻

[1] A. Smolic, “3D video and free viewpoint video-from capture to display,” Pattern Recognition, vol. 44, no. 9, pp. 1958-1968, 2011.

[2] M. Tanimoto, “Overview of free-viewpoint television,” IEEE Int. Conf. on Multimedia and Expo, 2009, pp. 1152-1153.

[5] P. Merkle, A. Smolic, K. Muller, and T. Wiegand, “Multi-view video plus depth representation and coding,” in Proc. of IEEE Int. Conf. on Image Processing, 2007, pp. 201-204.

[6] E. Ekmekcioglu and V. Velisavljevic, “Content adaptive enhancement of multi-view depth maps for free viewpoint video,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 2, pp. 352-361, 2011.

[7] Muller et al., “3D high-efficiency video coding for multi-view video and depth data,” IEEE Trans. on Image Processing, vol. 22, no. 9, pp. 3366-3378, 2013.