多視角加深度視訊編碼技術之研究

A study on multi-view video plus depth coding

陳彥諭(Yen Yu Chen)

指導教授 ： 江瑞秋

國立中正大學/工學院/電機工程學系/碩士(2011年)

摘要

隨著3D技術日新月異的發展，目前的3D視訊已可提供消費者絕佳的娛樂效果，因此電影、電視都紛紛搶搭這股3D熱潮，提供具有3D內容的影片。而要合成3D視訊，在影像擷取過程中，就不只是侷限一架攝影機而已，舉例來說，立體視訊就必須要包含兩支具有視差的視訊，才能觀看場景的深度感。而自由視角視訊更需要多視角視訊影像，因此視訊的壓縮與傳輸就成為編碼端的負擔。
為了合成自由視角視訊，多視角視訊加深度資訊兩種視訊資料都是必備的，缺一不可。若考量到編碼問題，我們得思考要如何分配這兩者之間的位元率比例，也就是說在彩色與深度視訊中，我們得找出最佳的平衡點，使得自由視角視訊有較好的合成品質。此外目前學術研究拍攝的多視角視訊，動輒上百架攝影機同步擷取影像，可想而知資料量非常龐大，因此資料減量的部份就顯得格外重要，如何找出較有效率壓縮多視角視訊的方法，也成為一重要的研究課題。
本論文第一部份為多視角視訊加深度資訊之位元率分配，而第二部份則會對多視角視訊作資料減量，在此我們會針對以上這兩個部份作探討。於位元率分配的部份，我們利用人眼視覺評估影像的方式，找出較佳的自由視角視訊，並可得知在各位元率的情況，彩色與深度視訊的位元率分配方式。另外第二部份我們會採用視角間量化參數調整、階層式畫面間量化參數的調整，以及移除彩度資訊的方式，對多視角視訊編碼作適當的資料減量。
我們經由主觀實驗發現在不同位元率時，建議使用者適當的位元率比例，於高位元率其深度頻寬占全部頻寬約60.13%左右，中位元率則占頻寬36.74%左右，低位元率時則須要花費23.31%的頻寬。而第二部份中，以靜態視訊為例，經由實驗發現視角間量化參數調整約可節省26.02%之位元率，階層式畫面間量化參數調整約可節省18.58%之位元率，彩度資訊資料減量約可節省3.2%之位元率，然後結合以上三種方法最多則可節省45.82%之位元率。
關鍵詞: 多視角視訊編碼、JMVC、位元率分配

關鍵字

多視角視訊編碼、JMVC、位元率分配

並列摘要

Multi-View Video (MVV) attracts a lot of interests by providing viewers several viewpoints and seems as one of the new video types. For free view point television (FTV) application, it contains multi-view video and multi-view depth. Video sequences as well as depth maps have to be compressed prior to transmission. Hence, joint bit allocation between texture videos and depth maps is an important research issue in 3D video coding. Furthermore, some challenges have to be conquered. For example, how to deal with the drastically increasing amount of data of multi-view video plus depth (MVD) format.
The quality of the free view point video (FVV) depends on smoothness among the synthesized video. Peak Signal-to-Noise Ratio (PSNR) is the well-known metric to evaluate the image quality. However, we believe that PSNR can not directly reflect the Human Visual System (HVS), especially for the synthesized video. So we use the Double Stimulus Continuous Quality Scale (DSCQS) to evaluate the free view point videos and find the best bit allocation between multi views and depth maps under various bit-rate scenarios. Besides, multi-view videos require huge memory and bandwidth. In the second part of this work, we propose three schemes to reduce the bit-rate of multi-view videos. For instance, quantization parameter (QP) adjustment between Hierarchical B frames, QP adjustment between views, and chrominance data reduction.
We suggest the appropriate bit-rate ratio after carrying out subjective experiments. In the high bit-rate, the depth bit rate is approximately 60.13% of total bandwidth, and a better free view point videos can be rendered. In the medium bit-rate scenario, the depth bit rate is approximately 36.74%. Similarly, in the low bit-rate scenario, 23.31% of the bandwidth is suggested. In the second part, the QP adjustment between Hierarchical B frames, the QP adjustment between views, and the chrominance data reduction achieve, respectively, 26.02%, 18.58% and 3.2% bit-rate saving for the low motion video. Furthermore, we combine the above three methods and up to 45.82% of bit-rate saving is achieved.

並列關鍵字

JMVC、MVC、Bit allocation、DIBR

參考文獻

[5] Ling Hou, Oscar Au, Xiaopeng Fan, Jiantao Zhou , “Maximum-likelihood versus maximum a posteriori based local illumination and color correction algorithm for multi-view video,” in Proc. of Multimedia Signal Processing(MMSP '09), 2009.

[6] Pei-Kuei Tsung, Hsin-Jung Yang, Pin-Chih Lin, Kuan-Yu Chen, Liang-Gee Chen, “Hybrid Color Compensation for Virtual View Synthesis in Multiview Video Application,” in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS), 2010.

[8] Han Oh, Yo-Sung Ho, “H.264-Based Depth Map Sequence Coding Using

[9] Min-Koo Kang, Cheon Lee, Jin Young Lee, Yo-Sung Ho, “Adaptive Geometry-Based Intra Prediction for Depth Video Coding,” in Proc. of IEEE International Conference on Multimedia and Expo (ICME) , 2010.

“Depth Map Coding with Distortion Estimation of Rendered View,” in Proc. of VIPC'10, 2010.