結合虛擬參考畫面之改良式多視角高效率視訊編碼器(MV-HEVC)

Improved Multiview High Efficiency Video Coding (MV-HEVC) with virtual reference

潘思清(Szu-Ching Pan)

指導教授 ： 林鼎然 ； 邱奕世

中原大學/電機資訊學院/電子工程研究所/碩士(2019年)

https://doi.org/10.6840/cycu201900440

摘要

高效率視訊編碼(HEVC)在多視角畫面應用，編碼順序會先以同幀不同視角來編碼，編碼時會以「同幀的第一視角」和「同視角不同幀」的畫面作為參考幀，論文方法中會改善中間視角的編碼品質，插入不存在的虛擬幀，進而達到本篇論文的最終目地提升中間視角之壓縮率。而為了生成出中間參考幀，且僅能以兩側視角為素材，在本文提出的方法中，利用深度學習補幀之方法，將兩視角當作成一般影像的前後兩幀，補幀出相似於中間視角之畫面，該畫面與中間視角的角度相接近，進而讓壓縮中間視角時的bitrate減少，所以可以把虛擬圖做為中間視角之參考幀。但是生成的虛擬圖仍有因補幀效果不佳造成的部分不完美區塊，所以本文再提出改善虛擬幀方法。本文先利用多視角優勢，依靠兩側視角畫面來計算與虛擬幀間的error，並找出表現不佳區塊，再以全部不佳區塊之PSNR來設定門檻值，將補幀效果較差的區塊檻篩選出來，並以inpainting的影像處理方法修補不佳畫面填補區塊，本文會分別比較原始HEVC方法、HEVC加入虛擬幀當作參考幀方法以及虛擬幀以inpainting修補方法的各種壓縮率比較，並以bitrate與PSNR做為壓縮率評比標準，實驗結果顯示加入虛擬幀後的結果最佳，與HM16.2原始編碼比較，bitrate能降低5~20%不等的情況下，PSNR還能提升約0.03~0.18 dB。

關鍵字

高效率視訊編碼 ； 影像組 ； 參考幀集

並列摘要

High-efficiency video coding (HEVC) is applied in multi-view images. The encoding order is first encoded in different views of the same frame. The encoding uses the "first view of the same frame" and "different frames of the same view" as the reference frame. The proposed method will improve the coding quality of the intermediate view, and the virtual frame that does not exist is inserted to be reference frame, thereby achieving the final goal of this study to improve the compression ratio of the intermediate view. In order to generate an intermediate reference frame, only the two sides of the view can be used as the material. In terms of the method proposed in this thesis, using the method of deep learning complement frame, the two views are regarded as two frames before and after the regular video, and the frame is similar to the intermediate view, which is close to the angle of the intermediate view, and then the compression is performed. The bitrate is reduced at the intermediate view, so the virtual frame can be used as a reference frame for the intermediate view. However, the generated virtual frame still has some imperfect blocks caused by the poor complementing effect, so this thesis proposes a method to improve the virtual frame. This thesis firstly uses the multi-view advantage, relies on other two-sided view image to calculate the error between the virtual frames, and finds the poorly performing block, and then sets the threshold value with the PSNR of all the bad blocks, and the area with poor complementing effect. The block is filtered out, and the poor image fill block is repaired by inpainting image processing method. The bitrate and PSNR are used as the compression ratio evaluation standard. The experimental results show that the result after adding the virtual frame is the best. Compared with the original encoding of HM16.2, the bitrate can be reduced by 5~20%, and the PSNR can be improved by 0.03~0.18 dB.

並列關鍵字

HEVC ； GOP ； RPS

參考文獻

[1]E. Herbst, S. Seitz, and S. Baker. Occlusion reasoning for temporal interpolation using optical flow. Technical report, August 2009.

[2]L. L. Rakˆet, L. Roholm, A. Bruhn, and J. Weickert. “Motion compensated frame interpolation with a symmetric optical flow constraint,” In Advances in Visual Computing, vol. 7431, pp. 447–457, 2012.

[3]T. Zhou, S. Tulsiani, W. Sun, J. Malik, and A. A. Efros. “View synthesis by appearance flow.” In European Conference on Computer Vision, vol. 9908, pp. 286–301, 2016. 1

[4]G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, pp. 1649-1668, 2012.

[5]T. Wiegand, J. R. Ohm, G. J. Sullivan, W.-J. Han, R. Joshi, T. K. Tan, and K. Ugur, “Special section on the joint call for proposals on high efficiency video coding (HEVC) standardization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, pp. 1661- 1666, January 2010.