運用修正版LeNet-5模型來有效降低HEVC幀內預測計算

Complexity Reduction on HEVC Intra Mode Decision with modified LeNet-5

丁海哲(Hai-Che Ting)

指導教授 ： 王家祥

國立清華大學/電機資訊學院/資訊系統與應用研究所/碩士(2018年)

摘要

HEVC與他的上一代編碼標準H.264/AVC節省了約50%的bitrate。但是預測的演算法比H.264/AVC還要來的複雜許多。HEVC擁有35個幀內預測模式，並將CU的大小種類分成了64 x 64、32 x 32、16 x 16、8 x 8以及4 x 4。在全幀內預測編碼模式當中，PU的模式決定時間佔了總編碼時間的60%〜70%。因此本篇論文的目標是藉由減少上述PU模式決定的計算複雜度，來節省整體的編碼時間。
本篇論文首先使用[18]的做法去做CU的分割，接著使用[6]的作法，先計算出當前PU的邊緣能量之後，藉由邊緣能量去判斷當前的PU是否為一個平坦的PU，如果是平坦的PU的話就只選擇0和1這兩種幀內預測模式當作候選模式，進入rate distortion optimization去選出當前PU的最佳幀內預測模式。如果不是平坦的PU，就針對 4 x 4以及 8 x 8的PU，藉由修正後的LeNet-5 CNN model去選擇出兩個候選模式，並和Most Probable Mode(MPM) list裡的前兩個模式一起成為候選模式進入rate distortion optimization，選出當前PU的最佳幀內預測模式。
我們從實驗結果發現，結合修正後的LeNet-5 Model以及邊緣能量提取，可以在BDBR沒有提升太多的情況之下，節省大量的編碼時間與運算量。
關鍵字：幀內預測、卷積神經網路、邊緣能量提取、視訊編碼

關鍵字

幀內預測 ； 卷積神經網路 ； 邊緣能量提取 ； 視訊編碼

並列摘要

HEVC is intended to provide significantly better coding efficiency than H.264/ AVC and its predecessors. One key contributor to this performance gain is the updated version of intra prediction that extended a large number of prediction directions on various sizes of prediction units (PUs), thus at a cost of very high computational complexity. More specifically, it has (at most) 35 intra modes, more PU blocks of size 64 x 64, 32 x 32, 16 x 16, 8 x 8, and 4 x 4 as well. Consequently, the PU mode decision and intra prediction would cost around 60% to 70% encoding time in the all-intra prediction HEVC encoding. Therefore, the goal of this study is intending to lower the computational complexity of HEVC intra prediction plus reduce the total encoding time of HEVC as well.
The key step of the proposed method is to intelligently elect an indispensable set of directions with the help of a modified LeNet-5 CNN model, thus reduce the computational complexity of further rate distortion optimization. About the modified LeNet-5 model, we first replace the tanh and sigmoid function with the Rectified Linear units, then we use zero padding to maintain the information of the input PU. Finally, we replace the Rectified Linear units of the 2nd convolutional layer with the maxout units. Besides, the edge strength extractor in [6] to determine the current PU is flat or not is adopted to skip most of the direction modes. And the early terminated CU partition technique in [18] is used to decrease the number of CUs. Finally, the candidates of neighboring PU is considered to be selected also. The experimental results demonstrate that the proposed method provides a decrease of up to 66.59% in the HEVC intra prediction processing time, with a little increase in the bit-rate (1.1% on average) and a reduction of 0.109% on average in PSNR values at most.

Key words: Intra prediction, CNN, edge strength extractor, early terminated CU partition, HEVC

並列關鍵字

Intra Prediction ； CNN ； edge strength extractor ； early terminated CU partition ； HEVC

參考文獻

[1] G. J. Sullivan, J. Ohm, Woo Jin Han, and T. Wiegand, “Overview of the high efficiency video coding (hevc) standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649–1668, 2013.

[2] G. J. Sullivan and T. Wiegand, “Video compression - from concepts to the h.264/avc standard,” Proceedings of the IEEE, vol. 93, no. 1, pp. 18–31, 2005.

[3] Gary J. Sullivan and Jens Rainer Ohm, “Recent developments in standardization of high efficiency video coding (hevc),” Proc. SPIE 7798, Applications of Digital Image Processing XXXIII, 77980V, September 2010, pp. 731–739.

[4] Guang Chen, Zhenyu Liu, T Ikenaga, and Dongsheng Wang, “Fast hevc intra mode decision using matching edge detector and kernel density estimation alike histogram generation,” in IEEE International Symposium on Circuits and Systems, Beijing, China, 19-23 May 2013, pp. 53–56.

[5] Thałsa L. Da Silva, Luciano V. Agostini, and Luis A. Da Silva Cruz, “Fast hevc intra prediction mode decision based on edge direction information,” in 20th European Signal Processing Conference, Bucharest, Romania, 27-31 Aug. 2012, pp. 1214–1218.