基於解碼分割資訊及深度學習之遺失視訊像素恢復演算法

A recovery method for lost video pixels based on decoding partition information and deep learning

丁采靈(Tsai-Ling Ding)

指導教授 ： 林鼎然

中原大學/電機資訊學院/電子工程研究所/碩士(2016年)

https://doi.org/10.6840/cycu201600317

摘要

本論文介紹新的運動向量恢復和錯誤隱藏的演算法，使用H.264/AVC
編碼的分割資訊。每個遺失像素位置的運動向量使用可用的相鄰運動向量，這有助於遺失的運動向量與他們之間距離反比的產生，運動外插的方法被使用在預測編碼分割資訊，從先前張畫面對應到當前畫面中遺失像素的不同級別的重疊，不同級別的重疊可以幫助判斷估計在遺失的MB中的編碼分割資訊，最後，被判斷在相同估計分割中的像素分享相同的運動向量，為了保持遺失MB估計移動物體的完整性。本論文所提出的分割的基於像素的運動向量的方法(PMVP)與最先進的Zhou’s方法[8]、Lin’s方法[9]和Lie’s方法[10]比較，對於在封包遺失率為3%、7%、16%和20%的總平均，PMVP分別比Zhou[8]高0.88 dB、1.02 dB、1.05 dB和1.01 dB；分別比Lin[9]高0.22 dB、0.32 dB、0.35 dB和0.33 dB；分別比Lie[10]高4.12 dB、4.98 dB、4.15 dB和3.88 dB。因此，所提出的PMVP在平均上，在所有方法中表現是最好的。在本論文中也介紹了深度學習的基本架構，及如何應用在錯誤補償。

關鍵字

錯誤隱藏 ； 運動向量恢復 ； H.264/AVC ； 解碼分割資訊 ； 深度學習

並列摘要

This paper presents a novel motion vector recovery and error concealment algorithm with the utilization of encoding partition information for H.264/AVC. The motion vectors for each missing pixel location are derived using available neighboring pixel motion vectors, which contribute to the generation of the missing motion vectors inversely proportional to the distance between them. The motion extrapolation method is used to project the encoding partition information from the reference frame into the current frame with different levels of overlapping of lost pixels. The different levels of overlapping can help determine the estimated encoding partition information in the lost MB. Finally, the pixels that are determined to be of the same estimated partition share the same motion vector in order to maintain the integrity of the estimated moving objects in the lost MB. This proposed pixel-based motion vector with partition (PMVP) method compares with the the state-of-the-art Zhou’s method [8], Lin’s method [9], and Lie’s method [10]. For total average in packet loss rates of 3%, 7%, 16%, and 20%, PMVP is better than Zhou [8] by 0.88 dB, 1.02 dB, 1.05 dB, and 1.01 dB, respectively; Lin [9] by 0.22 dB, 0.32 dB, 0.35 dB, and 0.33 dB, respectively; and Lie [10] by 4.12 dB, 4.98 dB, 4.15 dB, and 3.88 dB, respectively. Therefore, the proposed PMVP performs the best on average among all the methods. In this paper also describes the basic architecture of the deep learning, and how to apply error compensation.

並列關鍵字

Error concealment ； motion vector recovery ； H.264/AVC ； deep learning ； decoding partition information.

參考文獻

[3] "Video coding for low bit rate communication," Int. Telecommunication Union-Telecommunication (ITU-T) Rec. H.263, 1995.

[4] Q. Peng, T. Yang, and C. Zhu, "Block-based temporal error concealment for video packet using motion vector extrapolation " IEEE International Conference on Communications, Circuits and Systems and West Sino Expositions, vol. 1, pp. 10-14, July 2002.

[5] B. Yan and H. Gharavi, "A hybrid frame concealment algorithm for H.264/AVC," IEEE Transactions on Image Processing, vol. 19, pp. 98-107, January 2010.

[6] D. Agrafiotis, D. R. Bull, and C. N. Canagarajah, "Enhanced error concealment with mode selection," IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, pp. 960-973, August 2006.

[7] Y. Xu and Y. Zhou, "H.264 video communication based refined error concealment schemes," IEEE Transactions on Consumer Electronics, vol. 50, pp. 1135–1141, November 2004.