基於像素級轉換的多尺度深度線上影片穩定

Multi-Scale Deep Online Video Stabilization with Pixel-Based Warping

陳禹達(Yu-Ta Chen)

指導教授 ： 洪一平

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202002926

摘要

本篇論文提出一個深度學習的方法來解決線上影片穩定的問題，我們建立了一個多尺度的架構，只需輸入給神經網絡當前不穩定的幀和歷史穩定過的幀而不需要任何未來的影像，即可實時的將不穩定影片轉換成穩定影片，另外我們的方法會生成像素級的轉換圖，相較於過去方法使用一個單應矩陣或者切成網格式的轉換在每一個像素的轉換可以更準確，除此之外，我們還提出了一個二階段訓練方式，可以讓訓練出來的結果更具有穩健性。從我們的實驗結果可以發現，我們方法相較於傳統方法減少了扭曲的現象，且比現有的基於深度學習的線上穩定方法表現更好，另外，相較於最先進的幾個影片穩定方法，我們的方法目前是最為快速的。

關鍵字

線上影片穩定 ； 深度學習 ； 多尺度架構 ； 像素級轉換

並列摘要

In this thesis, a learning-based method is proposed to solve the online video stabilization problems. We build a multi-scale architecture and can stabilize the unstable videos in real time after feeding current unstable frame and historical stable frames to the neural network without using any future frames. Our network can estimate a pixel-based warping map to make the transformations of each pixel more precise than just calculating a global homography or multiple homographies. Besides, a two-stage training method is proposed to train our network, which makes the network more robust. Experimental results show that our algorithm achieves comparable performance with traditional methods and has better results than the state-of-the-art online stabilization methods based on learning. Moreover, our approach has the highest processing speed than the state-of-the-art methods.

並列關鍵字

online video stabilization ； deep learning ； multi-scale architecture ； pixel-based warping

參考文獻

[1] G. Zhang, W. Hua, X. Qin, Y. Shao, and H. Bao, "Video stabilization based on a 3D perspective camera model," The Visual Computer, vol. 25, no. 11, p. 997, 2009.

[2] C. Liang and F. Shi, "Fused video stabilization on the pixel 2 and pixel 2 xl," Google Res. Blog, Mountain View, CA, USA, Tech. Rep, vol. 11, 2017.

[3] B. M. Smith, L. Zhang, H. Jin, and A. Agarwala, "Light field video stabilization," in 2009 IEEE 12th international conference on computer vision, 2009: IEEE, pp. 341-348.

[4] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in 2011 International conference on computer vision, 2011: Ieee, pp. 2564-2571.

[5] H. Bay, T. Tuytelaars, and L. Van Gool, "Surf: Speeded up robust features," in European conference on computer vision, 2006: Springer, pp. 404-417.