以視覺顯著時空區塊為基礎之適應性視訊內容調整

Motion-Tolerance Contextual Visual Saliency Preserving For Video Resizing

羅毅修(Yi-Shiou Luo)

指導教授 ： 陳敦裕

元智大學/電機通訊學院/電機工程學系/碩士(2011年)

摘要

在現今調整影片大小的方法中大多都會產生視覺上的不連續性，尤其是在當影片內容中有劇烈的變動時更為明顯。為了去解決這個問題，在一個連續的影片當中要去保護視覺上的連續性就要應該要去考慮到所感興趣資訊的一個前後關係。在這篇論文當中，我們為了去偵測所感興趣的motion-tolerance，我們提出一個新穎的方法去模組visual dynamics基於spatiotemporal slice (STS), STS可以提供在影片當中較長時間上的資訊。首先我們利用STS所分析出的資訊去自動決定所需的patch，並產生codebook. 當產生出codebook之後我們利用裡面所包含的patch去在影片中計算出重要區塊所在的位置並且此範圍含蓋了我們所感興趣的焦點。為了在影片的縮放中並還保有視覺上的連續性，我們利用基於網格非均勻扭曲的方法並且依照STS所分析的資訊對網格建立縮放的限制。在實驗結果的量測當中，我們利用patch-based Kullback-Leibler divergence的方法去評估縮放後的圖與原圖之間的差異程度。在實驗結果當中，我們的方法是基於STS的分析並進一步的達到縮放效果，並且能有效的維持視覺感官上的連續性。

關鍵字

運動相關 ； 時空上的切片 ； 以方格為基楚 ； KL差異

並列摘要

State of the art methods for video resizing usually produce perceivable visual discontinuities, especially for videos with significant motion. To tackle this problem, contextual information about the focus of interest in consecutive video frames should be considered for preserving visual continuity. In this paper, in order to detect the focus of interest with motion-tolerance we propose a novel approach for modelling visual dynamics based on spatiotemporal slice (STS), which can provide rich visual patterns along a large temporal scale. Patch-based visual patterns are first computed to generate a codebook in the automatic specified spatiotemporal extent that is determined by contextual information in STS. The codebook is then used to compute its associated response in video frames and eventually an importance map for a video clip covering the focus of interest can be obtained. To preserve the visual continuity particularly for important area, a mesh-based non-homogeneous warping constrained by the trajectories in STS is guided by a multi-cue approach. For performance evaluation, a novel measure using patch-based Kullback-Leibler divergence (KL-divergence) is proposed to evaluate the deformation of the focus of interest. Experimental results show that the proposed approach of video resizing based on STS can effectively generate retargeted videos while maintain their isotropic manipulation and the continuous dynamics of visual perception.

並列關鍵字

motion-tolerance ； spatiotemporal slice ； patch-based ； codebook ； KL-divergence

參考文獻

[1] M. Rubinstein, A. Shamir, Shai Avidan, Improved seam carving for video retargeting, ACM Transactions on Graphics 27, 2008.

[2] L. Q. Chen, X. Xie, X. Fan, W. Y. Ma, H. J. Zhang, H. Q. Zhou, A visual attention model for adapting images on small displays, ACM Multimedia Systems Journal 9, 2003, pp. 353–364.

[3] M. Rubinstein, A. Shamir, S. Avidan, Improved seam carving for video retargeting, ACM Transactions on Graphics 27, 2008

[5] H. Liu, X. Xie, W. Y. Ma, H. J. Zhang, Automatic Browsing of Large Pictures on Mobile Devices, In Proceedings of ACM International Conference on Multimedia, 2003, pp. 148–155.

[6] B. Suh, H. Ling, B. B. Bederson, D. W. Jacobs, Automatic thumbnail cropping and its effectiveness, In Proceedings of UIST, 2003, pp.95–104.