視訊人物分割及背景替換

Human Segmentation from Video for Background Substitution

李郁慈

指導教授 ： 賴尚宏

國立清華大學/電機資訊學院/資訊系統與應用研究所/碩士(2012年)

https://doi.org/10.6843/NTHU.2012.00547

摘要

在本篇論文當中，我們提出了一個新演算法可以自動從視訊切割人物並替換影像背景得到一組新的影像序列。在這個任務中，如何快速且有效的取得人物分割結果是最主要的挑戰。首先，我們提出在視訊中追蹤人物動作的訊息來改進Random Walk演算法中的先前形狀模型，利用時間上的一致性來保留人型的完整。另外，我們合併亮度與邊緣資訊差異於節點間的權重定義並取能量最小化讓相似的點盡量收斂到同樣的分割。我們的實驗結果展現出我們可以有效率的獲得相當準確的人物分割結果。
另外，我們在多核心平台PACDuo上實作系統的初步架構，基於平台資源有限，我們提出使用TYPE和INDEX的演算法來有效降低計算量，最後再使用針對多核心的資源配置策略與局部的資料傳輸到DSP核心與ARM處理器同步運算。最後的實驗結果顯示兩種方法都有效的降低系統在多核心處理平台執行時間。

關鍵字

背景替換

並列摘要

In this thesis, we propose an automatic video conferencing system for background substitution. Since humans are the principal subject in these videos, our framework is based on human shape clues to separate humans from complex background and replace or blur the background for immersive communication. We first detect face position and size, track human boundary across frames, and propagate the segmentation likeihood to the next frame for obtaining the trimap to be used as input to the Random Walk algorithm. Besides, we also include gradient magnitude in edge weight to enhance the Random Walk segmentation results. In this part, we demonstrate the effectiveness of the proposed background substitution system through experiments on some real videos.
We also present a system based on a multi-core processing architecture. Two tables, TYPE and INDEX, are introduced to fast locate the required data for the close-form solution. We demonstrate the parallelization strategies for the proposed fast RW algorithm and face detection on heterogeneous multi-core embedded platform to make the most use of the system architecture. Compared to the single processor implementation, the experimental results show significant speedup of the parallelized human background substitution system on a multi-core embedded platform, which consists of an ARM processor and two DSP cores.

參考文獻

[3] S. Kwak, I. Park, J. Lee, H. Byun, and G. Bae, “Automatic Background Substitution using Monocular Camera and Temporal Foreground Probability Model,” Proceeding of 2nd International Conference on Ubiquitous Information Management and Communication (ICUIMC), 2008, pp. 506-510

[5] A. Criminisi, G. Cross, A. Blake, and V. Kolmogorov, “Bilayer Segmentation of Live Video,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 53-60.

[6] P. Yin, A. Criminisi, J. Winn, and I. Essa, “Bilayer Segmentation of Webcam Videos Using Tree-Based Classifiers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, vol. 33, pp. 30-42.

[7] F. Zhong, X. Qin, and Q. Peng, “Transductive Segmentation of Live Video with Non-stationary Background,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2189-2196.

[8] C. Zhang, Y. Rui, and L.W. He, “Light Weight Background Blurring for Video Conferencing Applications,” IEEE International Conference on Image Processing (ICIP), 2006, pp. 481-484.