用於視訊中人臉置換之方位與打光估計

我們提出一個用於視訊的人臉置換(face replacement)系統。這個系統以三維形變模型為基礎，對於來源人物(source subject)我們只需要拍攝一到兩張照片，就可以將擬被替換之目標人臉(target face)替換為同一方位(pose)、表情(expression)及光照(illumination) 下之來源人臉(source face)。關於人臉置換的相關研究不少，但多是用於圖片，市面上亦存在很多影像處理軟體可以作圖片上的人臉替換，前提是目標人臉和來源人臉的方位不能差太多，這類軟體在處理上非常耗時耗工，且需要具有一定的專業技術，我們所提出的系統相較於他們，因為是3D操作，可生成不同方位之人臉，使用上較為簡單且省時省力。這個系統可以應用於電影特效工業或一般的娛樂活動。我們的系統共由三個人的工作組合而成，總共分成以下幾個主要模組，包括：人臉校準、方位估計、人臉擷取、三維人臉模型生成、表情估計及生成、打光估計及人臉重新打光、人臉影像生成、影像合成。在這篇論文中，我的模組主要探討的主題是方位和打光估計。

關鍵字

人臉；人臉置換；方位估計；打光估計；三維模型

並列摘要

In this thesis, we present a system for face replacement in video. Our approach is based on 3D morphable model, so the input is reduced to one or two pictures for the source face to be placed in. Then, we can replace the target face in the target video with the source face, under similar pose, expression, and illumination. Most digital processing software can perform image face replacement only when the poses for the source and target faces are similar, and the manipulation process with those software is often time-consuming and labor-intensive. While most previous work also focuses on image face replacement, our system performs face replacement in video. Our system is useful in the special effect industries and entertainment. Our system consists of several modules: face alignment, pose estimation, expression estimation, face extraction, morphable model fitting, face synthesis, lighting estimation, and composition. In this thesis, we focus on pose and lighting estimation.

並列關鍵字

face ； face replacement ； pose estimation ； lighting estimation ； 3D model

參考文獻

[3] Blanz V, Vetter T: A morphable model for the synthesis of 3D faces. In Computer Graphics Proc.SIGGRAPH’99 (Los Angeles, 1999), pp. 187–194.

[5] D. Bitouk, N. Kumar, S. Dhillon, P. N. Belhumeur, and S. K. Nayar. Face swapping:automatically replacing faces in photographs. ACM Transactions on Graphics(SIGGRAPH), 27(3), 2008.

[6] H. Wang, R. Raskar, N. Ahuja, 2004. Seamless video editing. In Proceedings of the International Conference on Pattern Recognition (ICPR), 858–861.

[7] Y. Cheng. 3D Morphable Model based face replacement in video. Master’s thesis, CSIE Department,National Taiwan University, 2009.

[10] Basri, R. and Jacobs, D. W. 2003. Lambertian reflectance and linear subspaces. IEEE TPAMI 25, 218–233.

國際替代計量

用於視訊中人臉置換之方位與打光估計

全文下載

主題瀏覽