基於三維主動外觀模型以二維及三維影像定位大偏行角變化下之人臉特徵

基於外觀(appearance)的三維形變模型(Deformable model)能夠透過觀察到的外觀以及利用形變處理臉部特徵的定位。然而，當人臉於偏航角(yaw angle)左右擺動時，於二維影像上能見的臉部特徵會大量減少，使得演算法難以從臉部特徵的外觀獲取資訊，因此，傳統的方法在面臨大偏航角的臉部特徵定位是有困難的。由於外觀的能用資訊的減少，我們引入三維點雲(3D point cloud)資訊使得臉部特徵定位獲取更多額外的訊息。利用三維點雲的資訊，我們可以得到足夠的三維結構並且更精準的估測人臉的角度，然而，三維點雲卻缺少詳細的人臉特徵輪廓的資訊。在此篇論文中，我們使用了三維點雲定位(3D point cloud matching)做三維空間的臉部定位與模型的形變，並且利用傳統二維特徵定位的訊息做為搜尋空間與形變的限制，同時，將此演算法與三維主動外觀模形結合(3D Active Appearance Model)以獲得更好的結果。為了有效運用兩種資料，我們同時也估測了二維限制以及三維形變的可信度，因此，我們可以獲取臉部的三維結構等額外訊息並利用分析資訊的可信度以增加臉部特徵定位的成功率以及更精確的角度估測，最後應用此結果重建出人臉的立體結構。以下簡單描述我們的演算法：首先，三維主動外觀模型會先由使用者所標記的資料中學習模型的基底。接著，當相機回傳二維影像和三維點雲資訊時，基於二維外觀的特徵定位演算法會先透過模型基底的線性組合以形變模型做為二維空間形變的限制，接著，我們會藉由三維基底模型的形變去定位三維點雲的結構，而二維外觀特徵定位的結果會做為三維模型結構於影像平面上投影的限制，此限制強迫三維結構定位的投影結果必須滿足二維的定位結果。同時，二維限制的可信度以及三維點雲資訊的可信度會被計算，用以保證演算法的結果能同時滿足二維和三維的定位，此二階段的演算法會重複直到二維和三維的定位結果都已經收斂。由於此演算法的流程會利用外觀和三維點雲的結構主動調整三維外觀模型，在演算法收斂時，二維的臉部特徵定位、人臉的角度估測和三維立體結構皆可以從形變模型的結果獲得。

關鍵字

三維主動外觀模型；人臉特徵定位

並列摘要

Method using 3DAAM have advantages in handling face deformation and alignment. However, they tend to fail when human head is under wide yaw angles, due to the dramatic reduction of the visible facial features, which decreases the information useable by the alignment. As for 3D data alignment, due to the property of the 3D data which provides 3D structure, the alignment is able to capture the 3D structure and angle but loses the precise location of features. 3D and appearance based alignment suffers from different issues and is reliable in different situations. For dealing with wide yaw angle face alignment under the consideration of locating the features, and angle estimation, we introduce an approach to estimate the belief between 3D and 2D alignment. The algorithm incorporate 3D data alignment into 3D Active Appearance Model (3D AAM) fitting using a Time-of-Flight camera and a RGB camera. The proposed algorithm benefits from both 3D structure and texture information by estimating the belief between 3D and 2D alignment, and results in a more stable alignment with 126% improvement maximally, compared with the 3D AAM using 2D image only. Experiments demonstrate the improvement using real-world data. Its application to 3D structure recovery is also demonstrated.

並列關鍵字

3D Active Appearance Model ； Face Alignment

參考文獻

Amberg, B., Romdhani, S., & Vetter, T. (2007). Optimal step nonrigid icp algorithms for surface registration. In IEEE Conference on Computer Vision and Pattern Recognition.

Besland, P. J., McKay, & D., N. (1992). A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

Blanz, V. & Vetter, T. (2003). Face recognition based on fitting a 3d morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 1063–1074.

Bustard, J. & Nixon, M. (2010). 3d morphable model construction for robust ear and face recognition. In IEEE Conference on Computer Vision and Patern Recognition.

Cai, Q., Gallup, D., Zhang, C., & Zhang, Z. (2010). 3d deformable face tracking with a commodity depth camera. In European Conference on Computer Vision.

國際替代計量

基於三維主動外觀模型以二維及三維影像定位大偏行角變化下之人臉特徵

全文下載

主題瀏覽