Translated Titles

3D human face reconstruction and recognition by using the techniques of multi-view synthesizing method with the aids of the depth images





Key Words

三維人臉重建 ; 三維人臉辨識 ; 3D Face Reconstruction ; 3D Face Recognition



Volume or Term/Year and Month of Publication


Academic Degree Category




Content Language


Chinese Abstract

過去三維重建或辨識系統,大多數透過二維彩色影像和二維深度影像計算出三維空間座標,再進行三維影像,但這樣的運算量往往需耗費相當大的運算成本。本論文改採用可保留特徵向量方向性與彩色資訊的點雲系統,做為三維人臉重建與辨識的研究。點雲資料與二維影像不同的地方,傳統二維影像處理需不停計算二維影像每個像素點的資訊。點雲系統可以將二維彩色影像與二維深度影像合成為具三維空間座標的點雲模型做處理。可減少二維影像轉換三維模型運算複雜度,並建立三維空間座標KD-Tree查詢系統,加速三維模型查詢關鍵點的搜尋時間。 一般三維人臉重建取樣的樣本,往往需要使用設備昂貴的雷射掃描器,而本論文採用Microsoft KINCET感應器。它與昂貴的雷射掃描器相較下是屬於成本較為低廉的,KINECT具有深度資訊及彩色影像資訊,實驗環境以180°多視角掃描真實人臉三維表面影像,利用ICP演算法(Iteration Closest Point, ICP) 進行多視角人臉匹配,重建三維人臉模型。人臉辨識採用3D SIFT (3D Scale Invariant Feature Transform) 演算法提取人臉特徵關鍵點,並使用歐式距離計算三維空間座標特徵點與特徵點距離的權重關係。本論文提出的辨識方法在GavabDB公用資料庫辨識率可以達到83.6%。

English Abstract

In the past years, most of the three-dimensional reconstruction or recognition systems use two-dimensional image and its depth image to calculate the three-dimensional coordinates of the image to process the three-dimensional theme. Such operations usually take a considerable amount of computing costs. This research proposes another approach, point cloud, which can preserve feature vectors and color information, for three-dimensional face reconstruction and recognition. In the conventional 2D approach, it keeps tracking the information of each pixel of the 2D image. On the other hand, the point cloud system directly synthesizes the 2D image and its depth image into a point cloud model with 3D coordinates. Therefore it can reduce the computation complexity significantly. It can further construct a 3D space coordinate KD-Tree query system to accelerate the query search speed for searching the key points of the 3D coordinate. Generally, it uses some expensive equipments and laser scanners for three-dimensional facial reconstruction. In this research we try to use the Microsoft KINCET sensor to reconstruct the 3D human face. Compared with the expensive laser scanner, KINECT has the characteristics of cheap cost and can find the information of color image and depth image. In this research KINET is used to scan the human face in multi-view within 180. Then we use the iteration closest point (ICP) algorithm to match the multi-view human faces. By this approach the 3D data base group points of the human face can thus be established. The three-dimensional face model point cloud data via 3D SIFT (3D Scale Invariant Feature Transform) algorithm is applied to extract the feature key points. Then we use the three-dimensional coordinates of Euclidean distance to calculate the feature points and feature weights distance relationship to determine whether the face belongs to the same person. The experimental results show that under Gavab DB face database our approach has the recognition rate of 83.6%.

Topic Category 工學院 > 電機工程學系碩士班
工程學 > 電機工程
  1. [4] C. Zhang and T. Chen, “3A Self-Reconfigurable Camera Array,” in Eurographics Symposium on Rendering, 2004.
  2. [6] 陳煌杰:“獲取多方向影像於三維顏面模型重建 ”,國立台北科技大學土木防災研究所碩士論文,民國100年7月。
  3. [9] Ugur Halici and Tolga Inan, “3-D Face Recognition With Local Shape Descriptors,” IEEE Transactions on Information Forensics and Security, vol.7, no.2, pp.577,587, April 2012.
  4. [11] D. Huang, G. Zhang, M. Ardabilian, Wang, and L. Chen, “3D face recognition using distinctiveness enhanced facial repre- sentations and local feature hybrid matching,”In Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–7, 2010.
  5. [12] T. C. Faltemier, K. W. Bowyer, and P. J. Flynn, “A region ensemble for 3D face recognition,”IEEE Transactions on Information Forensics and Security, 3(1):62–73, 2008.
  6. [13] C. McCool, V. Chandran, S. Sridharan, and C. Fookes, “3D face verification using a free-parts approach,”Pattern Recognition Letter, vol. 29, no. 9, pp. 1190–1196, Jul. 2008.
  7. [14] K. Sobottka and I. Pitas, “Face localization and facial feature extraction based on shape and color information,” 1996 Internet Conference on Image Processing, pp.483-486 vol.3, 16-19 Sep 1996.
  8. [15] M. Yachida, H. Wu, and Q. Chen, “Face Detection from Color Images Using a Fuzzy Pattern Matching Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no.6, pp. 557-563, June 1999.
  9. [16] H. Wu, Q. Chen, and M. Yachida, “Face Detection from Color Images Using a Fuzzy Pattern Matching Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 6, pp. 557-563, June 1999.
  10. [17] K. C. Yow and R. Cipolla, “A Probabilistic Framework for Perceptual Grouping of Features for Human Face Detection Method,”The Second International Conference on Automatic Face and Gesture Recognition, pp. 16-21, 1996.
  11. [19] H. Drira, B. Ben Amor, A. Srivastava, M. Daoudi and R. Slama, “3D Face Recognition under Expressions, Occlusions, and Pose Variations, ” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol.35, no.9, pp.2270-2283, Sept. 2013.
  12. [20] T. Sakai, M. Nagao, and S. Fujibayashi, “Line Extraction and Pattern Detection in a Photograph,” Pattern Recognition, vol.1, pp. 233-248, 1969.
  13. [21] I. Craw, H. Ellis, and J. Lishman, “Automatic Extraction of Face Features,” Pattern Recognition Letters, vol. 5, pp.183-187, 1987.
  14. [26] P. J. Besl and N. D. McKay, “A Method for Registration of 3D Shapes,” Trans. PAMI, vol.14, no. 2, pp. 239-254, 1992.
  15. [27] J. Friedman, J. Bentley, and R. Finkel,“An algorithm for finding best matches in logarithmic expected time,” ACM Trans. Math. Software, no. 3, pp. 209–226, Sep.1977.
  16. [29] D. G. Lowe, “Object recognition from local scale-invariant features,” IEEE International Conference on The Proceedings of the Seventh, vol. 2, pp.1150-1157, Sep.1999.
  17. [31] A.P. Witkin,“Scale-space filtering,” In International Joint Conference on Artificial Intelligence, Karlsruhe, Germany, pp. 1019-1022, 1983.
  18. [33] P. Scovanner, S. Ali, and M. Shah,“A 3-dimensional Sift Descriptor and its Application to Action Recognition,” In In Proc. of MULTIMEDIA '07, pages 357-360, 2007.
  19. [1] http://www.face-rec.org/databases/
  20. [2] http://www.xbox.com/zh-TW/Kinect
  21. [3] B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams,M. Levoy, Mark H, “High Performance Imaging Using Large Camera Arrays,” In Proc. SIGGRAPH, 2005.
  22. [5] Wen-Kuo Hse and Pi-Ling Pai, “33D Modeling and Application from Panoramic Photography,” 2012台灣地理資訊學會年會暨學術研討會 , 2012.
  23. [7] 陳昭文,“多視角取像與三維重建技術,”影像與識別期刊, Vol.16 , No.2 , 2010.
  24. [8] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. A. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. J. Davison, and A. Fitzgibbon, “KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera,”In Symposium on User Interface Software and Technology (UIST), 2011.
  25. [10] Y. Lee, H. Song, U. Yang, H. Shin, and K. Sohn, “Local feature based 3D face recognition,”In Proceedings of Audio- and Video-Based Biometric Person Authentication, AVBPA, pp. 909–918, 2005.
  26. [18] C. Hesher, A. Srivastava, and G. Erlebacher, “A novel technique for face recognition using range imaging,” in Proc. 7th Int. Symp. Signal Processing and Its Applications, 2003, Jul. 2003, vol. 2, pp. 201–204.
  27. [22] A. Yuille, P. Hallinan, and D. Cohen, “Features Extraction From Faces Using Deformable Templates,” 1989 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '89), pp.104-109, 1989.
  28. [23] http://pointclouds.org
  29. [24] 朱德海,郭浩,蘇偉,點雲庫PCL學習教程,北京航空航天出版社,2012-10-1.
  30. [25] http://blogs.msdn.com/b/kinectforwindows/archive/2012/01/20/near-mode-what-it-is-and-isn-t.aspx
  31. [28] http://en.wikipedia.org/wiki/K-d_tree
  32. [30] C. Maes, T. Fabry, J. Keustermans, D. Smeets, P. Suetens, D. Vandermeulen, ”Feature Detection on 3D Face Surfaces for Pose Normalization and Recognition,” 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp.1-6, Sept. 2010.
  33. [32] http://wenku.baidu.com/view/cf66d0b465ce050876321327.html
  34. [34] http://www.gavab.es/recursos_en.html
  35. [35] A.B. Moreno, A. Sanchez, E. Frias-Martinez and J.F. Velez,“Three-dimensional facial surface modeling applied to recognition,” Engineering Applications of Artificial Intelligence, vol.22, pp.1233-1244, 2010.
  36. [36] M. H. Mousavi, K. Faez, and A. Asghari,“Three dimensional face recognition using svm classifier,” In ICIS ’08: Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science, pp. 208–213, Washington, DC, USA,2008.