正面化與自適性指數加權平均組合之基於深度學習的表情辨識

Frontalization and Adaptive Exponentially Weighted Average Ensemble Rule for Deep Learning Based Facial Expression Recognition

蔡開遠(Kai-Yuan Tsai)

指導教授 ： 丁建均

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2018年)

https://doi.org/10.6342/NTU201800990

摘要

如今，自動人臉表情辨識在人機介面及監控系統為一項極重要的技術，在模式識別及電腦視覺領域已經吸引大量的關注。
自動人臉表情辨識系統會接收一個輸入資料(靜態人臉影像或動態人臉序列)並且將其辨識為一個基本表情(生氣、難過、驚訝、開心、厭惡、恐懼、中立…等)， 我們的目標在著重於靜態人臉影像，並且辨識為七種表情狀態。在這篇論文中，我們提出使用人臉翻正演算法及自適性指數加權組合架構的卷積神經網路的人臉表情辨識系統。翻正演算法用於對齊小角度人臉旋轉(平面上及平面外)並且利用人臉偵測方法來去除多餘的背景雜訊達到資料歸一化，而自適性指數加權組合架構能夠藉由模型本身優劣程度找出適當的加權參數及組合方式強化自動表情辨識系統的穩定性。因此，根據我們提出的系統，在一些常見的資料庫進行實驗，模擬結果顯示以上提出的方法對於人臉表情辨識皆比過往的表情辨識算法結果要好。
關鍵字: 人臉表情;卷機神經網路;電腦視覺;人臉翻正化;階層式架構。

關鍵字

人臉表情 ； 卷機神經網路 ； 電腦視覺 ； 人臉翻正化 ； 階層式架構

並列摘要

Nowadays, Automatic Facial Expression Recognition (FER) is an important technique in human-computer interfaces and surveillance systems, has attracted significant attention in pattern recognition and computer vision.
Automatic systems for facial expression recognition receive the input (a static facial image or a facial image sequence) and classify it into one of the basic expressions (anger, sad, surprise, happy, disgust and fear, neutral and so on). Our work will focus on methods based on facial static images and it will consider the seven basic expressions. In this paper, we proposed a CNN based system with face frontalization and Hierarchical architecture for FER. The frontalized algorithm can align the small angle rotation (in-of-plane or out-of-plane) and use the face detection to remove the background noise, the adaptive exponentially weighted average ensemble rule can search the optimal weight according to the efficiency of classifier to improve the robust FER system.
As a result, we perform the proposed system on some popular databases, the simulation results show that it is very effective for facial expression recognition, we achieve an accuracy rate surpassing the state-of-the-art system.

Keyword: facial expression; convolutional neural networks; computer vision; face frontalization; hierarchical structure.

並列關鍵字

facial expression ； convolutional neural networks ； computer vision ； face frontalization ； hierarchical structure

參考文獻

[1] Ekman, Paul. "Facial expression and emotion." American psychologist 48.4 (1993): 384.

[2] P. Liu, S. Han, Z. Meng, and Y. Tong, “Facial expression recognition via a boosted deep belief network,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1805–1812.

[3] C. Shan, S. Gong, and P. W. McOwan, “Facial expression recognition based on local binary patterns: A comprehensive study,” Image and Vision Computing, vol. 27, no. 6, pp. 803–816, 2009.

[4] Y.-H. Byeon and K.-C. Kwak*, “Facial expression recognition using 3d convolutional neural network,” in International Journal of Advanced Computer Science and Applications, vol. 5, no. 12, 2014.

[5] J.-J. J. Lien, T. Kanade, J. Cohn, and C. Li, “Detection, tracking, and classification of action units in facial expression,” Journal of Robotics and Autonomous Systems, 1999.