應用深度學習方法於動態臉部情緒之極性分析

A Deep Learning Approach for Polarity Analysis of Dynamic Facial Emotion

鄒玉慧(Yu-Hui Zou)

指導教授 ： 林斯寅

中原大學/商學院/資訊管理研究所/碩士(2018年)

https://doi.org/10.6840/cycu201800817

摘要

過去許多研究嘗試使用電腦視覺處理、統計學方法、與各種機器學習演算法， 在影像處理與資料分析上已獲得不錯的成果。直到近幾年，深度學習(Deep Learning)的改良，更使得模型預測的精準度有了重大的突破，電腦視覺領域也紛 紛將深度卷積神經網路(Convolutional Neural Networks, CNN)的方法引入至影像 分析與辨識的過程中。然而，在各個領域中，有效的臉部表情辨識方法，對於感 知使用者的情緒與設計成功的人機互動系統，都是非常重要的一環，過去已有許 多研究著重在利用電腦視覺或機器學習方法來做情感計算(Affective Computing)， 包括近年來的深度學習方法，但尚無法精確找出使用者動態的情緒變化模式，也 無法在較少量訓練資料的環境下獲得準確的預測效果。因此，辨識動態臉部表情 與情緒變化模式來做進一步的應用，仍然是一個具有挑戰性的任務。例如在精神 醫學的領域中，基於外界刺激而產生的動態情緒變化模式的辨識，對於重度憂鬱 症的診斷預測，就是一個重要的課題。本研究為「應用深度學習方法於動態臉部 情緒之極性分析」，將提出一個動態臉部情緒辨識分析模型與方法，此方法整合 深度卷積神經網路、長短期記憶網路與臉部情緒辨識分析方法，能透過連續的臉 部表情偵測與辨識，並取得使用者動態的情緒變化模式進行比對。

關鍵字

深度學習 ； 卷積神經網路 ； 長短期記憶網路 ； 轉移學習 ； 動態表情辨識 ； 情感計算

並列摘要

In the past, many studies apply computer vision processing, statistical methods, and machine learning algorithms for image processing and data analysis that have yielded good results. Furthermore, recent years, the improvement of deep learning methods have led to a great breakthrough in the accuracy of model predictions. The computer vision field has also put deep learning methods into the image analysis and recognition process. Moreover, effective facial expression recognition is a very important part of perceiving the user's emotions and designing a successful human- computer interaction system in all fields. Many studies also applied computer vision, machine learning, and deep learning methods on affective computing. However, it is still difficult to recognize the patterns of dynamic emotional changes. And it can’t obtain accurate prediction results in the situation of lesser training data. Therefore, to recognize dynamic facial emotion and mood changes for further application is still a challenging task. For example, in the field of psychiatry, to recognize dynamic facial emotion for the diagnosis and prediction of major depressive disorder (MDD) is an important issue. This research project is titled “A Deep Learning Approach for Polarity Analysis of Dynamic Facial Emotion”. This study integrates facial emotion recognition analysis with convolutional neural networks (CNN) and long short-term memory (LSTM) to find out the dynamic patterns of facial emotional changes. Through continuous facial emotion detection and recognition analysis, to obtain the user's dynamic pattern of facial emotional changes for comparison.

並列關鍵字

deep learning ； convolutional neural networks ； long short-term memory ； transfer learning ； dynamic emotion recognition ； affective computing

參考文獻

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... others. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. ArXiv Preprint ArXiv:1603.04467.

Baum, L. E., & Petrie, T. (1966). Statistical Inference for Probabilistic Functions of Finite State Markov Chains. The Annals of Mathematical Statistics, 37(6), 1554–1563.

Beale, R., & Peter, C. (2008). The Role of Affect and Emotion in HCI. In Affect and Emotion in Human-Computer Interaction (pp. 1–11). Springer, Berlin, Heidelberg.

Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157–166.

Bengio, Yoshua. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1–127.