以台灣年長者為對象之基於深度卷積類神經網路的自動臉部表情辨識

Automatic Facial Expression Recognition for Taiwanese Elders with Deep Convolutional Neural Network

葉德緯(De-Wei Ye)

指導教授 ： 鄭士康

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202100072

摘要

本論文以深度卷積類神經網路實作可用於台灣年長者的臉部表情辨識模型，並探討年齡效應對於卷積類神經網路模型的表情認知影響。該模型結合多個臉部表情圖片的資料庫訓練，並搭配適當的數據平衡，使模型具有跨情境的穩健預測準確度。另藉由遷移學習領域的微調方法，能使模型在少量額外資料的幫助下，使台灣年長者的臉部表情辨識準確率進一步提升。實驗結果顯示，卷積類神經網路模型對台灣人的臉部表情辨識整體準確度優於人類與使用人工特徵的傳統電腦方法，其對不同年齡族群的辨識結果差異也較人類與傳統方法小。如同人類以及傳統電腦方法，卷積類神經網路模型從老人表情中接受到的情緒強度仍然比年輕人弱。而藉由應用可解釋人工智慧的方法，我們也視覺化卷積類神經網路模型分類的依據，並發現臉部肌肉的弱化與皺紋影響卷積類神經網路模型。整體而言，年齡效應對卷積類神經網路模型的影響與人類有相似之處，但實際上仍比較類似傳統人工特徵方法。

關鍵字

臉部表情辨識 ； 年長者看護 ； 深度學習 ； 遷移學習 ； 可解釋人工智慧

並列摘要

This thesis has implemented a robust cross-dataset facial expression recognition system for Taiwanese elders based on a deep convolutional neural network (CNN). We investigate how aging affects the recognition of expressions for the CNN model. The CNN model is trained with combined datasets with data balancing to let the model be robust against unseen datasets. Also, the recognition performance of Taiwanese elders is improved via fine-tuning on small amounts of extra data. The CNN model outperforms human raters and the handcrafted-feature method on Taiwanese faces. The recognition accuracy difference between old faces and young faces is smaller than human raters and the handcrafted-feature method. Still, the CNN model perceives weaker emotional intensity on old faces, and this property is similar to human raters and the handcrafted-feature method. With the assistance of the XAI method, we can visualize the discriminative parts in the CNN model. The visualization implies that weakened facial muscles and wrinkles still affect the expression recognition for the CNN model. Overall, the CNN model resembles the handcrafted-feature method more than human raters when perceiving facial expressions for the elderly.

並列關鍵字

Facial expression recognition ； Elderly care ； Deep learning ； Transfer learning ； Explainable artificial intelligence

參考文獻

[1] World Health Organization, “Global health and aging,” 2011. https://www.who.int/ageing/publications/global_health.pdf.

[2] C.-P. Liu, “Empathetic generative-based chatbot with emotion understanding via reinforcement learning,” Master’s thesis, Department of Electrical Engineering, National Taiwan University, 2020.

[3] J.-H. Jhan, “Empathetic and retrieval-based chatbot using deep reinforcement learning,” Master’s thesis, Graduate Institute of Communication Engineering, National Taiwan University, 2020.

[4] C. Sirithunge, A. B. P. Jayasekara, and D. Chandima, “Proactive robots with the perception of nonverbal human behavior: A review,” IEEE Access, vol. 7, pp. 77308–77327, 2019.

[5] S. Li and W. Deng, “Deep facial expression recognition: A survey,” IEEE Transactions on Affective Computing, 2020.