基於深度學習之連續臉部情緒模式辨識方法應用於學習情緒

近年來因為數位學習與應用的快速發展，在教學互動的環境之中，得知學生上課時的學習情緒(academic emotions)是非常重要的。學習情緒是一種在學習的過程中，學生會隨著學習行為的表現，而呈現出的臉部表情，教師能夠利用學生的學習情緒，來提供給學習者最適當的內容、反饋、提示、練習或測驗題目，是達成因材施教的教學理念很重要的一環，目的是根據使用的情緒來調整最適合學習者程度的教材內容，來提升學習者的學習成效與動機，未來也能夠應用在適性化學習(adaptive learning)上。然而，過去對於學習情緒的研究，大部分的方式是經由課後訪問學習者，來得知學習者對於上課內容的情緒。過去為了改善即時性的問題，也有相關研究嘗試使用臉部辨識與情緒辨識技術在學習情緒上，但此類的研究多以傳統辨識方法針對單一照片作分析與辨識為主，並未考慮到學習情緒的表達為一段時間之學習情況的連續表情。為了解決以上問題，本論文將提出一個基於深度學習之連續臉部情緒模式辨識方法應用於學習情緒。本論文實驗中將結合深度學習方法中的卷積神經網路和遞迴神經網路，分析與辨識學生的連續臉部學習情緒模式，得到學習情緒的辨識結果，並且幫助未來的學習系統能夠更快速且準確的瞭解學生的學習狀況，作出即時的回應，給予適合學生程度的教材或題目，來提升學習者的學習成效與動機。在本論文中，實驗結果得出卷積神經網路辨識模型的準確率為72.47%，卷積神經網路加遞迴神經網路實驗結果準確率為84.66%，兩種神經網路結合使用比單純使用卷積神經網路提高了約12%左右的準確率，證明卷積神經網路加上遞迴神經網路是可行的。

關鍵字

學習情緒；臉部情緒辨識；深度學習；卷積神經網路；遞迴神經網路

並列摘要

In recent years, due to the rapid development of digital learning and its applications, the concept of academic emotions has become more and more important in teaching interaction. Academic emotions is a kind of system process, which can automatic and dynamic adjustment of the next step in the system according to the performance of the learner's behavior and assessment results in the process to provide the most appropriate teaching content or testing materials. The purpose of academic emotions is improve their learning effectiveness and motivation. However, in the past, the adjustment method of traditional adaptive learning adjusts teaching content according to the learning feedback status until the learners complete a whole learning process, resulting in the problem of poor immediacy of adaptive presentation. In order to improve the problem of immediacy, some related researches have tried to apply face recognition and emotion recognition technology in academic emotions to detect the learner’s status in real time. However, this study will combine convolutional neural networks and recurrent neural networks in the deep learning method to analyze and recognize learners' continuous facial and learning emotion patterns on adaptive learning systems. The proposed model can improve the problem of immediacy feedback in adaptive learning by recognizing learning emotions in real time. It can also help the adaptive learning system to understand the learners learning status more quickly and accurately, and to respond instantly. This proposed method can give students the appropriate level of teaching materials in the process of adaptive learning to enhance learners' learning outcomes and motivations. The final accuracy in this study is 84.66%.the accuracy is better than CNN model.

並列關鍵字

academic emotions ； face emotion recognition ； deep learning ； convolutional neural networks ； recurrent neural networks

參考文獻

[1]Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., … others. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. ArXiv Preprint ArXiv:1603.04467.

Google Scholar

[2]Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1–127.

Google Scholar

[3]Ciresan, D. C., Meier, U., Masci, J., Maria Gambardella, L., &Schmidhuber, J. (2011). Flexible, high performance convolutional neural networks for image classification. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence (Vol. 22, p. 1237).

Google Scholar

[4]Ciregan, D., Meier, U., & Schmidhuber, J. (2012). Multi-column deep neural networks for image classification. In 2012 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (pp. 3642–3649). IEEE.

Google Scholar

[5]David E. Rumelhart, Geoffrey E. Hinton & Ronald J. Williams (1986)."Learning representations by back-propagating errors."Nature 323(1),533–536.

Google Scholar

國際替代計量

基於深度學習之連續臉部情緒模式辨識方法應用於學習情緒

全文下載

主題瀏覽