基於深度遷移學習於腦電波情緒分類

Adversarial-based deep transfer learning for EEG emotion classification

劉慶儀(Ching-Yi Liu)

指導教授 ： 陳永昇

國立交通大學/資訊學院/生醫工程研究所/碩士(2019年)

摘要

近年來，腦機介面系統成為科學界的主流，除了扮演大腦與外部設備相互溝通的工具，也透過神經生理訊號的偵測帶來情感腦機介面的研究。透過不同刺激材料激活不同腦區域為我們帶來有用的訊息來辨識腦部產生情感的狀態。近期有許多研究在腦連結的測量，例如相位鏈鎖值，應用神經網路在兩種情緒向度：正負向感受（負向/中性/正向）和喚醒度（低/中/高程度）得到不錯的分類結果。在本研究中，我們設計適合相位鏈鎖值特徵的神經網路，並加入注意力機制的輔助，篩選重要權重特徵，來進行分類。然而在腦電波訊號中不同的受試者間資料差異性過大是需要突破的難題，透過對抗式生成網路的概念延伸成對抗式遷移學習嘗試解決困難。在對抗式網路架構的部分包含來源編碼器、目標編碼器、領域分類器、情緒標記分類器。訓練的流程分為兩步驟，第一步，訓練來源編碼器以及情緒標記分類器，第二步，透過對抗式訓練可以將來源域的知識作為領域判斷器的基準，進而訓練目標編碼器以目標域的情緒標記來驗證。如此一來，對抗式訓練可以讓目標域的訓練特徵接近來源域的特徵，縮小其資料上的差距。我們選擇Wasserstein散度對抗式生成網路作為我們的對抗式訓練，其高效率以及穩定性讓我們在訓練時更好收斂，也利用開放式ＤＥＡＰ情緒分類資料庫來實作，最後發現透過注意力機制的幫助在神經網路的分類即可進步1-2%，而經由對抗式訓練的平均結果在正負向感受提升6.41％而在喚醒度提升了7.04%的結果，因此我們可以透過對抗式遷移學習的技術克服在個體間腦電波訊號的差異性。

關鍵字

深度學習 ； 遷移學習 ； 情緒分類 ； 腦電波

並列摘要

Over recent years, brain computer interface (BCI) systems have become an important topic. Aside from the original role as an inter-mediator between the human brain and external devices, the idea of neurophysiological signal detection was brought to fruition by various affective BCI studies. Functional connectivity among brain regions acitvated by emotional stimuli studies gives us essential information for recognizing the effective state of human brain. In the litereture, brain connectivity measurement as phase locking value (PLV) matrix has been applied to classify in two emotional dimensions: valence(negative/neutral/positive) and arousal(low/medium/high) using deep neural network models. Additionally, squeeze-and-excitation attention module was added into the neural network design to extract the essential information of model’s weights. However, differences among subjects not only unfortunately hinder potency of EEG data but also limit the classification performance. Hence, we propose an adversarial transfer learning architecture dealing with the cross-subjects issue in EEG classification. This archetecture comprises and two feature encoders (source encoder and target encoder), a domain classifier based on the generative adversarial networks (GAN) concept, and a label classifier. Furthermore, the training procedure is carried out by two steps, in which classifier training the source encoder and label classifier were first trained, followed by adversarial training for target encoder and domain classifier with purpose of diminishing the gap betweem source and target feature. Notably, by the virtue of the Wasserstein divergence GAN as adversarial component, the high efficacy and great stability was ensured in convergence training process. Moreover,the implementation of method in the public DEAP dataset for emotion recognition is implemented. According to our experiment, the attention mechanism gives around 5% improvement in CNN network architecture. Besides, the adversarial learning improve the accuracy of arousal and valence classification with 6.41% and 7.04%, respectively. Showing that the iner-subject variation in EEG data can be effectively reduced. Hence, the proposed adversarial learning promisingly ameliorate the state-of-art results of emotion classification.

並列關鍵字

deep learning ； transfer learning ； emotion recognition ； electroencephalography

參考文獻

[1] Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, and Mario Marchand. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4446, 2014.

[2] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.

[3] Ruo-Nan Duan, Jia-Yi Zhu, and Bao-Liang Lu. Differential entropy feature for eeg-based emotion classification. In 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), pages 81–84. IEEE, 2013.

[4] Paul Ekman and Dacher Keltner. Universal facial expressions of emotion. Segerstrale U, P. Molnar P, eds. Nonverbal communication: Where nature meets culture, pages 27–46, 1997.

[5] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495, 2014.