以雙向長短期記憶網路架構混和多時間粒度文字模態改善婚 姻治療自動化行為評分系統

在心理學領域的研究中，為了觀察人類的心理狀態，專家們時常會設計一套實驗流程，如諮詢、演出或討論等，希望藉由外在行為的刺激引出內在情緒的反應。然而，在分析整段互動過程時，不同時間長度的互動片段會隱含不同強度的情緒資訊，專家們便藉由彙整這些片段的資訊以便做出較完整且適合的決策。本論文受此概念啟發，將其應用於婚姻治療資料庫自動化行為評分系統中，藉此增強機器對於心理治療之互動過程評分之正確性。其計畫挑選長期患有婚姻問題的夫妻，讓夫妻雙方針對主題進行對話，將其過程中之聲音、影像以及文字記錄下來，藉由這些資訊可分析夫妻雙方互動過程之行為表現程度進而評估治療成效。本論文使用雙向長短時記憶網路(Bidirectional Long Short Term Memory)架構應用於文字模態中取出多時間粒度下之高階特徵，並結合文本層級之文章向量(Doc2Vec)做特徵篩選，以整合不同時間層次之行為表徵，最後加入語音模態進行二元分類器之機器學習，在六種行為編碼之表現上，丈夫和妻子的平均行為準確率分別達到了 79.3%和 82.4%，相較於過去論文的 74%和 75%[1]分別提升了 5.3%以及 7.4%。最後的實驗與結果展示了使用深度雙向長短期記憶網路能夠有效學習時間序列資訊的優點，其應用於各時間粒度行為強度之計算能夠增進整體演算法在婚姻治療之行為辨識準確率。

關鍵字

人類行為訊號處理；多時間粒度；深度學習；情感計算；自然語言處理

並列摘要

In psychology field research, experts generally design a standard experimental procedure, e.g., consultation, show or talk, to observe the mental state of human. They expect to trigger reactions of internal emotion by stimulating external behavior. However, when analyzing whole interaction process, different lengths of fragments of interaction including different strength of emotional information, and experts make more complete and suitable decision. Our work inspired by the conception and apply it on automatic behavior rating system of couple therapy database, to improve the accuracy of scoring interaction process of psychotherapy. This program recruit seriously and chronically distressed married couples, and let them make a problem-solving communication for specific topic, recording the audio, video and text of process, experts analyze the extent of behavior of couples interaction process to evaluate treatment effects by these information. This paper use Bidirectional Long Short Term Memory structure to extract multi- granular and high-level features for lexical modality, also combine Doc2Vec into document level with feature selection to integrate different temporal level of behavioral features, and finally join audio modality to train binary classifier with machine learning algorithm. For the performance of six behavioral codes, husband and wife's average accuracy of behavior achieve 79.3% and 82.4% separately, this enhance 5.3% and 7.4% average accuracy compared to 74% and 75% of previous paper[1]. Our experiments and results present the merit of use of Bidirectional Long Short Term Memory can learn time series information effectively, the computation of different level granularity of intensity of behavior improving the algorithm on couple therapy rating system.

並列關鍵字

behavioral signal processing ； multi-granularity ； deep learning ； affective computing ； natural language processing

參考文獻

[1] Xia, Wei, et al. "A dynamic model for behavioral analysis of couple interactions using acoustic features." Sixteenth Annual Conference of the International Speech Communication Association. 2015.