基於多項性生理訊號分析之階層雲霧架構於情感運算系統

Cascaded Cloud-edge Framework for Affective Computing System based on Multi-modal Analysis of Physiological Signals

王勝輝(Sheng-Hui Wang)

指導教授 ： 吳安宇

國立臺灣大學/電機資訊學院/電子工程學研究所/碩士(2018年)

https://doi.org/10.6342/NTU201801899

摘要

情感運算(affective computing) 為實現人機互動系統的關鍵技術，使得電腦、機器能夠理解使用者當下情緒或心境，進一步基於使用者當前心理狀態給予最適當的服務與回應。隨著智慧型物聯網的發展，有越來越多種穿戴式裝置搭配各種感測器，我們能夠有效獲得關於受測者的各種感測訊號，並且達到連續不間斷的偵測，進而分析情緒，在此趨勢下，生理訊號反映了真實生理狀態，很適合做為情緒辨識系統的輸入。未來的應用情境如智慧工廠(i-Factory)，若工作者的心理狀態能從收錄的生理訊號及時被機器所感測，系統根據情緒分析的結果而調整工作量，能夠進一步提升整個工廠的生產效率。
然而，連續的情緒偵測會累積大量資料，若是所有資料都仰賴雲端來進行資料分析，巨量的資料將使頻寬無法負荷，傳輸過程中層層的路由器也會產生嚴重的傳輸延遲，而造成整體系統效能的屏障，相對於雲端運算，在邊緣裝置上做分析運算，能有效降低資料傳輸流量及提升後端資訊分析運算的效率，但會因為資源受限而降低分類表現。因此，我們在此論文提出了一個具階層雲霧架構的情感辨識系統，讓多數資料能在邊緣端做處理，只將較難分辨的資料送往雲端做準確分析。在雲端伺服器端，基於訊號複雜度分析，萃取熵域特徵值(entropy-domain features)，提出一高準確度的多模式生理訊號處理(包含腦電、心電、膚電訊號)之情緒辨識架構；在邊緣裝置端，藉著極限學習機(extreme learning machine) 落實分析演算法於有限的能源與硬體計算資源，並由整體學習(Ensemble Learning)有效提升分析準確率。我們藉由將雲端與邊緣端兩者的優點結合，達到了高準確度兼具低能量消耗的成效。

關鍵字

情感運算 ； 生理訊號

並列摘要

Affective computing is a key function for human–computer interaction (HCI), which makes it possible for machines and computers to realize human’s emotion and mentality, and further give appropriate responses and services based-on human’s current mental status. As intelligent IoT develops, more and more wearable devices are equipped with different kinds of sensors. We can effectively get various signals sensed from subjects and have the access for continuously monitoring. For this trend, physiological signals reflecting nature responses can be good inputs for affective computing framework. A future application of this scenario is i-Factory. Once the mental status of worker can be perceived by analyzing physiological signals. Workload of each worker can be dynamically adjusted based on the prediction outcome. Consequently, the productivity efficiency of the whole factory can be enhanced.
However, continuously affect monitoring will accumulate a great amount of data. If all the data are analyzed by the cloud, they would result in lack of bandwidth. Multiple layers of routers in the transmission process would lead to large latency. All these would cause a barrier to overall system performance. By contrast, analyzing data in edge devices can significantly reduce the amount of transmitted data and increase the efficiency of computation in the cloud. But the classification accuracy is relatively low due to the resource constrained problem. As a consequence, we aim to propose a cascaded edge-cloud framework for emotion recognition. Large portion of data can be screened by edge devices and only the data that are hard to be recognized would be transmitted to cloud for accurate prediction. For cloud server, entropy-domain features are extracted to quantify the complexity of signal. A high-accuracy framework is established based on multi-modal analysis of physiological signals. For edge devices, extreme learning machine (ELM) is applied to classification in the scenario of restricted hardware and computation resources. Ensemble learning is then used to enhance prediction performance. Finally, we combine both edge and cloud framework to form a cascaded system and attain the results of both high accuracy and low energy consumption.

並列關鍵字

Affective Computing ； Physiological Signals

參考文獻

[1] F. Wallhoff, “The Facial Expressions and Emotions Database Homepage (FEEDTUM),” www.mmk.ei.tum.de /˜waf/fgnet/feedtum.html, Sept. 2005.

[2] F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier and B. Weiss, “A database of german emotional speech” Proc. 9th International Conference on Spoken Language Processing (INTERSPEECHICSLP 2005), Sept. 2005, pp. 1517-1520.

[3] J. Abdon Miranda-Correa M. Khomami Abadi N. Sebe I. Patras, “AMIGOS: A Dataset for Mood Personality and Affect Research on Individuals and Groups,” ArXiv e-prints 2017.

[4] J.A. Russell, “A Circumplex Model of Affect,” J. Personality and Social Psychology, vol. 39, no. 6, pp. 1161-1178, 1980.

[5] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari and M. Ayyash, "Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications," IEEE Communications Surveys & Tutorials, vol. 17, no. 4, pp. 2347-2376, Fourthquarter 2015.