以情緒表達改善增強式學習之研究

Inductive Policy Improvement for Reinforcement Learning by Emotional Expressions

陳煜穎(Yu-Ying Chen)

指導教授 ： 黃國勝 ； 陳昱仁

國立中正大學/工學院/電機工程學系/碩士(2014年)

摘要

在現今的機器學習演算法中，機器人能夠透過與環境的互動或人的教導來學習，然而這樣的方式需要專業人員的設計，本文提出一個架構，讓非專業的人能夠透過給予表情的方式來鼓勵或者責備機器人的行為，訓練機器人達到一個學習的成果。
在表情辨識部分利用視訊攝影機並結合模糊邏輯演算法，將人臉表情轉化為正向及負向兩種情形。然而因為人的表情並不是穩定的而且會變動，在增強式學習法中，如果直接將人的表情作為一個獎懲回饋將會導致機器人無所適從，因此我們利用情緒值E-value的方式，讓機器人擁有一個情感狀態，並使用信息理論中的相對熵改善增強式學習演算法，讓人能夠透過表情的回饋方式引導機器人學習。

關鍵字

相對熵 ； 情緒 ； 增強式學習 ； 模糊理論 ； 表情辨識

並列摘要

Machine learning has been applied in many domains, robot has ability to learn from environment’s interaction or human teaching. However, the learning algorithm may need experts to supply accurate knowledge of environment or design a feasible agent. In this thesis, we design a framework for non-expert humans to train robot’s behavior by encourage or discourage it through giving good or bad emotional facial expressions.
In our system, we use camera to capture human facial expressions. Then, we recognize the facial expressions and separate them into two types (good or bad emotion) through interval fuzzy type-2 logic algorithm. However, human emotion is variable or unstable, it cannot directly to use in the reinforcement learning. We utilize E-value and relative entropy of information theory to improve reinforcement learning to induct robot by giving robot human feedback.

並列關鍵字

relative entropy ； emotion ； reinforcement learning ； fuzzy theory ； facial expression recognition

參考文獻

[1]. R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998.

[2]. A. Ayesh, “Emotionally Motivated Reinforcement Learning Based Controller,” 2004 IEEE International Conference on Systems, Man, and Cybernetics, Vol. 1, pp. 874-878, 2004.

[3]. J. Broekens, “Emotion and Reinforcement: Affective Facial Expressions Facilitate Robot Learning,” Artifical Intelligence for Human Computing Lecture Notes in Computer Science, Vol. 4451, pp. 113-132, 2007.

[5]. A. L. Thomaz, Guy Hoffman, Cynthia Breazeal, “Reinforcement Learning with Human Teachers: Understanding How People Want to Teach Robots,” The 15th IEEE International Symposium on Robot and Human Interactive Communication, Sept. 2006.

[9]. M. Sridharan, “Augmented Reinforcement Learning for Interaction with Non-expert Humans in Agent Domains,” 2011 10th International Conference on Machine Learning and Applications and Workshops (ICMLA), Vol. 1, December 2011.