應用於人機互動之多人表情辨識與環境氛圍辨識系統

Recognition of Multi-user Facial Expression and Its Ambient Atmosphere Identification System for Human-Robot Interaction

林佩嫺(Pei-Hsien Lin)

指導教授 ： 羅仁權

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2012年)

https://doi.org/10.6342/NTU.2012.03340

摘要

在科技進步與人力結構的變遷下，人們開始追求更高的生活品質，在社會福利、醫療照護、居家看護、教育與各種服務等等的需求也相對提升。隨著智慧型機器人產業的發展逐漸受到重視，使機器人逐漸進入人們的生活中，並提供舒適安全與健康的生活是近年來學界與業界共同追求的遠景。因此，智慧型機器人產業的發展在世界各國中也列為前瞻優先發展的科技產業之一，而表情與環境氛圍辨識也成為人機互動領域中相當重要的議題。
本論文主題在於發展一個多人臉部的表情及由此多人表情感受此時的環境氛圍之辨識系統，智慧型機器人利用視覺影像系統取得環境中的影像瞭解氛圍並與人們產生互動。利用主動外觀模型(Active Appearance Models, AAM)來追蹤使用者的臉部特徵，並利用使用者的臉部特徵資訊去判斷臉部表情，在表情辨識系統中，我們主要結合兩方法來萃取臉部特徵，分別是特徵向量與主動外觀模型的差異量的方法，利用這些特徵與各表情間的相對關係來辨識表情。此外，我們將單人的表情辨識發展成在一個環境中能夠同時辨識多人情緒的氛圍偵測系統，藉此來得知此環境中氛圍為何，使機器人能理解人，並且不單只透過滑鼠或者鍵盤所下達命令，藉由這些臉部特徵的資訊，機器人便可以輕鬆的偵測環境中大部份人們的情緒變化，並據此做出適當的回應，達到人機互動的目的。
本論文裡提出的演算法與程式設計都是於Windows平台上以C++程式語言開發，藉由相對穩定的使用者臉部表情特徵取得，且依據辨識結果，將所有軟體硬體整合應用於實現在本實驗室自行開發的智慧型機器人上。

關鍵字

臉部特徵的匹配與追蹤 ； 主動外觀追蹤模型 ； 光流法 ； 臉部表情辨識 ； 環境氛圍辨識 ； 人機互動

並列摘要

Since technology developed rapidly and societal evolvement progressively, people begin to pursue higher quality of life and the requirements in social welfare, medical care, home care, education and other services are increased. With the progress of intelligent robot industry, how to integrate robots into the daily life and how robots provide a comfortable, safe and healthy life become common visions in both academia and industry in recently years. Thus, the development of intelligent robot industry has been one of the priority prospects of industries. Among these, facial expression and ambient atmosphere recognition have become one of significant issues in Human-Robot Interaction.
Emotional interaction with human beings is desirable for robots. The objective of this thesis is to implement an integrated system which has ability to track multiple people at the same time, to recognize their facial expressions, and to identify social ambiance. Consequently, the intelligent robot with vision systems can acquire the information of the ambient atmosphere and further interacts with people properly. Optical flow and component-based active appearance model are applied for facial features alignment and tracking. In our facial expression recognition scheme, we fuse Feature Vectors based Approach (FVA) and Differential-Active Appearance Model (AAM) Features based Approach (DAFA) to obtain not only apposite positions of feature points, but also more information about texture and appearance. With the obtained useful information, FVA can classify the emotions according to comparison with the distances and ratios of feature points, and DAFA can distinguish emotions from classical machine learning on a low dimensional manifold space. Furthermore, emotion recognition of multiple people at the same time is extended. Based on the proposed algorithm, multiple person emotion analysis and social ambient atmosphere identification can be achieved, which makes the relationship between people and robots much closer.
All the systems, user interface, software and applications proposed in this thesis are implemented with C++ programming language in Windows platform. With relative stable facial features extraction, the proposed algorithm is implemented in robots developed in our laboratory at the International Center of Excellence on Intelligent Robotics and Automation Research (iCeiRA) at National Taiwan University.

並列關鍵字

Alignment and Tracking of Facial Features ； Active Appearance Model (AAM) ； Facial Expression Recognition ； Ambient Atmosphere Identification ； Human-Computer Interactions

參考文獻

[20] 張書瑞(Shu-Ruei Chang),” 機器人臉部表情結合唇型同步與視覺專注應用於人機互動” , 臺灣大學機械工程學研究所學位論文, 2011

[1] B. Fasel and J. Luttin, “Automatic Facial Expression Analysis: Survey,” Pattern Recognition, vol. 36, no. 1, pp. 259-275, 2003.

[2] T. F. Cootes, G. J. Edwards, C. J. Taylor ,“Active Appearance Models “, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681–685, Jun. 2001.

[3] Y. Tian, T. Kanade, and J. Cohn, “Recognizing action units for facial expression analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 97–115, 2001.

[4] M.S. Bartlett, G. Littlewort, I. Fasel, and R. Movellan, “Real time face detection and facial expression recognition: Development and application to human computer interaction,” in Computer Vision and Pattern Recognition Workshop, vol. 5, 2003, pp. 53.