在複雜環境中偵測參與度並建立人機互動

Implementation of Pre-Engagement Detection on Human-Robot Interaction in Complex Environments

呂昕儒(Sin-Ru Lu)

指導教授 ： 黃漢邦

國立臺灣大學/工學院/機械工程學研究所/碩士(2020年)

https://doi.org/10.6342/NTU202002769

摘要

人機互動是一門研究人類與機器人互動的領域，其必須被引入以規範人類與機器人之間的互動行為。因此本論文探討機器人與人類的互動行為，並提出人機互動四層架構和人機模型，規範機器人在人類社會中做出符合人類行為的決策。為了使機器擁有與人類相似的決策行為，此篇文章整合了實驗室已經發展的深度學習模組──SRWGAN、情緒辨識模型和眼神追蹤模型，與現有的深度學習模組──FSA-Net、SlowFast和Google API，並且使用自行改良的隱藏式馬可夫模型進行資訊整合與預測，進而提出一個整合性的架構──機器人認知系統，能讓機器人在複雜環境中，找出互動對象，並且給予回應。
為了使機器人能夠從人類頭部姿態與眼神，推斷出適合開啟對話的時間，我們提出了參與度的舒適指標，讓機器人知道互動者處在的參與度狀態，且防止過長時間凝視對方造成的焦慮。另外，為了使機器人可以偵測使用者的回饋(語音與表情)，我們提出了自然性指標，讓機器人可以針對其數值採取適合的措施。
最後，所發展的雙臂服務型機器人──莫比，共20個自由度: 雙手臂12、雙手掌4、輪子2和頭部2，並且可以完成舉起2公斤負載的任務。所提出的接近模型、企圖模型、及人機互動模型，均在莫比機器人身上實現，實驗結果相當不錯。

關鍵字

人機互動 ； 人類行為 ； 認知系統 ； 參與度 ； 行為辨識

並列摘要

Human-robot interaction (HRI), which is the study of interactions between humans and robots, must be introduced to humans for regulating the behaviors of interaction be-tween humans and robots. Therefore, this thesis discusses the interactions between humans and robots, and proposes four layers of HRI and HRI model, which let robots do human-like decisions. In order for robots to have decision-making behaviors similar to humans, this thesis integrates deep learning model that have
been developed in our la-boratory: SRWGAN, emotion classifier, and eye tracker, and existing deep learning model: FSA-Net, SlowFast, and Google API. The improved hidden Markov model (HMM) is used to combine the above modules for overall integration and prediction. Finally, the robot cognitive system is proposed to allow the robot to identify and respond appropriately in complex environments.
In order to make the robot determine the suitable time to start conversation from the head pose and eyes, the engagement comfort index is proposed to identify the engagement state of interactors and prevent eye contact anxiety. In addition, to enable robots to detect user’s the feedback, including voice and facial expression, we propose the naturalness index so that the robots can take appropriate action against its values.
Finally, a dual-arm mobile robot constructed by NTU Robotics Laboratory, is called Mobi, with a total 20 degrees of freedom (DoFs) configured as follows: 6 on each arm, 2 on each hand, 2 on head and 2 on wheel. The arm-hand is designed to lift 2 kg payload for tasks. In experiments, the engagement model, intention model and HRI model are implemented on Mobi robot, and are further verified through engagement comfort index, and naturalness index. Mobi is given the task of navigating and guiding in a complex environment. The results are quite satisfactory.

並列關鍵字

HRI ； human behaviors ； cognitive system ； engagement ； action recognition

參考文獻

S. M. Anzalone, S. Boucenna, S. Ivaldi, and M. Chetouani, "Evaluating the Engagement with Social Robots," International Journal of Social Robotics, vol. 7, no. 4, pp. 465-478, 2015.

S. Bahrampour, N. Ramakrishnan, L. Schott, and M. Shah, "Comparative Study of Deep Learning Software Frameworks," arXiv preprint arXiv:1511.06435, 2015.

P. Baxter, J. Kennedy, A.-L. Vollmer, J. de Greeff, and T. Belpaeme, "Tracking Gaze over Time in HRI as a Proxy for Engagement and Attribution of Social Agency," Proceedings of the ACM/IEEE International Conference on Human-robot interaction, Bielefeld Germany, pp. 126-127, March 03–06, 2014.

D. Bohus and E. Horvitz, "Managing Human-Robot Engagement with Forecasts And... Um... Hesitations," Proceedings of the 16th International Conference on Multimodal Interaction, Istanbul Turkey, pp. 2-9, November, 2014.

J. H. Brockmyer, C. M. Fox, K. A. Curtiss, E. McBroom, K. M. Burkhart, and J. N. Pidruzny, "The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game-Playing," Journal of Experimental Social Psychology, vol. 45, no. 4, pp. 624-634, 2009.