人形機器人的動作模仿系統及其應用

Imitation System of Humanoid Robots and Its Applications

詹澤鋒(Ze-Feng Zhan)

指導教授 ： 黃漢邦

國立臺灣大學/工學院/機械工程學研究所/碩士(2022年)

https://doi.org/10.6342/NTU202202870

摘要

針對人形機器人的複雜全身動作規劃，本論文提出一個以模仿影片中人類動作來規劃近似人類動作的機器人動作的系統，稱作模仿系統。並且為我們的人形機器人搭建一套互動系統，使其有能力進行基本的人機互動。
在模仿系統中，首先要抓取模仿的人類目標的動作，我們利用3D人體關鍵點預測模型，取得影片中人體的關鍵點的3D座標。接下來透過本論文提出的映射方法，其中參考了機器人的控制方法和穩定性，將關鍵點3D座標轉換並得到機器人執行該動作所需的各種軌跡檔案。此外，我們提出幾種後處理方法去處理生成的軌跡。
而互動系統中，我們使用透過模仿系統得到的動作資料庫來規劃動作，並且建立了視覺和語音的系統，使機器人能辨識人類手勢或是姿勢、可以跟人進行對話，並接收語音資訊進行動作。另外也搭載了學長建立的音樂節奏辨識系統，能讓機器人跟上音樂節拍跳舞。這個系統使我們的人形機器人能和人互動。
最後我們透過上述系統，完成幾項簡單的人機互動場景，證明模仿系統動作規劃的方便快速及有效，以及我們的互動系統的完備。

關鍵字

人形機器人 ； 模仿 ； 3D關鍵點預測 ； 人機互動 ； MediaPipe ； 動作規劃 ； 動作資料庫

並列摘要

In this thesis, we propose an imitation system that imitates human motions in videos to plan robot actions that are similar to human motions, with the aim of the complicated whole-body action planning of humanoid robots. Additionally, we created an interaction system that will enable basic human-robot interaction for our humanoid robot.
To obtain the 3D coordinates of the key points on the human body, we used the 3D pose estimation model. The key points were then transformed to various trajectory files needed by the robot to complete the motion, using the mapping method proposed in this research, which refers to the control strategy and stability of the robot. In addition, we proposed some post-processing method to post-process the trajectories.
In the interaction system, we created a speech and vision system so that the robot could detect human gestures or postures and converse with people. It also has a music rhythm recognition system developed by seniors that enables the robot to dance to the beats of the song. Finally, through this system, we completed several human-robot interaction scenarios, which proved the convenience, and effectiveness of motion planning with an imitation system, and the completeness of the interaction system.

並列關鍵字

humanoid robot ； imitation system ； 3D pose estimation ； human-robot interaction ； MediaPipe ； motion planning ； motion database

參考文獻

[1] Yang Chengfu and Zheng Manjin's Masterpiece [Yang Style Zhengzi Tai-Chi 37 Styles]. Youtube. 2022, from https://www.youtube.com/watch?v=kBI9qOFliTU ab_channel=LeeChanTaichiTV

[2] M. Alibeigi, S. Rabiee, and M. N. Ahmadabadi, "Inverse Kinematics Based Human Mimicking System Using Skeletal Tracking Technology," Journal of Intelligent Robotic Systems, Vol. 85, No. 1, pp. 27-45, 2017.

[3] M. Andtfolk, L. Nyholm, H. Eide, and L. Fagerström, "Humanoid Robots in the Care of Older Persons: A Scoping Review," Assistive Technology, Vol. 1, No. 1, pp. 1-9, 2021.

[4] L. Biagiotti and C. Melchiorri, Trajectory Planning for Automatic Machines and Robots, 1st edition, New York: Springer Science Business Media, 2008.

[5] C. Breazeal, C. D. Kidd, A. L. Thomaz, G. Hoffman, and M. Berlin, "Effects of Nonverbal Communication on Efficiency and Robustness in Human-Robot Teamwork," Proc. of 2005 IEEE/RSJ international conference on intelligent robots and systems, Edmonton, AB, Canada, pp. 708-713, 2005.