基於音樂流派分類的實時人形機器人舞蹈系統

Real-time Humanoid Robot Dance System Based on Music Genre Classification

盧思陽(Swan Lu)

指導教授 ： 黄漢邦

國立臺灣大學/工學院/機械工程學研究所/碩士(2021年)

https://doi.org/10.6342/NTU202103102

摘要

現如今，人們對人工智能的追求不僅限於使用便利或智能的範疇。機器人的娛樂使用方面也逐漸被納入考量以及使用，而人形機器人的舞蹈即是人們對機器人模仿人類以及機器人的娛樂性的一個追求。
本文採用了實時節拍追蹤器以及一種獨特的人形機器人舞步設計器，設計了一個實時人形機器人舞蹈系統。我們提出了Via Ratio，配合實時節拍追蹤器以及音樂速度估計器，使機器人能夠實時地生成符合當下音樂速度的舞蹈。並在其中使用了浮體運動學以及實心動量矩陣，使人形機器人的運動更穩定和平滑。
而在對於音樂，我們也使用了神經網絡進行分類。在頻域上，我們提取了多種音樂的特征，使用兩個lstm網絡對音樂進行分類。動作的設計上，也會根據不同的風格設計出對應的舞蹈動作。以達到機器人舞蹈不死板、有觀賞性以及藝術的目標。

關鍵字

人形機器人 ； 實時節拍追蹤 ； 音樂分類 ； 神經網絡 ； 動量控制 ； 浮體運動學 ； 質心動量矩陣

並列摘要

Nowadays, people's pursuit of artificial intelligence is not limited to the scope of convenience or intelligence. The entertainment use of robots is gradually being consid-ered and used, and the dance of humanoid robots is one of people's pursuit of robots im-itating humans and the entertainment of robots.
This paper uses a real-time beat tracker and a unique humanoid robot dance step designer to design a real-time humanoid robot dance system. We proposed Via Ratio, with a real-time beat tracker and music speed estimator, so that the robot can generate dances that match the current music speed in real time. The floating body kinematics and solid momentum matrix are used in it to make the movement of the humanoid robot more stable and smooth.
For music, we also use neural networks for classification. In the frequency domain, we extracted the characteristics of a variety of music and used two lstm networks to clas-sify the music. In the design of movements, corresponding dance movements will also be designed according to different styles. In order to achieve the goal of robot dance not rigid, ornamental and artistic.

並列關鍵字

Humanoid robot ； Real-time beat tracking ； Music genre classification ； Neural network ； Momentum control ； Floating based kinematics ； Centroidal momentum matrix

參考文獻

[1] B. Abror and D. Pramadihanto, "Dance Motion Pattern Planning for K. Mei as Dancing Humanoid Robot," 2017 International Conference on Robotics, Biomimetics, and Intelligent Computational Systems (Robionetics), Bail, Indonesia, pp. 6-11, 2017.

[2] A. Augello, E. Cipolla, I. Infantino, A. Manfre, G. Pilato, and F. Vella, "Creative Robot Dance with Variational Encoder," arXiv preprint arXiv:1707.01489, 2017.

[3] A. Augello, I. Infantino, A. Manfrè, G. Pilato, F. Vella, and A. Chella, "Creation and Cognition for Humanoid Live Dancing," Robotics and Autonomous Systems, vol. 86, pp. 128-137, 2016.

[4] T. Bi, P. Fankhauser, D. Bellicoso, and M. Hutter, "Real-Time Dance Generation to Music for a Legged Robot," 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, pp. 1038-1044, 2018.

[5] S. Böck, F. Korzeniowski, J. Schlüter, F. Krebs, and G. Widmer, "Madmom: A New Python Audio and Music Signal Processing Library," Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, The Netherlands, [Online]. Available: https://doi.org/10.1145/2964284.2973795, 2016.