聞曲起舞

Dance Generation from Audio

陳家昕(Jia-Xin Chen)

指導教授 ： 陳煥宗

國立清華大學/電機資訊學院/資訊工程學系所/碩士(2022年)

摘要

本文提出了一種新的架構用以在給定音樂片段的情況下產出舞步。以往有關編舞生成的研究中，通常會利用到ＲＮＮ或Transformer這類費時也費硬體資源的架構。我們提出一個只使用卷積層的輕量網路的新架構，想探求在這樣條件下模型可以產出的成果。我們測試在非實驗室環境的影片上，計算產出舞步在拍點相關的指摽以及我們新提出的舞步自我相似指數，驗證了此方法的有效性。

關鍵字

跨模式感知 ； 特徵空間 ； 編舞

並列摘要

This thesis proposes a new learning-based method to generate dance poses from given music clips. Prior approaches often address the choreography generation tasks using models that comprise recurrent networks or transformers and thus make the tasks hardware-demanding and time-consuming. We propose a network architecture that uses convolution layers to explore the extent of lightweight approaches. The experimental results on video in the wild provide a baseline of several beat-related indices and a new self-similarity metric on dance sequence generation and validate the effectiveness of our method.

並列關鍵字

cross-modality ； latent representation ； choreography

參考文獻

[1]K. Chen, Z. Tan, J. Lei, S. Zhang, Y. Guo, W. Zhang, and S. Hu. Choreomas-ter: choreography-oriented music-driven dance synthesis.ACM Trans. Graph.,40(4):145:1–145:13, 2021.

[2]B. O. Community.Blender - a 3D modelling and rendering package. Blender Foun-dation, Stichting Blender Foundation, Amsterdam, 2018.

[3]R. Huang, H. Hu, W. Wu, K. Sawada, M. Zhang, and D. Jiang. Dance revolution:Long-term dance generation with music via curriculum learning. InInternationalConference on Learning Representations, 2021.

[4]H.-Y. Lee, X. Yang, M.-Y. Liu, T.-C. Wang, Y.-D. Lu, M.-H. Yang, and J. Kautz.Dancing to music. InNeurIPS, 2019.

[5]M. Lee, K. Lee, and J. Park. Music similarity-based approach to generating dancemotion sequence.Multim. Tools Appl., 62(3):895–912, 2013.