基於人聲旋律的流行音樂即時伴奏生成系統

Real-time pop music accompaniment generation accordingto vocal melody by deep learning models

林子軒(Tzu-Hsuan Lin)

指導教授 ： 蘇豐文

國立清華大學/電機資訊學院/資訊系統與應用研究所/碩士(2021年)

摘要

近年來，於現在的的音訊處理技術中，我們已經看見在許多技術，如伴奏生成、人聲採譜上都已經取得了一定的成果。然而至今卻還未有將兩者結合的成果出現過，因此，我們融合了目前這些出色的成果，並改進其效率，進而提出了新的「即時伴奏系統」。在這個系統中，我們優化了過去人聲採譜模型的運算效率，以及提出精簡過的HMM-base伴奏生成模型，以在有限的時間內實時生成伴奏。我們認為這個成果將會幫助許多單人歌手獨力創造更完整的表演。

關鍵字

深度學習 ； 即時 ； 伴奏生成 ； 流行音樂

並列摘要

Abstract
The goal of this work is to propose a real-time accompaniment system to assist singers in complete a simple demo by themselves.
By current audio signal technology, we have seen some achievements on accompani-ment generation and some on vocal transcription. Basing on these great works, we propose a novel “Real-time accompany generation system” to combine current state-of-arts and further improve the efficiency to reach real-time human interactive mode. To reach a ac-ceptable computing efficiency, we do a lot pruning on original model and apply DenseNet concept to enhance its gradient propagate.
In this system, we integrate efficiency improved vocal transcription model and sim-plified HMM-base accompaniment generation model which can better fit small training set situation to output musical accompaniment in limited time. We believe that this work will benefit many solo singers to deliver their live shows or demos by themselves whenever they need.
As a result, we reach real-time under 180 BPM which covers most of pop music and propose a highly improved vocal transcription model with 1/1000 parameters and 1/50 FLOPs.

並列關鍵字

DeepLearning ； Real-Time ； Accompaniment ； Pop-music

參考文獻

[4] Andrew McLeod, Rodrigo Schramm, Mark Steedman, and Emmanouil Benetos. Au-tomatic transcription of polyphonic vocal music. Applied Sciences, 7(12), 2017. ISSN 2076-3417. doi: 10.3390/app7121285. URL https://www.mdpi.com/ 2076-3417/7/12/1285.

[7] Ian Simon, Dan Morris, and Sumit Basu. Mysong: Automatic accompaniment gener-ation for vocal melodies. In Proceedings of the SIGCHI Conference on Human Fac-tors in Computing Systems, CHI ’08, page 725–734, New York, NY, USA, 2008. As-sociation for Computing Machinery. ISBN 9781605580111. doi: 10.1145/1357054. 1357169. URL https://doi.org/10.1145/1357054.1357169.

[8] Li Luo, Peng-Fei Lu, and Zeng-Fu Wang. A real-time accompaniment system based on sung voice recognition. In 2008 19th International Conference on Pattern Recog-nition, pages 1–4, 2008. doi: 10.1109/ICPR.2008.4761071.

[9] J. Salamon, E. Gomez, D. P. W. Ellis, and G. Richard. Melody extraction from poly-phonic music signals: Approaches, applications, and challenges. IEEE Signal Pro-cessing Magazine, 31(2):118–134, 2014. doi: 10.1109/MSP.2013.2271648.

[12] V. Rao and P. Rao. Vocal melody extraction in the presence of pitched accompa-niment in polyphonic music. IEEE Transactions on Audio, Speech, and Language Processing, 18(8):2145–2154, 2010. doi: 10.1109/TASL.2010.2042124.