使用人工智慧晶片實作之自動樂譜辨識與打擊樂演奏系統

Robotic Percussion System Incorporating an Automatic Sheet Music Recognition System Using Artificial Intelligence Chip

陳建豪(Chen Jian-Hao)

指導教授 ： 王偉彥

國立臺灣師範大學/科技與工程學院/電機工程學系/碩士(2022年)

https://doi.org/10.6345/NTNU202201582

摘要

近幾年的神經網路研究，針對高解析度光學影像辨識系統已達到成熟階段，然而龐大的卷積神經網路(Convolutional Neural Network, CNN)架構往往有著極大的計算成本，如何維持可接受的正確率並降低計算負擔是一個值得研究的方向。因此本論文使用專精電腦視覺任務的人工智慧晶片替換龐大的目標偵測CNN來偵測音符位置，並以自行設計之輕量CNN辨識音階資訊。將複雜的任務分配給兩個輕量CNN來實現一套光學樂譜辨識系統。本論文亦設計控制程式整合光學樂譜辨識與Delta機械手臂控制。透過鏡頭偵測與辨識拍攝到的紙本樂譜，並且以通用非同步收發傳輸器(Universal Asynchronous Receiver/Transmitter, UART)取得辨識結果。接著以辨識結果確定演奏順序後，驅動Delta機械手臂自動演奏鐵琴。最後以紙本樂譜實際測試本論文提出之光學樂譜辨識系統，驗證此系統的辨識正確率。

關鍵字

樂譜辨識 ； 深度學習 ； Delta 機械手臂 ； 人工智慧晶片

並列摘要

In recent years, neural network research has reached a mature stage for high-resolution optical image recognition systems. However, huge Convolutional Neural Network (CNN) architectures often have huge computational costs, and it is worth studying how to maintain acceptable accuracy and reduce the computational cost. Therefore, this thesis uses an artificial intelligence chip specializing in computer vision tasks to replace the huge target detection CNN for detecting music score coordinates. This thesis also proposes a lightweight CNN to recognize the music scale of detected music score. A complex task is assigned to two lightweight CNNs to implement an optical music score recognition (OMR) system. This thesis also proposes the control program to integrate OMR system and Delta robot. The OMR system detects music score from captured sheet music through the lens and transfers results with Universal Asynchronous Receiver/Transmitter (UART) to control program. The program drives Delta robot to play percussion after the playing order is determined with the recognition results. Finally, we tested the OMR system with sheet music to verify the accuracy of this system.

並列關鍵字

music score recognition ； deep learning ； delta robot ； artificial intelligence chip

參考文獻

[1] B. Dynamics. "Spot Arm - Mobile Manipulation." https://www.bostondynamics.com/products/spot/arm (accessed Aug. 30, 2022).

[2] 蔡自偉, "印刷樂譜辨識系統," 國立中山大學, 資訊工程學系碩士論文, 2004.

[3] 黃朝慶, "自動樂譜辨識與打擊樂機器人系統," 國立臺灣師範大學, 電機工程學系碩士論文, 2020.

[4] W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," (in en), Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115-133, Dec. 1943, doi: 10.1007/BF02478259.

[5] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," (in en), Nature, vol. 323, no. 6088, pp. 533-536, Oct. 1986, doi: 10.1038/323533a0.