利用即時視訊建構虛擬鍵盤樂器

On Building Virtual Piano System from Real-time Videos

蔡函辰(Han-Chen Tsai)

指導教授 ： 鄭士康

國立臺灣大學/電機資訊學院/資訊網路與多媒體研究所/碩士(2007年)

https://doi.org/10.6342/NTU.2007.01743

摘要

在科技高度發達的數位時代，科技媒體成為藝術創作的新表現媒介。資訊技術與多媒體的結合帶來了具創新性的數位內容，以及高附加價值的影音互動多媒體產業。本論文介紹並且實作一個互動影音系統；「虛擬鋼琴系統」。研究動機起源於數位虛擬樂器與音樂玩具等相關研究；應用數位影像處理和MIDI技術，結合影音互動即時系統。藉由攝影機對使用者的手部運動作取像偵測，再進行動作的分析，以即時產生相對音高之鋼琴琴聲。本研究主要以虛擬鋼琴彈奏為實作，在技術層面，探討科技和媒體技術，在應用面上可以和數位內容產業之互動式遊戲結合，未來更可延伸至可攜式數位學習載具等產業。

關鍵字

虛擬樂器 ； 互動 ； 即時

並列摘要

The information technology is highly developed nowadays. It is also used as a new tool for creating the contents. Combining information technology with multimedia can bring us innovative and value-added digital contents for multimedia industries. In this thesis, we will propose and implement an interactive multimedia system, virtual piano system, which integrates information technology with interactive multimedia contents. This research’s motivation is based on a concept originally derived from the digital musical instrument and music toys. In this thesis, we use a video camera to capture the movements of user’s fingertips. After detecting and analyzing the movement, the corresponding piano sound will be played in real-time. Digital image processing and MIDI technology are involved in this study. It has many potential capabilities for developing products for digital content industries such as interactive game and portable digital learning device.

並列關鍵字

virtual instrument ； interactive ； real-time

參考文獻

[2] M. Farbood, E. Pasztor, K. Jennings, "Hyperscore: A Graphical Sketchpad for Novice Composers." IEEE Computer Graphics and Applications, January-March 2004.

[3] G. Levin and Z. Lieberman, "In-Situ Speech Visualization in Real-Time Interactive Installation and Performance." The 3rd International Symposium on Non-Photorealistic Animation and Rendering, Annecy, France, 6/2004

[5] Z. Zhou, et al., “Multisensory Musical Entertainment System”, IEEE MultiMedia, Vol. 11, No 3, pp 88-101, 2004.

[9]K.L. Chung, Digital Image Processing and Computer Vision, Taiwan Dong Hwa Bookstore, Taipei, Taiwan, 2002. (in Chinese)

[11]Y. Wang, B. Yuan, A novel approach for human face detection from color images under complex background, Pattern Recognition, Vol. 34, No.10, pp 1983-1992, 2001.