以視覺為基礎之即時指揮手勢追蹤系統

A Vision-based Real-time Conductor Gesture Tracking System

莊謹萍(Chin-ping Chuang)

指導教授 ： 李忠謀

國立臺灣師範大學/理學院/資訊工程研究所/碩士(2007年)

摘要

隨著網路視訊的普及，網路攝影機品質日趨優良、價格也相對低廉，本研究旨在提出一個「指揮者手勢追蹤系統」，代替鍵盤與滑鼠作為輸入單元，讓使用者能透過視訊攝影機(Webcam)及個人電腦、運用基本的指揮動作，能夠即時追蹤使用者手勢的軌跡與方向變化、偵測音樂節拍所在的時間點。
本研究可分為兩個主要階段：第一階段為目標物追蹤，採用CAMSHIFT演算法來實現物件追蹤。CAMSHIFT演算法為平均位移演算法的改良，此演算法利用使用者所感興趣的顏色機率分佈特性，經由平均位移迭代的方式，找出其機率分佈圖的峰值，此峰值即為可能性最高之影像區塊並得到物體移動路徑。第二階段則利用兩種方法計算：K-曲率法則以及垂直分量低點偵測。K-曲率法則利用物體移動路徑各點之區率並計算找出其方向轉變；而垂直分量低點偵測則是找出物體移動的垂直低點，將此低點定義為音樂的節拍點。
本研究所開發之系統可以讓使用者自行選定偵測目標（如指揮棒）並準確偵測移動的軌跡，將使用者的指揮動作上方向的改變，轉變成音樂檔的節拍事件，其準確率平均可達86.46%以上。

關鍵字

手勢追蹤

並列摘要

In recent years, interaction between humans and computers is becoming more important. “Virtual Orchestra” is an Human Computer Interface (HCI) software which attempts to authentically reproduce a live orchestra using synthesized and sampled instruments sounds. Compared with the traditional HCIs, using vision-based gesture can provide a touch-free interface which is less bounding than mechanical instruments. In this research, we design a vision-based system that can track the hand motions of a conductor from webcam and extract musical beats from motions.
The algorithm used is based on a robust nonparametric technique for climbing density gradients to find the mode of probability distributions. For each frame, the mean shift algorithm converges to the mode of the distribution. Then, the CAMSHIFT algorithm is used to track the moving objects in a video scene. After acquiring the target center point continuously, we can form the trajectory of moving target (such as baton, conductor’s hand…etc). By computing an approximation of k-curvature for the trajectory, and the angle between these two motion vectors, we can compute the point of the change of direction.
In this thesis, a system was developed for interpreting a conductor’s gestures and translating theses gestures into musical beats that can be explained as the major part of the music. This system does not require the use of active sensing, special baton, or other constraints on the physical motion of the conductor.

並列關鍵字

Gesture Tracking ； CAMSHIFT

參考文獻

[2] B. A. Myers, "A Brief History of Human-Computer Interaction Technology", Interactions, vol. 5, pp. 44-54, 1998.

[5] D. Murphy, “Tracking a Conductor's Baton” , S?ren I. Olsen, Editor, Proceedings of the 12th Danish Conference on Pattern Recognition and Image Analysis, volume 2003/05 of DIKU technical report series, pp. 59-66, Copenhagen, Denmark, August 2003.

[6] R. Behringer, "Conducting Digitally Stored Music by Computer Vision Tracking", AXMEDIS '05: Proceedings of the First International Conference on Automated Production of Cross Media Content for Multi-Channel Distribution, pp. 271, 2005.

[7] The Church of Jesus Christ of Latter Day Saints. Conducting Course.

[11] R. Boulanger and M. Mathews, “The 1997 Mathews Radio-baton and Improvisation Modes”, Proceedings of the 1997 International Computer Music Conference, pp.395-398, Thessaloniki, Greece, 1997.