即時手部追蹤之虛擬樂器演奏系統

Designing Virtual Instruments by Real-Time Finger Tracking System

許木炘(Mu-hsen Hsu)

指導教授 ： 施國琛

國立中央大學/資訊電機學院/資訊工程學系/碩士(2014年)

摘要

人機互動漸漸成為電腦科學及資訊工程相關研究中的一個重要領域，並且讓人們可以使用各種方式與電腦或設備進行溝通。同時，許多研究者也希望從中發展出更多新的研究議題。Kinect是由微軟公司發行的一個3D感測裝置，使用者可以透過Kinect得到的環境深度資料應用在各種開發和研究。近年來，許多的開發者函式庫陸續的誕生，如微軟官方的 Kinect SDK或PrimeSense公司發布的OpenNI, 這些函式庫提供了包括人體骨架追蹤和深度資料分析的相關技術。然而，這些技術只僅限於人體大節點的追蹤如:手、頭、膝蓋..等。 如果需要更細部的偵測如手指，這些開放式函式庫就無法達成，然而 Leap Motion和Senz3d感測器可以達成手指的追蹤，並且提供了相關的函式庫，讓使用者開發自己的應用程式。雖然這些新一代的感測器有相當程度的精確度，並且提供了手指追蹤的函式，但是它有操作距離的限制，使用者必須在固定的位置操作，另外，它所提供的手指追蹤函式庫如果要應用在虛擬樂器的演奏上效率是比較不足的。在這篇論文中，作者設計了一個基於Kinect感測器的手指追蹤演算法，該系統可即時的分析深度資料並且追蹤到使用者的手指。系統可以應用在各個領域或軟體，我們將該系統設計為虛擬樂器的控制器，讓使用者可以透過自身的手勢彈奏樂器。

關鍵字

Kinect ； Leap Motion ； 手指追蹤 ； 虛擬樂器 ； OpenNI ； Kinect SDK

並列摘要

Human Computer Interaction is becoming a major component in computer science related fields allowing humans to communicate with machines in very simple ways exploring new dimensions of research. Kinect, the 3D sensing device introduced by Microsoft which allows developers to use depth data for research. Recently, there are several mature libraries such as OpenNI and Kinect SDK which provide functions of Human skeleton tracking. Therefore, users may find that these applications only work with big joints of human body. When it comes to the tracking of fingers, devices such as Leap Motion or Senz3D will be mentioned because it provide APIs (Application Programming Interface) and games, which users can communicate with the computer by their fingers. Although these new devices provides functions of finger tracking, the operating distance is short and users need to sit in a fixed position. On the other hand, the efficiency of current tracking method is hard to be used for Virtual Instrument. Here, in this paper, authors’ experimental efforts on detailed hand tracking including fingers based on Microsoft Kinect. Analyzing the depth information captured by Kinect in real time, users can apply the result in different scopes such as device controller and Virtual Instruments.

並列關鍵字

Kinect ； Leap Motion ； Finger Tracking ； Virtual Instrument ； OpenNI ； Kinect SDK

參考文獻

[9] Sertan Sentürk, Sang Won Lee, Avinash Sastry, Anosh Daruwalla, and Gil Weinberg. "Crossole: A Gestural Interface for Composition, Improvisation and Performance using Kinect." The international Conference on New Interfaces for Musical Expression, 2012

[12] Timothy K. Shih, “Spider King: Virtual Musical Instruments based on Microsoft Kinect”, The 6th IEEE International Conference on Ubi-Media Computing, 2013, Aizu-Wakamatsu, Japan, November 2-4.

[13] Charles V. Stewart, ”Robust Parameter Estimation in Computer Vision”, Society for Industrial and Applied Mathematics, 1999, Vol. 41, No. 3, pp. 513–537.

[1] J.L. Raheja, A. Chaudhary, and K. Singal, “Tracking of Fingertips and Centers of Palm Using KINECT”, Computational Intelligence, Modelling and Simulation (CIMSiM), 2011, pp. 248-252.

[2] Zhou Ren, Jingjing Meng, Junsong Yuan, and Zhengyou Zhang, “Robust hand gesture recognition with Kinect sensor”, MM '11 Proceedings of the 19th ACM international conference on Multimedia, 2011, pp. 759-760.