Design and Implementation of a Gesture Recognition Library for Touch-Based Devices

胡晉豪(Chin-Hao Hu)

指導教授 ： 鄭永斌

國立臺灣師範大學/理學院/資訊工程研究所/碩士(2011年)

摘要

觸控裝置（touch-based devices）已經深入日常生活中的每一個角落，越來越多人使用具備觸控裝置的手機與平板電腦。但是在開發使用觸控的手勢做為操作輸入介面的軟體目前使用的方法仍然是針對每一個SDK（software development kit）來開發，例如Apple iOS SDK是使用Cocoa Touch framework來處理多點觸控（multi-touch）的事件。
觸控技術最常見的一個應用是手勢的辨識，但是這些開發工具與裝置中還沒有一個完整的手勢辨識的函式庫提供給開發人員使用。程式設計師必須要使用event handlers或delegate的方式擷取使用者的每支手指在平面上的座標位置。以軟體工程的角度來看，即使只拿來辨識一支手指的手勢，也必需搭配額外的機制或加入辨識的演算法等方式來實作。
本論文提出一個手勢辨識函式庫在觸控裝置的實作方法。我們以client與server的架構來建置本研究的實驗環境。Client端為多點觸控裝置，重點在於能夠傳送手在觸控板上的位置或所辨識出的手勢給server端。Server端在收到訊息後會針對與之連結的應用程式做出相對應的動作。手勢辨識的部分重點在於實作$1與$N gesture recognizer演算法。前端擷取特徵的部分搭配SVM（Support Vector Machine）進行學習與分類以辨識出是何種手勢，除此之外也能夠讓使用者自行定義手勢並透過反覆的訓練（training）相同的手勢達到準確度提升的效果。

關鍵字

觸控裝置 ； 手勢 ； 辨識

並列摘要

Touch-based devices have been used in our daily life. Many people nowadays use mobile phones and tablets with multi-touch panels. However, when programmers develop software with multi-touch feature, the software inevitably depends on the device SDK such as iOS SDK on Apple product. iOS SDK provides a framework called Cocoa Touch to handle multi-touch events.
In multi-touch applications, gesture recognition can enrich computer human interaction experience, but the development tools of multi-touch devices have not provided a multi-touch gesture recognition library for developers. Developers must use event handlers and delegation method to listen the touch events and try to recognize whether these events form a gesture of interest. The implementation of gesture recognition, however, requires additional overhead efforts and cost, which can slow down the development of these applications.
This thesis presents a design and implementation of gesture recognition library via multi-touched device. We adopt gesture recognition algorithms for uni-stroke and multi-strokes touch events and then apply SVM(Support vector machine) as back-end to classify the input, Our system also allow user to define their own gestures and train the gestures to extend the gestures of interest. Finally, this work is validated by applying the control to xDIVA (eXtreme Debugging Information Visualization Assistant).

參考文獻

[1] Gorg et al. A Framework for Abstract Representation and Recognition of Gestures in Multi-touch Applications. Advances in Computer-Human Interactions, 2010. ACHI '10. Third International Conference on (2010) p. 143 – 147

[3] Rivière et al. 3D multitouch: when tactile tables meet immersive visualization technologies. SIGGRAPH '10: SIGGRAPH 2010 Talks (2010)

[7] Hoste. Software engineering abstractions for the multi-touch revolution. ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (2010) vol. 2.

[8] Katzakis and Hori. Mobile devices as multi-DOF controllers. 3D User Interfaces (3DUI), 2010 IEEE Symposium on (2010) pp. 139 – 140

[9] Sigchi. Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. (2007) pp. 1-10