基於階層式手部解析的手勢辨識

Hand Gesture Recognition with Hierarchical Hand Parsing

林孟萱(Lin, Meng Hsuan)

指導教授 ： 賴尚宏

國立清華大學/電機資訊學院/資訊系統與應用研究所/碩士(2016年)

摘要

在這篇論文中，我們提出了一套基於階層式手部解析的手勢辨識演算法來對單張深度影像進行手勢辨識。我們所提出的方法，首先從原圖中擷取出手部區域的三維資料點，並透過計算手的方向與垂直軸之間的轉換，以光軸為轉軸，將手的姿勢做旋轉歸一化。我們根據定義好的手部組態，將整個手部切分成十一個彼此不重疊的子區塊，並利用深度範圍特徵訓練出三階層的隨機森林分類器，手部像素點將參考分類器計算出的後驗機率來決定其所屬的子區塊。在第一層中將利用分類器判別像素點是否屬於手掌區塊，接著在第二層中，進一步辨識非手掌類別中的像素點所屬的手指類別，最後在第三層中，將會辨識出位於手指的像素點是屬於指尖區塊還是手指根部區塊。最後，解析完成的手部資訊將會被進一步利用來組成三種特徵，包含手部姿勢特徵、手指角度特徵、手部區塊比例特徵，並透過支持向量機器來達到手勢辨識的目的。在實驗部分，我們將提出的方法分成手部解析以及手勢辨識兩大部分，利用不同的手勢真實影像資料庫來呈現我們方法的效能。

關鍵字

手勢辨識 ； 手部解析 ； 手部骨架偵測

並列摘要

In this thesis, we proposed a hand gesture recognition algorithm based on hierarchical hand parsing from a single depth image. In the proposed system, we first normalize in-plane rotation of the hand pose. According to hand configuration, we propose to segment a hand into 11 non-overlapping parts with a novel 3-layer hierarchical Random Decision Forest (RDF) per-pixel classifier. In the first layer, the hand region is divided into two parts: palm and fingers. In the second layer, pixels are classified into different finger classes: thumb, index finger, middle finger, ring finger and pinky finger. In the third layer, a finger pixel is classified into upper and lower part. In each layer, per-pixel classification is executed to assign a set of posterior probabilities corresponding to different hand parts to each pixel based on depth-context features. To develop hand gesture recognition, the information of parsed hand is employed to compute three kinds of features including posture feature, finger angle feature and hand part ratio feature, for Support Vector Machines (SVMs). Our experiments show superior performance of hand parsing and gesture recognition by using the proposed algorithm compared to some previous methods on different real hand pose datasets.

並列關鍵字

hand gesture recognition ； hand parsing ； hand skeleton detection

參考文獻

[1] C. Oz, and M. C. Leu. Recognition of finger spelling of American Sign Language with artificial neural network using position/orientation sensors and data glove, International Symposium on Neural Networks, pp. 157-164, 2005.

[2] C. Oz, and M. C. Leu. Linguistic properties based on American sign language recognition with artificial neural networks using a sensory glove and motion tracker. Computational Intelligence and Bioinspired System, Neurocomputing, vol. 70, no. 16–18, pp. 2891-2901, 2007.

[3] P. Kumar, Verma J, Prasad S. Hand data glove: a wearable real-time device for human–computer interaction, International Journal of Advanced Science and Technology, vol. 43, pp. 15-25, 2012.

[5] M. de La Gorce, Martin, D. J. Fleet, and N. Paragios. Model-based 3d hand pose estimation from monocular video, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9):1793-1805, 2011.

[7] V. I. Pavlovic, R. Sharma, and T. S. Huang. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19:677–695, 1997.