運用三維空間深度資訊及手指特徵點之即時動態手勢辨識系統

A Real-time Dynamic Hand Gesture Recognition System Based on 3D Depth Sensing and Fingertip Features

陳郁元(Yu-Yuan Chen)

指導教授 ： 顏嗣鈞

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2011年)

https://doi.org/10.6342/NTU.2011.01380

摘要

在人機互動之領域中，如何取代傳統的鍵盤與滑鼠一直是一個相當熱門的研究主題。而運用手勢辨識的概念不但常在電影中看到，也在具有多點觸控能力的智慧型手機上成為流行。但是，觸控式螢幕尺寸的限制將會影響到手勢辨識的準確度及多元性。 因此，本論文之目的為利用三維空間深度資訊為主以達到即時動態手勢辨識的效應。 此外，本論文所開發之系統將在無多點觸控能力之螢幕的情況下，依舊能夠辨識出使用者所作的手勢。
本系統使用Kinect感應器以得到完整的三維深度資訊，並運用深度直方圖之機制將使用者的手在任何的環境背景下也能偵測出來。 而在使用三維K-means分群法之後，就算手有重疊，本系統也可正確的分辨出手的數量。本系統在K-curvature演算法上做為變化以執行手指特徵點的萃取，也利用有限狀態機依照特徵點的不同特性做為觸控式手勢的分類。
從實驗結果中發現，本系統能夠在任何亮度以及複雜度的背景中有效的辨識出使用者所執行的手勢並可以每秒處理30張影像以達到即時的效應。此外，使用者將不會受到觸控式螢幕限制的種種不便。

關鍵字

即時動態手勢辨識 ； 多點觸控 ； Kinect 感應器 ； 三維空間深度資訊 ； 手指特徵點萃取 ； 人機互動

並列摘要

Interactions between humans and computers have long been restricted to the traditional means of keyboard and mouse. The concept, that movements from one’s fingers or hands provide new possibilities of human-computer interactions, is inspired by the gestural interface in sci-fi movie“Minority Report” and later proved to be plausible with the prevalence of multi-touch devices such as the Apple iPhone. The objective of this thesis is to develop a real time system capable of recognizing multi-touch hand gestures with a touchless interface by taking advantage of 3D sensing capabilities of Kinect, a novel yet affordable range sensor.
The system utilizes accurate 3D data and a depth-histogram in order to perform hand localization from any arbitrary background. K-means is used in 3D to determine the number of clusters representing hands found in the environment even in the occurrences of occlusions caused by hand overlaps. A variation of k-curvature extracts the location of fingertips from the hand contours. Based on the number of fingertips detected and their movements, a finite state machine is used to classify different multi-touch hand gestures performed by the user.
An evaluation of the system shows reliable accuracy of multi-touch gesture recognitions in a cluttered background under various lighting conditions while providing efficient real-time performance at 30 fps. In addition, the system offers users freedom in performing gestures since they are no longer restricted by the small sizes of the touch screen or the monitor of the device.

並列關鍵字

real-time dynamic gesture recognition ； multi-touch ； Kinect ； 3D Depth Sensing ； fingertips extractions ； human-computer interaction

參考文獻

[2] P. Mistry and P. Maes, “SixthSense - a wearable gestural interface,” Proceedings of SIGGRAPH Asia 2009, 2009.

[4] W. Westerman, J. G. Elias, and A. Hedge, “Multi-touch: a new tactile 2-d gesture interface for human-computer interaction,” Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting, vol. 1, pp. 632-636, 2001.

[5] S. Mitra and T. Acharya, “Gesture recognition: a survey,” IEEE Transactions on Systems, Man, and Cybernetics – Part C, vol. 37, no. 3, pp. 311-324, 2007.

[6] T. Starner and A. Pentland, “Real time American sign language recognition from video using hidden markov models,” MIT Media Lab, Cambridge, Massachusetts, Tech. Rep. 375, 1995.

[7] C. C. Wang and K. C. Wang, “Hand posture recognition using adaboost with sift for human robot interaction,” Springer Lecture Notes in Control and Information Sciences, vol. 370, pp. 317-329, 2008.