使用SURF特徵與類神經網路的手勢辨識

The Hand Recognition Using SURF and Neural Network

尤郁傑(Yu-Yu Chieh)

指導教授 ： 涂世雄

中原大學/電機資訊學院/電機工程研究所/碩士(2015年)

https://doi.org/10.6840/cycu201500594

摘要

本篇論文中，我們提出了一個手勢辨識方法，可以藉由Kinect達到廣範圍即時辨識。我們使用加速穩健特徵(SURF)取得手部影像的特徵向量，然後再使用類神經網路辨識手部特徵。這可以有效地達到即時辨識。
辨識系統主要分成三個步驟。第一個步驟是擷取影像和影像處理。我們使用Kinect原有的骨架追蹤技術追蹤手掌位置，然後擷取手部的深度影像並將其轉為二元影像。我們會將二元影像與結合彩色及二元影像的合併影像做比較。第二個步驟是萃取特徵，我們使用修改過的SURF萃取手部影像的特徵向量。修改過的SURF簡化了特徵點偵測，直接抓取固定五個位置作為特徵點。最後一個步驟是手部辨識，我們藉由類神經網路辨識手部特徵。為了能在一到三公尺的範圍達到最佳的辨識效果，我們需要足夠的訓練資料改進類神經網路的權重值。

本篇論文中，我們研究的貢獻如下：
(1) 我們的辨識系統對於不同的照明有較強的穩健性。
(2) 我們的辨識系統具有較廣的辨識範圍。

關鍵字

手勢辨識 ； 加速穩健特徵 ； 類神經

並列摘要

In this thesis, we propose a method to recognize the hand gesture on a wide range of distance using Kinect and control the mouse cursor on computer monitor at real-time. We use Speeded-up Robust Feature to extract feature vectors from hand image. Then we recognize these features by neural network. It can be effective in hand recognition at real-time.
The system consists of three steps. The first step is hand image capturing and processing. We use the original skeleton tracking technology from Kinect to capture the hand’s depth image and we make the depth image be a binary image. We also compare the binary image with the image that consists of color image and binary image. The second step is feature extracting. We use the modified Speeded-up Robust Feature (SURF) to extract image’s feature vectors. The modified SURF simplifies the interest region detection that selected five fix regions. The final step is hand recognition. We recognize hand features by neural network. In order to get good efficiency at hand recognition between 1 and 3 meters, we need to use sufficient training data to improve the weights in the neural network.

The contributions of our research are as follows:
(1) Our hand recognition system has the robustness in different illuminations.
(2) Our hand recognition system provides the wide range of recognition.

並列關鍵字

hand recognition ； SURF ； neural network ； Kinect

參考文獻

[8] Pugeault, N.; Bowden, R.; “Spelling it out: Real-time ASL fingerspelling recognition”, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1114 – 1119, 6-13 Nov. 2011. DOI:10.1109/ICCVW.2011.6130290

[9] Helen Cooper, Brian Holt, Richard Bowden, "Sign Language Recognition", Chapter in Visual Analysis of Humans: Looking at People, Springer, pp. 539 - 562, 2011. DOI: 10.1007/978-0-85729-997-0_27

[10] E. Ohn-Bar and M.M. Trivedi, “Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations”, IEEE Transactions on Intelligent Transportation Systems, Volume: 15, Issue: 6, pp. 2368 – 2377, Dec. 2014. DOI: 10.1109/TITS.2014.2337331

[11] E. Ohn-Bar, S. Martin, A. Tawari, M. Trivedi, “Head, Eye, and Hand Patterns for Driver Activity Recognition”, 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 660 – 665, 24-28 Aug. 2014. DOI: 10.1109/ICPR.2014.124

[12] E. Ohn-Bar and M.M. Trivedi, “The Power Is in Your Hands: 3D Analysis of Hand Gestures in Naturalistic Video”, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 912 – 917, 23-28 June 2013.