Feedforward Neural Networks於連續手勢辨識之研究

Continuous Hand Gesture Recognition By Feedforward Neural Networks

朱晏呈(Chu Yen-Cheng)

指導教授 ： 黃文吉

國立臺灣師範大學/理學院/資訊工程學系/碩士(2019年)

https://doi.org/10.6345/NTNU201900308

摘要

本論文提出基於感測器的連續手勢辨識系統，使用者右手握著智慧手機做出動作，透過收集搭載在手機上的感測器－三軸陀螺儀與三軸加速度計產生的訊號，總計六維度的資料來構成手勢。面對手勢這種時間序列資料，加上為解決連續手勢中找出切割點(Spotting)的問題，本論文中提出基於Feedforward Neural Networks建立出的深度學習模型，整合現行架構中已被證實能夠更加有效利用Convolutional Neural Networks的結構－ResNet、GoogLeNet與Inception-ResNet，將這些概念與PairNet做結合。
實驗中使用透過手機蒐集的11種手勢，在測試時一次會輸入含有1~4個手勢的資料，進到事先訓練好的類神經網路模型之中，再經由後處理得到辨識結果，而這樣的演算法則能處理傳統方法上無法有效解決的Spotting問題。另外，根據提出的模型ResPairNet在連續手勢上的辨識率，比LSTM高出7%以上的結果也可推得－Feedforward Neural Networks在時間序列資料的處理上，比Recurrent Neural Networks更加強大、有效，將這些原本應用於影像領域的結構，套用到處理時間資料的問題上，能夠更進一步提升Feedforward Neural Networks得學習能力。

關鍵字

Feedforward Neural Network ； Continuous Hand Gesture Recognition ； Deep Learning ； Human-Machine Interface

並列摘要

無資料

並列關鍵字

none

參考文獻

[1] S. Mitra and T. Acharya, "Gesture Recognition: A Survey" in IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 3, pp. 311-324, May 2007.

[2] Rung-Huei Liang and Ming Ouhyoung, "A real-time continuous gesture recognition system for sign language," Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, 1998, pp. 558-567.

[3] T. Starner, J. Weaver and A. Pentland, "Real-time American sign language recognition using desk and wearable computer based video," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1371-1375, Dec. 1998

[4] A. Agarwal and M. K. Thakur, "Sign language recognition using Microsoft Kinect," 2013 Sixth International Conference on Contemporary Computing (IC3), Noida, 2013, pp. 181-185.

[5] E. Ohn-Bar and M. M. Trivedi, "Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations," in IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 6, pp. 2368-2377, Dec. 2014.