結合多感測器的手勢辨識於人機互動及智慧型服務機器人之應用

Multi-Sensor Based Hand Gesture Recognition for Human-Robot Interaction and the application of Intelligent Service Robotics

吳彥彰(Yen-Chang Wu)

指導教授 ： 羅仁權

國立臺灣大學/電機資訊學院/電機工程學研究所/碩士(2012年)

https://doi.org/10.6342/NTU.2012.02976

摘要

隨著科技的進步，機器人在生活中扮演著極為重要的角色。在現今社會中，人們有更多的機會接觸到機器人的功能，像是生活中常見的智慧型服務機器人以及救援機器人。因此人機互動在當今成為一個重要的研究課題。在這個主題下，我們研究出人機互動最直接的方式。這篇論文主要可分為兩個部分：第一部分,我們介紹一個結合式的手勢辨識方法。此方法是利用電腦視覺以及數位影像處理的方式來達成。第二部分，我們介紹了各式各樣人與機器人的互動方式。
在第一部分中，我們著重於介紹手勢辨識方法，手勢辨識對人機互動來說，是非常重要且快速的;符號語言對於有身心障礙的患者來說，是最常見也是最直接的工具。身心障礙者可以藉由手勢向看護或機器人進行溝通與傳達。因此，我們介紹一個結合式的手勢辨識方式，這方式運用兩個不同的辨識器來完成手勢辨識演算法；換言之，辨識器的目的就是要展現良好的識別力。為了達到良好辨識力，我們所結合的兩個辨識器能互補相互得優缺點。其中一個辨識器藉由骨架分析(HSR)來辨識手勢，另一個則是為了要藉由SVM 機器學習演算法，來辨識手勢。此外要我們利用多樣化的特徵來表示影像，並交給機器學習演算法，進行手勢辨識。
最後這兩個辨識器經由CAR 公式 的整合來決定手勢。在CAR 公式中包含了一系列的規則;例如轉換辨識器和利用結合辨識器的方式，此外我們在訓練過程中 所利用的手勢影像有兩種，一部分是利用 Bosphorus的手勢資料庫，其他一部分是我們自己所蒐集的手勢影像。總的來說，我們經過完整的概念和成功的實驗證明我們手勢辨識的方法，可以實用於自行開發的智慧型機器人上並且實現人機互動。
第二部分，我們介紹了一些人機器互動的方式以及展現智慧型服務機器人的應用。例如: 經由第一部分的手勢辨識與服務型機器人的互動，深度感測器與機器人互動及應用以及多感測器結合的身分辨識。結合不同的感測器，實現出創新的人機互動方法。

關鍵字

智慧型服務機器人 ； 電腦視覺 ； 數位影像處理 ； 機器學習 ； 人機器互動 ； CAR 方程式

並列摘要

With advances in technology, robots play an important role in our lives. Nowadays, we have more chance to see robots service in our society such as intelligent robot for rescue and for service. Therefore, Human-Robot interaction becomes an essential issue for research. In this thesis, we propose a direct method for Human-Robot interaction on service robot. Thesis can be broadly divided into two parts; in the first part, we introduce a combining method for hand gesture recognition which is using computer vision and image processing to achieve the goal and second the part, we show several methods for Human-Robot interaction.
In the first part, we focus on our hand gesture recognition method. Hand sign recognition is an essential and fast way for Human-Robot Interaction (HRI). Sign language is the most intuitive and direct way to communication for impaired or disabled people. Through the hand or body gestures, the disabled can efficiently let caregiver or robot know what message and order they want to convey. In this thesis, we propose a combinatorial hands gesture recognition algorithm which combines two distinct recognizers. These two recognizers collectively determine the hand’s gesture via a process called Combinatorial Approach Recognizer (CAR) equation. These two recognizers are aimed to complement the ability of discrimination. To achieve this goal, one recognizer recognizes hand gesture by hand skeleton recognizer (HSR), and the other recognizer is based on Support Vector Machines (SVM). In addition, the corresponding classifiers of SVM are trained using variety features such as Gabor feature, local binary pattern and raw data. Furthermore, the trained images are using Bosphorus Hand Database [15][20][43] , in addition to the images taken by us. A set of rules including recognizer switching and combinatorial approach recognizer CAR equation is devised to synthesize the distinctive methods. We have successfully demonstrated gesture recognition experimentally with successful proof of concept. Our experiment can be use in self-developed intelligent robots and to achieve Human-Robot interaction.
In the second, we introduce several ways of Human-Robot interactions and demonstrate the application of intelligent service robots, such as hand gesture recognition which is proposed in first part implements on service robot, RGB-D sensor application and multi-sensor based person identification. We associate different kind of sensor to achieve innovative human-computer interaction methods.

並列關鍵字

Intelligent Service Robot ； Computer Vision ； Digital Image Processing ； Machine Learning ； Human-Robot Interaction (HRI) ； Combinatorial Approach Recognizer ； CAR equation

參考文獻

[1] A. Barkoky and N.M. Charkari, “Static Hand Gesture Recognition of Persian Sign Numbers using Thinning Method,” ICMT 2011

[3] C. Benadelkader and P. Griffin, “A Local Region-Based Approach to Gender Classification from Face Images,” Proc. 2005 IEEE CS Conf.Computer Vision and Pattern Recognition, p. 52, 2005.

[4] C. L. Breazeal, Sociable Machines: Expressive Social Exchange Between Humans and Robots. Sc.D. dissertation, Department of Electrical Engineering and Computer Science, MIT, May, 2000

[6] L. Bretzner, I. Laptev, and T. Lindeberg, “Hand gesture recognition using multi-scale color features, hierarchical models and particle filtering,” in Proc. 5th IEEE Int. Conf. Autom. Face Gesture Recog., 2002, pp. 405–410.

[7] L. Breiman, ”Bagging predictors,” Machine Learning, 24, 123–140, 1996.