基於深度影像與膚色之即時手勢辨識技術

A Technique of Real-time Gesture Recognition Based on Depth Image and Skin Color

賴彥成(Yen-Cheng Lai)

指導教授 ： 陳彥霖

國立臺北科技大學/電資學院/資訊工程系研究所/碩士(2013年)

摘要

目前社會上的電子產品多樣化，對於電腦或是電子機械的操控已不再局限於傳統的鍵盤、滑鼠，其中以肢體動作為輸入訊號的體感操作，也成為一種廣泛運用的輸入型態。以往對於手部輪廓辨識可概略的分為三種：深度、膚色、輪廓訓練器，但以深度影像為主的方式會面臨深度影像缺洞的問題，而純粹以膚色為辨別基礎的準確率較差；本論文提出一種即時手勢辨識方法，結合深度資訊與膚色，可以改進Kinect在中近距離(約2公尺處)的手勢偵測效能與精準度。

　　本論文研究主題為對於以Microsoft研發之影像偵測裝置Kinect所進行的手勢偵測，結合改良RGB色差法技術，使RGB色差法應用範圍更廣泛，相較於傳統之膚色辨識法可提升 5% ~ 27%的辨識速度，與純粹深度輪廓辨識法相比可提升近五倍的辨識速度；搭配手部移動軌跡組合為研究成果即時手勢辨識，執行速度方面有每秒23張frame的即時效果。研究成果可運用在需要手勢操作之體感互動技術，如互動式聯網電視、遊戲娛樂、體感電腦操作…等，提供另一種手部肢體操作方式。

關鍵字

電腦視覺 ； 輪廓偵測 ； 手勢辨識 ； 膚色 ； 紅外線偵測 ； 深度影像 ； 色差法 ； 圖形識別 ； Kinect

並列摘要

Currently, there are many kinds of electronic products. The human-machine control methods of these consumer electronic products, such as computers, smart TVs, smart phones…etc., are no longer confined to the traditional keyboards and mouse. Using body motions as control signals for human-machine control has become popular interfaces. The conventional techniques for identifying the hand gestures can be classified roughly into three types: depth image based, skin color based, contour trainers based, but the depth information of the image-based approach will have the hole problems, and the sole skin color information cannot achieve sufficient recognition accuracy. To overcome the above-mentioned issues, this thesis presents a real-time gesture recognition method that combines depth map and color image by a RGB-D sensor, can achieve high detection performance and accuracy of hand gesture.

This thesis’s propose is recognition hand gesture and movement by Microsoft Kinect and improved RGB color technology, and thus the RGB color information can achieve a wider range of applications. As compared to traditional color identification methods, the results can improve the corresponding recognition rate by 8% to 30%, the sole depth-hole filling method can be improved by nearly five times of the computation speed. As a result, the combination of hand movement trajectories and contour recognition can obtain real-time accurate hand gesture. The computational times of the proposed method can achieve up to 23 frames per second. These research results can be widely applied in the gesture-based human-computer interaction applications, such as interactive network TV, games, interaction computer operation ... etc.

並列關鍵字

computer vision ； contour detection ； gesture recognition ； skin color ； infrared light ； depth map ； pattern recognition ； Kinect

參考文獻

[8] 顧正偉, “體感操作與 OpenNI 介紹,” 國家高速網路與計算中心科學視算與互動媒體實驗室, 教導教材, Dec. 2012.

[16] 許耀文,陳志勇,游志雲, “以手長與手掌寬建立手部表面積的估算式,” 勞工安全衛生研究季刊, Mar. 2006.

[1] R. M. Haralick, S. Sternberg, and X. Zhuang, “Image analysis using mathematical morphology,” IEEE Pattern Anal. Machine Intell., vol. PAMI-9, July 1987, pp. 532–550.

[4] XBOX官方網站, “Xbox 360 + Kinect”

[9] Chia-ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, Shawmin Lei, “Real-time hand tracking on depth images,” Visual Communications and Image Processing (VCIP), 2011, pp. 1-4.