基於深度和彩色影像的手勢辨識系統

A Hand Gesture Recognition System Based on Depth and Color Images

林哲亘(Zhe-Ken Lin)

指導教授 ： 繆紹綱

中原大學/工學院/電子工程研究所/碩士(2015年)

https://doi.org/10.6840/CYCU.2015.00195

摘要

手勢辨識為人機互動帶來了新的可能，它的應用範圍非常廣泛，包括遊戲、機器人操控和控制家電等。雖然現在已經有使用深度體感攝影機檢測人體骨架和姿勢的成熟技術，但是在辨識手勢的部分卻仍然沒有比較好的方法，因為我們的手只占影像的一小部分，所以細節比較難提取出來。本論文先用深度體感攝影機得到彩色影像和深度影像，再對彩色影像做膚色偵測；接著利用深度值將手的的部分提取出來；最後找出手的輪廓和凸包，利用手指頂點及手的凸缺陷所形成的三角形來判斷代表數字0到5的手勢。本文使用的方法只需要一台廉價的深度體感攝影機，不用額外的工具，也不需要建立資料庫，比起其他的方法便利許多，而且大多數的研究背景都不能太雜亂，也不能有前景物，本文移除了以上的限制，且辨識率可達91.6%。

關鍵字

手勢辨識 ； 凸包 ； 膚色偵測 ； 深度圖影像 ； 深度體感攝影機

並列摘要

Gesture recognition opens up a new possibility for the human-computer interaction. It can be used in a wide range of applications, including games, robotic manipulation, and appliances control. There are some mature methods of detecting the human skeleton and poses by depth somatosensory cameras; however, hands are relatively smaller than the human body and the details of hands are hard to be extracted from the images captured from a depth somatosensory camera. Therefore, there is still an unsolved problem to detect hands and recognize delicate gestures effectively. In this thesis, we first use a depth somatosensory camera to get color and depth images. Secondly, we perform the skin color detection on color images. Then, we extract the skin color with depth values. Finally, find contours of the hand and recognize gestures that represent number zero to five based on the triangles formed by fingertips and the convex. In this thesis, we only need a cheap depth somatosensory camera, without additional tools and creating databases. That is more convenient than other methods. Besides, our method can handle messy backgrounds and foreground objects that some other methods cannot. Experimental results show that the proposed work can achieve 91.6% recognition rate.

並列關鍵字

depth somatosensory camera ； depth image ； skin color detection ； gesture recognition ； convex hull

參考文獻

AAAI Press/The MIT Press, 1996.

[5] Z. Ren, J. Yuan, and Z. Zhang, “Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera,” in Proceedings of the 19th ACM International Conference on Multimedia, ser. MM ’11. New York, NY, USA: ACM, pp. 1093–1096, 2011.

[6] Y. Wen, C. Hu, and G. Yu, “A robust method of detecting hand gestures using depth sensors,” in Proceedings of IEEE Workshop on Haptic Audio Visual Environments and Games, pp. 72–77, 2012.

[14] A. R. Smith, “Color gamut transform pairs,” Computer Graphics, vol. 12, no. 3, pp. 12–19, 1978.

[15] G. R. Bradski, “Computer vision face tracking for use in a perceptual user interface,” in Proceedings of IEEE Workshop on Applications of Computer Vision, pp. 214–129, 1998.