以視覺為基礎之強韌多指尖偵測與人機介面應用

Robust Vision-based Multiple Fingertip Detection and Human Computer Interface Application

曾泓博(Hong-Bo Zeng)

指導教授 ： 鄭旭詠

國立中央大學/資訊電機學院/資訊工程學系/碩士(2012年)

摘要

隨著科技的發展，人們逐漸習慣使用更人性化的科技產品，直覺與容易操作的概念變得非常重要，使得利用手勢操作機器的方式也逐漸成為趨勢，取代了傳統的輸入裝置。對於以手勢操作的機器而言，多指尖偵測是很重要的一部分。多指尖偵測的相關研究主要可分為穿戴式(wearable)與非穿戴式(markerless)兩大類，前者因為需要穿戴額外的設備以標記指尖位置，必須隨身攜帶設備，較為不便，而且若需與人共用時會有衛生上的顧慮，所以後者改用膚色偵測找出手部區域，進而找出指尖位置。
本論文選用了非穿戴式的研究方法，以單一攝影機拍攝無穿戴物品的手部操作影像，並找出影像中的指尖位置。但是由於其他非穿戴式的研究因實驗環境設定或是手勢定義的關係，未考慮當手臂進入畫面時可能會產生錯誤偵測結果的問題，所以在此我們將指寬值加入至輪廓曲率(Contour & Curvature)資訊中，並將這些資訊整合成單一數值以表示其對應點為指尖位置的可能性(likelihood)，藉由此方法完成多指尖偵測，我們可以解決上述問題。最後再進一步實作出以手勢操作之簡單的人機介面系統。

關鍵字

人機介面 ； 電腦視覺 ； 指尖偵測

並列摘要

Intuitive and easy to use interfaces are very important to a successful product. With the development of technology, gesture-based machine interface has gradually become a trend to replace traditional input devices. For gesture-based machine interface, multiple fingertip detection is a crucial step. The studies of multiple fingertip detection can be classified into two main categories, wearable and markerless. For the former, the users need to wear additional equipment to facilitate the fingertip detection. Considering the inconveniency and the hygiene problem for wearable equipment, the latter requires no additional equipment to get the hand regions or the positions of fingertips.
This thesis is a markerless method. We only use a single camera to capture images and locate the fingertip accurately in the images. A lot of markerless approaches limited their experimental environment or gesture definitions. In addition, some of them used contour and distance to centroid information to locate fingertips. Most of these methods made the assumption that only hand regions are in the scene, and didn’t consider the problems that might happen when the arms are also in the scene. In this thesis, we proposed a multiple fingertip detection algorithm based on the likelihood value of Contour and Curvature information with the width data we added, which is more robust and flexible. Finally, we implement a human computer interface system using predefined gestures.

並列關鍵字

HCI ； fingertip detection ； computer vision

參考文獻

[3] "SixSense-a wearable gestural interface," 2009. [Online]

[5] D. Schmidt, M. K. Chong and H. Gellersen, "HandsDown: Hand-contour-based User Identification for Interactive Surfaces," in NordiCHI''10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, New York, 2010.

[7] K. Oka, Y. Sato and H. Koike, "Real-time Fingertip Tracking and Gesture Recognition," in IEEE Computer Graphics and Applications, 2002.

[10] "Augmented Reality," [Online]

[12] C. Liao, H. Tang, Q. Liu, P. Chiu and F. Chen, "FACT: Fine-grained Cross-media Interaction with Documents via a Portable Hybrid Paper-laptop Interface," in MM''10: Proceedings of the International Conference on Multimedia, Firenze, Italy, 2010.