以單一攝影機追蹤手部與模擬多點觸控螢幕

Virtual Multi-touch Screen Visual Hand Tracking by Single Camera

邱耀德(Yao-De Chiu)

指導教授 ： 謝禎冏

大同大學/電機資訊學院/資訊工程學系所/碩士(2009年)

摘要

在近幾年來的電子產品其傳統輸入裝置漸有被取代的走向，其中以手勢辨識取代輸入裝置的功能最受歡迎。為了避免環境因素如光線、背景、相機等的影響造成膚色、手或臉部偵測錯誤，大多數的系統設計為基於使用多個攝影機、紅外線攝影機、或是3D立體攝影機取得物體的三維座標、或限制使用於單純背景下以增加準確率。但是這些攝影機的價格較昂貴，或需要高的計算量，因此需要一個平價且列為標準配備的攝影機，以及低計算量且放寬使用限制的人機介面。本論文提出了一個藉由單一攝影機，開發出不需碰觸到螢幕的虛擬觸控人機介面。首先，我們利用Haar - like特徵找出人臉，再利用人臉的膚色取得適應性膚色模型，找出當前使用者的手，並判斷所偵測到的是否為移動中的手。接著將偵測到的手進行鏈碼分析，找出手腕以上的部分，再以PCA分析出手掌的質心和手掌的特徵向量；其中手掌的質心對應到視窗系統的滑鼠座標；手掌的特徵值與特徵向量是用來排除膚色但非手的物體，以克服環境因素造成的偵測失誤。最後再利用內建控制游標的功能，透過觸控螢幕的設計理念，設計近似觸控螢幕的虛擬觸控螢幕，本系統並可同時偵測雙手，我們以i-Phone的多點觸控做為模擬標的，擴充為虛擬多點觸控螢幕。經實驗測試於操作距離為兩公尺內，手部質心的穩定度94.9%，多點觸控手勢辨識率達95.1%，以及Touch的辨識率93.3%，證實本方法之實用性。

關鍵字

軌跡追蹤 ； 多點觸控 ； 虛擬觸控螢幕 ； 手勢辨識

並列摘要

In recent years, many hand gesture recognition systems have been developed to replace some traditional input devices. However, the skin color may look quite differently, depending on camera settings, illumination, shadows, people’s tans, and ethnic groups. To increase the accuracy of hand detection, systems deploying multiple cameras, infrared cameras, and 3D stereoscopic cameras are proposed. Furthermore, some restrictions are made such as lighting and uniform background. Still, the prices of these cameras are more expensive. In this paper, we present a virtual touch screen by visual hand tracking with single ordinary web camera. Firstly, human face is detected by haar–like textures. Face color sampling is used to build the adaptive skin color model for hand detection in unrestrictive environment such as complex background with moving objects. The person nearest to the camera is assumed to be the user whose hands are extracted and tracked. Secondly, extracted hands are represented by chain codes and analyzed to find the moving hand’s palm. The primary benefit of PCA arises from quantifying the importance of each dimension for describing the variability of a data set. Therefore, PCA is adapted to find the eigenvectors and the centroid of oval palm. The centroid of Palm corresponds to the cursor position in the window system. The eigenvectors and eigenvalues could be used to filter out non-palm moving objects. By detecting both palms of two hands, multi-touch interface can be simulated. I-Phone’s multi-touch hand gestures are recognized and accuracy rate is 95.1% in average. The stability of the cursor improved by motion analysis could achieve 94.9%. Touch recognition rate is 93.9% which demonstrate the feasibility of proposed system.

並列關鍵字

Hand gesture recognition ； trajectory tracking ； multi-touch ； virtual touch screen

參考文獻

[1] C. L. Huang and S .h. Jeng, “A model-based hand gesture recognition system”, Machine Vision and Appl., vol.12, pp.243-258, 2001.

[2] T.S Huang, Ying Wu, and John Lin, “3D model - based visual hand tracking”, Proc. of the 2002 IEEE int, Conf. on Multimedia and Expo, vol.1, pp.905-908, 2002.

[4] Wei Du and Hua Li, “Vision based gesture recognition system with single camera”, Proc. of ICSP2000, vol2, pp.1351-1357, 2000.

[9] J.F. Lichtenauer, G.A. ten Holt, E.A. Hendriks, and M.J.T. Reinders, “Sign Language Detection Using 3D Visual Cues”, IEEE, 2007.

[10] Jefferson Y. Han, “Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection”, UIST'05, 2005.