應用於智慧展示系統之手勢控制技術

隨著科技日漸發達，現今的展示系統已不同於以往的靜態平面展示，而趨向透過自然使用者介面(Natural User Interface；NUI)進行操作的互動式展示系統。自然使用者介面的發展使現今許多科技產品的操作跳脫傳統的鍵盤輸入，逐漸走向利用不同方法獲取人類最自然的身體反應，例如：觸控螢幕、語音辨識、手勢辨識、腦波控制…等。然而，觸控螢幕有衛生的考量，語音辨識受語言限制，而腦波控制需要配戴儀器，因此人們傾向透過不需接觸與佩戴任何裝置的手與機器做最直接的互動。透過相機抓取手勢並加以辨識的手勢辨識是可以應用於人機互動的方法之一。其中，手部追蹤是手勢辨識系統中的第一個工作。膚色偵測以及深度資訊都是可以被用來定位手以及辨識手勢的重要線索。透過膚色偵測，可以輕易找出畫面中的膚色區域，然而，當手臉重疊、背景有類膚色物件或是有其他人在背後走動，都可能影響追蹤結果。幸運的是，上述這些問題可以透過深度資訊解決。然而，手勢辨識一般都是判斷手掌與手指的資訊，在相同深度中的手臂區域，可能會影響系統判斷手掌與手指資訊，進而影響後端的手識辨識結果。這篇論文提出基於彩圖與深度資訊之動態手勢辨識演算法可以移除手臂區域，去除手臂區域對後端手識辨勢的影響。在這篇論文中，線性內插法被用來解決手部定位時，RGB攝影機與深度攝影機並非於同一點抓取資訊而造成的彩圖與深度圖不匹配的問題。而手勢辨識系統中的第二個工作就是辨識手勢，在取得手的定位之後，有很多方法可以定義手勢，有些是透過找取參考點去定義指尖點，再透過計數指尖點的數目與角度去辨識手勢；有些則是透過手部圖像的特徵去做手勢辨識，像是Haar-like 特徵就是個很常見的方法。而這些方法通常是靜態的手識辨識，一般需要透過特定手勢才能完成指令與機器溝通。本篇論文提出基於彩圖與深度資訊之動態手勢辨識演算法，透過最直覺的動態手勢與機器做互動，不需事先閱讀使用手冊也能輕易使用。此外，在本篇論文中，手部圖像中會有一些參考點被找出來去定義手指指尖的位置，進而手掌區域的方向就可以被找出以提高手部追蹤的準確度。根據手指指尖參考點的資訊，這篇論文提出基於彩圖與深度資訊之動態手勢辨識演算法，透過參考點的找尋進而移除手臂區域，去除手臂對後端辨識的影響，最後，所提出的基於彩圖與深度資訊之動態手勢辨識演算法應用於支援3D影像格式的Smart TV上使用。實驗結果顯示，所提出的手勢辨識演算法是令人滿意的，且應用在我們的互動式系統上是適用的。

關鍵字

手勢辨識；參考點；手部追蹤；智慧電視

並列摘要

As technology advances, today’s presentation system is different from the previous static poster presentation, and tends to be an interactive presentation system operates by a natural user interface. Human interaction is a trend technology for machine control. With interactions, people can easily understand the information or command given by others. There are many various techniques can be applied to the interactive machines such as Gesture Recognition, Haptic control, Voice control, Brainwave control, and so on. However, haptic control has considerations of the hygiene, Voice control limited by language, and Brainwave control need to wear device. So, people prefer to interact by their hands without any devices and touch. Gesture recognition is an approach to Human-Computer Interaction (HCI) which relies on identifying hand gesture captured by cameras. Hand tracking is the first task in a gesture recognition system. Skin-color detection and depth information are important clues which can be used to localize the hand and to perform gesture recognition. By using skin-color detection, the skin region in the color image can be found out easily. But, there are some situations is difficult to recognize objects if only apply skin-color information, for example, when the hands and face overlapping, if there are some skin-color liked objects in the background, if there are other people behind the user, and so on. Fortunately, these problems can be solved by the depth information. However, the arm region in the same depth might disturb the result of gesture recognition. This thesis proposed a dynamic gesture recognition system based on the color image and the depth image, which can remove the arm region and reduce the influence on the following recognition. The second task in a gesture recognition system is gesture recognition. After localize the hand, there are many approaches to define the gesture. Some researchers find some reference points to define the fingertips, and the gesture could be recognized by counting the number of fingertips; some researchers recognized the gesture by the features of the hand image, Haar-like feature is a common method used in gesture recognition. However, these approaches usually recognize by some static gestures. This thesis proposed a dynamic gesture recognition system based on the color image and the depth image, which can interact with the machine with dynamic gestures. In this thesis, a linear regression approach is used to overcome a mismatch problem between the color image and the depth image in the hand location finding. This mismatch problem is caused by the RGB camera and the depth cameras are not at the same point to catch the information. Moreover, some reference points in hand image are found out to recognize the position of fingertips, then the direction of hand area can be found by the proposed algorithm to improve the hand tracking accuracy. According to the information of the reference points in the fingertips, an adapted gesture recognition algorithm is proposed in this paper. Finally, the proposed gesture recognition algorithm is applied to our interactive machine on the smart TV which can support 3D video format. The experimental results show the proposed gesture recognition is satisfactory, and the result that applied on The Smart Assistant that is feasible.