結合平面觸控與3D手勢之手機互動介面設計

近年來隨著智慧型手機和觸控式介面的發達，人們已經相當熟悉手機觸控螢幕的2D操作手勢，不少人員也投入2D手勢操作的研究，期待能夠擴充功能又符合簡單直覺的原則。再加上最近許多深度攝影機的開發，以3D手勢操作電腦的研究也不斷推陳出新，期望人機互動能夠更直覺方便。然而3D手勢操作方式之前提往往基於「零接觸」，其動機部份乃企圖解決當人不方便觸控螢幕時，能夠在一段距離外徒手隔空操作。因為智慧型手機的配備以及處理效能日新月異，未來手機前後鏡頭甚至將擁有深度攝影機捕捉使用者3D手勢。因此，我們研究結合2D及3D兩者手勢在智慧型手機上的互動方式。也就是使用者同時一手觸控螢幕，同時，另一手在空中以3D手勢同步操作。在3D手勢方面，我們選擇簡單且使用者能夠輕鬆控制的捏取手勢（2-finger grasp）－也就是在空中以拇指和食指輕觸的手勢。利用其有抓取、撿選之隱喻，配合2D觸控以達雙手同步操作之目的。本研究利用3D空間的活動，提出不同的應用情境，包含一、圖文跨螢幕、跨程式拖移。二、2D物件的雙手操作。最後我們驗證使用者對於手機上空中抓取手勢的偏好，以及2D結合3D操作的接受度。

關鍵字

人機互動；自然使用者介面；多點觸控；行動平台；影像辨識； Kinect ；手勢辨識；雙手互動模式

並列摘要

With growing technology of smart phones with touch-screen, people become familiar with 2D gesture on screen. Many researchers have already studied the interaction techniques on touch-screen. They tried to find intuitive and convenient principles. Furthermore, there are more and more studies of 3D gestures after depth cameras become more popular. 3D gestures help human computer interface translate from GUI (Graphic User Interface) to NUI (Natural User Interface). However, most 3D gesture interactions are based on “touchless”. They intend to allow people control computers or mobiles in a distance without touching. Due to observation above, and the growing performance of mobiles, it is possible that a phone could equip with a depth camera in the near future. Hence, we design several principles for 2D (touch-screen gestures) and 3D gestures on and above phones simultaneously. That is, a user can touch-screen while use another hand in the air to control a phone at the same time. A user can complete tasks with the cooperation of two hands with fluent gestures. We select 2-finger grasp in the air because “2-finger grasp gestures” is easier and comfortable for a user. Also, “2-finger grasp” has a metaphor of “lift” or “pick”, which helps people to interact with phones. In this these, we combine 3D gestures and 2D touch gestures and present design principles. To prove our concept, we also implement two different applications. First one is an application that a user can move objects beyond current view. Second one is text selection and copy/paste. We demonstrate how to combine 2D and 3D gestures to complete tasks. Finally, we conducted user study to evaluate users’ preference and collected user feedback.

並列關鍵字

HCI ； NUI ； touch ； mobiles ； computer vision ； gesture recognition ； bimanual

參考文獻

[6] XTR3D. Available: http://xtr3d.com/

[10] M. Ortega and L. Nigay, "AirMouse: Finger Gesture for 2D and 3D Interaction," presented at the INTERACT (2), 2009.

[13] R. Y. Wang, J. Popovi, and #263, "Real-time hand-tracking with a color glove," presented at the ACM SIGGRAPH 2009 papers, New Orleans, Louisiana, 2009.

[17] J. Z. Zheng, S. De La Rosa, and A. M. Dollar, "An investigation of grasp type and frequency in daily household and machine shop tasks," in Robotics and Automation (ICRA), 2011, pp. 4169-4175.

[18] R. L. Graham and F. Yao, "Finding the convexhull of a simple polygon " Journal of Algorithms, vol. 4, pp. 324-331, 1982.

國際替代計量

結合平面觸控與3D手勢之手機互動介面設計

全文下載

主題瀏覽