整合眼動、唇動及語音作為人機互動輸入的裝置

Integration of Eye Movement, Lip Motion, and Speech Recognition as a Human-Computer Interactive Input Device

曾柏瀚(Bo-Han Zeng)

指導教授 ： 張國清

義守大學/電機資訊學院/資訊工程學系/碩士(2015年)

摘要

本論文創新將眼球追蹤儀(eye tracker)及Kinect作為人機互動的輸入裝置，我們利用語音辨識啟動程式，透過眼球追蹤技術用於眼動當作滑鼠游標，再偵測唇形辨識執行點擊的功能，整合多項便利性的設定讓使用者與電腦互動，並作為與外界溝通之橋梁。
在實驗的應用中，利用所設計的人機互動應用程式，讓使用者可以利用眼動，將游標移動到指定點，執行點擊的功能分為兩種模式：第一種點擊模式為眨眼大於30fps，即執行點擊的功能。第二種點擊模式為偵測唇形張開的動作進行點擊的功能。整合利用眼動和唇動，希望讓使用者可以在操作人機互動應用程式時，能夠有更多元的輸入控制使用方式。

關鍵字

眼動 ； 唇動 ； 人機介面互動

並列摘要

In this study, the innovation that use eye tracker and Kinect as a human-computer interactive input device. We firstly apply speech recognition to start the GUI program, and then use eye movement to control the computer mouse cursor by eye tracking technology, and also Kinect can recognize the lip motion for a mouse click. Let the users can use the computers and communicate with people more easily.
In the experimental application, we design the computer programs that eye tracker can let user move the mouse to the point position. And the selection functions performed separate two ways, one is winking more than 30fps, the other is the lip motion that using the mouth open to start the functions. Integration eye movement and lip motion to help the user can operate at the human-computer interaction applications, to have more multiple input device usage.

並列關鍵字

eye movement ； lip motion ； human-computer interaction

參考文獻

[1] Andrew T. Duchowski, “A breadth-first survey of eye-tracking applications. Behavior Research Methods Instruments and Computers”, 34(4), pp. 455-470, 2002.

[2] Andrew T. Duchowski, “Eye Tracking Methodology: Theory and Practice”, New York:Springer, 2nd Edition, 2007.

[3] Andrew Sears and Julie A. Jacko, “The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications”, 2nd Edition, 2008.

[4] Ware, C., and Harutune H. Mikaelian, “An Evaluation of an Eye Tracker as a Device for Computer Input”, In Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics interface CHI '87, ACM Press, pp. 183-188, 1987.

[5] Heo, H.; Lee, E. C.; Park, K. R.; Kim, C. J. & Whang, M. C., “A realistic game system using multi-modal user interfaces”, IEEE Trans. Consumer Electronics 56 (3) , pp. 1364-1372, 2010.