在擴增實境中使用點頭作為實用的免提輸入方式

NodEverywhere: Enabling Practical Handsfree Target Selection for Augmented Reality Headsets using Head Nodding and Deep Learning

謝沅璋(Yuan-Chang Hsieh)

指導教授 ： 陳彥仰

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2019年)

https://doi.org/10.6342/NTU201903969

摘要

免用手操作的方式可以幫助人們在雙手被佔用的情況下，仍可以正常的使用頭戴式擴增實境顯示器。目前主流的免用手部操作方式主要是透過頭部或眼神的停留來作為觸發方法，然而過去的作品並沒有深入的研究這兩種方法在一般生活情境下會不會有誤觸的情形，也就是使用者並沒有想要點擊卻觸發了。這篇論文研究使用點頭作為點擊的方式，並使用深度學習來辨識點頭的動作，最後嘗試了四種方法來判斷使用者想點擊的位置。
我們先根據ISO 9241-9設計了一個點擊目標的實驗，並比較點頭與頭部及眼神停留的差異。結果顯示點頭有93.85% 會點選到正確的位置。我們接著設計了三個實際的應用情境(Gmail,YouTube,Holograms)來比較三種方法，點頭的誤觸率明顯小於以停留為選擇的機制，從使用者給各個方法的排名，我們可以得知點頭在Gmail,YouTube等需要專注的情形下比頭部或眼神的停留還要適合。最後我們進行了一項8人研究，記錄各種常見AR任務中(交談，閱讀，觀看影片和步行)使用者得眼神資料與頭部移動的數據。結果顯示基於駐留的方法具有顯著的非自願點擊問題，並且使用點頭來選擇的方法僅在300秒內無意觸發2.45次。
根據以上的結果，我們認為NodEverywhere是一種有效的點擊確認方法。

關鍵字

互動科技 ； 頭部姿勢 ； 擴增實境 ； 免提輸入

並列摘要

Handsfree input allows people to interact with the real-world without occupying their hands and is especially important for augmented reality headsets. Currently, dwell time is used with eye gaze and head pointing as a handsfree selection technique. However, prior work on improving dwell-time have not addressed unintended selections (i.e. the Midas Touch problem) for general, everyday use.This paper presents NodEverywhere, which uses deep learning to accurately detect a single head nod and uses backtracking algorithm to determine where the user clicked.
We first conducted a 12-person target selection user study according to ISO 9241-9 and compared with head-dwell and gaze-dwell. Results showed that the performance of NodEverywhere has 93.85% accuracy.We then evaluated NodEverywhere in three real application scenarios (Gmail, Youtube, Holograms).The result demonstrated that NodEverywhere have lower unintentional triggered than dwell-based input.Additionally, users feel significantly less fatigue than when using dwell-based technique.The ranking from users showed that NodEverywhere is the most suitable confirmation technique especially in YouTube application.Finally, we conducted an 8-person study to record natural gaze and head movement for various common AR tasks, including having a conversation, reading, watching a video, and walking. Results showed that dwell-based techniques have significant Midas Touch problems, and the nodding gesture only unintentional triggered 2.45 times in 300 secs.Based on these findings, we expect NodEverywhere to be an efficient confirmation technique.

並列關鍵字

HCI ； head gesture ； augmented reality ； handsfree input

參考文獻

[1] Fove eye tracking virtual reality headset.

[2] Microsoft hololens.

[3] Pupil.

[4] R. Atienza, R. Blonna, M. I. Saludares, J. Casimiro, and V. Fuentes. Interaction techniques using head gaze for virtual reality. In 2016 IEEE Region 10 Symposium (TENSYMP), pages 110–114, May 2016.

[5] J. W. Davis and S. Vaks. A perceptual user interface for recognizing head gesture acknowledgements. In Proceedings of the 2001 Workshop on Perceptive User Interfaces, PUI ’01, pages 1–7, New York, NY, USA, 2001. ACM.