視線感知即時翻譯工具的開發與評估

The Development and Evaluation of a Gaze-aware Real-time Translation Tool

李宜軒(Yi-Hsuan Lee)

指導教授 ： 葉怡玉

國立臺灣大學/理學院/心理學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201700806

摘要

閱讀第二語言的文章時，讀者常需即時的翻譯，以便理解篇章內容。針對此需求，過去研究試圖以眼球追蹤來反映讀者的心智運作，並由此推算讀者當下的需求，俾利閱讀流暢，增進體驗。然而，過去的即時翻譯研究多以固定閾值做為是否提供協助的準則，並未考慮閱讀行為的個別差異。此外，研究多以主觀報告為評估標準，缺乏客觀的資料佐證。是故本研究模擬視線感知翻譯工具原型，參照心理學於眼動與閱讀的發現，採用眼動控制數學模型中詞彙特性與凝視時間的計算邏輯，分別以停滯時間與回視眼跳為顯示時機的計算方式，並提供中文翻譯與無意義刺激，以觀察因應閱讀偏差而觸發的註解，如何影響個體的眼動表現、認知負荷及閱讀策略。結果顯示系統主動呈現之「中文翻譯」顯著增進閱讀理解、降低工作負荷，卻也同時改變使用者的閱讀策略，包括延長凝視時間、減少略視現象、增加再閱讀的比例等。再者，目標詞的詞頻高低亦與其是否會被凝視、被反向閱讀有關。根據整體與區域性眼動指標，基於個人化的停滯時間所提供的註解對於詞彙處理歷程具明顯助益。本研究歸納視線互動之設計指南供後人參考，而此實驗結果能否應用至真實場域或各式載具上，尚待未來研究進一步釐清。

關鍵字

即時翻譯 ； 注意力感知系統 ； 視線互動 ； 眼球追蹤 ； 眼動誘發回饋

並列摘要

Non-native speakers often need instant translation for comprehending documents in foreign language. To meet this need, prior research has attempted to map users’ eye movement to cognitive processes for recognizing their intention in real-time and improving the reading experience. However, recent research in gaze-based interaction mostly used fixed dwell-time as the threshold to determine whether or not to provide assistance. This approach ignores individual differences in reading behavior. Moreover, the benefits of gaze-contingent feedback were based on subjective evaluation so that the impact on objective performance is still missing. This study developed a gaze-aware instant translation prototype based on the psychological aspects of eye movements in reading by incorporating a mathematical model of eye-movement control in reading for computing fixation durations as a function of lexical difficulty. The system automatically provided Chinese translations or a meaningless X mask based on dwell-time or regressive saccades to investigate how the annotations triggered by deviation in reading pattern affect a user’s eye movements, cognitive load and comprehension accuracy. The results showed that proactive translations could improve reading comprehension and reduce cognitive load. Instant annotations also influence a user’s reading strategy, including prolonging fixation durations, decreasing skipping rate and increasing regression rate. Furthermore, the frequency of the target words was related to their fixation patterns. According to global and local eye-movement measures, the annotations provided based on personalized thresholds of dwell time were significantly helpful for word processing. Considering the methodological issues and observed results, this thesis suggests design guidelines and recommendations for developing gaze-aware reading applications. Future research could investigate whether a personalized gaze-aware annotation could be applied to the real-world settings or adapted to different devices.

並列關鍵字

Attention Aware System ； eye tracking ； gaze-based interaction ； gaze-contingent feedback ； real-time translation

參考文獻

Ashby, J., Rayner, K., & Clifton, C. (2005). Eye movements of highly skilled and average readers: Differential effects of frequency and predictability. The Quarterly Journal of Experimental Psychology Section A, 58, 1065-1086.

Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.

Beatty, J. (1982). Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91, 276.

Biedert, R., Buscher, G., & Dengel, A. (2010). The eyebook–using eye tracking to enhance the reading experience. Informatik-Spektrum, 33, 272-281.

Brysbaert, M., Drieghe, D., & Vitu, F. (2005). Word skipping: Implications for theories of eye movement control in reading. Cognitive Processes in Eye Guidance, 3, 53-77.