基於手指穿戴式相機和聽覺回饋於室內空間之視障者尋物導引系統

我們開發出一個穿戴式系統來幫助視障者在室內空間尋物。對比之下，現行的系統皆在使用空間或是穿戴性上有所限制。我們的系統藉由一個手指穿戴式的小型相機擷取影像來檢測目標並預估它離使用者的距離，再利用骨傳導雙聲道耳機來表述目標物的種類、方向、以及位置資訊來引導使用者尋找物體。尤其我們結合了基於深度學習的目標檢測模型和基於模板匹配的物件追蹤方法，即便是在模型遺失檢測的情況下，也能得到可信賴的目標位置。我們訓練的模型在測試資料集中，檢測傢俱(例:桌子，櫃子)和日常用品（例:錢包，鑰匙）更是有辦法達到高的精確度(>85%每項物品) 我們招募12位視障者來執行使用者研究實驗，透過即時互動的實驗來檢驗我們的系統效能。在本次實驗中，不論是環境或是目標物都是我們在訓練模型時所沒有出現過的物體。我們從實驗中發現，我們的系統跟真人輔助導引的方式在任務成功率上並沒有統計上的顯著差異，這展示了我們系統的強大性能。最後在受測者訪談過程中，視障者指出我們的系統可以使他們了解該空間中的家具擺設，並且縮小需要搜尋的範圍和提昇對於次任務的效率。

關鍵字

視障輔助；物體偵測；影像辨識；深度學習；互動設計；即時系統

並列摘要

We introduce RingGuardian, a portable wearable system to support blind and visually impaired (BVI) users to perform room-level search of objects. In contrast, most previous methods focus on limited search space and/or with limited portability. RingGuardian captures images from a small finger-worn camera to detect and estimate the distance of objects. Then, a stereo headphone is used to guide the BVI user by conveying the object category, direction and distance information. In particular, we combine an extended deep-learning-based object detector with a template-based object tracker to obtain reliable object tracks even under missing detection. In our testing set, our detector achieves high precision (>85% per instance) at detecting furniture (e.g., table, cabinet) and daily necessities (e.g., wallet, key). We empirically evaluate our full system's performance through an experiment followed by a real-time interactive user study with 12 BVI participants conducting in an environment with object instances which are unseen during model training. We discover that our system and human-assistive guiding strategy have no statistically significant difference in trial success rates. This demonstrates the strong performance of our full system. Finally, in the interview session, BVI users indicate that RingGuardian allowed them to know the arrangement of furniture in the environment also narrow down the search space and increase the efficiency of the task.