- 期刊
針對台灣手語辭典建構一個SVG圖像檢索系統
CONSTRUCTING A SVG GRAPH RETRIEVAL SYSTEM FOR TAIWANESE SIGN LANGUAGE DICTIONARY
摘要
手語是一種視覺語言，基於認知心理的雙編碼理論，人類大腦將每個手語字詞的文字與視覺兩部份分別建立檢索的索引。目前台灣手語線上辭典的研究，其手勢的特徵檢索功能僅能透過單隻手的手型與位置。此外，圖像辨識演算法可以自動擷取出手勢的手語視覺特徵，至於這些手勢特徵的組合（即打法）是否符合語法或語意，仍需要為每個手語字詞預先建立其語法的與語意的編碼。所幸過去我們依據手語社群所認可的完整視覺符號（即手語特徵），提出一個台灣手語詞典檢索參考系統架構（Reference System Architecture for Taiwanese Sign Language Dictionary Retrieval, RSA4TSLDR），但尚未依此架構實作一個手語圖像檢索系統。基於此系統架構，每個手語字詞是一個包含多個手勢圖像所組成的序列，每個圖像由多個手勢特徵所組成。本研究欲實作此手語圖像檢索系統，當人們遭遇到不認識的手語生字時，輸入代表手語特徵的符號，就可以從手語辭典檢索目標字詞。因此，探討如何依據被手語社群認可的RSA4TSLDR系統架構，實作出一個手語圖像檢索系統，便顯得被期待與重要。因此，本研究目的乃基於RSA4TSLDR架構，開發一個台灣手語的SVG圖像檢索系統（SVG Graph Retrieval System for Taiwanese Sign-Language；SGRS4TS），以作為手語辭典編輯者及辭典檢索者的平台。本研究以可縮放向量圖像（SVG）來表示每個手語手勢，並使用XML Schema來描述手語字詞，以RDF描述手語字詞間的關係。最後，我們邀請使用者評估SGRS4TS。實驗結果顯示，SGRS4TS獲得令人信服的性能表现。而且，採用RDF技術的SGRS4TS可以提升性能，優於未採用RDF技術的SGRS4TS。
並列摘要
Sign language is a visual language. Based on the dual coding theory of cognitive psychology, a human brain indexes the text and the visual parts of each sign language word separately. The research on the Taiwanese sign language online dictionary, its gesture feature retrieval function can only use the hand shape and position of a single hand. Besides, the current image recognition algorithms can automatically extract the visual features of sign language; however, whether the composition of these gesture features conforms to syntax or semantics, it is still necessary to pre-establish the syntactic and semantic encodings for each sign language word. Fortunately, in the past, we proposed a Reference System Architecture for Taiwanese Sign Language Dictionary Retrieval (RSA4TSLDR) based on the complete visual symbols recognized by the sign language community (i.e. sign language features), but we had not yet implemented a sign language graph retrieval system based on this architecture. Based on this system architecture, each sign language word is a sequence composed of multiple gesture images, and each image is composed of multiple gesture features. This study plans to implement such a graph retrieval system. When people encounter an unknown sign language word, they can retrieve the target word from the sign language dictionary by inputting symbols that represent the features of the sign language. Therefore, it is expected and important to explore how to implement a sign language graph retrieval system based on the RSA4TSLDR system architecture, recognized by the sign language community. Therefore, the aim of this research is to develop a SVG graph retrieval system for Taiwanese Sign-Language (SGRS4TS) based on the RSA4TSLDR architecture to serve as a platform for sign language dictionary editors and dictionary searchers. We use scalable vector images (SVG) to represent each sign language gesture, use XML Schema to describe sign language words, and use RDF to describe the relationship between sign language words. Finally, we invite users to evaluate SGRS4TS. The experimental results show that SGRS4TS gets convincing performance. Moreover, SGRS4TS adopting RDF technology can improve its performance and is better than SGRS4TS without RDF technology.