Research and Design of Sign Language Bidirectional Translation System

Jinhui Ran ； Liwen Dong ； Yaogui Xiang

《International Core Journal of Engineering》 8卷6期 (2022/06) Pp. 59-63

https://doi.org/10.6919/ICJE.202206_8(6).0008

摘要

In order to facilitate the normal social life of hearing-impaired people and improve their social integration ability, this paper studies the sign language bidirectional translation system. The system includes speech recognition module, chinese word segmentation module, sign language display module and sign language recognition module. Firstly, the acoustic model based on convolutional neural network is established to extract the features of the received speech information, and the corresponding text is obtained by speech recognition. Then, the jieba tool is used to complete the text segmentation to obtain the character sequence. At the same time, the corresponding sign language animation is searched and displayed through the cloud corpus. Finally, the sign language is divided into two types: static sign language and dynamic sign language, which are recognized and translated by convolutional neural network and bidirectional long short-term memory neural network respectively. In summary, the two-way translation of voice or text and sign language is realized. This system also has the characteristics of convenient operation, strong practicability and good application prospect.

關鍵字

Sign Language Recognition ； Deep Learning ； Speech Recognition ； Sign Language Translation

參考文獻

Zheng Xuan. Promoting the Professionalization of Chinese Sign Language Interpretation: Reflections on the Epidemic Situation of COVID-19 [J]. Disability Research, 2020,(01):24-32.

Zhao Jinlong. Research and Deployment of Sign Language Recognition Algorithm Based on Convolutional Neural Network [D]. Harbin Engineering University, 2021.

Chen Jianguo, Deng Xiaojun, Tan Mengxu, Zhang Guanzheng, Yu Shunhong. Design and Implementation of Sign Language Communication Platform Based on Unity [J]. Computer Knowledge and Technology, 2020,16(09):55-5.

Yang Shuying, Tian Di, Guo Yangyang, Zhao Min. Development of Simulation Sign Language Translation System [J]. Computer Simulation, 2022,39(02):278-282+418.

Li Qiang. Static Sign Language Recognition System Based on Convolutional Neural Network [D]. Jilin University, 2020.