A Study on Chinese Spelling Check Using Confusion Sets and N-gram Statistics

Chuan-Jie Lin ； Wei-Cheng Chu

《中文計算語言學期刊》 20卷1期 (2015/06) Pp. 23-47

並列摘要

This paper proposes an automatic method to build a Chinese spelling check system. Confusion sets were expanded by using two language resources, Shuowen Jiezi and the Four-Corner codes, which improved the coverages of the confusion sets. Nine scoring functions which utilize the frequency data in the Google Ngram Datasets were proposed, where the idea of smoothing was also adopted. Thresholds were also decided in an automatic way. The final system achieved far better than our baseline system in CSC 2013 Evaluation Task.

並列關鍵字

Chinese Spelling Check ； Confusion Set Expansion ； Google Ngram Scoring Function

參考文獻

de Amorim, R.C.,Zampieri, M.(2013).Effective Spell Checking Methods Using Clustering Algorithms.Recent Advances in Natural Language Processing.(Recent Advances in Natural Language Processing).

Blair, C.(1960).A program for correcting spelling errors.Information and Control.3,60-67.

Carlson, A.,Fette, I.(2007).Memory-Based Context-Sensitive Spelling Correction at Web Scale.Proceedings of the 6th International Conference on Machine Learning and Applications.(Proceedings of the 6th International Conference on Machine Learning and Applications).

Carlson, A.,Rosen, J.,Roth, D.(2001).Scaling up context-sensitive text correction.Proceedings of the 13th Innovative Applications of Artificial Intelligence Conference.(Proceedings of the 13th Innovative Applications of Artificial Intelligence Conference).

Chang, C.H.(1994).A pilot study on automatic chinese spelling error correction.Journal of Chinese Language and Computing.4,143-149.