Chinese Chunking Based on Maximum Entropy Markov Models

Guang-Lu Sun ； Chang-Ning Huang ； Xiao-Long Wang ； Zhi-Ming Xu

《中文計算語言學期刊》 11卷2期 (2006/06) Pp. 115-135

https://doi.org/10.30019/IJCLCLP.200606.0002

並列摘要

This paper presents a new Chinese chunking method based on maximum entropy Markov models. We firstly present two types of Chinese chunking specifications and data sets, based on which the chunking models are applied. Then we describe the hidden Markov chunking model and maximum entropy chunking model. Based on our analysis of the two models, we propose a maximum entropy Markov chunking model that combines the transition probabilities and conditional probabilities of states. Experimental results for two types of data sets show that this approach achieves impressive accuracy in terms of the F-score: 91.02% and 92.68%, respectively. Compared with the hidden Markov chunking model and maximum entropy chunking model, based on the same data set, the new chunking model achieves better performance.

並列關鍵字

Chinese Chunking ； Maximum Entropy Markov Models ； Chunking Specification ； Feature Template ； Smoothing Algorithm

參考文獻

Abney, S.(1991).Parsing by Chunks.Principle-Based Parsing, Kluwer Academic Publishers, Dordrecht.257-278.

Berger, A.,S. A. Della Pietra,V. J. Della Pietra(1996).A Maximum Entropy Approach to Natural Language Processing.Computational Linguistics.22(1),39-71.

Bikel, D.M.,D. Chiang(2000).Two Statistical Parsing Models Applied to The Chinese Treebank.In Proceedings of the second Chinese Language Processing Workshop.1-6.

van den Bosch, A.,S. Buchholz(2002).Shallow Parsing on the Basis of Words Only: a Case Study.In Proceedings of the 40th Annual Meeting of ACL.433-440.

Carroll, J.,T. Briscoe,G. Carroll,M. Light,D. Prescher,M. Rooth,S. Federici,S. Montemagni,V. Pirrelli,I. Prodanof,M. Vannocchi(1997).Phrasal Parsing Software.(Sparkle Work Package 3).