Performance Analysis and Visualization of Machine Translation Evaluation

Jian-Min Yao ； Yun-Qian Qu ； Qiang Lv ； Qiao-Ming Zhu ； Jing Zhang

《中文計算語言學期刊》 11卷3期 (2006/09) Pp. 297-314

https://doi.org/10.30019/IJCLCLP.200609.0006

並列摘要

Automatic translation evaluation is popular in development of MT systems, but further research is necessary for better evaluation methods and selection of an appropriate evaluation suite. This paper is an attempt for an in-depth analysis of the performance of MT evaluation methods. Difficulty, discriminability and reliability characteristics are proposed and tested in experiments. Visualization of the evaluation scores, which is more intuitional, is proposed to see the translation quality and is shown as a natural way to assemble different evaluation methods.

並列關鍵字

Machine Translation ； Performance ； Analysis ； Visualization ； Clustering ； Natural Language Processing

參考文獻

ALPAC(1966).Languages and machines: computers in translation and linguistics.A report by the Automatic Language Processing Advisory Committee, National Research Council.

Brew, C.,H.S. Thompson(1994).Automatic evaluation of computer generated text: a progress report on the TextEval project.Proceedings of the Human Language Technology Workshop.108-113.

Darwin, M.(2001).Trial and Error: An Evaluation Project on Japanese English MT Output Quality.Proceedings of the MT Summit.57-63.

Doyon, J.,K. Taylor,J. White(1988).The DARPA Machine Translation Evaluation Methodology: Past and Present.Proceedings of the AMTA.

Forner, M.,J. White(2001).Predicting MT fidelity from noun-compound handling.Proceedings of the Workshop MT Evaluation: Who Did What To Whom held in conjunction with Machine Translation Summit VIII.45-48.