Development of a Mandarin-English Bilingual Speech Recognition System with Unified Acoustic Models

Qing-Qing Zhang ； Jie-Lin Pan ； Yong-Hong Yan

《Journal of Information Science and Engineering》 26卷4期 (2010/07) Pp. 1491-1507

https://doi.org/10.6688/JISE.2010.26.4.21

並列摘要

This paper presents our recent work on the development of a grammar-constrained, Mandarin-English bilingual Speech Recognition System (MESRS) for real-world music retrieval. Two of the main difficult issues in handling the bilingual speech recognition for realworld applications are tackled: One is to balance the performance and the complexity of the bilingual speech recognition system; the other is to effectively deal with the matrix language accents in embedded language. A unified bilingual acoustic model, which is derived by the novel Two-pass phone-clustering method based on the Confusion Matrix (TCM), is developed to solve the first problem. To deal with the second problem, several nonnative model modification approaches are investigated on the unified acoustic models. Compared to the existing log-likelihood phone-clustering method, the proposed TCM method with effective incorporation of limited amounts of nonnative adaptation data and adaptive modification, relatively reduces the Phrase Error Rate (PER) by 10.9% for nonnative English phrases and the PER on Mandarin phrases decreases favorably, and besides, the recognition rate for bilingual code-mixing phrases achieves an 8.9% relative PER reduction.

並列關鍵字

bilingual speech recognition ； two-pass phone clustering ； confusion matrix ； nonnative adaptation ； model retraining