Latent Semantic Language Modeling and Smoothing

Jen-Tzung Chien ； Meng-Sung Wu ； Hua-Jui Peng

《中文計算語言學期刊》 9卷2期 (2004/08) Pp. 29-44

https://doi.org/10.30019/IJCLCLP.200408.0003

並列摘要

Language modeling plays a critical role for automatic speech recognition. Typically, the n-gram language models suffer from the lack of a good representation of historical words and an inability to estimate unseen parameters due to insufficient training data. In this study, we explore the application of latent semantic information (LSI) to language modeling and parameter smoothing. Our approach adopts latent semantic analysis to transform all words and documents into a common semantic space. The word-to-word, word-to-document and document-to-document relations are, accordingly, exploited for language modeling and smoothing. For language modeling, we present a new representation of historical words based on retrieval of the most relevant document. We also develop a novel parameter smoothing method, where the language models of seen and unseen words are estimated by interpolating the κ nearest seen words in the training corpus. The interpolation coefficients are determined according to the closeness of words in the semantic space. As shown by experiments, the proposed modeling and smoothing methods can significantly reduce the perplexity of language models with moderate computational cost.

並列關鍵字

language modeling ； parameter smoothing ； speech recognition ； and latent semantic analysis

參考文獻

Bell, T. C.,Written, I. H.(1991).The zero-frequency problem: Estimating the Probabilities of novel events in adaptive text compression.IEEE Transaction on Information Theory.37(4),1085-1094.

Bellegarda, J. R.(1998).A Multi-span Language Modeling Framework for Large Vocabulary Speech Recognition.IEEE Transactions on Speech and Audio Processing.6(5),456-467.

Bellegarda, J. R.(1997).Proc. IEEE Workshop on Automatic Speech Recognition and Understanding.

Bellegarda, J. R.(2000).Exploiting latent semantic information in statistical language modeling.Proceeding of IEEE.88(8),1279-1296.

Bellegarda, J. R.(2000).Large vocabulary Speech recognition with multi-span statistical language models.IEEE Transactions on Speech and Audio Processing.8(1),76-84.