In this paper, a mandarin speech recognition system based on tri-phone model was constructed. However, there are several practical problems when tri-phone models are applied in the speech recognition system. First, in the speech recognition system, many tri-phone models have only few occurrences in the training data, hence there is no sufficient data for robust parameter estimation of these rarely seen tri-phone models. Second, there are a large number of tri-phone models missing in the training corpus. Unseen tri-phone models are unavoidable when building cross-word tri-phone systems. We use decision tree and more training data to solve these problems.