考慮發聲特徵用於個人化電腦輔助發音訓練之對話遊戲

Dialogue Game Considering Articulatory Features for Personalized Computer-Aided Pronunciation Training

吳全勳(Chuan-Hsun Wu)

指導教授 ： 李琳山

國立臺灣大學/電機資訊學院/資訊工程學研究所/碩士(2015年)

https://doi.org/10.6342/NTU.2015.02187

摘要

本論文提出了一套在電腦輔助語言學習 (Computer-Assisted Language Learning,CALL) 中考慮發聲特徵 (Articulatory Feature) 之對話遊戲 (Dialogue Game) 架構。本論文中使用自動發音評量系統與餐廳情境對話之劇本，並利用連續狀態馬可夫決策程序(Markov Decision Process, MDP) 作為系統之模型， 並以增強式學
習 (Reinforcement Learning, RL) 訓練出系統之對化管理決策。此外，本論文亦採用由真實學習者語料庫，包括華語教師標註之發音偏誤類型 (Pronunciation Error Pattern)，訓練得到之學習者模擬模型，來產生模擬學習者來訓練系統模型。
過去相關研究少有發聲特徵結合電腦輔助語言學習的思考，本論文特提出了此全新構想。 主要考量來自前人的作品中由於永遠有若干低頻發音單位，若學習者說不好， 系統將必須耗費相對多練習回合，以實際練習到這些低頻的發音單位。 為改善此現象，本論文考慮以下重要假設：當某一發音單位出現頻率極低時，練習與該單位有高比例相同發聲特徵之其他發音單位，亦可視為一種虛擬而有進步效果之練習。 此一假設為本論文之基礎，雖然吾人並不曾有機會在實驗中證實此假設成立。因此本論文結合發聲特徵設定，希望以此虛擬練習次數之設定，彌補在前人系統中上述的缺陷。
本論文中建構出考量發聲特徵之華語學習對話樹遊戲，訓練系統適性提供練習對話語句給予不同發音情況的學習者。 並當語句缺乏某發音單位時，可以其他有高比例發聲特徵相同的發音單位，作為替代的虛擬練習， 亦可進一步給予不同發聲特徵不同權重，此設計使系統更專注於學習者表現不佳或練習不足之發音單位， 或練習該發音單位中高比例的發聲特徵之組合，以提供較多練習機會於這些發音單位。 實驗證實與分析顯示本論文中所提出方法之有其成效並可行，如果上述假設可以成立。

關鍵字

發聲特徵 ； 電腦輔助語言學習 ； 對話系統

並列摘要

In this thesis we propose a new dialogue game framework considering Articulatory Features (AFs) for personalized Computer-Assisted Language Learning (CALL). We use an automatic pronunciation evaluator and a set of dialogue scripts for reastaurant scenarios, with policy for selecting learning sentence trained by Reinforcement Learning (RL), based on continuous state Markov Decision Process (MDP) as the system’s model, We utilize a corpus of real learner data, including pronunciation Error Patterns (EP) annotated by Mandarin teachers, to train a learner simulation model, in order to produce a huge quantity of simulated learners for MDP training.

This thesis proposes a new concept of considering Articulatory Features (AFs) in a dialogue game for Computer-Assisted Language Learning (CALL). In the previous work, the learner has to go through longer dialogue paths (more dialogue turns) to practice some rare and ill-pronounced pronunciation units. Here the new approach is based on an important hypothesis: practicing other pronunciation unitswith highproportion of the same set of AFs of a considered rare unit, taken as ’pseudo practice’, can somehow offer improvement to the pronunciation of the considered rare unit. We further set different weights for different AFs within different pronunciation units, so as to have the system concentrated on those rare or ill-pronounced units. Experimental results verify the feasibility of the proposed framework based on the hypothesis above.

並列關鍵字

Articulatory Feature ； Computer-Assisted Language Learning ； Dialogue System

參考文獻

[1] Michael Levy, Computer-Assisted Language Learning: Context and Conceptualization., ERIC, 1997.

[2] Simon King and Paul Taylor, “Detection of phonological features in continuous speech using neural networks,” Computer Speech & Language, vol. 14, no. 4, pp. 333–353, 2000.

[3] Hua Yuan, Ji Xu, Junhong Zhao, and Jia Liu, “Improve low-resource non-native mispronunciation detection with native speech by articulatory-based tandem feature,” in Signal and Information Processing (ChinaSIP), 2013 IEEE China Summit & International Conference on. IEEE, 2013, pp. 127–131.

[4] Abhijeet Sangwan and John HL Hansen, “Automatic analysis of mandarin accented english using phonological features,” Speech Communication, vol. 54, no. 1, pp. 40–54, 2012.

[5] Chung-Hsien Wu, Han-Ping Shen, and Yan-Ting Yang, “Phone set construction based on context-sensitive articulatory attributes for code-switching speech recognition,” in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012, pp. 4865–4868.