自動化校長即席演講評分系統建置之初探

為了因應大型評量的需要，自動化的口語評量系統為目前評量發展的新趨勢。近年來，自動化評分系統已逐漸應用於教育測驗、課程教學、心理計量等領域，本研究擬透過行為特徵處理的方式（behavioral signal processing），來建置一套結合音訊、視訊之自動化口語評分系統。樣本為儲訓校長，演說內容為教學、校務等相關議題。本研究提出多模態計算架構，結合培訓計畫的音訊與視訊資料，並透過詞袋模型（bag of words）與費雪矢量編碼（Fisher-vector encoding），以建置自動即席演講評分系統。研究顯示，只要蒐集到的樣本在口語表現上的變異性夠大，即能有效促進機器學習的效果。此外，由本研究的實驗可看出，機器和真人的評分，甚至會高於真人評分者彼此間的相關性。然而，未來在增進系統的辨識率方面，可透過加入文本資料、處理音視訊錄製過程中的雜訊、蒐集更多演講樣本、評分者的評分等，來提高自動化系統的準確率。

關鍵字

口語評量；自動化評分系統；機器學習

並列摘要

Automatic scoring system is the current trend for large scale assessment. Automatic scoring system has became a popular research tool in the field of educational assessment, curriculum teaching and psychological measurement. In this work, we employ behavioral signal processing (BSP)-based methodology to develop a computational framework that can automate the scoring process of pre-service school principals' oral presentations given at the yearly training program. Using the audio-video feature extraction approach with session-level representation techniques based on bag-of-word and Fisher-vector encoding, we can then characterize each pre-service school principal's multimodal behavior during an impromptu speech examination for automatic scoring. For the future study, we will include lexical content and annotation; collect more samples and raters which could potentially improve the accuracy of this system.

並列關鍵字

automatic scoring system ； machine learning ； oral evaluation

參考文獻

Ochoa, X., Worsley, M., Weibel, N., & Oviatt, S. (2016). Multimodal learning analytics data challenges. Proceedings of the 6th International Conference on Learning Analytics and Knowledge, 498-499. Retrieved from http://dx.doi.org/10.1145/2883851.2883913

余民寧（2011）。教育測驗與評量：成就測驗與教學評量（第三版）。臺北市：心理。