以自然語言技術自動評估學生答案之研究

Automatic Assessment of Students’ Answers by Natural Language Processing Techniques

曹家豪

指導教授 ： 侯文娟

國立臺灣師範大學/理學院/資訊工程研究所/碩士(2010年)

摘要

為了增進老師與學生之間的互動以提高學生學習的意願，最主要的事是讓老師能迅速的了解學生的學習狀況，一個智慧型電腦系統應該要具備有自動評量學生分數的功能，當老師提出了問題後，一開始我們先建立評量的文件集，有了這些文件集後，我們依照下列的步驟去擷取出有意義的資訊：(1)為了得到句法的相關資訊，我們一開始對文件做詞性標記，(2)因為標點符號以及十進位數字會對我們造成干擾的資訊，我們也對此進行去除的動作，(3)為了聚集更好的資訊，我們也對句子進行正規化以及還原成字根的步驟，(4)擷取另外的資訊。在這篇論文中我們將評量的問題改成以分類的角度來進行實驗，即將學生分數分成兩個類別，其中一個類別是得分為6分到10分，另外一個類別就是0到5分。我們得到平均精確度為65.2%，並且從初步的二分類擴展到多分類，precision由原本的65.2%提高到70.8%藉由SVM進行的實驗得到了一個令人振奮的結果，在未來希望能有更進一步的成果。

關鍵字

自由文本 ； 自然語言處理 ； 支撐向量機

並列摘要

For improving the interaction between students and teachers, it is fundamental for teachers to understand students' learning levels. An intelligent computer system should be able to automatically evaluate students' answers when the teacher asks some questions. We first built the assessment corpus. With the corpus, we applied the following procedures to extract the relevant information: (1) apply the part-of-speech tagging such that the syntactic information is extracted, (2) remove the punctuation and decimal numbers because they play the noise roles, and (3) for grouping the information, apply the stemming and normalization procedure to sentences, (4) extract other features. In this study, we treated the assessment problem as the classifying problem, i.e., classifying students’ scores as two classes such as above/below 6 out of 10. We got an average of 65.2% precision rate, and extended from binary to multiple categories, precision from the original 65.2% to 70.8%. The experiments with SVM show exhilarating results and some improving efforts will be further made in the future.

並列關鍵字

free-text assessment ； natural language processing ； support vector machine

參考文獻

5. Foltz, T., Kintsch, W., Landauer, T.: The Measurement of Textual Coherence with Latent Semantic Analysis. Discourse Processes vol. 25, no. 2-3, Special Issue: Quantitative Approaches to Semantic Knowledge Representations (1998)

6. Friedman, C., Hripcsak G., DuMouchel, W., Johnson, S.B., Clayton, P.D.: Natural language processing in an operational clinical information system. Nat.Lang.Eng.

pp. 83-108. (1995)

7. Gheorghiu, R. and VanLehn, K. XTutor: An Intelligent Tutor System for Science and Math Based on Excel. Lecture Notes in Computer Science Springer Berlin / Heidelberg Volume 5091/2008 pp.749-751(2009)

9. Jackson, P. and Moulinier, I.: Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization. John Benjamins Publishing Company, (2002)