擴展反應型論述題反應之自動化評估方法 － 以教師教學能力為例

Automated Scoring Methods for Extended Response Essay Item － Case Study of Instructional Abilities

張宴晟(Yen-Sheng Chang)

指導教授 ： 張國恩 ； 宋曜廷

國立臺灣師範大學/教育學院/資訊教育學系/碩士(2007年)

摘要

擴展反應型論述題指的是受試者在回答論述題時，可以根據自己最佳的判斷組織、整合和評鑑適切的想法及觀念於答案中。雖然擴展反應型論述題比起其他選擇反應型題型更能夠測量學習者較複雜、較高層次的學習效果，但是在批改上卻非常耗費教師的時間。自動批改系統雖然有助於減輕教師批改上的負擔，然而現階段卻沒有一個可實際應用的工具，能針對擴展反應型論述題進行自動評分。因此，本文的目的在探討國內外論述題自動批改的相關研究，並提出一個以概念比對為基礎的自動評分模型，探討此模型的效能及未來發展方向。
本研究以立台北教育大學某課程的89位大一學生為對象，擬定10題有關教師教學能力的問題製成問捲進行自動評分實驗。研究結果發現：(1) 本實驗顯示本文所提之方法有相當好的效果，然而距離可應用階段所需的正確率仍有差距；(2) 語料庫的品質會大大的影響了本文所提方法之自動批改效能。

關鍵字

擴展反應型論述題自動批改 ； 概念比對

並列摘要

The Extended Responses Essay refers when the subjects answer to the discussed ques-tions basing on their best judgments, integration and evaluation of the relevance of ideas and concepts to answer. Although the Extended Response type items can measure more complicated and higher level of learning abilities than the Selecting Response type items, but to mark the answer is much more time-consuming. Although the automatic scoring system will help to reduce the burden of teacher's marking, there is no practical applica-tion tool at this stage that can automatically score Extended Response Essay items. Therefore, the purpose of this paper aims to explore the automatic scoring technique and research and put forward an automatic scoring model based on concept matching. Besides, we propose the effectiveness of this model and the future development direction.
The participants of this research are 89 freshmen of National Taipei university of Education students. We collected the data of 10 questionnaires of instructional abilities to our automatic scoring system experiment. The results showed that: (1) This experiment showed that this method is referred to very good effect, but there is still a gap to apply in practical application, (2) the quality of corpus greatly affects the automatic scoring per-formance.

並列關鍵字

automatic scoring ； concept matching

參考文獻

陳稼興、謝佳倫、許芳誠(2000)。以遺傳演算法為基礎的中文斷詞研究。電子商

林千翔、張嘉惠(2006)。基於特製隱藏式馬可夫模型之中文斷詞研究。ROCLING

Wang, H. C., Chang, C. Y., & Li, T. Y. (2005, November). Automated scoring for creative problem solving ability with ideation-explanation modeling. Paper presented at the 2005 International Conference on Computers in Education, Singapore.

Valenti, S., Neri, F., & Cucchiarelli, A. (2003). An Overview of Current Research on Automated Essay Grading. Journal of Information Technology Education, Vo-lume 2, 2003. Wang, H. C., Kumar, R., Rose, C. P., Li, T. Y., & Chang, C. Y. (2007). A Hybrid Ontology Directed Feedback Selection Algorithm for Sup-porting Creative Problem Solving Dialogues. Proceedings of 20th International Joint Conference on Artificial Intelligence.

Li, G. C., K. Y. Liu., & Y. K. Zhang. (1998). Identifying Chinese Word and Processing Different Meaning Structures. Journal of Chinese Information Processing, Vol. 2, pp. 45-53.