運用寫作評量軟體批改高中生英文作文之研究

Utilizing Automated Writing Evaluation Software in Correcting High School Students' Compositions

鄭惠文(Huei-Wen Cheng)

指導教授 ： 陳浩然

國立臺灣師範大學/文學院/英語學系在職進修碩士班/碩士(2013年)

摘要

在英文學習領域中，隨著網際網路的發達與全球化的影響，英文寫作扮演日益重要的角色。學生期盼能有更多的練習，以便因應將來各種需要以英文寫作的場合。但在台灣的高中裡，因班級人數眾多，批改作文及給予適當回饋建議對英文老師是一大負擔。近年來有些寫作評量軟體可以提供學生立即的批改與回饋，可以減輕老師的負擔。但隨著越來越多人使用這些軟體，有必要針對系統優缺點進行進一步研究。本研究檢驗一個新的寫作評量軟體Correct English所提供的寫作錯誤回饋並發現此軟體的優點與缺點。研究對象為90位來自兩所公立高中的高三學生。本研究收集此軟體針對146篇高中學生作文所提供的各種寫作錯誤回饋並加以分析；並進一步比較軟體提供的回饋與老師提供的回饋之間的差異。結果顯示此軟體提供了40種不同的回饋訊息，但大約有三分之一的回饋是錯誤訊息。此外，與老師提供的回饋相比，此系統仍然無法偵測許多學生的常見錯誤，如介系詞、時態、詞性與冠詞等；也無法提供句子改寫的功能。本研究建議使用此類寫作評量軟體的老師須注意其在教學上的使用。融合老師的引導與詳細的指示，英文老師便能善用現有的軟體功能來幫助學生寫作。

關鍵字

錯誤回饋 ； 自動寫作評量 ； 英文作文

並列摘要

In respect of English learning, the ability of writing well has been more and more important with the advances of the Internet and the trend of globalization. Learners expect to have more practices to prepare themselves for various occasions in which they have to write in English. However, for high schools in Taiwan, with more than 40 students in a class, grading and giving feedback on students’ writing has been a burden for English teachers in Taiwan. In recent years, there have been some automated writing evaluation (AWE) systems developed to provide learners with computerized feedback. These systems seem to be an alternative way to help teachers in the process of correcting essays since AWE systems promise to provide immediate score and feedback. Targeting at a newly-developed AWE system, this study aims to investigate the employment of the AWE system in correcting high school students’ compositions and find out whether the system can provide usable feedback for its users to revise their writing.
A total of 90 12th grade students from two senior high schools in Taipei were recruited in the study. Each of the students was asked to write two compositions on the assigned topics. An automated writing evaluation program called Correct English was used to generate the computerized feedback messages. At the same time, two human raters corrected and commented on the compositions. Afterwards, the computer-generated feedback messages on writing errors were compared with those of human raters’.
The results showed that Correct English provided 40 types of error feedback. Among the error feedback messages provided by the AWE system, one third of them were false alarms, which would confuse the learners. In addition, compared with the errors identified by human raters, there were still many common errors left untreated by the AWE system, such as errors in prepositions, verb tense, word form, and articles. Besides, human raters rewrote the sentences or provide suggestions when there were unclear or ungrammatical expressions, but the AWE system was not able to offer feedback on sentence level.
Based on the major findings of this study, it is suggested that language teachers should pay attention to the use of AWE systems in class. Teacher’s guidance, specific instructions and follow-up activities should be incorporated so that instructors can make the best use of the available functions to assist learners to become better writers.

並列關鍵字

error feedback ； automated writing evaluation ； English composition

參考文獻

Yeh, Y., Liou, H. C. & Yu, Y. T. (2007). The influence of automated essay evaluation and bilingual concordancing on EFL students’ writing. English Teaching & Learning, 31 (1), 117-160.

features in Criterion. Paper presented at the National Council on Measurement in Education (NCME) held between April12-16, 2004, in San Diego, CA

Burstein, J., Chodorow, M., & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing service. AI Magazine, 25(3), 27-36

Burston, J. (2001) Computer-mediated feedback in composition correction. CALICO Journal, 19 (1) 37-50

Chen, H. J. (2006). Examining the Scoring Mechanism and Feedback Quality of My Access. Proceedings of Tamkang University Conference on Second Language Writing.