Comparing Grammar Feedback Provided by Teachers with an Automated Writing Evaluation System

文法自動批改系統與教師對學生寫作錯誤之回饋

Hao-Jan Howard Chen ； Huei-Wen Sarah Cheng ； Ting-Yu Chirstine Yang

《英語教學期刊》 41卷4期 (2017/12) Pp. 99-131

https://doi.org/10.6330/ETL.2017.41.4.04

摘要

Writing has always been the focus in Taiwanese high school English classrooms. However, with often more than 40 students in a class, grading and giving feedback on students' writing can be a burden. Automated writing evaluation (AWE) systems have recently emerged that provide learners with computerized feedback and serve as an alternative way to help teachers correct essays. This study aims to investigate the employment of a newly-developed AWE system, Correct English, in correcting students' writing and uncover whether the system can provide feedback which can be used by learners to revise their texts. Ninety 12th-graders were recruited, each producing two compositions on assigned topics for the system and two human raters to correct. The computer-generated feedback was compared with that of the human raters. The results showed that one third of the feedback messages provided by Correct English were false alarms, and many common errors were left untreated. Additionally, human raters can provide revisions or suggestions on unclear or ungrammatical sentences that is beyond the capabilities of Correct English. Although Correct English can help with correcting basic errors, there remains much room for improvement. Language learners should therefore use its feedback with caution and seek teachers' help when needed.

關鍵字

grammatical errors ； automated writing evaluation ； teacher feedback

並列摘要

寫作為臺灣高中英文教學中的焦點之一，然在一班超過40人的情況下，作文批改成為教師的沉重負擔。近年出現之作文自動批改系統（AWE）與回饋功能已逐漸成為教師的輔助工具。為檢視AWE對學生作文之回饋品質，本研究蒐集了90位高三學生之英文作文，分別採用著名英文作文批改系統Correct English進行批改，以及兩位英語教師給予回饋。結果發現Correct English提供之回饋中約三分之一有誤，且相較於英語教師，Correct English未能指出許多常見之寫作偏誤。此外，英語教師可針對句意模糊或不完整的句子提出修正建議，然而Correct English卻無法提供可比擬的回饋。因此，雖然作文自動批改系統如：Correct English可提供一些基本文法的更正及回饋，但仍有相當大的改善空間。學生在使用作文自動批改系統時也應注意回饋的正確性且隨時尋求教師之協助。

並列關鍵字

文法錯誤 ； 自動寫作評量系統 ； 教師回饋

參考文獻

Shermis, M. D.(Ed.),Burstein, J. C.(Ed.)(2003).Automated essay scoring: A cross-disciplinary approach.Mahwah, NJ:Lawrence Erlbaum Associates.

Burstein, J.,Chodorow, M.,Leacock, C.(2003).Criterion online essay evaluation: An application for automated evaluation of student essays.15th Annual Conference on Innovative Applications of Artificial Intelligence.(15th Annual Conference on Innovative Applications of Artificial Intelligence).:

Burstein, J.,Marcu, D.(2003).A machine learning approach for identification of thesis and conclusion statements in student essays.Computers and the Humanities.37,455-467.

Cavaleri, M.,Dianati, S.(2016).You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students.Journal of Academic Language and Learning.10(1),223-236.

Chen, C.-F. E.,Cheng, W.-Y.(2006).The use of a computer-based writing program: Facilitation or frustration?.23rd International Conference on English Teaching and Learning.(23rd International Conference on English Teaching and Learning).: