- 期刊
Comparing Automated Writing Evaluation Systems With Human Teaching and Grading Assistant: A Case Study in EFL College Class With Grammarly
比較自動化寫作評分系統與人類助教評分的異同：以EFL大學課堂使用Grammarly為例
摘要
The development of automated writing assessment systems aims to assist teachers in grading, editing compositions, and providing learners with immediate corrective feedback to enhance writing skills. Grammarly, as a promising writing assessment system, identifies grammar errors, punctuation mistakes, corrects lengthy sentences, and addresses incorrect syntax. This study applied and evaluated Grammarly's instructional functionality in a college English as a Foreign Language (EFL) class. To examine the strengths and weaknesses of Grammarly, the research compared Grammarly with five recruited English teaching and grading assistants in assessing and proofreading essays from 20 students in an EFL career English course. The comparison includes overall scores, numbers and types of error corrections, and performance in providing modification suggestions. The results show significant correlations between Grammarly's grading and four teaching assistants, but Grammarly tended to be more stringent. Grammarly suggested more than double the number of errors, covering more error types, and modification suggestions compared to the five assistants. Grammarly and two assistants exhibit high correlations in proofreading, detecting grammar, spelling, and formatting errors. However, human assistants are more excellent in evaluating semantic fluency and vocabulary breadth due to their understanding of students' intended meanings even for grammatically inappropriate or less clear sentences. The study concludes that Grammarly can offer substantial benefits in large-class EFL writing instruction, making it a viable assessment tool. However, human grading assistants still provide essential assessment and proofreading functions.
並列摘要
自動化寫作評分系統的創建是為了協助教師評分、修改作文並給予學習者立即性的糾錯回饋以增強寫作技能。許多更好、更進階的自動化寫作評分系統不斷地被設計出來，以便能更符合教師們及學習者的需求。Grammarly是一個十分具潛力的寫作評分系統，可以判定文法上的錯誤、標點符號的錯誤、冗長句的修正、不正確的句法等。本研究檢視Grammarly系統並評估它在大班寫作教學的功能，比較Grammarly和五位招募的英文助教對生涯英語課程20位學生作文的評量與檢誤，比較的方向包含總分、錯誤糾正和修改建議的表現，5位助教在Test of English for International Communication（TOEIC）英語能力檢測分數在870～920間。結果顯示Grammarly的評分與其中4位助教有高相關，但是評分比助教嚴格；Grammarly提出的錯誤數量、錯誤類型以及修改建議三方面比5位助教多出兩倍以上，Grammarly和其中2位助教在檢誤的相關達顯著，能檢查出文法、拼字與格式型態的錯誤，但是因為5位助教能理解學生潛在所欲表達的意義，因此在語意流暢與單字使用廣度這兩方向的評量比較有效。本研究發現Grammarly在大班寫作教學的評量可提供許多助益，老師可以選用Grammarly作為大班英文寫作的評量工具，但是人類助教還是能提供重要的評量功能。