以代符基底比對程式輸出之程式作業評分機制之研發與評估

Development and evaluation of program assignment grading mechanism through token-based program output matching

陳彥志(Yan-Jhih Chen)

指導教授 ： 周志岳

元智大學/資訊學院/資訊工程學系/碩士(2011年)

https://doi.org/10.6838/YZU.2011.00179

摘要

程式設計相關課程常常使用程式作業讓學生練習程式實作，然而程式作業的批改相當費時費力。學生在繳交完程式作業後常常無法即時地得到回饋，導致無法在很快的時間內更正其錯誤而影響了學習成效。目前已有一些電腦輔助系統可以即時地批改學生的程式作業，但是大部分系統採用完整字串比對機制來判斷程式執行輸出的正確性，並沒有支援依程式執行輸出正確性比例評分的功能。本研究研發具備代符基底比對機制之電腦輔助評分系統來針對學生的程式作業之輸出正確性比例進行評分。此電腦輔助評分系統在學生繳交程式作業後立即編譯學生的程式、執行測試個案、比對執行輸出的正確性、然後根據老師所設定的代符基底比對評分方案來計算該作業之評分。系統將評分顯示給學生作為形成性評量結果以便學生可以修正其程式錯誤後再次上傳，此評分也可以作為老師評分時的參考以減輕老師評分之負擔。本研究研發了代符基底比對機制，並提出了五類代符基底比對規則來評估學生程式輸出的正確性比率，並根據此正確性比率來評分。此程式作業評分系統實際運用在二個學期的程式設計課程當中以評估此系統的評分準確性、對學生的幫助度、以及學生對此評分機制的態度。此評估得到三個主要的結果：第一，採用代符基底比對機制的電腦評分準確率可達90%，其中有七成五的電腦評分結果和人工評分結果完全相同。其評分準確性高於採用完整字串比對機制的評分準確性。第二，具備代符基底比對機制之電腦評分系統所提供的形成性評量回饋可以協助學生修改其程式之錯誤。第三，問卷結果顯示學生們認為代符基底比對機制的電腦評分是準確的，並認為此評分系統有助於他們，也喜歡此系統。整體而言，採用代符基底比對機制之程式作業評分系統比起完整字串比對機制有較高的評分準確性，也能夠有效地協助學生發現並修正其程式作業的錯誤。

關鍵字

程式作業 ； 電腦輔助評分 ； 代符基底比對機制

並列摘要

Programming assignment is wide-applied in computer science courses to enable students to practice programming; however, marking programming assignments is time-consuming so that students cannot receive immediate marking feedback to quickly correct their errors in assignments. Some researchers developed systems to assist teachers and teaching assistants in marking programming assignments; however, most systems used entire output matching mechanism to judge execution correctness of programs. This project developed a token-based output matching mechanism, including five kinds of token-based output matching rules, to marking student programming assignments. When students submit their programs, the system will compile the programs, execute the programs by test cases, match program output to judge correctness, and use a token-based scheme to grade student program. The system provides the grading result as a formative assessment result to students and enables them to correct their errors in program and to re-submit the program. The system can also help teachers to lessen their load of marking programming assignments. The system was applied and evaluated in two programming courses. The evaluation results show that the grading accuracy of the computer grading is 90% and about 75% computer grading of token-based program output matching mechanism is the same as human grading. The results also show that providing the computer grading as a formative assessment result can help students refine their programs. In addition, students like the program assignment grading mechanism and consider the system grading correct and helpful.

並列關鍵字

programming assignments ； computer assisted marking ； token-based matching mechanism

參考文獻

Ala-Mutka, K. M. (2005). A survey of automated assessment approaches for programming assignments, Computer Science Education, 15 (2), pp. 83-102.

Cheang, B., Kurnia, A., Lim, A., & Oon, W.-C. (2003). On automated grading of Programming Assignments in an academic institution. Computers & Education, 41, 121 – 131.

Joy, M. & Luck, M. (1999). Plagiarism in programming assignments, IEEE Transactions on Education, 42 (2), pp. 129-133.

Li, J, Pan, W, Zhang , R, Chen,F, Nie, S, He, X,.(2010). Design and Implementation of Semantic Matching Based Automatic Scoring System for C Programming Language. In Proceedings of Edutainment 2010, LNCS 6249, pp. 247-257.

Linn, M. C. & Dalbey, J.(1985). Cognitive consequences of Programming Instruction: Instruction, Access, and Ability. Educational Psychologist, 20(4): 191-206.