自動化評斷同儕回饋品質相對等第演算法：自動化方法與結果分析

Automatically Ranking Algorithm of Peer Feedback: the Method and Result Analysis

黃大珉(Da-Min Huang)

指導教授 ： 葉丙成

國立臺灣大學/電機資訊學院/電信工程學研究所/碩士(2017年)

https://doi.org/10.6342/NTU201704378

摘要

在現今的教育中，為學生的作業、考試評分已經不是專屬於老師或是助教的工作。除了由專業人士評分之外，同儕互評也是近年來逐漸興起的學生評量方式之一。從傳統的紙本互評，到現在也逐漸有一些線上同儕回饋系統，讓學生可以依照自己的時間規劃去完成同儕互評的流程。而同儕互評的趨勢，也促使許多學者進而投入同儕互評的品質計算方式與互評效果之研究。
在進行同儕互評時，如果想要提升同儕互評帶來的學習成效，那麼互評系統內的同儕互評回饋品質就是一項重要的指標。但在目前為止，計算回饋品質的方式仍舊處在人為評分的階段，並且沒有一套明確的標準與規範。此外，在面對大量的同儕互評回饋時，評斷回饋品質往往變成一項耗時費工的任務，是目前同儕互評回饋評量的瓶頸所在。
在本研究中，將使用機器學習的技術開發出一套可以幫助降低在計算回饋品質時所花費時間的演算法，稱為「隨機森林自動評斷回饋等第演算法」。此演算法會自動評斷互評回饋相對於其它互評回饋的等第，也就是互評回饋依照其回饋品質在整體互評回饋中的等級。本研究利用在課程中蒐集到的同儕互評回饋數據，並且為現行的評量方式加上新的規定來解決評定標準不夠明確的狀況，進而給出較為客觀準確的同儕互評回饋品質量化分數。將這些分析後的數據加以處理，用來作為產生演算法預測模型的訓練資料，進而發展出一套可以自動為同儕回饋評論預測相對等第的演算法。並進一步透過實驗數據證明，此演算法可以應用在網路開放式評論中，用來預測使用者給出評論的相對等第。

關鍵字

線上互評系統 ； 同儕互評 ； 同儕回饋 ； 回饋品質 ； 機器學習

並列摘要

Peer assessment has been gradually implemented in education nowadays. The role of evaluating students’ performance can be not only done by teachers, but also by students themselves. With the support of modern Internet, the process of peer assessment can be done online. Students do not have to evaluate others' work in class, and they can easily provide their feedbacks with a simple connection to an online peer assessment system. The trend has attracted many researchers' attention and put effort in studying the effect of peer assessment on students.
To improve learning performance of students in peer assessment, the quality of peer feedback is an important factor. Although there has been many methods for evaluating the peer feedback quality, the process is still being done manually, which is time consuming. There is no clear rule and standard for feedback quality so far.
This research is trying to establish an algorithm called “Random Forest Automatically Ranking Algorithm” with techniques used in machine learning to help reduce time spent on calculating quality of peer feedback. This algorithm automatically determines rank of some peer feedback relative to other peed feedback, which means it grades every peer feedback with its quality relative to others’ quality. This research uses peer feedback data collected from course to get quantified quality of peer feedback and add new rules to method for evaluating the peer feedback quality to get more objective quality of peer feedback. We use processed peer feedback data as train data for generating predict model of this algorithm to automatically rank peer feedback. Furthermore, it is confirmed by experimental data that Random Forest Automatically Ranking Algorithm can also be applied to open comments on the internet to predict rank of comments given by online users.

並列關鍵字

Online peer assessment system ； peer assessment ； peer grading ； peer feedback ； feedback quality ； machine learning

參考文獻

[1] Lan Li, Xiongyi Liu, Allen L. Steckelberg, “Assessor or assessee: How student learning improves by giving and receiving peer feedback,” British Journal of Educational Technology, vol. 41, no. 3, pp. 525-536, 2010.

[2] Nicol, David J., and Debra Macfarlane‐Dick, “Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice,” Studies in higher education, vol. 31, no. 2, pp. 199-218, 2006.

[3] Blischak, John D., Emily R. Davenport, and Greg Wilson, “A quick introduction to version control with Git and GitHub,” PLoS Comput Biol, 12.1, 2016.

[4] Ertmer, Peggy A., et al. Demetriadis, “Using peer feedback to enhance the quality of student online postings: An exploratory study,” Journal of Computer‐Mediated Communication, 12.2 (2007): 412-433.

[5] Bangert-Drowns, Robert L., et al, “The instructional effect of feedback in test-like events,” Review of educational research, 61.2 (1991): 213-238.