Variable Results of Using Explicit Behavioral Criteria to Improve Validity of Students' Self-Evaluation in Problem-based Learning

以測量明確的行為來提昇醫學生問題導向學習自我評估的準確度

梁繼權(Kai-Kuen Leung) ； 王維典(Wei-Dean Wang)

《Journal of Medical Education》 12卷4期 (2008/12) Pp. 216-224

https://doi.org/10.6145/jme.200812_12(4).0003

摘要

Purpose: To test the hypothesis that using explicit behavioral criteria for evaluation can improve the accuracy of students' self-evaluation in problem-based learning (PBL). Methods: Second to fourth-year medical students of a seven-year curriculum were studied. PBL performance of students was assessed by students using a student version of the Tutotest (S-Tutotest(superscript C)) for self-evaluation, and by facilitating tutors using the tutor version of the Tutotest (T-Tutotest(superscript C)) and a global rating score for evaluation and feedback. A multiple-choice achievement test was taken by students at the end of the PBL course. Results: 315 students' evaluations were obtained with a 78.9% completion rate. Students' self-ratings were poorly correlated to tutors' ratings. Tutors' Tutotest ratings were moderately correlated to their global ratings. Students' achievement test scores were poorly correlated to tutors' Tutotest ratings and global ratings, and uncorrelated with students' self-ratings. Students of poor achievers tended to overestimate and students of good achievers tended to underestimate their performance. Conclusions: Our study did not fully support Arthur's argument that explicit criteria of performance are important for the validity improvement of self-evaluation. Results of student evaluation varied with methods of evaluation. Explicit behavior criteria are useful for average grade students to use self-evaluation for PBL learning but not so for students with poor or good academic achievement.

關鍵字

醫學教育 ； 問題導向學習 ； 學習評估 ； 自我評估

並列摘要

Purpose: To test the hypothesis that using explicit behavioral criteria for evaluation can improve the accuracy of students' self-evaluation in problem-based learning (PBL). Methods: Second to fourth-year medical students of a seven-year curriculum were studied. PBL performance of students was assessed by students using a student version of the Tutotest (S-Tutotest(superscript C)) for self-evaluation, and by facilitating tutors using the tutor version of the Tutotest (T-Tutotest(superscript C)) and a global rating score for evaluation and feedback. A multiple-choice achievement test was taken by students at the end of the PBL course. Results: 315 students' evaluations were obtained with a 78.9% completion rate. Students' self-ratings were poorly correlated to tutors' ratings. Tutors' Tutotest ratings were moderately correlated to their global ratings. Students' achievement test scores were poorly correlated to tutors' Tutotest ratings and global ratings, and uncorrelated with students' self-ratings. Students of poor achievers tended to overestimate and students of good achievers tended to underestimate their performance. Conclusions: Our study did not fully support Arthur's argument that explicit criteria of performance are important for the validity improvement of self-evaluation. Results of student evaluation varied with methods of evaluation. Explicit behavior criteria are useful for average grade students to use self-evaluation for PBL learning but not so for students with poor or good academic achievement.

並列關鍵字

medical education ； problem-based learning ； evaluation ； self-assessment

參考文獻

Ward M,Gruppen L,Regehr G(2002).Measuring selfevaluation: current state of the art.Adv Health Sci Educ Theory Pract.7,63-80.

Gordon MJ(1991).A review of the validity and accuracy of self-evaluations in health professions training.Acad Med.66,762-769.

Mattheos N,Nattestad A,Falk-Nilsson E(2004).The interactive examination: assessing students'self-evaluation ability.Med Educ.38,378-389.

Farnill D,Hayes SC,Todisco J(1997).Interviewing skills: self-evaluation by medical students.Med Educ.31,122-127.

Barnsley L,Lyon PM,Ralston SJ(2004).Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence.Med Educ.38,358-367.