RATER DOMINANCE IN DISCUSSION AS A RESOLUTION METHOD

Alireza Ahmadi

《Taiwan Journal of TESOL》 17卷1期 (2020/04) Pp. 141-165

https://doi.org/10.30397/TJTESOL.202004_17(1).0006

摘要

Rater subjectivity has long been an intriguing topic. The use of discussion as a resolution method is a practical way to reduce this subjectivity. However, the efficacy of discussion depends on whether different raters get equally engaged in it or one rater tends to dominate others. This study investigated whether and how rater dominance occurs in discussion. To this end, three discussion sessions in which five Iranian raters negotiated to resolve discrepancies in rating were analyzed. Findings indicated raters were unequally engaged in discussions, so rater dominance obviously existed. However, it did not necessarily display itself in more turn-takings, a higher amount of speech or changes in scoring. The joint construction of discourse was found to affect raters' understanding of the rating criteria and scoring method. This in turn played a key role in how dominance was realized. The findings illuminate the complexity of rater dominance as a highly context-dependent issue.

關鍵字

rater dominance ； discussion ； resolution method ； speaking assessment ； EFL learners

參考文獻

Ahmadi, A., & Sadeghi, E. (2016). Assessing English language learners’ oral performance: A comparison of monologue, interview, and group oral test. Language Assessment Quarterly, 13(4), 341–358.

Attali, Y. (2016). A comparison of newly trained and experienced raters on a standardized writing assessment. Language Testing, 33(1), 99-115.

Baker, B. A. (2012).Individual differences in rater decision-making style: An exploratory mixed-methods study. Language Assessment Quarterly, 9(3), 225–248.

Barkaoui, K. (2007). Rating scale impact on EFL essay marking: A mixed-method study. Assessing Writing, 12(2), 86–107.

Barkaoui, K. (2008). Effects of scoring method and rater experience on ESL essay rating processes and outcomes (Unpublished doctoral dissertation). University of Toronto, Toronto, Canada.