Applying the MFRM to Investigating the Scoring Validity of Translation Tasks Used in the High-Stakes Tests

Wen-Ta Tseng ； John-Michael L. Nix ； Tzi-Ying Su ； Yi-Hung Lin

《應用英語期刊》 8期 (2015/12) Pp. 61-95

並列摘要

The purpose of this study is to describe a many-faceted Rasch model (MFRM) for the measurement of translation ability in the context of EFL testing. The MFRM model is a multivariate extension of Rasch measurement models that can be used to provide a framework for calibrating both raters and translation tasks within the context of EFL assessment with subjectively scored constructed response items. First, the continuing necessity for the use of translation tasks within the context of EFL learning and assessment and concomitant issues in trait-ability measurement are explained. Next, the use of the MFRM for solving measurement problems encountered in the subjective assessment of sentential translation ability is presented. Four facets-rater severity, rater experience, item difficulty, and test-taker trait ability were defined. Items and raters were then evaluated based on their logit scores and the consistency of performance to investigate impact of rater group membership (expert or novice) on item scores. Finally, implications for applying the MFRM in regular L2 assessment regimens at the tertiary level are discussed.

並列關鍵字

Translation test ； item response theory ； rating scale ； many-faceted Rasch model