Effects of Score Transformation on the Composite Scores under the Multivariate Proficiency Distribution Using IRT

原始至量尺分數轉換法在多向度能力分配下對由試題反映理論模式產生量尺總分的影響

章舜雯(Shun-Wen Chang) ； 滕欣(Shin Teng) ； 盧家鋒(Chia-Feng Lu)

《測驗學刊》 60卷3期 (2013/09) Pp. 457-488

摘要

本研究探討原始至量尺分數轉換法在多向度能力分配中，對使用試題反應理論模式產生量尺總分的影響；在測驗組合、各學科所測量的能力彼此互為相關的情境下，本研究建立一多向度能力分配，根據Kolen、Wang 與Lee（2012）的IRT模式、使用實徵資料，探討直線、常態化以及正弦反函數等三種不同的轉換法以試題反應理論模式產生量尺總分的效果，研究中以包含五個測驗學科的2008年國中基測第一次測驗5,000筆考生的隨機樣本進行。評鑑結果的標準是分別針對不同轉換方式，比較量尺總分在統計與測量方面的特性、分數分配圖形、總誤差量與信度值，以及測量標準誤。研究結果顯示，不同的分數轉換法所得量尺總分有著不同的分數特性，其描述統計值、分數分配圖形與測量標準誤皆有所不同。原始至量尺分數有不同的轉換形式，量尺總分又通常以各學科量尺分數分別加總而成，所以分數的轉換法在考生量尺總分上扮演著極為重要的角色。本研究應用Kolen等人的IRT模式建立多向度能力分配，探討不同分數轉換法對於應用試題反應理論模式產生量尺總分的影響，研究的結果對透過不同轉換法所得量尺總分的特性提供了有用的訊息。

關鍵字

多向度能力分配 ； 原始至量尺分數轉換 ； 測驗組合 ； 量尺總分 ； 試題反應理論

並列摘要

This study was designed to examine the composite scores that combine the individual components of the test battery using scale scores resulting from different raw-toscale score transformations under the multivariate proficiency distribution using IRT. The purpose was to evaluate the impact of the three conversion approaches of the linear, the normalizing, and the arcsine transformation on the composite scores obtained by using IRT with the consideration that correlations existed among the examinees' proficiencies. The effects of the transformation were explored via the special case of Kolen, Wang, and Lee's (2012) IRT modeling using empirical data. The five tests of the Basic Competence Test were employed, with a random sample of 5,000 examinees drawn from the data in 2008. The analyses included the summary statistics and frequency distributions of the composite scores, overall SEMs, reliability values and the CSEMs. The results showed that the different transformations led to diverse outcomes of the composite score attributes. Their descriptive statistics, frequency distributions as well as CSEMs were not the same among the various conversion procedures. Composite scores are often formed based on scale scores attained from different forms of conversion; the role of the transformation can be critical in influencing the test results of the examinees' composite scores. Assessing the effects of the score transformation through Kolen et al.'s modeling has helped to understand more about the characteristics of the composite scores under the multivariate environment via IRT.

並列關鍵字

composite score ； IRT ； multivariate proficiency distribution ； raw-to-scale score transformation ； test battery

參考文獻

Brennan, R. L.(Ed.)(1989).Methodology used in scaling the ACT Assessment and P-ACT+.Iowa City, IA:American College Testing Program.

Chang, S. W.,Teng, S.,Wu, Y. T.(2010).Explorations of composite scores under the multivariate proficiency distribution using IRT.Annual meeting of the National Council on Measurement in Education.(Annual meeting of the National Council on Measurement in Education).:

Chang, S.W.(2006).Methods in scaling the Basic Competence Test.Educational and Psychological Measurement.66(6),907-929.

Feldt, L. S.(1984).Some relationships between the binomial error model and classical test theory.Educational and Psychological Measurement.44,883-891.

Freeman, M. F.,Tukey, J. W.(1950).Transformations related to the angular and square root.The Annals of Mathematical Statistics.21,607-611.