題組之相關特性對電腦化適性測驗測量精準度的影響

The Influences of the Features of Testlet on Computerized Adaptive Testing

陳柏熹(Po-Hsi Chen) ； 黃宏宇(Hung-Yu Huang) ； 王文中(Wen-Chung Wang)

《測驗學刊》 55卷1期 (2008/04) Pp. 129-150

https://doi.org/10.7108/PT.200804.0129

摘要

本研究主要目的在探討當CAT中含有題組（例如語文測驗中的克漏字測驗、閱讀測驗……等）而違反IRT局部獨立性的假定時，如何使用題組反應模式來進行CAT，並分析在含有題組的情況下，題組的相關特性（題組效果大小、題組題數佔CAT總題數的比例、題組施測順序）對CAT之能力估計精準度的影響。研究結果顯示：在含有題組的CAT中，題組的存在會降低CAT的能力估計精準性。而且隨著題組效果愈大，或是題組佔CAT總題數的比例愈高，CAT的能力估計精準性會更差。而題組試題的施測順序也會影響CAT的效能，先施測題組的信度比先施測單題高；先施測題組的均方根誤(RMSE)比先施測單題低。而且此現象在題組佔總題數比例較高時或CAT總題數較高時更加明顯。本研究建議未來在含有題組的測驗中若要進行CAT，應考慮上述題組特性的影響，以使CAT發揮較大的優勢。

關鍵字

題組 ； 試題反應理論 ； 題組反應理論 ； 電腦化適性測驗

並列摘要

Testlet-based items have been widely used in large-scale tests. Fitting standard item response models to testlet responses ignores the possible dependence between the items within a testlet. Such an item response analysis tends to overestimate the precision of measures obtained from testlets, and yield biased estimation for item parameters. This study assesses the influences of random testlet effects on the measurement efficiency of computerized adaptive testing (CAT). A simulation study was conducted. Three independent variables were manipulated: a) the magnitude of random testlet effects, b) the percentage of the testlet-based items, and c) the administered order of the testlet-based items. The dependent variables were test reliability, conditional standard error of estimation, and the root mean square of error (RMSE). Results indicated that a) the larger the random testlet effects and the higher the percentage of the testlet-based items in the testlet CAT, the lower the test reliability and the higher the conditional standard error would the testlet CAT yield, especially for those examinees with very extreme levels on the latent trait, b) administered the testlet- items first in CAT would yields higher reliability and lower RMSE. The influences of the magnitude of random testlet effects, the percentage of the testlet-based items, and the administered order of the testlet-based items, were larger in the long-test condition than in the short-test condition. We suggested that when there are testlet-based items in CAT, the testlet related feature should be taken into account in order to preserve the priorities of CAT.

並列關鍵字

testlet ； item response model ； terstlet response model ； computerized adaptive testing

參考文獻

陳柏熹(2006)。能力估計方法對多向度電腦化適性測驗的影響。教育心理學報。38(2)，195-211。

Adams, R. J.,Wilson, M.,Wang, W. C.(1997).The multidimensional random coefficients multinomial logit model.Applied Psychological Measurement.21,1-23.

Andrich, D.(1978).A rating formulation for ordered response categories.Psychometrika.43,561-573.

Birnbaum, A.,F. M. Lord,M. R. Novick (Eds.)(1968).Statistical theories of mental test scores.Reading, MA:Addison-Wesley.

Bock, R D.,Aitkin, W.(1981).Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm.Psychometrika.46,443-459.