The Analytical Determination of Item Exposure Rates and Trait Estimate Precision in Computerized Adaptive Testing

分析“電腦適性測驗”的試題曝光率及能力估計精確度

陳淑英(Shu-Ying Chen)

《測驗學刊》 51卷1期 (2004/06) Pp. 103-115

https://doi.org/10.7108/PT.200406.0103

摘要

試題曝光率及能力估計精確度是「電腦適性測驗」（Computerized Adaptive Testing, 簡稱CTA）成功與否的重要關鍵，到目前為止，CAT在這方面的研究，大都是以電腦類比的方式來探討，雖然電腦類比的研究方式有許多優點，其研究結果的準確性去常有很大的爭議，當模擬的次數不夠多時，模擬的研究結果常會和真正的結果有相當大的差距，因此電腦模擬研究方法通常是當結果無法以理論推導而得時的一個概似方法。目前大部分的CAT研究，都還停留在以電腦類比的方式來進行，如果CAT的試題曝光率及能力估計精確度，可以用理論推導而得，那麼CAT在這方面的研究效率及準確性就可以顯著地改善。
本研究的主要目的是探討如何籍著機率理論來推導試題曝光率及能力估計精確度，並以此推導出的結果來評估電腦模擬研究在三個不同模擬次數下（100、500及1000）的效率及準確性。研究結果顯示，電腦適性測驗的試題曝光率及能力估計精確度確實可能透過簡單的機率理論推導出來，電腦模擬研究的準確性除了受到模擬次數的影響外，也受到測驗長度的影響，測驗長度愈長、電腦模擬研究的準確性愈高。對一個包含二十道題的電腦適性測驗來說，使用500個模擬者的電腦模擬研究，便可充分提供準確的結果。

關鍵字

電腦適性測驗 ； 試師曝光率 ； 能力估計精确度 ； 模擬研究

並列摘要

Precision of trait estimation and determination of item exposure rates are always important considerations in conducting computerized adaptive tests (CATs). To date, simulation techniques have been used in research concerning these two considerations. Even though conducting simulation studies has some advantages over using real human subjects, the simulation results may not be accurate. The purpose of this study was to investigate the precision of CAT trait estimation and to determine item exposure rates by using an analytical approach (referred to as Tree). The results thus obtained were used to evaluate those observed from a simulation study, where three different levels of replications, 100, 500, and 1,000 (referred to as Sl00, S500, and S1K respectively) were implemented.
Based on the results observed from this study, the differences among S100, S500, S1K, and Tree decreased as test length increased. For test length as long as 20 items, the differences among S500, S1K, and Tree were negligible. In other words, a simulation study with 500 replications can provide results as accurate as the analytical approach for a 20-item test.

並列關鍵字

computerized adaptive testing ； item exposure rate ； trait estimate precision ； simulation study

參考文獻

Davey, T.,Parshall, C. G.(1995).New algorithms for item selection and exposure control with computerized adaptive testing.

Lord, F. M.(1980).Applications of item response theory to practical testing problems.

Stocking, M. L.,Lewis, C.(1998).Controlling item exposure conditional on ability in computerized adaptive testing.Journal of Educational and Behavior Statistics.23

Stocking, M. L.,Lewis, C.(2000).Computerized adaptive testing: Theory and practice.

Sympson, J. B.,Hetter, R. D.(1985).Proceedings of the 27th Annual Meeting of the Military Testing Association.