Effects of Changes in the Examinees' Ability Distribution on the Exposure Control Methods in CAT

考生能力分配改變對電腦適性測驗曝光控制法之影響

章舜雯(Shun-Wen Chang) ； 涂柏原(Bor-Yaun Twu)

《測驗年刊》 48卷1期 (2001/01) Pp. 167-190

摘要

試題機密是電腦適性測驗界極度關心的眾多實際問題之一。為了要達
到試題機密的要求,專家們藉著在試題選擇的過程中加入試題曝光控制法, 以直接控制熱門試題的曝光程度。這些試圖爆光控制法通常在正式使用前建立試題曝光控制參數, 作為正式進行適性測驗時控制試題曝光率的標準。然而,正式施測的情境與建立試題曝光控制參數所擬定的情境可能有所差異,導致試題曝光控制法無法完全達到控制試題曝光率的目的。本研究探討這兩個情境
中,考生能力分配改變的因素對the Sympson and Hetter method (SH 法)與the Davey and Parshall method (DP法) 的影響, 以及確認the Stocking and Lewis conditional multinomial method (SLC 條件多項式法)是否不受正式施測時考生能力分配因素所影響。本研究採用不 同的考生能力分配之組合,以電腦程式模擬電腦適性測驗程序進行。研究結果顯示,考生能 力分配的改變對SH 與DP 法在控制試題最大曝光觀察值上產生影響, 並以SH 法所受的影 響較為嚴重。研究結果支持SLC 法在控制試題曝光率的表現上,不會受到正式參與適性測 驗考生的能力分配所影響。

關鍵字

電腦適性測驗；試題曝光率；曝光控制法；能力分配

並列摘要

Item security is one of the practical issues that substantially concern the makers of high stakes tests for the continuous testing context of CATs. To satisfy the security requirements of CATs, efforts have been made to directly control the exposure rates of optimal items by incorporating statistical methods into the item selection procedure. Most methods of exposure control employ exposure control parameters derived in advance of the operational CAT situations. Since differences likely occur between the exposure control parameter derivation stage and the operational CAT administrations, the exposure control methods may not fully accomplish the goal of controlling item exposure. This study explored the effects of the distribution changes on the performance of the Sympson and Hetter (SH) and the Davey and Parshall (DP) procedures and provided an examination of the assertion that the Stocking and Lewis conditional multinomial (SLC) procedure would function independently of any real examinee population. Simulations were carried out for this study using various combinations of the intended and real ability distributions. The results showed that
the changes in the examinees' ability distribution affected both SH and DP procedures in their control of the observed maximum exposure rates and the effects were more profound for the SH method. The performance of the SLC method was demonstrated to be independent of the examinees' ability distributions in the operational CAT administrations.

並列關鍵字

Computerized Adaptive Testing ； Item Exposure Rate ； Exposure Control Method ； Ability Distribution

參考文獻

Birnbaum, A.(1968).Statistical theories of mental test scores.Reading, Mass:Addison-Wesley.

Davey, T.,Fan, M.(2000).Annual meeting of the National Council on Measurement in Education.New Orleans:

Davey, T.,Parshall, C. G.(1995).Annual meeting of the American Educational Research Association.San Francisco:

Hetter, R. D.,Sympson, J. B.(1997).Computerized adaptive testing: From inquiry to operation.Washington, DC:American Psychological Association.

Kingsbury, G. G.,Zara, A. R.(1989).Procedures for selecting items for computerized adaptive tests.Applied Measurement in Education.2(4),359-375.