電腦適性測驗線上曝光率控管之研究

Investigating Item Exposure Control on the Fly in Computerized Adaptive Testing

吳玫玲(Mei-Ling Wu) ； 陳淑英(Shu-Ying Chen)

《測驗學刊》 55卷1期 (2008/04) Pp. 1-32

https://doi.org/10.7108/PT.200804.0001

摘要

本研究的主要目的是評估線上曝光控管法－SHOF在電腦適性測驗（簡稱CAT）實施初期，考生人數不多時的控管成效，為了深入探討此方法，本研究進行以下四種曝光控管法的比較：
1. SH: The Sympson & Hetter (1985) procedure
2. SHO: The Sympson & Hetter online procedure
3. MLV: The Modified van der Linden & Veldkamp procedure
4. SHOF: The Sympson & Hetter online procedure with Freeze
研究結果顯示，四種方法的控管成效隨著考生人數增加而趨於近似，當考生人數為10,000人時，四種方法的控管成效相近，皆能妥善控管曝光率；但在CAT實施初期，考生人數不多時，四種方法的控管成效就有明顯差異，唯有SHOF可有效控管曝光率，其他三種方法都不能確保試題的最大曝光率在安全範圍內；此外，當題庫異動或施測母群改變時，SHOF的控管成效仍不受影響，曝光率仍能獲得穩定控管，整體來說，SHOF明顯改善SH、SHO與MLV的缺失，應可作為實務上曝光率控管的重要參考。

關鍵字

測驗安全性 ； 試題曝光率控管 ； 電腦適性測驗 ； 線上曝光率控管

並列摘要

The purpose of this study is to investigate the effects of the item exposure control method the Sympson & Hetter online procedure with freeze (SHOF) on test security and measurement precision. The SHOF procedure is an online version of the Sympson and Hetter procedure and can provide item exposure control on the fly, especially for earlier examinees. Four exposure control methods were investigated and evaluated thoroughly in this study. They are as follows:
1. SH: The Sympson & Hetter (1985) procedure
2. SHO: The Sympson & Hetter online procedure
3. MLV: The Modified van der Linden & Veldkamp procedure
4. SHOF: The Sympson & Hetter online procedure with Freeze
Results indicated that the four exposure control methods performed similarly as the number of examinees increased. When the number of examinees was small, however, item exposure can be controlled only by implementing the SHOF procedure. The other three methods failed to control item exposure rates for examinees who took tests earlier. In addition, the performance of the SHOF procedure was not affected by the change of item pool or distribution of examinee population. Thus, the SHOF procedure would be a better alternative than the other three procedures for controlling item exposure in CATs.

並列關鍵字

computerized adaptive testing ； item exposure control ； item exposure control on the fly ； test security

參考文獻

Chang, S. W.(2002).The impact of pool modifications on the exposure control parameters of CAT items.Psychological Testing.49,265-291.

Chang, S. W.,Ansley, T. N.,Twu, B, Y.(2002).Performance of item exposure control methods to computerized adaptive testing: further explorations.Psychological Testing.49,235-263.

Chang, S. W.,Twu, B. Y.(2001).Effects at changes in the examinees` ability distribution on the exposure control methods in CAT.Psychological Testing.48,167-190.

Chen, S. Y.(2004).Controlling item exposure on the fly in computerized adaptive testing.Paper presented at the Annual Meeting of tire Taiwanese Psychological Association.(Paper presented at the Annual Meeting of tire Taiwanese Psychological Association).:

Chen, S. Y.,Ankenmann, R. D.(2004).Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing.Journal of Educational Measurement.41,149-174.