以二次規劃方式控制初始試題機率達成試題曝光率之最佳化

Controlling the Probability of the Initial Items to Decrease Item Exposure Rates

翁相擇(Xiang-Zer Weng)

指導教授 ： 何榮桂

國立臺灣師範大學/教育學院/資訊教育學系/碩士(2001年)

摘要

現代測驗理論確實在理論上較古典測驗理論更為人所信服，但是現代測驗理論卻遲遲未能邁入實用的階段。直至近年來，托福(TOEFL)測驗及GRE測驗開始採用電腦化適性測驗之後，因為電腦化適性需要當代測驗理論的配合，方確立現代測驗理論在實際面的價值。而現代測驗理論進入實用階段之後，首需考量的課題便是試題曝光率的問題。因為對於電腦化適性測驗而言，建置題庫需要花費相當大的成本，因此，為使題庫可以長久使用，題庫的保密性便成為重要的課題。而題庫保密性是否良好，其中之一便需考量試題曝光率的控制。
本研究重點便在於試題曝光率的控制。主要的構想即是分析題庫中每一題試題出現的期望次數，找出初始試題出現之機率，以使得試題曝光率達到最佳化，也即先分析所有初始試題其所有可能的情況，以推導出題庫中每一題試題出現的期望次數。然後利用非線性規劃的技術，計算出每一題初始試題出現之機率，以使得試題曝光率達到最佳化。最後再進行模擬研究以驗證此分析是否正確，且試題曝光率較未控制初始試題出現機率時之為佳。經由本研究所發展之系統，預期可以找出題庫中使用率過低的試題亦或是永遠也用不到之試題，以控管題庫的品質。此外利用本研究所發展之系統，可以找出不適合作為初始試題的題目。
關鍵字：電腦化適性測驗、試題曝光率、非線性規劃、二次規劃

關鍵字

電腦化適性測驗 ； 試題曝光率 ； 非線性規劃 ； 二次規劃

並列摘要

This study proposed a new approach that controls the probabilities of the initial items with quadratic programming to decrease item exposure rates. The purpose of propose method is expected to accomplish the CAT without changing any process in general CAT process. In most researches about item exposure rates control, these ways change some steps in general CAT process that often leads to pick up an item which is not most informative as next item, so these ways improve item exposure rates, but lengthen the test length. And these ways lose the sprit, efficiency, which CAT wants.
In this research, the way we propose indeed not only improves the item exposure rates obviously, but also promises that will shorten the test length and test overlap rate. In short, the way is analyzing all the possible responses, and counting the expected values of each item in the item bank for each initial item. Then, by the expected values of each item in the item bank for each initial item that we find, we make use of quadratic programming to find the optimum probabilities of initial items to decrease item exposure rates. Though, the degree of improvements for item exposure rates is not better than most ways that were often used, but we make it possible to shorten the test length and improve the item exposure rates.
Keywords: Computerized adaptive testing (CAT), item exposure rates, item response theory (IRT), nonlinear programming, quadratic programming

並列關鍵字

Computerized adaptive testing (CAT) ； item exposure rates, item response theory (IRT) ； nonlinear programming ； quadratic programming

參考文獻

Chang, S. W., & Twu, B. Y. (1998). A comparative study of item exposure control methods in computerized testing. ACT research report series; 98-3.

Chang, H. H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.

Chang, H. H., & Ying, Z. (1999). a-Straitified multistage computerized adaptive testing. Applied Psychological Measurement, 23, 263-278.

Davey, T., & Parxhall, C. G. (1995). New algorithms for item selection and exposure control with computerized adaptive testing. Paper presented at the annual meeting of the American Educational Research Association. April, 1995, San Franciso, CA.

McBride, J. R., & Martin, J. T. (1983). Reliability and validity of adaptive ability tests in a military setting. In D. J. Weiss (Ed.) New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 223-236). New York, Academic Press.