可回溯的電腦化適性測驗

Computerized adaptive tests with limiting answer review

林信雄(Lin, Xinxiong)

指導教授 ： 何榮桂

國立臺灣師範大學/教育學院/資訊教育學系/碩士(2006年)

摘要

電腦化適性測驗（computerized adaptive tests, CAT）考量到能力估計值演算法之準確度，不允許受試者對已作答的題目進行回溯，因此無形中提高了受試者的測驗焦慮指數，亦造成受試者的真實能力更難以被精確地測量。本研究嘗試提出一簡單合理的演算法（RCAT演算法），以供可回溯的CAT作為能力估計值重新計算之用。研究採用模擬方式評估演算法之良窳。
研究結果顯示，使用RCAT演算法之可回溯CAT可更精準測得受試者的真實能力值，且RCAT完成測驗後之測驗標準誤與CAT不相上下，只是在使用RCAT時，需要使用較長的測驗長度作為代價，方可達到與CAT相同水準的測驗標準誤。

關鍵字

項目反應理論 ； 回溯 ； 電腦化適性測驗

並列摘要

The popularity of computerized adaptive testing (CAT) has been increasing year by year. Item review is not permitted in many adaptive tests because test makers consider it does not follow the logic on which adaptive tests are based. Studies show that the no-revision policy causes increased anxiety to examinees. Consequently, this anxiety can decrease the examinee’s performance on adaptive tests, which would increase the error in their ability estimates. The purpose of this study is to suggest and to evaluate a RCAT algorithm for re-estimating examinee’s ability in CAT with limited answer review. The RCAT algorithm was examined through a simulation study.
The results in this study show that RCAT is more precise than CAT in estimating examinee’s ability. Comparison between the RCAT and CAT conditions yields no significant differences in estimated measurement error. However, RCAT needs longer test length than CAT to achieve the level of standard error we set. It’s the cost that RCAT pays.

並列關鍵字

item response theory(IRT) ； review ； computerized adaptive testing(CAT)

參考文獻

何榮桂(1997)。遠距測驗—Dear CAT 的設計與實施。物理教育，1(1)，51-62頁。

Baker, F. B. (1985). The basic of item response theory. Portmouth, NH: Heinemann.

Gershon, R., & Bergstrom, B. (1995, April). Does cheating on CAT pay: Not. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff Publishing.

Hambleton, R. K., & Swaminathan, H. & Rogers, H. J. (1991). Fundamentals of item response theory. Sage Publications Inc.