試題作答時間在電腦適性測驗上的應用－改善具洩題資訊考生的能力估計精確性

Using Response Times to Improve Measurement Precision for Examinees with Item Pre-knowledge in Computerized Adaptive Testing

陳俊宏(Jyun-Hong Chen) ； 陳淑英(Shu-Ying Chen)

《測驗學刊》 57卷4期 (2010/12) Pp. 459-483

https://doi.org/10.7108/PT.201012.0459

摘要

當測驗的試題外洩時，考生的能力將被不當高估，為了改善擁有洩題資訊考生的能力估計精確性，McLeod、Lewis和Thissen(2003)提出試題分享反應理論(SIRT)。在SIRT中，考生正確作答一道試題的機率，是由考生的能力與試題被記憶機率－p(m)，所共同決定。SIRT因考量p(m)對作答的影響，故在洩題發生時，其能力估計精確性較傳統IRT佳。雖然如此，SIRT的成效取決於p(m)是否準確估計，準確估計p(m)是SIRT在執行上的重要課題。
試題作答時間(RT)應能提升p(m)的估計準確性。當考生擁有洩題資訊時，除了能夠正確作答試題外，其作答時間亦較為快速。因此，在p(m)估計上，RT應可提供重要訊息。本研究嘗試利用RT來估計p(m)，再根據SIRT以改善具洩題資訊考生的能力估計精確度。研究結果顯示，p(m)可以透過RT準確估計，具洩題資訊考生能力不當高估的情形可透過SIRT獲得明顯改善。

關鍵字

洩題偵測 ； 測驗安全 ； 試題作答時間 ； 電腦適性測驗

並列摘要

Examinees' scores would be seriously inflated when item sharing occurs. To improve the precision of ability estimate for examinees with item pre-knowledge, sharing item response theory (SIRT) was proposed. Based on SIRT, the probability that an examinee answers an item correctly depends not only on the examinee's ability level, but also on the probability that the item has been memorized, p (m). Even though it is not trivial to have p m accurately estimated, response times could be used to improve the accuracy of the estimate of p (m). When examinees have item pre-knowledge, they would answer items not only correctly but also quickly. Thus, to identify examinees with item pre-knowledge effectively and to estimate p m accurately, response times as well as item responses should be considered. The purpose of this study is to improve measurement precision for examinees with item pre-knowledge by using SIRT, where p (m) is estimated based on a procedure proposed by van der Linden & van Krimpen-Stoop (2003) with consideration of response times. Results indicated that examinees with item pre-knowledge were effectively detected and the precision of their ability estimates was significantly improved when response times were taken into account in estimating p (m).

並列關鍵字

computerized adaptive testing ； item sharing detection ； response times ； test security

參考文獻

陳俊宏(2007)。電腦適性測驗中因應共謀行為的模型探討(碩士論文)。國立中正大學心理學研究所。

Barrada, J. R.,Olea, J,Ponsoda, V.,Abad, F. J.(2006).Estrategias de seleccion de items en un Test Adaptativo Informatizado para la evaluacio'n de ingles escrito.Psicothema.18,828-834.

(Chen, S. Y. (2008). Investigating response times for examinees with item pre-knowledge in computer based testing. Manuscript in preparation.).

McLeod, L.,Lewis, C.,Thissen, D.(2003).A bayesian method for the detection of item preknowledge in computerized adaptive testing.Applied Psychological Measurement.27(2),121-137.

van der Linden, W. J.(ed.),Hambleton, R. K.(ed.)(1997).Handbook of modern item response theory.New York:Springer.