局部試題依賴偵測方法之偵測效果比較

Compare the Detection Result by Using Different Local Item Dependent Detection Methods

陶君浩(Chun-Hao Tao)

指導教授 ： 陳柏熹

國立臺灣師範大學/教育學院/教育心理與輔導學系/碩士(2013年)

摘要

本研究旨在比較Rasch題組模式之題組效果估計、題組-殘差主成分分析及Q3指標等局部試題依賴偵測方法的偵測效果，包含兩個子研究。研究一為模擬研究，主要是操弄不同的題組效果高低及不同的題組內試題數，模擬了500名與1500名受測者在不同題組情境下的作答反應，再利用Rasch題組模式之題組效果估計、題組-殘差主成分分析及Q3指標分別對各個題組進行局部試題依賴的偵測，最後透過題組效果的參數回覆情形、題組-殘差主成分分析及Q3指標的偵測結果以及各偵測方法之偵測結果與題組效果真值的Spearman's ρ係數來瞭解各種局部試題依賴偵測方法之偵測效果的差異情形。而在研究二的實徵研究部分，研究者則是以93～98年國民中學基本學力測驗英文科的題組資料為例，進行上述三種局部試題依賴偵測方法之偵測結果的比較。
主要發現如下：
一、隨著受測者人數及題組內試題數的增加，題組效果參數的回覆情形會逐漸變好，即Rasch題組模式的題組效果估計會越來越準確；但在高題組效果時，其估計的精準度卻相對較差。
二、不論是在何種情境之下，Q3指標的偵測效果均一致地優於其他兩個偵測方法的偵測效果。
三、對於93～98年國民中學基本學力測驗英文科的題組而言，不同方法的偵測結果有差異，其中又Rasch題組模式之題組效果估計的偵測結果與其他兩者差異最大。
四、根據Q3指標的偵測結果，93～98年國中基測英文科的題組大致沒有局部試題依賴的情形，僅93-2-3、93-2-5、93-2-6、94-1-8及97-2-3這幾個題組可能是存有局部試題依賴的問題。

關鍵字

題組 ； 局部試題依賴 ； 題組效果

並列摘要

The purpose of this research is to compare the detection result by using testlet effect estimates of the Rasch testlet model, testlet-residual based principal component analysis and the Q3 statistics. The research is composed of two sub-researches. Study 1 is a simulation study. In study 1, first, testlet effects (high/ low), sample sizes (500/1500) and the item numbers within testlet (2/4/6/8) were manipulated. Testlet effect estimates of the Rasch testlet model, testlet-residual based principal component analysis and the Q3 statistics were used to detected local item dependent for each testlet. The parameters recovery of testlet effect, the detection result of testlet-residual based principal component analysis and the Q3 statistics, and the Spearman's ρ coefficient of local item dependent detection result with the true value of testlet effect were used to compare the detection result of different local item dependent detection methods. Study 2 is an empirical study. These three local item dependent detection methods were compared and applied to the data from the English subject of Basic Competence Test for Junior High School Students(2004～2009).
The main results are the following：
1.As sample sizes and the item numbers in each testlet were increased, the parameters recovery of testlet effect decreased to an acceptable level. It means the testlet effect estimates of the Rasch testlet model will be more and more accurate when the sample sizes or item numbers increases. However, in the condition of high testlet effect, the estimated accuracy of testlet effect were decreased instead.
2.No matter in what situations, the detection result of the Q3 statistics was better than the other two detection methods.
3.The detection results have varied between these three local item dependent detection methods on the english subject of Basic Competence Test for Junior High School Students(2004～2009), testlet effect estimates of the Rasch testlet model especially.
4.According to the detection result of the Q3 statistics, there were no local item dependent for each testlet on the english subject of Basic Competence Test for Junior High School Students(2004～2009), except for 93-2-3, 93-2-5, 93-2-6, 94-1-8, and 97-2-3 .

並列關鍵字

testlet ； local item dependent ； testlet effect

參考文獻

陳柏熹、黃宏宇、王文中（2008）。題組之相關特性對電腦化適性測驗測量精準度的影響。測驗學刊，55（1），129-150。

Andrich, D. (1985). A latent trait model for items with response dependencies: implications for test construction and analysis. In S. Embretson (Ed.), Test design: contributions from psychology, education, and psychometrics (pp. 245-273) . New York, NY: Academic Press.

Bradlow, E.T., Wainer, H., & Wang, X. (1999). A Bayesian random effects models for testlet. Psychometrika, 64(2), 153-168.

Chen, W. & Thissen, D. (1997). Local dependence indices for item pairs using item response theory. Educational and Behavioral Statistics, 22, 265-289.

Chou, Y.-T. & Wang, W. (2010). Checking Dimensionality in Item Response Models With Principal Component Analysis on Standardized Residuals. Educational and Psychological Measurement,70(5), 717-731.