Evaluation of Mean and Covariance Structure Analysis Model in Detecting Differential Item Functioning of Polytomous Items: in Comparison with GMH, and Poly-SIBTEST

Evaluation of Mean and Covariance Structure Analysis Model in Detecting Differential Item Functioning of Polytomous Items: in Comparison with GMH, and Poly-SIBTEST

柯明錦(Ming-Jin Ke)

指導教授 ： 蔡蓉青教授

國立臺灣師範大學/理學院/數學系在職進修碩士班/碩士(2010年)

摘要

無資料

關鍵字

DIF ； MIMIC ； MACS ； DIFFTEST ； GMH ； Poly-SIBTEST

並列摘要

Item bias is a well-known and important issue in educational testing and therefore the investigation of Differential Item Functioning (DIF) has long been proven valuable in evaluating the fairness or quality of an item. In this thesis, a simulation study was conducted for polytomous responses to evaluate the efficacy of using the multiple-group Structural Equation Model (SEM) to detect DIF, in comparison with two nonparametric DIF indices, Poly-SIBTEST and Generalized Mantel-Haenzel (GMH) methods. The multiple group SEM model was used to generate data with five ordered response categories. Items exhibiting DIF were modified by allowing their threshold parameters to differ between the focal and reference groups. Five factors were manipulated: three test lengths (10, 20, 30), three sample sizes (500, 1000, and 3000), two ability distributions, four percentages of DIF items (0%, 10%, 20%, and 30%), and four different size proportions of the focal and reference groups (80/20, 70/30, 60/40, and 50/50), to produce all the conditions of the data sets. Each condition was replicated 100 times to facilitate Type I error and Power calculations of the three detection procedures under consideration.
Our results suggested that the DIFFTEST approach under MG-MACS to DIF detection had the smallest overall Type I error rate and comparable or higher Power than GMH and Poly-SIBTEST in all conditions. When polytomous data were generated under the MG-MACS model, the DIFFTEST procedure was viable for DIF detection even for tests with as few as ten items. In conclusion, DIFFTEST performed the best for detecting DIF in polytomous items while comparing to GMH and Poly-SIBTEST, for yielding lower Type I error rates and higher Power for different types of DIF conditions.

參考文獻

Tsai, L.T., Yang, C.C., Wang, W.C., & Shih. C.L. (2008). A simulation study on using MIMIC model to assess the accuracy of differential item functioning (in Chinese), Psychological Testing, 55, 287-312.

Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.

Bolt, D. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15, 113-141.

Camilli, G., & Shepard, L. A. (1994). Methods for Identifying Biased Test Items. Sage:Thousand Oaks.

Chang, H. H. & Mazzeo, J.(1994). The unique correspondence of the item response function and item category response functions in polytomously scored item response models. Psychometrika, 59, 391-404.