Differential Item Functioning Analyses in Large-Scale Educational Surveys: Key Concepts and Modeling Approaches for Secondary Analysts

大型教育調查研究中的差別試題功能：次級分析中的核心概念及建模方法

朱小姝(Xiao-Shu Zhu) ； 安德魯˙儒普(André A. Rupp) ； 高靜(Jing Gao)

《教育科學研究期刊》 56卷1期 (2011/03) Pp. 91-127

摘要

型教育評量研究常採用多階段抽樣的設計（multi-stage sampling design），透過對母群體之抽樣單位進行分層以抽取受測者。此外，還會採用複雜題本設計（complex booklet design）的方式將題目組成多份測驗題本。在此情況下，欲確保公正測量出不同受測群體的能力，關鍵在於能夠有效偵測所採用的題目是否具差別試題功能（differential item functioning, DIF）。本文旨在介紹探討在大型教育評量複雜設計之下能用以偵測差別試題功能的建模方法，並應用六種可用於偵測DIF的多階層廣義線性模式（hierarchical generalized linear models, HGLMs），再透過電腦模擬比較它們偵測DIF的效力。接著又將這些模式應用到國際數學與科學教育成就趨勢調查研究（TIMSS）的實證數據上，藉以探測是否存在一致性的性別DIF（uniform gender DIF）。

關鍵字

複雜題本設計 ； 差別試題功能 ； 多階層廣義線性模式 ； 多階段抽樣設計通

並列摘要

Many educational surveys employ a multi-stage sampling design for students, which makes use of stratification and/or clustering of population units, as well as a complex booklet design for items from an item pool. In these surveys, the reliable detection of item bias or differential item functioning (DIF) across student groups is a key component for ensuring fair representations of different student groups. In this paper, we describe several modeling approaches that can be useful for detecting DIF in educational surveys. We illustrate the key ideas by investigating the performance of six hierarchical generalized linear models (HGLMs) using a small simulation study and by applying them to real data from the Trends in Mathematics and Science Study (TIMSS) study where we use them to investigate potential uniform gender DIF.

並列關鍵字

complex booklet design ； DIF ； HGLMs ； multi-stage sampling design

參考文獻

(2009).(TIMSS&PIRLS International Study Center. (2009). TIMSS 2007 international database and user guide. Retrieved June 12, 2010, from http://timss.bc.edu/TIMSS2007/idb_ug.html).http://timss.bc.edu/TIMSS2007/idb_ug.html

American Educational Research Association,American Psychological Association,National Council on Measurement in Education(1999).Standards for educational an psychological testing.Washington, DC:American Educational Research Association.

Beaton, A. E.(Ed.)(1987).Implementing the new design: The NAEP 1983-84.,::Educational Testing Service.

Binder, D. A.(1983).On the variances of asymptotically normal estimators from complex surveys.International Statistical Review.51(3),279-292.

Binici, S.(2008).Random-effect differential item fuctioning via herarchical generalized linear model and generalized linear latent mixed model: A comparison of estimation methods.Florida State University.