兼顧認知成分分析與難度參數變異的圖形推理試題模型發展策略

An Iterative Revising Approach for Developing the Item Models of Figural Matrix Test

蘇義翔(I-Hsiang Su) ； 洪碧霞(Pi-Hsia Hung)

《數位學習科技期刊》 3卷3期 (2011/07) Pp. 95-108

摘要

當代電腦化適性測驗的發展往往因為試題編製成本居高不下而受到限制，晚近電腦科技與認知科學的進展，讓更富經濟效能的試題複製理論和技術漸臻成熟。然而，試題複製所產生的參數不確定性，自然會造成能力估計的誤差。因此，控制試題參數的不確定性成為自動化命題技術探討的重要課題。本研究以認知成分分析為基礎，針對國小學童圖形推理作業，建立試題構成原則。文中提出19個試題模型，複製產生103個試題，並針對616名國小四、五、六年級學生施測後，以題目反應理論單參數模式估計試題難度參數。整體而言，試題模型內難度變異的平均極小（σ^2=.053），顯示研究中所採用難度參數變異與試題認知成分交互檢核的模型微調策略，可以依據構念內含的認知成分，有效將試題模型內的難度參數變異降低到合理的水準。

關鍵字

參數不確定性 ； 期望反應函數 ； 試題模型 ； 圖形推理測驗 ； 難度參數

並列摘要

Implementation of computerized adaptive testing is limited due to costly expense and time-consuming item production. Taking advantage of the advanced computer technology and cognitive science, more economical and efficient item cloning theories and technologies are evolved into feasibility. However, item parameter uncertainty induced by item cloning causes ability estimate error inevitably. How to control item parameter uncertainty becomes a critical issue on the automated item generation. Based on cognitive component analysis literatures, the present study attempts to establish item generation principles for figural matrix tasks. Nineteen item models are developed to generate 103 isomorphic item variants. The response data of 616 4th to 6th grade students is used for Item Response Theory (IRT) one-parameter model item parameter calibration. Whenever the variance of difficulty parameters within an item model is relatively large, a related component will be identified to establish a new item model. Overall, the average variance of item difficulty parameter within item models is very small (σ^2=.053). The results suggest that the item modeling mechanism accomplish a reliable control for item parameter.

並列關鍵字

parameter uncertainty ； expected response function ； item model ； figure matrix test ； difficulty parameter

參考文獻

俞筱鈞(1994)。瑞文氏圖形推理測驗系列指導手冊。臺北市:中國行為科學社。

Babcock, R. L.(2002).Analysis of age differences in types of errors on the Raven's advanced progressive matrices.Intelligence.30,485-503.

Bejar, I. I.(1996).Generative response modeling: Leveraging the computer as a test delivery medium.Princeton, NJ:Educational Testing Service.

Bejar, I. I.,Lawless, R.,Morley, M. E.,Wagner, M. E.,Bennett, R. E.,Revuelta, J.(2003).A feasibility study of on-the-fly item generation in adaptive testing.The Journal of Technology, Learning, and Assessment.2(3),3-29.

Carpenter, P. A.,Just, M. A.,Shell, P.(1990).What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test.Psychological Review.97,404-431.