建構反應題的認知診斷模型

在線上學習中，如何能快速精準的了解學習者是否已經習得課程中的知識或概念，一直是個重要的問題。建構反應題可提供較為詳細的學習者訊息，但在線上測驗中建置建構反應題的自動計分機制需要耗用較多之人力與時間。因此，如何能合併選擇題及建構反應題進行測驗，以降低建置自動計分機制所需的人力與時間，同時能提升自動計分的精準度，也是待探討的議題。在本研究中，將提出一種應用在自動計分機制情境下的統計模型，用於提高數學建構反應題的自動評分診斷概念之準確性，以及比較不同自動計分方式的成效評估，並以模擬研究探討選擇題與建構反應題有效的組合方式，以實徵研究探討新方法的可行性。研究結果顯示，基於建構反應題的認知診斷模型和自動評分機制能提高診斷學習者概念之精準度，而使用適當題數的選擇題合併建構反應題進行測驗時，則可在不損及估計精準度的狀況下降低測驗的長度及施測所需的時間。

關鍵字

DINA ；自動計分；認知診斷；建構反應題

並列摘要

Multiple choice items are usually used for simple scoring. Constructed response items provide more diagnostic information about learners, however, it is costly and time-consuming to develop constructed response items for automated scoring process. To apply both multiple choices and constructed response items in online assessment would be more efficient. To reach this purpose, an automated scoring method for constructed response items and a statistical model for the assessment with both multiple choice and constructed response items are used in the present study. The study collected both simulated data and empirical data to evaluate the performance of the proposed model. The results show that the proposed cognitive diagnostic model for both multiple choice and constructed response items outperforms those only considering multiple choice items. As well, the efficacy of test development and data analysis was improved.

並列關鍵字

automated scoring ； cognitive diagnosis ； constructed-response item ； DINA

參考文獻

Schuwirth, L. W., & van der Vleuten, C. P. (2011). Programmatic assessment: From assessment of learning to assessment for learning. Med Teach, 33, 478-485. doi: 10.3109/0142159X.2011.565828

Wang, S., Yang, Y., Culpepper, S. A., & Douglas, J. A. (2017). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden markov model with covariates. Journal of Educational and Behavioral Statistics, 43(1), 57-87. doi: 10.3102/1076998617719727

Bejar, I. I. (2011). A validity-based approach to quality control and assurance of automated scoring. Assessment in Education: Principles, Policy & Practice, 18, 319-341. doi: 10.1080/0969594X.2011.555329

Chakraborty, U. K., Konar, D., Roy, S., & Choudhury, S. (2019). Automatic short answer grading using rough concept clusters. International Journal of Advanced Intelligence Paradigms, 14(3/4), 260-280. doi: 10.1504/IJAIP.2019.103413

Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272. doi:10.1177/01466210122032064

國際替代計量

建構反應題的認知診斷模型

全文下載

主題瀏覽