- 學位論文
兩階段馬可夫鏈模式與Rasch模式於健康專業認證筆試之分析與比較：以麻醉專科考試為例
Two-state Markov Chain Model and Rasch Model Applied to Certification Examinations in Health Professionals: An Illustration in Anesthesiology
摘要
背景 常規醫事人員專業認證筆試之目的在於評估考生是否具備執業能力之基礎，其測驗結果同時受到考生能力與題目難易程度所影響，因此，基於預判頻率理論，具備同時評估考生能力與題目難易程度之Rasch模式於筆試資料應用愈趨廣泛。 然而，具次序性之筆試資料並不適用Rasch模式，因此，於本論文將結合兩階段馬可夫鏈模式(two-state Markov chain)與Rasch模式，並以實證資料加以說明。 目的 本論文目的在於展示如何於Rasch模式中導入兩階段馬可夫鏈模式(two-state Markov chain)用以處理次序型筆試資料，並提出兩模式於個別考生能力與試題難易程度之相關應用理論。 方法 本論文以2007至2016年臺灣麻醉專科醫師筆試資料進行實證，兩階段馬可夫鏈模式將用於「對到錯」(Pcw)與「錯到對」(Pwc)兩階段之條件機率評估。其結果亦與Rasch模式所得結果進行比較，於兩階段馬可夫迴歸模式中拓展其隨機效應(random effect)，用以評估兩階段中共變數(如年齡、性別及受訓醫院所屬區域等)之影響。本研究將以貝氏馬可夫鏈蒙地卡羅方法(Bayesian Markov Cain Monte Carlo method)進行參數估計，其SAS程式碼(Proc MCMC)亦呈現於本論文中。 結果 本論文利用兩階段馬可夫鏈模式分析臺灣地區2007-2016十年間麻醉專科筆試資料，Pwc及Pcw兩個條件機率的估計結果分別為0.6957 (95% CI:0.6860-0.7044)及0.2326 (95% CI:0.2276-0.2374)，由此可得該考試考生長程答對機率為74.95% (95% CI:74.49%-75.42%)，接近Rasch模式得到的73.50%。考生能力指數估計結果為1.09 [=log (Pwc/Pcw)]，也與Rasch模式得到的1.02接近。此外，拓展兩階段馬可夫模式可進一步考慮考生特性與隨機效應，本資料顯示考生性別與年齡與考生能力有關：男性顯著低於女性、年齡長者分數較低。而混合效應模式證明考題難度的異質性(隨機效應在Pwc及Pcw的標準差分別為1.6781及 1.8416)高於考生的異質性(隨機效應在Pwc及Pcw的標準差分別為0.4399及 0.4875)。在未考慮隨機效益之下，二階馬可夫鏈(DIC=41935.14)的模式配適較一階馬可夫鏈(DIC=42872.54)佳，但在一階馬可夫鏈模式加入考生能力與考題難度的隨機效應後，其模式配適則優於二階馬可夫鏈模式。 結論 本論文利用兩階段馬可夫鏈模式評估麻醉專科筆試資料，並證明其分析結果與傳統Rasch模式分析方法之間的連結性。在未來數位化時代，兩階段馬可夫鏈模式可作為專業認證考試發展序列性適性考試之評估工具。
並列摘要
Background Conventional written examination for accrediting health professionals is often based on pre-determined frequentist type with the Rasch model that models the corrected probability of answering the question as a function of individual ability and the difficulty of item. However, using the Rasch model may not be appropriate for sequential type written test. The two-state Markov chain (MC) method is therefore proposed. The link between it and the Rasch model is postulated and illustrated with one empirical data. Aims The specific aims of this thesis are to demonstrate how to apply two-state MC to modelling sequential type written test and to propose the theorem of linking two-state MC model with the Rasch model in terms of individual ability and the difficulty of item. Methods The empirical data used for illustration are derived from Taiwanese board certification examination in anaesthesiology from 2007 to 2016. The proposed two-state (wrong and correct) MC model was applied to estimating two parameters, i.e. two conditional probabilities (Pwc (wrongcorrect) and Pcw (correct wrong)). The results of two-state MC were compared with those based on the Rasch model. Two-state MC regression model with random effect was extended to assess the effect of relevant covariates (such as age, gender, and region) on two parameters of two-state MC model. Bayesian Markov Cain Monte Carlo (MCMC) method was used to estimate the parameters of interest for the Rasch model and two-state MC model. Computer algorithms with SAS Proc MCMC were also developed in this thesis. Results Based on the estimated 0.6957 (95% CI:0.6860-0.7044) of Pwc and 0.2326 (95% CI:0.2276-0.2374) of Pcw using the overall empirical data, the long-run corrected probability (74.95% (95% CI:74.49%-75.42%)) derived from the equilibrium distribution of two-state MC model was close to the estimated 73.50% based on the Rasch model. The overall estimated log (Pwc/Pcw) (1.09) was very close to the estimated individual ability (1.02 logit scale) based on the Rasch model. The application of two-state MC regression model to Taiwanese board certification examination in anaesthesiology from 2007 to 2016 based on sequential type test found the following relationships after adjustment for the relevant covariates with each other and also random effect of individual ability and item difficulty: age was inversely associated the corrected probability; females were more likely to have correct answer than males; the corrected probability varied by region. The heterogeneity of item difficulty (sigma=1.6781 and 1.8416 for Pwc and Pcw, respectively) was greater than that of individual ability (sigma=0.4399 and 0.4875 for Pwc and Pcw, respectively). Without considering covariates with random effects, the second-order Markov model (DIC=41935.14) had a better performance compared with the first-order Markov model (DIC=42872.54). However, the consideration of variation of individual ability and item difficulty with two random effect using the mixed Markov regression model outnumbered second-order Markov model. Conclusions This thesis demonstrates alternative use of two-state Markov chain model for the assessment of sequential type written test. The findings support the theorem of linking two-state MC model with the Rasch model. The proposed two-state MC model and its regression one may provide a new insight into the development of a prototype of adaptive sequential test that is very useful for certification of health professionals in era of digital age.
參考文獻
延伸閱讀
- 張光宜、潘瑛如（2021）。麻醉專科醫師訓練里程碑計畫成效評量暨甄審筆試測驗分析。台灣教育研究期刊，2(2)，209-241。https://www.airitilibrary.com/Article/Detail?DocID=P20220316001-202103-202203240015-202203240015-209-241
- 杜慧真（2018）。術前麻醉評估門診流程改善之行動研究 -以中部某醫學中心為例〔碩士論文，中臺科技大學〕。華藝線上圖書館。https://www.airitilibrary.com/Article/Detail?DocID=U0099-2705202017053275
- 魏令芳（2016）。麻醉學領域與麻醉部門之書目計量與研究前沿比較分析〔博士論文，國立臺灣大學〕。華藝線上圖書館。https://doi.org/10.6342/NTU201602279
- 陳偉德、林正介、黃秋錦、蔡崇豪、蔡長海（2006）。建置迷你臨床演練評量(mini-CEX)之經驗與試辦成果。Journal of Medical Education，10(3)，232-239。https://doi.org/10.6145/jme.200609_10(3).0007
- 許天維、曾建維、梁榮進、王柏婷、永井正武（2012）。The Use of Rasch Model GSP Chart and Grey Structural Model Analysis for Vocational Education and Training Courses: Taking Enterprise Ethics and Professional Ethics Courses as an Example。教育研究與發展期刊，8(4)，53-79。https://doi.org/10.6925/SCJ.201212_8(4).0003