國泰綜合醫院試辦High-Stakes OSCE從測驗結果探討試題良莠的測試方案

Score Analysis of High-Stakes OSCE to Develop Strategies for Station Development Experience at Cathay General Hospital, Taiwan

蔡明霖(Ming-Lin Tsai) ； 連恆煇(Heng-Hui Lien) ； 張少樺(Shao-Hua Chang) ； 林涵柔(Han-Jou Lin) ； 劉榮森(Jung-Sen Liu) ； 洪焜隆(Kun-Long Hung)

《輔仁醫學期刊》 10卷3期 (2012/09) Pp. 141-149

摘要

目的：從一次標準化的high-stakes OSCE考生成績分析來回顧考題發展過程中試題信效度、難易度、鑑別度、及格標準設定的優缺點並期望能改善往後OSCE試題的發展流程。方法：國泰醫院在民國100年12月24日根據台灣醫學教育學會公告的high-stake OSCE標準舉行了一場七年級實習醫學生的測驗，測驗進行順利，考生的成績除了以紙本回饋作為日後改進的參考外，詳細的資料經一一審視分析包括每位考生每站評分表得分（checklist）、整體評分（global rating）、考官及SP的建議、各考題之間的相關性統計、考試過程的錄影回顧，來檢討每一考題之測試效果。結果：從考生的成績來看此次考題的設置：1. 及格標準設定普遍太低。2. 有某些考題的低鑑別度可反應出是由於考官評分太過於寬鬆。3. 在同一類型的考題如理學檢查或問診題群中，若某題分數出異常相關性，很可能是考題本身因素需再加修改。4. 在OSCE中，PE、問診題彼此皆呈高度相關，此類題目大致可歸納為醫病互動，但另一題群- 操作技術（procedure skills）4 題，彼此間皆無相關，且每題和總分也呈低相關，其原因需進一步詳加探討。結論：1. High-stakes OSCE能有效測試考生臨床技術能力。2. 根據同一標準下進行的high-stakes OSCE的成果分析可有效再檢測考題的信效度。3. 技術性考站的發展需從更廣泛的面向加以思考。4. 配題藍圖中對於考生鑑別診斷的規範需再加強。

關鍵字

OSCE ； 臨床技術

並列摘要

Background and Purpose: Through a detailed analysis of results from an index of reliability, validity, difficulty, and discrimination of a standard high-stakes Objective Structured Clinical Examination (OSCE), we wanted to establish standard protocols for station development. Methods: A high-stakes OSCE was performed on December 24, 2011 for interns at Cathay General Hospital, Taiwan according to updated standards of the Taiwan Association of Medical Education. The OSCE was smoothly performed, and scores and suggestions by the rater for each student were directly fed back to the students for formative purposes. All scores including a global rating and scores from the checklist were analyzed in detail, including the score of the checklist and global rating at each station for every student, suggestions by raters and standardized patients (SPs), the associated statistics between each station, and a video recording of the examination process to evaluate the effects of each station. Results: The results of this high-stakes OSCE test were as follows: 1. the preset baseline for passing the test was too low; 2. the low discriminative ability of some stations was possibly caused by measurement errors due to leniency by the raters; 3. in the same type of stations like physical examination or history taking, high associations between different stations meant that the contents of these stations needed to be corrected; and 4. in the high-stakes OSCE, a high association was noted between the physical examination and history taking stations which both belong to the same type: communication between doctors and patients. However, low associations among these 4 stations of procedure skills need to be further discussed and studied. Conclusions: 1. The high-stakes OSCE can effectively evaluate procedure skills; 2. an analysis of the high-stakes OSCE can check on the reliability and validity of every station; 3. development of procedure skills stations needs to be further and more broadly evaluated; and 4. the method for discriminating student abilities needs more discussion.

並列關鍵字

OSCE ； clinical skills