An Innovative Approach to Support the Reliabilities of Case-based Discussion Assessment

Heng-Hui Lien ； Tien-Shang Huang ； Yu-Chun Chiu ； Huey-Ling Chen ； Ming-Been Lee ； Yen-Yuan Chen

《Journal of Medical Education》 24卷4期 (2020/12) Pp. 173-182

https://doi.org/10.6145/jme.202012_24(4).0003

摘要

Purpose: To convincingly examine the reliabilities of Case-based Discussion (CbD), we developed three videotaped clinical scenarios played by standardized patients and standardized residents. The objective of this study was to examine the test-retest reliability, inter-rater reliability and internal consistency reliability of CbD to assess medical trainees' clinical reasoning and decision-making. Methods: We developed three clinical scenarios and videotaped as CbD I, CbD II, and CbD III for guaranteeing that all assessors rated exactly the same scenario. We purposefully recruited the assessors with fruitful experiences of assessing medical trainees' clinical reasoning and decision-making from four governmental and private university-affiliated medical centers, and two Veterans Affairs Council-affiliated medical centers. Each assessor was required: (1) to watch the videotaped discussions in consecutive order starting from CbD I in a single rating session; and (2) to assess each resident's clinical reasoning and decision-making in each discussion. We estimated the test-retest reliability, interrater reliability, and internal consistency reliability of CbD using Pearson's correlation coefficients (PCC), intra-class correlation coefficient (ICC), and internal consistency reliability (Cronbach's alpha), respectively. Results: We found that: (1) an assessor gave similar rating results in face of the same case-based discussion as indicated by acceptable test-retest reliability (PCC = .70~.78, P ＜ .01); (2) assessors gave similar rating results to a resident's clinical reasoning and decision-making as indicated by a good to excellent interrater reliability (PCC = .73~.88, P ＜ .01); and (3) all the items of CbD reflected uni-dimensionally a construct-a resident's clinical reasoning and decision-making as supported by a much better internal consistency reliability (Cronbach's alpha = .92~.95). Conclusions: The study results convincingly showed that CbD is a reliable assessment tool for assessing a medical trainee's or medical learner's clinical reasoning and decision-making, and can be widely used in clinical encounters to assess clinical reasoning and decision-making.

關鍵字

case-based discussion ； reliability

參考文獻

Van Der Vleuten CP: The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1996;1(1):41-67; DOI: 10.1007/BF00596229.

Bullock A, Barnes E, Ryan B, et al. : Case-based discussion supporting learning and practice in optometry. Ophthalmic Physiol Opt. 2014;34(5):614-21; DOI: 10.1111/opo.12151.

Couto TB, Farhat SC, Geis GL, et al.: High-fidelity simulation versus case-based discussion for teaching medical students in Brazil about pediatric emergencies. Clinics (Sao Paulo). 2015;70(6):393-9. ; DOI: 10.6061/clinics/2015(06)02.

Mohanaruban A, Flanders L, Rees H: Case-based discussion: perceptions of feedback. Clin Teach. 2018;15(2):126-31; DOI: 10.1111/tct.12654.

Mehta F, Brown J, Shaw NJ.: Do trainees value feedback in case-based discussion assessments? Med Teach. 2013;35(5):e1166-72; DOI: 10.3109/0142159X.2012.731100.