Strides in Development of Medical Education

Strides in Development of Medical Education

Current Issue

By Issue

By Author

By Subject

Author Index

Keyword Index

About Journal

Aims and Scope

Indexing and Abstracting

Journal Team

Editorial Policy

Peer Review Process

Publication Ethics

Journal Metrics

Review of Checklist and Global Rating Form Scoring Methods in Objective Structured Clinical Examination Stations: A Narrative Review

Document Type : Original Article

Authors

1 Health Services Management Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran

2 Department of Medical Education, Education Development Center, Kerman University of Medical Sciences, Kerman, Iran

3 Department of Midwifery, School of Nursing and Midwifery, Torbat Heydariyeh University of Medical Sciences, Torbat Heydariyeh, Iran

Abstract

Background: The objective structured clinical examination (OSCE) is broadly applied to assess the clinical competence of medical students. Despite the widespread use of the OSCE, the scoring methods used in this test can be a potential source of measurement error and affect the accuracy of the scores.
Objectives: This study was aimed to investigate the checklist and global rating form scoring methods in OSCE stations.
Methods: In this narrative review study, the keywords “OSCE”, “Reliability”, “Validity”, “Utility”, “Global”, and “Checklist” were searched in Scopus, Web of Science, EMBASE, and Medline databases between 2010 and March 2021. After removing duplicates and considering the direct relationship of the articles to the study’s objectives, full-text articles were reviewed.
Results: In total , 30 articles were retrived, of which 19 cases were finalized. The results of retrived articles were divided into five categories, including the correlation of scores, subjectivity and objectivity, validity and reliability, ease of use, and the need to train assessors, and the assessed capabilities in scoring by checklists and global rating forms.
Conclusion: There is widespread disagreement on the superiority of checklist over global rating form. It is recommended that both scoring methods be used in combination to achieve maximum reliability and validity, to assess student skills based on objective criteria, to avoid applying assessor mentality, and to gain maximum validity in test results.

Keywords

References
  1. Halman S, Fu AY, Pugh D. Entrustment within an objective structured clinical examination (OSCE) progress test: Bridging the gap towards competency-based medical education. Med Teach. 2020 Nov;42(11):1283-1288. doi: 10.1080/0142159X.2020.1803251. [PMID: 32805146].
  2. Alshammari E. Implementing eOSCE during COVID-19 lockdown. Journal of Advanced Pharmacy Education & Research. Jan-Mar. 2020;10(1):175.
  3. Lee YH, Lin SC, Wang PY, Lin MH. Objective structural clinical examination for evaluating learning efficacy of Cultural Competence Cultivation Programme for nurses. BMC Nurs. 2020 Dec 1;19(1):114. doi: 10.1186/s12912-020-00500-3. [PMID: 33292191]. [PMCID: PMC7709429].
  4. Daniels VJ, Pugh D. Twelve tips for developing an OSCE that measures what you want. Med Teach. 2018 Dec;40(12):1208-1213. doi: 10.1080/0142159X.2017.1390214. [PMID: 29069965].
  5. Homer M, Fuller R, Hallam J, Pell G. Shining a spotlight on scoring in the OSCE: Checklists and item weighting. Med Teach. 2020 Sep;42(9):1037-1042. doi: 10.1080/0142159X.2020.1781072. [PMID: 32608303].
  6. Kristina SA, Wijoyo Y. Assessment of Pharmacy Students' Clinical Skills using Objective Structured Clinical Examination (OSCE): A Literature Review. Systematic Reviews in Pharmacy. 2019;10(1):55-60. doi:10.5530/srp.2019.1.9.
  7. Onwudiegwu U. Osce: Design, Development and Deployment. J West Afr Coll Surg. Jan-Mar 2018;8(1):1-22. [PMID: 30899701]. [PMCID: PMC6398515].
  8. Govaerts MJ, van der Vleuten CP, Holmboe ES. Managing tensions in assessment: moving beyond either–or thinking. Med Educ. 2019 Jan;53(1):64-75. doi: 10.1111/medu.13656. [PMID: 30289171]. [PMCID: PMC6586064].
  9. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018; 18: 313. doi: 10.1186/s12909-018-1421-x. [PMCID: PMC6302424]. [PMID: 30572876].
  10. Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015 Dec 1;24(4):230-5. doi:10.1179/2047480615Z.000000000329.
  11. Sim JH, Aziz YF, Vijayanantha A, Mansor A, Vadivelu J, Hassan H. A closer look at checklist scoring and global rating for four OSCE stations: Do the scores correlate well?. Education in Medicine Journal. 2015 Jun 1;7(2): 39-44. doi:5959/eimj.v7i2.341.
  12. Wan M, Canalese R, Lam L, Petersen R, Quinlivan J, Frost G. Comparison of criterion-based checklist scoring and global rating scales for the Objective Structured Clinical Examination (OSCE) in pre-clinical year medical students. Med Educ. 2011; 45 (Suppl. 3): 1–15.
  13. Turner K, Bell M, Bays L, Lau C, Lai C, Kendzerska T, Evans C, Davies R. Correlation between global rating scale and specific checklist scores for professional behaviour of physical therapy students in practical examinations. Education Research International. 2014 Jan 1;2014. doi:10.1155/2014/219512.
  14. Inayah AT, Anwer LA, Shareef MA, Nurhussen A, Alkabbani HM, Alzahrani AA, et al. Objectivity in subjectivity: do students’ self and peer assessments correlate with examiners’ subjective and objective assessment in clinical skills? A prospective study. BMJ Open. 2017 May 9;7(5):e012289. doi: 10.1136/bmjopen-2016-012289. [PMID: 28487454]. [PMCID: PMC5623435].
  15. CarlLee S, Rowat J, Suneja M. Assessing entrust able professional activities using an orientation OSCE: identifying the gaps. J Grad Med Educ. 2019 Apr;11(2):214-220. doi: 10.4300/JGME-D-18-00601.2. [PMID: 31024656]. [PMCID: PMC6476083].
  16. Setyonugroho W, Kennedy KM, Kropmans TJ. Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: a systematic review. Patient Educ Couns. 2015 Jun 27;S0738-3991(15)00277-3. [PMID: 26149966].
  17. Massey D, Byrne J, Higgins N, Weeks B, Shuker MA, Coyne E, et al. Enhancing OSCE preparedness with video exemplars in undergraduate nursing students. A mixed method study. Nurse Educ Today. 2017 Jul;54:56-61. doi: 10.1016/j.nedt.2017.02.024. [PMID: 28477564].
  18. Rawlings MA, Johnson BJ. Reliability and Validity of OSCE for Assessment of BSW Competency: Lessons Learned. Nurse Educ Today. 2017 Jul;54:56-61. doi: 10.1016/j.nedt.2017.02.024. [PMID: 28477564].
  19. Majumder MA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. doi: 10.2147/AMEP.S197275. eCollection 2019. [PMID: 31239801]. [PMCID: PMC6556562].
  20. Huang HP, Chao LF, Wang YH, Ni YM, Jane SW. The establishment and examination of the reliability, validity, discrimination, and difficulty of nursing objective structured clinical examination (OSCE). Hu Li Za Zhi. 2017 Dec;64(6):67-76. doi: 10.6224/JN.000084. [PMID: 29164548].
  21. Khan AS, Shabi-ul-Hassan S. Comparison of assessment scores of candidates for communication skills in an OSCE, by examiners, candidates and simulated patients. Creative Education. 2012 Oct 23;3(06):931. doi:10.4236/ce.2012.326141.
  22. Sturpe DA, Huynh D, Haines ST. Scoring objective structured clinical examinations using video monitors or video recordings. Am J Pharm Educ. 2010 Apr 12;74(3):44. doi: 10.5688/aj740344. [PMID: 20498737]. [PMCID: PMC2865410].
  23. Palter VN, Orzech N, Reznick RK, Grantcharov TP. Validation of a structured training and assessment curriculum for technical skill acquisition in minimally invasive surgery: a randomized controlled trial. Ann Surg. 2013 Feb; 257(2): 224-30. doi: 10.1097/SLA.0b013e31827051cd. [PMID: 23013806].
  24. Gillis ME, Scott SA, Richardson CG, Oxner WM, Gauthier L, Wilson DA, Glennie RA. Developing and Assessing the Feasibility of Implementing a Surgical Objective Structured Clinical Skills Examination (S-OSCE). J Surg Educ. Jul-Aug 2020;77(4):939-946. doi: 10.1016/j.jsurg.2020.02.018. [PMID: 32179030].
  25. Muthusami A, Mohsina S, Sureshkumar S, Anandhi A, Elamurugan TP, Srinivasan K, et al. Efficacy and feasibility of Objective Structured Clinical Examination in the internal assessment for surgery postgraduates. J Surg Educ. May-Jun 2017;74(3):398-405. doi: 10.1016/j.jsurg.2016.11.004. [PMID: 27913082].
  26. Macluskey M, Durham J, Balmer C, Bell A, Cowpe J, Dawson L, et al. Dental student suturing skills: a multicentre trial of a checklist‐based assessment. Eur J Dent Educ. 2011 Nov;15(4):244-9. doi: 10.1111/j.1600-0579.2010.00665.x. [PMID: 21985209].
  27. Elshama SS. How to Design and Apply an Objective Structured Clinical Examination (OSCE) in Medical Education?. Iberoamerican Journal of Medicine. 2020 Nov;3(1): 51-5.
  28. Patrício MF, Julião M, Fareleira F, Carneiro AV. Is the OSCE a feasible tool to assess competencies in undergraduate medical education?. Med Teach. 2013 Jun;35(6):503-14. doi: 10.3109/0142159X.2013.774330. [PMID: 23521582].
  29. Solà-Pola M, Morin-Fraile V, Fabrellas-Padrés N, Raurell-Torreda M, Guanter-Peris L, Guix-Comellas E, Pulpón-Segura AM. The usefulness and acceptance of the OSCE in nursing schools. Nurse Educ Pract. 2020 Jan 31;43:102736. doi: 10.1016/j.nepr.2020.102736. [PMID: 32058920].
  30. Ma IW, Zalunardo N, Pachev G, Beran T, Brown M, Hatala R, McLaughlin K. Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation. Adv Health Sci Educ Theory Pract. 2012 Oct;17(4):457-70. doi: 10.1007/s10459-011-9322-3. [PMID: 21877217].
  31. Piumatti G, Cerutti B, Perron NJ. Assessing communication skills during OSCE: need for integrated psychometric approaches. BMC Med Educ. 2021 Feb 16;21(1):106. doi: 10.1186/s12909-021-02552-8. [PMID: 33593345]. [PMCID: PMC7887794].
  32. Monti M, Klöckner-Cronauer C, Hautz SC, Schnabel KP, Breckwoldt J, Junod-Perron N, et al. Improving the assessment of communication competencies in a national licensing OSCE: lessons learned from an experts’ symposium. BMC Med Educ. 2020 May 26;20(1):171. doi: 10.1186/s12909-020-02079-4. [PMID: 32456675]. [PMCID: PMC7249637].
  33. Hopwood J, Myers G, Sturrock A. Twelve tips for conducting a virtual OSCE. Med Teach. 2020 Oct 20;1-4. doi: 10.1080/0142159X.2020.1830961. [PMID: 33078984].
  34. Donohoe CL, Reilly F, Donnelly S, Cahill RA. Is There Variability in Scoring of Student Surgical OSCE Performance Based on Examiner Experience and Expertise?. J Surg Educ. Sep-Oct 2020;77(5):1202-1210. doi: 10.1016/j.jsurg.2020.03.009. [PMID: 32336628].
  35. Dias EP, Ferreira MA. Twelve tips for essential aspects in the elaboration of Global Medical Curricula in a context of Medical Competencies, EPAs, and Milestones. MedEdPublish. 2021 Feb 8;10. doi:10.15694/mep.2021.000039.1.
  36. Hodges B, McIlroy JH. Analytic global OSCE ratings are sensitive to level of training. Med Educ. 2003 Nov;37(11):1012-6. doi: 10.1046/j.1365-2923.2003.01674.x. [PMID: 14629415].
  37. Ilgen JS, Ma IW, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation‐based assessment. Med Educ. 2015 Feb;49(2):161-73. doi: 10.1111/medu.12621. [PMID: 25626747].
  38. Salajegheh M, Gandomkar R, Mirzazadeh A, Sandars J. Identification of capacity development indicators for faculty development programs: A nominal group technique study. BMC Med Educ. 2020 May 24;20(1):163. doi: 10.1186/s12909-020-02068-7. [PMID: 32448229]. [PMCID: PMC7245937].
  39. Salajegheh M, Sandars J, Norouzi A, Mirzazadeh A, Gandomkar R. Psychometric evaluation of a questionnaire to evaluate organizational capacity development for faculty development programs. J Educ Health Promot. 2020 Sep 28;9:233. doi: 10.4103/jehp.jehp_292_20. [PMID: 33209925]. [PMCID: PMC7652081].
  40. Zimmermann P, Kadmon M. Standardized Examinees: Development of a new tool to evaluate factors influencing OSCE scores and to train examiners. GMS J Med Educ2020 Jun 15;37(4):Doc40. doi: 10.3205/zma001333. [PMID: 32685668]. [PMCID: PMC7346289].
  41. Mastour H, Eslami Hasan Abadi S, Nili MR, Delavar A, Zarei E. Real Time Electronic Formative Assessment in Medical Education and Its Impact on Motivational Beliefs and Self-regulation Strategies. Future of Medical Education Journal. 2018 Jun 1;8(1):42-6. doi:10.22038/FMEJ.2018.32080.1204.
Strides in Development of Medical Education
Volume 18, Issue 1
December 2021
Pages 1-6
Files
Share
How to cite
Statistics