Strides in Development of Medical Education

Document Type : Original Article

Authors

1 Specialist in Emergency Medicine, Associate Professor Department of Emergency Medicine, Center for Educational Research in Medical Sciences, Tehran University of Medical Sciences, Iran

2 Ph.D. Candidate of Medical education, Educational Development Center, Tehran University of Medical Sciences, Iran

Abstract

Background & Objectives: Different methods have been introduced to determine the cutscores of exams However they have been rarely used in our country In this study we used four methods to set a standard for preinternship objective structured clinical examination (OSCE) in Tehran University of Medical Sciences Methods: A prefixed score of 60% was considered in the first approach In the Angoff method a panel including 11faculty members was invited to estimate the probability of passing each station for a borderline student and mean of all estimations was considered as the station standard In borderline regression method a linear regression model for each station was calculated in which the students checklist scores and general scores were considered as dependent and independent variables respectively The station standard was calculated based on grade 2 in Likert scale In Cohens method we set the standard at 60% of the 95th percentile point Results : The standards for the total test in the prefixed score Angoff borderline regression and Cohens methods were respectively 60 4915 4239 and 4274 According to these standards the percentages of passing students were respectively 219% 676% 933% and 924% Conclusion: Using four methods to set standard of an OSCE showed that this approach for determining exams cutscore has significant effect on the calculated standard and the percent of students passing the exam

Keywords

  1. Norcini JJ. Setting standards on educational tests. Medical Education 2003;37:464–469
  2. Cizek GJ, Bunch MB. Standard setting: A guide to establishing and evaluating performance standards for tests. Thousand Oaks, CA: Sage Publications, Inc. 2007: 2022.
  3. Min Liu, Keh-Min Liu. Setting pass scores for clinical skills assessment. Kaohsiung Journal of Medical Sciences 2008;24(12):656-663
  4. Angoff WH. Scales, norms and equivalent scores. In: Thorndike RL, ed. Educational Measurement. 2nd ed. Washington, DC: American Council on Education 1971:508-600
  5. Kramer A, Muijtjens A, Jansen K, Dusman H, Tan L, van der Vleuten C. Comparison of a rational and an empirical standard setting procedure for an OSCE. Objective structured clinical examinations. Medical Education 2003: 2: 132–139.
  6. Cohen-Schotanus J, Vleuten Cees van der. A standard setting method with the best performing students as point of reference: Practical and affordable. Medical Teacher 2010; 32: 154–160
  7. Chesser A, Laing M, Miedzybrodzka Z, et al. Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. Medical Education 2004; 38: 825–831
  8. Davison I, Bullock A. Evaluation of the Introduction of the Objective Structured Public Health Examination. The Research Office School of Education University of Birmingham 2007.
  9. Mortaz Hejri S, Jalili M, Labaf A. Setting Standard Threshold Scores for an Objective Structured Clinical Examination using Angoff Method and Assessing the Impact of Reality Chacking and Discussion on Actual Scores. Iranian Journal of Medical education 2012;11(8):885-894. [In Persian]
  10. Kaufman DM, Mann KV, Muijtjens AMM, van der Vleuten CPM. A comparison of standard setting procedures for an OSCE in undergraduate medical education. Academic Medicine 2000;75:267–71.
  11. Cusimono MD, Rothman AI. The Effect of Incorporating Normative Data into a CriterionReferenced Standard Setting in Medical Education. Academic Medicine 2003;78(10 suppl):88-90
  12. Schoonheim-Klein M, Muijtjens A, Habets L and et al. Who will pass the dental OSCE? Comparison of the Angoff and the borderline regression standard setting method .European Journal of Dental Education 2009;13:162–171
  13. Norcini JJ, Stillman PL, Sutnick AI, et al. Scoring and Standard Setting with Standardized Patients. Evaluation & the Health Professions 1993;16(3):322-332.
  14. Yudkowsky R, Downing SM, Popescu Mihaela. Setting standards for performance tests: A pilot study of a three-level Angoff method. Academic Medicine 2008; 83 (10 suppl):13-16
  15. Kane MT, Crooks TJ, Cohen AS. Designing and evaluating standard-setting procedures for licensure and certification tests. Advanced Health Science Education Theory Practice 1999;4:195–207
  16. Pell G, Fuller R, Homer M, Roberts T. How to measure the quality of the OSCE: A review of metrics. AMEE guide no. 49. Medical teacher 2010;32:802–811