Document Type : Brief report
Authors
1 General Practitioner, M.Sc. in Medical Education, Medical Education Development Center, Hamedan University of Medical Sciences, Hamedan, Iran
2 General Practitioner, M.Sc. in Medical Education, Instructor, Department of General Education, School of Medicine, Hamedan University of Medical Sciences, Hamedan, Iran
Abstract
Background & Objective: The aim of this study was to survey the effect of presenting feedback on improvement of multiple choice questions (MCQ) design in annual residency exams Methods: In this semiexperimental study the structure and taxonomy in MCQs of annual residency exams in 2009 were analyzed through Millmans checklist Based on this data an interventional feedback on strengths and weaknesses was presented to question designers who had experience of attending MCQ workshops and designing questions in the past The principles of giving technical feedback in medical education such as objectivity right timing and being direct respectful and without judgment were considered After the annual residency exam in 2010 the questions of designers in both years were compared in respect to Millmans principals and taxonomy before and after the intervention Results: From 2500 questions 2035 questions were entered into the study 652% of the questions developed in 2009 had no structural flaw this increased to 828% after the intervention which was a significant difference (P < 0001) Moreover there was a significant increase in the percentage of questions with a high taxonomy from 38% in 2009 to 531% in 2010 (P < 0001) Conclusion: Giving interventional feedback based on its techniques in medical education to experienced designers of MCQs can affect the improvement of designing quality of questions
Keywords
- Bazargan A. Educational assessment. 5th ed. Tehran: Samt; 2007. [In Persian]
- Gharib M, Khakbazan Z, et al. Educational Process in Medicine. 1th ed. Tehran: Tehran University of Medical Sciences; 2005. [In Persian]
- SabooriKashani A, Mohamadi M, Zeinaloo A, Razavi M. Standardization of student ُُs evaluation: Two testing theories. 1th ed. Tehran: Tehran sadra; 2004. [In Persian]
- Zolfaghari B, Adibi N, Darakhshanfar S, Tansaz M. Academic achievement tests in medical sciences. 1st ed: Isfahan: Medical University Educational Development center Publisher; 2001. [In Persian]
- Vaezi AA, Hosseini M, Aghily H, Mirjalili MR. The investigation of applying multiplechoice question making rules in final exams of the faculty members of ShahidSadoughi Yazd University of Medical Sciences. The Journal of Medical Education & Development Center of Yazd. 2007; 2(1)3138. [In Persian]
- Haghshenas M, Vahidshahi K. Evaluation of multiple choice questions in the school of medicine, Mazandaran University of in medical sciences, the first semester of 2007. Strides Development of Medical Education 2008; 5(2):120-27. [In Persian]
- Bighlarkhani M, Meyari A, Falah M, Zamanian A. The study of Multiple Choice Questions designed in residency annual exams in Hamedan University Medical Sciences 2007&2008. Final Report of research project 10856, Hamedan university of Medical sciences. 2009. Available From: http://dme.umsha.ac.ir/uploads/MCQ%20Te rm...%20pezesh1.7cross-sectional.pdf [In Persian]
- Meyari A, Beiglarkhani M. The effect of education on improvement of multiple choice questions' designing in annual residency exams of dental faculty. Iranian Journal of Medical Education 2012; 12(1):36-45. [In Persian]
- Vahidshahi K, Mohagheghi MA. Comparison of some quality criteria of multiple choice questions in residency annual exam 2007 & 2008. Available From: http://cgme.behdasht.gov.ir/uploads/264_78 1_N4_Moghayese.html
- Ministry of health, Treatment and Medical Education. Iranian council for graduate medical education.[Analysis of structure &results of Residency annual exam 57;2010. Available From:http://cgme.behdasht.gov.ir/uploads/2 64_781_N10_TahlilBord.pdf [In Persian]
- Millman J. The new handbook of teacher evaluation. Assessing elementary and secondary school teachers. New York: Sage publication;1981.
- Saif AA. Educational Measurement, Assessment and Evaluation. Fourth edition, 25th revision. Tehran: Nashredoran; 2007. [In Persian]
- Kaveh M, Bahreini M. Analytical assessment of multiple-choice tests at Mashhad Medical School. Medical Journal of Mashhad University of Medical Sciences 2002; 45(76):89-95. [In Persian]
- Shakurnia A, KhosraviBorojeni A, Mozaffari A, Elhampour H. An evaluation of exam questions designed by faculty members, emphasizing on the multiple choice question Jundishapur structure, University of Ahvaz Medical Sciences, 2007. Strides in Development of Medical Education 2010; 6(2):129-138. [In Persian]
- Arab M. The study of Multiple Choice Questions flaws in Hamedan medical school 1999. Journal of Tehran University of Medical Sciences: Abstract of 4th National Congress of Medical Education 2000: 51. [In Persian]
- Doripour F, KarimiSh, Pakaein J. Evaluation of MCQ'S of nursing students' health courses 2006. Strides in Development of Medical Education. Abstract of 8th National Congress of Medical Education 2006. [In Persian]
- Rasoulinejad A, Vakili Z. Comparison of taxonomy of Kashan medical University's residency exams. Strides in Development of Medical Education. Abstract of 8th National Congress of Medical Education 2006. [In Persian]
- Ende J. Feedback in Clinical Medical Education. The Journal of the American Medical Association 1983; 250(6):777-81.
- Downing SM. Assessment of knowledge with written test formas. International Handbook of research in medical education; 2: 647-72.
- Smith JD, Prideaux D. Developing the accredited postgraduate assessment program for Fellowship of the Australian College of Rural and Remote Medicine.Rural and Remote Health 2007: 7(4):805.
- Huxham GJ, Naeraa N. Is Bloom's taxonomy reflected in the response pattern to MCQ items? Medical Education 1980; 14(1):23-6.
- McCourbie P. Improving the fairness of multiple choice questions: A literature review. Medical Teacher 2004: 26(8):70912.
- Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item writing flaws in multiple choice questions used in high stakes nursing assessments. Nurse Education in Practice 2006; 6(6):354-63.
)