Strides in Development of Medical Education

An Evaluation of Exam Questions Designed by Faculty Members Emphasizing on the Multiple Choice Question Structure Ahvaz Jundishapur University of Medical Sciences 2007

Document Type : Original Article

Authors

Abdolhussein Shakurnia 1
Azar Khosravi Borojeni 2
Alireza Mozaffari 3
Houssein Elhampour 4

1 M.Sc. in Immunology, Instructor of Immunology Dept., School of Medicine, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

2 Ph.D. in Microbiology, Associate Professor of Microbiology Dept., School of Medicine, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

3 Specialist in Internal Medicine, Associate Professor of Internal Medicine Dept., School of Medicine, Ahvaz Jundishapur University of Medical Sciences,Ahvaz, Iran

4 M.Sc. in Educational Sciences, Instructor of Educational Sciences Dept., School of Educational Sciences and Psychology, Shahid Chamran University,Ahvaz, Iran

Abstract
  Background & Objective : Evaluation of educational achievement is an important part of an educational process such an evaluation is possible via holding exams Multiple Choice Question (MCQ) as an objective testing method is the most common tool for evaluating students Hence this study was conducted to evaluate the exam questions designed by faculty members with emphasis on the MCQ structure in Ahvaz Jundishapur University of Medical Sciences   Methods : In this descriptive study questions designed by 197 nonclinician faculty members were analyzed Data was collected using a checklist including demographic information properties of exam questions and items related to comparing MCQs with design rules of such questions   Results : The results revealed that 81% of the questions developed by faculty members were essay 5532% and 3654% were objective and a combination of essay and objective respectively Among a total of 5208 evaluated questions 6496% contained no structural flaw The most common flaws included ambiguous or unclear question stems (1594%) use of negative verb in the stem (1222%) using “all” or “none” among option (1085%) and use of blank space in the beginning or middle of the stem (1041%) Almost 90% of the MCQs were at cognitive level (taxonomy I)   Conclusion : Due to the importance of developing standard questions corresponding to guidelines for MCQs in acceptable exams regular revision of exam questions and further training courses for designing questions is recommended

Keywords

Exam
Faculty members
MCQ
  1. Guilbert JJ. Educational handbook for health personnel. 6th ed. Geneva; World Health Organization: 1987.
  2. Sepasi H, Attari YA. The study of psychometric characteristics of Shahid Chamran University Faculty members final test scores. Journal of Education and Psychology (Special Issue an Education) 2006; 3(4): 1-20. [In Persian]
  3. Sepasi H. A comparison between multiplechoice and True-False test item in measuring Shahid Chamran University undergraduate students’ academic achievement in measurement and evaluation course with the control of students’ absences. Journal of Education and Psychology 1998; 5(1&2): 31-56. [In Persian]
  4. Moatamedi A, Seif A. Survey on evaluation methods of faculties of evaluation method course in Training Teacher Center and related factors. Master’s dissertations. Allameh Tabatabaei University 1994. [In Persian]
  5. Kaveh Tabatabaei MS, Bahreini Tousi MH, Modabber Azizi MJ, Ebrahimzadeh S, Bahreini Tousi V, Bahreini Tousi K. Analyses survey on multiple choice question in medical school in Mashhad University of medical sciences in 2002. Medical Journal of Mashhad University of Medical Sciences 2003; 45 (76): 89-95. [In Persian]
  6. Holsgrove G, Elzubeir M. Imprecise terms in UK medical multiple-choice questions: What examiners think they mean. Med Educ 1998; 32 (4): 343-50.
  7. Sepasi H. A study of cognitive domain and analysis of psychometric characteristics of test terms in final examinations of Arabic, calculus and biology courses of the third grade high school girls in different Scio-economic statusin Khuzestan, Iran. Journal of Education and Psychology 2006; 13(4): 57-78. [In Persian]
  8. Ghadimi Moghaddam MM. Survey on special knowledge Faculties of Tehran Azad University (9 Area). Master theses. Azad University of Tehran; 2004. [In Persian]
  9. Arab M. Survey on defect of multiple choice questions in medical school of Hamadan Medical University in 2000. 4th National Congress of Medical Education. Tehran: University of Medical Sciences; 2001: 51. [In Persian]
  10. Shakurnia A, Khosravi A, Shariati A, Zareei A. Survey on multiple choice questions of academic members in Ahvaz Jundishapur University of Medical Sciences in 2005. Abstract of the 8th National Congress of Medical Education. Kerman University of Medical Sciences; 2007: 44. [In Persian]
  11. Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item writing flaws in multiplechoice questions used in high stakes nursing assessments. Nur Educ Today 2006; 26 (8): 662-71.
  12. Seif A. Assessment methods and educational evaluation. Tehran; Nashre Douran: 1999. [In Persian]
  13. Rezaei H, Shoughia Shafagh Aria F, Doulati Y, Mohagheghi MA, Malekzadeh R. Evaluation techniques in medical education. Tehran: Ammeh Publication; 1999. [In Persian]
  14. Zolfaghari B, Adibi N, Darakhshanfar S, Tansaz M, Karbasi A, Niroumand P. Academic achievement tests in medical sciences. Isfahan: Medical University Educational Development center; 2001. [In Persian]
  15. Mehrens AW, Lehman JI. Measurement and evaluation in education and psychology. New York: Holt, Rinehart and Winston Inc; 1984.
  16. Kiyamanesh A. Multiple choice questions and its effect on student achievements’, content and teaching methods. Taalim va Tarbiat 1995; 8 (3): 57-77. [In Persian]
  17. Vaezi AA, Morovvati MA, Heidari MR. Obeying pertinent rules for making multiple choice questions in final exams designed by faculty members. Abstract of the 9th National congress of Medical Education. Yazd: University of Medical Sciences; 2008: 250. [In Persian]
  18. Khoshkam M, Shajari Zh. Trend in Processing of structural designing general science exam in the study in 1999 in Tehran. Research in Medical Sciences 1997; 3(Supple. 1): 160. [In Persian]
  19. Allah Verdi-pour H, Bashirian S. Knowledge and attitude of faculty members about evaluation of educational progress. Scientific Journal of Hamadan University of Medical Sciences 2000; 7 (4): 20-4. [In Persian]
  20. Kaveh Tabatabaei MS, Bahreini Tousi MH, Modabber Azizi MJ, Ebrahimzadeh S, Bahreini Tousi V, Bahreini Tousi K. Evaluation methods in clinical ward and medical student’s views’ in 2002. Medical Journal of Mashhad University of Medical sciences. 2003; 45 (76): 97-103. [In Persian]
  21. Latifi M, Shakurnia A. Comparing faculty members and experts’ points of view about questions designed in Ahvaz University of Medical Sciences. Abstract of the 8th National Congress of Medical Education. Kerman University of Medical Sciences; 2007: 159. [In Persian]
  22. Mohammadi Zeidi A, Heydarnia A, Khalaj M. Comparing test preparation methods in the school of public health before and after test standardization workshop. Abstract of the 9th National Congress of Medical Education. Yazd University of Medical Sciences; 2008: 146. [In Persian]
  23. Gorg Yaraghi M, Nazem M, Ebrahimi A, Avizhgan M, Mazaheri M, Esmaeili A. The effect of an intervention on test preparation in surgery ward, 2007. Abstract of the 9th National Congress of Medical Education. Yazd University of Medical Sciences; 2008: 25. [In Persian]
  24. Dorripour F, Karimi S, Pakaein J. Evaluation of multiple choice questions in hygiene course of nursing students, 2006. Abstract of the 8th National Congress of Medical Education. Kerman University of Medical Sciences; 2007: 67. [In Persian]
  25. Jozefowicz RF, Koeppen BM, Case S, Galbraith R, Swanson D, Glew RH. The quality of in-house medical school examinations. Acad Med 2002; 77 (2): 156-61.
  26. Gorg Yaraghi M, Ebrahimi A, Ashirioun V, Azadbakht S, Avizhgan M, Esmaeili A, et al. An analysis of multiple choice questions for undergraduate medical students and designing necessary steps to correct students’ evaluation process. Abstract of the 9th National Congress of Medical Education. Yazd University of Medical Sciences; 2008: 27. [In Persian]
  27. Vahidshahi K, Mahmoudi M, Shahbaz Nezhad L, Shakeri S. The quality of multiple choice questions in the first semester, 2006, School of Medicine, Mazandaran University of Medical Sciences. Abstract of the 9th National Congress on Medical Education. Yazd: Yazd University of Medical Science; 2008: 41. [In Persian]
  28. Khoddam H, Jouybari L, Sanagoo A. Nursing clinical instructors’ skills in designing question. IJME 2005; 5 (2): 191-2. [In Persian]
  29. Sellappah S, Hussey T, Blackmore AM, McMurray A. The use of questioning strategies by clinical teacher. J Adv Nurs 1998; 28 (1): 142-8.
  30. Masters JC, Hulsmeyer BS, Pike ME, Leichty K, Miller MT, Verst AL. Assessment of multiple-choice questions in selected test banks accompanying text books used in nursing education. J Nurs Educ 2001; 40 (1): 25–32.
  31. Fattahi Bafghi A, Karimi H. The effect of education on academic members’ designing multiple choice questions regarding face validity and content validity. Abstract of the 7th National Congress of Medical Education. Tabriz University of Medical Sciences; 2005: 63. [In Persian]
Strides in Development of Medical Education
Volume 6, Issue 2
2010
Pages 129-138

Home

Editorial Policy

Submit Manuscript

Guide for Authors

Peer Review Process

Guide for Reviewers

Publication Ethics Policy

Forms

Suggestion and Complain

FAQ

Contact Us