Strides in Development of Medical Education

Abstract Background and Objective: One of the potential strategies for ensuring the quality of educational programs is adopting a systematic approach to its evaluation. Current evidence indicates the lack of high quality program evaluation activities in the field of medical education. The aim of this study was to review the current status of program evaluation activities in Tehran University of Medical Sciences, Tehran, Iran, and formulate guidelines to promote program evaluation activities at the University level. Methods: A survey was conducted to investigate the current conditions of program evaluation using a questionnaire in 2012. Then, the comprehensive course evaluation guidelines, consisting of 22 items, were developed based on literature review, survey results, and experts’ opinions. Finally, each affiliated school developed its own evaluation plan. The evaluation taskforce reviewed evaluation plans using a checklist. Results: Using one tool or resource, 9 schools (90%) conducted course evaluation at least once. The views of students, faculty, staff or alumni were used occasionally. Moreover, 4 schools (40%) reported the evaluation results. After reviewing 14 submitted course plans based on the checklist, 51 feedbacks were provided. Most and least feedbacks were related to evaluation design and implementation and evaluation infrastructure, respectively. Conclusion: The process of developing guidelines and plans resulted in stakeholders reaching a common understanding of course evaluation, and in turn, creating evaluation capacity and more accountability.

Abstract Background and Objective: Outpatient educational environments are used for the teaching of a broad range of medical disciplines. Therefore, it is of utmost importance to assess these environments with valid and reliable tools. The Ambulatory Care Learning Education Environment Measure (ACLEEM) can be used for this purpose. The objective of this study was to assess the educational environment at outpatient clinics in teaching hospitals of Kerman University of Medical Sciences, Kerman, Iran.
Methods: This descriptive-analytical study which was cross-sectional in nature was conducted in Kerman University of Medical Sciences in 2013-14. The study population, based on census sampling, consisted of 70 residents specializing in internal, gynecological, pediatric, and surgical disciplines. The data collection tool was a valid and reliable questionnaire consisting of 2 sections, demographic questions and 50 questions in 8 domains. Data were analyzed using SPSS software and descriptive tests (frequency, mean, and standard deviation) and analytical tests (ANOVA and t-test).
Results: The total score of the questionnaire was 62.90 which showed the partially desirable status of the outpatient educational environment in the studied population. In terms of the total scores for each domain, the domains of clinical teacher competence (14.91 ± 2.99), clinical activity and patient care (12.85 ± 2.78), and time allocation for nonclinical activities (8.94 ± 2.93) obtained the highest scores, respectively. There was no significant relationship between sex, age, and educational year and mean score of each domain (P > 0.05). However, this relationship was significant for the field of study variable (P < 0.05). The highest mean score was related to gynecology residents (65.68 ± 5.7) and the lowest mean score was related to surgery residents (60.60 ± 5.9).
Conclusion: The findings showed that the resident's perspective of the outpatient environment is not satisfactory. Therefore, it is expected that the related authorities provide a favorable and conducive environment for teaching and learning through planning.

Abstract Background and Objective: In order to promote quality of education, teaching performances of all academic staff are evaluated by students every semester. This study aims to address the changing trend of staff scores in different semesters, and factors affecting it, in Kerman University of Medical Sciences, Iran, using multilevel modeling. Moreover, the results were compared to that of simple linear regression modeling. Methods: In the present analytical study, the scores of 336 academic staff of Kerman University of Medical Sciences during 2008 to 2012 were extracted from the students’ monitoring website. The tutor’s code was used for the identification of their scores in different semesters and was entered into the multilevel models. In order to investigate the effects of gender, work experience (in years), school (seven schools), and academic rank (master, assistant professor, associate professor, professor), the simple regression and multilevel models were compared. To study the significance of the random intercepts, the likelihood ratio test was used. In addition, to study the goodness of fit of the models, the Akaike information criteria (AIC), Bayesian information criteria (BIC), and the mean squared error (MSE) were used. Results: The results showed that time had a significant positive impact on the improvement of staff scores. However, the scores of male and female staff were not significantly different. The scores of professors were significantly higher than assistant professors. In addition, the score of dental school staff was significantly higher than that of medical school staff. The comparison of the goodness of fit of models showed that the multilevel modeling provided a better fit to the longitudinal data. In the linear regression model, variables such as work experience and academic rank (professors in comparison to assistant professors and nursing school staff in comparison to pharmacology school staff) were falsely considered significant, due to the lack of consideration of the dependence of longitudinal observations of the evaluation and correction of standard errors. Conclusion: The multilevel model, due to the consideration of the dependence of longitudinal observations of the evaluation, provides a better fit to data. Moreover, the incorrect use of the linear regression model, considering longitudinal observations to be independent, led to erroneous conclusions. The results of the present study, in terms of the goodness of fit of the models, showed a positive trend in scores of academic staff of Kerman University of Medical Sciences. In other words, the scores have improved over time.

Abstract Background and Objective: In the present study, the evaluation domain of ambulatory care training in Firoozgar Hospital, Tehran, Iran, was assessed based on the Iranian national standards of undergraduate medical education related to ambulatory education using the Baldrige scoring matrix. It also offered some suggestions for the promotion of education quality by use of the gap analysis method between the current condition and these standards. Methods: This descriptive analytic study was a kind of evaluation research performed using the standard checklist published by the undergraduate medical education council. Document evaluation, survey, and observation were performed based on the Baldrige excellence model. After the confirmation of validity and reliability of the checklist, the establishment level of national standards was evaluated in the clinics of Firoozgar Hospital in the evaluation domain. Data were analyzed according to the national standards of the domain of ambulatory education and the Baldrige scoring matrix. Finally, the quality of the current condition was determined as very satisfactory, satisfactory, medium, weak, and very weak. Results: In the clinics of Firoozgar Hospital, 25.50% of the national standards related to ambulatory education in the evaluation domain were observed and this domain was at a medium level. Conclusion: Due to the medium level of the evaluation domain, it is suggested that ambulatory education authorities provide the necessary facilities and reformations by considering the national standards of ambulatory education in the domains that were determined as weak and need quality promotion.

Abstract Background and Objective: Due to the remarkable advantages of virtual education, the increasing development of online education institutions has been approved by education and training authorities all around the world. In spite of their remarkable growth, online education institutions are facing significant challenges such as lack of trust from the society. These challenges are mainly associated with the uncertainties regarding the quality of online teaching and learning. Therefore, it seems essential to pay especial attention to the concept of quality in virtual learning institutions and its measurement for which a comprehensive framework of quality criteria and indicators is required. The main purpose of this study was to extract the indicators and attributes of quality evaluation in virtual education from the existing literature to facilitate the development of a holistic quality evaluation framework for online learning in higher education. Methods: In this qualitative research, meta-synthesis method based on the Walsh and Downe approach was employed. The research method comprises of 6 steps (including goal definition, systematic search for related papers, evaluation of documents, extracting related content, content analyses, and interpretation of findings). From the total number of 438 potentially related academic documents, 41 were selected and analyzed. Results: Through the six step method, 12 quality indicators of online education have been extracted in 4 aspects which include 47 attributes. The extracted indicators for quality evaluation, considering the variety of approaches presented in the literature, provide a comprehensive framework in which the target population, such as online learners, instructors, and employers, has been considered. Conclusion: The main indicators for evaluating the quality of virtual education comprised of students, instructors, courseware/content, learning management system, unit/course structure, technological infrastructure, teaching, learning, support services, professional development programs, evaluation and assessment, and graduates. In the developed quality evaluation framework, the measurement attributes of each indicator have been extracted from the literature.

Abstract Background & Objective: Quality assessment of educational services is one of the most important actions in line with an improvement in the services. If the qualitative dimension of services is not taken into account, we will observe significant academic failure and reduction in creativity among the learners. Therefore, one of the salient steps in improving the quality of educational services is quality assessment from the viewpoints of its users. The aim of the present study was to determine the quality of educational services from the viewpoints of residents based on the SERVQUAL model. Methods: In this descriptive-analytical study, 204 residents from the educational hospitals of Kerman,  Iran,  participated.  The  participants  were  selected  using  simple  random  sampling method. The data collection tool was the Evaluation of Educational Quality Questionnaire that evaluates  educational  quality  based  on  the  SERVQUAL  gap  analysis  method.  This  tool compares respondents’ perceptions and expectations of the quality of educational services in 5 domains  (reliability,  responsibility,  confidence,  assurance,  empathy,  and  tangible  physical). Content validity of this questionnaire was approved by experts' opinions and its reliability was confirmed using Cronbach's alpha (r = 0.88). To analyze data, SPSS software was used for performing descriptive (mean, standard deviation) and analytical (independent t-test) tests. Results: The findings showed that there was a negative gap between residents' perceptions and expectations of the quality of educational services in all 5 dimensions. The highest and lowest level  of  gap  was  related to the  tangible physical  (-1.50)  dimension  and the responsibility (-0.51) dimension, respectively. Conclusion: There is a negative gap between residents' perceptions and expectations in different dimensions of educational services quality based on the SERVQUAL model. In order to improve the quality of education, a review of educational infrastructures (environment and facilities) and education management seems necessary. This requires the efforts of relevant authorities toward the development and improvement of these infrastructures.

Abstract Background & Objective: The evaluation of educational programs is the most important aspect of any educational system. If performed correctly, it will provide valuable information on designing and implementation of educational programs and a basis for the evaluation of educational practices. Methods: This present study was performed in 2014 on 22 PhD students attending the Schools of Traditional  Medicine  of  Mashhad,  Kerman,  and  Shahid  Beheshti  Universities  of  Medical Sciences, Iran, in 2009-2013. The data collection tool consisted of a questionnaire containing open- and closed-ended questions. The reliability and validity of the questionnaire have been approved. The quantitative data obtained were analyzed and categorized using Microsoft Excel. Results: The general objectives of traditional pharmacy consist of mission, perspective, goals, and professional duties. Each course was evaluated from the view point of students. This showed that more than half of the students agreed on the overviews and details (general objective,  prerequisites,  content  titles,  student  evaluation,  and  educational  sources).  The educational  sources  of  some  courses  such  as  Iranian  Traditional  Medicine  (foundations  and principles) with 43.9%, Sanitation and Prevention Principles with 33%, and Identification of Raw Material and Production of Traditional Medicine with 36.8% obtained the less than half of the positive votes of students. Moreover, 42.1% of students believed that the courses of Identification of Raw Material, Production of Traditional Medicine, and Identification of Traditional Medicine Forms were not presented in accordance with the educational program. Conclusion: The participants mostly approved the aims and overviews of this field of study each of its courses. The proposed recommendations show that the units and content of some courses, and their educational sources and priorities require revision and reformation. In general, due to being a new field, traditional medicine contains ambiguities which must be clarified through further evaluation and elimination of deficiencies observed in the current survey.

Abstract Background & Objective: Owing to the fact that seven years has passed since the establishment of the PhD program in reproductive health in Iran, its evaluation is essential. Hence, the aim of this study was to evaluate the PhD program in reproductive health in Iran based on the Context- Input-Product-Process (CIPP) evaluation model. Methods: This descriptive study was conducted in 2013 in the nursing and midwifery schools of Iran in which a PhD program in reproductive health was being taught (Tehran, Shahid Beheshti, Isfahan, Shahroud, and Mashhad Universities). The study population consisted of all heads of departments of midwifery, faculty members of PhD programs in reproductive health, heads of libraries in nursing and midwifery schools, graduates and reproductive health PhD students of medical sciences universities in Iran. Data collection tools consisted of five researcher-made questionnaires based on the CIPP model and a checklist of educational facilities and equipment assessment. Content and face validity were evaluated based on expert opinion. Cronbach’s alpha coefficient was calculated in order to obtain the reliability of the questionnaires. Data were analyzed using descriptive statistics and mean of groups were compared using One-way ANOVA. Results: The status of context indicators of PhD programs in reproductive health was evaluated as appropriate by 60% of department heads, 83.3% faculty members, 83.3% graduates, and 74.1% students. Managers, graduates, and students evaluated the status of input indicators of PhD programs in reproductive health as fairly appropriate. Moreover, faculty members, graduates, and students evaluated the status of process indicators of PhD program in reproductive health as fairly appropriate. In addition, in the product domain, managers, faculty members, and students evaluated the status of product indicators of PhD programs in reproductive health as fairly appropriate. Conclusion: The results of this study showed that the status of PhD programs in reproductive health was fairly appropriate; therefore, it seems necessary to improve the indicators that cause the program status to be fairly appropriate.

Abstract Background and Objective: The evaluation of the teaching of lecturers by students is necessary in order to provide feedback for lecturers on improving the quality of student education, evaluating teacher's duties, and to help administrators in making decisions on employing or promoting faculty members. Therefore, the aim of this paper was to design a questionnaire related to education design indices and to classify these factors in the evaluation of teaching of lecturers by students. It should be noted that Danielson’s Framework for Teaching Evaluation Instrument, 2013 Edition, was used in the designing of the questionnaire. Methods: This was a cross-sectional study. Using the Krejcie and Morgan table and the omission of unreturned and incomplete questionnaires, a total of 297 questionnaires were studied. The questionnaires were investigated through validity coefficient, exploratory factor analysis, confirmatory factor analysis, and mean. Data were analyzed using SPSS and Amos software. Results: Through exploratory factor analysis, 8 subscales were achieved. In addition, confirmatory factor analysis approved the abovementioned indexes available in the subbranches of student evaluation. In assessing teaching indexes, the most important aspect is interaction between teachers and students, and punctuality, presenting new topics, mastery over educational material, student participation in discussions, using educational technologies, lesson design, and evaluation. Conclusion: In general, the reliability and validity, face validity, content coefficient, and construct validity of the scale were satisfactory. It can be used in internal researches.

Abstract Background & Objective: In recent years, with increasing awareness of limitations of traditional assessment methods in the measurement of learner capabilities, assessment methods have undergone many changes. This survey addresses the extent to which educational departments in Tehran University of Medical Sciences, Tehran, Iran, use various student assessment methods. Methods: The present cross-sectional study was conducted using a researcher developed tool to gather information about student assessment methods in 2012. Based on Miller’s pyramid of assessment, common student assessment methods were classified into written and oral assessment, clinical reasoning assessment, clinical skills assessment, and workplace-based assessment. Study sample consisted of all educational departments. Sampling was performed using the census method, which determined the use or lack of use of each method of assessment at different educational levels. The data collected were analyzed using descriptive statistics. Results: The response rate was 70.43%; 81 of 115 departments completed the questionnaire. The most frequently used methods by departments were written and oral exams. Among them, the multiple choice test was the most widely applied assessment method. Patient management problem (PMP) was the most broadly used method to assess clinical reasoning. Moreover, among clinical skills assessment, objective structured clinical examination (OSCE) was the most commonly applied method in medical clinical courses. Conclusion: Graduates of medical universities must acquire capabilities far beyond the acquisition of theoretical knowledge, but assessment methods used by departments do not necessarily assess their capabilities. The results of this study emphasize the need for the revision of medical student assessment programs.

Abstract Background & Objective: The main purpose and role of higher education is to train professional workforce in the fields required by society through training them the necessary skills for their field. Today, the promotion of educational quality is considered as the basis of an authentic higher educational system. Therefore, the most efficient way to improve educational quality is educational evaluation in order to understand the success rate of faculty members on achieving this goal. Nevertheless, researches indicate that the evaluation of faculty members seems to be one of the most controversial issues of medical education and needs further attention and research. Obtaining faculty member's experiences, which is the main goal of this study, seems to be valuable. Therefore, the main objective of this study was to explore faculty member's experiences of the educational evaluation system. Methods: The present qualitative study was carried out using content analysis in Mazandaran University of Medical Sciences, Iran, in 2013. Participants included 8 faculty members, who volunteered to take part in the study. Data was collected through semi-structured interviews and purposeful sampling and continued until saturation. Results: From the results, 3 main themes of lack of a clear objective for the evaluation system, inappropriate methods of evaluating faculty members, and the impact of their characteristics on the evaluation were achieved. Conclusion: The study results show the necessity of comprehensive fundamental reform in the method of educational evaluation of faculty members, improvement of evaluators’ awareness and commitment, improvement of education quality, and ultimately, promotion of higher educational systems. Moreover, the provision of solutions for the enhancement of faculty members’ efficacy and motivation is required. On the other hand, it is essential to diminish their worries and concerns through creating an authentic, justice-oriented evaluation system, and ultimately, to achieve maximum improvement in educational policies and programs.

Abstract Background and Objective: Continuous promotion of education quality necessitates assessment that can be performed based on each of the elements forming the educational system (input, process, product, output, and result). This assessment results in the improvement of educational system quality. Therefore, the present study has been conducted with the purpose of evaluating the causal model of the relationship between presented services quality and assessment of professors.
Methods: The present research was a descriptive analytical study. The statistical population consisted of all nursing students of Urmia University of Medical Sciences, Urmia, Iran, among which 73 individuals were selected through systematic random sampling. Data were gathered using the two questionnaires of educational services quality and faculty members assessment. The Educational Service Quality Questionnaire is based on the SERVQUAL model and consists of the elements of tangible factors, reliability, responsiveness, assurance, and empathy. The Faculty Members Assessment Questionnaire consists of the elements of teaching methods, dominance and academic ability, education discipline, and personal and social characteristics of the faculty members. The validity of the questionnaires was determined based on the viewpoints of experts in management and educational sciences. The combined reliability of the questionnaires was measured using the Dillon-Goldstein coefficient and Average Variance Extracted. The combined reliability of both questionnaires was higher than 0.7 and Average Variance Extracted was higher than 0.5. Data were analyzed using one way t-test, structural equation modeling method using partial least square (PLS), self-test system, Q2 Stone-Geisser index, and goodness of fit index through SPSS, Microsoft Excel, VisualPLS, and SmartPLS.
Results: The relationship of educational services quality with faculty members’ assessment factors (teaching method, educational discipline and faculty members’ personal and social characteristic) was significant. The service quality variant has the greatest effect on personal and social characteristics of faculty members (0.27). The absolute (0.41) and relative (0.91) goodness of fit index shows that the tested model has a good level of fit.
Conclusion: Paying attention to service quality factors could have a positive role in effective evaluation of faculty members by students. As a result of the assessment of faculty members by student, the performance and activities of faculty members can be improved.

Abstract Background and Objective: Evaluation of students before entering the clinical field can be a reliable criterion for assessing the quality of students’ clinical skills. This study was performed with the aim of evaluating nursing students using objective structured clinical examination (OSCE) method in order to assess students’ competence before entering the clinical environment. Methods: In this cross-sectional study, the knowledge and practice of 39 nursing students were examined during 2 consecutive days before enter the clinical field in determined stations of the nursing skills laboratory of Rafsanjan University of Medical Sciences, Iran, using relevant checklists. Data were analyzed using SPSS software and descriptive and analytic statistics such as independent t-test and Spearman’s rank correlation coefficient. Results: Of the 39 participants, 22 (56.4%) were female, 17 (43.6%) were male. The mean age of participants was 21.85 ± 0.96. The highest score was related to writing of drug form, kardex, and nursing report (94.8%), and ECG procedure (74.7%), respectively. The least score belonged to tracheostomy suction (12.8%), and catheterization and subcutaneous injection procedures (25.6%), respectively. Conclusion: According to the findings and considering the weaknesses of students, effective steps can be taken toward the improvement of their performance.

Abstract Background & Objective: The use of an integrated and approved method in clinical evaluation is of great importance. The aim of this research was the comparison of the 360- degree and common evaluation methods in clinical skills evaluation of undergraduate surgical technology students in Kerman University of Medical Sciences, Iran, during the operating room technical training course. Methods: This single group interventional study was performed on 4th semester students in surgical technology at Kerman University of Medical Sciences in the operating room of Shahid Bahonar Hospital of Kerman. The data collection tool consisted of a questionnaire which was completed by 24 students and 2 lecturers.. Data were analyzed using SPSS software and paired and independent t-tests. Results: Findings showed that mean age of students was 21.20 ± 0.56 years and most of them were females. Mean and standard deviation of total score of the common and 360-degree evaluation methods were 17.66 ± 0.86 and 17.88 ± 0.88, respectively. There was no significant differences between the mean scores of these methods (P = 0.001). Students had relative satisfaction with the 360-degree evaluation method (score = 6.12 ±  64.77) and relative dissatisfaction with the common evaluation method (score = 38.11 ± 08.49). There was a significant difference between students satisfaction with these two methods (P  < 0.001). Lecturers had relative dissatisfaction with the common evaluation method (score = 47.51 ± 6.64) and complete satisfaction with the 360-degree evaluation method (score = 85.98 ± 9.84). Conclusion: The students’ 360-degree evaluation score was similar to that obtained through their evaluation by lecturers. Considering students’ relative satisfaction and lecturers’ total satisfaction with the 360-degree method, in comparison to their relative dissatisfaction with the common method, it seems that using this method can be effective in the modification of their dissatisfaction with the current clinical evaluation method.

Abstract Background & Objective: In recent years, there has been an increase in the number of medical universities that have launched doctoral nursing education programs in their centers; thus, monitoring the quality of these programs to identify their strengths and weaknesses is very important. This study was conducted to evaluate the quality of doctoral nursing education in a university center in Iran. Methods: This descriptive and comparative study was performed on 33 lecturers and students/graduates using the census method in 2015. Data were collected using the quality of nursing doctoral education (QNDE) Questionnaire. Results: There was no statistically significant difference between the mean scores of the two groups, but this difference was significant in the program domain (P = 0.036). The participants’ answers only showed significant differences in a few statements of the program, resources, and evaluation domains (P < 0.050). A significant difference was observed between the views of lecturers and students in terms of advanced computer facilities and internet access, and appropriate research infrastructure for facilitating research and education (P < 0.050). Conclusion: Weaknesses such as lack of resources, infrastructure, library holdings, and regular and timely feedback to program evaluators require attention in order to be decreased or resolved. Great differences in the scores of overall course quality and curriculum evaluation between the two groups revealed the need for further research for the identification of factors effective on the quality of doctoral education programs from the view of both groups.

Abstract Background & Objective: The desire for portfolio application, as a suitable method for evaluation of clinical students, has recently expanded. This study investigated the implementation of portfolio in the evaluation of medical students. Methods: In the current study, medical students in their educational rotations in health centers in Mashhad, Iran, in 2011 were evaluated by using portfolio. The scores of medical students' performance and health centers supervisors' evaluation of them were assessed during 10 months and compared with scores from a similar period in the past. Data were analyzed using t-test and SPSS software. Results: Portfolios were completed by all medical students during their rotation in health centers and and evaluation of student was done based on portfolio scoring. Mean performance scores of medical students and health center supervisors' evaluation of them had increased compared of them by portfolio evaluation had increased compared with traditional students' evaluation method in similar past period. Conclusion: Using portfolios for evaluation of medical students' performance in community educational fields is proposed as a performance-based approach.

Abstract Background  &  Objective:  Clinical  environments  have  a  crucial  role  on  medical  students' training. Thus, the aim of this study was to assess clinical environments based on the (Dundee Ready  Education  Environment  Measure)  DREEM  model  from  the  viewpoint  of  interns  and residents in hospitals affiliated with Kerman University of Medical Sciences, Iran, in 2012. Methods:  This  was  a  descriptive-analytic  study.  The  data  collection  tool  was  the  DREEM Questionnaire with 50 questions (5-point Likert scale) in the 5 domains of learning, teachers, educational environment, student's academic self-perceptions, and student's social self-perceptions. The study environment consisted of 4 main wards (internal, surgical, pediatrics, and gynecology) of hospitals affiliated with Kerman University of Medical Sciences. The study subjects consisted of 63 interns and 73 residents. Data was analyzed in SPSS software using Students' t-test and ANOVA. Results: Mean score of perception of educational environment in interns was 161.17 ± 22.30 and in residents was 157.45 ± 21.14. The comparison of different areas of clinical environment evaluation only showed a significant difference between the two groups in the area of student's social self-perceptions (P < 0.05). The interns' score was higher than that of the residents. No significant differences were observed between hospitals and the studied wards. Conclusion: The students' perceptions of their educational environment in clinical wards were desirable. Despite different literature's recommendation of using DREEM in order to evaluate weaknesses and strengths of clinical environments, the concurrent use of other methods and instruments for the assessment of the efficacy of this questionnaire is recommended.

Abstract

Abstract