Document Type : Original Article
Authors
1 Associate Professor, Department of Community and Family Medicine, Afzalipour Faculty of Medicine, Kerman University of Medical Sciences, Kerman, Iran
2 Medical Student, Afzalipour Faculty of Medicine, Kerman University of Medical Sciences, Kerman, Iran
3 Assistant Professor, Department of Medical Ethics and History of Medicine, School of Persian Medicine, Kerman University of Medical Sciences, Kerman, Iran
Abstract
Background: Medical ethics courses play a pivotal role in medical education, aiming to enhance the moral decision-making capabilities of medical students. As such, the evaluation of medical ethics education programs within medical faculties is of paramount importance for the improvement of these initiatives.
Objectives: This study aimed to appraise the psychometric properties of the Context, Input, Process, Product (CIPP) evaluation model in medical ethics education.
Methods: This cross-sectional study was conducted on 136 clerckship medical students, who were enrolled in a medical ethics course at Kerman University of Medical Sciences, Kerman, Iran. The participants completed a 40-item researcher-made questionnaire, which was designed based on the CIPP evaluation model. The questionnaire was divided into four distinct sections, each corresponding to the context, input, process, and product aspects of the model. The face and content validity of the instrument was established by an expert panel, consisting of 10 faculty members of medical education and medical ethics. The reliability of the questionnaire was also determined by calculating its internal consistency using Cronbach’s alpha coefficient. Moreover, the construct validity of the questionnaire was assessed via confirmatory factor analysis, using the goodness of fit indices. Data were analyzed in SPSS version 19 and Lisrel 8.8.
Results: The content validity index and content validity ratio of the questionnaire were measured to be 0.97 and 0.89, respectively. The internal consistency of different sections of the questionnaire ranged between 0.71 and 0.87. In the confirmatory factor analysis, the model showed acceptable goodness of fit indices.
Conclusion: In this study, the psychometric properties of the CIPP evaluation model for medical ethics education were found to be acceptable and applicable.
Highlights
Habibeh Ahmadipour: (Google Scholar) (PubMed)
Amirhossein Alirezaie: (Google Scholar) (PubMed)
Mina Mobasher: (Google Scholar) (PubMed)
Keywords
Background
Medical ethics courses are one of the key components of medical education due to the necessity of a humanistic approach in medical treatment, which mandates practitioners in clinical settings to continually develop and refine their knowledge, attitudes, and capabilities. Moreover, medical schools need to pay particular attention to medical ethics training programs that aim to improve medical students’ moral decision-making capabilities (1). Recently, the increased complexity of the healthcare system has resulted in the establishment of medical ethics training programs around the world (2, 3). In Iran, a significant reform project was undertaken by the Medical Ethics and History of Medicine Research Center of Tehran University of Medical Sciences (TUMS, Tehran, Iran) in 2005. This project, which was carried out in collaboration with the Medical Education Development Office, aimed to enhance the medical ethics programs. Following this reform, a paradigm shift in the approach to medical ethics education was reported by 2011 (4).
One of the most challenging aspects of education is assessing the quality and effectiveness of education and instruction. The evaluation of educational programs can provide insight into the degree to which the objectives of a program have been fulfilled (5). It is generally a crucial method for improving quality, enhancing strengths, and mitigating potential shortcomings (6, 7). Stufflebeam et al. developed the Context, Input, Process, and Product (CIPP) model in the late 1960’s as a robust framework for evaluating educational initiatives (8). Over 200 doctoral dissertations, journal publications, and assessment studies related to the CIPP model have been identified, spanning across various countries and disciplines (9). Additionally, the CIPP model has been used in numerous research studies in Iran to assess educational initiatives, as seen in the long-term educational clinical faculty development program (10), the externship curriculum for public health courses by Yazd University of Medical Sciences (11), and the master's program in neonatal intensive care nursing (12).
Four elements make up the CIPP model, including context, input, process, and product evaluations. The needs of the intended audience and the setting in which the program is executed are evaluated through context evaluation. Input evaluation focuses on the program resources, including financing, employees, and materials. The process of program implementation is also examined in this model to determine if the program is proceeding according to the plan, and finally, product evaluation scrutinizes the outcomes of the program, assessing whether it has successfully achieved its intended objectives and goals (8).
The CIPP model offers a comprehensive framework for evaluating educational programs across all levels, ranging from individual classrooms to entire districts and even state-wide education systems. It is a comprehensive, flexible, interactive, and decision-making-focused model that can be tailored to the specific demands of any program (7). According to previous research, it is a decision-making model that can provide a comprehensive framework to promote the efficiency of educational programs by taking into account the context, input, process, and product areas (13). It can also help educators discover “the gap between theory and practice” in education programs (14).
Generally, the CIPP model serves as a versatile and practical framework that can be utilized by a diverse group of individuals, “including evaluators, program specialists, researchers, developers, policymakers, leaders, administrators, committees, and even laypeople”. This model assists evaluators in identifying and improving program weaknesses (7). Overall, utilizing this instrument facilitates the recognition of four key aspects: the identification of significant needs, the justification of the developed program, the effective implementation of the program, and the substantial impact of the program (8).
Objectives
Kerman University of Medical Sciences (Kerman, Iran) has planned to offer medical ethics courses, using various creative educational approaches and a new curriculum, established and reformed by the Medical Ethics and History of Medicine Research Center of Tehran University of Medical Sciences (Tehran, Iran). The CIPP model was employed in this study to examine the quality of this program. This study also aimed to evaluate the psychometric characteristics of the CIPP model.
Methods
Study Design and Setting: This cross-sectional study was conducted on 136 medical students in the clerckship phase at Kerman University of Medical Sciences, Kerman, Iran, during 2022. The participants entered the study using the convenience sampling method. According to a study by Kyriazos (15), a sample size larger than 100 is necessary for a confirmatory factor analysis (CFA) model with three to four indicators per factor. For inclusion in the study, the participants must have completed the medical ethics course, which carries two credits according to the approved curriculum, at the onset of their clerckship period. Additionally, the participants were required to provide oral informed consent. If a student failed to answer more than 10% of the questions, they were excluded from the study.
Data Collection Tools: A researcher-made questionnaire was used to collect data from 136 out of 218 (62.4%) medical students in their clerckship phase at Kerman University of Medical Sciences. It was designed according to the CIPP model structure (7, 8) in line with the medical ethics curriculum in Iran (4) by reviewing the literature and relevant studies (5, 9, 10, 11, 13). This tool focuses on four key questions: “Have important needs been identified?”, “Is the program design justifiable?”, “Is the developed program functioning effectively?”, and “Has the program made a significant impact?”.
Considering the four domains of the model
(i.e., context, input, process, and product), multiple questions designed by the research team were assigned to each domain by an expert panel, including 10 faculty members of medical education and medical ethics. The context, input, process, and product sections of the tool contained thirteen, seven, eight, and nine items, respectively. However, two items of the process domain were removed due to a factor loading below 0.30. The response to each item was scored on a three-point scale (2 for “Yes”, 1 for “No idea”, and 0 for “No”). In the product domain, three items were reverse scored. In addition to the multiple-choice items, the product section of the tool also included three open-ended questions:
Question 38: What strategies can enhance the teaching of medical ethics?”
Question 39: What barriers hinder the achievement of this course objectives?
Question 40: Do you have any suggestions regarding the objectives of the medical ethics course? If so, please share them.
Validity and Reliability of the Questionnaire: To evaluate the construct validity of the questionnaire and verify the structural model proposed by the researchers, a confirmatory factor analysis was conducted. The goodness of fit indices used in this analysis included χ2/df, root mean square error of approximation (RMSEA), standardized root means square residual (SRMR), incremental fit index (IFI), relative fit index (RFI), normed fit index (NFI), and comparative fit index (CFI). Data were analyzed using SPSS version 22. Lisrel version 8.8 was employed for the confirmatory factor analysis.
Ethical Considerations: After receiving the ethical approval code for the study (IR.KMU.AH.1400.019), the questionnaires were completed anonymously and voluntarily. The approximate time to complete the questionnaire was 15-20 minutes, and the researcher was available to provide necessary guidance while the students were filling out the questionnaire.
Results
Based on the content validity assessment, the content validity index of the tool utilized in this study was 0.97, and its content validity ratio was 0.89. In the confirmatory factor analysis, the factor loadings of all items were greater than or equal to 0.3, except for items 30 and 31 related to the product domain, which were removed from the model. Table 1 presents the factor loadings of the items, along with Cronbach’s alpha, composite reliability, and the average variance explained (AVE). Almost all the fit indices of this model in the confirmatory factor analysis were at an acceptable level (χ2/df=1090.83/557=1.96, P=0.001, RMSEA=0.08, SRMR=0.09, CFI=0.93, IFI=0.93, RFI=0.90, NFI=0.9).
Table 2 demonstrates the correlations between different dimensions of the medical ethics education evaluation tool based on the CIPP model. According to this table, all dimensions of the model were significantly correlated. However, in the path analysis, a significant correlation was only found between some of the dimensions.
Discussion
The present study evaluated the psychometric properties of a researcher-made questionnaire for medical ethics education based on the CIPP evaluation model from the perspective of students at Kerman University of Medical Sciences (Kerman, Iran). The results of the exploratory and confirmatory factor analyses showed that the questionnaire can be used to measure the quality of medical ethics education in four dimensions, including the context, input, process, and product of the CIPP model. The questionnaire was found to be valid and reliable for gathering data on students’ experiences with medical ethics education.
To the best of our knowledge, no study has yet evaluated medical ethics training by using the CIPP model. However, some studies have applied other instruments to evaluate medical ethics training programs. For example, a cohort study surveyed 111 students at Glasgow University, using the vignettes of the Ethics in Health Care Survey Instrument (16). Moreover, an interactive online course of ethics education in pediatrics was evaluated by using the Constructivist On-Line Learning Environment Survey (COLLES) (17). Additionally, using a cross-sectional descriptive questionnaire survey, the ethics education programs at Turkish medical schools were reviewed (18). In another study that evaluated the quality of the medical ethics curriculum from the viewpoint of intern students in Mashhad, Iran, the internal quality was assessed using the nine facets of Frances Klein’s model, while the Kirkpatrick model was employed to evaluate its external quality (19).
Table 1. The factor loadings, T values, Cronbach's alpha coefficients, Average Variance Extracted (AVE), and Composite Reliability (CR) of the different dimensions of the medical ethics education evaluation tool using the CIPP model
|
Factor |
Item |
Loading |
T value |
Cronbach's alpha coefficients |
AVE |
CR |
|
Context |
Is the goal of medical ethics education clear to you? |
0.41 |
4.74 |
0.87 |
0.42 |
0.90 |
|
Are instructional resources and materials used in medical ethics courses practical? |
0.51 |
6.16 |
||||
|
Is the volume of work offered in terms of quantity and content appropriate? |
0.40 |
4.55 |
||||
|
Are there appropriate and effective relationships between teachers and students? |
0.42 |
4.90 |
||||
|
Are you satisfied with the way you are evaluated in the medical ethics course? |
0.52 |
6.34 |
||||
|
Participating in medical ethics classes helped me learn how to perform ethical analysis in clinical settings. |
0.73 |
9.68 |
||||
|
Participating in medical ethics classes helped me learn how to use ethical analysis tools in clinical settings. |
0.68 |
8.66 |
||||
|
Attending medical ethics classes helped me understand the importance of respecting patients’ rights, especially in the educational environment. |
0.70 |
9.15 |
||||
|
Participating in the medical ethics course helped me understand the importance of respecting other health services. |
0.66 |
8.36 |
||||
|
Participating in the medical ethics course helped me understand the importance of getting informed consent for effective communication between the doctor and the patient. |
0.61 |
7.68 |
||||
|
Participating in the medical ethics course helped me understand the importance of keeping the patient 's secrets and respecting his/her privacy in building an effective physician-patient relationship. |
0.65 |
8.25 |
||||
|
Participating in the medical ethics course helped me understand the importance of managing conflicts of interest in preserving patient’s confidence. |
0.80 |
10.89 |
||||
|
Participating in the class on medical ethics helped me understand the importance the professional behavior in protecting public trust. |
0.76 |
10.11 |
||||
|
Input |
Are the contents of medical ethics course relevant to the educational goals of the general medical training programs? |
0.62 |
6.23 |
0.76 |
0.41 |
0.83 |
|
Is the content of medical ethics course adjusted to students ' needs? |
0.61 |
6.19 |
||||
|
Do you think that the medical ethics course meets the purpose of teaching this course? |
0.60 |
6.05 |
||||
|
Is the volume of medical ethics training proportionate to the unit's rate? |
0.41 |
4.35 |
||||
|
Do teachers qualify to teach medical ethics? |
0.58 |
5.94 |
||||
|
Are you motivated enough to study medical ethics? |
0.46 |
4.88 |
||||
|
In your opinion, do you can deal with the ethical issues and respond to the patient 's needs? |
0.57 |
5.83 |
||||
|
Process |
Are the materials presented to you in the medical ethics course clear and understandable? |
0.54 |
5.26 |
0.78 |
0.40 |
0.83 |
|
Do medical ethics teachers use auxiliary instructional methods (such as slides, movies, etc.) to better teach? |
0.30 |
3.14 |
||||
|
Do medical ethics professors use various and appropriate activities (such as problem solving, question and answer, group discussion, etc.) to engage the students? |
0.42 |
4.22 |
||||
|
Does your teaching hospital play a role or cooperates with the faculty in medical ethics education? |
0.40 |
4.09 |
||||
|
Are you satisfied with the services of the faculty providing medical ethics training course? |
0.67 |
5.91 |
||||
|
Do you think that the medical ethics course will be useful to you? |
0.70 |
6.10 |
||||
|
Do you think that the educational experience you have gained during medical ethics has been satisfactory? |
0.73 |
6.26 |
||||
|
Participating in the medical ethics course helped me understand the ethical challenges in providing health services. |
0.63 |
5.68 |
||||
|
Table 1. The factor loadings, T values, Cronbach's alpha coefficients, Average Variance Extracted (AVE), and Composite Reliability (CR) of the different dimensions of the medical ethics education evaluation tool using the CIPP model (continue) |
||||||
|
Factor |
Item |
Loading |
T value |
Cronbach's alpha coefficients |
AVE |
CR |
|
Product |
Attending medical ethics classes helped me be more prepared to accept my future role as |
0.50 |
4.13 |
0.71 |
0.40 |
0.80 |
|
According to the medical ethics course materials: if patient tissue samples such as blood |
0.34 |
3.28 |
|
|
|
|
|
According to the medical ethics course materials: in teaching hospitals, students' learning outcome should be the priority, not the patient's safety. However, there is a possibility of mistakes in the teaching hospital. |
0.30 |
2.69 |
|
|
|
|
|
According to the medical ethics course materials: If I am going to talk with my colleagues |
0.61 |
4.85 |
|
|
|
|
|
According to the medical ethics course materials: if an error occurs during the treatment of a |
0.77 |
5.42 |
|
|
|
|
|
According to the medical ethics course materials: If one of my colleagues behaves inappropriately |
0.86 |
6.59 |
|
|
|
|
|
According to the medical ethics course materials: If the patient behaves inappropriately |
0.40 |
3.41 |
|
|
|
|
Table 2: Correlations between different dimensions of the medical ethics education evaluation tool based on the CIPP model
|
Factor |
Item |
Input |
Context |
Process |
Product |
|
Input |
Correlation coefficient |
1.000 |
.776** |
.790** |
.321** |
|
P |
. |
<0.001 |
<0.001 |
<0.001 |
|
|
Context |
Correlation coefficient |
.776** |
1.000 |
.818** |
.501** |
|
P |
<0.001 |
. |
<0.001 |
<0.001 |
|
|
Process |
Correlation coefficient |
.790** |
.818** |
1.000 |
.421** |
|
P |
<0.001 |
<0.001 |
. |
<0.001 |
Bold values indicate a significant correlation in the path analysis.
The CIPP evaluation model has been used to evaluate various educational programs in several studies. For instance, researchers at the Shiraz Medical School (Shiraz, Iran) utilized the CIPP model to evaluate the integrated basic sciences medical curriculum. This assessment focused on the quality of the connection between basic sciences and clinical courses, as well as the method of delivering these courses. The portfolio technique and semi-structured interviews were used to assess the product, and the portfolio technique was used to evaluate the students' learning and reflection. The semi-structured interviews were conducted to assess the students' perceptions of the curriculum effectiveness and the additional changes required to improve it (20). Considering the students’ opinions and suggestions, the results of this study were similar to ours in terms of product assessment.
Moreover, in another study conducted at Babol Dental School (Babol, Iran), the CIPP model was employed to assess educational programs across three departments from the students’ perspective, with a particular emphasis on the attainment of educational objectives (7). The researchers also used the CIPP model to evaluate faculty development programs for 80 clinical faculty members, and the features of the questionnaire were also examined psychometrically (10). Additionally, another study used the CIPP approach to survey 40 public health students for evaluating the externship curriculum of public health courses. The results showed that the CIPP model can be used to improve the scientific content and efficiency of this program (11). It appears that the CIPP evaluation model can determine the effectiveness, shortcomings, and strengths of educational programs.
Conclusion
The psychometric properties of the CIPP evaluation model were acceptable and applicable for medical ethics education programs. Therefore, this model can be effectively utilized to evaluate the quality of medical ethics education in a comprehensive and precise manner.
- Miles SH, Lane LW, Bickel J, Walker RM, Cassel CK. Medical ethics education: coming of age. Acad Med. 1989 Dec;64(12):705-14. doi: 1097/00001888-198912000-00004. [PMID: 2686674]
- Perkins HS, Geppert CM, Hazuda HP. Challenges in teaching ethics in medical schools. Am J Med Sci. 2000 May;319(5):273-8. doi: 1016/S0002-9629(15)40752-9. [PMID: 10830549]
- Core Committee, Institute for International Medical Education. Global minimum essential requirements in medical education. Med Teach. 2002 Mar;24(2):130-5. doi: 1080/01421590220120731. [PMID: 12098431]
- Asghari F, Mirzazadeh A, Samadi A, Safa AN, Jafarian A, Farahani AV, et al. Reform in medical ethics curriculum: a step-by-step approach based on available resources. J Med Ethics Hist Med. 2011; 4:8. [PMID: 23908750] [PMCID: PMC3713913]
- Tabari M, Nourali Z, Khafri S, Gharekhani S, Jahanian I. Evaluation of educational programs of pediatrics, orthodontics, and restorative departments of Babol dental school from the perspective of the students based on the CIPP model. Caspian J Dent Res. 2016;5(2):8-16.
- Vatankhah S, Salemi A. A Study on Evaluation System of Hospitals Affiliated to Iran University of Medical Sciences Using Balance Score Cards, Tehran, Iran. Journal of Health Administration. 2010; 12(38): 49-58. [In Persian]
- Stufflebeam DL, Coryn CLS. Daniel Stuffelbum’s CIPP Model for Evaluation. In: Stufflebeam DL, Coryn CLS, eds. Evaluation Theory, Models, and Applications. 2nd ed. San Francisco CA: Jossey‑Bass; 2014:309-13.
- Stufflebeam DL. The Relevance of the CIPP Evaluation Model for Educational Accountability. J Res Develop Edu. 1971; 5:19-25.
- Zhang G, Zeller N, Griffith R, Metcalf D, Williams J, Shea C, et al. Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement. 2011; 15(4); 57-84.
- Salajegheh M. Design, implementation, and evaluation of a long-term educational faculty development program for clinical faculty. J Med Educ Dev. 2022; 15(46): 54-60. doi: 52547/edcj.15.46.54.
- Mazloomy Mahmoudabad SS, Moradi L. Evaluation of Externship curriculum for public health Course in Yazd University of Medical Sciences using CIPP model. Educ Strategy Med Sci 2018; 11(3): 28-36. [In Persian]
- Hemati Z, Irajpour A, Allahbakhshian M, Varzeshnejad M, AbdiShahshahani M. Evaluating the Neonatal Intensive Care Nursing MSc Program Based on CIPP Model in Isfahan University of Medical Sciences. Iran J Med Educ. 2018; 18: 324-32. [In Persian]
- Nasr VG, Guzzetta NA, Mossad EB. Fellowship training in pediatric cardiac anesthesia: history, maturation, and current status. J Cardiothorac Vasc Anesth. 2019 Jul;33(7):1828-1834. doi: 1053/j.jvca.2018.08.016. [PMID: 30243872]
- Waters LD. (Dissertations). Outcome Measures of Graduates of a Master’s Degree in Nursing Program: Assessing Congruence of Perceptions of Graduates and Employers. Newark, Delaware, USA: University of Delaware; 1987.
- Kyriazos TA. Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. 2018; 9(08): 2207. doi: 4236/psych.2018.98126.
- Goldie J, Schwartz L, McConnachie A, Morrison J. The impact of three years' ethics teaching, in an integrated medical curriculum, on students′ proposed behaviour on meeting ethical dilemmas. Med Educ. 2002 May; 36(5): 489-97. doi:1046/j.1365-2923.2002.01176.x. [PMID: 12028400]
- Kidszun A, Forth FA, Matheisl D, Busch F, Kaltbeitzel L, Kurz S. Ethics education in pediatrics: Implementation and evaluation of an interactive online course for medical students. GMS J Med Educ. 2022 Nov 15;39(5): Doc55. doi: 3205/zma001576. [PMID: 36540566] [PMCID: PMC9733484]
- Volkan Kavas M, Isil Ulman Y, Demir F, Artvinli F, Şahiner M, Demirören M, et al. The state of ethics education at medical schools in Turkey: taking stock and looking forward. BMC Med Educ. 2020 May 24;20(1):162. doi: 1186/s12909-020-02058-9. [PMID: 32448274] [PMCID: PMC7245803]
- Mohebbi Amin S, Rabiei M, Keizoori A H. A review of students’ evaluation of the medical ethics curriculum. Iranian Journal of Medical Ethics and History of Medicine. 2015; 8(3):77-86. [In Persian]
- Rooholamini A, Amini M, Bazrafkan L, Dehghani MR, Esmaeilzadeh Z, Nabeiei P, et al. Program evaluation of an integrated basic science medical curriculum in Shiraz Medical School, using CIPP evaluation model. J Adv Med Educ Prof. 2017;5(3):148-154. [PMID: 28761888] [PMCID: PMC5522906]
)