Background

The process of becoming a physician and developing a professional identity is a complex and multifaceted process. The initial years of medical school are particularly challenging and can significantly influence this developmental path (1, 2). During this period, students must navigate new social networks, environments, and responsibilities, including adjusting socially to peers and roommates, acquiring essential academic skills like time management and problem-solving, familiarizing themselves with the culture, policies, and expectations of the university and medical profession, and taking responsibility for self-care, among many other demands (3). To thrive in medical school, incoming students need to rapidly adapt to an educational culture and learning environment that differ substantially from their previous experiences (4). Furthermore, due to the interpersonal nature of the medical profession, which requires ongoing communication with patients and their families, medical students and physicians need to develop a range of social and emotional skills (5), including effective communication, conflict resolution, and negotiation, as well as higher-order abilities such as decision-making, clinical reasoning, and critical thinking (6). However, the selection of medical students is typically based primarily on academic and scientific criteria, often overlooking non-academic skills (7). Consequently, medical schools admit students from diverse social and economic backgrounds, with varying personality traits, psychological characteristics, and non-academic competencies (8). In the early years of medical education, students encounter significant stress due to multiple simultaneous challenges. These include fear of academic failure, concerns about ridicule or negative feedback, feelings of alienation, peer and parental pressures, and the need to adjust to dormitory life (8, 9). Additionally, students must manage the substantial increase in the scope and rigor of the medical curriculum compared with secondary education, limited understanding of academic expectations, heavy workloads, and insufficient time for rest. Altogether, these factors make the initial years of medical training particularly demanding and stressful for many students (2). Thus, one of the key responsibilities of colleges and universities is to help students navigate these challenges and prepare them for a successful academic path (10). Over the past few decades, rising concerns about the quality of medical care and health services have drawn the attention of educational policymakers and curriculum developers to innovative approaches in medical education worldwide. In this context, alongside disciplinary knowledge and technical skills, the development of soft skills and competencies essential for effective professional practice has become a central focus for medical education planners (11). Consequently, curricular interventions have been implemented internationally to ensure that the range of competencies necessary for professional performance and effective interaction within healthcare settings is systematically incorporated into undergraduate medical education (12-14). Iran is not exempt to this global trend. Consequently, starting in the first semester of the 2015-2016 academic year, Medical Etiquette Courses were incorporated into the basic sciences phase of the undergraduate medical curriculum. With the introduction of new educational programs, however, important questions arise regarding their quality, learner satisfaction, and educational effectiveness. Educational evaluation is a complex and structured process involving the collection and analysis of relevant information to make informed judgments about the quality and value of an educational program. Such evaluations enable identification of curricular strengths and weaknesses and assess program effectiveness and efficiency after implementation. Based on these assessments, decisions can be made about revising, continuing, or expanding the program. In educational settings, evaluation is carried out through the use of structured criteria designed to determine the value, quality, utility, performance, and overall significance of a program (15). Several models have been proposed for evaluating educational programs. In this study, the CIPP evaluation model, one of the most recognized and influential frameworks, was employed. Developed in the 1970s by Stufflebeam at the Center for Studies in Evaluation at Ohio University (16), the Context, Input, Process, and Product (CIPP) model provides a comprehensive and systematic approach to program evaluation, serving as the theoretical foundation for this research. By assessing the design, implementation, and outcomes of educational programs, the CIPP model helps educators and policymakers determine how well programs achieve their objectives while highlighting strengths and areas for improvement. Accordingly, this study aimed to examine the validity and reliability of a researcher-developed evaluation instrument based on the CIPP model for assessing Medical Etiquette Courses in the undergraduate medical curriculum at Kerman University of Medical Sciences and Shahid Beheshti University of Medical Sciences in Iran.

Objectives

The purpose of this research was to identify the validity and reliability of a researcher-made evaluation scale, grounded in the principles of Stufflebeam’s CIPP Evaluation Model, for assessing “Medical Etiquette Courses” in the Iranian undergraduate medical curriculum.

Methods

Study design: This descriptive study was conducted at two Iranian universities of medical sciences with the aim of developing a valid and reliable evaluation instrument, based on the CIPP evaluation model, for assessing Medical Etiquette Courses within the undergraduate medical curriculum in Iran.

Setting: Two medical schools, Kerman University of Medical Sciences, Kerman, Iran, and Shahid Beheshti University of Medical Sciences, Tehran, Iran, collaborated in the development of the evaluation scale. At both institutions, Medical Etiquette Courses are implemented into the basic sciences phase of the undergraduate medical curriculum. Since the study aimed to examine the psychometric properties of the scale, the sample size was determined using a ratio of approximately five participants per item (17). Ultimately, 171 medical students in their clerkship phase, who had previously completed the Medical Etiquette Courses during the basic sciences phase, were included. The study was conducted in the following steps:

Step 1: Questionnaire item generation

To develop the questionnaire items, initially, the literature related to curriculum and course evaluation was reviewed, and then based on the latest Stufflebeam model (18) and the general specifications of the "Medical Etiquette Courses" in undergraduate medical curriculum of Iran, appropriate items were designed for each domain of questionnaire by the research team. Then, through three sessions of group discussion, the research team reached a consensus regarding the content, structure, order and sequence, and the number of items in each domain. In this way, 32 items were compiled for four evaluation domains: context (7 items), input (5 items), process (9 items), and output (11 items).

Step 2: Internal consistency

A pilot study was conducted with 20 medical students to assess the internal consistency using Cronbach’s alpha. According to Jerry Sigudla and Jeanette E. Maritzany (19), value above 0.8 is a good reliability, between 0.6 and 0.8 is an acceptable reliability, and a Cronbach’s alpha coefficient value below 0.6 is an unacceptable reliability for exploratory research. These students were incorporated into the main study.

Step 3: Face validity

Five medical students were independently asked to review the translated questionnaire and provide feedback on the clarity, appropriateness, and relevance of the items. Based on their feedback, several items were reworded to improve clarity; however, no items were removed.

Step 4: content validity

At this step, the questionnaire was evaluated by ten faculty members in medical education and medical ethics from both universities, all of whom had experience teaching medical etiquette and medical ethics courses. The experts assessed the relevance of the items, and the Content Validity Index (CVI) and Content Validity Ratio (CVR) were subsequently calculated. In addition, the research team conducted a group discussion during which each item was systematically reviewed and refined with respect to level of difficulty, appropriateness, and potential ambiguity, as well as the need for item deletion, modification, or consolidation.

Step 5: convergent validity

Convergent validity was evaluated to determine whether items that are theoretically related are also empirically correlated. Composite Reliability (CR) values greater than 0.7 supports the convergent validity of the instrument.

Step 6: Construct validity

To assess the adequacy of the sample size and the suitability of the data for factor analysis, exploratory factor analysis (EFA) was conducted using the Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test of sphericity. A KMO value of 0.80 or higher was considered indicative of adequate sampling, and a significant Bartlett’s test (P < 0.05) supported the factorability of the data. Although EFA was used to confirm the suitability of the data (e.g., KMO, Bartlett’s test, item correlations), the number of factors retained was determined based on a theoretical framework and expert panel review, rather than statistical criteria such as eigenvalues or scree plots. Subsequently, confirmatory factor analysis (CFA) was performed to evaluate the fit of this theory-driven model, and relevant goodness-of-fit indices were calculated (20). Following Bryant and Stone (21), reporting a balanced set of indices, one absolute fit index, one incremental fit index, one parsimonious fit index, and the χ² statistic, was considered sufficient to provide a robust evaluation of model fit.

Statistical Analysis: Data analysis was performed using SPSS version 22. Descriptive statistics, including mean, standard deviation, frequency, and percentage, were used to summarize the data. Both exploratory and confirmatory factor analyses were conducted. To enable comparison across different domains, item scores were transformed to a 0–100 scale, resulting in domain scores ranging from 0 to 100. Based on previous research (21), scores were categorized as follows: ≤50 as undesirable, 51–70 as fairly desirable, and 71–100 as desirable.
Chi-square tests were used to compare these frequencies across groups, with a significance level set at p ≤ 0.05. Confirmatory factor analysis was conducted using LISREL version 8.8 and PLS version 3.

Ethical considerations: After obtaining ethical approval with the code IR.NASRME.REC.1400.173, the questionnaires were administered anonymously and voluntarily. It took participants approximately 15-20 minutes to complete the questionnaire, and the researcher was present during this time to guide the students if needed.

Results

A total of 171 medical students participated in the study. Their mean age was 20.88 ± 1.32 years, and the majority were male (55.4%), from Shahid Beheshti University of Medical Sciences (63.7%), and in their third year (86.9%).

As mentioned earlier; after shaping the item pool and conducting three focus group discussions, 32 items were developed across four domains: context (7 items), input (5 items), process (9 items), and output (11 items).

Validity of the instrument

1. Content validity: Ten faculty members from both universities, experienced in teaching medical etiquette and medical ethics, assessed the instrument's items for relevance. The Content Validity Ratio (CVR) was 0.96, and the Content Validity Index (CVI) was 0.97, both falling within the acceptable range.
2. Construct validity: In exploratory factor analysis, the Kaiser-Meyer-Olkin (KMO) measure was 0.86, indicating a satisfactory sample size, and Bartlett's test of sphericity was statistically significant (df = 496, χ²=2246.75, P<0.001), confirming the appropriateness of factor analysis. The average variance explained (R²) for the four constructs ranged from 24% to 34%, with an overall average of 29%, demonstrating that the model provides a reasonable and acceptable explanation for the observed variables. Confirmatory Factor Analysis (CFA) was performed to assess the fit of a theoretical model recommended and refined by our expert panel. All items in the CFA had factor loadings above 0.3. Items with loadings between 0.3-0.4 were theoretically justified and did not compromise the scale's integrity in other analyses, such as item analysis.

Factor loadings and composite reliability are presented in Table 1. In the structural equation method, statistically significant correlations (Figure 1) were found and almost all fit indices of the model were in the acceptable range (Table 2).

Table 3 shows the central and dispersion statistics and the level of desirability of different domains of CIPP model for evaluating the current state of teaching medical etiquette courses in medical students.

The frequency of the desirability levels in different domains of the CIPP model did not show a statistically significant difference based on gender. (P>0.05) This frequency did not show any significant statistical difference based on the university, except for the context, in which, the frequency of the desirable level was significantly higher in Kerman University (71%) than Shahid Beheshti (47.7 %) (P=0.009).

Among fourth year students, the frequency of the undesirable, relatively desirable, and desirable levels of input domain were 13.6 %, 31.8 %, and 54.5 % in fourth-year students, and 44.5 % ,21.9 %, and 33.6 % in third year students, which were statistically significant (P=0.02). The other three dimensions of the model had no statistically significant difference in this regard.

Table 1. Factor Loadings of Items, Cronbach's Alpha, and Composite Reliability (CR) for evaluating the current state of teaching medical etiquette courses by using the CIPP Model

Table 2. Appropriate fit indicators for evaluating the current state of teaching medical etiquette courses by using the CIPP model

Table 3. Central and dispersion statistics and the level of desirability of different domains of the model

Discussion

The present study investigated the psychometric characteristics of the researcher-made questionnaire based on the CIPP model for evaluating the “Medical Etiquette Courses” of the undergraduate medical curriculum. The findings from both exploratory and confirmatory factor analyses indicated that the questionnaire is suitable for assessing the quality of Medical Etiquette Courses across the four CIPP dimensions: Context, Input, Process, and Product.

The instrument demonstrated acceptable validity and reliability to gather students’ perspectives on their experiences in Medical Etiquette Courses. To the best of our knowledge, the CIPP model has not been used for evaluating Medical Etiquette Courses or other soft skills programs in Iran. Nevertheless, several studies have employed alternative tools or approaches to assess soft skills training and have reported their experiences and outcomes.

Soft skills encompass both interpersonal and intrapersonal abilities (22, 23) and complement scientific and technical competencies (24). Intrapersonal skills relate to an individual’s capacity to manage personal success, including time and stress management and creative thinking. Interpersonal skills involve managing relationships with others to enhance performance and include competencies such as motivation, leadership, and effective communication (25). Various educational strategies have been employed to develop soft skills in higher education, particularly in medical education. Among these, problem-based learning (PBL) has been widely implemented as a curriculum innovation (26). By engaging students in problem-solving activities, PBL fosters critical thinking, idea generation, and the acquisition of knowledge, skills, and behaviors essential for competent medical practice (27, 28). Moreover, research has shown that communication skills training can enhance empathy among medical students (29). Together, these findings highlight the importance of not only teaching soft skills but also systematically evaluating them within medical education programs (30). A study investigating and prioritizing soft skills in the dental profession, based on their relevance to professional success and quality, found that both faculty and students considered six key areas essential: moral-professional values, artistic skills, communication skills, interpersonal skills, and technical expertise (31). Similarly, in the present study, the product domain which reflects the perceived usefulness of the course from the students’ perspective indicated that students recognized the teaching of soft skills as important and relevant to their future professional roles.

One study found that first-year students demonstrated the greatest improvement in awareness, particularly when exposed to early soft skills training. The results showed significant enhancement in students’ communication skills and confidence in interacting with patients following the training. Similar findings have been reported in other studies as well (32). The Faculty of Medical Sciences at the University of Malaysia (USM) launched the Community and Family Case Study Program (CFCS) in 1981 (33), which has been continuously developed since its inception. The CFCS is a community-based educational initiative that combines research methods with community engagement, aiming to equip students with the knowledge, values, and skills necessary to become competent and ethical medical professionals (33). In a related study, a series of active learning activities were conducted in 2016 for third-year medical students, involving 330 participants. The study aimed to evaluate whether these interventions could enhance students’ soft skills. Students completed a self-assessment questionnaire, and their performance was additionally evaluated by four peers. These studies highlight the importance of student-centered approaches in teaching soft skills. Similarly, the process domain of our questionnaire evaluated medical students’ perspectives on the teaching strategies employed. In our study, only about 37% of students reported satisfaction with the teaching methods, indicating a need for further improvement in this area. Supporting this, a study at the University of Malaysia found that cultivating personal skills and values provides a meaningful learning experience and promotes the development of personal competence (34).

A study at the University of Pretoria examined the professional development of medical students, focusing on physician–patient interaction skills and professional socialization. Using Grounded Theory, researchers explored students’ perspectives on the role of mentors in developing soft skills. Data were collected via focus groups, in-depth and personal interviews, and biographical drawings, then analyzed inductively. The results showed no significant differences between student groups. Students described effective role models as physicians who are clinically and academically skilled, prioritize patient care, demonstrate strong interpersonal abilities, and inspire others. Conversely, weak role models were seen as negatively impacting students and undermining the ethical climate and learning environment (35). In the United Kingdom, a simulated training program was created to prepare medical students with the non-technical skills needed for their roles as junior doctors. A study evaluating the program assessed students’ perceptions of teaching quality, key learning factors, and readiness for future clinical duties. Using a mixed-methods design with questionnaires and focus groups, the study emphasized the benefits of simulated learning environments, especially those that closely mirror real clinical settings and allow students to practice essential skills (36). Students who participated in the Medical Etiquette Courses at Kerman and Shahid Beheshti Universities of Medical Sciences showed improvements in eleven areas based on peer evaluations: information presentation, technological skills, creativity, communication, leadership, life-planning, adaptability, self-reliance, humility, and punctuality. The authors concluded that, even without a formal soft skills curriculum, the active learning activities implemented during the pre-clinical phase effectively enhanced multiple essential competencies for medical professionals. They therefore recommended incorporating such active learning approaches into the undergraduate medical curriculum (37). The CIPP evaluation model has been widely used to assess various educational programs, including several studies in Iran. For instance, it has been applied to evaluate the doctoral program in reproductive health (38) and the educational curriculum at Kerman Medical School (39). The model has also been applied in the development and assessment of evaluation tools for medical ethics education (40). In this study, we created a questionnaire based on the CIPP framework to evaluate the effectiveness of the Medical Etiquette Course in promoting soft skills among Iranian medical students. Our results further suggest that students generally perceive the course positively in terms of its impact on their professional development.

Limitations: One limitation of this study is the sample size of fewer than 200 participants. This is partially offset by the relatively simple model, which includes multiple indicators per factor, no missing data, and mostly factor loadings above 0.6. Another limitation is the use of convenience sampling, which may increase the risk of selection bias and limit the generalizability of the findings. Additionally, reliance on self-report measures could introduce social desirability bias or inaccuracies. Despite the instrument showing acceptable psychometric properties, two points deserve attention: first, several items were retained despite low factor loadings, reflecting a compromise between theoretical completeness and statistical robustness, suggesting future research might refine or reword these items; second, other forms of validity assessment, such as discriminant validity, were not evaluated in this study.

Conclusion

The psychometric evaluation of the CIPP-based instrument for Medical Etiquette Courses showed acceptable reliability and validity, supporting its use for assessing these training programs. The CIPP model thus provides a strong framework for systematically and meaningfully evaluating the quality of medical etiquette education. By incorporating continuous feedback, this approach facilitates evidence-based decision-making and promotes sustainable educational development. Future research could apply this instrument across multiple academic centers with larger, randomly selected cohorts and explore its use for longitudinally assessing improvements in the medical etiquette curriculum.