Background

Improving the quality of the care system is an important factor affecting patient health and safety (1, 2). In this context, the care system needs efficient nurses equipped with problem-solving and clinical decision-making skills (1, 3). As appropriate clinical training can enhance self-efficacy (4), preparing students as future nurses is crucial for recognizing patients' educational needs and providing safe care (5, 6).

Bandura defined self-efficacy as a person's belief in their ability to perform effectively in their roles (7). Self-efficacious individuals adapt to the requirements of specific situations, whereas those with low self-efficacy face serious challenges when performing certain tasks (1). For this reason, nursing students with higher self-efficacy are more willing to engage in nursing care (8).

Self-efficacy in the clinical performance of nursing students reflects the educational standards of nursing schools (4). Improving the knowledge, skills, and attitudes of nursing students is considered vital to ensure safe practice and to uphold clinical standards through training and proper evaluation of clinical practice (9, 10). Therefore, it is necessary for nursing instructors to enhance students' professional knowledge and skills via innovative teaching and evaluation methods (11). In this context, nursing instructors focus on adopting teaching and evaluation methods that integrate students' theoretical knowledge with clinical skills in practical settings. One of the main approaches is to conduct a pre-internship exam using an objective structured clinical examination (OSCE) (12, 13).

In the new nursing curriculum, final-year undergraduate students are present in the hospital and at the patient's bedside. Therefore, improving skills such as self-efficacy through workshops is necessary. In addition, evaluation via pre-internship tests can assess hidden skills, such as self-efficacy, provide timely and appropriate feedback, and increase students' awareness of their strengths and weaknesses in clinical settings. This approach gives students an opportunity to work on their skills and competence before entering clinical environments (14). In fact, pre-internship tests and workshops assess skills that students cannot acquire through written exams (15, 16).

In Iran, nursing students participate in a comprehensive exam before graduation, and some nursing schools conduct an OSCE pre-internship exam prior to the internship course (10).

OSCE refers to the evaluation of students' competence using simulated clinical scenarios (17) that focus on observable behaviors (18). Globally, OSCE is used in nursing curricula to measure clinical competence; nevertheless, it is a relatively new addition to Iran's nursing curriculum (10). The simulated nature of OSCE allows students to envision the clinical environment and understand the practical application of their knowledge (17). It also provides an opportunity for students to practice clinical decision-making skills in a safe setting, without compromising patient safety (19). Dr. Ronald Harden first introduced OSCE in
1975 to assess the clinical competence of medical students (20). OSCE is more realistic than written tests and serves as an alternative for evaluating clinical qualifications, with less subjectivity than clinical observation. In OSCE, students are assessed by external examiners as they progress through a series of structured stations with standardized patients to test specific skills such as interviewing, communication, clinical judgment, and physical examination (21).

Considering that self-efficacy is a vital component for independent nursing practice, it seems necessary to develop and implement educational programs that strengthen self-efficacy. Although the effectiveness of educational programs on various outcomes has been proven, their effects on self-efficacy as a key outcome have been less evaluated (7).

Objectives

The present study was conducted with the aim of determining the effect of the design, implementation, and evaluation of the OSCE pre-internship test on self-efficacy in the clinical performance of nursing students before entering the internship.

Methods

Design & Participants: This study was semi-experimental with a one-group pretest-posttest design, conducted on undergraduate nursing students at Qazvin University of Medical Sciences. Inclusion criteria included sixth-semester nursing students, willingness to participate in the study, and obtaining a passing grade in all theoretical and clinical courses. In total, 56 students participated in the study via census.

Intervention

The research was conducted in three steps: design, implementation and evaluation.

First step: design

At this step, scientific and executive committees were created with specific job descriptions. The members of the scientific committee included the dean of the nursing faculty, the vice president of education, the director of the faculty's education development office, the directors of the educational groups, and a faculty member responsible for conducting the exam. All members of the scientific committee had participated in at least one OSCE familiarization workshop. The main tasks of the scientific committee for the exam included providing the table of specifications, checklists, the number of stations and their content, determining the members of the executive committee, establishing the passing score, and supervising the correct implementation of the exam. The members of the executive committee included the person in charge of conducting the pre-internship test, the person in charge of the education department, the administrative-financial deputy of the faculty, and the examiners of the stations. The tasks of the executive committee included determining the exact time of the exam, scheduling the educational workshops, developing the scenario for each station, ensuring the timely registration of grades for all theoretical courses and internships, preparing the final list of students allowed to participate in the exam, and preparing the exam environment. Analyzing and announcing the results of the exam to the Vice-Chancellor no later than one week after the exam was the responsibility of the executive committee. In addition, the executive committee was obliged to inform the managers of the educational groups about the list of students who failed to achieve a passing score on one or more stations, determine the type and amount of compensatory clinical courses, and report on the planning and implementation of these courses to the vice president of education.

The pre-internship test via OSCE was held at eight stations after the end of the sixth semester exams. The stations included Station 1 (physical examination), Station 2 (airway suction), Station 3 (nasogastric tube insertion), Station 4 (pressure ulcer dressing), Station 5 (infant cardiopulmonary resuscitation), Station 6 (patient education), Station 7 (vaccination), and Station 8 (injections). According to the content of each station, mannequins and simulators were used. Students were asked to demonstrate specific clinical skills at each station. The criteria for passing the test were to acquire at least 70% of the evaluation checklist score in each of the stations. In the case of failure at any station, after coordinating with the director of the nursing department, the student was required to complete an internship unit (equivalent to 51 hours) under the supervision of a professor in the relevant department and obtain a passing grade in the course's practical exam. Each student could participate in the exam for each section a maximum of three times, and in the event of failure to obtain a passing score on the third attempt, the decision regarding the student's academic status was made by the educational council of the college and university.

The second step: implementation

Before the pre-internship exam (pre-test and post-test), a briefing session was held by the member of the academic staff responsible for the exam in the presence of all students. In the briefing meeting, the approvals of the executive committee meeting, the purpose and necessity of the test, the manner, frequency, time, and place of the test, the characteristics of the stations and their number, the rotation order of the students in the stations, the test resources, the evaluation method, and the criteria for passing the test were all explained. The OSCE pre-internship test was conducted at two times: before and after the workshops. First, the OSCE pre-test was conducted in eight stations. After the completion of student evaluations, educational workshops were held. Following the workshops and two weeks after the pre-test, the OSCE post-test was conducted in eight stations. The purpose of the training workshops was to outline the standards of procedures and the basic principles of care. In the workshops, the content and educational scenarios of each station were deliberately not mentioned. During the pre-exam and post-exam phases, the examiner of each station designed a scenario related to the corresponding station using nursing textbooks. The content validity of the scenarios and the agreement on the items of the checklists were established during several meetings with the OSCE executive committee members and seven faculty members, and the necessary revisions were made. For scoring in the checklists, each item was assigned three points (not performing the procedure = zero, performing the procedure to some extent = one, and performing the procedure completely = three). On the day of the exam (pre-test and post-test), after providing the necessary explanations, students' personal equipment and cell phones were collected by the supervisors, who were part of the administrative staff of the exam center. The students were called to the stations from the large hall of the quarantine center in groups of eight. The test was conducted in the simulation and clinical skills training center of Qazvin University of Medical Sciences under completely standard conditions. This center is equipped with 16 separate rooms as training stations, numerous mannequins and educational moulages, a dressing room, a large hall for group work or quarantine, as well as a central control room. The time allocated to each station was six minutes. Students took turns moving from one station to another, where a clinical scenario was presented at each station. Students' skills at each station were evaluated by the examiner faculty members using a checklist.

The third step: evaluation

The Kirkpatrick model was used to evaluate the effect of the OSCE pre-internship test. This model proposes a four-stage evaluation that includes reactions, learning, behavior, and results. The first level describes the attitudes and satisfaction of the participants regarding the learning activity, which can be assessed via a survey of the participants in order to evaluate their subjective response to the test. The second level measures the change in the learning of the participants, which is evaluated by better performance in the post-test or improved performance in the exams. In addition, the correlation between the OSCE score and clinical performance can be examined. The third level assesses the change in clinical behavior and performance, which is evaluated by clinical educators. The fourth level represents the final result, which educational design leads to improved patient care. It is usually difficult to measure this level (22, 23).

1- Examining students’ reaction: The attitude and satisfaction of the students toward the OSCE exam were evaluated using a questionnaire to measure the students' feedback. This questionnaire was designed based on a literature review and then presented to 10 nursing lecturers of Qazvin University of Medical Sciences for content and face validity. After approving the validity, a final 17-item self-report questionnaire with a two-point Likert scale (agree, disagree) was designed.

2- Students' learning: The students' learning was evaluated by comparing the total pre-internship score and the score of each station separately in the pre-test and post-test. The correlation between the pre-internship test score in the post-test and the self-efficacy scores for the students’ clinical performance was evaluated immediately after the post-test and three months later. The correlation between the pre-internship test score in the post-test and the students’ 7th-semester GPA was also measured. Self-efficacy in the clinical performance of the nursing students as an outcome variable was evaluated and compared at three time points of measurement: before the pre-test, immediately after the post-test, and three months after the post-test (in the middle of the 7th semester) using the self-efficacy questionnaire in clinical practice. This questionnaire was designed and psychometrically evaluated in 2009 by Cheraghi and colleagues in Iran. The questionnaire has 37 questions in four subscales of patient assessment (12 items), nursing diagnosis and planning (9 items), implementation (10 items), and evaluation (6 items). Each item is given a score from 1 (I'm not sure) to 4 (I'm sure), and the total score ranges from 37 to 148, where a higher score indicates greater self-efficacy. Cronbach's alpha coefficient (ɑ = 0.96) showed adequate internal consistency of the final instrument, and the retest with a two-week interval indicated adequate stability of the instrument (r = 0.94) (24). In the present study, the Cronbach's alpha coefficient of the questionnaire was 0.84, and the retest showed its appropriate stability (r = 0.91).

3- Change in students' behavior: In this research,
the correlation was examined between pre-internship test scores in the post-test and the self-efficacy assessment of the students’ clinical performance at the third time point, measured three months after the post-test. In addition, self-efficacy scores for the students’ clinical performance were compared at three measurement points.

4- Results: In the present study, the relationship between the pre-internship test score in the post-test and the students’ 7th-semester GPA was evaluated. In addition, the correlation between the students' 7th-semester GPA scores and their self-efficacy scores in clinical practice was examined immediately after the post-test and three months later (Figure 1).

Data Analysis: Mean and standard deviation were used to describe quantitative variables, while frequency and percentage were used to describe qualitative variables. Data distribution was determined via histogram and the Kolmogorov-Smirnov test. Spearman correlation, Wilcoxon tests, and Friedman's repeated measures tests were used to analyze the data. Data were analyzed using SPSS software version 26 (IBM Corporation, Armonk, NY). A p-value of > 0.05 was considered the significance threshold.

Results

In this study, 56 students were evaluated. Of the participants, 53.6% (30 people) of the participants were female, with an average age of 23.0 ± 3.7 years and a GPA of 18.1 ± 1.2 in the 7th semester.

Table 1 shows the students' feedback about the pre-internship test. The majority had a positive understanding of the test. The students perceived the OSCE pre-internship examination as a useful educational test and learning opportunity that adequately linked theory to clinical practice. Most of the students stated that the test included the content and clinical skills taught in the previous semesters. In addition, the students considered the test to be fair. They stated that they were fully informed about the conduct, nature, objectives, and importance of the test before the test.

The participants described the behavior of the examiners, the number of stations, and the location of the exam as appropriate. Most of the students pointed out that the test was less stressful than traditional tests. Nevertheless, 30.4% of the students considered it a stressful test.

The findings showed a significant increase in the mean scores of eight stations as well as the score of each station separately in the pre-test and post-test (P < 0.001). The highest and lowest mean scores were related to the physical examination stations (3.51 ± 0.63) and vaccination (1.83 ± 0.20), respectively (Table 2).

The findings showed an increase in the mean total score of self-efficacy in clinical practice before the pre-test (79.44 ± 10.29), immediately after the post-test (100 ± 15.98), and three months after the post-test (133.96 ± 15.53). The increase in the mean scores across all subscales and the total score of self-efficacy in clinical practice was significant (P < 0.001) (Table 3 and Figure 2).

Table 4 presents the relationship between the students' self-efficacy scores and their pre-internship test scores after the post-test. As expected, before performing the pre-test and participating in the workshops, there was no significant relationship between the self-efficacy scores in clinical practice and the pre-test (P = 0.405); however, a positive correlation was found between scores immediately after the post-test (P < 0.001) and three months later (P = 0.007).

The participants described the behavior of the examiners, the number of stations, and the location of the exam as appropriate. Most of the students pointed out that the test was less stressful than traditional tests. Nevertheless, 30.4% of the students considered it a stressful test.

The findings showed a significant increase in the mean scores of eight stations as well as the score of each station separately in the pre-test and post-test (P < 0.001). The highest and lowest mean scores were related to the physical examination stations (3.51 ± 0.63) and vaccination (1.83 ± 0.20), respectively (Table 2).

The findings showed an increase in the mean total score of self-efficacy in clinical practice before the pre-test (79.44 ± 10.29), immediately after the post-test (100 ± 15.98), and three months after the post-test (133.96 ± 15.53). The increase in the mean scores across all subscales and the total score of self-efficacy in clinical practice was significant (P < 0.001) (Table 3 and Figure 2).

Table 4 presents the relationship between the students' self-efficacy scores and their pre-internship test scores after the post-test. As expected, before performing the pre-test and participating in the workshops, there was no significant relationship between the self-efficacy scores in clinical practice and the pre-test (P = 0.405); however, a positive correlation was found between scores immediately after the post-test (P < 0.001) and three months later (P = 0.007).

In Table 5, the relationship between the students' 7th-semester GPA and the desired variables is presented. A significant positive relationship was found between the pre-internship exam score in the post-exam stage and the students' 7th-semester GPA. Therefore, it seems that the pre-internship test in the 6th semester, along with educational workshops, can improve the academic achievement of students in the 7th semester. In addition, a significant positive relationship was observed between self-efficacy in clinical practice immediately after the post-test and three months later and the 7th-semester GPA. This suggests that the increase in self-efficacy in clinical practice led to improved performance among the nursing students in the 7th semester. Over time, many factors can play a role in enhancing students' performance; therefore, this finding requires repeating similar studies with longer follow-up periods (Table 5).

Discussion

The main goal of nursing education is to train competent nurses and help students increase self-confidence and self-efficacy. The findings showed a significant increase in the mean scores of all stations and the scores of each station separately in the pre-test and post-test. Therefore, the effectiveness of the OSCE pre-internship test and educational workshops on the students' learning was confirmed. In a systematic review, Vincent et al. reported that students and instructors worldwide consider the correct implementation of the OSCE to be a more realistic and valid method of assessing student clinical ability (16). Kirwan et al., in a meta-synthesis study of midwifery students' views on the OSCE exam, stated that the OSCE should be administered when students have learned basic nursing skills before entering clinical practice. This will help them remember the main clinical techniques at the patient's bedside. In addition, they emphasized the importance of providing opportunities for students to learn to practice with other students under the supervision of the instructor, familiarizing themselves with the equipment in the OSCE environment, and conducting a mock OSCE test to prepare students for the main test (17).

In the present study, the OSCE test was used as a learning and evaluation tool. A mock OSCE test was conducted as a pre-test to evaluate the clinical performance of the students before participating in the OSCE pre-internship test (post-test), which served as the main test, and feedback was given to the students. Between the pre-test and post-test, in order to improve learning, practice sessions were held in the OSCE environment, and educational workshops were conducted by the examiners. These preparations led the students to be satisfied with the pre-internship test and to consider it an opportunity to learn. In the current study, the implementation of the pre-internship test resulted in a significant increase in self-efficacy in clinical performance at three measurement points. Supporting our findings, Montgomery et al. reported that the OSCE is a valuable evaluation and learning tool for developing nurses' knowledge, clinical skills, confidence, competence, and self-efficacy, so that students can effectively perform their professional roles (25). In another study, the relationship between the students' self-efficacy scores and OSCE scores showed that this test can provide rich learning experiences despite being stressful. In other words, self-efficacy protects students from the negative effects of stress and anxiety on their clinical performance (12). According to the principles of Bandura's self-efficacy theory, if nursing students have high self-efficacy, they will have a greater ability to perform psychomotor skills. Dynamic self-efficacy is created by reacting to four sources of information: performing a behavior, observing others, receiving feedback, and controlling emotions such as stress and anxiety during a procedure. These sources of information show that performance has a significant influence on self-efficacy (26). Contrary to the findings of the present study, some previous studies found that, despite the satisfaction and improved clinical performance of students after taking the OSCE exam compared to traditional teaching methods, the students’ self-efficacy scores did not show a significant increase (19, 27). This difference in findings may be due to variations in the clinical environments of different countries. In the current research, most of the nursing students perceived the OSCE pre-internship test as a useful educational tool that provided an opportunity for learning by reducing the gap between theory and clinical practice. The results of previous research have shown that the OSCE evaluation method is a satisfactory experience for evaluating clinical competence and should be implemented repeatedly during students' education (16, 28, 29). The evaluator's behavior can play a key role in improving students' performance during the test. In the present study, the participants evaluated the behavior of the examiner as appropriate and fair. Meanwhile, in Alkhateeb et al.'s study, students were dissatisfied with the examiner's discrimination and unfair behavior during the exam (30). In another study, students mentioned that direct observation by examiners during the exam, through the transfer of negative or positive impulses, can decrease or increase their self-confidence. Therefore, it is necessary for evaluators to receive sufficient training about the OSCE before conducting the test (31). In the present study, to address the aforementioned limitations, scientific and executive committees were formed with specific job descriptions.

The majority of our students stated that the pre-internship test was less stressful than other tests. The results of Vincent et al.'s research showed that students experienced less exam anxiety in the OSCE compared to traditional evaluation methods (16), which was consistent with the findings of the present study. Nevertheless, about a third of the students in the present study considered participating in the OSCE a stressful experience. Unusual and excessive stress causes disturbances in mental processes (32) and can prevent the demonstration of real competence, thereby impairing the validity of the OSCE. In addition, a high level of anxiety can disrupt students' ability to learn from the test and cause a decrease in self-confidence (12). Raziani et al. recommended using methods to reduce the anxiety and worry of nursing students regarding the OSCE. These included providing opportunities for students to prepare, repeated practice sessions, holding a mock test similar to the main test, or conducting preliminary workshops and question-and-answer sessions (31). In the present study, the aforementioned strategies were considered in the implementation of the OSCE pre-internship test. Most of the students stated that the time allocated to each station was suitable, which could be attributed to the pre-test. Holding a mock OSCE test helps allocate more realistic time for each station, reduce stress, and enhance students' understanding of the test (21).

Strengths and Limitations: One strength of the present study was that the OSCE pre-internship test was conducted at two points (pre-test and post-test), which had a positive effect on the high preparation of the students before the main test. This led to positive feedback and increased the students' satisfaction regarding the implementation of the test. Another strength was holding the test in the clinical skills simulation center, which had a standard building and was equipped with the latest mannequins and moulages. Among the limitations of the present study, the small number of the participants due to the nature of the study can be noted, and it is suggested that a study be conducted with a larger sample size to increase generalizability.

Conclusion

The present study showed that the correct implementation of the OSCE pre-internship test can have a positive effect on the self-efficacy of students' clinical performance. Therefore, it is suggested that this method of teaching and evaluation be given more attention in the nursing curriculum. In addition, holding the OSCE pre-internship exam at the end of the sixth academic semester should be included in the nursing curriculum. To reduce students' stress and improve results, it is suggested that the OSCE be conducted in accordance with the standards several times during the four-year education period. The present study can serve as a guide for conducting the pre-internship test in other faculties.