Background

Along with the rapid development of technology and Internet-based education, electronic learning methods (e-learning) have expanded, shifting traditional classes to virtual learning environments. A new method of education, called blended education, was introduced by Marsh and others in 2003 as the second generation of virtual education, comprising a combinatorial approach of electronic and face-to-face teaching strategies, thereby using a combination of student-centered and teacher-centered methods. This approach tries to combine the principles of cognitive learning and collective constructivism and bring together the educational system’s elements of awareness, ability, and creativity. The use of this educational method in the electronic education system has several advantages, such as achieving high-quality learning, flexibility in organizing and presenting educational materials by professors, improving the learning process by creating a series of mental challenges, engaging learners in the organization of educational affairs, explicit explaining the educational content and assessment criteria (1-3).

After the rapid movement toward distance learning in the spring of 2020 due to the COVID-19 pandemic, lab-course education demanded the employment of new alternatives. Although traditional practical learning seems to remain the main part of educating lab courses, with the progress in the world of online and electronic learning, e-learning is increasingly employed to deliver these courses, bringing challenges that need to be addressed. Due to the necessity of the minimum attendance to educational laboratories in special conditions (e.g., the COVID-19 pandemic), the effective learning of practical skills by students is compromised during lab-based courses, especially among pharmacy students. So, adjoining new electronic technologies to the educational content of practical courses can help solve these problems. Blended education (traditional-electronic) can profoundly boost the achievement of learning objectives and learners’ enthusiasm (4).

Objectives

In this study, an educational booklet was designed for the lab-based course of pharmacognosy for pharmacy students with the help of simple augmented reality (AR). This novel booklet was used to deliver educational content during one academic semester. Finally, the effects of this method on the level of satisfaction and learning of practical skills by students were investigated.

Methods

Study Features and Educational Materials: This was an interventional and educational scholarship study with a controlled pre-test/post-test design. The lab-based pharmacognosy course was delivered to pharmacy students studying in their 7^th semester as a pilot for one semester. The new booklet was designed based on a traditional booklet; however, several experiments were recorded in the form of educational micro-learning videos (about 5-minute length), displaying how the experiments were performed. The simplest form of augmented reality was applied, i.e., short movies and related photos were annexed to the booklet using QR codes. The educational videos were uploaded to the cloud space of a Gmail account, and the related URLs were applied to generate QR codes by
https://www.qr-code-generator.com/.

Research Entry and Exit Criteria: All the students who registered for the lab-based pharmacognosy course in the second semester of the academic year 2021-2022 were eligible to enter the study. The students were adolescents between the age of 21 and 22 years old. The aims of the study were explained to them, and those who agreed and gave informed consent were admitted to the study. Exclusion criteria included failure to participate in one of the tests (i.e., pre-test or post-test) or deliver the tasks assigned (i.e., logbook preparation). In addition, students who had not studied the educational booklet before initiating the lab course were excluded. The consort flow chart of the study is presented in
Figure 1.

Training of Students with the New Educational Booklet: Students were randomly divided into two groups of control and intervention. Because of limited physical space in the laboratory, students in each group were further divided into three subgroups and educated in separate classes. The control group (30 students in three subgroups) was trained using the traditional booklet, followed by performing practical laboratory skills. The control group received the training one week before the intervention group to avoid their access to the AR-based booklet. Students in the intervention group (n = 35 in three subgroups) were trained using the new educational booklet and then continued the routine course in the laboratory. Finally, both groups of students went through the practical implementation of the experiments. At the end of the research, students in the control group were granted access to the novel AR-based booklet to observe justice and maintain equal learning opportunities.

Evaluation of Learning Efficacy: Pre-test and post-test assessments were conducted for all students (control and intervention) to check the students’ basic knowledge and their learning achievements. The pre-test and post-test included three short-answer questions from the content of the respective lab booklet designed by the course-holding professor.

Evaluation of Practical Skills: During the implementation of experiments in the laboratory, students’ practical skills were evaluated according to a student evaluation checklist designed and completed by the professor and the lab assistant instructor based on predetermined indicators (e.g., the number of test repetitions, number of obvious errors, number of troubleshooting questions, improper use of equipment, ability to perform the tests completely, missing a step, correctly detecting end-points, correct understanding of materials, performing the test within the time specified, and log book preparation). The score of each item ranged between 0 (the poorest performance) and
5 (the best performance). This checklist was evaluated by five expert professors of pharmacognosy, and the CVI and CVR of the questions were calculated above 0.8.

Evaluation of Students’ Satisfaction: The students’ satisfaction with the novel educational method was assessed using the modified questionnaire for user interface satisfaction (QUIS) (5, 6). This questionnaire was originally designed for evaluating mobile-based applications, so it was slightly modified to be applicable for the evaluation of this AR-based booklet. The modified questionnaire consisted of 29 questions, two of which were related to identity information. The remaining 27 questions were related to usability and satisfaction, including general queries about the usability, capabilities, information richness, terminology, educational power, and general efficacy of the new booklet. The score of each question ranged between 0 and 9. A score of 0-2 was designated as poor, 3-5 as fair, 6-8 as good, and nine as excellent levels of satisfaction and ability. The content validity index (CVI) and content validity ratio (CVR) were calculated based on the amendments received from ten experts. The CVI values were calculated between 0.75 and 0.99, and CVR ranged from 0.84 to 0.9. Cronbach’s alpha coefficient of the whole instrument was calculated to be 0.919.

Figure 1: The CONSORT flow chart of the pre-test and post-test steps in this randomized controlled trial

Statistical Analysis: Data were analysed using SPSS software (IBM^®21). Plots were drawn by GraphPad Prism 8.0. Descriptive statistics, including mean and standard deviation (SD), were calculated for each numerical scale variable. Normality was analysed by the Kolmogorov-Smirnov and Shapiro-Wilk tests using non-parametric analysis by SPSS. A Sig. value of > 0.05 in both tests indicated the normal distribution of the data. The paired t-test was employed to compare mean differences between the pre-test and post-test scores in each group, and ANCOVA was used to compare mean differences in post-test scores between the two groups after adjusting for pre-test scores. Finally, ANOVA was applied to compare the scores of practical skills between all six subgroups of the students participating in the lab course. P values below 0.05 were considered statistically significant.

Results

The Effect of Using an AR-Based Combined Training Booklet on Pre-Test and Post-Test Scores: The mean scores of the pre-test and post-test are shown in Table 1. This study included six subgroups of students who attended the laboratory on six different days. The first three subgroups of the students (i.e., the control group) were educated by the traditional booklet, and the other three subgroups (i.e., the intervention group) were trained using the new AR-based booklet. The normality of data distribution was affirmed in both groups. The mean pre-test grade of students in the intervention group was slightly higher than that of their counterparts in the control group, but this difference was not statistically significant. Regarding post-test grades, the results of Levene’s test showed the equality of error variances, and ANCOVA showed no statistically significant difference in the mean post-test grades between the two groups. Of note, a slight increase in the mean of the post-test grades was seen in the intervention group compared to the control group (1.999 for the intervention group vs. 1.788 for the control group).

Table 1. Mean ± SD of Students’ Pre-Test and Post-Test Grades in the Control and Intervention Groups

Within-Group Comparison of Pre-Test and Post-Test Scores: Considering the fact that pre-test and post-test scores in each group are dependent on each other, the paired sample student t-test was performed to check if attending the laboratory course boosted the post-test scores of the students. A decrease in the mean of post-test scores compared to pre-test scores was observed in both groups (Table 1), and the difference between the means of the pre-test and post-test scores was statistically significant in both groups (p-value<0.05).

The Effect of the AR-Based Booklet on Students’ Practical Skills: The students’ practical laboratory skills were evaluated and scored according to the criteria mentioned previously (Table 2). A two-step comparison at two levels was applied (i.e., first, all six subgroups (three control and three intervention subgroups) were individually compared to each other, and then skills were compared between the control and intervention groups). The two levels of analysis included comparing individual components separately and then as a whole index of practical skills. Comparing the mean of each component between the control and intervention groups showed a statistically significant difference in the number of obvious errors, the number of troubleshooting questions, the improper use of equipment, and the misidentification of materials. There was no significant difference between the three subgroups of either the control or the intervention group. An increase was seen in the mean scores of all components except for the duration of test conduction and logbook preparation. Finally, the mean score of laboratory skills obtained by the students in the intervention group was significantly higher than that of the students in the control group (p-value <0.05)
(Figure 2).

Students’ Satisfaction with the AR-Based Booklet: In this study, a modified form of the QUIS was used to check the level of functionality and user satisfaction with the AR-based booklet developed.

The questionnaire was completed by the students in the intervention group. The general opinion regarding the new booklet, with a mean score of 7.34 ± 1.3, was satisfactory (i.e., between 6-8). The lowest scores were related to the items of "the ability to work continuously with the booklet" (6.9±2.1) and "the clarity of supplementary educational references" (6.5±2.8)
(Figure 3). The highest score was reported for "the ease of working with the booklet and its display capabilities", with α score of 0.944.

Figure 2: Comparison of practical skills of students between the intervention and control groups; each bar in each group is related to a subgroup.

*indicates a p-value<0.01.

Table 2. Students’ Scores Regarding the Components Used as Indexing Factors for Practical Skills

SD: Standard deviation

Figure 3: Mean scores of different components of the QUIS questionnaire. The number of valid questionnaires evaluated was n= 30.

Discussion

In this study, a novel educational AR booklet was designed, combining the traditional text booklet and e-learning technology, to deliver lab-based pharmacognosy courses to pharmacy students. It is of utmost importance to improve traditional teaching methods, particularly by using information and distance learning technologies. The novel booklet designed here employed augmented reality to better convey practical skills to pharmacy students, and it was implemented for one academic semester. However, the results showed that the new booklet could not significantly improve students’ grades, evidenced by no profound change in their post-test scores. On the other hand, the students were satisfied with the AR-based booklet provided in the context of a blended learning method. Similarly, a previously published work showed that the use of blended learning technology in a microbiology laboratory did not significantly contribute to achieving learning objectives, noting that the students who attended the virtual laboratory course obtained similar grades to their peers who, completely or partially, were physically present in the laboratory. Nevertheless, the results showed that the knowledge gained was satisfactory, and the participants valued the experience (7).

It has been proven that learning in laboratories can be augmented by providing essential information virtually during lab work. Augmented reality seems particularly suitable for providing basic information during lab courses as it can integrate physical and virtual tasks. Virtual information can be displayed in close spatial proximity to the corresponding lab environment. In fact, VR can be a basic component for the effective delivery of multimedia education, thus reducing the cognitive load of learners, strengthening their productive processing, and, finally, enhancing the acquisition of conceptual knowledge. In a previous study, researchers successfully developed a tablet-based AR application to support learning practical physics skills among higher education students (8). However, in our study, learning output did not show a significant improvement among the students who used the novel AR-based educational booklet based on paper exams. This may be explained by the poor involvement of students or their failure to study the novel booklet or watch the microlearning media embedded in the booklet adequately. Nevertheless, this novel educational booklet could improve the practical skills of students in the laboratory and deliver scores equivalent to the traditional in-person lab course, highlighting this booklet as a tool worthy of further development and evaluation.

In another study, students attending a general chemistry lab course, although deemed hands-on activities and exposition to new scientific instruments necessary for learning practical laboratory skills in chemistry labs, often considered these tools as black boxes, so they had no knowledge about how to use them or what capabilities they had. Becoming acquainted with laboratory instruments is an important part of laboratory training. In another study, an application known as Augmented Reality in the Educational Laboratory (ARiEL) was designed using AR technology to connect students to instruments’ analytical information, and the results showed that ARiEL could reduce students’ anxiety when using instruments and improve their intellectual attachment to the data retrieved by the instrument (9). The fact that we could not find a significant role for this novel educational booklet in boosting students’ learning might be related to the simplicity and the lack of attractiveness of the technology used for students. Therefore, using more advanced and intriguing technologies, such as what is used in ARiEL, can be considered in future studies to achieve more promising results.

Although blended education can be attractive for students, it cannot replace face-to-face training methods. In a study in 2010, the attitudes of pharmacy students towards face-to-face or blended teaching of a pharmacokinetics course were evaluated, demonstrating that face-to-face interactions of students with each other and with the instructor were ranked higher than online interactions (10).

Our study had several limitations, which might have affected the results. This study was conducted only in one pharmacognosy lab-based course during a single semester and only in one faculty (Pharmacy School). Our primary goal was to preliminary investigate the effectiveness and attractiveness of the novel AR-based educational booklet and its influence on students’ skills. In future studies, it is recommended to include more courses and disciplines to assess the applicability of this novel booklet. Moreover, due to Internet limitations, the training videos prepared in this study had minimum coverage of essential educational concepts and the details of experiments. Better access to the Internet can allow for uploading more media and should be considered in future studies. In general, AR-based methods have been promising, given that the essential tools and infrastructure are available, which allows for more advanced methods to be applied. Therefore, by employing novel technologies such as AR or VR, we recommend developing more comprehensive AR-based booklets to be employed in a more controlled manner to achieve the best educational outcomes in lab-based courses (11).

Conclusion

Facing the COVID-19 pandemic forced the educational system to seek different educational methods primarily based on virtual platforms. Since virtual education had not been generally applied in Iran before COVID-19, especially for lab-based courses, Iranian teachers and students, like many others around the globe, faced many challenges in adopting these teaching and learning courses. Blending learning has been introduced as an efficient method for lab-based courses in higher education (2, 12, 13). Here, we developed a novel educational package based on blended learning in which educational movies were adjoined to texts using QR codes. The implementation of this novel booklet for training a pharmacognosy lab-based course to pharmacy students could not improve the post-test scores of the students compared to the control group who received the traditional learning method. However, the students expressed their eagerness for and satisfaction with the novel booklet. Further studies are required to assess the effectiveness of novel advanced technologies, such as AR or VR, in promoting the learning and practical skills of pharmacy students participating in lab-based courses.