Background

In every society, many organizations and institutions provide various services to people from different walks of life. Among these organizations, educational services delivered by universities not only play an important role in students’ lives but also affect all parts of society (1). Universities and higher education institutions are considered the main centers of thought and science production in every society and play a significant role in the presence and activity of thinkers, scholars, and students in the community (2). Universities maximize productivity and efficient use of opportunities in the country through careful planning for educational, research, and human resources development. Besides, they improve the education quality following national and international scientific and educational developments (3).

The quality of services is one of the factors contributing to the success and sustainability of any organization, including the higher education system. Its sustainable development requires simultaneous and balanced growth of quantitative dimensions. Quality in Higher Education is a multidimensional concept influenced by several criteria, including the educational status, academic system, curricula, institutional conditions, and standards of the academic discipline (4). Since service satisfaction is the distinguishing element among organizations in the competitive market, student satisfaction is a determining factor in evaluating the quality of services provided by higher education organizations (5). Improving the quality of education and research, alongside providing specialized educational services, are the ultimate goals of any educational system. Hence, failing to achieve these goals results in wasting economic resources and reducing self-esteem and instability in the students’ personal and social temperament (6).

Meanwhile, Information and Communication Technology (ICT) provides the opportunity to combine the art and the science of teaching and education to create an approach called e-learning (7). E-learning has ascertained many educational goals, including synchronous learning, cooperative learning, self-assessment, and self-direction. Today, e-learning in higher education is at a rapid development stage; however, it encounters problems like increasing demand for services and competition among educational centers. Hence, it is essential to improve its efficiency by adopting appropriate solutions based on evaluation evidence (8). E-learning is an online system that facilitates teaching and learning with the advent of various web-based software tools and applications. This modern educational approach incorporates asynchronous and synchronous tools to support learning processes. E-learning enhances knowledge acquisition, clinical skill development, and learner satisfaction across various domains in medical education (9-11).

Program evaluation reveals the existing and desired status and the strengths, weaknesses, opportunities, and obstacles of e-learning systems. Students play a vital role in enhancing the quality of service and evaluating educational systems as they are the main clientele and recipients of system services. Moreover, their perspectives are the key indicator of a university’s overall quality (7, 12).

One method used to evaluate the quality of universities’ services is the SERVQUAL model developed by Parasuraman (13). This tool measures students' perceptions in five different dimensions of service delivery including; tangibility (facilities, equipment, staff, faculty members, and communication channels), reliability (responsibility to do services reliably), responsiveness (tendency to cooperate and assist students), assurance (competence and ability of staff and faculty members to create a sense of trust and confidence among students), and empathy (special treatment to every single student regarding their moods to convince them to realize that university system can understand them) (14). This tool examines service quality by comparing students’ expectations and perceptions in each dimension. Parasuraman et al define the quality of service as the difference between clients’ expectations and perceptions of services, known as the expectations-perception gap. This definition is represented by the equation (Quality = Perception-Expectation). Clients’ expectations are their beliefs about the provided services that work as reference points when evaluating the services. The purpose of the customer's perception is to find out how one evaluates the service received, i.e., the customer's judgment of the service. Therefore, perceived quality is a form of insight related to satisfaction and results of comparison between expectations and perceptions of performance (8). A negative score of the gap indicates that perception is lower than expectations, and students are not satisfied with the services. On the other hand, a positive score shows that the provided services are better than expectations (14).

Nowadays, the Internet has diminished the necessity of physical presence on university campuses in many instances. Consequently, governments are encouraging the adoption of e-learning solutions. Hence, the number of higher education institutes implementing strategies and policies for integrating e-learning technology into their educational processes is increasing. Therefore, ensuring the quality of e-services from different viewpoints, including students, is necessary to provide effective e-learning systems (15, 16).

Despite the evaluations made at different universities based on the SERVQUAL model, we did not find any research examining the quality gap of e-learning services (17). There are different dimensions across different service industries and even within the same service industry. Moreover, their customers are heterogeneous. So, a customized SERVQUAL model for specific contexts is beneficial (18). Hence, regarding the importance of educational service quality, the need for high-quality online educational services, and the essence of developing a customized SERVQUAL tool, this study aimed to develop a valid and reliable tool for evaluating the quality of educational services in e-learning centers.

Objectives

This study aimed to design and evaluate the psychometric properties of a tool for assessing the quality of educational services in e-learning centers, based on the SERVQUAL model.

Methods

This descriptive-analytical study started with a review of the literature on the quality of educational services based on the SERVQUAL model and gathered the questionnaires and factors affecting service quality in higher education. Then, we compiled all the questionnaires’ items, reviewed them carefully, and omitted or merged similar items. We conducted the psychometric assessment methodology for this draft of the questionnaire to ensure its validity and reliability.

Instrument development: In the first stage of this research, 12 studies addressing the questionnaires and factors regarding the quality of educational services based on the SERVQUAL model were collected (1, 3, 5, 6, 13, 14, 17, 19-23). Then, their items were collected and indexed. 88 questions were extracted from these studies. After omitting similar or irrelevant questions, 56 questions remained. An expert panel modified the items to make them compatible with an e-learning environment. The experts consisted of four e-learning and educational specialists and five university teachers with more than three years of experience in e-teaching. Modification of the questions was performed in two rounds of the expert panel.

In the next step, the face validity of the instrument was assessed by other experts, including five faculty members with higher education/e-learning expertise or with at least 3 years of e-teaching experience, and also seven junior and senior e-students. To assess the content validity, Content Validity Ratio (CVR) and Content Validity Index (CVI) were applied To determine the CVR, experts evaluated each item on a three-part scale of "necessary", "useful but not necessary," and "not necessary". According to Lawshe, C. H. (1975) formula, if the calculated value for each question was greater than 0.57, the content validity of that question was then accepted. The Waltz & Bausell method was also applied to determine the CVI (24). For each question, three items of relevance (not relevant, relatively relevant, relevant, and completely relevant), simplicity (not simple, relatively simple, simple, and quite simple), and clarity (not clear, relatively clear, clear, and very clear) were questioned. The scores for the options of "completely relevant" and "relevant", " simple " and " quite simple "," clear " and " very clear" were added up together. Then the overall scores were divided by the number of experts. If the calculated value for each question was higher than 0.79, it was considered acceptable. The final instrument consisted of 20 pairs of questions in two parts to measure students' expectations and perceptions of the quality of educational services. The instrument items were based on a five-point Likert scale, i.e., “quite important”, “important”, “relatively important”, “less important” and “the least important” for the expectations section; and “very good”, “good”, “average”, “poor” and “very poor” in the perceptions section. It should be noted that each item was scored between 1 and 5. The tool was named Service Quality in E-Learning Centers (SQELC).

Exploratory Factor Analysis was also administered to examine the construct validity (25).

 To assess the reliability of the tool, the test-retest approach was applied, and Cronbach's alpha was used to evaluate its internal consistency.

Study population and sampling strategy: The statistical population was students of the Tehran University of Medical Sciences (TUMS) who studied in virtual post-graduate programs. All majors and courses in these programs were delivered through virtual and blended approaches.

The required sample size for assessing reliability through test-retest was 20 participants, while the recommended sample size for evaluating construct validity ranges from 100 to 200 participants. Therefore, all eligible students were included in the study using a census sampling method. There are several methods to determine the adequacy of the sample size for factor analysis; however, the most widely accepted and reliable approach is the calculation of the Kaiser-Meyer-Olkin (KMO) index. A KMO value greater than 0.7 indicates sufficient sample adequacy to conduct factor analysis (26, 27). All analyses were performed in IBM SPSS Statistics for Windows, version 17 (IBM Corp., Armonk, N.Y., USA).

Results

Content Validity: Out of the 56 items assessed for CVI and CVR, 20 items were ultimately approved (Table 1).

Table 1. CVI and CVR of the approved items

Construct Validity: The questionnaire was delivered to 110 students of virtual programs at TUMS to conduct Exploratory Factor Analysis (EFA). 72 out of 110 students (47 women and 25 men) filled the questionnaire (response rate: 65.5%). The descriptive statistics and characteristics of the participants are presented in Table 2.

Table 2. Characteristics of the study participants

Kaiser Mayer Olkin Index (KMO) at EFA was 0.773, indicating a sufficient sample size. Bartlett's Test of Sphericity (chi-square = 695.89, P < 0.001) also showed the suitability of samples for factor analysis. Five factors were extracted according to the design background, and the Varimax rotation method was applied to determine the items of each factor. In addition, coefficients less than 0.45 were omitted. According to the results, these five factors explained 67.17% of the variance, which is an acceptable value (Table 3).

Table 3. Extracted components and Total Variance Explained for SQELC tool

The scatter plot (Figure 1) also confirms the suitability of the 5-factor solution.

Table 4 shows the rotating factor matrix. Although it is recommended that factors consist of at least three items to ensure internal consistency and stability, there are instances in the literature for retaining two-item factors when they demonstrate strong conceptual coherence and satisfactory factor loadings (26). In our analysis, both items within Factor 5 loaded exclusively on this factor, suggesting a distinct underlying construct. Furthermore, the content of these items reflects a specific, meaningful domain within the broader construct being measured.

According to the results of factor analysis and research background, the same names of factors of the original questionnaires were chosen. The names and questions of the factors and the internal consistency of each factor are presented in Table 5.

Reliability: The reliability of the instrument was examined through the test-retest method. Reliability in a 20-person sample after two weeks of replication, using the Interclass Correlation Coefficient (ICC), was 0.769. Cronbach's alpha for all items also showed 0.824 internal consistency for the whole instrument. The results of the questionnaire analysis among participants are presented in Table 6.

Table 4. Factor loadings of items on extracted components after varimax rotation in exploratory factor analysis

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization

Table 5. Components' names, their items, and internal consistencies

Table 6. Comparing expectations and perceptions of quality services in E-Learning Schools using SQELC

Discussion

This study aimed to design a service quality assessment instrument for e-learning centers based on the SERVQUAL model and evaluate its psychometric properties. The final tool for evaluating service quality in e-learning centers consisted of 20 items and five extracted factors named the same as the main SERVQUAL model.

E-learning is an instructional strategy to enhance teaching-learning processes through web-based systems, software tools, and applications (28). Specifications of e-learning, such as being student-centered and self-directed, and its capabilities for deploying different types of interaction, have made it a persistent and effective approach in which a wide range of asynchronous and synchronous tools are used (29, 30). Differences between e-learning and face-to-face instruction show that the dimensions and factors affecting their quality are different. E-learning aspects can be divided into system design, system delivery, and system outcome, which should be considered to measure their quality (22).

Positive attitude towards students and realizing their needs, providing effective feedback, and appropriate facilities such as appropriate hardware and software, guidelines, and actively announcing them are the main parts of the factor "responsiveness". The factor "assurance" consists of the appropriate delivery of assignments, curriculum, and e-content through media such as learning management systems. Providing sufficient and timely resources, assignments, communication facilities like discussion groups, experienced instructors, and access to reliable internet are composed of the factors of "Reliability". Sense of trust for students, preparing an expedient environment, and respectful behavior of managers are considered as "Empathy". Finally, inspiring students' creativity and considering their achievements, and preparing a feedback system for improving the quality of services are classified as "tangibility".

Sugant determined four dimensions for e-learning services to be evaluated as service quality: Content (design and presentation, structure, completeness), usability (attractive interface, ease of navigation, interactivity, progress tracking), technology (fast, reliable, support), and responsiveness (assessment and evaluation) (31).

In SQELC, items 2, 5, 9, 11, and 12 cover components of the dimension "content"; items 1, 13, 14, and 16 cover components of the dimension "usability"; items 18 and 20 are related to "technology " and items 3, 15, and 16 are according to the dimension "responsiveness".

Ayuni and Mulyana (32) assessed the relationship between service quality and satisfaction, and loyalty of the students in the e-learning system. They used four dimensions for e-learning service quality, consisting of teaching dimension, administrative services, support services, and system, which have been developed previously (33). Items 5, 7, 10, and 17 are according to administrative services, items 2, 9, 11, 12, 13, and 15 are according to teaching dimension, items 3, 6, 7, 8, 10, 17, 18, and 19 are compatible with support services, and finally, the dimension of "system" can be evaluated through items 14, 16, 17, 18, and 20.  It is visible that some items overlap between different dimensions.

The results of SQELC can lead to the analysis of deficiencies in e-learning centers. It also provides conductive solutions to reduce the observed quality gaps in educational services. The results obtained from the Virtual School of TUMS showed that perceptions and expectations were close in all aspects of the quality of educational services, including accountability, reliability, assurance, and empathy. Although this gap was negative in all dimensions, in other words, students' expectations were beyond their perceptions of the actual status quo; the difference was not significant in terms of effect size (gap size), while its statistical significance may be because of the high sample size. The existence of a quality gap in the dimension of accountability can indicate students’ dissatisfaction regarding unclear announcements of the school or even its inadequacy in terms of providing feedback on students’ assignments in the virtual setting. This illustrates the need for instructors and professors to engage with students and provide feedback on their homework. The style and type of implementation of e-courses, where students have less access to teachers and where traditional classes are not available, doubles the importance of providing feedback on students' homework. Quality of service in terms of reliability means the ability to perform the service reliably and or act with commitment. The service gap in this dimension can reflect students' dissatisfaction with the quality of the Internet and how they interact in the LMS environment, such as the expected time for students to deliver homework, the provision of tutorials, and even facilitating discussions in the virtual classroom by instructors. The dimension of assurance in SQELC indicates the competence and ability of university staff and faculty members to induce a sense of trust. The negative gap in performed services in this dimension indicates that, from the student's point of view, assignments are not appropriate and lesson-related, and the school has failed to provide confidence among students to make them feel safe and secure. The tangible and physical dimension means that the conditions and environment of the setting are available to provide services, including facilities, equipment, staff, and communication channels.

In addition, the empathy (the factor 4 with three items) means a personalized attitude toward one’s mood. In this case, it means that students believe that the university staff perceive each individual of them. Proper behavior and respectful treatment towards students can bring students' expectations up to the level of their perception in this dimension. The negative gap in service quality in the empathy dimension indicates that students do not have appropriate mechanisms to express their views and suggestions.

Limitations: One limitation of this study is that it focuses solely on one university. It is recommended that the SQELC results be evaluated across multiple medical universities to better understand the accuracy of its results. Additionally, the available sample size was not satisfactory. Although the statistical indices showed their adequacy, it is recommended to apply the tool to a large population. Furthermore, investigating the optimal results for different academic fields would be valuable.

Conclusion

In this study, SQELC, an instrument of service quality assessment for e-learning centers, was devised. This instrument uses student feedback to evaluate service quality. It is suggested that other stakeholders, such as teachers and staff, also be considered in plans. The results obtained in evaluating the quality of services in e-learning centers are considered as guidelines for management, instructors, and staff to be informed of the strengths and weaknesses of the services so that they can improve the quality of educational services by removing the weaknesses as well as improving the strengths.