Background

Learning management systems (LMSs) are
web-based e-learning platforms which provide an environment for delivering course information, content, and learning activities, principally supporting asynchronous interaction between students and instructors (1). Universities have invested heavily in providing LMS to guarantee the enhancement of students’ learning experiences in the information age (2). LMS definitions and functions revolve around teaching and learning management. Turnbull et al. (3) define LMS as online learning technologies employed to make, manage, and implement course materials. Meanwhile, evidence indicates that the ecosystem of LMSs will change and expand soon with new technologies such as the Internet of Things (4) and artificial intelligence (5). Considering the current usage and future developments, suitable policy-making is required for LMS management and administration. Most LMSs are governed by institutional policies that provide general guidelines on utilizing information and communication technology (3). Clear policies outline the proper technology infrastructures as well as contribute to developing, managing, and working with LMSs (6, 7).

LMS adoption in developing countries has more challenges than in developed ones (8, 9). Different factors are identified as major reasons for developing countries' failure in LMS utilization including technological, organizational, and self-factors, which result in lack of user satisfaction (9). To overcome such challenges, LMS should be designed and adopted as a learning community, i.e., LMS stakeholders should consider serious participation in expanding the learning ecosystem (10).

According to (11) the Ministry of Health and Medical Education (MOHME), the authority body supervising universities of medical sciences had announced no clear policy for e-learning and educational technologies until 2013 in Iran. At that time, most of the 52 universities of medical sciences did not have any LMSs, and only a few had their own systems, which were used sparsely. One of the main agendas of this policy was to provide LMS infrastructure for universities of medical sciences. This duty was assigned to the Smart (Virtual) University of Medical Sciences. Thus, the design and creation of a national LMS, named NAVID, was initiated in 2016. Two of the authors, “Mojtahedzadeh and Mohammadi”, were principal instructional designers of the project. It was made freely available to all universities of medical sciences in 2018 and some universities employed it voluntarily. With the outbreak of the COVID-19 pandemic, all universities of medical sciences adopted NAVID LMS (12).

After about 5 years of using NAVID LMS as a national platform, many studies have addressed NAVID as the medium for their teaching-learning processes across different universities (12-17), indicating its popularity. Indeed, although the availability of the NAVID LMS as a national infrastructure was identified as an opportunity for depicting the strategic direction of medical sciences education (18), a few studies have focused on evaluating the LMS itself from different perspectives (19). Accordingly, this study aimed to explore the strengths, weaknesses, opportunities, and threats (i.e., within the SWOT framework) of this experience. We believe that the results of this study will assist authorities of higher education institutions, especially those in developing regions confronting similar budgetary and resource challenges, in understanding the benefits and limitations of this experience.

Objectives

This study was structured to identify the strengths, weaknesses, opportunities, and threats (within the SWOT framework) associated with implementing a national LMS across Iranian universities of medical sciences.

Methods

Study Design: We applied a descriptive qualitative research method to explore participants’ lived experiences and perceptions of the NAVID LMS. Sandelowski (20) emphasized that in descriptive qualitative research, a shared understanding of a phenomenon is developed by examining participants' experiences, facilitating discovery. Whereas much of this process focuses on detailed description, interpretation is applied selectively when necessary. Meanwhile, a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis is one of the effective situation analysis techniques organizations utilize to provide in-depth information on their position according to the internal and external environmental elements (21). Strengths and weaknesses refer to the organizational internal elements that facilitate or interfere with goal achievement; on the other hand, opportunities and threats address the external ones. This analysis is widely employed in different fields including education (22).

Participants and Setting: Data were collected through semi-structured interviews with 16 participants who were actively involved in utilizing and adopting the NAVID LMS. The sample selection was based on three key indicators: i) active professors specializing in e-learning and users of NAVID, ii) managers of Medical Education Development Centers (MEDCs) responsible for facilitating e-learning, and iii) technical experts from MEDCs who served as NAVID administrators. The participants took part from 11 universities of varying sizes across the country. Table 1 reports the demographic characteristics of the participants. We selected the participants through a purposeful sampling method, following the principle of maximum diversity, and continued the sampling process until reaching qualitative data saturation in each participant group (23) based on their stance about LMS adoption.

Table 1. Demographic Characteristics of the Study Participants

Conducting Semi-Structured Interviews: We conducted semi-structured interviews for data collection. The interview probing questions were as follows:

-     What strengths do you think NAVID LMS has?

-     Has it succeeded in achieving the intended goals as a national LMS?

-     What weaknesses do you think NAVID LMS has?

-     Are the services provided by NAVID LMS aligned with the needs, conditions, and facilities of your university?

-     What are the opportunities for the further development of NAVID LMS?

-     What are the threats interfering with the further development of NAVID LMS?

-     What are the benefits and limitations of the national design as well as provision of NAVID LMS in a developing country like Iran?

Further, more detailed questions were asked, if necessary. The interviews lasted between 30 to 90 minutes. All interviews were audio recorded and transcribed verbatim.

Content Analysis of the Interview Transcripts: We employed directed content analysis, where the SWOT analysis framework serves as the initial structure to collect and categorize data (24). We thoroughly reviewed and systematically coded the transcripts using Microsoft Word 2016. We then returned the transcript and extracted codes of each interview to the related interviewee to check for accuracy and final feedback.

We utilized some strategies indicated by Creswell and Poth (25) to ensure the validity and reliability of the findings:

Triangulation: We collected data from multiple sources, including university professors with e-learning experience, directors of e-learning units, and LMS admins, to capture all main stakeholders’ views.

-     Member checking: To eliminate the researchers’ bias and bring the findings closer to reality, the transcripts and extracted codes were returned to the interviewees to guarantee the credibility of the results.

-     Rich and thick description: We described the study setting, participants' characteristics, data collection and coding processes, as well as the results in detail to ensure the transferability of the findings.

-     Prolonged engagement: The interviewer developed mutual trust and built rapport with the interviewees. Further, we took ample time to engage with the data and thoroughly read the transcripts, allowing us to gain a profound understanding of both the transcripts and the coding.

-     Intercoder reliability: Two researchers analyzed the transcribed data independently to extract codes and then checked the final results to reach an agreement on the codes, subthemes, and assigned themes.

Results

Through the content analysis of the data, we extracted 235 codes, which were classified into 21 subthemes and arranged under four predetermined themes: i) strengths, ii) weaknesses, iii) opportunities, and iv) threats of NAVID LMS adoption. These subthemes were 7 strengths, 6 weaknesses, 4 opportunities, and 4 threats (Table 2).

Herein, we further elaborate on each subtheme with participants’ illustrative comments.

NAVID strengths

Appropriate instructional design: According to the participants, NAVID LMS had an appropriate instructional design following the educational principles and international experiences. Adequate modules, suitable configuration, appropriate fonts and colors, graphic appeal, English-Persian views, and balanced cognitive load were other positive points mentioned for NAVID LMS. For example, one of the e-learning office directors stated:

NAVID benefits from an appropriate instructional design. I think performing the instructional design process for an educational platform based on the experiences and interests of the users would make a lot of difference in its usage (Participant No. 8).

Providing basic modules for e-teaching: Some participants emphasized that NAVID covers basic users’ needs through the possibility of uploading different types of content; interacting with students by using assignments, forums, and messaging modules; grading; generating reports, and creating exams. In this regard, Participant No. 2 who was a medical education professor stated:

Even though NAVID lacks numerous features, it is popular because university professors often do not require many modules and may even prefer straightforward platforms with sufficient functionality. Fortunately, NAVID meets the bare essentials of an acceptable LMS.

User-friendliness: Many participants indicated that users do not require extensive training to log in and use NAVID, though training videos were available. In this regard, one of the university professors stated:

NAVID LMS is convenient and user-friendly.

Table 2. Themes, Subthemes, and Their Frequency Mentioned by Interviewees about NAVID LMS within the SWOT Framework

It indicates where NAVID modules are located and how they function, which is crucial in determining software usage (Participant No. 6).

Ongoing updates: Interviewees believed that the NAVID team had made efforts in several cases such as fixing bugs, improving the existing modules, and adding new ones such as the content repository module. For instance, one of the professors expressed his experience of NAVID development as follows:

NAVID LMS has significantly improved compared to the first versions. Continuous software updates show the users that a dedicated team supports it (Participant No. 6).

Fetching data from the Student Administrative Software (SAS): Along the interviews, each university NAVID was connected to its SAS through an automatic web service for fetching the data of courses along with their students and teachers. This was identified as a primary factor in NAVID's widespread usage. In this regard, one of the NAVID admins stated:

Automated fetching of the courses’ data and participants is the most pragmatic approach making an LMS accessible for each teacher. Professors were amazed when they logged in to LMS and could have access to their students as well as courses at the first instance (Participant No. 14).

Reducing e-learning costs: The participants noted that many universities lacked the budget to implement an LMS prior to launching NAVID. Some interviewees highlighted the government's role in providing free LMS, which was essential for e-learning dissemination in Iran during and before the pandemic. One of the e-learning experts from a middle-sized university mentioned:

For a developing country like Iran with serious budget constraints, if every university provides an LMS for itself, it will increase the costs of the system in public universities that rely on governmental budgets (Participant No. 1).

Equity in access to e-learning infrastructure: According to some participants, NAVID LMS ameliorated the educational equity in access to e-learning infrastructure in universities of medical sciences. It helped during the COVID-19 pandemic and prevented the closure of education, especially in underprivileged universities. One of the directors of the e-learning offices outlined:

When a facility such as LMS is provided, some professors become willing to use it. Meanwhile, the COVID-19 pandemic made it a necessity to do so. NAVID was an equal opportunity for professors and students, especially in smaller and newly established universities (Participant No. 9).

NAVID weaknesses

Limitations in the modules: The participants addressed some weaknesses for both content delivery and interactive NAVID modules technically (such as limitations in file size upload) and pedagogically (such as the need to group students for assigning different contents or assignments). The NAVID exam module was stated as its most problematic feature not being competitive with other LMSs. The participants emphasized that NAVID should either be linked to specialized electronic exam software, or its exam module requires significant development.

Limitation in the notification system: NAVID did not have an appropriate notification system such as emailing or messaging as described by many participants, one of whom noted:

One of the few advantages of our previous LMS compared to NAVID was sending notification emails for activities. NAVID lacks this feature, requiring users to check their dashboards regularly to avoid missing any activity (Participant No. 6).

Limitations in the reports: Some interviewees stated that although NAVID reports for student activities were complete, they lacked enough data for teachers’ activities, especially for the courses delivered by several teachers. One of the NAVID admins stated that:

Authorities of universities need detailed reports of teachers’ activities for evaluation purposes. An LMS should support such a need, especially in the courses delivered by several teachers (Participant No. 13).

Restrictions in connection with other software: Although each university’s NAVID was connected to its SAS, the participants noted the necessity of NAVID being linked to other university software such as electronic exam system, online class platforms, and teacher evaluation software, as stated by one of the e-learning office directors:

I understand that every university uses its own electronic exam or online class software, making it difficult for the NAVID team to plan for such personalized connections for each university; meanwhile, this is a crucial function (Participant No. 10).

Being slow to load: This limitation was indicated by the LMS admins, especially when they had to fetch data from SAS or get reports.

Delayed technical support: Some participants believed that the technical support team was slow in resolving bugs and errors. One of the interviewees expressed his concern as follows:

It is disappointing for students and professors to face bugs and errors while working with the LMS, and this worsens when it takes time to resolve (Participant No. 3).

NAVID opportunities

International commercialization: The participants believed that NAVID had the potential to be introduced to the LMS international market as a commercial low-cost LMS suitable for low-income institutions in the region. In this regard, one of the participants noted:

“International commercialization of NAVID LMS is not out of reach if only the team improves in technical and support aspects” (Participant No. 13).

Making NAVID LMS an open-source software: Some interviewees suggested making NAVID an open-source software to benefit from collaborative development with the help of software experts to improve the current modules and add new ones. In this regard, an interviewee asserted:

Considering the budget constraints for expanding the NAVID team, making it an open-source software may be a reasonable solution for enhancing the modules and creating new features as well as add-ons such as an e-portfolio or a better exam tool (Participant No. 8).

Stakeholders’ participation in LMS improvement: Several participants focused on the national reputation of NAVID as an opportunity to invite users and experts from the LMS industry to criticize NAVID from both technical and pedagogical aspects. In this regard, an e-learning office director mentioned his idea for the development of NAVID as follows:

NAVID team should organize a software bug-finding contest and invite different people including software programming experts, active users, and professionals to find its bugs or suggest new ideas for better performance (Participant No. 10).

Building an independent research and development team: The participants had the idea of virtually gathering an interdisciplinary team, other than the one involved in NAVID implementation and maintenance to conduct ongoing needs assessments and visualize the future for NAVID LMS. For example, one of the participants stated:

Research and development of such a nationally utilized LMS needs an independent team, not involved in its maintenance and support. This team can contribute virtually and should consist of expert e-learning instructional designers, who understand the principles of computer science, along with software technical members, who are aware of the basics of virtual learning pedagogy (Participant No. 13).

NAVID threats

The highly competitive LMS market: According to some participants, the closure of universities during the COVID-19 pandemic resulted in a significant investment in the private sector for developing new LMSs or customizing open-source ones such as Moodle. Some highlighted that NAVID should be a dynamic software with ongoing improvement to survive in such a competitive market.

Lack of appropriate rules and regulations for e-learning: The participants criticized the lack of up-to-date and clear strategies for e-learning along with its application in biomedical programs at both national and university levels. They were concerned about the continuation of adopting e-learning or blended learning after the pandemic, which would hinder NAVID usage. One of the e-learning office directors describes this concern as follows:

Unfortunately, following the COVID-19 pandemic and universities’ re-openness, we do not have clear rules and regulations for continuing the appropriate implementation of blended or e-learning strategies in face-to-face programs. We should have a clear plan to benefit from the provided infrastructure and professors’ experiences of using them (Participant No. 8).

Insufficient financial resources: Most participants indicated that NAVID LMS did not have sufficient financial support for development and maintenance. For instance, one of the e-learning office directors of a large university stated:

Authorities and policymakers should be aware that if NAVID LMS is not supported financially, it will disappear in competition with other LMSs. Then, small and medium-sized universities with serious budget constraints cannot afford to provide another LMS, which means not using e-learning at all (Participant No. 8).

Limitations in technological infrastructure and facilities: According to some participants, the shortage of hardware infrastructure posed a significant threat to NAVID software. Many university managers and users addressed issues such as NAVID had hardware limitations.

Discussion

In this study, we explored the nationally developed and implemented NAVID LMS’ strengths, weaknesses, opportunities, and threats by conducting semi-structured interviews with the stakeholders and performing content analysis of the interviews within the SWOT framework.

We identified seven points of strength for NAVID LMS, including appropriate instructional design, providing basic modules for e-teaching, user-friendliness, ongoing updates, fetching data from SAS, reducing e-learning costs, and providing equity in access to e-learning infrastructure. Some studies have highlighted the attention and significance of instructional design for LMS creation along with its important role in users’ satisfaction. The proper design of LMS motivates learners to study (26). Accordingly, the most frequently noted strength was its effective instructional design. This may be due to consideration of educational principles in addition to the users’ characteristics and interests as well as paying attention to the good configuration and graphics creating an environment with a low cognitive load.

The participants believed that NAVID provided basic modules for e-teaching in accordance with some studies conducted using NAVID LMS (12, 15). Working with these NAVID modules was reported to be simple and convenient, as confirmed by another study (19). LMS user-friendliness is associated with better students’ perceived usefulness (27) as well as greater satisfaction with the tool (28) and an indicator of its easier adoption (29, 30). Monitoring and updating of the NAVID LMS has been stated to be one of the challenges faced in the delivery and maintenance stages of implementing e-learning (31).

Perceived usefulness and user satisfaction significantly influence adoption rates of digital tools, serving as key indicators of successful adoption (27, 28). Furthermore, maintaining and updating e-learning platforms emerges as a persistent challenge throughout both in the delivery and maintenance stages of implementing e-learning (29). Although the participants lacked extensive experience with the quality of international LMS, they acknowledged that the NAVID LMS team effectively updated the software based on their feedback. Similarly, prior research has affirmed that continuous improvement, regular updates, and the expansion of LMS functions as well as services for users in higher education institutions are among the primary goals of successful LMS implementation (6).

Further, Uziak et al. (32) stated that in their experience no connection was established between their university SAS and the pioneer Blackboard LMS, which was a serious problem. NAVID LMS team technically handled this concern by providing each university NAVID admin with the integration of data of all courses with their instructors and students from each university’s SAS into NAVID along with users’ authentication service at the beginning of each semester. In addition, based on our findings, reducing educational costs and enhancing equity in access to educational resources have been mentioned as benefits of using LMSs (28, 33).

In spite of the above-mentioned strengths, six items were extracted for NAVID LMS weaknesses, including limitations in the modules, the notification system, and the reports; restrictions in connection with other software; slow loading and delayed technical support. Some of the limitations in NAVID modules can be categorized as adding further features and specifications into the LMS. Meanwhile, the adoption of numerous specifications makes an LMS complex and prohibits the users’ acceptance (34). NAVID's simple instructional design contributed to its widespread adoption.

In addition to university SAS, NAVID LMS was connected to some university Adobe Connect systems for holding synchronous sessions, which was a good experience (19). Such integrations to related software such as virtual classes and electronic exams are among the criteria for adopting an LMS and provide better usage of e-learning facilities for users (35).

Slow loading, as the next point of weakness for NAVID LMS, may be caused by some factors. It can be related to the student and faculty members' Internet connection limitations (27, 29, 36). It can also be the consequence of hardware or software issues. As reported by other researchers, the COVID-19 pandemic urged an extreme load on servers that hosted LMSs and many users began to utilize these systems (37). This caused slowness and technical delays in working with the NAVID LMS, particularly at the beginning of the COVID-19 outbreak. Finally, the last mentioned weakness for NAVID LMS was delayed technical support of users. Accountable and fast technical support service has been stated to be one of the main success factors of LMS implementation in other experiences for other LMSs as well (28, 29, 32, 36). In addition, another study specifically on NAVID implementation addressed the same issue (14).

In addition to the internal factors, the participants stated some opportunities for NAVID LMS in the external environment including international commercialization, making NAVID LMS an open-source software, stakeholders’ participation in LMS improvement, as well as building an independent research and development team. Regarding the chance of international commercialization of NAVID LMS, we assume that the multi-lingual architecture of the software along with its user-friendliness and some good features provide such an opportunity. Meanwhile, the software coding structure could be revised to move towards open coding. Thus, there is a need for a clear policy on its commercialization or open-source delivery. If the open-source model is followed, service-based business models should be adopted instead of traditional license-based ones (38).

Stakeholders' participation in recognizing the bugs and improving the modules, as another opportunity, allows for receiving ideas in various ways, such as inviting experts from the LMS industry or asking for users’ participation for its improvement. Moodle LMS bug tracker is a good example of such an opportunity through which developers, experts, and ordinary users contribute to reporting the bugs and improving the software (39). Ultimately, forming a research and development team with an interdisciplinary academic background and observing issues from different angles constituted another opportunity which could keep LMS developers close to emerging technical and scientific developments. The importance of such teams and the benefit of their virtual development for software projects have been recommended in the literature (40).

The threats of NAVID LMS revolve around economical, structural, and digital infrastructure components. The first threat was the competitiveness of the LMS market in Iran and the activity of private companies, which have increased since the COVID-19 pandemic. This competition has also been mentioned for well-known learning management systems (LMSs) such as Blackboard (41). Out-of-date as well as insufficient rules and regulations related to e-learning dissemination at the governmental level have been noted as another threat to NAVID LMS usage. This challenge of legal legitimacy in the Iranian higher education context has been stated in another study (42). Thus, authorities should develop and legitimize effective policies as well as regulations focused on post-Covid conditions. The lack of financial resources to maintain and expand e-learning is another threat that has limited the work on emerging and advanced technologies. The challenge of affording hardware and software costs of e-learning is also a concern in other contexts (43). This needs a thorough investigation of the cost-effectiveness of any action before its adoption. Eventually, the participants stated that the technological infrastructure of e-learning in universities was insufficient and had raised obstacles to using e-learning. This limitation is aligned with other experiences and warrants a more accurate examination of the prerequisites of LMS adoption (44).

Limitations: Despite the research team's efforts to conduct a SWOT analysis of the NAVID LMS, the lack of participation from senior managers and policymakers of the MOHME, as well as administrators of other active e-learning LMS across the country, stands out as a major shortcoming of this study. The participation of these stakeholders could have greatly enhanced the findings by allowing for a deeper analysis the external opportunities and threats.

Conclusion

The results highlighted internal and external factors essential for developing strategies to enhance strengths, address weaknesses, leverage opportunities, and mitigate threats in adopting and maintaining an LMS, particularly the NAVID LMS. According to these findings, several key suggestions should be considered to improve the quality of the NAVID LMS. Firstly, strengthening support and design teams through an interdisciplinary approach will help eliminate bugs and ameliorate the system quality based on audience’s needs and evolving technological advancements. Further, the communication between users and the support team can be facilitated by launching an interactive website and developing mobile-friendly applications, ensuring greater accessibility and responsiveness. Transitioning from a closed-source approach to an open-source model will enable experienced experts to contribute to system improvements, leveraging collective knowledge and expertise. In addition, integrating Navid with essential domestic and international platforms will enhance teaching quality and optimize the management of the learning process. Ultimately, ensuring continuous financial investment is crucial for regular updates and upgrades, maintaining the system’s effectiveness and long-term sustainability.