Large Language Model as an Emerging Technology in Medical Education: An Analysis of Scholarly Collaboration, Citation, and Vocabulary Co-occurrence

Document Type : Original Article

Authors

1 Visiting Faculty in Medical Library and Information Sciences Department, School of Allied Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran

2 Associate Professor, Department of Knowledge and Information Science; Faculty of Educational Sciences and Psychology, Shahid Chamran University of Ahvaz; Ahvaz, Iran

3 PhD Candidate in Knowledge and Information Science, Faculty of Educational Sciences and Psychology, Shahid Chamran University of Ahvaz; Ahvaz, Iran

4 PhD Candidate in Knowledge and Information Science, Faculty of Educational Sciences and Psychology, Alzahra University, Tehran, Iran

10.22062/sdme.2025.200769.1504
Abstract
Background: The emergence of large language models has led to significant progress in various fields. In recent years, researchers have had a special focus on its applications in medical education.
Objectives: This study explores the social and intellectual framework of research on
large language models as a growing technology in medical education across the Web of Science database.
Methods: The present study is a quantitative study performed using scientometric techniques. The statistical population consisted of texts containing 643 documents from the year of this study. Following the Prisma guidelines, studies were screened based on their titles, abstracts, and keywords, leading to the selection of 233 documents for the research population. No restrictions (such as year, time, country) were applied to the search. Data analysis was undertaken using VOSviewer software.
Results: The study found that the United States was the most active country, generating 90 papers and receiving 537 citations. Author Lee, Hyunsoo, received 163 citations, with JMIR Medical Education being among the most frequent authors and journals with 30 papers and 352 citations. The keywords “artificial intelligence” with 144 repetitions, “ChatGPT” with 131 repetitions, “medical education” with 97 repetitions, and “large language models” with 89 repetitions make them highly frequent words. Four clusters were formed from the keywords obtained in the study”.
Conclusion: The research in large language models and medical education is growing rapidly, with organizations revealing a strong interest in international cooperation. This field has great potential for development. Nevertheless, countries with more publications have not necessarily received more citations.

Highlights

Shahnaz Khademizadeh: (Google Scholar) (PubMed)

Zeinab Jozi: (Google Scholar) (PubMed)

Hajar Beigi: (Google Scholar) (PubMed)

 

Keywords


  1. Nassiri K, Akhloufi MA. Recent advances in large language models for healthcare. BioMedInformatics. 2024;4(2):1097-143. doi: 3390/biomedinformatics4020062.
  2. Ravi A, Neinstein A, Murray SG. Large language models and medical education: preparing for a rapid transformation in how trainees will learn to be doctors. ATS Sch. 2023 Jun 14;4(3):282-292. doi: 34197/ats-scholar.2023-0036PS. [PMID: 37795112] [PMCID: PMC10547030]
  3. Safranek CW, Sidamon-Eristoff AE, Gilson A, Chartash D. The role of large language models in medical education: applications and implications. JMIR Med Educ. 2023 Aug 14:9:e50945. doi: 2196/50945. [PMID: 37578830] [PMCID: PMC10463084]
  4. Bahl L, Baker J, Cohen P, Jelinek F, Lewis B, Mercer R. Recognition of continuously read natural corpus. InICASSP'78. IEEE International Conference on Acoustics, Speech, and Signal Processing 1978 Apr 10(3), 422-424. IEEE. doi: 1109/ICASSP.1978.1170402.
  5. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. Journal of Machine Learning Research. 2003;3:993-1022.
  6. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Advances in Neural Information Processing Systems. 2017;30:1-11. doi: 2196/46900.
  7. Heilbron M, Armeni K, Schoffelen JM, Hagoort P, De Lange FP. A hierarchy of linguistic predictions during natural language comprehension. PNAS. 2022 Aug 9;119(32):e2201968119. doi: doi: 1073/pnas.2201968119.
  8. Shao CY, Li H, Liu XL, Li C, Yang LQ, Zhang YJ, et al. Appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in different language contexts: survey study. Interact J Med Res. 2023 Aug 14:12:e46900. doi: 2196/46900. [PMID: 37578819] [PMCID: PMC10463083]
  9. Kleinig O, Gao C, Bacchi S. This too shall pass: the performance of ChatGPT‐3.5, ChatGPT‐4 and New Bing in an Australian medical licensing examination. Med J Aust. 2023 Sep 4;219(5):237. doi: 5694/mja2.52061. [PMID: 37528548]
  10. Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023 Jun 1:9:e48291. doi: 2196/48291. [PMID: 37261894] [PMCID: PMC10273039]
  11. Liu L, Qu S, Zhao H, Kong L, Xie Z, Jiang Z, et al. Global trends and hotspots of ChatGPT in medical research: a bibliometric and visualized study. Front Med (Lausanne). 2024 May 16:11:1406842. doi: 3389/fmed.2024.1406842. [PMID: 38818395] [PMCID: PMC11137200]
  12. Li Z, Li F, Fu Q, Wang X, Liu H, Zhao Y, et al. Large language models and medical education: a paradigm shift in educator roles. Smart Learning Environments. 2024;11(1):26. doi: 1186/s40561-024-00313-w.
  13. Wu Y, Zheng Y, Feng B, Yang Y, Kang K, Zhao A. Embracing ChatGPT for medical education: exploring its impact on doctors and medical students. JMIR Med Educ. 2024 Apr 10:10:e52483. doi: 2196/52483. [PMID: 38598263] [PMCID: PMC11043925]
  14. Eysenbach G. The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Med Educ. 2023 Mar 6:9:e46885. doi: 2196/46885. [PMID: 36863937] [PMCID: PMC10028514]
  15. Khalpey Z, Kumar U, King N, Abraham A, Khalpey AH. Large language models take on cardiothoracic surgery: a comparative analysis of the performance of four models on American Board of Thoracic Surgery exam questions in 2023. 2024 Jul 22;16(7):e65083. doi: 10.7759/cureus.65083. [PMID: 39171020] [PMCID: PMC11337141]
  16. Kitamura FC. ChatGPT is shaping the future of medical writing but still requires human judgment. 2023 Apr;307(2):e230171. doi: 10.1148/radiol.230171. [PMID: 36728749]
  17. Morreel S, Mathysen D, Verhoeven V. Aye, AI! ChatGPT passes multiple-choice family medicine exam. Med Teach. 2023 Jun;45(6):665-6. doi: 1080/0142159X.2023.2187684. [PMID: 36905610]
  18. Lee H. The rise of ChatGPT: Exploring its potential in medical education. Anat Sci Educ. 2024 Jul-Aug;17(5):926-931. doi: 1002/ase.2270. [PMID: 36916887]
  19. Civaner MM, Uncu Y, Bulut F, Chalil EG, Tatli A. Artificial intelligence in medical education: a cross-sectional needs assessment. BMC Med Educ. 2022 Nov 9;22(1):772. doi: 1186/s12909-022-03852-3. [PMID: 36352431] [PMCID: PMC9646274]
  20. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023 Feb 9;2(2):e0000198. doi: 1371/journal.pdig.0000198. [PMID: 36812645] [PMCID: PMC9931230]
  21. De Bellis N. Bibliometrics and citation analysis: from the science citation index to cybermetrics. Journal of the American Society for Information Science and Technology; 2010;61(1):205- doi: 10.1002/asi.21181.
  22. Mayta-Tovalino F, Espinoza-Carhuancho F, Alvitez-Temoche D, Mauricio-Vilchez C, Munive-Degregori A, Barja-Ore J. Scientometric analysis on the use of ChatGPT, artificial intelligence, or intelligent conversational agent in the role of medical training. Medical Education. 2024 Mar 1;25(2):100873. doi: 1016/j.edumed.2023.100873.
  23. Mosavi A, Imre F, Hung VT. ChatGPT and large language models in healthcare; a bibliometrics analysis and review. Proceedings of the IEEE 11th International Conference on Computational Cybernetics and Cyber-Medical Systems; 2024 Apr 4-6; Hanoi, Vietnam. 2024: 350-4.
  24. Wang S, Yang L, Li M, Zhang X, Tai X. Medical Education and Artificial Intelligence: Web of Science–Based Bibliometric Analysis (2013-2022). JMIR Med Educ. 2024 Oct 10:10:e51411. doi: 2196/51411. [PMID: 39388721] [PMCID: PMC11486481]
  25. VOSviewer workbook. VOSviewer: A Guide for Research Visualization. [cited 2023 Jul 8]. Available from: https://www.griffith.edu.au/__data/assets/pdf_file/0030/1905483/VOSviewer-guide-2023.pdf?utm_source=chatgpt.com
  26. Van Eck NJ, Waltman L. VOSviewer: Visualizing scientific landscapes. Version 1.6.20. Leiden: Centre for Science and Technology Studies, Leiden University. [cited 2023 Oct 31]. Available from: https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.20.pdf.
  27. Van Eck N, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. 2010 Aug;84(2):523-38. doi: 10.1007/s11192-009-0146-3. [PMID: 20585380] [PMCID: PMC2883932]
  28. Chan S. Knowledge Management in Smart Organizations. InKnowledge Management. In: Mezgar I. Integration of ICT in smart organizations. Hershey, Pennsylvania: IGI Global Scientific Publishing; 2008: 3385-408.
  29. Liu Q, Ali NL, Lee HY. Applying VOSviewer in a bibliometric review on English language teacher education research: An analysis of narratives, networks and numbers. Cogent Education. 2025;12(1):2449728. doi: 1080/2331186X.2025.2449728.
  30. Hosseini E, Ghaebi A, Baradar R. Bibliometrics and Mapping of Co-words in the Field of Linked Data. Scientometrics Research Journal, 2021; 7((Issue 1, spring & summer)): 91-116. doi: 22070/rsci.2020.4904.1333.
  31. Fallah Barzeghar M, khalili A. Bibliometric Analysis and Systematic Review of Global Theoretical Context in the Field of Urban Health. Journal of Iranian Architecture & Urbanism(JIAU). 2022;13(2):351-72. [In Persian]
  32. Raman R, Lathabhai H, Diwakar S, Nedungadi P. Early research trends on ChatGPT: insights from altmetrics and science mapping analysis. International Journal of Emerging Technologies in Learning (iJET). 2023;18(19):13-31. doi:3991/ijet.v18i19.41793.
  33. Yang X, Chen A, PourNejatian N, Shin HC, Smith KE, Parisien C, et al. A large language model for electronic health records. NPJ Digit Med. 2022 Dec 26;5(1):194. doi: 1038/s41746-022-00742-2. [PMID: 36572766] [PMCID: PMC9792464]
  34. Hildt E. What Is the Role of Explainability in Medical Artificial Intelligence? A Case-Based Approach. Bioengineering (Basel). 2025 Apr 2;12(4):375. doi: 3390/bioengineering12040375. [PMID: 40281735] [PMCID: PMC12025101]
  35. Ahn S. The impending impacts of large language models on medical education. Korean J Med Educ. 2023 Mar;35(1):103-107. doi: 3946/kjme.2023.253. [PMID: 36858381] [PMCID: PMC10020064]
  36. Weber MT, Noll R, Marchl A, Facchinello C, Grünewaldt A, Hügel C, et al. MedBot vs RealDoc: efficacy of large language modeling in physician-patient communication for rare diseases. Journal of the American Medical Informatics Association. 2025 May;32(5):775-83. doi: 1093/jamia/ocaf034.