ArticlePakistan journal of medical sciences2025
Exploring the role of DeepSeek-R1, ChatGPT-4, and Google Gemini in medical education: How valid and reliable are they?
Article in Pakistan journal of medical sciences, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 9 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
9 citing papers in PubMed, 1 synthesis or guideline pooled it.
- The effectiveness of GAI-assisted teaching methods in medical education: a systematic review and meta-analysis.Frontiers in public health · 2026Pooled it
- Applications of DeepSeek in Medicine: Bibliometric Analysis and Scoping Review.Journal of medical Internet research · 2026Article
- Use of Artificial Intelligence in Undergraduate Medical Student Assessment, Scientific Writing & Peer Review: Promise, Pitfalls, and the Need for Educational Governance.Pakistan journal of medical sciences · 2026Article
- Evaluation of the approaches of different specialties and artificial intelligence systems on traumatic dental injuries.BMC oral health · 2026Article
- Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility.Diagnostics (Basel, Switzerland) · 2026Article
- Blind spots in artificial intelligence systems: poor identification of self-generated medical images-evidence-based cross-sectional study.Frontiers in medicine · 2026Article
- Comparing Chinese-language large language models for caregiver questions about developmental dysplasia of the hip.Frontiers in pediatrics · 2026Article
- Decoding Alexander the Great's gastrointestinal cause of death using artificial wisdom: An artificial intelligence-human inquiry into a medical mystery.World journal of gastroenterology · 2025Article
- Comparative performance of ChatGPT-4o, ChatGPT-5, and gemini 2.5 flash on Persian internal medicine subspecialty board exams.Scientific reports · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Objective: In recent years, Artificial Intelligence (AI) has led to rapid advancements in science, technology, industries, healthcare settings, and medical education. A Chinese-built large language model, DeepSeek-R1, inspires the scientific community as an affordable and open alternative to earlier established US-based AI models, ChatGPT-4 and Google Gemini 1.5 Pro. This study aimed to explore the role of "DeepSeek-R1, ChatGPT-4 and Google Gemini 1.5 Pro" and to assess the validity and reliability of these AI tools in medical education. Methods: The current cross-sectional study was performed in the Department of Physiology, College of Medicine, King Saud University, Riyadh, Saudi Arabia during the period January 25, 2025, to February 28, 2025. The Multiple-Choice Questions (MCQs) bank was created with a pool of basic medical sciences (60 MCQs) and clinical medical sciences (40 MCQs). The one hundred MCQs were prepared from various medical textbooks, journals, and examination pools. The MCQs were individually entered into the given area of the "DeepSeek-R1, ChatGPT-4 and Google Gemini 1.5 Pro" to assess the level of knowledge in various disciplines of medical sciences. Results: The marks obtained in basic medical sciences by DeepSeek R1 47/60 (78.33%), ChatGPT-4 47/60 (78.33%), and Google Gemini 1.5 Pro 49/60 (81.7%). However, in clinical medical sciences, the marks obtained by DeepSeek R1 were 35/40 (87.5%), ChatGPT-4 36/40 (90.0%), and Google Gemini 1.5 Pro 33/40 (82.5%). The total marks obtained by DeepSeekR1 were 82/100 (82.0%), Chat GPT-4 84/100 (84.0%), and Google Gemini-1.5 Pro 82/100 (82.0%). Conclusions: The Chinese-based DeepSeek-R1, the US-based ChatGPT-4, and Google Gemini-1.5 Pro achieved similar scores, exceeding 80% marks, in various medical sciences subjects. The study findings demonstrate that the knowledge, validity, and reliability levels of DeepSeek R1, ChatGPT-4, and Google Gemini 1.5 Pro are similar for their potential future use in medical education.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.