ArticleHealthcare (Basel, Switzerland)2023
ChatGPT Knowledge Evaluation in Basic and Clinical Medical Sciences: Multiple Choice Question Examination-Based Performance.
Article in Healthcare (Basel, Switzerland), 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 44 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
44 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- The performance of ChatGPT and other large language models on multiple-choice questions in biomedical disciplines: A meta-analysis.Anatomical sciences education · 2026Pooled it
- Performance of large language models in medical licensing examinations: a systematic review and meta-analysis.Journal of educational evaluation for health professions · 2025Pooled it
- Guideline-based clinical reasoning in periodontology education: a comparative study of residents and large language models.BMC medical education · 2026Article
- Susceptibility of Assessment Types to AI-Generated Content in Digital Health and Health Information Management Education: Quasi-Experimental Pilot Study.JMIR medical education · 2026Article
- Performance of 5 AI Models on United States Medical Licensing Examination Step 1 Questions: Comparative Observational Study.JMIR AI · 2026Article
- Use of ChatGPT among Student Nurses in Baguio City, Philippines: A Cross-sectional Study.Acta medica Philippina · 2026Article
- Performance comparison of large language models on pediatric dentistry questions in the Turkish dentistry specialization examination.BMC medical education · 2025Article
- Comparing ChatGPT and DeepSeek for Assessment of Multiple-Choice Questions in Orthopedic Medical Education: Cross-Sectional Study.JMIR formative research · 2025Article
- Comparative performance of large language models in answering periodontology questions from the Turkish Dental Specialty Examination: a cross-sectional study on accuracy and coverage.BMC oral health · 2025Article
- Evaluation of ChatGPT's performance on answering pediatric urology questions based on association guidelines.Canadian Urological Association journal = Journal de l'Association des urologues du Canada · 2025Article
- Collaborative intelligence in AI: Evaluating the performance of a council of AIs on the USMLE.PLOS digital health · 2025Article
- Article
- Exploring the role of DeepSeek-R1, ChatGPT-4, and Google Gemini in medical education: How valid and reliable are they?Pakistan journal of medical sciences · 2025Article
- Comparison of artificial intelligence systems in answering prosthodontics questions from the dental specialty exam in Turkey.Journal of dental sciences · 2025Article
- Performance of ChatGPT in answering the oral pathology questions of various types or subjects from Taiwan National Dental Licensing Examinations.Journal of dental sciences · 2025Article
- Areas of research focus and trends in the research on the application of AIGC in healthcare.Journal of health, population, and nutrition · 2025Article
- While GPT-3.5 is unable to pass the Physician Licensing Exam in Taiwan, GPT-4 successfully meets the criteria.Journal of the Chinese Medical Association : JCMA · 2025Article
- Large Language Models in Biochemistry Education: Comparative Evaluation of Performance.JMIR medical education · 2025Article
- Artificial intelligence-based chatbot assistance in clinical decision-making for medically complex patients in oral surgery: a comparative study.BMC oral health · 2025Article
- Exploring medical students' intention to use of ChatGPT from a programming course: a grounded theory study in China.BMC medical education · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
The Chatbot Generative Pre-Trained Transformer (ChatGPT) has garnered great attention from the public, academicians and science communities. It responds with appropriate and articulate answers and explanations across various disciplines. For the use of ChatGPT in education, research and healthcare, different perspectives exist with some level of ambiguity around its acceptability and ideal uses. However, the literature is acutely lacking in establishing a link to assess the intellectual levels of ChatGPT in the medical sciences. Therefore, the present study aimed to investigate the knowledge level of ChatGPT in medical education both in basic and clinical medical sciences, multiple-choice question (MCQs) examination-based performance and its impact on the medical examination system. In this study, initially, a subject-wise question bank was established with a pool of multiple-choice questions (MCQs) from various medical textbooks and university examination pools. The research team members carefully reviewed the MCQ contents and ensured that the MCQs were relevant to the subject's contents. Each question was scenario-based with four sub-stems and had a single correct answer. In this study, 100 MCQs in various disciplines, including basic medical sciences (50 MCQs) and clinical medical sciences (50 MCQs), were randomly selected from the MCQ bank. The MCQs were manually entered one by one, and a fresh ChatGPT session was started for each entry to avoid memory retention bias. The task was given to ChatGPT to assess the response and knowledge level of ChatGPT. The first response obtained was taken as the final response. Based on a pre-determined answer key, scoring was made on a scale of 0 to 1, with zero representing incorrect and one representing the correct answer. The results revealed that out of 100 MCQs in various disciplines of basic and clinical medical sciences, ChatGPT attempted all the MCQs and obtained 37/50 (74%) marks in basic medical sciences and 35/50 (70%) marks in clinical medical sciences, with an overall score of 72/100 (72%) in both basic and clinical medical sciences. It is concluded that ChatGPT obtained a satisfactory score in both basic and clinical medical sciences subjects and demonstrated a degree of understanding and explanation. This study's findings suggest that ChatGPT may be able to assist medical students and faculty in medical education settings since it has potential as an innovation in the framework of medical sciences and education.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.