Evidence map›Paper›PMID 40922522›Full record

ArticleEuropean journal of clinical investigation2025

Influence of medical educational background on the diagnostic quality of ChatGPT-4 responses in internal medicine: A pilot study.

Nicolò Gilardi, Massimo Ballabio, Francesco Ravera, Lorenzo Ferrando, Mario Stabile, Andrea Bellodi, Giovanni Talerico, Benedetta Cigolini, Carlo Genova, Federico Carbone and 4 more

Abstract read
In one paragraph

Article in European journal of clinical investigation, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 3 papers.

0numbers the graph read from it
0cells of the map it votes in
3citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

3 citing papers in PubMed.

  1. Article
  2. Article
  3. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

14 authors.

Nicolò GilardiDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.ORCID https://orcid.org/0009-0005-0487-1772
Massimo BallabioDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Francesco RaveraDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Lorenzo FerrandoDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Mario StabileDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Andrea BellodiIRCCS Policlinico San Martino, Genoa, Italy.
Giovanni TalericoInternal Medicine, Azienda Ospedaliera San Giovanni Addolorata, Rome, Italy.
Benedetta CigoliniDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Carlo GenovaDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Federico CarboneFirst Clinic of Internal Medicine, Department of Internal Medicine, University of Genoa, Genoa, Italy.ORCID https://orcid.org/0000-0003-2957-4078
Fabrizio MontecuccoFirst Clinic of Internal Medicine, Department of Internal Medicine, University of Genoa, Genoa, Italy.ORCID https://orcid.org/0000-0003-0823-8729
Christian BraccoS.C Medicina Interna AO S. Croce e Carle, Cuneo, Italy.
Alberto BallestreroDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.
Gabriele ZoppoliDepartment of Internal Medicine and Medical Specialties (DiMI), Università degli Studi di Genova, Genoa, Italy.

Funding

Fondazione AIRC per la ricerca sul cancro ETS IG 21761
6 · The paper itself

Abstract

This pilot study evaluated the influence of medical background on the diagnostic quality of ChatGPT-4's responses in Internal Medicine. Third-year students, residents and specialists summarised five complex NEJM clinical cases before querying ChatGPT-4. Diagnostic ranking, assessed by independent experts, revealed that residents significantly outperformed students (OR 2.33, p = .007); though overall performance was low. These findings indicate that user expertise and concise case summaries are critical for optimising AI diagnostics, highlighting the need for enhanced AI training and user interaction strategies.

Indexed as

Clinical CompetenceInternal MedicineInternship and ResidencyStudents, MedicalFemaleGenerative Artificial IntelligenceHumansMalePilot Projectsartificial intelligenceChatGPT‐4clinical decision makingdiagnostic rankinginternal medicinelarge language models

Identifiers

PMID40922522
PMCPMC12517243

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.