ArticleBMC medical education2025
Integrating AI into clinical education: evaluating general practice trainees' proficiency in distinguishing AI-generated hallucinations and impacting factors.
Article in BMC medical education, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 18 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
18 citing papers in PubMed.
- Assessing the integration of digitalization and AI in general medicine curricula in German-speaking Europe: a comprehensive survey.Medical education online · 2026Article
- AI-Empowered Nuclear Medicine Education, Part 1: Theoretical and Ethical Foundation.Journal of nuclear medicine technology · 2026Article
- AI-Empowered Nuclear Medicine Education, Part 3: Practical AI Applications for Learners.Journal of nuclear medicine technology · 2026Article
- Evaluating the Performance and Fragility of Large Language Models on the Self-Assessment for Neurological Surgeons.Neurosurgery · 2026Article
- Current Landscape of Generative AI Use as a Search Engine Among Resident Physicians: Cross-Sectional Study.JMIR AI · 2026Article
- Microlearning in medical histology: a longitudinal evaluation of faculty-created five-minute videos using five theoretical frameworks and Kirkpatrick assessment.BMC medical education · 2026Article
- Awareness, Educational Needs, and Curriculum Preferences Regarding AI and Medical Big Data Education Among Clinical Medicine Undergraduates: Cross-Sectional Survey Study.JMIR formative research · 2026Article
- Large Language Model-Generated Differential Diagnoses in Radiology Education: Comparison with a Standard Casebook.Diagnostics (Basel, Switzerland) · 2026Article
- AI and the transformation of clinical reasoning: A classroom perspective.PLOS digital health · 2026Article
- Validation of the Use of a Large Language Model for Detecting Sentiment in Student Course Evaluation.Family medicine · 2026Article
- Large Language Model-Generated Patient Education Materials in Pediatric Ophthalmology: A Scoping Review.Clinical ophthalmology (Auckland, N.Z.) · 2026Review
- AI and algorithmic literacy among health workers: a scoping review through a digital health literacy lens.Frontiers in public health · 2026Article
- Feasibility of a Specialized Large Language Model for Postgraduate Medical Examination Preparation: Single-Center Proof-Of-Concept Study.JMIR formative research · 2025Article
- GastroGPT Pioneering Specialized AI in Gastroenterology: Strengths, Pitfalls, and the Road to Clinical Integration.JGH open : an open access journal of gastroenterology and hepatology · 2025Article
- Evaluating large language models using national endodontic specialty examination questions: are they ready for real-world dentistry?BMC medical education · 2025Article
- Leveraging AI to Investigate Child Maltreatment Text Narratives: Promising Benefits and Addressable Risks.JMIR pediatrics and parenting · 2025Article
- Article
- Artificial intelligence in perioperative medicine education: A feasibility test of case-based learning.Journal of perioperative practiceArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
12 authors.
Funding
Abstract
objectiveTo assess the ability of General Practice (GP) Trainees to detect AI-generated hallucinations in simulated clinical practice, ChatGPT-4o was utilized. The hallucinations were categorized into three types based on the accuracy of the answers and explanations: (1) correct answers with incorrect or flawed explanations, (2) incorrect answers with explanations that contradict factual evidence, and (3) incorrect answers with correct explanations.
methodsThis multi-center, cross-sectional survey study involved 142 GP Trainees, all of whom were undergoing General Practice Specialist Training and volunteered to participate. The study evaluated the accuracy and consistency of ChatGPT-4o, as well as the Trainees' response time, accuracy, sensitivity (d'), and response tendencies (β). Binary regression analysis was used to explore factors affecting the Trainees' ability to identify errors generated by ChatGPT-4o.
resultsA total of 137 participants were included, with a mean age of 25.93 years. Half of the participants were unfamiliar with AI, and 35.0% had never used it. ChatGPT-4o's overall accuracy was 80.8%, which slightly decreased to 80.1% after human verification. However, the accuracy for professional practice (Subject 4) was only 57.0%, and after human verification, it dropped further to 44.2%. A total of 87 AI-generated hallucinations were identified, primarily occurring at the application and evaluation levels. The mean accuracy of detecting these hallucinations was 55.0%, and the mean sensitivity (d') was 0.39. Regression analysis revealed that shorter response times (OR = 0.92, P = 0.02), higher self-assessed AI understanding (OR = 0.16, P = 0.04), and more frequent AI use (OR = 10.43, P = 0.01) were associated with stricter error detection criteria.
conclusionsThe study concluded that GP trainees faced challenges in identifying ChatGPT-4o's errors, particularly in clinical scenarios. This highlights the importance of improving AI literacy and critical thinking skills to ensure effective integration of AI into medical education.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.