ArticleJMIR medical informatics2023
Applications of the Natural Language Processing Tool ChatGPT in Clinical Practice: Comparative Study and Augmented Systematic Review.
Article in JMIR medical informatics, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 28 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
28 citing papers in PubMed, 4 syntheses or guidelines pooled it, 73 citations in OpenAlex.
- Evaluation of a large language model (ChatGPT) versus human researchers in assessing risk-of-bias and community engagement levels: a systematic review use-case analysis.European journal of public health · 2025Pooled it
- Generative artificial intelligence use in evidence synthesis: A systematic review.Research synthesis methods · 2025Pooled it
- The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review.Journal of the American Medical Informatics Association : JAMIA · 2025Pooled it
- Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis.Journal of biomedical science · 2025Pooled it
- ChatGPT-4-Based Automated Preliminary Clinical Reporting in Myocardial Perfusion Imaging: A Pilot Evaluation.Diagnostics (Basel, Switzerland) · 2026Article
- Artificial Intelligence Resources for the Screening of Titles and Abstracts in Systematic Reviews: A Scoping Review.Cochrane evidence synthesis and methods · 2026Review
- Value and Credibility of Meta-Analysis: Tutorial on Enhancing Methodological Rigor and AI-Powered Efficiency.Journal of medical Internet research · 2026Article
- Natural language processing to enhance rheumatoid arthritis care in clinical studies: a scoping review of applications, data, approaches, challenges and future directions.Rheumatology international · 2026Article
- Large Language Models for Clinical Narrative Processing: Methods, Applications, and Challenges.Methods and protocols · 2026Article
- Ontology-driven generation of parameters for health technology assessment models: a prompt engineering study.International journal of technology assessment in health care · 2026Article
- Use of artificial intelligence for health science in low- and middle-income countries: NIH portfolio landscape, gaps and opportunities.BMJ open · 2026Article
- Fine-tuning and structured prompting strategies for question answering over full-text biomedical research articles.PloS one · 2026Article
- Artificial intelligence and ChatGPT literacy among surgical healthcare professionals: knowledge, attitudes, and perceived clinical utility.Frontiers in digital health · 2026Article
- Information Extraction of Doctoral Theses Using Two Different Large Language Models vs Health Services Researchers: Development and Usability Study.JMIR formative research · 2025Article
- RADEX: a rule-based clinical and radiology data extraction tool demonstrated on thyroid ultrasound reports.European radiology · 2025Article
- Cross language transformation of free text into structured lobectomy surgical records from a multi center study.Scientific reports · 2025Article
- Assessing Patient-Reported Satisfaction With Care and Documentation Time in Primary Care Through AI-Driven Automatic Clinical Note Generation: Protocol for a Proof-of-Concept Study.JMIR research protocols · 2025Article
- Breaking Bones, Breaking Barriers: ChatGPT, DeepSeek, and Gemini in Hand Fracture Management.Journal of clinical medicine · 2025Article
- Generative Artificial Intelligence for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations: An ISPOR Working Group Report.Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research · 2025Review
- Exploring the Credibility of Large Language Models for Mental Health Support: Protocol for a Scoping Review.JMIR research protocols · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThis research integrates a comparative analysis of the performance of human researchers and OpenAI's ChatGPT in systematic review tasks and describes an assessment of the application of natural language processing (NLP) models in clinical practice through a review of 5 studies.
objectiveThis study aimed to evaluate the reliability between ChatGPT and human researchers in extracting key information from clinical articles, and to investigate the practical use of NLP in clinical settings as evidenced by selected studies.
methodsThe study design comprised a systematic review of clinical articles executed independently by human researchers and ChatGPT. The level of agreement between and within raters for parameter extraction was assessed using the Fleiss and Cohen κ statistics.
resultsThe comparative analysis revealed a high degree of concordance between ChatGPT and human researchers for most parameters, with less agreement for study design, clinical task, and clinical implementation. The review identified 5 significant studies that demonstrated the diverse applications of NLP in clinical settings. These studies' findings highlight the potential of NLP to improve clinical efficiency and patient outcomes in various contexts, from enhancing allergy detection and classification to improving quality metrics in psychotherapy treatments for veterans with posttraumatic stress disorder.
conclusionsOur findings underscore the potential of NLP models, including ChatGPT, in performing systematic reviews and other clinical tasks. Despite certain limitations, NLP models present a promising avenue for enhancing health care efficiency and accuracy. Future studies must focus on broadening the range of clinical applications and exploring the ethical considerations of implementing NLP applications in health care settings.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.