Evidence map›Paper›PMID 39008829›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2024

Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency.

Cathy Shyr, Randall W Grout, Nan Kennedy, Yasemin Akdas, Maeve Tischbein, Joshua Milford, Jason Tan, Kaysi Quarles, Terri L Edwards, Laurie L Novak and 3 more

Abstract read
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Reply to Layne et al.'s Letter to the Editor.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  11. Article
  12. Article
  13. Article
  14. Review
  15. Artificial intelligence: Revolutionizing pediatric emergency care - A narrative review.International journal of critical illness and injury science
    Review
  16. Review
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

13 authors.

Cathy ShyrDepartment of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.ORCID 0000-0001-7466-0034
Randall W GroutDepartment of Pediatrics, Indiana University School of Medicine, Indianapolis, IN 46202, United States.ORCID 0000-0002-2911-6782
Nan KennedyVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Yasemin AkdasDivision of Emergency Medicine Research, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
Maeve TischbeinVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Joshua MilfordVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Jason TanVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Kaysi QuarlesVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Terri L EdwardsVanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Laurie L NovakDepartment of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
Jules WhiteDepartment of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37240, United States.
Consuelo H WilkinsDepartment of Medicine, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
Paul A HarrisDepartment of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.

Funding

Catalyzing and Harmonizing Operational Innovation for Recruitment (CHOIR)U24TR004432 · NCATS · VANDERBILT UNIVERSITY MEDICAL CENTER · PI Paul A. Harris, CONSUELO HOPKINS WILKINS · 2023 to 2026
$20.9M
Development and implementation of statistical machine learning methods to shorten rare disease odysseysK99LM014429 · NLM · VANDERBILT UNIVERSITY MEDICAL CENTER · PI SHYR, CATHY · 2024 to 2025
$181k
NCATS NIH HHS 1K99LM014429-01NCATS NIH HHS U24 TR004432NLM NIH HHS K99 LM014429
6 · The paper itself

Abstract

objectiveReturning aggregate study results is an important ethical responsibility to promote trust and inform decision making, but the practice of providing results to a lay audience is not widely adopted. Barriers include significant cost and time required to develop lay summaries and scarce infrastructure necessary for returning them to the public. Our study aims to generate, evaluate, and implement ChatGPT 4 lay summaries of scientific abstracts on a national clinical study recruitment platform, ResearchMatch, to facilitate timely and cost-effective return of study results at scale. MATERIALS AND

methodsWe engineered prompts to summarize abstracts at a literacy level accessible to the public, prioritizing succinctness, clarity, and practical relevance. Researchers and volunteers assessed ChatGPT-generated lay summaries across five dimensions: accuracy, relevance, accessibility, transparency, and harmfulness. We used precision analysis and adaptive random sampling to determine the optimal number of summaries for evaluation, ensuring high statistical precision.

resultsChatGPT achieved 95.9% (95% CI, 92.1-97.9) accuracy and 96.2% (92.4-98.1) relevance across 192 summary sentences from 33 abstracts based on researcher review. 85.3% (69.9-93.6) of 34 volunteers perceived ChatGPT-generated summaries as more accessible and 73.5% (56.9-85.4) more transparent than the original abstract. None of the summaries were deemed harmful. We expanded ResearchMatch's technical infrastructure to automatically generate and display lay summaries for over 750 published studies that resulted from the platform's recruitment mechanism. DISCUSSION AND

conclusionImplementing AI-generated lay summaries on ResearchMatch demonstrates the potential of a scalable framework generalizable to broader platforms for enhancing research accessibility and transparency.

Indexed as

Abstracting and IndexingArtificial IntelligenceBiomedical ResearchHumansInformation Disseminationartificial intelligencelarge language modelResearchMatchreturn of study resultstext summarization

Identifiers

PMID39008829
PMCPMC11413467

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.