Evidence map›Paper›PMID 39781102›Full record

ArticleJB & JS open access

Source Characteristics Influence AI-Enabled Orthopaedic Text Simplification: Recommendations for the Future.

Saman Andalib, Sean S Solomon, Bryce G Picton, Aidin C Spina, John A Scolaro, Ariana M Nelson

Abstract read
In one paragraph

Article in JB & JS open access. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 8 papers.

0numbers the graph read from it
0cells of the map it votes in
8citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

8 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Saman AndalibUniversity of California, Irvine, School of Medicine, Irvine, California.ORCID https://orcid.org/0009-0002-3281-8256
Sean S SolomonUniversity of California, Irvine, School of Medicine, Irvine, California.ORCID https://orcid.org/0000-0002-4001-6141
Bryce G PictonUniversity of California, Irvine, School of Medicine, Irvine, California.ORCID https://orcid.org/0000-0003-3539-6167
Aidin C SpinaUniversity of California, Irvine, School of Medicine, Irvine, California.ORCID https://orcid.org/0000-0003-3994-9646
John A ScolaroDepartment of Orthopaedic Surgery, University of California, Irvine, Medical Center, Orange, California.ORCID https://orcid.org/0000-0001-7926-5017
Ariana M NelsonDepartment of Anesthesiology, University of California, Irvine, Medical Center, Orange, California.ORCID https://orcid.org/0000-0003-1575-1635

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Background: This study assesses the effectiveness of large language models (LLMs) in simplifying complex language within orthopaedic patient education materials (PEMs) and identifies predictive factors for successful text transformation. Methods: We transformed 48 orthopaedic PEMs using GPT-4, GPT-3.5, Claude 2, and Llama 2. The readability, quantified by the Flesch-Kincaid Reading Ease (FKRE) and Flesch-Kincaid Grade Level (FKGL) scores, was measured before and after transformation. Analysis included text characteristics such as syllable count, word length, and sentence length. Statistical and machine learning methods evaluated the correlations and predictive capacity of these features for transformation success. Results: All LLMs improved FKRE and FKGL scores (p < 0.01). GPT-4 showed superior performance, transforming PEMs to a seventh-grade reading level (mean FKGL, 6.72 ± 0.99), with higher FKRE and lower FKGL than other models. GPT-3.5, Claude 2, and Llama 2 significantly shortened sentences and overall text length (p < 0.01). Importantly, correlation analysis revealed that transformation success varied substantially with the model used, depending on original text factors such as word length and sentence complexity. Conclusions: LLMs successfully simplify orthopaedic PEMs, with GPT-4 leading in readability improvement. This study highlights the importance of initial text characteristics in determining the effectiveness of LLM transformations, offering insights for optimizing orthopaedic health literacy initiatives using artificial intelligence (AI). Clinical Relevance: This study provides critical insights into the ability of LLMs to simplify complex orthopaedic PEMs, enhancing their readability without compromising informational integrity. By identifying predictive factors for successful text transformation, this research supports the application of AI in improving health literacy, potentially leading to better patient comprehension and outcomes in orthopaedic care.

Identifiers

PMID39781102
PMCPMC11703440

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.