ArticleScience advances2025
Delving into LLM-assisted writing in biomedical publications through excess vocabulary.
Article in Science advances, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 34 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
34 citing papers in PubMed.
- Human medical documentation significantly outperforms ChatGPT-4o in critical clinical dimensions: A blinded comparative assessment in paediatric orthopaedics.Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA · 2026Trial
- Accelerated use of artificial intelligence-associated vocabulary in Japanese surgical journals after 2023.Surgery today · 2026Article
- Authors, reviewers, editors… smitten and bitten by the artificial intelligence (AI) bug!Indian journal of anaesthesia · 2026Article
- Article
- Evolution of Artificial Intelligence Adoption inOrthopaedic journal of sports medicine · 2026Article
- The rise of large language models and the direction and impact of US federal research funding.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Reply to Topaz and Bahl: Interpreting a post-2022 lexical shift in academic prose.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- AI will reorganize science. Will research remain a human enterprise?Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- An automatic consult reply system for therapeutic plasma exchange using retrieval-augmented generation.Vox sanguinis · 2026Article
- Large Language Models and the Future of Peer-Reviewed Medical Publishing: Challenges, Editorial Responses, and a Framework for Responsible Integration.Journal of Korean medical science · 2026Review
- Artificial Intelligence for Discovery in Life Sciences.Bioconjugate chemistry · 2026Review
- Artificial Intelligence Use in Medical Writing: On the Meaning and Limits of Disclosure.JMA journal · 2026Article
- The detectability paradox: bilingual medical report generation with open-weight models and the limits of human oversight.Journal of the American Medical Informatics Association : JAMIA · 2026Article
- Linguistic shifts in dental scientific writing in the large language model era.Research integrity and peer review · 2026Article
- Reimagining plant science training in the era of generative artificial intelligence: a global perspective.The Plant cell · 2026Review
- On the makeup of high-impact reviews in ecology.Eco-Environment & Health · 2026Article
- Will the widespread use of large language models in scientific writing undermine scientists' critical thinking?PLoS biology · 2026Article
- Who Accommodates Whom? Bidirectional Linguistic Accommodation and Progressive Interpersonal Convergence in Human-AI Conversations.Behavioral sciences (Basel, Switzerland) · 2026Article
- Academic journals' AI policies fail to curb the surge in AI-assisted academic writing.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- General-Purpose Models for the Chemical Sciences: LLMs and Beyond.Chemical reviews · 2026Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations, can produce inaccurate information, and reinforce existing biases. Yet, many scientists use them for their scholarly writing. But how widespread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: We study vocabulary changes in more than 15 million biomedical abstracts from 2010 to 2024 indexed by PubMed and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the COVID pandemic.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.