ArticleJCO clinical cancer informatics2024
Characterizing the Increase in Artificial Intelligence Content Detection in Oncology Scientific Abstracts From 2021 to 2023.
Article in JCO clinical cancer informatics, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 13 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
13 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- The role of reviewers in the era of systematic reviews and meta-analysis: A practical guide for researchers.Biomolecules & biomedicine · 2025Guideline
- Artificial intelligence in endoscopy and colonoscopy: a comprehensive bibliometric analysis of global research trends.Frontiers in medicine · 2025Pooled it
- Blinded by the Bot: Benchmarking GPT and Gemini Against Human Authors in Otolaryngology Reviews.World journal of otorhinolaryngology - head and neck surgery · 2026Article
- AI text detection in dentistry: a comparative analysis across generative models.Research integrity and peer review · 2026Article
- Article
- [How to detect scientific texts generated with artificial intelligence?]Revista medica del Instituto Mexicano del Seguro Social · 2026Article
- Humanization of AI-generated abstracts in oral radiology: Role of chatbots and dedicated platforms.Brazilian oral research · 2026Article
- Accuracy of Artificial Intelligence Detection Software for Residency Personal Statements.Journal of graduate medical education · 2025Article
- ChatGPT-4o Compared With Human Researchers in Writing Plain-Language Summaries for Cochrane Reviews: A Blinded, Randomized Non-Inferiority Controlled Trial.Cochrane evidence synthesis and methods · 2025Article
- Can ChatGPT pass the Turkish Orthopedics and Traumatology Board Examination? Turkish orthopedic surgeons versus artificial intelligence.Ulusal travma ve acil cerrahi dergisi = Turkish journal of trauma & emergency surgery : TJTES · 2025Observational
- Article
- AI detectors are poor western blot classifiers: a study of accuracy and predictive values.PeerJ · 2025Article
- Variability in Journal and Publisher Policies on Artificial Intelligence Use in Manuscript Preparation: An Orthopaedic Perspective.JB & JS open accessArticle
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
purposeArtificial intelligence (AI) models can generate scientific abstracts that are difficult to distinguish from the work of human authors. The use of AI in scientific writing and performance of AI detection tools are poorly characterized.
methodsWe extracted text from published scientific abstracts from the ASCO 2021-2023 Annual Meetings. Likelihood of AI content was evaluated by three detectors: GPTZero, Originality.ai, and Sapling. Optimal thresholds for AI content detection were selected using 100 abstracts from before 2020 as negative controls, and 100 produced by OpenAI's GPT-3 and GPT-4 models as positive controls. Logistic regression was used to evaluate the association of predicted AI content with submission year and abstract characteristics, and adjusted odds ratios (aORs) were computed.
resultsFifteen thousand five hundred and fifty-three abstracts met inclusion criteria. Across detectors, abstracts submitted in 2023 were significantly more likely to contain AI content than those in 2021 (aOR range from 1.79 with Originality to 2.37 with Sapling). Online-only publication and lack of clinical trial number were consistently associated with AI content. With optimal thresholds, 99.5%, 96%, and 97% of GPT-3/4-generated abstracts were identified by GPTZero, Originality, and Sapling respectively, and no sampled abstracts from before 2020 were classified as AI generated by the GPTZero and Originality detectors. Correlation between detectors was low to moderate, with Spearman correlation coefficient ranging from 0.14 for Originality and Sapling to 0.47 for Sapling and GPTZero.
conclusionThere is an increasing signal of AI content in ASCO abstracts, coinciding with the growing popularity of generative AI models.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.