ArticleHealth information and libraries journal2025
Use of large language models to identify pseudo-information: Implications for health information.
Article in Health information and libraries journal, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
1 citing paper in PubMed.
- Use of large language models to identify pseudo-information: Implications for health information.Health information and libraries journal · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
1 author.
Funding
Abstract
backgroundOpen-access scientific research is an essential source of health-related information and self-education. Artificial intelligence-based large language models (LMMs) may be used to identify erroneous health information.
objectiveTo investigate to what extent LMMs can be used to identify pseudo-information.
methodsFour common LMM applications (ChatGPT-4o, Claude 3.5 Sonnet, Gemini and Copilot) were used to investigate their capability to indicate erroneous information provided in an open-access article.
resultsInitially, ChatGPT-4o and Claude were able to mark the provided article as an unreliable information source, identifying most of the inaccuracy problems. The assessments provided by Gemini and Copilot were inaccurate, as several critical aspects were not identified or were misinterpreted. During the validation phase, the initially accurate assessment of ChatGPT-4o was not reproducible, and only Claude was able to detect several critical issues in this phase. The verdicts of Copilot and Gemini remained largely unaltered. DISCUSSION: Large heterogeneity exists between LMMs in identifying inaccurate pseudo-information. Replication in LMM output may constitute a significant hurdle in their application.
conclusionThe accuracy of LMMs needs to be further improved until they can be reliably used by patients for health-related online information and as assistant tools for health information and library services workers without restriction.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.