ArticleFrontiers in neurology2023
The utility of ChatGPT in the assessment of literature on the prevention of migraine: an observational, qualitative study.
Article in Frontiers in neurology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed, 18 citations in OpenAlex.
- Mapping applications and evaluations of LLM-enabled AI chatbots for health purposes: a scoping review.Frontiers in public health · 2026Article
- Application of Artificial Intelligence in the Headache Field.Current pain and headache reports · 2024Review
- ChatGPT as an Information Source for Patients with Migraines: A Qualitative Case Study.Healthcare (Basel, Switzerland) · 2024Article
- ChatGPT for Automated Qualitative Research: Content Analysis.Journal of medical Internet research · 2024Article
- The Clinical Relevance of Artificial Intelligence in Migraine.Brain sciences · 2024Review
- Could generative artificial intelligence replace fieldwork in pain research?Scandinavian journal of pain · 2024Article
- ChatGPT's Inconsistency in the Diagnosis of Alzheimer's Disease.Journal of Alzheimer's disease reports · 2024Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: It is not known how large language models, such as ChatGPT, can be applied toward the assessment of the efficacy of medications, including in the prevention of migraine, and how it might support those claims with existing medical evidence. Methods: We queried ChatGPT-3.5 on the efficacy of 47 medications for the prevention of migraine and then asked it to give citations in support of its assessment. ChatGPT's evaluations were then compared to their FDA approval status for this indication as well as the American Academy of Neurology 2012 evidence-based guidelines for the prevention of migraine. The citations ChatGPT generated for these evaluations were then assessed to see if they were real papers and if they were relevant to the query. Results: ChatGPT affirmed that the 14 medications that have either received FDA approval for prevention of migraine or AAN Grade A/B evidence were effective for migraine. Its assessments of the other 33 medications were unreliable including suggesting possible efficacy for four medications that have never been used for the prevention of migraine. Critically, only 33/115 (29%) of the papers ChatGPT cited were real, while 76/115 (66%) were "hallucinated" not real papers and 6/115 (5%) shared the names of real papers but had not real citations. Conclusion: While ChatGPT produced tailored answers on the efficacy of the queried medications, the results were unreliable and inaccurate because of the overwhelming volume of "hallucinated" articles it generated and cited.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.