ArticleThe Lancet regional health. Europe2026
Real-world use and evaluation of a generative AI chatbot for Parkinson's disease information: a prospective observational study.
Article in The Lancet regional health. Europe, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: Patient-facing medical AI chatbots are entering clinical use faster than evidence can characterise their real-world safety. We evaluated one deployed system and introduce CARE-LLM, a proposed framework for conversation-level post-market surveillance. Methods: We conducted a prospective, observational, conversation-level evaluation of jAImes, a retrieval-augmented AI information system for Parkinson's disease deployed publicly in Germany, across its first 129 days of operation (Nov 11, 2025-Mar 20, 2026), analysing all 2035 conversations (6146 messages). CARE-LLM (Conversation-level AI Real-world Evaluation) combines automated triage of every conversation, structured expert review of flagged cases, sampling-based sensitivity validation against the adequate class, and a failure-class feedback loop. Findings: AI-assisted triage classified 1803/2035 conversations (88·6%) as good, 224/2035 (11·0%) as partially adequate, and 8/2035 (0·4%) as inadequate. Expert review of 45 flagged conversations confirmed five critical events. Independent re-review of 100 randomly sampled 'good'-rated conversations identified four (4/100, 4%; 95% CI 1·6-9·8) with clinically critical errors missed by the triage-a conditional false-negative rate within the 'good' stratum, not an overall critical-event rate. Confirmed critical events-five flagged, four sampled-spanned three failure classes (knowledge boundary, robustness, and escalation failures) and included an inappropriate memantine recommendation in Parkinson's disease dementia and explicit suicidal ideation that did not trigger the intended emergency response. Interpretation: Tightly scoped, retrieval-augmented, patient-facing AI achieved favourable automated triage classifications while still harbouring clinically critical failures invisible to automated quality metrics alone; detecting them required conversation-level evaluation with clinical expert adjudication. Funding: Deutsche Forschungsgemeinschaft, Interdisciplinary Center for Clinical Research Würzburg, and German Ministry of Education and Research (BMBF); technical development of jAImes was commissioned and financed by Parkinson Stiftung Deutschland.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.