ArticleDigital health
Design and multidimensional expert evaluation of a national health information portal-integrated generative AI chatbot for health literacy and awareness.
Article in Digital health. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
0 citing papers in PubMed.
No citing paper in PubMed yet.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Background: Generative AI (GenAI) shows promise for health literacy and awareness, with large language models (LLMs) increasingly used for health information-seeking despite concerns about misinformation. National health information portals provide authoritative, evidence-based content but lack conversational interfaces. Retrieval-augmented generation (RAG) offers a potential solution by grounding responses in verified health information portal content, yet evidence on such integration remains limited. Objective: This study aimed to design a national health information portal-integrated generative AI chatbot to support access to trustworthy health information and to evaluate its response characteristics relative to general-purpose LLMs through a multidimensional expert assessment. Methods: A RAG-enhanced chatbot (NZIP AI assistant) was designed for the Czech Republic's National Health Information Portal and evaluated by eleven experts who compared it with ChatGPT, Microsoft Copilot, and Google Gemini across 67 health-related questions. Responses were rated on trustworthiness, clarity, user-friendliness, and relevance using a 5-point Likert scale (1 = best, 5 = worst). Content analysis examined response characteristics. Results: In total, 267 assessments were collected. NZIP AI assistant achieved the highest trustworthiness score (mean 1.21) but ranked lowest on clarity (1.53), user-friendliness (1.46), and relevance (1.65). General-purpose LLMs demonstrated the inverse pattern. Content analysis revealed that the NZIP AI assistant provided source citations in 100% of responses, compared with 0-3% for general-purpose LLMs; included explicit safety disclaimers in 85% of responses, compared with 5-18%; and declined 10.4% of queries outside its knowledge base. Conclusions: National health information portals can integrate generative AI with RAG architecture to deliver verified, source-grounded health information, though trade-offs in user-oriented dimensions warrant consideration in future implementations. Still, this expert evaluation assessed response characteristics rather than direct effects on health literacy, awareness, or health outcomes.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.