ArticleBMJ health & care informatics2019
Evaluating the quality of voice assistants' responses to consumer health questions about vaccines: an exploratory comparison of Alexa, Google Assistant and Siri.
Article in BMJ health & care informatics, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 25 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
25 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Conversational AI and Vaccine Communication: Systematic Review of the Evidence.Journal of medical Internet research · 2023Pooled it
- Assessment of vaccine information accuracy across large language models.Frontiers in public health · 2026Article
- Exploring the Utility of Digital Voice Assistants for Primary Care Patients, Including Those With Physical and Visual Disabilities: Cross-Sectional Study.JMIR mHealth and uHealth · 2025Article
- Virtual Assistants' Response to Queries About Nicotine Replacement Therapy: A Mixed-Method Analysis.Evaluation & the health professions · 2025Article
- Are Virtual Assistants Trustworthy for Medicare Information: An Examination of Accuracy and Reliability.The Gerontologist · 2024Article
- The accuracy of artificial intelligence-based virtual assistants in responding to routinely asked questions about orthodontics.The Angle orthodontist · 2023Article
- Feasibility and Acceptability of Chatbots for Nutrition and Physical Activity Health Promotion Among Adolescents: Systematic Scoping Review With Adolescent Consultation.JMIR human factors · 2023Article
- Voice Assistants' Responses to Questions About the COVID-19 Vaccine: National Cross-sectional Study.JMIR formative research · 2023Article
- Voice Assistants as Consultants for Male Patients with Sexual Dysfunction: A Reliable Option?International journal of environmental research and public health · 2023Article
- Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study.Journal of medical Internet research · 2022Review
- Anthropomorphizing Technology: A Conceptual Review of Anthropomorphism Research and How it Relates to Children's Engagements with Digital Voice Assistants.Integrative psychological & behavioral science · 2022Article
- Design and Formative Evaluation of a Virtual Voice-Based Coach for Problem-solving Treatment: Observational Study.JMIR formative research · 2022Article
- Evaluating Voice Assistants' Responses to COVID-19 Vaccination in Portuguese: Quality Assessment.JMIR human factors · 2022Article
- Ensuring Interrater Reliability When Evaluating Voice Assistants. Comment on "Evaluating Voice Assistants' Responses to COVID-19 Vaccination in Portuguese: Quality Assessment".JMIR human factors · 2022Article
- Reliability of Commercial Voice Assistants' Responses to Health-Related Questions in Noncommunicable Disease Management: Factorial Experiment Assessing Response Rate and Source of Information.Journal of medical Internet research · 2021Article
- Mitigating Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: Exploratory Mixed Methods Experiment.Journal of medical Internet research · 2021Article
- Rationale and Methods of Evaluation for ACHO, A New Virtual Assistant to Improve Therapeutic Adherence in Rural Elderly Populations: A User-Driven Living Lab.International journal of environmental research and public health · 2021Observational
- Artificial Intelligence Can Improve Patient Management at the Time of a Pandemic: The Role of Voice Technology.Journal of medical Internet research · 2021Article
- Delivering Perinatal Health Information via a Voice Interactive App (SMILE): Mixed Methods Feasibility Study.JMIR formative research · 2021Article
- Patterns and Influencing Factors of eHealth Tools Adoption Among Medicaid and Non-Medicaid Populations From the Health Information National Trends Survey (HINTS) 2017-2019: Questionnaire Study.Journal of medical Internet research · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
objectiveTo assess the quality and accuracy of the voice assistants (VAs), Amazon Alexa, Siri and Google Assistant, in answering consumer health questions about vaccine safety and use.
methodsResponses of each VA to 54 questions related to vaccination were scored using a rubric designed to assess the accuracy of each answer provided through audio output and the quality of the source supporting each answer.
resultsOut of a total of 6 possible points, Siri averaged 5.16 points, Google Assistant averaged 5.10 points and Alexa averaged 0.98 points. Google Assistant and Siri understood voice queries accurately and provided users with links to authoritative sources about vaccination. Alexa understood fewer voice queries and did not draw answers from the same sources that were used by Google Assistant and Siri.
conclusionsThose involved in patient education should be aware of the high variability of results between VAs. Developers and health technology experts should also push for greater usability and transparency about information partnerships as the health information delivery capabilities of these devices expand in the future.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.