ArticleJournal of medical Internet research2020
Evaluating Smart Assistant Responses for Accuracy and Misinformation Regarding Human Papillomavirus Vaccination: Content Analysis Study.
Article in Journal of medical Internet research, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed, 1 synthesis or guideline pooled it, 47 citations in OpenAlex.
- A Systematic Review on Healthcare Artificial Intelligent Conversational Agents for Chronic Conditions.Sensors (Basel, Switzerland) · 2022Pooled it
- Trial
- The Answer Bot Effect (ABE): A powerful new form of influence made possible by intelligent personal assistants and search engines.PloS one · 2022Trial
- Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review.Journal of the American Medical Informatics Association : JAMIA · 2024Article
- Voice Assistants' Responses to Questions About the COVID-19 Vaccine: National Cross-sectional Study.JMIR formative research · 2023Article
- Social Media Perceptions and Internet Verification Skills Associated With Human Papillomavirus Vaccine Decision-Making Among Parents of Children and Adolescents: Cross-sectional Survey.JMIR pediatrics and parenting · 2022Article
- Evaluating Voice Assistants' Responses to COVID-19 Vaccination in Portuguese: Quality Assessment.JMIR human factors · 2022Article
- Infodemiological Examination of Personal and Commercial Tweets About Cannabidiol: Term and Sentiment Analysis.Journal of medical Internet research · 2021Article
- Factors Influencing Willingness to Share Health Misinformation Videos on the Internet: Web-Based Survey.Journal of medical Internet research · 2021Article
- Mitigating Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: Exploratory Mixed Methods Experiment.Journal of medical Internet research · 2021Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors at 1 institution in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundAlmost half (46%) of Americans have used a smart assistant of some kind (eg, Apple Siri), and 25% have used a stand-alone smart assistant (eg, Amazon Echo). This positions smart assistants as potentially useful modalities for retrieving health-related information; however, the accuracy of smart assistant responses lacks rigorous evaluation.
objectiveThis study aimed to evaluate the levels of accuracy, misinformation, and sentiment in smart assistant responses to human papillomavirus (HPV) vaccination-related questions.
methodsWe systematically examined responses to questions about the HPV vaccine from the following four most popular smart assistants: Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. One team member posed 10 questions to each smart assistant and recorded all queries and responses. Two raters independently coded all responses (κ=0.85). We then assessed differences among the smart assistants in terms of response accuracy, presence of misinformation, and sentiment regarding the HPV vaccine.
resultsA total of 103 responses were obtained from the 10 questions posed across the smart assistants. Google Assistant data were excluded owing to nonresponse. Over half (n=63, 61%) of the responses of the remaining three smart assistants were accurate. We found statistically significant differences across the smart assistants (N=103, χ
conclusionsSmart assistants appear to be average-quality sources for HPV vaccination information, with Alexa responding most reliably. Cortana returned the largest proportion of inaccurate responses, the most misinformation, and the greatest proportion of results with negative sentiments. More collaboration between technology companies and public health entities is necessary to improve the retrieval of accurate health information via smart assistants.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.