Evidence map›Paper›PMID 33404508›Full record

ArticleJournal of medical Internet research2021

Utilization of Self-Diagnosis Health Chatbots in Real-World Settings: Case Study.

Xiangmin Fan, Daren Chao, Zhan Zhang, Dakuo Wang, Xiaohua Li, Feng Tian

Open access · goldAbstract readCase Reports
In one paragraph

Article in Journal of medical Internet research, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 53 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
53citing papers in PubMed, 2 pooled it
9.0field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

53 citing papers in PubMed, 2 syntheses or guidelines pooled it, 143 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Trial
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Review
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. AI education for clinicians.EClinicalMedicine · 2025
    Review
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors at 4 institutions in 3 countries.

Xiangmin Fan *The Institute of Software, Chinese Academy of Sciences, Beijing, China.ORCID 0000-0003-4223-2320
Daren Chao *Department of Computer Science, University of Toronto, Toronto, ON, Canada.ORCID 0000-0003-1813-419X
Zhan Zhang *School of Computer Science and Information Systems, Pace University, New York, NY, United States.ORCID 0000-0001-6973-6903
Dakuo Wang *IBM Research, Cambridge, MA, United States.ORCID 0000-0001-9371-9441
Xiaohua LiZuoshouyisheng Inc, Beijing, China.ORCID 0000-0001-6641-0457
Feng Tian *The Institute of Software, Chinese Academy of Sciences, Beijing, China.ORCID 0000-0002-6598-419X
Chinese Academy of Sciences · CNCambridge Scientific (United States) · USPace University · USUniversity of Toronto · CA

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundArtificial intelligence (AI)-driven chatbots are increasingly being used in health care, but most chatbots are designed for a specific population and evaluated in controlled settings. There is little research documenting how health consumers (eg, patients and caregivers) use chatbots for self-diagnosis purposes in real-world scenarios.

objectiveThe aim of this research was to understand how health chatbots are used in a real-world context, what issues and barriers exist in their usage, and how the user experience of this novel technology can be improved.

methodsWe employed a data-driven approach to analyze the system log of a widely deployed self-diagnosis chatbot in China. Our data set consisted of 47,684 consultation sessions initiated by 16,519 users over 6 months. The log data included a variety of information, including users' nonidentifiable demographic information, consultation details, diagnostic reports, and user feedback. We conducted both statistical analysis and content analysis on this heterogeneous data set.

resultsThe chatbot users spanned all age groups, including middle-aged and older adults. Users consulted the chatbot on a wide range of medical conditions, including those that often entail considerable privacy and social stigma issues. Furthermore, we distilled 2 prominent issues in the use of the chatbot: (1) a considerable number of users dropped out in the middle of their consultation sessions, and (2) some users pretended to have health concerns and used the chatbot for nontherapeutic purposes. Finally, we identified a set of user concerns regarding the use of the chatbot, including insufficient actionable information and perceived inaccurate diagnostic suggestions.

conclusionsAlthough health chatbots are considered to be convenient tools for enhancing patient-centered care, there are issues and barriers impeding the optimal use of this novel technology. Designers and developers should employ user-centered approaches to address the issues and user concerns to achieve the best uptake and utilization. We conclude the paper by discussing several design implications, including making the chatbots more informative, easy-to-use, and trustworthy, as well as improving the onboarding experience to enhance user engagement.

Indexed as

AdultArtificial IntelligenceFemaleHumansMaleResearch DesignSocial MediaTelemedicineartificial intelligencecase studychatbotconversational agentdiagnosiseHealthhuman–artificial intelligence interactionreal worldself-diagnosisuser experience

Identifiers

PMID33404508
PMCPMC7817366
OpenAlexW3090265966

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.