SynthesisInfectious diseases of poverty2020
Crowdsourcing in health and medical research: a systematic review.
Synthesis in Infectious diseases of poverty, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 64 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
64 citing papers in PubMed.
- A digital citizen science intervention to reduce HIV stigma and promote HIV testing: a randomized clinical trial among adolescents and young adults in Kazakhstan.Sexual health · 2025Trial
- Dissemination Strategies From an HIV Self-Testing Intervention Among Adolescents and Young Adults in Nigeria: Crowdsourcing Open Call Study.JMIR formative research · 2026Article
- Article
- Application of Innovation Tournament methodology to inform de-implementation strategies: lessons learned in addressing mammography overscreening in older women.Implementation science communications · 2026Article
- Evaluating Crowdsourced Data Collection for Carceral Death Surveillance: Pilot Study Using Amazon Mechanical Turk.JMIR formative research · 2026Article
- Using a crowdsourcing open call to generate hepatitis B and C stigma-reduction materials with medical students in China.BMC public health · 2026Article
- Nontraditional Data in Pandemic Preparedness and Response: Identifying and Addressing First- and Last-Mile Challenges.Journal of medical Internet research · 2026Article
- Incidence, mortality, and cumulative risk of cancer in adolescents and young adults in Switzerland.European journal of epidemiology · 2026Article
- Community engagement and implementation science: a hermeneutic review for implementation scientists.Implementation science communications · 2026Review
- Benchmarking large language models for predictive modeling in biomedical research with a focus on reproductive health.Cell reports. Medicine · 2026Article
- Testing the effects of segmented crowdsource-selected messages to improve intentions to follow colorectal cancer screening recommendations: study protocol for a randomized controlled trial.BMC public health · 2026Article
- Multimodal LLM vs. Human-Measured Features for AI Predictions of Autism in Home Videos.Algorithms · 2025Article
- The Validation of the Nomophobia Questionnaire Using a Modern Psychometric Approach: An Item Response Theory Analysis of 5087 Participants.Brain and behavior · 2025Article
- Integrating Digital Health Innovations to Achieve Universal Health Coverage: Promoting Health Outcomes and Quality Through Global Public Health Equity.Healthcare (Basel, Switzerland) · 2025Review
- The sustainability of health interventions implemented in Africa: an updated systematic review on evidence and future research perspectives.Implementation science communications · 2025Review
- Article
- How to (or how not to) implement crowdsourcing for the development of health interventions: lessons learned from four African countries.Health policy and planning · 2024Article
- Usability, Engagement, and Report Usefulness of Chatbot-Based Family Health History Data Collection: Mixed Methods Analysis.Journal of medical Internet research · 2024Article
- Understanding the Appropriate and Beneficial Use of Before and After Photos in Breast Surgery: A North American Survey.Plastic surgery (Oakville, Ont.) · 2024Article
- Fostering citizen-engaged HIV implementation science.Journal of the International AIDS Society · 2024Article
4 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
15 authors.
Funding
Abstract
backgroundCrowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health.
methodsWe followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach.
resultsWe screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled.
conclusionsCrowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine.
trial registrationPROSPERO: CRD42017052835. December 27, 2016.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.