ReviewJournal of general internal medicine2018
Comparing Amazon's Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature.
Review in Journal of general internal medicine, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 183 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
183 citing papers in PubMed.
- Effects of Patient Portal Message Framing on Treatment Preferences and Expectations for Degenerative Meniscus Tears: Randomized Exploratory Cross-Sectional Survey Study.JMIR formative research · 2026Trial
- Assessing perceived empathy based on genetic counselor gender using a randomized, hypothetical prenatal genetic counseling scenario design.Journal of genetic counseling · 2025Trial
- Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment.Journal of medical Internet research · 2024Trial
- Crowdsourced Feedback to Improve Resident Physician Error Disclosure Skills: A Randomized Clinical Trial.JAMA network open · 2024Trial
- Assessing Interventions on Crowdsourcing Platforms to Nudge Patients for Engagement Behaviors in Primary Care Settings: Randomized Controlled Trial.Journal of medical Internet research · 2023Trial
- Incorporating Social Networks and Event-Specific Information in a Personalized Feedback Intervention to Reduce Drinking Among Young Adults.Alcohol and alcoholism (Oxford, Oxfordshire) · 2022Trial
- A randomized controlled trial of a video intervention shows evidence of increasing COVID-19 vaccination intention.PloS one · 2022Trial
- Examining the role of COVID-19 testing availability on intention to isolate: A Randomized hypothetical scenario.PloS one · 2022Trial
- Framing Effects on Decision-Making for Diagnostic Genetic Testing: Results from a Randomized Trial.Genes · 2021Trial
- The Influence of Cost Information on Treatment Choice: A Mixed-Methods Study.The Journal of hand surgery · 2020Trial
- Effect of Intensivist Communication in a Simulated Setting on Interpretation of Prognosis Among Family Members of Patients at High Risk of Intensive Care Unit Admission: A Randomized Trial.JAMA network open · 2020Trial
- HPV vaccination intention among unvaccinated international and domestic college students in the U.S.: A cross-sectional survey.Human vaccines & immunotherapeutics · 2026Article
- Exploring the Links between Individual Types of Adverse Childhood Experiences and Self-Reported Psychiatric and Neurocognitive Conditions.Journal of child & adolescent trauma · 2026Article
- Mental health service experiences regarding sexual orientation: Relations with demographics, depression, and suicidality among queer adults.Psychological services · 2026Article
- Examining the Decision to Take a Dementia Risk Test: Considerations for Privacy in Screening At-Home.Dementia and geriatric cognitive disorders · 2026Article
- Reply: Patient Preferences for Utilization of Telemedicine in Aesthetic Surgery Consultations.Aesthetic plastic surgery · 2026Article
- Deciphering the Data: Health Numeracy and Its Impact on Decision-Making in Breast Augmentation.Aesthetic plastic surgery · 2026Article
- Perceived Utility of Cognitive Behavioral Therapy in People With Bowel Disorders of Gut-Brain Interaction.Neurogastroenterology and motility · 2026Article
- Examining the Contrast Avoidance Model to Understand the Association between Generalized Anxiety Disorder and Sleep Difficulties.The Psychiatric quarterly · 2026Article
- Trajectories of spiritual well-being across the first year of the COVID-19 pandemic.The American journal of orthopsychiatry · 2026Article
123 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe goal of this article is to conduct an assessment of the peer-reviewed primary literature with study objectives to analyze Amazon.com 's Mechanical Turk (MTurk) as a research tool in a health services research and medical context.
methodsSearches of Google Scholar and PubMed databases were conducted in February 2017. We screened article titles and abstracts to identify relevant articles that compare data from MTurk samples in a health and medical context to another sample, expert opinion, or other gold standard. Full-text manuscript reviews were conducted for the 35 articles that met the study criteria.
resultsThe vast majority of the studies supported the use of MTurk for a variety of academic purposes. DISCUSSION: The literature overwhelmingly concludes that MTurk is an efficient, reliable, cost-effective tool for generating sample responses that are largely comparable to those collected via more conventional means. Caveats include survey responses may not be generalizable to the US population.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.