Trial reportJournal of medical Internet research2023
Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial.
Trial report in Journal of medical Internet research, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
26 citing papers in PubMed.
- Habit-formation strategies enhance long-term outcomes of a self-compassionate touch micropractice: a randomized controlled trial.Scientific reports · 2026Trial
- Digital Cognitive Behavioral Therapy Reduces Heavy Drinking Among a Non-Treatment Sample of People With Alcohol Use Disorder: A Feasibility Trial With CBT4CBT.Alcohol, clinical & experimental research · 2026Trial
- Trial
- A pilot randomised controlled trial of an online self-management programme for people with persistent pain post-knee replacement (Kneed).Clinical rehabilitation · 2025Trial
- Detecting and Preventing Fraudulent Participation in Qualitative Research: Content Analysis of Two Multisite Studies.Journal of medical Internet research · 2026Article
- Identifying and Analyzing Bot-Generated Responses in Online Health Care Surveys: Methodological Study.JMIR medical informatics · 2026Article
- A characterization of infants' and children's time-activity patterns for use in soil and dust exposure estimation.Journal of exposure science & environmental epidemiology · 2026Article
- Exploratory Study: The Impact of Online Coordinative Exercise in a Small Latinx Youth Sample.Pediatric reports · 2026Article
- Chronic Pelvic Pain Prevalence Increases Across Categories of Temporomandibular Disorder Severity in Women: A Large Cross-Sectional Study.Canadian journal of pain = Revue canadienne de la douleur · 2026Article
- A pilot feasibility and acceptability evaluation of Trans Care: A CBPR digital health intervention for transgender and nonbinary adults experiencing gender dysphoria.International journal of transgender health · 2026Article
- Protocol for the 'Beyond 50' prospective observational cohort study: investigating the impact of physical and psychosocial factors on healthy ageing.BMC public health · 2025Observational
- Lessons Learned Identifying and Controlling Fraudulent Participation in Online Randomized Trials.Journal of medical Internet research · 2025Article
- Challenges and Opportunities in Recruiting a Very Large Sample of Pregnant Individuals: Secondary Analysis of an Online Nationwide Randomized Controlled Trial.JMIR pediatrics and parenting · 2025Article
- Imposters, Bots, and Other Threats to Data Integrity in Online Research: Scoping Review of the Literature and Recommendations for Best Practices.Online journal of public health informatics · 2025Review
- Assessment of digital therapeutics in decentralized clinical trials: A scoping review.PLOS digital health · 2025Article
- Identifying and counteracting fraudulent responses in online recruitment for health research: a scoping review.BMJ evidence-based medicine · 2025Article
- Population Recruitment Strategies in the Age of Bots: Insights from the What Is on Your Plate Study.Current developments in nutrition · 2025Article
- Introducing Novel Methods to Identify Fraudulent Responses (Sampling With Sisyphus): Web-Based LGBTQ2S+ Mixed-Methods Study.Journal of medical Internet research · 2025Article
- Effective Recruitment or Bot Attack? The Challenge of Internet-Based Research Surveys and Recommendations to Reduce Risk and Improve Robustness.Interactive journal of medical research · 2025Article
- Evaluation of the Florida Newborn Screening Program Education Campaign.International journal of neonatal screening · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors.
Funding
Abstract
backgroundEvaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated "bots" and participant deception.
objectiveThis paper uses a case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues.
methodsWe recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of deception: "bots," that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information.
resultsOf the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification.
conclusionsData management protocols are necessary to detect automated bots and manual participant deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media.
trial registrationISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.