ArticlePsychonomic bulletin & review2012
Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments.
Article in Psychonomic bulletin & review, 2012. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 233 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
233 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- How do voice acoustics affect the perceived trustworthiness of a speaker? A systematic review.Frontiers in psychology · 2025Pooled it
- The effect of working memory maintenance on long-term memory.Memory & cognition · 2019Pooled it
- The effects of acute physical and cognitive exercises on sequential motor skill learning: An exploratory study.PloS one · 2025Trial
- A digital intervention for cognitive deficits following COVID-19: a randomized clinical trial.Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology · 2024Trial
- Large and small financial incentives may motivate COVID-19 vaccination: A randomized, controlled survey experiment.PloS one · 2023Trial
- Trial
- The Daily Activity Study of Health (DASH): A pilot randomized controlled trial to enhance physical activity in sedentary older adults.Contemporary clinical trials · 2021Trial
- Enhanced switching and familial susceptibility for psychosis.Brain and behavior · 2018Trial
- When "Bouba" equals "Kiki": Cultural commonalities and cultural differences in sound-shape correspondences.Scientific reports · 2016Trial
- Personality trait and state associations with cognitive functioning and variability in middle-aged and older adults.The journals of gerontology. Series B, Psychological sciences and social sciences · 2026Article
- Intact Visual Correlation Detection in Autistic Adults.Autism research : official journal of the International Society for Autism Research · 2026Article
- Judgment of Learning Reactivity for Emotional Faces.Behavioral sciences (Basel, Switzerland) · 2026Article
- Daily Negative Affect and Reaction Time Inconsistency in Emerging Adults: Ecological Momentary Assessment Study.JMIR mHealth and uHealth · 2026Article
- Remote Sensory-Cognitive Assessment in Children with Autism: Evaluating Feasibility and Performance Outcomes.Behavioral sciences (Basel, Switzerland) · 2026Article
- Introducing the SAGE study: a multimodal protocol for testing a GABAergic mechanism of age-related episodic memory impairment across the sexes.BMJ neurology open · 2026Article
- Conditioned generalization of social anxiety effects based on positive and negative social evaluations.Scientific reports · 2025Article
- Distractor avoidance and early quitting in visual search.Attention, perception & psychophysics · 2025Article
- Spatial response-code association for loudness but not brightness.Perception · 2025Article
- Temporal Relationships Between Depression, Self-Efficacy, and Physical Activity in Individuals With Stroke.Archives of rehabilitation research and clinical translation · 2025Article
- Understanding Navon: A detailed structural and conceptual analysis of a basic local-global task.Psychonomic bulletin & review · 2025Review
173 more citing papers are in PubMed but not listed here.
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
Abstract
With the increasing sophistication and ubiquity of the Internet, behavioral research is on the cusp of a revolution that will do for population sampling what the computer did for stimulus control and measurement. It remains a common assumption, however, that data from self-selected Web samples must involve a trade-off between participant numbers and data quality. Concerns about data quality are heightened for performance-based cognitive and perceptual measures, particularly those that are timed or that involve complex stimuli. In experiments run with uncompensated, anonymous participants whose motivation for participation is unknown, reduced conscientiousness or lack of focus could produce results that would be difficult to interpret due to decreased overall performance, increased variability of performance, or increased measurement noise. Here, we addressed the question of data quality across a range of cognitive and perceptual tests. For three key performance metrics-mean performance, performance variance, and internal reliability-the results from self-selected Web samples did not differ systematically from those obtained from traditionally recruited and/or lab-tested samples. These findings demonstrate that collecting data from uncompensated, anonymous, unsupervised, self-selected participants need not reduce data quality, even for demanding cognitive and perceptual experiments.
Indexed as
Identifiers
22829343What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.