ReviewNature neuroscience2009
Circular analysis in systems neuroscience: the dangers of double dipping.
Review in Nature neuroscience, 2009. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to 2 registered trials, which are not on this map. Cited by 1,428 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neurocognitive and fMRI Activation Changes Observed at Baseline and After Whole Brain Radiation vs. Radiosurgery for Patients With Cerebral Metastases: a Prospective Case-control Analysis.
The Relation of Genetic Factors, Food Cues, and Self-Regulation With Excess Consumption and Adiposity in Children
Who cites it
1,428 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- Heartbeat-Evoked Responses in M/EEG: A Systematic Review of Methods With Suggestions for Analysis and Reporting.Psychophysiology · 2026Pooled it
- Automated Analysis of Pelvic Radiographs for Hip Dysplasia Screening Using Artificial Intelligence in Children with Cerebral Palsy: A Systematic Review.Medicina (Kaunas, Lithuania) · 2026Pooled it
- Gradient Organization of Space, Time, and Numbers in the Brain: A Meta-analysis of Neuroimaging Studies.Neuropsychology review · 2024Pooled it
- Gaboxadol increases resting theta and alpha power without affecting evoked responses in fragile X syndrome in a home-based setting.Journal of neurodevelopmental disorders · 2026Trial
- Effects of the KCNQ (Kv7) Channel Opener Ezogabine on Resting-State Functional Connectivity of Striatal Brain Reward Regions, Depression, and Anhedonia in Major Depressive Disorder: Results From a Randomized Controlled Trial.Biological psychiatry · 2025Trial
- Brain connectomics markers for response prediction to transcranial magnetic stimulation in cocaine use disorder.Scientific reports · 2025Trial
- High-order brain interactions in ketamine during rest and task: a double-blinded cross-over design using portable EEG on male participants.Translational psychiatry · 2024Trial
- Preliminary evidence that ketamine alters anterior cingulate resting-state functional connectivity in depressed individuals.Translational psychiatry · 2023Trial
- Neurocognitive Mechanisms of Preventive Cognitive Therapy as Relapse Prevention in Remitted Major Depressive Disorder: A Randomized Controlled Trial.Biological psychiatry global open science · 2026Article
- Childhood maltreatment impacts the activity of the neural empathy but not theory of mind network: findings from a transdiagnostic sample.European archives of psychiatry and clinical neuroscience · 2026Article
- Decoding object categories from electroencephalography during free viewing reveals early information evolution compared to passive viewing.Journal of vision · 2026Article
- Big data research: the importance of the data-generating process and domain expertise.World psychiatry : official journal of the World Psychiatric Association (WPA) · 2026Article
- Moral and perceptual conflict processing are implemented by distinct brain activation and functional connectivity patterns.PLoS biology · 2026Article
- The mediating effect of neuroticism on the association between brain signal variability in the prefrontal cortex and depression among healthy individuals.Brain imaging and behavior · 2026Article
- Leveraging machine learning to personalize depression treatment: A pre-registered study of 828 adults randomized to a digital single-session intervention or waitlist.Clinical psychological science : a journal of the Association for Psychological Science · 2026Article
- Sex-specific biological aging clocks across organs and omics.Nature medicine · 2026Article
- Cortical Network Reorganization During Balance Control in Chronic Stroke: Premotor-Parietal Behavioral Uncoupling and Directional Prefrontal Compensation.medRxiv : the preprint server for health sciences · 2026Article
- Autistic children show reduced activity in left superior temporal cortex in response to social-emotional meaning in language.NeuroImage. Clinical · 2026Article
- Severe PTSD symptoms magnify episodic memory-encoding deficits and amygdala-ACC attenuation during unpredictable threat in male combat veterans.Molecular psychiatry · 2026Article
- Intact neurophysiological markers of death denial in long-term ayahuasca users.Psychopharmacology · 2026Article
1,368 more citing papers are in PubMed but not listed here.
Corrections and comments
- Commented on by
Authors and funding
4 authors.
Funding
Abstract
A neuroscientific experiment typically generates a large amount of data, of which only a small fraction is analyzed in detail and presented in a publication. However, selection among noisy measurements can render circular an otherwise appropriate analysis and invalidate results. Here we argue that systems neuroscience needs to adjust some widespread practices to avoid the circularity that can arise from selection. In particular, 'double dipping', the use of the same dataset for selection and selective analysis, will give distorted descriptive statistics and invalid statistical inference whenever the results statistics are not inherently independent of the selection criteria under the null hypothesis. To demonstrate the problem, we apply widely used analyses to noise data known to not contain the experimental effects in question. Spurious effects can appear in the context of both univariate activation analysis and multivariate pattern-information analysis. We suggest a policy for avoiding circularity.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.