ArticleJournal of personality and social psychology2011
Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011).
Article in Journal of personality and social psychology, 2011. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 260 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
260 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- A systematic review and Bayesian meta-analysis of 30 years of stress generation research: Clinical, psychological, and sociodemographic risk and protective factors for prospective negative life events.Psychological bulletin · 2024Pooled it
- Characterizing Interoceptive Differences in Autism: A Systematic Review and Meta-analysis of Case-control Studies.Journal of autism and developmental disorders · 2023Pooled it
- Robust Bayesian meta-analysis: Model-averaging across complementary publication bias adjustment methods.Research synthesis methods · 2023Pooled it
- Causal effects of social media use on self-esteem, mindfulness, sleep and emotional well-being: a social media restriction study.Frontiers in public health · 2025Trial
- The relationship between frontal alpha asymmetry and behavioral and brain activity indices of reactive inhibitory control.Journal of neurophysiology · 2024Trial
- Auricular Transcutaneous Vagus Nerve Stimulation Specifically Enhances Working Memory Gate Closing Mechanism: A System Neurophysiological Study.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2023Trial
- Video meeting signals: Experimental evidence for a technique to improve the experience of video conferencing.PloS one · 2022Trial
- Empirical Investigation of the Verbal Cues Involved in Delivering Experiential Metaphors.International journal of environmental research and public health · 2021Trial
- Applying behavioural science to increase uptake of the NHS Health Check: a randomised controlled trial of gain- and loss-framed messaging in the national patient information leaflet.BMC public health · 2019Trial
- Exploring the association between whole blood Omega-3 Index, DHA, EPA, DHA, AA and n-6 DPA, and depression and self-esteem in adolescents of lower general secondary education.European journal of nutrition · 2019Trial
- Trial
- The Role of Patients' Treatment Expectations in Ventricular Assist Device Support: Results From a Convergent Mixed Methods Study.Artificial organs · 2026Article
- The benefit of simultaneous versus sequential presentation of memoranda in working memory: It depends on the stimuli.Memory & cognition · 2026Article
- How Uncertainty Affects Children's Exploration and Exploitation in Statistical Learning.Developmental science · 2026Article
- Social Network Types in Autistic Adults and Its Associations with Mastery, Quality of Life, and Autism Characteristics.Journal of autism and developmental disorders · 2026Article
- Language Access is a Catalyst for Early Numerical Abilities.Developmental science · 2026Article
- Bayes factor hypothesis testing in meta-analyses: Practical advantages and methodological considerations.Research synthesis methods · 2026Review
- Article
- Neurodiversity and mental health in adulthood: exploring the unique contributions of autism and ADHD to internalising problems.Scientific reports · 2026Article
- Assessing Questionable and Responsible Research Practices in Psychology Master's Theses.Behavioral sciences (Basel, Switzerland) · 2026Article
200 more citing papers are in PubMed but not listed here.
Corrections and comments
- Commented on by
- Commented on by
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Does psi exist? D. J. Bem (2011) conducted 9 studies with over 1,000 participants in an attempt to demonstrate that future events retroactively affect people's responses. Here we discuss several limitations of Bem's experiments on psi; in particular, we show that the data analysis was partly exploratory and that one-sided p values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem's data with a default Bayesian t test and show that the evidence for psi is weak to nonexistent. We argue that in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal. We conclude that Bem's p values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.
Indexed as
Identifiers
21280965What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.