ArticlePeerJ2016
Distributions of p-values smaller than .05 in psychology: what is going on?
Article in PeerJ, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis.PloS one · 2019Pooled it
- Coregistration of EEG and eye-tracking in infants and developing populations.Attention, perception & psychophysics · 2025Review
- How do psychology researchers interpret the results of multiple replication studies?Psychonomic bulletin & review · 2023Article
- Big little lies: a compendium and simulation ofRoyal Society open science · 2023Article
- Article
- Multimodality matters in numerical communication.Frontiers in psychology · 2023Review
- Article
- Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field.Royal Society open science · 2022Article
- Transparent, Open, and Reproducible Prevention Science.Prevention science : the official journal of the Society for Prevention Research · 2022Article
- P-hacking in clinical trials and how incentives shape the distribution of results across phases.Proceedings of the National Academy of Sciences of the United States of America · 2020Article
- Moderation of classic social psychological effects by demographics in the U.S. adult population: New opportunities for theoretical advancement.Journal of personality and social psychology · 2019Article
- Research practices and statistical reporting quality in 250 economic psychology master's theses: a meta-research investigation.Royal Society open science · 2019Article
- The Prevalence of Marginally Significant Results in Psychology Over Time.Psychological science · 2019Article
- The Weak Spots in Contemporary Science (and How to Fix Them).Animals : an open access journal from MDPI · 2017Review
- Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature.PLoS biology · 2017Article
- Article
- Article
- Conducting Meta-Analyses Based on p Values: Reservations and Recommendations for Applying p-Uniform and p-Curve.Perspectives on psychological science : a journal of the Association for Psychological Science · 2016Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psychological literature, and a potential increase thereof over time. We indeed found evidence for a bump just below .05 in the distribution of exactly reported p-values in the journals Developmental Psychology, Journal of Applied Psychology, and Journal of Personality and Social Psychology, but the bump did not increase over the years and disappeared when using recalculated p-values. We found clear and direct evidence for the QRP "incorrect rounding of p-value" (John, Loewenstein & Prelec, 2012) in all psychology journals. Finally, we also investigated monotonic excess of p-values, an effect of certain QRPs that has been neglected in previous research, and developed two measures to detect this by modeling the distributions of statistically significant p-values. Using simulations and applying the two measures to the retrieved test results, we argue that, although one of the measures suggests the use of QRPs in psychology, it is difficult to draw general conclusions concerning QRPs based on modeling of p-value distributions.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.