ArticlePloS one2016
p-Curve and p-Hacking in Observational Research.
Article in PloS one, 2016. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 44 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
44 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Publication and related biases in health services research: a systematic review of empirical evidence.BMC medical research methodology · 2020Pooled it
- Pooled it
- Psychometric evaluation of the Questionnaire of Life Satisfaction (FLZBMC psychology · 2026Article
- Benzbromarone as a Novel Candidate for Preventing Alzheimer's Disease: Evidence From Real-World Data Screening and in Vitro Validation.Clinical and translational science · 2026Article
- Causal Inference Tools for Pharmacovigilance: Using Causal Graphs to Identify and Address Biases in Disproportionality Analysis.Drug safety · 2026Article
- Trends in the distribution of P values in epidemiology journals: a statistical, P-curve, and simulation study.American journal of epidemiology · 2025Article
- Societal restraint of behavior during the pre-vaccine pandemic saved working-age men but not women.Science advances · 2025Article
- Spurious precision in meta-analysis of observational research.Nature communications · 2025Article
- Niche Overlap in Forest Tree Species Precludes a Positive Diversity-Productivity Relationship.Plants (Basel, Switzerland) · 2025Article
- Sports Metaresearch: An Emerging Discipline of Sport Science and Medicine.Sports medicine (Auckland, N.Z.) · 2025Review
- Chrono-immunotherapy as a low-hanging fruit for cancer treatment? A call for pragmatic randomized clinical trials.Journal for immunotherapy of cancer · 2025Review
- Improving Replication in Endometrial Omics: Understanding the Influence of the Menstrual Cycle.International journal of molecular sciences · 2025Review
- Misstatements, misperceptions, and mistakes in controlling for covariates in observational research.eLife · 2024Review
- Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses.Research synthesis methods · 2024Article
- The inconsistency of p-curve: Testing its reliability using the power pose and HPA debates.PloS one · 2024Article
- Problems and alternatives of testing significance using null hypothesis andFood science and biotechnology · 2023Review
- The logical structure of experiments lays the foundation for a theory of reproducibility.Royal Society open science · 2023Article
- Big little lies: a compendium and simulation ofRoyal Society open science · 2023Article
- Article
- Article
Corrections and comments
- Commented on by
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
The p-curve, the distribution of statistically significant p-values of published studies, has been used to make inferences on the proportion of true effects and on the presence of p-hacking in the published literature. We analyze the p-curve for observational research in the presence of p-hacking. We show by means of simulations that even with minimal omitted-variable bias (e.g., unaccounted confounding) p-curves based on true effects and p-curves based on null-effects with p-hacking cannot be reliably distinguished. We also demonstrate this problem using as practical example the evaluation of the effect of malaria prevalence on economic growth between 1960 and 1996. These findings call recent studies into question that use the p-curve to infer that most published research findings are based on true effects in the medical literature and in a wide range of disciplines. p-values in observational research may need to be empirically calibrated to be interpretable with respect to the commonly used significance threshold of 0.05. Violations of randomization in experimental studies may also result in situations where the use of p-curves is similarly unreliable.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.