ArticleRoyal Society open science2023
The logical structure of experiments lays the foundation for a theory of reproducibility.
Article in Royal Society open science, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
10 citing papers in PubMed.
- Improving science will require more than just training.Clinical psychological science : a journal of the Association for Psychological Science · 2026Article
- Sometimes Less Is More: On Sample Size in Medical Research.Journal of evaluation in clinical practice · 2026Review
- How can we make sound replication decisions?Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Dialogues about the practice of science.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- A Systematic Review on the Evolution of Power Analysis Practices in Psychological Research.Psychologica Belgica · 2025Article
- Can a Good Theory Be Built Using Bad Ingredients?Computational brain & behavior · 2024Article
- Statistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate.Entropy (Basel, Switzerland) · 2024Article
- Better Accuracy for Better Science . . . Through Random Conclusions.Perspectives on psychological science : a journal of the Association for Psychological Science · 2024Article
- Research Into Evidence-Based Psychological Interventions Needs a Stronger Focus on Replicability.Clinical psychology in Europe · 2023Article
- Mixed News about the Bad News Game.Journal of cognition · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
Abstract
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current 'crisis' reflects inadequate attention to a theoretical understanding of results reproducibility.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.