ArticleRoyal Society open science2021
The case for formal methodology in scientific reform.
Article in Royal Society open science, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 27 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
27 citing papers in PubMed.
- Improving science will require more than just training.Clinical psychological science : a journal of the Association for Psychological Science · 2026Article
- Spatial Filters-In Search of the Virtual Electrode.The Journal of neuroscience : the official journal of the Society for Neuroscience · 2026Article
- Unlocking the benefits of transparent and reusable science for climate risk management.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- What is critical metascience and why is it important?Frontiers in psychology · 2026Review
- Dialogues about the practice of science.Proceedings of the National Academy of Sciences of the United States of America · 2025Article
- Article
- Novelty, Consistency, Transparency: The Trilemma of Psychological Sciences and its Consequences on Open Science Practices.International review of social psychology · 2025Article
- Novel Uncertainty Quantification Through Perturbation-Assisted Sample Synthesis.IEEE transactions on pattern analysis and machine intelligence · 2024Article
- The Problem-Ladenness of Theory.Computational brain & behavior · 2024Article
- What Makes a Good Theory? Interdisciplinary Perspectives.Computational brain & behavior · 2024Article
- Bridging theory and data: A computational workflow for cultural evolution.Proceedings of the National Academy of Sciences of the United States of America · 2024Article
- Statistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate.Entropy (Basel, Switzerland) · 2024Article
- A tutorial on fitting joint models of M/EEG and behavior to understand cognition.Behavior research methods · 2024Article
- The replication crisis has led to positive structural, procedural, and community changes.Communications psychology · 2023Review
- Perspectives on scientific error.Royal Society open science · 2023Review
- Open Science Practices in Communication Sciences and Disorders: A Survey.Journal of speech, language, and hearing research : JSLHR · 2023Article
- Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes.Royal Society open science · 2023Review
- Using preregistration as a tool for transparent fNIRS study design.Neurophotonics · 2023Article
- The logical structure of experiments lays the foundation for a theory of reproducibility.Royal Society open science · 2023Article
- Evaluating causal psychological models: A study of language theories of autism using a large sample.Frontiers in psychology · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
4 authors.
Funding
Abstract
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.