ArticleeLife2021
Consensus-based guidance for conducting and reporting multi-analyst studies.
Article in eLife, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 25 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
25 citing papers in PubMed.
- TOP 2025: An update to the Transparency and Openness Promotion Guidelines.Research integrity and peer review · 2026Article
- Many AI analysts, one dataset: Navigating the agentic data science multiverse.Proceedings of the National Academy of Sciences of the United States of America · 2026Article
- Article
- Multiverse simulation to explore the impact of analytical choices on type I and type II errors in a reaction time study.Behavior research methods · 2025Article
- Emergency Department Vital Sign Variability Is Associated with Hematoma Progression in Spontaneous Intracerebral Hemorrhage.Journal of clinical medicine · 2025Article
- Big team science reveals promises and limitations of machine learning efforts to model physiological markers of affective experience.Royal Society open science · 2025Article
- Marital status and risk of cardiovascular disease - a multi-analyst study in epidemiology.European journal of epidemiology · 2025Article
- Introducing synchronous robustness reports.Nature human behaviour · 2025Article
- Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology.BMC biology · 2025Article
- What is the vibration of effects?BMJ evidence-based medicine · 2025Article
- Article
- Toward a more credible assessment of the credibility of science by many-analyst studies.Proceedings of the National Academy of Sciences of the United States of America · 2024Article
- Prevalence, patterns and predictors of paranormal beliefs in The Netherlands: a several-analysts approach.Royal Society open science · 2024Article
- Heterogeneity in effect size estimates.Proceedings of the National Academy of Sciences of the United States of America · 2024Article
- The potential emerging role of piRNA/PIWI complex in virus infection.Virus genes · 2024Review
- Subjective evidence evaluation survey for many-analysts studies.Royal Society open science · 2024Article
- Participatory action research to develop and implement multicomponent, multilevel strategies for implementing colorectal cancer screening interventions in American Indian communities in New Mexico.Implementation science communications · 2024Article
- Raising awareness of uncertain choices in empirical data analysis: A teaching concept toward replicable research practices.PLoS computational biology · 2024Article
- Excess death estimates from multiverse analysis in 2009-2021.European journal of epidemiology · 2023Article
- How to build up big team science: a practical guide for large-scale collaborations.Royal Society open science · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
54 authors.
Funding
Abstract
Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.