ArticlePhilosophical transactions. Series A, Mathematical, physical, and engineering sciences2018
Improving reproducibility by using high-throughput observational studies with empirical calibration.
Article in Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 56 papers, 2 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
56 citing papers in PubMed, 2 syntheses or guidelines pooled it.
- Yogurt, cultured fermented milk, and health: a systematic review.Nutrition reviews · 2021Pooled it
- Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis.Lancet (London, England) · 2019Pooled it
- From Product-Quality Complaints to Patient-Safety Triage: A Structured Narrative Review and Proposed PRCS-TRACE Framework.Healthcare (Basel, Switzerland) · 2026Review
- Retinal Artery Occlusion and Incident Dementia in the All of Us Research Program: Detection Bias Calibration with Prespecified Negative-Control Outcomes.Journal of clinical medicine research · 2026Article
- Comparative Cardiovascular Effectiveness of Glucagon-Like Peptide 1 Receptor Agonists and Sodium-Glucose Cotransporter 2 Inhibitors in Diabetes Mellitus.Journal of the American College of Cardiology · 2026Article
- Trust in Observational Research.Journal of the American College of Cardiology · 2026Article
- Optimizing cohort criteria for multi-country analysis of women experiencing menopause in administrative databases.BMC medical research methodology · 2026Article
- Semaglutide and diabetic retinopathy: an OHDSI network study.BMJ open diabetes research & care · 2025 · on this mapObservational
- Assessing Covariate Balance With Small Sample Sizes.Statistics in medicine · 2025Article
- Assessing Covariate Balance with Small Sample Sizes.medRxiv : the preprint server for health sciences · 2025Article
- Predicting ICU Admission Risk in Children with Respiratory Syncytial Virus.Infectious diseases and therapy · 2025Article
- Objective study validity diagnostics: a framework requiring pre-specified, empirical verification to increase trust in the reliability of real-world evidence.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- The challenges of replication: A worked example of methods reproducibility using electronic health record data.PloS one · 2025Article
- Glucagon-Like Peptide 1 Receptor Agonists and Chronic Lower Respiratory Disease Among Type 2 Diabetes Patients: Replication and Reliability Assessment Across a Research Network.Pharmacoepidemiology and drug safety · 2025Article
- Bayesian Posterior Interval Calibration to Improve the Interpretability of Observational Studies.Statistical analysis and data mining · 2024Article
- Targeted learning with an undersmoothed LASSO propensity score model for large-scale covariate adjustment in health-care database studies.American journal of epidemiology · 2024Article
- Comparative Effectiveness of Second-Line Antihyperglycemic Agents for Cardiovascular Outcomes: A Multinational, Federated Analysis of LEGEND-T2DM.Journal of the American College of Cardiology · 2024Article
- The State of Use and Utility of Negative Controls in Pharmacoepidemiologic Studies.American journal of epidemiology · 2024Review
- Shrinkage with shrunken shoulders: Gibbs sampling shrinkage model posteriors with guaranteed convergence rates.Bayesian analysis · 2023Article
- A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases.NPJ digital medicine · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors.
Funding
Abstract
Concerns over reproducibility in science extend to research using existing healthcare data; many observational studies investigating the same topic produce conflicting results, even when using the same data. To address this problem, we propose a paradigm shift. The current paradigm centres on generating one estimate at a time using a unique study design with unknown reliability and publishing (or not) one estimate at a time. The new paradigm advocates for high-throughput observational studies using consistent and standardized methods, allowing evaluation, calibration and unbiased dissemination to generate a more reliable and complete evidence base. We demonstrate this new paradigm by comparing all depression treatments for a set of outcomes, producing 17 718 hazard ratios, each using methodology on par with current best practice. We furthermore include control hypotheses to evaluate and calibrate our evidence generation process. Results show good transitivity and consistency between databases, and agree with four out of the five findings from clinical trials. The distribution of effect size estimates reported in the literature reveals an absence of small or null effects, with a sharp cut-off at
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.