SynthesisJournal of clinical epidemiology2011
Optimism bias leads to inconclusive results-an empirical study.
Synthesis in Journal of clinical epidemiology, 2011. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 24 papers, 5 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
24 citing papers in PubMed, 5 syntheses or guidelines pooled it.
- Accuracy of Event Rate and Effect Size Estimation in Major Cardiovascular Trials: A Systematic Review.JAMA network open · 2024Pooled it
- Pooled it
- Using Bayesian statistics to estimate the likelihood a new trial will demonstrate the efficacy of a new treatment.BMC medical research methodology · 2017Pooled it
- Survival of patients with non-small cell lung cancer without treatment: a systematic review and meta-analysis.Systematic reviews · 2013Pooled it
- Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review.PloS one · 2013Pooled it
- Reporting practices and impact of withdrawal of life-sustaining treatment on outcomes in acute brain injury clinical trials: a literature review and simulation study.Critical care (London, England) · 2026Review
- The role of research ethics committees in addressing optimism in sample size calculations: a meta-epidemiological study.Research integrity and peer review · 2025Article
- Effect size estimation in cooperative group- and industry-sponsored phase 3 oncology trials.Journal of the National Cancer Institute · 2025Article
- Article
- Increasing Power in Phase III Oncology Trials With Multivariable Regression: An Empirical Assessment of 535 Primary End Point Analyses.JCO clinical cancer informatics · 2024Article
- Back to the basics: Guidance for designing good literature searches.Research in social & administrative pharmacy : RSAP · 2024Article
- Optimism Bias in the Design of Phase III Randomized Control Trials Evaluating PD-1/PD-L1 Targeting Monoclonal Antibodies.The oncologist · 2022Review
- Principal investigators over-optimistically forecast scientific and operational outcomes for clinical trials.PloS one · 2022Article
- Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?The oncologist · 2021Article
- Design analysis indicates Potential overestimation of treatment effects in randomized controlled trials supporting Food and Drug Administration cancer drug approvals.Journal of clinical epidemiology · 2018Article
- 'Optimism bias' in contemporary national clinical trial network phase III trials: are we improving?Annals of oncology : official journal of the European Society for Medical Oncology · 2018Article
- Article
- Do we need randomized clinical trials in extracorporeal respiratory support? We are not sure.Intensive care medicine · 2017Article
- Setting the research agenda for improving health care in musculoskeletal disorders.Nature reviews. Rheumatology · 2015Review
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
9 authors.
Funding
Abstract
objectiveOptimism bias refers to unwarranted belief in the efficacy of new therapies. We assessed the impact of optimism bias on a proportion of trials that did not answer their research question successfully and explored whether poor accrual or optimism bias is responsible for inconclusive results. STUDY
designSystematic review.
settingRetrospective analysis of a consecutive-series phase III randomized controlled trials (RCTs) performed under the aegis of National Cancer Institute Cooperative groups.
resultsThree hundred fifty-nine trials (374 comparisons) enrolling 150,232 patients were analyzed. Seventy percent (262 of 374) of the trials generated conclusive results according to the statistical criteria. Investigators made definitive statements related to the treatment preference in 73% (273 of 374) of studies. Investigators' judgments and statistical inferences were concordant in 75% (279 of 374) of trials. Investigators consistently overestimated their expected treatment effects but to a significantly larger extent for inconclusive trials. The median ratio of expected and observed hazard ratio or odds ratio was 1.34 (range: 0.19-15.40) in conclusive trials compared with 1.86 (range: 1.09-12.00) in inconclusive studies (P<0.0001). Only 17% of the trials had treatment effects that matched original researchers' expectations.
conclusionFormal statistical inference is sufficient to answer the research question in 75% of RCTs. The answers to the other 25% depend mostly on subjective judgments, which at times are in conflict with statistical inference. Optimism bias significantly contributes to inconclusive results.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.