ArticleBMC medical research methodology2010
Can we rely on the best trial? A comparison of individual trials and systematic reviews.
Article in BMC medical research methodology, 2010. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 12 papers, 9 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
12 citing papers in PubMed, 9 syntheses or guidelines pooled it.
- Switching antipsychotics versus continued current treatment in people with non-responsive schizophrenia.The Cochrane database of systematic reviews · 2025Pooled it
- Novel functional insights into ischemic stroke biology provided by the first genome-wide association study of stroke in indigenous Africans.Genome medicine · 2024Pooled it
- Natriuretic peptide-guided treatment for heart failure: a systematic review and meta-analysis.BMJ evidence-based medicine · 2020Pooled it
- Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study.Journal of clinical epidemiology · 2019Pooled it
- Glucocorticoids for acute viral bronchiolitis in infants and young children.The Cochrane database of systematic reviews · 2013Pooled it
- Influence of trial sample size on treatment effect estimates: meta-epidemiological study.BMJ (Clinical research ed.) · 2013Pooled it
- Intraocular pressure-lowering effects of commonly used fixed-combination drugs with timolol: a systematic review and meta-analysis.PloS one · 2012Pooled it
- Epinephrine for bronchiolitis.The Cochrane database of systematic reviews · 2011Pooled it
- Steroids and bronchodilators for acute bronchiolitis in the first two years of life: systematic review and meta-analysis.BMJ (Clinical research ed.) · 2011Pooled it
- Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers.BMC medical research methodology · 2020Article
- Article
- Reaching consensus on the physiotherapeutic management of patients following upper abdominal surgery: a pragmatic approach to interpret equivocal evidence.BMC medical informatics and decision making · 2012Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
3 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundThe ideal evidence to answer a question about the effectiveness of treatment is a systematic review. However, for many clinical questions a systematic review will not be available, or may not be up to date. One option could be to use the evidence from an individual trial to answer the question?
methodsWe assessed how often (a) the estimated effect and (b) the p-value in the most precise single trial in a meta-analysis agreed with the whole meta-analysis. For a random sample of 200 completed Cochrane Reviews (January, 2005) we identified a primary outcome and extracted: the number of trials, the statistical weight of the most precise trial, the estimate and confidence interval for both the highest weighted trial and the meta-analysis overall. We calculated the p-value for the most precise trial and meta-analysis.
resultsOf 200 reviews, only 132 provided a meta-analysis of 2 or more trials, with a further 35 effect estimates based on single trials. The average number of trials was 7.3, with the most precise trial contributing, on average, 51% of the statistical weight to the summary estimate from the whole meta-analysis. The estimates of effect from the most precise trial and the overall meta-analyses were highly correlated (rank correlation of 0.90).There was an 81% agreement in statistical conclusions. Results from the most precise trial were statistically significant in 60 of the 167 evaluable reviews, with 55 of the corresponding systematic reviews also being statistically significant. The five discrepant results were not strikingly different with respect to their estimates of effect, but showed considerable statistical heterogeneity between trials in these meta-analyses. However, among the 101 cases in which the most precise trial was not statistically significant, the corresponding meta-analyses yielded 31 statistically significant results.
conclusionsSingle most precise trials provided similar estimates of effects to those of the meta-analyses to which they contributed, and statistically significant results are generally in agreement. However, "negative" results were less reliable, as may be expected from single underpowered trials. For systematic reviewers we suggest that: (1) key trial(s) in a review deserve greater attention (2) systematic reviewers should check agreement of the most precise trial and the meta analysis. For clinicians using trials we suggest that when a meta-analysis is not available, a focus on the most precise trial is reasonable provided it is adequately powered.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.