SynthesisSystematic reviews2020
Journal impact factor, trial effect size, and methodological quality appear scantly related: a systematic review and meta-analysis.
Synthesis in Systematic reviews, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers, 4 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed, 4 syntheses or guidelines pooled it.
- Stem cell-derived exosome treatment for acute spinal cord injury: a systematic review and meta-analysis based on preclinical evidence.Frontiers in neurology · 2025Pooled it
- The impact of radiomics for human papillomavirus status prediction in oropharyngeal cancer: systematic review and radiomics quality score assessment.Neuroradiology · 2022Pooled it
- Methodological approaches for assessing certainty of the evidence in umbrella reviews: A scoping review.PloS one · 2022Pooled it
- Should low-quality evidence dominate high-level evidence? A systematic review and meta-analysis of systematic reviews of musculoskeletal physical therapy interventions.The Journal of manual & manipulative therapy · 2021Pooled it
- Positivity bias in scientific research: Solutions for change.Science advances · 2026Review
- Methodological Rigor and Integrity of Systematic Reviews and Meta-Analyses From Nursing-Affiliated Institutions in North African Countries: The FALCON-1 Study.Cochrane evidence synthesis and methods · 2026Article
- Assessment of Selective Reporting Biases in Studies Included in Campbell Systematic Reviews: A Systematic Review.Campbell systematic reviews · 2026Review
- Marginal Improvements in Risk of Bias of Nephrology Randomized Controlled Trials Over Time: A Meta-research Study of Cochrane Reviews of Randomized Controlled Trials.Kidney medicine · 2026Article
- Ending publication bias: A values-based approach to surface null and negative results.PLoS biology · 2025Article
- Reported methodological quality of medical systematic reviews: Development of an assessment tool (ReMarQ) and meta-research study.Research synthesis methods · 2025Article
- Bibliometric analysis of research trends on the combination of radiotherapy and PARP inhibitors in solid tumors.Frontiers in pharmacology · 2025Review
- A pilot study investigating the relationship between journal impact factor and methodological quality of real-world observational studies.Frontiers in research metrics and analytics · 2025Article
- The artificial intelligence revolution...in unethical publishing: Will AI worsen our dysfunctional publishing system?The Journal of general physiology · 2024Article
- EJHP's impact factor moves on up.European journal of hospital pharmacy : science and practice · 2022Article
- Faculty appointment and promotion in Taiwan's medical schools, a systematic analysis.BMC medical education · 2022Article
- Definition and Characteristics of Mesenchymal Stromal Cells in Preclinical and Clinical Studies: A Scoping Review.Stem cells translational medicine · 2022Article
- Methodology over metrics: current scientific standards are a disservice to patients and society.Journal of clinical epidemiology · 2021Article
- Evolving trends of systematic reviews on virtual reality for stroke rehabilitation.Archives of physiotherapyArticle
- Adherence to the PRISMA statement and its association with risk of bias in systematic reviews published in rehabilitation journals: A meta-research study.Brazilian journal of physical therapyReview
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
7 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundAs systematic reviews' limited coverage of the medical literature necessitates decision-making based on unsystematic review, we investigated a possible advantage of systematic review (aside from dataset size and systematic analysis): does systematic review avoid potential bias in sampling primary studies from high impact factor journals? If randomized controlled trials (RCTs) reported in higher-impact journals present different treatment benefits than RCTs reported in lower-impact journals, readers who focus on higher-impact journals for their rapid literature reviews may introduce bias which could be mitigated by complete, systematic sampling.
methodsWe randomly sampled Cochrane Library (20 July 2005) treatment reviews that measured mortality as a binary outcome, published in English or French, with at least five RCTs with one or more deaths. Our domain-based assessment of risk of bias included funding source, randomness of allocation sequence, blinding, and allocation concealment. The primary analysis employed logistic regression by a generalized linear model with a generalized estimating equation to estimate the association between various factors and publication in a journal with a high journal impact factor (JIF).
resultsFrom the 29 included systematic reviews, 189 RCTs contributed data. However, in the primary analyses comparing RCT results within meta-analyses, there was no statistically significant association: unadjusted odds of greater than 50% mortality protection in high-JIF (> 5) journals were 1.4 (95% CI 0.42, 4.4) and adjusted, 2.5 (95% CI 0.6, 10). Elements of study quality were weakly, inconsistently, and not statistically significantly correlated with journal impact factor.
conclusionsJournal impact factor may have little to no association with study results, or methodological quality, but the evidence is very uncertain.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.