ArticlePloS one
Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study.
Article in PloS one. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 39 papers, 3 of them syntheses that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
39 citing papers in PubMed, 3 syntheses or guidelines pooled it.
- An appraisal of the methodology and quality of evidence of systematic reviews on the efficacy of prone positional ventilation in adult patients with acute respiratory distress syndrome: an umbrella review.Internal and emergency medicine · 2023Pooled it
- Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: cross sectional meta-research study.BMJ (Clinical research ed.) · 2022Pooled it
- Temporal Changes in Effect Sizes of Studies Comparing Individuals With and Without Autism: A Meta-analysis.JAMA psychiatry · 2019Pooled it
- Improving peer review of systematic reviews and related review types by involving librarians and information specialists as methodological peer reviewers: a randomised controlled trial.BMJ evidence-based medicine · 2025Trial
- Oral Structural and Functional Impairment in Relation to Depressive Outcomes in Older Adults: A Systematic Review and Meta-Analysis of Observational Studies.Diseases (Basel, Switzerland) · 2026Review
- Open Science Practices in Systematic Reviews From 2014 to 2024: A Cohort Study of 300 Systematic Reviews.Cochrane evidence synthesis and methods · 2026Article
- Data sharing in acupuncture meta-analyses: Associations with journal policies and practical considerations.Integrative medicine research · 2026Article
- Enhancing the quality of systematic reviews and meta-analyses.BJPsych open · 2025Article
- Search strategies for systematic reviews in reproductive medicine: a narrative review and practical guide.Journal of assisted reproduction and genetics · 2025Review
- Transparency in Cognitive Training Meta-analyses: A Meta-review.Neuropsychology review · 2025Review
- Transparency, quality, and statistical consistency of meta-analytic systematic reviews in clinical child and adolescent psychology (2022-2024): study protocol for a meta-review.Frontiers in psychology · 2025Article
- Searching and reporting in Campbell Collaboration systematic reviews: A systematic assessment of current methods.Campbell systematic reviews · 2024Review
- Effect of librarian collaboration on otolaryngology systematic review and meta-analysis quality.Journal of the Medical Library Association : JMLA · 2024Article
- Guidance for systematic reviews in journal author instructions: Findings and recommendations for editorial teams.Cochrane evidence synthesis and methods · 2024Article
- Addressing the challenges of reconstructing systematic reviews datasets: a case study and a noisy label filter procedure.Systematic reviews · 2024Article
- Open reproducible scientometric research with Alexandria3k.PloS one · 2023Article
- A suggested data structure for transparent and repeatable reporting of bibliographic searching.Campbell systematic reviews · 2022Article
- Health sciences and medical librarians conducting research and their experiences asking for co-authorship.Journal of the Medical Library Association : JMLA · 2022Article
- Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?Postgraduate medical journal · 2022Article
- A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000-2020).Behavior research methods · 2022Review
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
2 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundA high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines.
objectivesTo examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy.
methodsWe identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility.
results272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy.
conclusionsDespite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.