Evidence map›Paper›PMID 41501837›Full record

ArticleSystematic reviews2026

Methodological quality of systematic intervention reviews on vaccination.

Antonia Pilic, Louise Henaff, Christoph A Steffen, Madeleine Batke, Hanna Helene Linß, Iksha Thebe, Nina Rehr, Catalina Hamon Pinilla, Amrita John, Ole Wichmann and 2 more

Abstract read
In one paragraph

Article in Systematic reviews, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

12 authors.

Antonia PilicRobert Koch Institute, Immunization Unit, Berlin, Germany. pilicA@rki.de.ORCID 0000-0001-6502-7982
Louise HenaffWorld Health Organization (WHO), 20 Avenue Appia, Geneva, 1211, Switzerland.
Christoph A SteffenWorld Health Organization (WHO), 20 Avenue Appia, Geneva, 1211, Switzerland.
Madeleine BatkeRobert Koch Institute, Immunization Unit, Berlin, Germany.
Hanna Helene LinßRobert Koch Institute, Immunization Unit, Berlin, Germany.
Iksha ThebeRobert Koch Institute, Immunization Unit, Berlin, Germany.
Nina RehrRobert Koch Institute, Immunization Unit, Berlin, Germany.
Catalina Hamon PinillaRobert Koch Institute, Immunization Unit, Berlin, Germany.
Amrita JohnRobert Koch Institute, Immunization Unit, Berlin, Germany.
Ole WichmannRobert Koch Institute, Immunization Unit, Berlin, Germany.
Vanessa PiechottaRobert Koch Institute, Immunization Unit, Berlin, Germany.
Thomas HarderRobert Koch Institute, Immunization Unit, Berlin, Germany.

Funding

World Health Organization 001
6 · The paper itself

Abstract

backgroundSystematic reviews (SRs) are pivotal in vaccination recommendation development. However, the reliability of their findings relies on methodological rigor. This study examines the methodological quality of vaccination-related SRs and aims to identify influencing factors.

methodsWe used the SYSVAC registry to draw 120 SRs on the efficacy or effectiveness of vaccination using a randomized block design. SRs published from 2011 to 2023 were considered. SR characteristics were extracted, and methodological quality was assessed by two reviewers using AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews 2). SRs were scored between 0 and 16 and categorized as high, moderate, low, or critically low quality. Mann-Whitney U test, chi-squared test, Fisher's exact test, and multivariable linear regression analysis were used to assess the influence of SR characteristics on AMSTAR 2 score. Secondary analysis of critically low-rated SRs was conducted to identify limitations in critical items.

resultsOut of 120 SRs, 110 SRs were rated as of critically low-quality. The majority of critically low-rated SRs lacked the justification for excluding individual studies (n = 103, 93.6%) and protocol registration (n = 85, 77.3%). Median AMSTAR 2 score across all SRs was 10 (range 2-16). SRs published after 2017, Cochrane reviews, and SRs from authors with no conflicts of interest (CoI) had higher methodological quality than those published before 2017, non-Cochrane reviews, and SRs from authors with CoI (mean difference AMSTAR 2 score 2, 6, and 2, respectively; all p < 0.05). SRs published before 2017 had significant limitations in protocol registration, study selection in duplicate, and risk of bias assessment; and non-Cochrane reviews in protocol registration, justification for excluding individual studies, funding sources of studies, and impact of risk of bias assessment of individual studies (all p < 0.05). Management of conflicts was described only in a quarter of SRs with CoI (n = 9/43, 20.9%). The adjusted analysis showed that only publication year after 2017 and Cochrane review status had a significant independent relation on the AMSTAR 2 score (R

conclusionThe findings underscore the critical methodological shortcomings in vaccination-related SRs. Future efforts should prioritize adherence to established methodological standards and transparency, to enhance the impact of SRs in individual and decision-making.

Indexed as

Research DesignSystematic Reviews as TopicVaccinationHumansAMSTAR 2Methodological qualityMethodologyNational Immunization Technical Advisory Group (NITAG)Systematic reviewVaccination

Identifiers

PMID41501837
PMCPMC12870254

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.