ArticleBMC medical research methodology2019
A descriptive analysis of the characteristics and the peer review process of systematic review protocols published in an open peer review journal from 2012 to 2017.
Article in BMC medical research methodology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 7 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
7 citing papers in PubMed, 36 citations in OpenAlex.
- Does the disconnect between the peer-reviewed label and reality explain the peer review crisis, and can open peer review or preprints resolve it? A narrative review.Naunyn-Schmiedeberg's archives of pharmacology · 2026Review
- Factors associated with successful publication for systematic review protocol registration: an analysis of 397 registered protocols.Systematic reviews · 2023Article
- Methodological quality of systematic reviews on treatments for Alzheimer's disease: a cross-sectional study.Alzheimer's research & therapy · 2022Article
- Article
- Is reusing text from a protocol in the completed systematic review acceptable?Systematic reviews · 2021Article
- A tutorial on methodological studies: the what, when, how and why.BMC medical research methodology · 2020Review
- Response to letter to the editor.Pediatric surgery international · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
5 authors at 2 institutions in 1 country.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundAn a priori design is essential to reduce the risk of bias in systematic reviews (SRs). To this end, authors can register their SR with PROSPERO, and/or publish a SR protocol in an academic journal. The latter has the advantage that the manuscript for the SR protocol is usually peer-reviewed. However, since authors ought not to begin/continue the SR before their protocol has been accepted for publication, it is crucial that SR protocols are processed in a timely manner. Our main aim was to descriptively analyse the peer review process of SR protocols published in 'BMC Systematic Reviews' from 2012 to 2017.
methodsWe systematically searched MEDLINE via PubMed for all SR protocols published in 'BMC Systematic Reviews' between 2012 and 2017, except for protocols for overviews, scoping reviews or realist reviews. Data were extracted from the SR protocols and Open Peer Review reports. For each round of peer review, two researchers judged the extent of revision (minor/major) based on the reviewer reports. Their content was further investigated by two researchers in a random 10%-sample using PRISMA-P as a guideline. All data were analysed descriptively.
resultsWe identified 544 eligible protocols published in 'BMC Systematic Reviews' between 2012 and 2017. Of those, 485 (89.2%) also registered the SR in PROSPERO, the majority (87.4%) before first submission of the manuscript for the SR protocol (median 49 days). The absolute number of published SR protocols increased from 2012 to 2017 (21 vs 145 protocols), as did the median processing time (61 vs 142 days from submission to acceptance) and the proportion of protocols requiring a major revision after first peer review (19.1% vs 52.4%). Reviewer comments most frequently addressed the PRISMA-P item 'Eligibility criteria'. Overall, 76.0% of the reviewer comments suggested more transparency.
conclusionsThe number of published SR protocols increased over the years, but so did the processing time. In 2017, it took several months from submission to acceptance, which is critical from an author's perspective. New models of peer review such as post publication peer review for SR protocols should be investigated. This could probably be realized with PROSPERO.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.