SynthesisBMC medical research methodology2019
Tools used to assess the quality of peer review reports: a methodological systematic review.
Synthesis in BMC medical research methodology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
26 citing papers in PubMed.
- Trial
- Discovery Stack Pilot demonstrates the feasibility and outcomes of a scientist-designed peer-review model that separates quality and impact.PLoS biology · 2026Article
- Paying peer reviewers: benefits, risks, and challenges.Naunyn-Schmiedeberg's archives of pharmacology · 2026Review
- Does the disconnect between the peer-reviewed label and reality explain the peer review crisis, and can open peer review or preprints resolve it? A narrative review.Naunyn-Schmiedeberg's archives of pharmacology · 2026Review
- Exploring reviewer self-assessment in the context of academic peer review.Scientific reports · 2025Article
- Scientific Publication Speed of Korean Medical Journals during the COVID-19 Era.Healthcare informatics research · 2024Article
- The Peer Review Process.Respiratory care · 2024Article
- A structured, journal-led peer-review mentoring program enhances peer review training.Research integrity and peer review · 2024Article
- How to Peer Review a Neurology Education Manuscript.Neurology. Education · 2023Review
- Prevalence of Short Peer Reviews in 3 Leading General Medical Journals.JAMA network open · 2023Article
- Reviewer training for improving grant and journal peer review.The Cochrane database of systematic reviews · 2023Article
- Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review.Research integrity and peer review · 2023Article
- Comparison between the General Assessment of Hospitalised Patient Tool and the Barthel Index: A Retrospective Study.Nursing reports (Pavia, Italy) · 2023Article
- Relationship between journal impact factor and the thoroughness and helpfulness of peer reviews.PLoS biology · 2023Article
- Characterization of Comments About bioRxiv and medRxiv Preprints.JAMA network open · 2023Article
- Knowledge and motivations of training in peer review: An international cross-sectional survey.PloS one · 2023Article
- Applications and Current Medico-Legal Challenges of Telemedicine in Ophthalmology.International journal of environmental research and public health · 2022Review
- Article
- Peer review analyze: A novel benchmark resource for computational analysis of peer reviews.PloS one · 2022Article
- Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
6 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
backgroundA strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research.
methodsWe conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis.
resultsWe identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of 'quality'. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer's comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18).
conclusionSeveral tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.