Evidence map›Paper›PMID 42902408›Full record

ArticleSystematic reviews2026

Unified tools for assessing the methodological quality of intervention effects in rapid reviews: a scoping review.

Deborah Edwards, Emily C Clark, Judit Csontos, Maureen Dobbins, Elizabeth Gillen, Juliet Hounsome, Sarah E Neil-Sztramko, Ruth Lewis, Mala Mann, Gillian Prue

Abstract readScoping Review
PubMed Full text at PMC Publisher
In one paragraph

Article in Systematic reviews, 2026. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Not yet cited in PubMed.

0numbers the graph read from it
0cells of the map it votes in
0citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

0 citing papers in PubMed.

No citing paper in PubMed yet.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Deborah EdwardsJBI Wales Centre For Evidence Based Care, School of Healthcare Sciences, Cardiff University, Cardiff, Wales, UK. edwardsdj@cardiff.ac.uk.ORCID http://orcid.org/0000-0003-1885-9297
Emily C ClarkNational Collaborating Centre for Methods and Tools (NCCMT), Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, Canada.
Judit CsontosJBI Wales Centre For Evidence Based Care, School of Healthcare Sciences, Cardiff University, Cardiff, Wales, UK.
Maureen DobbinsNational Collaborating Centre for Methods and Tools (NCCMT), School of Nursing, McMaster University, Hamilton, Canada.
Elizabeth GillenJBI Wales Centre For Evidence Based Care, School of Healthcare Sciences, Cardiff University, Cardiff, Wales, UK.
Juliet HounsomeSpecialist Unit for Review Evidence (SURE), Cardiff University Library Service, Cardiff, Wales, UK.
Sarah E Neil-SztramkoNational Collaborating Centre for Methods and Tools (NCCMT), Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, Canada.
Ruth LewisNorth Wales Medical School, College of Medicine and Health, Bangor University, Bangor, Wales, UK.
Mala MannSpecialist Unit for Review Evidence (SURE), Cardiff University Library Service, Cardiff, Wales, UK.
Gillian PrueSchool of Nursing and Midwifery, Queen's University Belfast, Belfast, UK.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundEvaluating the methodological quality of primary studies is a crucial aspect of evidence syntheses, such as rapid reviews. Rapid reviews often include both randomised controlled trials (RCTs) and non-randomised studies (NRSIs) requiring multiple design-specific quality assessment tools. This can complicate workflows and reduce efficiency. Unified tools, designed to assess methodological quality across diverse study designs, may offer a more consistent and streamlined approach. This scoping review aimed to comprehensively identify and describe unified tools for assessing methodological quality across both RCTs and NRSIs.

methodsThe review followed JBI scoping review methodology and was reported in line with PRISMA-ScR guidance. Searches were conducted in MEDLINE (Ovid), Embase (Ovid), and CINAHL (EBSCO) from 1998 to 2024, alongside grey literature searches, citation tracking, targeted website searches, and consultation with experts in evidence synthesis. Study selection was performed by two independent reviewers. A tool was defined as any structured instrument developed to support users in assessing methodological quality.

resultsA total of 55 publications were included, identifying 29 unique unified tools. These were categorised by structure: scales (n = 14), checklists with an additional summary judgement (n = 7), simple checklists (n = 5), domain-based tools (n = 2), and other tools (n = 1). All tools were designed to assess both RCTs and NRSIs, with many also applicable to additional study designs, including descriptive studies (n = 15), case reports or series (n = 9), qualitative research (n = 7), and systematic reviews (n = 3). Unified tools were developed using a mixture of approaches including literature reviewing (n = 25), expert consensus (n = 20), and stakeholder consultations (n = 17) and pilot testing (n = 17). Evaluation of psychometric properties was conducted to varying levels across different domains of validity and reliability.

conclusionThis review identified a diverse set of unified tools for assessing methodological quality across both RCTs and NRSIs. However, variation in tool structure, the availability of accompanying guidance, and the limited evaluation of psychometric properties, particularly inter-rater reliability and criterion validity, indicate potential issues for adoption in rapid reviews. Unified tools need further refinement and validation to be embedded in the rapid review process. SYSTEMATIC REVIEW REGISTRATION: https://osf.io/nyteu/.

Indexed as

Research DesignReview Literature as TopicHumansRandomized Controlled Trials as TopicMethodological qualityMethodologyRapid reviewsScoping reviewsUnified framework

Identifiers

PMID42902408
PMCPMC13685391

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.