Evidence map›Paper›PMID 29898777›Full record

ReviewSystematic reviews2018

Evaluation of the reliability, usability, and applicability of AMSTAR, AMSTAR 2, and ROBIS: protocol for a descriptive analytic study.

Allison Gates, Michelle Gates, Gonçalo Duarte, Maria Cary, Monika Becker, Barbara Prediger, Ben Vandermeer, Ricardo M Fernandes, Dawid Pieper, Lisa Hartling

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in Systematic reviews, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 26 papers, 8 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
26citing papers in PubMed, 8 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

26 citing papers in PubMed, 8 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Frontiers in nutrition · 2023
    Pooled it
  7. Pooled it
  8. Association betweenBMJ open · 2020
    Pooled it
  9. Review
  10. Article
  11. Review
  12. Article
  13. Article
  14. Review
  15. Review
  16. Review
  17. Review
  18. Article
  19. Review
  20. The Effectiveness of Community Occupational Therapy Interventions: A Scoping Review.International journal of environmental research and public health · 2021
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Allison GatesAlberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada.
Michelle GatesAlberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada.
Gonçalo DuarteClinical Pharmacology Unit, Instituto de Medicina Molecular, University of Lisbon, Av. Professor Egas Moniz, 1649-028, Lisbon, Portugal.
Maria CaryCentre for Health Evaluation & Research (CEFAR), National Association of Pharmacies, Rua Marechal Saldanha, no 1, 1249-069, Lisbon, Portugal.
Monika BeckerDepartment für Humanmedizin, Institut für Forschung in der Operativen Medizin, Universität Witten/Herdecke, Witten, Germany.
Barbara PredigerDepartment für Humanmedizin, Institut für Forschung in der Operativen Medizin, Universität Witten/Herdecke, Witten, Germany.
Ben VandermeerAlberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada.
Ricardo M FernandesClinical Pharmacology Unit, Instituto de Medicina Molecular, University of Lisbon, Av. Professor Egas Moniz, 1649-028, Lisbon, Portugal.
Dawid PieperDepartment für Humanmedizin, Institut für Forschung in der Operativen Medizin, Universität Witten/Herdecke, Witten, Germany.
Lisa HartlingAlberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. hartling@ualberta.ca.ORCID 0000-0001-8341-3991

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundSystematic reviews (SRs) of randomised controlled trials (RCTs) can provide the best evidence to inform decision-making, but their methodological and reporting quality varies. Tools exist to guide the critical appraisal of quality and risk of bias in SRs, but evaluations of their measurement properties are limited. We will investigate the interrater reliability (IRR), usability, and applicability of A MeaSurement Tool to Assess systematic Reviews (AMSTAR), AMSTAR 2, and Risk Of Bias In Systematic reviews (ROBIS) for SRs in the fields of biomedicine and public health.

methodsAn international team of researchers at three collaborating centres will undertake the study. We will use a random sample of 30 SRs of RCTs investigating therapeutic interventions indexed in MEDLINE in February 2014. Two reviewers at each centre will appraise the quality and risk of bias in each SR using AMSTAR, AMSTAR 2, and ROBIS. We will record the time to complete each assessment and for the two reviewers to reach consensus for each SR. We will extract the descriptive characteristics of each SR, the included studies, participants, interventions, and comparators. We will also extract the direction and strength of the results and conclusions for the primary outcome. We will summarise the descriptive characteristics of the SRs using means and standard deviations, or frequencies and proportions. To test for interrater reliability between reviewers and between the consensus agreements of reviewer pairs, we will use Gwet's AC DISCUSSION: Appraising the methodological and reporting quality of SRs is necessary to determine the trustworthiness of their conclusions. Which tool may be most reliably applied and how the appraisals should be used is uncertain; the usability of newly developed tools is unknown. This investigation of common (AMSTAR) and newly developed (AMSTAR 2, ROBIS) tools will provide empiric data to inform their application, interpretation, and refinement.

Indexed as

BiasRandomized Controlled Trials as TopicReproducibility of ResultsEvidence-Based MedicineHumansQuality Assurance, Health CareResearch DesignOverviews of reviewsQuality assessmentReliabilityRisk of biasSystematic reviewsValidity

Identifiers

PMID29898777
PMCPMC6000957

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.