Evidence map›Paper›PMID 30764774›Full record

ArticleBMC medical research methodology2019

Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience.

Katarina Maticic, Marina Krnic Martinic, Livia Puljak

Open access · goldAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical research methodology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 15 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
15citing papers in PubMed, 2 pooled it
4.0field-weighted citation impact, top 5% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

15 citing papers in PubMed, 2 syntheses or guidelines pooled it, 41 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Review
  4. Article
  5. Review
  6. Review
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Review
  14. Article
  15. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors at 2 institutions in 1 country.

Katarina MaticicPharmacy Mirna Jelic Hrgic, Brodski Stupnik, Croatia.
Marina Krnic MartinicDepartment of ENT, Head and Neck Surgery, University Hospital Center, Split, Croatia.
Livia PuljakCenter for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia. livia.puljak@gmail.com.ORCID http://orcid.org/0000-0002-8467-6061
Catholic University of Croatia · HRUniversity of Split · HR

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundReporting quality of systematic reviews' (SRs) abstracts is important because this is often the only information about a study that readers have. The aim of this study was to assess adherence of SR abstracts in the field of anesthesiology with the reporting checklist PRISMA extension for Abstracts (PRISMA-A) and to analyze to what extent will the use of PRISMA-A yield concordant ratings in two raters without prior experience with the checklist.

methodsWe analyzed reporting quality of SRs with meta-analysis of randomized controlled trials of interventions published in the field of anesthesiology from 2012 to 2016 by using 12-item PRISMA-A checklist. After calibration exercise, two authors without prior experience with PRISMA-A scored the abstracts. Primary outcome was median adherence to PRISMA-A checklist. Secondary outcome was adherence to individual items of the checklist. We analyzed whether there was improvement in reporting of SR abstracts over time. Additionally, we analyzed discrepancies between the two raters in scoring individual PRISMA-A items.

resultsOur search yielded 318 results, of which we included 244 SRs. Median adherence to PRISMA-A checklist was 42% (5 items of 12). The majority of analyzed SR abstracts (N = 148, 61%) had a total adherence score under 50%, and not a single one had adherence above 75%. Adherence to individual items was very variable, ranging from 0% for reporting SR funding, to 97% for interpreting SR findings. Overall adherence to PRISMA-A did not change over the analyzed 5 years before and after publication of PRISMA-A in 2013. Even after calibration exercise, discrepancies between the two raters were found in 275 (9.3%) out of 2928 analyzed PRISMA-A items. Cohen's Kappa was 0.807. In the item about the description of effect there were discrepancies in 59% of the abstracts between the raters.

conclusionReporting quality of systematic review abstracts in the field of anesthesiology is suboptimal, and did not improve after publication of PRISMA-A checklist in 2013. We need stricter adherence to reporting checklists by authors, editors and peer-reviewers, and interventions that will help those stakeholders to improve reporting of systematic reviews. Some items of PRISMA-A checklist are difficult to score.

Indexed as

AbstractPRISMAReportingSystematic review

Identifiers

PMID30764774
PMCPMC6376734
OpenAlexW2934309103

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.