Evidence map›Paper›PMID 32423382›Full record

SynthesisBMC medical research methodology2020

Methodological tools and sensitivity analysis for assessing quality or risk of bias used in systematic reviews published in the high-impact anesthesiology journals.

Marija Franka Marušić, Mahir Fidahić, Cristina Mihaela Cepeha, Loredana Gabriela Farcaș, Alexandra Tseke, Livia Puljak

Abstract readMeta-Analysis
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in BMC medical research methodology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 66 papers, 45 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
66citing papers in PubMed, 45 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

66 citing papers in PubMed, 45 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

6 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Marija Franka MarušićSchool of Medicine, University of Split, Split, Croatia.
Mahir FidahićMedical Faculty, University of Tuzla, Tuzla, Canton Tuzla, Bosnia and Herzegovina.
Cristina Mihaela CepehaThe Victor Babes, University of Medicine and Pharmacy, Timisoara, Romania.
Loredana Gabriela FarcașIuliu Hațieganu University of Medicine and Pharmacy, Faculty of Medicine, Cluj-Napoca, Romania.
Alexandra TsekeMedical School, National and Kapodistrian University of Athens, Athens, Greece.
Livia PuljakCenter for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia. livia.puljak@gmail.com.ORCID 0000-0002-8467-6061

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundA crucial element in the systematic review (SR) methodology is the appraisal of included primary studies, using tools for assessment of methodological quality or risk of bias (RoB). SR authors can conduct sensitivity analyses to explore whether their results are sensitive to exclusion of low quality studies or a high RoB. However, it is unknown which tools do SR authors use for assessing quality/RoB, and how they set threshold for quality/RoB in sensitivity analyses. The aim of this study was to assess quality/RoB assessment tools, the types of sensitivity analyses and quality/RoB thresholds for sensitivity analyses used within SRs published in high-impact pain/anesthesiology journals.

methodsThis was a methodological study. We analyzed SRs published from January 2005 to June 2018 in the 25% highest-ranking journals within the Journal Citation Reports (JCR) "Anesthesiology" category. We retrieved the SRs from PubMed. Two authors independently screened records, full texts, and extracted data on quality/RoB tools and sensitivity analyses. We extracted data about quality/RoB tools, types of sensitivity analyses and the thresholds for quality/RoB used in them.

resultsOut of 678 analyzed SRs, 513 (76%) reported the use of quality/RoB assessments. The most commonly reported tools for assessing quality/RoB in the studies were the Cochrane tool for risk of bias assessment (N = 251; 37%) and Jadad scale (N = 99; 15%). Meta-analysis was conducted in 451 (66%) of SRs and sensitivity analysis in 219/451 (49%). Most commonly, sensitivity analysis was conducted to explore the influence of study quality/RoB (90/219; 41%) on the results. Quality/RoB thresholds used for sensitivity analysis for those studies were clearly reported in 47 (52%) articles that used them. The quality/RoB thresholds used for sensitivity analyses were highly heterogeneous and inconsistent, even when the same tool was used.

conclusionsA quarter of SRs reported using quality/RoB assessments, and some of them cited tools that are not meant for assessing quality/RoB. Authors who use quality/RoB to explore the robustness of their results in meta-analyses use highly heterogeneous quality/RoB thresholds in sensitivity analyses. Better methodological consistency for quality/RoB sensitivity analyses is needed.

Indexed as

AnesthesiologyPeriodicals as TopicBiasHumansResearch ReportSystematic Reviews as TopicBiasQuality assessmentSensitivity analysisSystematic review

Identifiers

PMID32423382
PMCPMC7236513

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.