Evidence map›Paper›PMID 31382898›Full record

ArticleBMC medical research methodology2019

Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook.

Ognjen Barcot, Matija Boric, Tina Poklepovic Pericic, Marija Cavar, Svjetlana Dosenovic, Ivana Vuka, Livia Puljak

Open access · goldAbstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in BMC medical research methodology, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 24 papers, 10 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
24citing papers in PubMed, 10 pooled it
5.3field-weighted citation impact, top 3% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

24 citing papers in PubMed, 10 syntheses or guidelines pooled it, 53 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Article
  12. Article
  13. Article
  14. Review
  15. Article
  16. Review
  17. Review
  18. Article
  19. Review
  20. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors at 2 institutions in 1 country.

Ognjen BarcotDepartment of Surgery, University Hospital Split, Spinciceva 1, Split, Croatia.
Matija BoricDepartment of Surgery, University Hospital Split, Spinciceva 1, Split, Croatia.
Tina Poklepovic PericicDepartment for Research in Biomedicine and Health, University of Split, School of Medicine, Soltanska 2, Split, Croatia.
Marija CavarDepartment of Radiology, University Hospital Split, Spinciceva 1, Split, Croatia.
Svjetlana DosenovicDepartment of Anesthesiology and Intensive Care, University Hospital Split, Spinciceva 1, Split, Croatia.
Ivana VukaDepartment for Research in Biomedicine and Health, University of Split, School of Medicine, Soltanska 2, Split, Croatia.
Livia PuljakCenter for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia. livia.puljak@gmail.com.ORCID 0000-0002-8467-6061
University of Split · HRCatholic University of Croatia · HR

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundAssessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool are provided in the Cochrane Handbook for Systematic Reviews of Interventions (The Cochrane Handbook). The purpose of this study was to analyse whether Cochrane authors use adequate judgments about the RoB for random sequence generation of RCTs included in Cochrane reviews.

methodsWe extracted authors' judgments (high, low or unclear RoB) and supports for judgments (comments accompanying judgments which explain the rationale for a judgment) for random sequence generation of included RCTs from RoB tables of Cochrane reviews using automated data scraping. We categorised all supporting comments, analysed the number and type of various supporting comments and assessed adequacy of RoB judgment for randomisation in line with recommendations from the Cochrane Handbook.

resultsWe analysed 10,103 RCTs that were included in 704 Cochrane reviews. For 5,706 RCTs, randomisation was not described, but for the remaining RCTs, it was indicated that randomisation was performed using computer/software/internet (N = 2,850), random number table (N = 883), mechanical method (N = 359) or it was incomplete/inappropriate (N = 305). Overall, 1,220/10,103 trials (12%) did not have a RoB judgment in line with Cochrane Handbook guidance about randomisation. The highest proportion of misjudgements was found for trials with high RoB (28%), followed by those with low (20%) or unclear (3%). Therefore, one in eight judgments for the analysed domain in Cochrane reviews was not in line with Cochrane Handbook, and one in four if the judgment was "high risk".

conclusionAuthors of Cochrane reviews often make judgments about the RoB related to random sequence generation that are not in line with instructions given in the Cochrane Handbook, which compromises the reliability of the systematic reviews. Our results can help authors of both Cochrane and non-Cochrane reviews which use Cochrane RoB tool to avoid making common mistakes when assessing RoB in included trials.

Indexed as

BiasSystematic Reviews as TopicHumansJudgmentRandomized Controlled Trials as TopicReproducibility of ResultsResearch DesignCochraneRandomisationRisk of biasSelection biasSequence generationSystematic reviews

Identifiers

PMID31382898
PMCPMC6683577
OpenAlexW2964693286

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.