Evidence map›Paper›PMID 29540417›Full record

SynthesisBMJ open2018

Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review.

Matthew J Page, Joanne E McKenzie, Julian P T Higgins

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in BMJ open, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 122 papers, 56 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
122citing papers in PubMed, 56 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

122 citing papers in PubMed, 56 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Irish Dancing Injuries and Associated Risk Factors: A Systematic Review.International journal of environmental research and public health · 2023
    Pooled it

62 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Matthew J PageSchool of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
Joanne E McKenzieSchool of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
Julian P T HigginsPopulation Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.

Funding

Medical Research Council MC_UU_12013/9Medical Research Council MR/K025643/1Medical Research Council MR/M025209/1
6 · The paper itself

Abstract

backgroundSeveral scales, checklists and domain-based tools for assessing risk of reporting biases exist, but it is unclear how much they vary in content and guidance. We conducted a systematic review of the content and measurement properties of such tools.

methodsWe searched for potentially relevant articles in Ovid MEDLINE, Ovid Embase, Ovid PsycINFO and Google Scholar from inception to February 2017. One author screened all titles, abstracts and full text articles, and collected data on tool characteristics.

resultsWe identified 18 tools that include an assessment of the risk of reporting bias. Tools varied in regard to the type of reporting bias assessed (eg, bias due to selective publication, bias due to selective non-reporting), and the level of assessment (eg, for the study as a whole, a particular result within a study or a particular synthesis of studies). Various criteria are used across tools to designate a synthesis as being at 'high' risk of bias due to selective publication (eg, evidence of funnel plot asymmetry, use of non-comprehensive searches). However, the relative weight assigned to each criterion in the overall judgement is unclear for most of these tools. Tools for assessing risk of bias due to selective non-reporting guide users to assess a study, or an outcome within a study, as 'high' risk of bias if no results are reported for an outcome. However, assessing the corresponding risk of bias in a synthesis that is missing the non-reported outcomes is outside the scope of most of these tools. Inter-rater agreement estimates were available for five tools.

conclusionThere are several limitations of existing tools for assessing risk of reporting biases, in terms of their scope, guidance for reaching risk of bias judgements and measurement properties. Development and evaluation of a new, comprehensive tool could help overcome present limitations.

Indexed as

BiasOutcome Assessment, Health CareBiomedical ResearchHumansInformation DisseminationRisk Assessmentbias (epidemiology)checklistpublication biasreview literature as topic

Identifiers

PMID29540417
PMCPMC5857645

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.