Evidence map›Paper›PMID 32487022›Full record

SynthesisBMC medical research methodology2020

Publication and related biases in health services research: a systematic review of empirical evidence.

Abimbola A Ayorinde, Iestyn Williams, Russell Mannion, Fujian Song, Magdalena Skrybant, Richard J Lilford, Yen-Fu Chen

Abstract readSystematic Review
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in BMC medical research methodology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 5 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
16citing papers in PubMed, 5 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

16 citing papers in PubMed, 5 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Article
  7. Review
  8. Article
  9. Review
  10. Review
  11. Article
  12. Review
  13. Article
  14. Review
  15. Article
  16. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Abimbola A AyorindeWarwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK.
Iestyn WilliamsHealth Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK.
Russell MannionHealth Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK.
Fujian SongNorwich Medical School, University of East Anglia, Norwich, UK.
Magdalena SkrybantInstitute of Applied Health Research, University of Birmingham, Birmingham, UK.
Richard J LilfordInstitute of Applied Health Research, University of Birmingham, Birmingham, UK.
Yen-Fu ChenWarwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK. Y-F.Chen@warwick.ac.uk.ORCID 0000-0002-9446-2761

Funding

Department of Health 15/71/06Health Services and Delivery Research Programme HS&DR 15/71/06National Institute for Health Research Applied Research Collaboration West Midlands
6 · The paper itself

Abstract

backgroundPublication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR.

methodsDatabases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively.

resultsAfter screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases.

conclusionsMethodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. SYSTEMATIC REVIEW REGISTRATION: PROSPERO 2016 CRD42016052333.

Indexed as

Health Services ResearchResearch DesignBiasHumansPublication BiasDissemination biasFunnel plotsGrey literatureHealth services researchOutcome reporting biasPublication biasResearch methodologyResearch publicationResearch registrationSystematic review

Identifiers

PMID32487022
PMCPMC7268600

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.