Evidence map›Paper›PMID 29663281›Full record

ArticleJournal of general internal medicine2018

Empirical Comparison of Publication Bias Tests in Meta-Analysis.

Lifeng Lin, Haitao Chu, Mohammad Hassan Murad, Chuan Hong, Zhiyong Qu, Stephen R Cole, Yong Chen

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Journal of general internal medicine, 2018. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 222 papers, 125 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
222citing papers in PubMed, 125 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

222 citing papers in PubMed, 125 syntheses or guidelines pooled it.

  1. Red cell distribution width as a prognostic predictor for colorectal cancer: a meta-analysis.Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico · 2026
    Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Guideline
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

162 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

7 authors.

Lifeng LinDepartment of Statistics, Florida State University, Tallahassee, FL, USA.
Haitao ChuDivision of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA. chux0051@umn.edu.
Mohammad Hassan MuradEvidence-Based Practice Center, Mayo Clinic, Rochester, MN, USA.
Chuan HongDepartment of Biostatistics, Harvard School of Public Health, Boston, MA, USA.
Zhiyong QuSchool of Social Development and Public Policy, Beijing Normal University, Beijing, China.
Stephen R ColeDepartment of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, USA.
Yong ChenDepartment of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA, USA.

Funding

Women's CancerP30CA077598 · NCI · UNIVERSITY OF MINNESOTA TWIN CITIES · PI Douglas Yee · 1998 to 2026
$100.4M
Research CoreU54MD008620 · NIMHD · UNIVERSITY OF MINNESOTA · PI KONETY, BADRINATH R, SHIKANY, JAMES M · 2013 to 2017
$13.5M
Prevention of Lower Urinary Tract Symptoms in Women: Bladder Health Scientific and Data Coordinating Center (PLUS-SDCC) (U01)U01DK106786 · NIDDK · UNIVERSITY OF MINNESOTA · PI HARLOW, BERNARD L, RUDSER, KYLE · 2015 to 2019
$8.4M
Dynamic learning for post-vaccine event prediction using temporal information in VAERSR01AI130460 · NIAID · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI CHEN, YONG, TAO, CUI · 2017 to 2021
$3.4M
A General Framework to Account for Outcome Reporting Bias in Systematic ReviewsR01LM012607 · NLM · UNIVERSITY OF PENNSYLVANIA · PI CHEN, YONG · 2017 to 2020
$1.4M
Statistical Methods and Software for Multivariate Meta-analysisR21LM012197 · NLM · UNIVERSITY OF MINNESOTA · PI CHU, HAITAO · 2015 to 2016
$385k
Bayesian Methods and Software for Patient-Centered Network Meta-Analysis of BinarR21AI103012 · NIAID · UNIVERSITY OF MINNESOTA · PI CHU, HAITAO · 2013 to 2014
$385k
Joint Meta-Regression Methods Accounting for Postrandomization VariablesR21LM012744 · NLM · UNIVERSITY OF MINNESOTA · PI CHU, HAITAO · 2017 to 2018
$376k
Aiding Effective Decision Making in Dental Research Using Network Meta-analysisR03DE024750 · NIDCR · UNIVERSITY OF MINNESOTA · PI CHU, HAITAO, JOHN, MIKE · 2015 to 2016
$292k
6 · The paper itself

Abstract

backgroundDecision makers rely on meta-analytic estimates to trade off benefits and harms. Publication bias impairs the validity and generalizability of such estimates. The performance of various statistical tests for publication bias has been largely compared using simulation studies and has not been systematically evaluated in empirical data.

methodsThis study compares seven commonly used publication bias tests (i.e., Begg's rank test, trim-and-fill, Egger's, Tang's, Macaskill's, Deeks', and Peters' regression tests) based on 28,655 meta-analyses available in the Cochrane Library.

resultsEgger's regression test detected publication bias more frequently than other tests (15.7% in meta-analyses of binary outcomes and 13.5% in meta-analyses of non-binary outcomes). The proportion of statistically significant publication bias tests was greater for larger meta-analyses, especially for Begg's rank test and the trim-and-fill method. The agreement among Tang's, Macaskill's, Deeks', and Peters' regression tests for binary outcomes was moderately strong (most κ's were around 0.6). Tang's and Deeks' tests had fairly similar performance (κ > 0.9). The agreement among Begg's rank test, the trim-and-fill method, and Egger's regression test was weak or moderate (κ < 0.5).

conclusionsGiven the relatively low agreement between many publication bias tests, meta-analysts should not rely on a single test and may apply multiple tests with various assumptions. Non-statistical approaches to evaluating publication bias (e.g., searching clinical trials registries, records of drug approving agencies, and scientific conference proceedings) remain essential.

Indexed as

Meta-Analysis as TopicEmpirical ResearchHumansPublication BiasSystematic Reviews as TopicCochrane Libraryfunnel plotmeta-analysispublication biasstatistical test

Identifiers

PMID29663281
PMCPMC6082203

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.