Evidence map›Paper›PMID 30978228›Full record

SynthesisPloS one2019

Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis.

Robbie C M van Aert, Jelte M Wicherts, Marcel A L M van Assen

Abstract readMeta-Analysis
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in PloS one, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 112 papers, 49 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
112citing papers in PubMed, 49 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

112 citing papers in PubMed, 49 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Effectiveness of surgical interventions for heavy eye syndrome: a systematic review and meta-analysis with meta-regression.Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie · 2025
    Pooled it
  6. Publication Bias in Randomized Controlled Trials of Hypospadias Surgical Repair: A Systematic Review.International journal of urology : official journal of the Japanese Urological Association · 2025
    Pooled it
  7. Pooled it
  8. Pooled it
  9. Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it

52 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Robbie C M van AertDepartment of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands.ORCID 0000-0001-6187-0665
Jelte M WichertsDepartment of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands.
Marcel A L M van AssenDepartment of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (CDSR; representing meta-analyses from medicine). Publication bias was assessed on all homogeneous subsets (3.8% of all subsets of meta-analyses published in Psychological Bulletin) of primary studies included in meta-analyses, because publication bias methods do not have good statistical properties if the true effect size is heterogeneous. Publication bias tests did not reveal evidence for bias in the homogeneous subsets. Overestimation was minimal but statistically significant, providing evidence of publication bias that appeared to be similar in both fields. However, a Monte-Carlo simulation study revealed that the creation of homogeneous subsets resulted in challenging conditions for publication bias methods since the number of effect sizes in a subset was rather small (median number of effect sizes equaled 6). Our findings are in line with, in its most extreme case, publication bias ranging from no bias until only 5% statistically nonsignificant effect sizes being published. These and other findings, in combination with the small percentages of statistically significant primary effect sizes (28.9% and 18.9% for subsets published in Psychological Bulletin and CDSR), led to the conclusion that evidence for publication bias in the studied homogeneous subsets is weak, but suggestive of mild publication bias in both psychology and medicine.

Indexed as

Data Interpretation, StatisticalMedicinePsychologyPublication BiasDatabases, FactualData ManagementHumansMonte Carlo MethodQuality ControlSelection Bias

Identifiers

PMID30978228
PMCPMC6461282

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.