Evidence map›Paper›PMID 36978059›Full record

SynthesisBMC medicine2023

The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses.

Ziyu Liu, Fahad M Al Amer, Mengli Xiao, Chang Xu, Luis Furuya-Kanamori, Hwanhee Hong, Lianne Siegel, Lifeng Lin

Open access · goldAbstract readMeta-Analysis
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in BMC medicine, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 11 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
11citing papers in PubMed, 3 pooled it
4.2field-weighted citation impact, top 6% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

11 citing papers in PubMed, 3 syntheses or guidelines pooled it, 16 citations in OpenAlex.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Review
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors at 8 institutions in 4 countries.

Ziyu LiuDepartment of Statistics, Florida State University, Tallahassee, FL, USA.
Fahad M Al AmerDepartment of Mathematics, College of Science and Arts, Najran University, Najran, Saudi Arabia.
Mengli XiaoDepartment of Biostatistics and Informatics, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
Chang XuMinistry of Education Key Laboratory for Population Health Across-Life Cycle & Anhui Provincial Key Laboratory of Population Health and Aristogenics, Anhui Medical University, Anhui, China.
Luis Furuya-KanamoriUQ Centre for Clinical Research, Faculty of Medicine, University of Queensland, Herston, Australia.
Hwanhee HongDepartment of Biostatistics and Bioinformatics, School of Medicine, Duke University, Durham, NC, USA.
Lianne SiegelDivision of Biostatistics, University of Minnesota School of Public Health, Minneapolis, MN, USA.
Lifeng LinDepartment of Epidemiology and Biostatistics, Mel and Enid Zuckerman College of Public Health, University of Arizona, Tucson, AZ, USA. lifenglin@arizona.edu.ORCID 0000-0002-3562-9816
Anhui Medical University · CNDuke University · USFlorida State University · USNajran University · SAThe University of Queensland · AUUniversity of Arizona · USUniversity of Colorado Anschutz Medical Campus · USUniversity of Minnesota · US

Funding

Statistical Methods and Software for Multivariate Meta-analysisR01LM012982 · NLM · UNIVERSITY OF MINNESOTA · PI LIN, LIFENG, SIEGEL, LIANNE · 2019 to 2022
$1.3M
Joint modeling of continuous and binary data in meta-analysisR03MH128727 · NIMH · UNIVERSITY OF ARIZONA · PI LIN, LIFENG · 2022 to 2023
$146k
NIMH NIH HHS R03 MH128727NLM NIH HHS R01 LM012982
6 · The paper itself

Abstract

backgroundStudies included in a meta-analysis are often heterogeneous. The traditional random-effects models assume their true effects to follow a normal distribution, while it is unclear if this critical assumption is practical. Violations of this between-study normality assumption could lead to problematic meta-analytical conclusions. We aimed to empirically examine if this assumption is valid in published meta-analyses.

methodsIn this cross-sectional study, we collected meta-analyses available in the Cochrane Library with at least 10 studies and with between-study variance estimates > 0. For each extracted meta-analysis, we performed the Shapiro-Wilk (SW) test to quantitatively assess the between-study normality assumption. For binary outcomes, we assessed between-study normality for odds ratios (ORs), relative risks (RRs), and risk differences (RDs). Subgroup analyses based on sample sizes and event rates were used to rule out the potential confounders. In addition, we obtained the quantile-quantile (Q-Q) plot of study-specific standardized residuals for visually assessing between-study normality.

resultsBased on 4234 eligible meta-analyses with binary outcomes and 3433 with non-binary outcomes, the proportion of meta-analyses that had statistically significant non-normality varied from 15.1 to 26.2%. RDs and non-binary outcomes led to more frequent non-normality issues than ORs and RRs. For binary outcomes, the between-study non-normality was more frequently found in meta-analyses with larger sample sizes and event rates away from 0 and 100%. The agreements of assessing the normality between two independent researchers based on Q-Q plots were fair or moderate.

conclusionsThe between-study normality assumption is commonly violated in Cochrane meta-analyses. This assumption should be routinely assessed when performing a meta-analysis. When it may not hold, alternative meta-analysis methods that do not make this assumption should be considered.

Indexed as

Cross-Sectional StudiesHumansOdds RatioSample SizeCochrane LibraryEffect measureHeterogeneityMeta-analysisNormality assumptionQ–Q plot

Identifiers

PMID36978059
PMCPMC10053115
OpenAlexW4361217902

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.