Evidence map›Paper›PMID 32720712›Full record

SynthesisBiometrics2020

Testing small study effects in multivariate meta-analysis.

Chuan Hong, Georgia Salanti, Sally C Morton, Richard D Riley, Haitao Chu, Stephen E Kimmel, Yong Chen

Abstract readMeta-Analysis
PubMed Full text at PMC Publisher
In one paragraph

Synthesis in Biometrics, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 40 papers, 25 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
40citing papers in PubMed, 25 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

40 citing papers in PubMed, 25 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Pooled it
  6. Use of PD-1/PD-L1 inhibitors in non-small cell lung cancer in elderly patients: a meta-analysis and systematic review of randomized clinical trials.Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico · 2026
    Pooled it
  7. Pooled it
  8. Pooled it
  9. Effectiveness of unguided web-based interventions for the treatment of depressive symptoms in improving the quality of life: a meta-analysis.Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation · 2025
    Pooled it
  10. Pooled it
  11. Pooled it
  12. Pooled it
  13. Pooled it
  14. Pooled it
  15. Pooled it
  16. Pooled it
  17. Pooled it
  18. Pooled it
  19. Pooled it
  20. Pooled it
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

7 authors.

Chuan HongDepartment of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.ORCID 0000-0001-7056-9559
Georgia SalantiInstitute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.
Sally C MortonDepartment of Statistics, Virginia Tech, Blacksburg, Virginia.
Richard D RileyCentre for Prognosis Research, School of Medicine, Keele University, Staffordshire, UK.
Haitao ChuDivision of Biostatistics, University of Minnesota, Minneapolis, Minnesota.
Stephen E KimmelDepartment of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.
Yong ChenDepartment of Biostatistics, Epidemiology & Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania.ORCID 0000-0003-0835-0788

Funding

Dynamic learning for post-vaccine event prediction using temporal information in VAERSR01AI130460 · NIAID · UNIVERSITY OF TEXAS HLTH SCI CTR HOUSTON · PI CHEN, YONG, TAO, CUI · 2017 to 2021
$3.4M
A General Framework to Account for Outcome Reporting Bias in Systematic ReviewsR01LM012607 · NLM · UNIVERSITY OF PENNSYLVANIA · PI CHEN, YONG · 2017 to 2020
$1.4M
Statistical Methods and Software for Multivariate Meta-analysisR01LM012982 · NLM · UNIVERSITY OF MINNESOTA · PI LIN, LIFENG, SIEGEL, LIANNE · 2019 to 2022
$1.3M
Statistical Methods and Software for Multivariate Meta-analysisR21LM012197 · NLM · UNIVERSITY OF MINNESOTA · PI CHU, HAITAO · 2015 to 2016
$385k
NIAID NIH HHS R01 AI130460NLM NIH HHS R01 LM012607NLM NIH HHS R01 LM012982NLM NIH HHS R21 LM012197Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung 320030 179158
6 · The paper itself

Abstract

Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias, and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection processes can be involved in the reporting of multivariate outcomes. For example, some studies may be completely unpublished while others may selectively report multiple outcomes. In this paper, we propose a score test as an overall test of small study effects in multivariate meta-analysis. Two detailed case studies are given to demonstrate the advantage of the proposed test over various naive applications of univariate tests in practice. Through simulation studies, the proposed test is found to retain nominal Type I error rates with considerable power in moderate sample size settings. Finally, we also evaluate the concordance between the proposed tests with the naive application of univariate tests by evaluating 44 systematic reviews with multiple outcomes from the Cochrane Database.

Indexed as

Research DesignMultivariate AnalysisPublication BiasSample SizeSystematic Reviews as Topiccomparative effectiveness researchcomposite likelihoodoutcome reporting biaspublication biassmall study effectsystematic review

Identifiers

PMID32720712
PMCPMC7736122

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.