Evidence map›Paper›PMID 37207016›Full record

ArticleHealth services & outcomes research methodology2023

A tutorial comparing different covariate balancing methods with an application evaluating the causal effects of substance use treatment programs for adolescents.

Andreas Markoulidakis, Khadijeh Taiyari, Peter Holmans, Philip Pallmann, Monica Busse, Mark D Godley, Beth Ann Griffin

Open access · hybridAbstract read
In one paragraph

Article in Health services & outcomes research methodology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers.

0numbers the graph read from it
0cells of the map it votes in
19citing papers in PubMed
7.4field-weighted citation impact, top 2% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

19 citing papers in PubMed, 33 citations in OpenAlex.

  1. Trial
  2. Observational
  3. Observational
  4. Article
  5. Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Assessing Covariate Balance with Small Sample Sizes.medRxiv : the preprint server for health sciences · 2025
    Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors at 3 institutions in 2 countries.

Andreas MarkoulidakisCentre for Trials Research, Cardiff University, Cardiff, Wales UK.ORCID 0000-0002-4595-0582
Khadijeh TaiyariSchool of Medicine, Cardiff University, Cardiff, Wales UK.
Peter HolmansDivision of Psychological Medicine and Clinical Neurosciences, School of Medicine, Cardiff University, Cardiff, Wales UK.
Philip PallmannSchool of Medicine, Cardiff University, Cardiff, Wales UK.
Monica BusseSchool of Medicine, Cardiff University, Cardiff, Wales UK.
Mark D GodleyChestnut Health Systems, Normal, IL USA.
Beth Ann GriffinRAND Corporation, Arlington, VA USA.
Cardiff University · GBChestnut Health Systems · USRAND Corporation · US

Funding

Improving Causal Inference Tools for Addiction ResearchersR01DA045049 · NIDA · RAND CORPORATION · PI GRIFFIN, BETH ANN, MCCAFFREY, DANIEL F · 2018 to 2020
$2.4M
Medical Research Council MR/L010305/1NIDA NIH HHS R01 DA045049
6 · The paper itself

Abstract

Randomized controlled trials are the gold standard for measuring causal effects. However, they are often not always feasible, and causal treatment effects must be estimated from observational data. Observational studies do not allow robust conclusions about causal relationships unless statistical techniques account for the imbalance of pretreatment confounders across groups and key assumptions hold. Propensity score and balance weighting (PSBW) are useful techniques that aim to reduce the observed imbalances between treatment groups by weighting the groups to look alike on the observed confounders. Notably, there are many methods available to estimate PSBW. However, it is unclear a priori which will achieve the best trade-off between covariate balance and effective sample size for a given application. Moreover, it is critical to assess the validity of key assumptions required for robust estimation of the needed treatment effects, including the overlap and no unmeasured confounding assumptions. We present a step-by-step guide to the use of PSBW for estimation of causal treatment effects that includes steps on how to evaluate overlap before the analysis, obtain estimates of PSBW using multiple methods and select the optimal one, check for covariate balance on multiple metrics, and assess sensitivity of findings (both the estimated treatment effect and statistical significance) to unobserved confounding. We illustrate the key steps using a case study examining the relative effectiveness of substance use treatment programs and provide a user-friendly Shiny application that can implement the proposed steps for any application with binary treatments.

Indexed as

Balancing weightsCausal treatment effectPropensity scoreSensitivity analysisUnmeasured confounding

Identifiers

PMID37207016
PMCPMC10188586
OpenAlexW3161326545

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.