Evidence map›Paper›PMID 27340369›Full record

ArticleHealth services & outcomes research methodology

Estimating causal effects: considering three alternatives to difference-in-differences estimation.

Stephen O'Neill, Noémi Kreif, Richard Grieve, Matthew Sutton, Jasjeet S Sekhon

Abstract read
In one paragraph

Article in Health services & outcomes research methodology. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 57 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
57citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

57 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Employment Responses to a Partner's Disability Onset ("Care Shocks"): Do Working Conditions Matter?The journals of gerontology. Series B, Psychological sciences and social sciences · 2025
    Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. How do family doctors respond to reduced waiting times for cancer diagnosis in secondary care?The European journal of health economics : HEPAC : health economics in prevention and care · 2024
    Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

5 authors.

Stephen O'NeillDepartment of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
Noémi KreifDepartment of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
Richard GrieveDepartment of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
Matthew SuttonManchester Centre for Health Economics, Institute of Population Health, The University of Manchester, Manchester, UK.
Jasjeet S SekhonDepartment of Political Science and Department of Statistics, University of California at Berkeley, Berkeley, CA USA.

Funding

Department of Health SRF-2013-06-016Medical Research Council G106/1173Medical Research Council MR/L012332/1
6 · The paper itself

Abstract

Difference-in-differences (DiD) estimators provide unbiased treatment effect estimates when, in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel trends over time. This assumption is implausible in many settings. An alternative assumption is that the potential outcomes are independent of treatment status, conditional on past outcomes. This paper considers three methods that share this assumption: the synthetic control method, a lagged dependent variable (LDV) regression approach, and matching on past outcomes. Our motivating empirical study is an evaluation of a hospital pay-for-performance scheme in England, the best practice tariffs programme. The conclusions of the original DiD analysis are sensitive to the choice of approach. We conduct a Monte Carlo simulation study that investigates these methods' performance. While DiD produces unbiased estimates when the parallel trends assumption holds, the alternative approaches provide less biased estimates of treatment effects when it is violated. In these cases, the LDV approach produces the most efficient and least biased estimates.

Indexed as

Difference-in-differencesMatchingPay-for-performancePolicy evaluationSynthetic control method

Identifiers

PMID27340369
PMCPMC4869762

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.