Evidence map›Paper›PMID 33573595›Full record

ReviewBMC medical research methodology2021

Conceptualising natural and quasi experiments in public health.

Frank de Vocht, Srinivasa Vittal Katikireddi, Cheryl McQuire, Kate Tilling, Matthew Hickman, Peter Craig

Abstract readReview
PubMed Full text at PMC Publisher
In one paragraph

Review in BMC medical research methodology, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 64 papers, 4 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
64citing papers in PubMed, 4 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

64 citing papers in PubMed, 4 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Pooled it
  5. Trial
  6. Trial
  7. Risk of bicycle collisions and 'safety in numbers': a natural experiment using the local introduction of e-scooters in England.Injury prevention : journal of the International Society for Child and Adolescent Injury Prevention · 2026
    Article
  8. Article
  9. Review
  10. Article
  11. Article
  12. Article
  13. Observational
  14. Article
  15. Review
  16. Article
  17. Article
  18. Review
  19. Article
  20. Article

4 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Frank de VochtPopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK. frank.devocht@bristol.ac.uk.ORCID 0000-0003-3631-627X
Srinivasa Vittal KatikireddiMRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Bristol, UK.
Cheryl McQuirePopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
Kate TillingPopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
Matthew HickmanPopulation Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK.
Peter CraigMRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Bristol, UK.

Funding

Chief Scientist Office SCAF/15/02Chief Scientist Office SPHSU13Chief Scientist Office SPHSU17Medical Research Council MC_UU_00011/3Medical Research Council MC_UU_00022/2Medical Research Council MC_UU_12017/13School for Public Health Research SPHR-PROG-PCA-WP3
6 · The paper itself

Abstract

backgroundNatural or quasi experiments are appealing for public health research because they enable the evaluation of events or interventions that are difficult or impossible to manipulate experimentally, such as many policy and health system reforms. However, there remains ambiguity in the literature about their definition and how they differ from randomized controlled experiments and from other observational designs. We conceptualise natural experiments in the context of public health evaluations and align the study design to the Target Trial Framework.

methodsA literature search was conducted, and key methodological papers were used to develop this work. Peer-reviewed papers were supplemented by grey literature.

resultsNatural experiment studies (NES) combine features of experiments and non-experiments. They differ from planned experiments, such as randomized controlled trials, in that exposure allocation is not controlled by researchers. They differ from other observational designs in that they evaluate the impact of events or process that leads to differences in exposure. As a result they are, in theory, less susceptible to bias than other observational study designs. Importantly, causal inference relies heavily on the assumption that exposure allocation can be considered 'as-if randomized'. The target trial framework provides a systematic basis for evaluating this assumption and the other design elements that underpin the causal claims that can be made from NES.

conclusionsNES should be considered a type of study design rather than a set of tools for analyses of non-randomized interventions. Alignment of NES to the Target Trial framework will clarify the strength of evidence underpinning claims about the effectiveness of public health interventions.

Indexed as

Public HealthResearch DesignBiasCausalityHumansEvaluationsNatural experimentsPublic healthPublic health policyQuasi experiments

Identifiers

PMID33573595
PMCPMC7879679

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.