Evidence map›Paper›PMID 33826756›Full record

ReviewClinical pharmacology and therapeutics2022

When Can Nonrandomized Studies Support Valid Inference Regarding Effectiveness or Safety of New Medical Treatments?

Jessica M Franklin, Richard Platt, Nancy A Dreyer, Alex John London, Gregory E Simon, Jonathan H Watanabe, Michael Horberg, Adrian Hernandez, Robert M Califf

Abstract readReview
In one paragraph

Review in Clinical pharmacology and therapeutics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 19 papers.

0numbers the graph read from it
0cells of the map it votes in
19citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

19 citing papers in PubMed.

  1. Article
  2. Article
  3. Review
  4. Article
  5. Article
  6. Review
  7. Observational
  8. Article
  9. Partnering with Family Organizations in Research: Experiences from the Field.International journal of environmental research and public health · 2025
    Article
  10. Article
  11. Article
  12. Review
  13. Article
  14. Article
  15. Article
  16. An Overview of Phase 2 Clinical Trial Designs.International journal of radiation oncology, biology, physics · 2022
    Review
  17. Article
  18. Review
  19. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Jessica M FranklinOptum Epidemiology, Boston, Massachusetts, USA.
Richard PlattHarvard Pilgrim Health Care Institute, Harvard Medical School, Boston, Massachusetts, USA.
Nancy A DreyerIQVIA Real World Solutions, Cambridge, Massachusetts, USA.
Alex John LondonPhilosophy Department & Center for Ethics and Policy, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA.
Gregory E SimonKaiser Permanente Washington Health Research Institute, Seattle, Washington, USA.
Jonathan H WatanabeSchool of Pharmacy and Pharmaceutical Sciences, University of California Irvine, Irvine, California, USA.
Michael HorbergKaiser Permanente Mid-Atlantic Permanente Research Institute and Mid-Atlantic Permanente Medical Group, Bethesda, Maryland, USA.
Adrian HernandezDuke Clinical Research Institute, Durham, North Carolina, USA.
Robert M CaliffVerily Life Sciences and Google Health, Cambridge, Massachusetts, USA.

Funding

Randomized Cardiovascular Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology (RCT DUPLICATE)R01HL141505 · NHLBI · BRIGHAM AND WOMEN'S HOSPITAL · PI SCHNEEWEISS, SEBASTIAN G., WANG, SHIRLEY · 2019 to 2023
$3.6M
NHLBI NIH HHS R01 HL141505
6 · The paper itself

Abstract

The randomized controlled trial (RCT) is the gold standard for evaluating the causal effects of medications. Limitations of RCTs have led to increasing interest in using real-world evidence (RWE) to augment RCT evidence and inform decision making on medications. Although RWE can be either randomized or nonrandomized, nonrandomized RWE can capitalize on the recent proliferation of large healthcare databases and can often answer questions that cannot be answered in randomized studies due to resource constraints. However, the results of nonrandomized studies are much more likely to be impacted by confounding bias, and the existence of unmeasured confounders can never be completely ruled out. Furthermore, nonrandomized studies require more complex design considerations which can sometimes result in design-related biases. We discuss questions that can help investigators or evidence consumers evaluate the potential impact of confounding or other biases on their findings: Does the design emulate a hypothetical randomized trial design? Is the comparator or control condition appropriate? Does the primary analysis adjust for measured confounders? Do sensitivity analyses quantify the potential impact of residual confounding? Are methods open to inspection and (if possible) replication? Designing a high-quality nonrandomized study of medications remains challenging and requires broad expertise across a range of disciplines, including relevant clinical areas, epidemiology, and biostatistics. The questions posed in this paper provide a guiding framework for assessing the credibility of nonrandomized RWE and could be applied across many clinical questions.

Indexed as

BiasConfounding Factors, EpidemiologicData AnalysisEvidence-Based MedicineHumansNon-Randomized Controlled Trials as TopicTherapeutics

Identifiers

PMID33826756
PMCPMC9291272

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.