Evidence map›Paper›PMID 34260087›Full record

ArticleClinical pharmacology and therapeutics2022

Prevalence of Avoidable and Bias-Inflicting Methodological Pitfalls in Real-World Studies of Medication Safety and Effectiveness.

Katsiaryna Bykov, Elisabetta Patorno, Elvira D'Andrea, Mengdong He, Hemin Lee, Jennifer S Graff, Jessica M Franklin

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Clinical pharmacology and therapeutics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers, 3 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
20citing papers in PubMed, 3 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

20 citing papers in PubMed, 3 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Pooled it
  4. Observational
  5. Article
  6. Article
  7. APPRAISE: A Tool for Appraising Potential for Bias in Real-World Evidence Studies on Medication Effectiveness or Safety.Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research · 2025
    Article
  8. Review
  9. Review
  10. The EU project Real4Reg: unlocking real-world data with AI.Health research policy and systems · 2025
    Article
  11. Article
  12. Review
  13. Article
  14. Article
  15. Review
  16. Review
  17. Article
  18. A causal roadmap for generating high-quality real-world evidence.Journal of clinical and translational science · 2023
    Review
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Katsiaryna BykovDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
Elisabetta PatornoDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
Elvira D'AndreaDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
Mengdong HeDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
Hemin LeeDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
Jennifer S GraffNational Pharmaceutical Council, Washington, DC, USA.
Jessica M FranklinDivision of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.

Funding

RESEARCH TRAINING IN PEDIATRIC EMERGENCY MEDICINET32HD040128 · NICHD · CHILDREN'S HOSPITAL BOSTON · PI Kenneth D. Mandl · 2001 to 2026
$5.0M
Novel Approaches to Monitor the Safety and Effectiveness of Newly Marketed Diabetes Medications in Older Adults Considering Frailty and MultimorbidityK08AG055670 · NIA · BRIGHAM AND WOMEN'S HOSPITAL · PI PATORNO, ELISABETTA · 2018 to 2021
$661k
National Pharmaceutical CouncilNIA NIH HHS K08 AG055670NICHD NIH HHS T32 HD040128
6 · The paper itself

Abstract

Many real-word evidence (RWE) studies that utilize existing healthcare data to evaluate treatment effects incur substantial but avoidable bias from methodologically flawed study design; however, the extent of preventable methodological pitfalls in current RWE is unknown. To characterize the prevalence of avoidable methodological pitfalls with potential for bias in published claims-based studies of medication safety or effectiveness, we conducted an English-language search of PubMed for articles published from January 1, 2010 to May 20, 2019 and randomly selected 75 studies (10 case-control and 65 cohort studies) that evaluated safety or effectiveness of cardiovascular, diabetes, or osteoporosis medications using US health insurance claims. General and methodological study characteristics were extracted independently by two reviewers, and potential for bias was assessed across nine bias domains. Nearly all studies (95%) had at least one avoidable methodological issue known to incur bias, and 81% had potentially at least one of the four issues considered major due to their potential to undermine study validity: time-related bias (57%), potential for depletion of outcome-susceptible individuals (44%), inappropriate adjustment for postbaseline variables (41%), or potential for reverse causation (39%). The median number of major issues per study was 2 (interquartile range (IQR), 1-3) and was lower in cohort studies with a new-user, active-comparator design (median 1, IQR 0-1) than in cohort studies of prevalent users with a nonuser comparator (median 3, IQR 3-4). Recognizing and avoiding known methodological study design pitfalls could substantially improve the utility of RWE and confidence in its validity.

Indexed as

BiasCase-Control StudiesCohort StudiesData AnalysisDatabases, FactualData MiningDrug-Related Side Effects and Adverse ReactionsHumansInsurance Claim ReviewMethodsPrevalenceResearch Design

Identifiers

PMID34260087
PMCPMC8678198

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.