Evidence map›Paper›PMID 32827027›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2020

Large-scale evidence generation and evaluation across a network of databases (LEGEND): assessing validity using hypertension as a case study.

Martijn J Schuemie, Patrick B Ryan, Nicole Pratt, RuiJun Chen, Seng Chan You, Harlan M Krumholz, David Madigan, George Hripcsak, Marc A Suchard

Erratum issuedAbstract readComparative StudyValidation Study
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 21 papers.

0numbers the graph read from it
0cells of the map it votes in
21citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

21 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Application of a digital quality measure for cancer diagnosis in Epic Cosmos.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Data Consult Service: Can we use observational data to address immediate clinical needs?Journal of the American Medical Informatics Association : JAMIA · 2021
    Article
  17. Increasing trust in real-world evidence through evaluation of observational data quality.Journal of the American Medical Informatics Association : JAMIA · 2021
    Article
  18. Article
  19. Observational
  20. Article
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

9 authors.

Martijn J SchuemieEpidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA.
Patrick B RyanEpidemiology Analytics, Janssen Research and Development, Titusville, New Jersey, USA.
Nicole PrattQuality Use of Medicines and Pharmacy Research Centre, University of South Australia, Adelaide, Australia.
RuiJun ChenDepartment of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA.
Seng Chan YouDepartment of Biomedical Informatics, Ajou University School of Medicine, Suwon, Korea.
Harlan M KrumholzDepartment of Medicine, Yale University School of Medicine, New Haven, Connecticut, USA.
David MadiganDepartment of Statistics, Columbia University, New York, New York, USA.
George HripcsakDepartment of Biomedical Informatics, Columbia University Medical Center, New York, New York, USA.
Marc A SuchardDepartment of Biostatistics, Fielding School of Public Health, University of California, Los Angeles, Los Angeles, California, USA.

Funding

Yale Clinical and Translational Science Award (U Component)UL1TR001863 · NCATS · YALE UNIVERSITY · PI John H. Krystal, LUCILA OHNO-MACHADO · 2016 to 2026
$102.9M
DISCOVERING AND APPLYING KNOWLEDGE IN CLINICAL DATABASESR01LM006910 · NLM · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI HRIPCSAK, GEORGE M · 2000 to 2023
$10.6M
NCATS NIH HHS UL1 TR001863NLM NIH HHS R01 LM006910
6 · The paper itself

Abstract

objectivesTo demonstrate the application of the Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) principles described in our companion article to hypertension treatments and assess internal and external validity of the generated evidence. MATERIALS AND

methodsLEGEND defines a process for high-quality observational research based on 10 guiding principles. We demonstrate how this process, here implemented through large-scale propensity score modeling, negative and positive control questions, empirical calibration, and full transparency, can be applied to compare antihypertensive drug therapies. We assess internal validity through covariate balance, confidence-interval coverage, between-database heterogeneity, and transitivity of results. We assess external validity through comparison to direct meta-analyses of randomized controlled trials (RCTs).

resultsFrom 21.6 million unique antihypertensive new users, we generate 6 076 775 effect size estimates for 699 872 research questions on 12 946 treatment comparisons. Through propensity score matching, we achieve balance on all baseline patient characteristics for 75% of estimates, observe 95.7% coverage in our effect-estimate 95% confidence intervals, find high between-database consistency, and achieve transitivity in 84.8% of triplet hypotheses. Compared with meta-analyses of RCTs, our results are consistent with 28 of 30 comparisons while providing narrower confidence intervals.

conclusionWe find that these LEGEND results show high internal validity and are congruent with meta-analyses of RCTs. For these reasons we believe that evidence generated by LEGEND is of high quality and can inform medical decision-making where evidence is currently lacking. Subsequent publications will explore the clinical interpretations of this evidence.

Indexed as

Databases, FactualMeta-Analysis as TopicAntihypertensive AgentsComputer Communication NetworksConfidence IntervalsHumansHypertensionObservationPropensity ScoreRandomized Controlled Trials as TopicTreatment OutcomeAntihypertensive Agentsempirical calibrationhypertensionobservational studiesopen sciencetreatment effects

Identifiers

PMID32827027
PMCPMC7481033

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.