Evidence map›Paper›PMID 34667878›Full record

ArticleLearning health systems2021

A roadmap to operationalize and evaluate impact in a learning health system.

Claire Allen, Katie Coleman, Kayne Mettert, Cara Lewis, Emily Westbrook, Paula Lozano

Abstract read
In one paragraph

Article in Learning health systems, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 54 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
54citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

54 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Article
  4. Article
  5. Article
  6. Factors Influencing Physician Adherence to Venous Thromboembolism Risk Assessment Model Recommendations: A Best-Worst Scaling.American journal of cardiovascular drugs : drugs, devices, and other interventions · 2026
    Article
  7. Article
  8. Article
  9. Article
  10. Review
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Opportunities for the informatics community to advance learning health systems.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

6 authors.

Claire AllenKaiser Permanente Washington Health Research Institute Seattle Washington USA.
Katie ColemanKaiser Permanente Washington Health Research Institute Seattle Washington USA.
Kayne MettertKaiser Permanente Washington Health Research Institute Seattle Washington USA.
Cara LewisKaiser Permanente Washington Health Research Institute Seattle Washington USA.
Emily WestbrookKaiser Permanente Washington Health Research Institute Seattle Washington USA.
Paula LozanoKaiser Permanente Washington Health Research Institute Seattle Washington USA.

Funding

AHRQ HHS K12 HS026369
6 · The paper itself

Abstract

backgroundMany health systems invest in initiatives to accelerate translation of knowledge into practice. However, organizations lack guidance on how to develop and operationalize such Learning Health System (LHS) programs and evaluate their impact. Kaiser Permanente Washington (KPWA) launched our LHS program in June 2017 and developed a logic model as a foundation to evaluate the program's impact.

objectiveTo develop a roadmap for organizations that want to establish an LHS program, understand how LHS core components relate to one another when operationalized in practice, and evaluate and improve their progress.

methodsWe conducted a narrative review on LHS models, key model components, and measurement approaches.

resultsThe KPWA LHS Logic Model provides a broad set of constructs relevant to LHS programs, depicts their relationship to LHS operations, harmonizes terms across models, and offers measurable operationalizations of each construct to guide other health systems. The model identifies essential LHS inputs, provides transparency into LHS activities, and defines key outcomes to evaluate LHS processes and impact. We provide reflections on the most helpful components of the model and identify areas that need further improvement using illustrative examples from deployment of the LHS model during the COVID-19 pandemic.

conclusionThe KPWA LHS Logic Model is a starting point for future LHS implementation research and a practical guide for healthcare organizations that are building, operationalizing, and evaluating LHS initiatives.

Indexed as

implementation sciencelearning health systemprogram evaluationquality improvement

Identifiers

PMID34667878
PMCPMC8512726

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.