Evidence map›Paper›PMID 28339559›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2017

Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures.

Jill Boylston Herndon, Krishna Aravamudhan, Ronald L Stephenson, Ryan Brandon, Jesley Ruff, Frank Catalanotto, Huong Le

Abstract readValidation Study
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2017. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 5 papers.

0numbers the graph read from it
0cells of the map it votes in
5citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

5 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Article
  5. Assessing the validity of existing dental sealant quality measures.Journal of the American Dental Association (1939) · 2018
    Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors.

Jill Boylston HerndonKey Analytics and Consulting, LLC, Gainesville, Florida, USA (Dr Herndon was with the Department of Health Outcomes and Policy, University of Florida College of Medicine, Gainesville, Florida, USA, when this study was conducted).
Krishna AravamudhanDental Quality Alliance, American Dental Association, Chicago, Illinois, USA.
Ronald L StephensonStephenson Developers Group, Clearwater, Florida, USA.
Ryan BrandonExan Group, Vancouver, Canada.
Jesley RuffAmerican Dental Partners, Inc., Wakefield, Massachusetts, USA.
Frank CatalanottoDepartment of Community Dentistry and Behavioral Science, University of Florida College of Dentistry, Gainesville, Florida, USA.
Huong LeAsian Health Services, Oakland, California and National Network for Oral Health Access, Denver, Colorado, USA.

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

objectiveTo describe the stakeholder-engaged processes used to develop, specify, and validate 2 oral health care electronic clinical quality measures. MATERIALS AND

methodsA broad range of stakeholders were engaged from conception through testing to develop measures and test feasibility, reliability, and validity following National Quality Forum guidance. We assessed data element feasibility through semistructured interviews with key stakeholders using a National Quality Forum-recommended scorecard. We created test datasets of synthetic patients to test measure implementation feasibility and reliability within and across electronic health record (EHR) systems. We validated implementation with automated reporting of EHR clinical data against manual record reviews, using the kappa statistic.

resultsA stakeholder workgroup was formed and guided all development and testing processes. All critical data elements passed feasibility testing. Four test datasets, representing 577 synthetic patients, were developed and implemented within EHR vendors' software, demonstrating measure implementation feasibility. Measure reliability and validity were established through implementation at clinical practice sites, with kappa statistic values in the "almost perfect" agreement range of 0.80-0.99 for all but 1 measure component, which demonstrated "substantial" agreement. The 2 validated measures were published in the United States Health Information Knowledgebase.

conclusionThe stakeholder-engaged processes used in this study facilitated a successful measure development and testing cycle. Engaging stakeholders early and throughout development and testing promotes early identification of and attention to potential threats to feasibility, reliability, and validity, thereby averting significant resource investments that are unlikely to be fruitful.

Indexed as

Electronic Health RecordsMeaningful UseQuality Indicators, Health CareAdolescentChildChild, PreschoolDatasets as TopicDental CariesEvidence-Based DentistryHumansPediatric DentistryUnited StatesYoung Adultelectronic health recordshealth care quality indicatorsmeaningful usequality measure developmentquality measure testing

Identifiers

PMID28339559
PMCPMC7651948

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.