Evidence map›Paper›PMID 34590684›Full record

ArticleJournal of the American Medical Informatics Association : JAMIA2022

Synergies between centralized and federated approaches to data quality: a report from the national COVID cohort collaborative.

Emily R Pfaff, Andrew T Girvin, Davera L Gabriel, Kristin Kostka, Michele Morris, Matvey B Palchuk, Harold P Lehmann, Benjamin Amor, Mark Bissell, Katie R Bradwell and 79 more

Abstract read
In one paragraph

Article in Journal of the American Medical Informatics Association : JAMIA, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 58 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
58citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

58 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Electronic health record data quality assessment and tools: a systematic review.Journal of the American Medical Informatics Association : JAMIA · 2023
    Pooled it
  3. Article
  4. Article
  5. Differential COVID-19 Outcomes Across Lysosomal Disorders.medRxiv : the preprint server for health sciences · 2026
    Article
  6. Article
  7. Article
  8. Article
  9. Article
  10. Article
  11. Making the case for an International Childhood Cancer Data Partnership.Journal of the National Cancer Institute · 2025
    Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Article
  17. National COVID Cohort Collaborative data enhancements: a path for expanding common data models.Journal of the American Medical Informatics Association : JAMIA · 2025
    Article
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

89 authors.

Emily R PfaffDepartment of Medicine, UNC Chapel Hill School of Medicine, Chapel Hill, North Carolina, USA.ORCID 0000-0002-6840-9756
Andrew T GirvinPalantir Technologies, Denver, Colorado, USA.
Davera L GabrielSection of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
Kristin KostkaThe OHDSI Center at the Roux Institute, Northeastern University, Portland, Maine, USA.
Michele MorrisDepartment of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.
Matvey B PalchukTriNetX LLC, Cambridge, Massachusetts, USA.
Harold P LehmannDepartment of Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA.
Benjamin AmorPalantir Technologies, Denver, Colorado, USA.
Mark BissellPalantir Technologies, Denver, Colorado, USA.
Katie R BradwellPalantir Technologies, Denver, Colorado, USA.
Sigfried GoldSection of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.ORCID 0000-0001-7853-6137
Stephanie S HongSection of Biomedical Informatics and Data Science, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
Johanna LoombaUniversity of Virginia, Charlottesville, Virginia, USA.
Amin MannaPalantir Technologies, Denver, Colorado, USA.
Julie A McMurryCenter for Health AI, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA.
Emily NiehausPalantir Technologies, Denver, Colorado, USA.
Nabeel QureshiPalantir Technologies, Denver, Colorado, USA.
Anita WaldenDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA.
Xiaohan Tanner ZhangJohns Hopkins University School of Medicine, Baltimore, Maryland, USA.
Richard L ZhuJohns Hopkins University School of Medicine, Baltimore, Maryland, USA.ORCID 0000-0002-4289-7632
Richard A MoffittDepartment of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA.ORCID 0000-0003-2723-5902
Melissa A HaendelUniversity of Colorado Anschutz Medical Campus, Aurora, Colorado, USA.ORCID 0000-0001-9114-8737
Christopher G ChuteSchools of Medicine, Public Health, and Nursing, Johns Hopkins University, Baltimore, Maryland, USA.ORCID 0000-0001-5437-2545
N3C Consortium
William G Adams
Shaymaa Al-Shukri
Alfred Anzalone
Ahmad Baghal
Tellen D Bennett
Elmer V Bernstam
Elmer V Bernstam
Mark M Bissell
Brian Bush
Thomas R Campion
Victor Castro
Jack Chang
Deepa D Chaudhari
Wenjin Chen
San Chu
James J Cimino
Keith A Crandall
Mark Crooks
Sara J Deakyne Davies
John DiPalazzo
David Dorr
Dan Eckrich
Sarah E Eltinge
Daniel G Fort
George Golovko
Snehil Gupta
Melissa A Haendel
Janos G Hajagos
David A Hanauer
Brett M Harnett
Ronald Horswell
Nancy Huang
Steven G Johnson
Michael Kahn
Kamil Khanipov
Curtis Kieler
Katherine Ruiz De Luzuriaga
Sarah Maidlow
Ashley Martinez
Jomol Mathew
James C McClay
Gabriel McMahan
Brian Melancon
Stephane Meystre
Lucio Miele
Hiroki Morizono
Ray Pablo
Lav Patel
Jimmy Phuong
Daniel J Popham
Claudia Pulgarin
Carlos Santos
Indra Neil Sarkar
Nancy Sazo
Soko Setoguchi
Selvin Soby
Sirisha Surampalli
Christine Suver
Uma Maheswara Reddy Vangala
Shyam Visweswaran
James von Oehsen
Kellie M Walters
Laura Wiley
David A Williams
Adrian Zai

Funding

Vanderbilt Institute for Clinical and Translational Research (VICTR) -Identifying correlates of functional immunity in SARS-CoV-2 convalescent plasmaUL1TR002243 · NCATS · VANDERBILT UNIVERSITY MEDICAL CENTER · PI Paul A. Harris, Wesley H Self · 2017 to 2026
$130.7M
University of Pittsburgh Clinical and Translational Science InstituteUL1TR001857 · NCATS · UNIVERSITY OF PITTSBURGH AT PITTSBURGH · PI REIS, STEVEN E · 2016 to 2025
$129.3M
UCLA Clinical Translational Science InstituteUL1TR001881 · NCATS · UNIVERSITY OF CALIFORNIA LOS ANGELES · PI ARLEEN F. BROWN, ARASH NAEIM · 2016 to 2026
$118.1M
Clinical and Translational Science InstituteUL1TR001872 · NCATS · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · PI COLLARD, HAROLD R, JACOBY, VANESSA · 2016 to 2025
$112.1M
Project-005UL1TR001445 · NCATS · NEW YORK UNIVERSITY SCHOOL OF MEDICINE · PI BREDELLA, MIRIAM ANTOINETTE, HOCHMAN, JUDITH S · 2015 to 2025
$103.5M
Yale Clinical and Translational Science Award (U Component)UL1TR001863 · NCATS · YALE UNIVERSITY · PI John H. Krystal, LUCILA OHNO-MACHADO · 2016 to 2026
$102.9M
Phenotypic Diversity in COVID-19UL1TR001878 · NCATS · UNIVERSITY OF PENNSYLVANIA · PI FITZGERALD, GARRET A · 2016 to 2025
$102.4M
Transform Dissemination and Implementation Science in CTSA ProgramsUL1TR002319 · NCATS · UNIVERSITY OF WASHINGTON · PI John K. Amory · 2017 to 2026
$100.0M
Clinical and Translational Science AwardUL1TR001873 · NCATS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI REILLY, MUREDACH P · 2016 to 2025
$99.0M
WU INSTITUTE OF CLINICAL AND TRANSLATIONAL SCIENCESUL1TR002345 · NCATS · WASHINGTON UNIVERSITY · PI William G. Powderly · 2017 to 2026
$97.8M
The Harvard Clinical and Translational Science CenterUL1TR002541 · NCATS · HARVARD MEDICAL SCHOOL · PI NADLER, LEE MARSHALL · 2018 to 2022
$93.0M
Implementing a Maternal health and PRegnancy Outcomes Vision for Everyone (IMPROVE)UL1TR002378 · NCATS · EMORY UNIVERSITY · PI Andres J Garcia, Elizabeth O. Ofili · 2017 to 2026
$92.1M
NCATS NIH HHS U24 TR002306NCATS NIH HHS U24TR002306NCATS NIH HHS UL1 TR001412NCATS NIH HHS UL1 TR001414NCATS NIH HHS UL1 TR001420NCATS NIH HHS UL1 TR001422NCATS NIH HHS UL1 TR001425NCATS NIH HHS UL1 TR001433NCATS NIH HHS UL1 TR001436NCATS NIH HHS UL1 TR001439NCATS NIH HHS UL1 TR001442NCATS NIH HHS UL1 TR001445NCATS NIH HHS UL1 TR001449NCATS NIH HHS UL1 TR001450NCATS NIH HHS UL1 TR001453NCATS NIH HHS UL1 TR001855NCATS NIH HHS UL1 TR001857NCATS NIH HHS UL1 TR001860NCATS NIH HHS UL1 TR001863NCATS NIH HHS UL1 TR001866NCATS NIH HHS UL1 TR001872NCATS NIH HHS UL1 TR001873NCATS NIH HHS UL1 TR001876NCATS NIH HHS UL1 TR001878NCATS NIH HHS UL1 TR001881NCATS NIH HHS UL1 TR001998NCATS NIH HHS UL1 TR002001NCATS NIH HHS UL1 TR002003NCATS NIH HHS UL1 TR002014NCATS NIH HHS UL1 TR002240NCATS NIH HHS UL1 TR002243NCATS NIH HHS UL1 TR002319NCATS NIH HHS UL1 TR002345NCATS NIH HHS UL1 TR002366NCATS NIH HHS UL1 TR002369NCATS NIH HHS UL1 TR002373NCATS NIH HHS UL1 TR002377NCATS NIH HHS UL1 TR002378NCATS NIH HHS UL1 TR002384NCATS NIH HHS UL1 TR002389NCATS NIH HHS UL1 TR002489NCATS NIH HHS UL1 TR002494NCATS NIH HHS UL1 TR002529NCATS NIH HHS UL1 TR002535NCATS NIH HHS UL1 TR002537NCATS NIH HHS UL1 TR002538NCATS NIH HHS UL1 TR002541NCATS NIH HHS UL1 TR002544NCATS NIH HHS UL1 TR002550NCATS NIH HHS UL1 TR002553NCATS NIH HHS UL1 TR002556NCATS NIH HHS UL1 TR002645NCATS NIH HHS UL1 TR002649NCATS NIH HHS UL1 TR002733NCATS NIH HHS UL1 TR002736NCATS NIH HHS UL1 TR003015NCATS NIH HHS UL1 TR003017NCATS NIH HHS UL1 TR003096NCATS NIH HHS UL1 TR003098NCATS NIH HHS UL1 TR003107NCATS NIH HHS UL1 TR003142NCATS NIH HHS UL1 TR003167NCATS NIH HHS UL1 TR004419NIDDK NIH HHS K23 DK059311NIGMS NIH HHS U54 GM104938NIGMS NIH HHS U54 GM104940NIGMS NIH HHS U54 GM104941NIGMS NIH HHS U54 GM104942NIGMS NIH HHS U54 GM115428NIGMS NIH HHS U54 GM115458NIGMS NIH HHS U54 GM115516NIGMS NIH HHS U54 GM115677
6 · The paper itself

Abstract

objectiveIn response to COVID-19, the informatics community united to aggregate as much clinical data as possible to characterize this new disease and reduce its impact through collaborative analytics. The National COVID Cohort Collaborative (N3C) is now the largest publicly available HIPAA limited dataset in US history with over 6.4 million patients and is a testament to a partnership of over 100 organizations. MATERIALS AND

methodsWe developed a pipeline for ingesting, harmonizing, and centralizing data from 56 contributing data partners using 4 federated Common Data Models. N3C data quality (DQ) review involves both automated and manual procedures. In the process, several DQ heuristics were discovered in our centralized context, both within the pipeline and during downstream project-based analysis. Feedback to the sites led to many local and centralized DQ improvements.

resultsBeyond well-recognized DQ findings, we discovered 15 heuristics relating to source Common Data Model conformance, demographics, COVID tests, conditions, encounters, measurements, observations, coding completeness, and fitness for use. Of 56 sites, 37 sites (66%) demonstrated issues through these heuristics. These 37 sites demonstrated improvement after receiving feedback. DISCUSSION: We encountered site-to-site differences in DQ which would have been challenging to discover using federated checks alone. We have demonstrated that centralized DQ benchmarking reveals unique opportunities for DQ improvement that will support improved research analytics locally and in aggregate.

conclusionBy combining rapid, continual assessment of DQ with a large volume of multisite data, it is possible to support more nuanced scientific questions with the scale and rigor that they require.

Indexed as

COVID-19Cohort StudiesData AccuracyHealth Insurance Portability and Accountability ActHumansUnited StatesCOVID-19data accuracyelectronic health records

Identifiers

PMID34590684
PMCPMC8500110

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.