ArticleBMC medical research methodology2023
Data quality considerations for evaluating COVID-19 treatments using real world data: learnings from the National COVID Cohort Collaborative (N3C).
Article in BMC medical research methodology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 16 papers, 1 of them a synthesis that pooled it.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
16 citing papers in PubMed, 1 synthesis or guideline pooled it.
- Electronic health record data quality assessment and tools: a systematic review.Journal of the American Medical Informatics Association : JAMIA · 2023Pooled it
- Augmenting structured diagnoses through effective use of pre-trained large language models on clinical notes.JAMIA open · 2026Article
- A mixed-methods study comparing clinical and patient-reported perspectives on long COVID: insights from electronic health records and narrative journal entries.BMC infectious diseases · 2026Observational
- Long COVID Persistence and Surveillance Gaps Across 58 US Hospitals.JAMA network open · 2026Article
- Beyond Missingness: Systematizing Methods for Comprehensive Data Fitness Assessment in Clinical Research.Journal of medical Internet research · 2026Article
- Data science for pediatric infectious disease: utilizing COVID-19 as a model.Current opinion in infectious diseases · 2025Review
- Increasing the Utility of Real-World Data to Inform Public Health Decision Making Through a US-based Private-Public Partnership: 10 Lessons Learned from a Principled Approach to Rapid Pandemic RWE Generation.Therapeutic innovation & regulatory science · 2025Article
- The health data utility and the resurgence of health information exchanges as a national resource.Journal of the American Medical Informatics Association : JAMIA · 2025Article
- Data linkage multiplies research insights across diverse healthcare sectors.Communications medicine · 2025Review
- Article
- Coronavirus Disease 2019 (COVID-19) Real World Data Infrastructure: A Big-Data Resource for Study of the Impact of COVID-19 in Patient Populations With Immunocompromising Conditions.Open forum infectious diseases · 2025Article
- Long-Term Mortality Following SARS-CoV-2 Infection in Rural Versus Urban Dwellers With Autoimmune or Inflammatory Rheumatic Disease: A Retrospective Cohort Analysis From the National COVID Cohort Collaborative.Arthritis care & research · 2025Article
- Building a collaborative ecosystem across the IDeA-CTR networks in response to a public health emergency.Journal of clinical and translational science · 2025Article
- Influence of Prior SARS-CoV-2 Infection on COVID-19 Severity: Evidence from the National COVID Cohort Collaborative.medRxiv : the preprint server for health sciences · 2024Article
- Using National COVID cohort collaborative (N3C) data to explore impact of COVID-19 infection on kidney function in patients receiving lithium therapy.AMIA ... Annual Symposium proceedings. AMIA Symposium · 2024Article
- Identification and Prediction of Clinical Phenotypes in Hospitalized Patients With COVID-19: Machine Learning From Medical Records.JMIR formative research · 2023Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
11 authors.
Funding
Abstract
backgroundMulti-institution electronic health records (EHR) are a rich source of real world data (RWD) for generating real world evidence (RWE) regarding the utilization, benefits and harms of medical interventions. They provide access to clinical data from large pooled patient populations in addition to laboratory measurements unavailable in insurance claims-based data. However, secondary use of these data for research requires specialized knowledge and careful evaluation of data quality and completeness. We discuss data quality assessments undertaken during the conduct of prep-to-research, focusing on the investigation of treatment safety and effectiveness.
methodsUsing the National COVID Cohort Collaborative (N3C) enclave, we defined a patient population using criteria typical in non-interventional inpatient drug effectiveness studies. We present the challenges encountered when constructing this dataset, beginning with an examination of data quality across data partners. We then discuss the methods and best practices used to operationalize several important study elements: exposure to treatment, baseline health comorbidities, and key outcomes of interest.
resultsWe share our experiences and lessons learned when working with heterogeneous EHR data from over 65 healthcare institutions and 4 common data models. We discuss six key areas of data variability and quality. (1) The specific EHR data elements captured from a site can vary depending on source data model and practice. (2) Data missingness remains a significant issue. (3) Drug exposures can be recorded at different levels and may not contain route of administration or dosage information. (4) Reconstruction of continuous drug exposure intervals may not always be possible. (5) EHR discontinuity is a major concern for capturing history of prior treatment and comorbidities. Lastly, (6) access to EHR data alone limits the potential outcomes which can be used in studies.
conclusionsThe creation of large scale centralized multi-site EHR databases such as N3C enables a wide range of research aimed at better understanding treatments and health impacts of many conditions including COVID-19. As with all observational research, it is important that research teams engage with appropriate domain experts to understand the data in order to define research questions that are both clinically important and feasible to address using these real world data.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.