Evidence map›Paper›PMID 36162166›Full record

Observational studyInternational journal of medical informatics2022

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records.

Iacopo Vagliano, Martijn C Schut, Ameen Abu-Hanna, Dave A Dongelmans, Dylan W de Lange, Diederik Gommers, Olaf L Cremer, Rob J Bosman, Sander Rigter, Evert-Jan Wils and 40 more

Abstract readObservational Study
In one paragraph

Observational study in International journal of medical informatics, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.

0numbers the graph read from it
0cells of the map it votes in
2citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

2 citing papers in PubMed.

  1. Article
  2. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

50 authors.

Iacopo VaglianoDepartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands. Electronic address: i.vagliano@amsterdamumc.nl.
Martijn C SchutDepartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
Ameen Abu-HannaDepartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
Dave A DongelmansNational Intensive Care Evaluation (NICE) foundation, Amsterdam, The Netherlands; Department of Intensive Care Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands.
Dylan W de LangeNational Intensive Care Evaluation (NICE) foundation, Amsterdam, The Netherlands; Department of Intensive Care Medicine, University Medical Center Utrecht, University Utrecht, Utrecht, The Netherlands.
Diederik GommersDepartment of Intensive Care, Erasmus Medical Center, Rotterdam, The Netherlands.
Olaf L CremerIntensive Care, UMC Utrecht, Utrecht, The Netherlands.
Rob J BosmanICU, OLVG, Amsterdam, The Netherlands.
Sander RigterDepartment of Anesthesiology and Intensive Care, St. Antonius Hospital, Nieuwegein, The Netherlands.
Evert-Jan WilsDepartment of Intensive Care, Franciscus Gasthuis & Vlietland, Rotterdam, The Netherlands.
Tim FrenzelDepartment of Intensive Care Medicine, Radboud University Medical Center, Nijmegen, The Netherlands.
Remko de JongIntensive Care, Bovenij Ziekenhuis, Amsterdam, The Netherlands.
Marco A A PetersIntensive Care, Canisius Wilhelmina Ziekenhuis, Nijmegen, The Netherlands.
Marlijn J A KampsIntensive Care, Catharina Ziekenhuis Eindhoven, Eindhoven, The Netherlands.
Dharmanand RamnarainDepartment of Intensive Care, ETZ Tilburg, Tilburg, The Netherlands.
Ralph NowitzkyIntensive Care, Haga Ziekenhuis, Den Haag, The Netherlands.
Fleur G C A NooteboomIntensive Care, Laurentius Ziekenhuis, Roermond, The Netherlands.
Wouter de RuijterDepartment of Intensive Care Medicine, Northwest Clinics, Alkmaar, The Netherlands.
Louise C Urlings-StropIntensive Care, Reinier de Graaf Gasthuis, Delft, The Netherlands.
Ellen G M SmitIntensive Care, Spaarne Gasthuis, Haarlem en Hoofddorp, The Netherlands.
D Jannet Mehagnoul-SchipperIntensive Care, VieCuri Medisch Centrum, Venlo, The Netherlands.
Tom DormansIntensive care, Zuyderland MC, Heerlen, The Netherlands.
Cornelis P C de JagerDepartment of Intensive Care, Jeroen Bosch Ziekenhuis, Den Bosch, The Netherlands.
Stefaan H A HendriksIntensive Care, Albert Schweitzerziekenhuis, Dordrecht, The Netherlands.
Sefanja AchterbergICU, Haaglanden Medisch Centrum, Den Haag, The Netherlands.
Evelien OostdijkICU, Maasstad Ziekenhuis Rotterdam, Rotterdam, The Netherlands.
Auke C ReidingaICU, SEH, BWC, Martiniziekenhuis, Groningen, The Netherlands.
Barbara Festen-SpanjerIntensive Care, Ziekenhuis Gelderse Vallei, Ede, The Netherlands.
Gert B BrunnekreefDepartment of Intensive Care, Ziekenhuisgroep Twente, Almelo, The Netherlands.
Alexander D CornetDepartment of Intensive Care, Medisch Spectrum Twente, Enschede, The Netherlands.
Walter van den TempelDepartment of Intensive Care, Ikazia Ziekenhuis Rotterdam, Rotterdam, The Netherlands.
Age D BoelensAnesthesiology, Antonius Ziekenhuis Sneek, Sneek, The Netherlands.
Peter KoetsierIntensive Care, Medisch Centrum Leeuwarden, Leeuwarden, The Netherlands.
Judith LensICU, IJsselland Ziekenhuis, Capelle aan den IJssel, The Netherlands.
Harald J FaberICU, WZA, Assen, The Netherlands.
A KarakusDepartment of Intensive Care, Diakonessenhuis Hospital, Utrecht, The Netherlands.
Robert EntjesDepartment of Intensive Care, Adrz, Goes, The Netherlands.
Paul de JongDepartment of Anesthesia and Intensive Care, Slingeland Ziekenhuis, Doetinchem, The Netherlands.
Thijs C D RettigDepartment of Anesthesiology, Intensive Care and Pain Medicine, Amphia Ziekenhuis, Breda, The Netherlands.
M C ReulandDepartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
Sesmu ArbousIntensivist, LUMC, Leiden, The Netherlands.
Lucas M FleurenDepartment of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
Tariq A DamDepartment of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
Patrick J ThoralDepartment of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
Robbert C A LalisangPacmed, Amsterdam, The Netherlands.
Michele TonuttiPacmed, Amsterdam, The Netherlands.
Daan P de BruinPacmed, Amsterdam, The Netherlands.
Paul W G ElbersDepartment of Intensive Care Medicine, Laboratory for Critical Care Computational Intelligence, Amsterdam Medical Data Science, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
Nicolette F de KeizerDepartment of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands; National Intensive Care Evaluation (NICE) foundation, Amsterdam, The Netherlands.
Dutch COVID-19 Research Consortium, the Dutch ICU Data Sharing Against COVID-19 Collaborators

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

purposeTo assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data.

methodsObservational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors.

resultsA total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors.

conclusionIn our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Indexed as

COVID-19Electronic Health RecordsHospital MortalityHumansIntensive Care UnitsNetherlandsRegistriesRetrospective StudiesCovid-19 [C01.748.610.763.500]Critical care [E02.760.190]Electronic Health Record [E05.318.308.940.968.625.500]In-hospital mortality [E05.318.308.985.550.400]Machine learning [G17.035.250.500]Prognosis [E01.789]

Identifiers

PMID36162166
PMCPMC9492397

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.