Evidence map›Paper›PMID 41402410›Full record

ArticleNPJ digital medicine2025

SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models.

Alvise Dei Rossi, Matteo Metaldi, Michal Bechny, Irina Filchenko, Julia van der Meer, Markus H Schmidt, Claudio L A Bassetti, Athina Tzovara, Francesca D Faraci, Luigi Fiorillo

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in NPJ digital medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 1 paper.

0numbers the graph read from it
0cells of the map it votes in
1citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

1 citing paper in PubMed.

  1. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Alvise Dei RossiInstitute of Digital Technologies for Personalized Healthcare ∣ MeDiTech, University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland.
Matteo MetaldiInstitute of Digital Technologies for Personalized Healthcare ∣ MeDiTech, University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland.
Michal BechnyInstitute of Digital Technologies for Personalized Healthcare ∣ MeDiTech, University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland.
Irina FilchenkoSleep Wake Epilepsy Center ∣ NeuroTec, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
Julia van der MeerSleep Wake Epilepsy Center ∣ NeuroTec, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
Markus H SchmidtSleep Wake Epilepsy Center ∣ NeuroTec, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
Claudio L A BassettiSleep Wake Epilepsy Center ∣ NeuroTec, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
Athina TzovaraInstitute of Informatics, University of Bern, Bern, Switzerland.
Francesca D FaraciInstitute of Digital Technologies for Personalized Healthcare ∣ MeDiTech, University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland.
Luigi FiorilloInstitute of Digital Technologies for Personalized Healthcare ∣ MeDiTech, University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland. luigi.fiorillo@supsi.ch.

Funding

University of Wisconsin Institute for Clinical and Translational ResearchUL1TR002373 · NCATS · UNIVERSITY OF WISCONSIN-MADISON · PI ELIZABETH S BURNSIDE, Allan R. Brasier · 2017 to 2026
$75.9M
INSTITUTIONAL CLINICAL AND TRANSLATIONAL SCIENCE AWARDUL1RR024134 · NCRR · UNIVERSITY OF PENNSYLVANIA · PI FITZGERALD, GARRET A · 2006 to 2011
$70.4M
Institute for Clinical and Translational ResearchUL1TR001079 · NCATS · JOHNS HOPKINS UNIVERSITY · PI FORD, DANIEL ERNEST · 2013 to 2017
$60.1M
CTSA INFRASTRUCTURE FOR PEDIATRIC RESEARCHUL1RR024989 · NCRR · CASE WESTERN RESERVE UNIVERSITY · PI DAVIS, PAMELA B · 2007 to 2011
$53.7M
Wake Forest Clinical and Translational Science AwardUL1TR001420 · NCATS · WAKE FOREST UNIVERSITY HEALTH SCIENCES · PI ARD, JAMY D, FOLEY, KRISTIE L · 2015 to 2023
$32.3M
Clinical and Translational Science Collaborative of Northern Ohio, Catalyzing Linkages to Equity in Health (CLE Health)UM1TR004528 · NCATS · CASE WESTERN RESERVE UNIVERSITY · PI GRACE A MCCOMSEY · 2023 to 2026
$32.1M
INSTITUTIONAL CTSA (UW-MADISON): CLINICAL TRIALSUL1RR025011 · NCRR · UNIVERSITY OF WISCONSIN-MADISON · PI DREZNER, MARC KENNETH · 2007 to 2011
$29.5M
Clinical and Translational Science AwardUL1TR000040 · NCATS · COLUMBIA UNIVERSITY HEALTH SCIENCES · PI GINSBERG, HENRY N · 2012 to 2015
$26.2M
Study of Osteoporotic FracturesR01AG005407 · NIA · UNIVERSITY OF CALIFORNIA SAN FRANCISCO · PI CUMMINGS, STEVEN RON, YAFFE, KRISTINE · 1986 to 2016
$23.2M
Zosuquidar trihydrochloride for acute myeloid leukemia &refractory anemiaM01RR000080 · NCRR · UNIVERSITY HOSPITALS OF CLEVELAND · PI DEARBORN, DORR GELLATLY · 1985 to 2007
$22.2M
EPIDEMIOLOGY OF SLEEP-DISORDERED BREATHING IN ADULTSR01HL062252 · NHLBI · UNIVERSITY OF WISCONSIN-MADISON · PI PEPPARD, PAUL E · 1999 to 2013
$14.1M
APPLES: Apnea Positive Pressure Long-Term Efficacy StudyU01HL068060 · NHLBI · STANFORD UNIVERSITY · PI KUSHIDA, CLETE A · 2002 to 2007
$14.1M
NCATS NIH HHS UL1 TR000040NCATS NIH HHS UL1 TR001079NCATS NIH HHS UL1 TR001420NCATS NIH HHS UL1 TR002373NCATS NIH HHS UM1 TR004528NCRR NIH HHS M01 RR000080NCRR NIH HHS UL1 RR024134NCRR NIH HHS UL1 RR024989NCRR NIH HHS UL1 RR025011NHLBI NIH HHS HHSN268201500003CNHLBI NIH HHS HHSN268201500003INHLBI NIH HHS K23 HL004426NHLBI NIH HHS K24 HL127307NHLBI NIH HHS N01 HC095159NHLBI NIH HHS N01 HC095160NHLBI NIH HHS N01 HC095161NHLBI NIH HHS N01 HC095162NHLBI NIH HHS N01 HC095163NHLBI NIH HHS N01 HC095164NHLBI NIH HHS N01 HC095165NHLBI NIH HHS N01 HC095166NHLBI NIH HHS N01 HC095167NHLBI NIH HHS N01 HC095168NHLBI NIH HHS N01 HC095169NHLBI NIH HHS R01 HL046380NHLBI NIH HHS R01 HL062252NHLBI NIH HHS R01 HL070837NHLBI NIH HHS R01 HL070838NHLBI NIH HHS R01 HL070839NHLBI NIH HHS R01 HL070841NHLBI NIH HHS R01 HL070842NHLBI NIH HHS R01 HL070847NHLBI NIH HHS R01 HL070848NHLBI NIH HHS R01 HL071194NHLBI NIH HHS R01 HL071515NHLBI NIH HHS R01 HL098433NHLBI NIH HHS R01 HL106410NHLBI NIH HHS R24 HL114473NHLBI NIH HHS T32 HL007567NHLBI NIH HHS U01 HL053916NHLBI NIH HHS U01 HL053931NHLBI NIH HHS U01 HL053934NHLBI NIH HHS U01 HL053937NHLBI NIH HHS U01 HL053938NHLBI NIH HHS U01 HL053941NHLBI NIH HHS U01 HL063463NHLBI NIH HHS U01 HL064360NHLBI NIH HHS U01 HL068060NHLBI NIH HHS U01 HL083075NHLBI NIH HHS U01 HL083129NIAMS NIH HHS R01 AR035582NIAMS NIH HHS R01 AR035583NIAMS NIH HHS R01 AR035584NIA NIH HHS R01 AG005394NIA NIH HHS R01 AG005407NIA NIH HHS R01 AG008415NIA NIH HHS R01 AG021918NIA NIH HHS R01 AG026720NIA NIH HHS R01 AG027574NIA NIH HHS R01 AG027576NIA NIH HHS R01 AG036838NIA NIH HHS R01 AG058680NIA NIH HHS T32 AG000212NIBIB NIH HHS R01 EB025018NICHD NIH HHS R01 HD079411NINR NIH HHS R01 NR002707swissuniversities P-5B_80 SleePyLand 2024-25
6 · The paper itself

Abstract

Automatic sleep staging with deep learning has advanced considerably, yet clinical adoption remains hindered by limited generalization, model bias, and inconsistent evaluation practices. We present SLEEPYLAND, an open-source framework comprising ~ 220,000 h of in-domain and ~ 84,000 h of out-of-domain polysomnographic recordings, spanning diverse ages, disorders, and hardware configurations. We release pre-trained state-of-the-art models, evaluating them across single- and multi-channel EEG/EOG setups. We introduce SOMNUS, an ensemble that integrates models via soft-voting, achieving robust performance across 24 datasets (macro-F1, 68.7-87.2%), outperforming individual models in 94.9% of cases and exceeding prior state-of-the-art. Exploiting the Bern-Sleep-Wake-Registry (N = 6633), we show that while SOMNUS improves generalization, no model architecture consistently minimizes model demographic/clinical bias. On multi-annotated datasets, SOMNUS surpasses the best human scorer (macro-F1, 85.2% vs 80.8% on DOD-H, and 80.2% vs 75.9% on DOD-O), more closely reproducing consensus. Finally, ensemble disagreement metrics predict scorer ambiguity (ROC-AUC 82.8%), providing reliable proxies for human uncertainty.

Identifiers

PMID41402410
PMCPMC12816009

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.