ArticleJournal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine2025
Automated analysis of the American Academy of Sleep Medicine Inter-Scorer Reliability gold-standard polysomnogram dataset.
Article in Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 2 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
2 citing papers in PubMed.
- Performance of nine large language models on real-world polysomnography interpretation for obstructive sleep apnea: a multi-dimensional comparative analysis.European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery · 2026Article
- Letter to the editor regarding 'a deep learning approach for gastroscopic manifestation recognition based on Kyoto Gastritis Score'.Annals of medicine · 2025Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
15 authors.
Funding
Abstract
STUDY
objectivesWe compared the performance of a comprehensive automated polysomnogram analysis algorithm-CAISR (Complete Artificial Intelligence Sleep Report)-to a multiexpert gold-standard panel, crowdsourced scorers, and experienced technicians for sleep staging and detecting arousals, respiratory events, and limb movements.
methodsA benchmark dataset of 57 polysomnogram records (Inter-Scorer Reliability dataset) with 200 30-second epochs scored per American Academy of Sleep Medicine guidelines was used. Annotations were obtained from (1) the American Academy of Sleep Medicine multiexpert gold-standard panel, (2) American Academy of Sleep Medicine Inter-Scorer Reliability (ISR) platform users ("crowd," averaging 6,818 raters per epoch), (3) 3 experienced technicians, and (4) CAISR. Agreement was assessed via Cohen's kappa (κ) and percent agreement.
resultsAcross tasks, CAISR achieved performance comparable to that of experienced technicians but did not match consensus-level agreement between the multiexpert gold standard and the crowd. For sleep staging, CAISR's agreement with multiexpert gold standard was 82.1% (κ = 0.70), comparable to experienced technicians but below the crowd (κ = 0.88). Arousal detection showed 87.81% agreement (κ = 0.45), respiratory event detection 83.18% agreement (κ = 0.34), and limb movement detection 94.89% agreement (κ = 0.11), each aligning with performance equivalent to experienced technicians but trailing crowd agreement (κ = 0.83, 0.78, and 0.86 for detection of arousal, respiratory events, and limb movements, respectively).
conclusionsCAISR achieves experienced technician-level accuracy for polysomnogram scoring tasks but does not surpass the consensus-level agreement of a multiexpert gold standard or the crowd. These findings highlight the potential of automated scoring to match experienced technician-level performance while emphasizing the value of multirater consensus. CITATION: Tripathi A, Nasiri S, Ganglberger W, et al. Automated analysis of the American Academy of Sleep Medicine Inter-Scorer Reliability gold-standard polysomnogram dataset.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.