ArticleClinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology2015
Inter-expert and intra-expert reliability in sleep spindle scoring.
Article in Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology, 2015. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 35 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
35 citing papers in PubMed, 74 citations in OpenAlex.
- EEG Quantification: A New Dawn for an Old Tool.Revista de neurologia · 2026Article
- From sleep staging to spindle detection: a case study on end-to-end automated sleep analysis.Scientific reports · 2026Article
- OpenSpindleNet: An open-source deep learning network for reliable sleep spindle detection in scalp and intracranial EEG.Computers in biology and medicine · 2025Article
- Sleep Identification Enabled by Supervised Training Algorithms (SIESTA): An Open-Source Platform for Automatic Sleep Staging of Rodent Electrocorticographic and Electromyographic Data.Journal of biological rhythms · 2025Article
- Discovering EEG biomarkers of Lennox-Gastaut syndrome through unsupervised time-frequency analysis.Epilepsia · 2025Article
- Structure of Spectral Composition and Synchronization in Human Sleep on the Whole Scalp: A Pilot Study.Brain sciences · 2024Article
- A personalized semi-automatic sleep spindle detection (PSASD) framework.Journal of neuroscience methods · 2024Article
- Individualized temporal patterns dominate cortical upstate and sleep depth in driving human sleep spindle timing.bioRxiv : the preprint server for biology · 2024Article
- Factors affecting discrepancies between scorers in manual sleep spindle detections in single-channel electroencephalography in young adult males.Frontiers in sleep · 2024Article
- Switching state-space modeling of neural signal dynamics.PLoS computational biology · 2023Article
- Hierarchical fusion detection algorithm for sleep spindle detection.Frontiers in neuroscience · 2023Article
- [Sleep spindles-Function, detection and use as biomarker for diagnostics in psychiatry].Der Nervenarzt · 2022Review
- Advanced sleep spindle identification with neural networks.Scientific reports · 2022Article
- Effect of a Recliner Chair with Rocking Motions on Sleep Efficiency.Sensors (Basel, Switzerland) · 2021Article
- Article
- Heart Rate and Heart Rate Variability Change with Sleep Stage in Dairy Cows.Animals : an open access journal from MDPI · 2021Article
- Machine learning prediction of sleep stages in dairy cows from heart rate and muscle activity measures.Scientific reports · 2021Article
- Analysis and visualization of sleep stages based on deep neural networks.Neurobiology of sleep and circadian rhythms · 2021Article
- Large-scale assessment of consistency in sleep stage scoring rules among multiple sleep centers using an interpretable machine learning algorithm.Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine · 2021Article
- Validation of an Automatic Arousal Detection Algorithm for Whole-Night Sleep EEG Recordings.Clocks & sleep · 2020Article
Corrections and comments
PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.
Authors and funding
8 authors at 5 institutions in 2 countries.
Funding
Abstract
objectivesTo measure the inter-expert and intra-expert agreement in sleep spindle scoring, and to quantify how many experts are needed to build a reliable dataset of sleep spindle scorings.
methodsThe EEG dataset was comprised of 400 randomly selected 115s segments of stage 2 sleep from 110 sleeping subjects in the general population (57±8, range: 42-72 years). To assess expert agreement, a total of 24 Registered Polysomnographic Technologists (RPSGTs) scored spindles in a subset of the EEG dataset at a single electrode location (C3-M2). Intra-expert and inter-expert agreements were calculated as F1-scores, Cohen's kappa (κ), and intra-class correlation coefficient (ICC).
resultsWe found an average intra-expert F1-score agreement of 72±7% (κ: 0.66±0.07). The average inter-expert agreement was 61±6% (κ: 0.52±0.07). Amplitude and frequency of discrete spindles were calculated with higher reliability than the estimation of spindle duration. Reliability of sleep spindle scoring can be improved by using qualitative confidence scores, rather than a dichotomous yes/no scoring system.
conclusionsWe estimate that 2-3 experts are needed to build a spindle scoring dataset with 'substantial' reliability (κ: 0.61-0.8), and 4 or more experts are needed to build a dataset with 'almost perfect' reliability (κ: 0.81-1). SIGNIFICANCE: Spindle scoring is a critical part of sleep staging, and spindles are believed to play an important role in development, aging, and diseases of the nervous system.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.