ArticleBehavior research methods2024
The complexity of measuring reliability in learning tasks: An illustration using the Alternating Serial Reaction Time Task.
Article in Behavior research methods, 2024. The graph could read no effect estimate from its abstract, so it casts no vote on the map. An erratum has been issued. Cited by 19 papers.
What it found
Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.
The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.
The trial behind it
Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.
Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.
Who cites it
19 citing papers in PubMed.
- Longitudinal evidence for decreasing statistical learning abilities across childhood.Nature communications · 2026Article
- Obsessive-compulsive tendencies shift the balance between competitive neurocognitive functions.iScience · 2026Article
- No gender differences in predictive processing.Scientific reports · 2026Article
- Differences in characteristics of two aspects of procedural learning in action video game players.PeerJ · 2026Article
- The interplay between executive functions and updating predictive representations.Scientific reports · 2025Article
- Individual differences in probabilistic learning and updating predictive representations in individuals with obsessive-compulsive tendencies.BMC psychiatry · 2025Article
- Enhancing retrieval capacity of the predictive brain through dorsolateral prefrontal cortex intervention.Cerebral cortex (New York, N.Y. : 1991) · 2025Article
- Reliability of the serial reaction time task: If at first you don't succeed, try, try, try again.Quarterly journal of experimental psychology (2006) · 2024Article
- On the reliability of value-modulated attentional capture: An online replication and multiverse analysis.Behavior research methods · 2024Article
- Top-down and bottom-up oscillatory dynamics regulate implicit visuomotor sequence learning.Cerebral cortex (New York, N.Y. : 1991) · 2024Article
- Evidence for a competitive relationship between executive functions and statistical learning.NPJ science of learning · 2024Article
- Reliability of individual differences in distractor suppression driven by statistical learning.Behavior research methods · 2024Article
- Resting network architecture of theta oscillations reflects hyper-learning of sensorimotor information in Gilles de la Tourette syndrome.Brain communications · 2024Article
- Identifying Transfer Learning in the Reshaping of Inductive Biases.Open mind : discoveries in cognitive science · 2024Article
- Intact predictive processing in autistic adults: evidence from statistical learning.Scientific reports · 2023Article
- The reliability of the serial reaction time task: meta-analysis of test-retest correlations.Royal Society open science · 2023Article
- Modality-specific and modality-independent neural representations work in concert in predictive processes during sequence learning.Cerebral cortex (New York, N.Y. : 1991) · 2023Article
- Inhibitory control hinders habit change.Scientific reports · 2022Article
- Long-term memory predictors of adult language learning at the interface between syntactic form and meaning.PloS one · 2022Article
Corrections and comments
- Erratum issued
Authors and funding
4 authors.
Funding
No grant is acknowledged in the PubMed record.
Abstract
Despite the fact that reliability estimation is crucial for robust inference, it is underutilized in neuroscience and cognitive psychology. Appreciating reliability can help researchers increase statistical power, effect sizes, and reproducibility, decrease the impact of measurement error, and inform methodological choices. However, accurately calculating reliability for many experimental learning tasks is challenging. In this study, we highlight a number of these issues, and estimate multiple metrics of internal consistency and split-half reliability of a widely used learning task on a large sample of 180 subjects. We show how pre-processing choices, task length, and sample size can affect reliability and its estimation. Our results show that the Alternating Serial Reaction Time Task has respectable reliability, especially when learning scores are calculated based on reaction times and two-stage averaging. We also show that a task length of 25 blocks can be sufficient to meet the usual thresholds for minimally acceptable reliability. We further illustrate how relying on a single point estimate of reliability can be misleading, and the calculation of multiple metrics, along with their uncertainties, can lead to a more complete characterization of the psychometric properties of tasks.
Indexed as
Identifiers
What OpenQuestion holds
Registered trials
Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.