Evidence map›Paper›PMID 31494250›Full record

SynthesisNeuroImage2019

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

Stephanie Noble, Dustin Scheinost, R Todd Constable

Registry-linked trialAbstract readMeta-AnalysisSystematic Review
In one paragraph

Synthesis in NeuroImage, 2019. The graph could read no effect estimate from its abstract, so it casts no vote on the map. It is linked to trial NCT07479966 (Resting-state Connectivity and Individual Differences in Affective Placebo Responsiveness), which is not on this map. Cited by 335 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
335citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

NCT07479966 enrolling by invitationnot on this mapstarted 2022, after this paper: background citation

Resting-state Connectivity and Individual Differences in Affective Placebo Responsiveness

TypeobservationalSponsorStefanie BrassenRan2022 to 2026Enrolled50ConditionsCondition: HealthyArmsNo intervention
3 · Its place in the literature

Who cites it

335 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Heterogeneity in measurement of NIHSS: a systematic review and meta-analysis.Neurological sciences : official journal of the Italian Neurological Society and of the Italian Society of Clinical Neurophysiology · 2025
    Pooled it
  3. Trial
  4. Trial
  5. Trial
  6. Trial
  7. Article
  8. Article
  9. Article
  10. Article
  11. Review
  12. Review
  13. Article
  14. Article
  15. Article
  16. Article
  17. Article
  18. Functional brain organization is stable within individuals across years.bioRxiv : the preprint server for biology · 2026
    Article
  19. Article
  20. Article

275 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Stephanie NobleInterdepartmental Neuroscience Program, Yale University, USA. Electronic address: stephanie.noble@yale.edu.
Dustin ScheinostInterdepartmental Neuroscience Program, Yale University, USA; Department of Radiology and Biomedical Imaging, Yale School of Medicine, USA; Department of Statistics and Data Science, Yale University, USA; Child Study Center, Yale School of Medicine, USA.
R Todd ConstableInterdepartmental Neuroscience Program, Yale University, USA; Department of Radiology and Biomedical Imaging, Yale School of Medicine, USA; Department of Neurosurgery, Yale School of Medicine, USA.

Funding

Yale Clinical and Translational Science Award (U Component)UL1TR001863 · NCATS · YALE UNIVERSITY · PI John H. Krystal, LUCILA OHNO-MACHADO · 2016 to 2026
$102.9M
Constrained Network-Based Multiple Comparison CorrectionK00MH122372 · NIMH · YALE UNIVERSITY · PI NOBLE, STEPHANIE MARIE · 2019 to 2021
$242k
Improving Reliability and Validity of fMRI Statistical MethodsF99NS108557 · NINDS · YALE UNIVERSITY · PI NOBLE, STEPHANIE MARIE · 2018 to 2018
$43k
NCATS NIH HHS UL1 TR001863NIMH NIH HHS K00 MH122372NINDS NIH HHS F99 NS108557
6 · The paper itself

Abstract

backgroundOnce considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed.

objectivesWe aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability.

methodsA search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC.

resultsFrom an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a "poor" ICC of 0.29 (95% CI = 0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage.

conclusionThis study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.

Indexed as

Brain MappingMagnetic Resonance ImagingBrainHumansImage Processing, Computer-AssistedNeural PathwaysReproducibility of ResultsEdge-levelfMRIIntraclass correlation coefficientSystematic reviewValidity

Identifiers

PMID31494250
PMCPMC6907736

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.