Evidence map›Paper›PMID 34799630›Full record

ArticleScientific reports2021

Effect of data leakage in brain MRI classification using 2D convolutional neural networks.

Ekin Yagis, Selamawet Workalemahu Atnafu, Alba García Seco de Herrera, Chiara Marzi, Riccardo Scheda, Marco Giannelli, Carlo Tessa, Luca Citi, Stefano Diciotti

Abstract read
In one paragraph

Article in Scientific reports, 2021. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 48 papers, 2 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
48citing papers in PubMed, 2 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

48 citing papers in PubMed, 2 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Article
  4. Article
  5. Article
  6. Article
  7. Review
  8. Article
  9. Article
  10. Article
  11. Review
  12. Article
  13. Article
  14. Review
  15. Article
  16. Article
  17. Article
  18. Using Deep Learning to Simultaneously Reduce Noise and Motion Artifacts in Brain MR Imaging.Magnetic resonance in medical sciences : MRMS : an official journal of Japan Society of Magnetic Resonance in Medicine · 2025
    Article
  19. Article
  20. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

9 authors.

Ekin Yagis *School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK.
Selamawet Workalemahu Atnafu *Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy.
Alba García Seco de HerreraSchool of Computer Science and Electronic Engineering, University of Essex, Colchester, UK.
Chiara MarziDepartment of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy.
Riccardo SchedaDepartment of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy.
Marco GiannelliUnit of Medical Physics, Pisa University Hospital "Azienda Ospedaliero-Universitaria Pisana", Pisa, Italy.
Carlo TessaDivision of Radiology, Versilia Hospital, Azienda USL Toscana Nord Ovest, Lido di Camaiore, LU, Italy.
Luca CitiSchool of Computer Science and Electronic Engineering, University of Essex, Colchester, UK.
Stefano DiciottiDepartment of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", University of Bologna, Via dell'Università 50, 47521, Cesena, Italy. stefano.diciotti@unibo.it.

Funding

Alzheimer's Disease Neuroimaging Initiative - SupplementU01AG024904 · NIA · NORTHERN CALIFORNIA INSTITUTE RES &EDUC · PI WEINER, MICHAEL W · 2004 to 2015
$121.0M
Smartphone-Based "Burst" Cognitive AssessmentsP01AG003991 · NIA · WASHINGTON UNIVERSITY · PI JOHN MORRIS · 1985 to 2026
$69.5M
WASHINGTON UNIVERSITY ALZHEIMERS DISEASE RESEARCH CENTERP50AG005681 · NIA · WASHINGTON UNIVERSITY · PI FAGAN, ANNE · 1985 to 2019
$52.1M
The natural history of AB accumulation in preclinical ADP01AG026276 · NIA · WASHINGTON UNIVERSITY · PI MORRIS, JOHN · 2005 to 2025
$49.5M
Research Education ComponentP30AG066444 · NIA · WASHINGTON UNIVERSITY · PI DAVID M. HOLTZMAN · 2020 to 2026
$28.7M
MORPHOMETRY BIOMEDICAL INFORMATICS RESEARCH NETWORKU24RR021382 · NCRR · MASSACHUSETTS GENERAL HOSPITAL · PI ROSEN, BRUCE R · 2004 to 2008
$24.1M
STRUCTURE, FUNCTION AND COGNITION IN SCHIZOPHRENIAP50MH071616 · NIMH · WASHINGTON UNIVERSITY · PI BARCH, DEANNA · 2004 to 2008
$11.2M
Functional-anatomic exploration of cognitive controlR01AG021910 · NIA · WASHINGTON UNIVERSITY · PI BUCKNER, RANDY L · 2005 to 2009
$1.2M
European Union's Horizon 2020 research and innovation programme 824153 "POTION"NCRR NIH HHS U24 RR021382NIA NIH HHS P01 AG003991NIA NIH HHS P01 AG026276NIA NIH HHS P30 AG066444NIA NIH HHS P50 AG005681NIA NIH HHS R01 AG021910NIA NIH HHS U01 AG024904NIMH NIH HHS P50 MH071616
6 · The paper itself

Abstract

In recent years, 2D convolutional neural networks (CNNs) have been extensively used to diagnose neurological diseases from magnetic resonance imaging (MRI) data due to their potential to discern subtle and intricate patterns. Despite the high performances reported in numerous studies, developing CNN models with good generalization abilities is still a challenging task due to possible data leakage introduced during cross-validation (CV). In this study, we quantitatively assessed the effect of a data leakage caused by 3D MRI data splitting based on a 2D slice-level using three 2D CNN models to classify patients with Alzheimer's disease (AD) and Parkinson's disease (PD). Our experiments showed that slice-level CV erroneously boosted the average slice level accuracy on the test set by 30% on Open Access Series of Imaging Studies (OASIS), 29% on Alzheimer's Disease Neuroimaging Initiative (ADNI), 48% on Parkinson's Progression Markers Initiative (PPMI) and 55% on a local de-novo PD Versilia dataset. Further tests on a randomly labeled OASIS-derived dataset produced about 96% of (erroneous) accuracy (slice-level split) and 50% accuracy (subject-level split), as expected from a randomized experiment. Overall, the extent of the effect of an erroneous slice-based CV is severe, especially for small datasets.

Indexed as

Image Interpretation, Computer-AssistedMagnetic Resonance ImagingNeural Networks, ComputerNeuroimagingAgedAged, 80 and overAlzheimer DiseaseBrainCase-Control StudiesCross-Sectional StudiesDeep LearningFemaleHumansMaleMiddle AgedParkinson Disease

Identifiers

PMID34799630
PMCPMC8604922

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.