Evidence map›Paper›PMID 36138025›Full record

ArticleScientific data2022

Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images.

Iulian Emil Tampu, Anders Eklund, Neda Haj-Hosseini

Abstract read
In one paragraph

Article in Scientific data, 2022. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 33 papers.

0numbers the graph read from it
0cells of the map it votes in
33citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

33 citing papers in PubMed.

  1. Article
  2. Review
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Review
  9. Review
  10. Review
  11. Article
  12. Review
  13. Article
  14. Review
  15. Article
  16. Article
  17. Article
  18. Article
  19. Article
  20. Review
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

3 authors.

Iulian Emil TampuDepartment of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden. iulian.emil.tampu@liu.se.ORCID 0000-0002-7582-1706
Anders EklundDepartment of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden.ORCID 0000-0001-7061-7995
Neda Haj-HosseiniDepartment of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden.ORCID 0000-0002-0555-8877

Funding

Åke Wiberg Stiftelse (Åke Wiberg Foundation) M19-0455, M20-0034, M21-0083Forskningsrådet i Sydöstra Sverige (Medical Research Council of Southeast Sweden) 931466Vetenskapsrådet (Swedish Research Council) (2018-05250)VINNOVA (Swedish Governmental Agency for Innovation Systems) 2017-02447
6 · The paper itself

Abstract

In the application of deep learning on optical coherence tomography (OCT) data, it is common to train classification networks using 2D images originating from volumetric data. Given the micrometer resolution of OCT systems, consecutive images are often very similar in both visible structures and noise. Thus, an inappropriate data split can result in overlap between the training and testing sets, with a large portion of the literature overlooking this aspect. In this study, the effect of improper dataset splitting on model evaluation is demonstrated for three classification tasks using three OCT open-access datasets extensively used, Kermany's and Srinivasan's ophthalmology datasets, and AIIMS breast tissue dataset. Results show that the classification performance is inflated by 0.07 up to 0.43 in terms of Matthews Correlation Coefficient (accuracy: 5% to 30%) for models tested on datasets with improper splitting, highlighting the considerable effect of dataset handling on model evaluation. This study intends to raise awareness on the importance of dataset splitting given the increased research interest in implementing deep learning on OCT data.

Indexed as

Deep LearningTomography, Optical CoherenceHumans

Identifiers

PMID36138025
PMCPMC9500039

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.