Evidence map›Paper›PMID 33118182›Full record

SynthesisMedical physics2020

Reproducibility analysis of multi-institutional paired expert annotations and radiomic features of the Ivy Glioblastoma Atlas Project (Ivy GAP) dataset.

Sarthak Pati, Ruchika Verma, Hamed Akbari, Michel Bilello, Virginia B Hill, Chiharu Sako, Ramon Correa, Niha Beig, Ludovic Venet, Siddhesh Thakur and 6 more

Open access · greenAbstract readMeta-Analysis
In one paragraph

Synthesis in Medical physics, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 28 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
28citing papers in PubMed, 1 pooled it
2.8field-weighted citation impact, top 7% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

28 citing papers in PubMed, 1 synthesis or guideline pooled it, 42 citations in OpenAlex.

  1. Pooled it
  2. Article
  3. Article
  4. Article
  5. Article
  6. Article
  7. Article
  8. Review
  9. Article
  10. Article
  11. Review
  12. Review
  13. Article
  14. Article
  15. Article
  16. Article
  17. Review
  18. Article
  19. Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

16 authors at 6 institutions in 1 country.

Sarthak PatiCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0003-2243-8487
Ruchika VermaDepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.ORCID http://orcid.org/0000-0003-4870-128X
Hamed AkbariCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0001-9786-3707
Michel BilelloDepartment of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0001-6313-5437
Virginia B HillDepartment of Radiology, Feinberg School of Medicine, Northwestern University, Chicago, IL, 60611, USA.ORCID http://orcid.org/0000-0001-9146-9370
Chiharu SakoCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0003-3243-3954
Ramon CorreaDepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.ORCID http://orcid.org/0000-0003-2247-7777
Niha BeigDepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.ORCID http://orcid.org/0000-0002-1150-5886
Ludovic VenetCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0003-4203-5378
Siddhesh ThakurCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0003-4807-2495
Prashant SeraiCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0002-4672-4413
Sung Min HaCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0002-3526-9101
Geri D BlakeUniversity of Arkansas for Medical Sciences, Little Rock, AR, USA.ORCID http://orcid.org/0000-0002-4475-298X
Russell Taki ShinoharaCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0001-8627-8203
Pallavi TiwariDepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.ORCID http://orcid.org/0000-0001-9477-4856
Spyridon BakasCenter for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, 19104, USA.ORCID http://orcid.org/0000-0001-8734-6482
University of Pennsylvania · USCase Western Reserve University · USNorthwestern University · USPenn Center for AIDS Research · USThe Ohio State University · USUniversity of Arkansas for Medical Sciences · US

Funding

Statistical atlases of brain tumor MRI:do imaging phenotypes predict progression?R01NS042645 · NINDS · UNIVERSITY OF PENNSYLVANIA · PI DAVATZIKOS, CHRISTOS · 2002 to 2024
$8.7M
Biomedical Image Computing and Informatics ClusterS10OD023495 · OD · UNIVERSITY OF PENNSYLVANIA · PI DAVATZIKOS, CHRISTOS · 2017 to 2017
$1.9M
RadxTools for assessing tumor treatment response on imagingU01CA248226 · NCI · CASE WESTERN RESERVE UNIVERSITY · PI TIWARI, PALLAVI, VISWANATH, SATISH EASWAR · 2020 to 2022
$1.4M
The Federated Tumor Segmentation (FeTS) platform: An intuitive tool facilitating secure multi-institutional collaborationU01CA242871 · NCI · UNIVERSITY OF PENNSYLVANIA · PI BAKAS, SPYRIDON · 2019 to 2021
$1.1M
CCCC Brain Tumor Pilot AwardCWRU Technology Validation Start-Up FundDana Foundation David Mahoney Neuroimaging GrantDepartment of Defense (DoD) Peer Reviewed Cancer Research Program W81XWH-18-1-0404NCI NIH HHS U01 CA242871NCI NIH HHS U01 CA248226NIH HHS S10 OD023495NIH HHS U01CA242871NINDS NIH HHS R01 NS042645The V Foundation Translational Research Award
6 · The paper itself

Abstract

purposeThe availability of radiographic magnetic resonance imaging (MRI) scans for the Ivy Glioblastoma Atlas Project (Ivy GAP) has opened up opportunities for development of radiomic markers for prognostic/predictive applications in glioblastoma (GBM). In this work, we address two critical challenges with regard to developing robust radiomic approaches: (a) the lack of availability of reliable segmentation labels for glioblastoma tumor sub-compartments (i.e., enhancing tumor, non-enhancing tumor core, peritumoral edematous/infiltrated tissue) and (b) identifying "reproducible" radiomic features that are robust to segmentation variability across readers/sites. ACQUISITION AND VALIDATION

methodsFrom TCIA's Ivy GAP cohort, we obtained a paired set (n = 31) of expert annotations approved by two board-certified neuroradiologists at the Hospital of the University of Pennsylvania (UPenn) and at Case Western Reserve University (CWRU). For these studies, we performed a reproducibility study that assessed the variability in (a) segmentation labels and (b) radiomic features, between these paired annotations. The radiomic variability was assessed on a comprehensive panel of 11 700 radiomic features including intensity, volumetric, morphologic, histogram-based, and textural parameters, extracted for each of the paired sets of annotations. Our results demonstrated (a) a high level of inter-rater agreement (median value of DICE ≥0.8 for all sub-compartments), and (b) ≈24% of the extracted radiomic features being highly correlated (based on Spearman's rank correlation coefficient) to annotation variations. These robust features largely belonged to morphology (describing shape characteristics), intensity (capturing intensity profile statistics), and COLLAGE (capturing heterogeneity in gradient orientations) feature families. DATA FORMAT AND USAGE NOTES: We make publicly available on TCIA's Analysis Results Directory (https://doi.org/10.7937/9j41-7d44), the complete set of (a) multi-institutional expert annotations for the tumor sub-compartments, (b) 11 700 radiomic features, and (c) the associated reproducibility meta-analysis. POTENTIAL APPLICATIONS: The annotations and the associated meta-data for Ivy GAP are released with the purpose of enabling researchers toward developing image-based biomarkers for prognostic/predictive applications in GBM.

Indexed as

GlioblastomaCohort StudiesHumansMagnetic Resonance ImagingReproducibility of ResultsglioblastomaIvyGAPMRIradiomicsreproducibilitysegmentation

Identifiers

PMID33118182
PMCPMC8382093
OpenAlexW3096019617

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.