Evidence map›Paper›PMID 36537895›Full record

ArticleRadiology2023

Inconsistent Partitioning and Unproductive Feature Associations Yield Idealized Radiomic Models.

Mishka Gidwani, Ken Chang, Jay Biren Patel, Katharina Viktoria Hoebel, Syed Rakin Ahmed, Praveer Singh, Clifton David Fuller, Jayashree Kalpathy-Cramer

Abstract read
PubMed Full text at PMC Publisher
In one paragraph

Article in Radiology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 20 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
20citing papers in PubMed, 1 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

20 citing papers in PubMed, 1 synthesis or guideline pooled it.

  1. Pooled it
  2. Temporalis muscle biomarkers from routine brain MRI and risk of dementia in two independent cohorts.Alzheimer's & dementia : the journal of the Alzheimer's Association · 2026
    Article
  3. Review
  4. Article
  5. Article
  6. Review
  7. Review
  8. Article
  9. Article
  10. Review
  11. Article
  12. Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
  18. Review
  19. Article
  20. Review
4 · The record

Corrections and comments

5 · Who and what money

Authors and funding

8 authors.

Mishka GidwaniFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0002-8477-2539
Ken ChangFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0001-6956-5059
Jay Biren PatelFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).
Katharina Viktoria HoebelFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0002-1881-7065
Syed Rakin AhmedFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0002-1615-8633
Praveer SinghFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0001-6641-2030
Clifton David FullerFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0002-5264-3994
Jayashree Kalpathy-CramerFrom the Athinoula A. Martinos Center for Biomedical Imaging (M.G., K.C., J.B.P., K.V.H., S.R.A., P.S., J.K.C.) and Department of Radiology (J.K.C.), Massachusetts General Brigham, 13th St, Building 149, Room 2301, Charlestown, MA 02129; Case Western School of Medicine, Cleveland, Ohio (M.G.); Harvard-MIT Division of Health Sciences and Technology, Cambridge, Mass (J.B.P., K.V.H.); Harvard Graduate Program in Biophysics, Harvard Medical School, Harvard University, Cambridge, Mass (S.R.A.); Geisel School of Medicine at Dartmouth, Dartmouth College, Hanover, NH (S.R.A.); and Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Tex (C.D.F.).ORCID 0000-0001-8906-9618

Funding

Tumor Evolution and Metastasis ProgramP30CA016672 · NCI · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI Fabien Delerue · 1985 to 2026
$290.8M
Targeting the p53 pathway for reversal of oral premalignancyP50CA097007 · NCI · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI MILLS, GORDON B. · 2002 to 2012
$26.1M
Development of functional magnetic resonance imaging-guided adaptive radiotherapy for head and neck cancer patients using novel MR-Linac deviceR01DE028290 · NIDCR · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI CHRISTODOULEAS, JOHN PAUL, FULLER, CLIFTON DAVID · 2019 to 2023
$4.3M
Using Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) to Establish Objective Clinical Outcome Measures for Mandibular OsteoradionecrosisR01DE025248 · NIDCR · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI LAI, STEPHEN Y · 2016 to 2020
$2.9M
imaging Radiation-Associated Dysphagia (iRAD)R01CA218148 · NCI · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI FULLER, CLIFTON DAVID, HUTCHESON, KATHERINE ARNOLD · 2017 to 2019
$810k
Fellow and Resident Radiation Oncology iNtensive Training in Imaging and Informatics to Empower Research Careers (FRONTI2ER)R25EB025787 · NIBIB · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI DAS, PRAJNAN, FULLER, CLIFTON DAVID · 2018 to 2022
$535k
Using Dynamic Contrast-Enhanced Magnetic Resonance Imaging (DCE-MRI) to Establish Objective Clinical Outcome Measures for Mandibular OsteoradionecrosisR56DE025248 · NIDCR · UNIVERSITY OF TX MD ANDERSON CAN CTR · PI LAI, STEPHEN Y · 2015 to 2015
$470k
NCI NIH HHS P30 CA016672NCI NIH HHS P50 CA097007NCI NIH HHS R01 CA218148NIBIB NIH HHS R25 EB025787NIDCR NIH HHS R01 DE025248NIDCR NIH HHS R01 DE028290NIDCR NIH HHS R56 DE025248
6 · The paper itself

Abstract

Background Radiomics is the extraction of predefined mathematic features from medical images for the prediction of variables of clinical interest. While some studies report superlative accuracy of radiomic machine learning (ML) models, the published methodology is often incomplete, and the results are rarely validated in external testing data sets. Purpose To characterize the type, prevalence, and statistical impact of methodologic errors present in radiomic ML studies. Materials and Methods Radiomic ML publications were reviewed for the presence of performance-inflating methodologic flaws. Common flaws were subsequently reproduced with randomly generated features interpolated from publicly available radiomic data sets to demonstrate the precarious nature of reported findings. Results In an assessment of radiomic ML publications, the authors uncovered two general categories of data analysis errors: inconsistent partitioning and unproductive feature associations. In simulations, the authors demonstrated that inconsistent partitioning augments radiomic ML accuracy by 1.4 times from unbiased performance and that correcting for flawed methodologic results in areas under the receiver operating characteristic curve approaching a value of 0.5 (random chance). With use of randomly generated features, the authors illustrated that unproductive associations between radiomic features and gene sets can imply false causality for biologic phenomenon. Conclusion Radiomic machine learning studies may contain methodologic flaws that undermine their validity. This study provides a review template to avoid such flaws. © RSNA, 2022

Indexed as

Machine LearningHumansRetrospective StudiesROC Curve

Identifiers

PMID36537895
PMCPMC10068883

What OpenQuestion holds

Textmetadata
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.