Evidence map›Paper›PMID 32483644›Full record

ArticleEuropean radiology2020

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.

Isabella Fornacon-Wood, Hitesh Mistry, Christoph J Ackermann, Fiona Blackhall, Andrew McPartlin, Corinne Faivre-Finn, Gareth J Price, James P B O'Connor

Abstract read
In one paragraph

Article in European radiology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 120 papers, 6 of them syntheses that pooled it.

0numbers the graph read from it
0cells of the map it votes in
120citing papers in PubMed, 6 pooled it
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

120 citing papers in PubMed, 6 syntheses or guidelines pooled it.

  1. Pooled it
  2. Pooled it
  3. Guideline
  4. Pooled it
  5. Guideline
  6. The application of radiomics in laryngeal cancer.The British journal of radiology · 2021
    Pooled it
  7. Trial
  8. Trial
  9. Trial
  10. Article
  11. Article
  12. Article
  13. Radiomics in laryngeal squamous cell carcinoma: state of the art.Acta otorhinolaryngologica Italica : organo ufficiale della Societa italiana di otorinolaringologia e chirurgia cervico-facciale · 2026
    Review
  14. Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Review

60 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

8 authors.

Isabella Fornacon-WoodDivision of Cancer Sciences, University of Manchester, Manchester, UK. Isabella.fornacon-wood@postgrad.manchester.ac.uk.ORCID http://orcid.org/0000-0002-3736-2967
Hitesh MistryDivision of Cancer Sciences, University of Manchester, Manchester, UK.
Christoph J AckermannDepartment of Medical Oncology, Spital STS AG, Thun, Switzerland.
Fiona BlackhallDivision of Cancer Sciences, University of Manchester, Manchester, UK.
Andrew McPartlinDepartment of Clinical Oncology, The Christie Hospital NHS Foundation Trust, Manchester, UK.
Corinne Faivre-FinnDivision of Cancer Sciences, University of Manchester, Manchester, UK.
Gareth J PriceDivision of Cancer Sciences, University of Manchester, Manchester, UK.
James P B O'ConnorDivision of Cancer Sciences, University of Manchester, Manchester, UK.

Funding

Cancer Research UK 28683Cancer Research UK 29420Cancer Research UK C147/A18083Cancer Research UK C147/A25254Cancer Research UK C19221/A22746
6 · The paper itself

Abstract

objectiveTo investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.

methodsThe statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.

resultsThe reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.

conclusionIBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics. KEY POINTS: • Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.

Indexed as

SoftwareCarcinoma, Non-Small-Cell LungHead and Neck NeoplasmsHumansImage Processing, Computer-AssistedLung NeoplasmsPrognosisProportional Hazards ModelsReproducibility of ResultsRetrospective StudiesSmall Cell Lung CarcinomaTomography, X-Ray ComputedBiomarkersPrognosisReliability of resultsTomography, x-ray computedTranslation

Identifiers

PMID32483644
PMCPMC7553896

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.