Evidence map›Paper›PMID 38001421›Full record

ArticleBMC medical research methodology2023

Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm.

Ashleigh Ledger, Jolien Ceusters, Lil Valentin, Antonia Testa, Caroline Van Holsbeke, Dorella Franchi, Tom Bourne, Wouter Froyman, Dirk Timmerman, Ben Van Calster

Abstract read
In one paragraph

Article in BMC medical research methodology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 6 papers.

0numbers the graph read from it
0cells of the map it votes in
6citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

6 citing papers in PubMed.

  1. Review
  2. Machine learning prediction of germlineBMJ health & care informatics · 2025
    Article
  3. Review
  4. Article
  5. Article
  6. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

10 authors.

Ashleigh LedgerDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
Jolien CeustersDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
Lil ValentinDepartment of Obstetrics and Gynecology, Skåne University Hospital, Malmö, Sweden.
Antonia TestaDepartment of Woman, Child and Public Health, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy.
Caroline Van HolsbekeDepartment of Obstetrics and Gynecology, Ziekenhuis Oost-Limburg, Genk, Belgium.
Dorella FranchiPreventive Gynecology Unit, Division of Gynecology, European Institute of Oncology IRCCS, Milan, Italy.
Tom BourneDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
Wouter FroymanDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
Dirk TimmermanDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
Ben Van CalsterDepartment of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium. ben.vancalster@kuleuven.be.ORCID 0000-0003-1613-7450

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

backgroundAssessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic.

methodsThis retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery ≤ 120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125.

resultsMost tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold.

conclusionAlthough several models had similarly good performance, individual probability estimates varied substantially.

Indexed as

Ovarian NeoplasmsAlgorithmsCA-125 AntigenFemaleHumansLogistic ModelsRetrospective StudiesUncertaintyCA-125 AntigenCalibrationMachine learningMulticlass modelsOvarian NeoplasmsPrediction models

Identifiers

PMID38001421
PMCPMC10668424

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.