Evidence map›Paper›PMID 39837647›Full record

ArticleJournal of applied clinical medical physics2025

Evaluation of AI-based auto-contouring tools in radiotherapy: A single-institution study.

Tingyu Wang, James Tam, Thomas Chum, Cyril Tai, Deborah C Marshall, Michael Buckstein, Jerry Liu, Sheryl Green, Robert D Stewart, Tian Liu and 1 more

Abstract readEvaluation Study
In one paragraph

Article in Journal of applied clinical medical physics, 2025. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 10 papers.

0numbers the graph read from it
0cells of the map it votes in
10citing papers in PubMed
–field-weighted citation impact
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

10 citing papers in PubMed.

  1. Article
  2. Article
  3. Article
  4. Review
  5. Article
  6. Article
  7. Article
  8. Article
  9. Review
  10. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

11 authors.

Tingyu WangDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.ORCID https://orcid.org/0009-0009-0787-365X
James TamDepartment of Radiation Oncology, The Mount Sinai Hospital, New York, New York, USA.
Thomas ChumDepartment of Radiation Oncology, The Mount Sinai Hospital, New York, New York, USA.
Cyril TaiDepartment of Radiation Oncology, Mount Sinai Union Square, New York, New York, USA.
Deborah C MarshallDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Michael BucksteinDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Jerry LiuDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Sheryl GreenDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Robert D StewartDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Tian LiuDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.
Ming ChaoDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, New York, USA.

Funding

Novel Functional Anatomic and Biomarker Indices of Radiation-Induced Female Sexual Toxicities in a Multi-Center CohortDP5OD031876 · OD · ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI · PI MARSHALL, DEBORAH CATHERINE · 2021 to 2025
$2.1M
Mount Sinai Health SystemNIH HHS DP5 OD031876
6 · The paper itself

Abstract

backgroundAccurate delineation of organs at risk (OARs) is crucial yet time-consuming in the radiotherapy treatment planning workflow. Modern artificial intelligence (AI) technologies had made automation of OAR contouring feasible. This report details a single institution's experience in evaluating two commercial auto-contouring software tools and making well-informed decisions about their clinical adoption.

methodsA cohort of 36 patients previously treated at our institution were selected for the software performance assessment. Fifty-eight OAR structures from seven disease sites were automatically segmented with each tool. Five radiation oncologists with different specialties qualitatively scored the automatic OAR contours' clinical usability by a 4-level scale (0-3), termed as quality score (QS), representing from "0: not usable" to "3: directly usable for a clinic." Additionally, quantitative comparison with clinically approved contours using Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD95) was performed in complement to QS from physicians.

resultSoftware A achieved an average QS of 2.17 ± 0.69, comparable to Software B's average QS of 2.17 ± 0.72. Software B performed better with more OARs (42 vs. 37) that required minor or no modification than Software A. Major modifications were needed for 13 out of 58 automated contours from both tools. Both DSC and HD95 scores for the two tools were comparable to each other, with DSC: 0.67 ± 0.23 versus 0.66 ± 0.21 and HD95: 13.07 ± 15.84 versus 15.55 ± 18.45 for Software A and Software B, respectively. Correlation coefficients between the physician score and the quantitative metrics suggested that the contouring results from Software A aligned more closely with the physician's evaluations.

conclusionBased on our study, either software tool could produce clinically acceptable contours for about 65% of the OAR structures. However, further refinement is necessary for several challenging OARs to improve model performance.

Indexed as

Artificial IntelligenceImage Processing, Computer-AssistedNeoplasmsOrgans at RiskRadiotherapy Planning, Computer-AssistedSoftwareAlgorithmsHumansRadiotherapy DosageRadiotherapy, Intensity-Modulatedartificial intelligenceimage segmentationorgan at riskRadiation therapytreatment planning

Identifiers

PMID39837647
PMCPMC11969097

What OpenQuestion holds

Textmetadata
LicenceCC BY
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.