Evidence map›Paper›PMID 36937399›Full record

ArticleFrontiers in oncology2023

How smart is artificial intelligence in organs delineation? Testing a CE and FDA-approved Deep-Learning tool using multiple expert contours delineated on planning CT images.

Silvia Strolin, Miriam Santoro, Giulia Paolani, Ilario Ammendolia, Alessandra Arcelli, Anna Benini, Silvia Bisello, Raffaele Cardano, Letizia Cavallini, Elisa Deraco and 15 more

Open access · goldFull text read
In one paragraph

Article in Frontiers in oncology, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 17 papers.

0numbers the graph read from it
0cells of the map it votes in
17citing papers in PubMed
6.2field-weighted citation impact, top 3% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

17 citing papers in PubMed, 27 citations in OpenAlex.

  1. Article
  2. Article
  3. Article
  4. Review
  5. Article
  6. Article
  7. Article
  8. Article
  9. Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives.Strahlentherapie und Onkologie : Organ der Deutschen Rontgengesellschaft ... [et al] · 2025
    Review
  10. Article
  11. Performance evaluation of MVision AI Contour+ in gastric MALT lymphoma segmentation.Reports of practical oncology and radiotherapy : journal of Greatpoland Cancer Center in Poznan and Polish Society of Radiation Oncology · 2025
    Article
  12. Article
  13. Review
  14. Article
  15. Article
  16. Review
  17. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

25 authors at 2 institutions in 1 country.

Silvia StrolinDepartment of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Miriam SantoroDepartment of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Giulia PaolaniDepartment of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Ilario AmmendoliaDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Alessandra ArcelliDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Anna BeniniDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Silvia BiselloDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Raffaele CardanoDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Letizia CavalliniDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Elisa DeracoDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Costanza Maria DonatiDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Erika GaliettaDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Andrea GaluppiDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Alessandra GuidoDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Martina FerioliDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Viola LaghiDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Federica MediciDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Maria NtretaDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Natalya RazganiayevaDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Giambattista SiepeDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Giorgio TolentoDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Daria VallerossaDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Alice ZamagniDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Alessio Giuseppe MorgantiDepartment of Radiation Oncology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Lidia StrigariDepartment of Medical Physics, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
Azienda USL di Bologna · ITUniversity of Bologna · IT

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

Background: A CE- and FDA-approved cloud-based Deep learning (DL)-tool for automatic organs at risk (OARs) and clinical target volumes segmentation on computer tomography images is available. Before its implementation in the clinical practice, an independent external validation was conducted. Methods: At least a senior and two in training Radiation Oncologists (ROs) manually contoured the volumes of interest (VOIs) for 6 tumoral sites. The auto-segmented contours were retrieved from the DL-tool and, if needed, manually corrected by ROs. The level of ROs satisfaction and the duration of contouring were registered. Relative volume differences, similarity indices, satisfactory grades, and time saved were analyzed using a semi-automatic tool. Results: Seven thousand seven hundred sixty-five VOIs were delineated on the CT images of 111 representative patients. The median (range) time for manual VOIs delineation, DL-based segmentation, and subsequent manual corrections were 25.0 (8.0-115.0), 2.3 (1.2-8) and 10.0 minutes (0.3-46.3), respectively. The overall time for VOIs retrieving and modification was statistically significantly lower than for manual contouring (p<0.001). The DL-tool was generally appreciated by ROs, with 44% of vote 4 (well done) and 43% of vote 5 (very well done), correlated with the saved time (p<0.001). The relative volume differences and similarity indexes suggested a better inter-agreement of manually adjusted DL-based VOIs than manually segmented ones. Conclusions: The application of the DL-tool resulted satisfactory, especially in complex delineation cases, improving the ROs inter-agreement of delineated VOIs and saving time.

Indexed as

deep learning toolindependent external validationquality metricssegmentationtime saved

Identifiers

PMID36937399
PMCPMC10019504
OpenAlexW4322766744

What OpenQuestion holds

Textfull text, public
LicenceCC BY
reference markers read1
measurements read48
table measurements read1
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.