Evidence map›Paper›PMID 33458300›Full record

ArticlePhysics and imaging in radiation oncology2020

Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy.

Femke Vaassen, Colien Hazelaar, Ana Vaniqui, Mark Gooding, Brent van der Heyden, Richard Canters, Wouter van Elmpt

Open access · goldAbstract read
In one paragraph

Article in Physics and imaging in radiation oncology, 2020. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 120 papers, 1 of them a synthesis that pooled it.

0numbers the graph read from it
0cells of the map it votes in
120citing papers in PubMed, 1 pooled it
18.0field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

120 citing papers in PubMed, 1 synthesis or guideline pooled it, 192 citations in OpenAlex.

  1. Pooled it
  2. Article
  3. Review
  4. Article
  5. Article
  6. Review
  7. Quantifying manual editing time in autocontouring via audit log analysis.Physics and imaging in radiation oncology · 2026
    Article
  8. Article
  9. Article
  10. Article
  11. Article
  12. Predicting the Effort Required to Manually Mend Auto-Segmentations.IEEE journal of biomedical and health informatics · 2026
    Article
  13. Article
  14. Article
  15. Article
  16. Review
  17. Article
  18. Article
  19. Article
  20. Article

60 more citing papers are in PubMed but not listed here.

4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

7 authors at 3 institutions in 2 countries.

Femke VaassenDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Colien HazelaarDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Ana VaniquiDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Mark GoodingMirada Medical Ltd., Oxford, United Kingdom.
Brent van der HeydenDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Richard CantersDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Wouter van ElmptDepartment of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Maastricht University Medical Centre · NLMaastro Clinic · NLMirada Medical (United Kingdom) · GB

Funding

No grant is acknowledged in the PubMed record.

6 · The paper itself

Abstract

background and purposeIn radiotherapy, automatic organ-at-risk segmentation algorithms allow faster delineation times, but clinically relevant contour evaluation remains challenging. Commonly used measures to assess automatic contours, such as volumetric Dice Similarity Coefficient (DSC) or Hausdorff distance, have shown to be good measures for geometric similarity, but do not always correlate with clinical applicability of the contours, or time needed to adjust them. This study aimed to evaluate the correlation of new and commonly used evaluation measures with time-saving during contouring. MATERIALS AND

methodsTwenty lung cancer patients were used to compare user-adjustments after atlas-based and deep-learning contouring with manual contouring. The absolute time needed (s) of adjusting the auto-contour compared to manual contouring was recorded, from this relative time-saving (%) was calculated. New evaluation measures (surface DSC and added path length, APL) and conventional evaluation measures (volumetric DSC and Hausdorff distance) were correlated with time-recordings and time-savings, quantified with the Pearson correlation coefficient, R.

resultsThe highest correlation (R = 0.87) was found between APL and absolute adaption time. Lower correlations were found for APL with relative time-saving (R = -0.38), for surface DSC with absolute adaption time (R = -0.69) and relative time-saving (R = 0.57). Volumetric DSC and Hausdorff distance also showed lower correlation coefficients for absolute adaptation time (R = -0.32 and 0.64, respectively) and relative time-saving (R = 0.44 and -0.64, respectively).

conclusionSurface DSC and APL are better indicators for contour adaptation time and time-saving when using auto-segmentation and provide more clinically relevant and better quantitative measures for automatically-generated contour quality, compared to commonly-used geometry-based measures.

Indexed as

Added path lengthAutomatic delineationContouring timeDice similarity coefficientHausdorff distanceRadiotherapySurface DSCTime-saving

Identifiers

PMID33458300
PMCPMC7807544
OpenAlexW2994739006

What OpenQuestion holds

Textmetadata
LicenceCC BY-NC-ND
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.