Evidence map›Paper›PMID 37725868›Full record

ArticleMedical image analysis2023

UNesT: Local spatial representation learning with hierarchical transformer for efficient medical segmentation.

Xin Yu, Qi Yang, Yinchi Zhou, Leon Y Cai, Riqiang Gao, Ho Hin Lee, Thomas Li, Shunxing Bao, Zhoubing Xu, Thomas A Lasko and 5 more

Open access · greenAbstract read
In one paragraph

Article in Medical image analysis, 2023. The graph could read no effect estimate from its abstract, so it casts no vote on the map. Cited by 31 papers.

0numbers the graph read from it
0cells of the map it votes in
31citing papers in PubMed
11.0field-weighted citation impact, top 1% of its field
1 · What the graph read from it

What it found

Each row is one number read from the abstract, on the scale the paper reported it, with its interval. Left of the dashed line favours the treatment, right favours the comparator. Under each row is the sentence it came from. New to these charts? A ten-minute tutorial.

The abstract states no effect estimate the extractor could read, or names no intervention and outcome on the map, so this paper lights no cell and moves no belief. It is still indexed, cited and linked below.

2 · The registry

The trial behind it

Trials whose registry record cites this paper, or whose number appears in the abstract. A trial that started after this paper was published is citing it as background, not reporting it.

Neither the registry nor the abstract names a trial number. If this is a trial report, that itself is worth knowing.

3 · Its place in the literature

Who cites it

31 citing papers in PubMed, 91 citations in OpenAlex.

  1. Article
  2. Article
  3. Article
  4. Review
  5. Article
  6. Article
  7. Phenotype discovery of traumatic brain injury segmentations from heterogeneous multi-site data.Proceedings of SPIE--the International Society for Optical Engineering · 2026
    Article
  8. Article
  9. Foundation Models Meet Medical Image Interpretation.Research (Washington, D.C.) · 2026
    Review
  10. Review
  11. Article
  12. Top-k Bottom All butDiagnostics (Basel, Switzerland) · 2025
    Article
  13. Article
  14. Article
  15. Article
  16. Scalable, reproducible, and cost-effective processing of large-scale medical imaging datasets.Proceedings of SPIE--the International Society for Optical Engineering · 2025
    Article
  17. A 4D atlas of diffusion-informed spatial smoothing windows for BOLD signal in white matter.Proceedings of SPIE--the International Society for Optical Engineering · 2025
    Article
  18. Article
  19. Assessing the role of volumetric brain information in multiple sclerosis progression.Computational and structural biotechnology journal · 2025
    Article
  20. Article
4 · The record

Corrections and comments

PubMed lists nothing against this paper. Absence here is not a guarantee, only a check that was made.

5 · Who and what money

Authors and funding

15 authors at 5 institutions in 2 countries.

Xin YuDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA.
Qi YangDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA.
Yinchi ZhouDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA.
Leon Y CaiDepartment of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37212, USA.
Riqiang GaoDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA; Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, 08540, USA.
Ho Hin LeeDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA.
Thomas LiDepartment of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37212, USA.
Shunxing BaoDepartment of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, 37212, USA.
Zhoubing XuDigital Technology and Innovation, Siemens Healthineers, Princeton, NJ, 08540, USA.
Thomas A LaskoDepartment of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, 37212, USA.
Richard G AbramsonDepartment of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37212, USA; Annalise-AI, Pty, Ltd, USA.
Zizhao ZhangGoogle Cloud AI, USA.
Yuankai HuoDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA; Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, 37212, USA.
Bennett A LandmanDepartment of Computer Science, Vanderbilt University, Nashville TN, 37212, USA; Department of Biomedical Engineering, Vanderbilt University, Nashville, TN, 37212, USA; Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, 37212, USA; Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, 37212, USA.
Yucheng TangDepartment of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, 37212, USA; Nvidia Corporation, USA. Electronic address: yuchengt@nvidia.com.
Vanderbilt University · USGoogle (United States) · USNvidia (United Kingdom) · GBSiemens Healthcare (United States) · USVanderbilt University Medical Center · US

Funding

The Vanderbilt Institute for Clinical and Translational Research (VICTR)UL1TR000445 · NCATS · VANDERBILT UNIVERSITY MEDICAL CENTER · PI BERNARD, GORDON RAPHAEL · 2012 to 2016
$41.4M
MEDICAL SCIENTIST TRAINING PROGRAMT32GM007347 · NIGMS · VANDERBILT UNIVERSITY · PI WILLIAMS, CHRISTOPHER S. · 1985 to 2023
$26.3M
Vanderbilt University Biomolecular Multimodal Imaging Center for 3-Dimensional Tissue MappingU54DK120058 · NIDDK · VANDERBILT UNIVERSITY · PI CAPRIOLI, RICHARD M, SPRAGGINS, JEFFREY M · 2018 to 2021
$5.7M
Controlling Quality and Capturing Uncertainty in Advanced Diffusion Weighted MRIR01EB017230 · NIBIB · VANDERBILT UNIVERSITY · PI LANDMAN, BENNETT A. · 2015 to 2024
$5.1M
Medical Scientist Training ProgramT32GM152284 · NIGMS · VANDERBILT UNIVERSITY · PI Christopher S. Williams · 2024 to 2026
$4.8M
Computer-Assisted functional neurosurgeryR01EB006136 · NIBIB · VANDERBILT UNIVERSITY · PI DAWANT, BENOIT M. · 2006 to 2013
$4.0M
Computer-Assited Functional NeurosurgeryR01NS095291 · NINDS · VANDERBILT UNIVERSITY · PI DAWANT, BENOIT M., DHAESE, PIERRE F. · 2015 to 2018
$2.4M
NCATS NIH HHS UL1 TR000445NIBIB NIH HHS R01 EB006136NIBIB NIH HHS R01 EB017230NIDDK NIH HHS U54 DK120058NIGMS NIH HHS T32 GM007347NIGMS NIH HHS T32 GM152284NINDS NIH HHS R01 NS095291
6 · The paper itself

Abstract

Transformer-based models, capable of learning better global dependencies, have recently demonstrated exceptional representation learning capabilities in computer vision and medical image analysis. Transformer reformats the image into separate patches and realizes global communication via the self-attention mechanism. However, positional information between patches is hard to preserve in such 1D sequences, and loss of it can lead to sub-optimal performance when dealing with large amounts of heterogeneous tissues of various sizes in 3D medical image segmentation. Additionally, current methods are not robust and efficient for heavy-duty medical segmentation tasks such as predicting a large number of tissue classes or modeling globally inter-connected tissue structures. To address such challenges and inspired by the nested hierarchical structures in vision transformer, we proposed a novel 3D medical image segmentation method (UNesT), employing a simplified and faster-converging transformer encoder design that achieves local communication among spatially adjacent patch sequences by aggregating them hierarchically. We extensively validate our method on multiple challenging datasets, consisting of multiple modalities, anatomies, and a wide range of tissue classes, including 133 structures in the brain, 14 organs in the abdomen, 4 hierarchical components in the kidneys, inter-connected kidney tumors and brain tumors. We show that UNesT consistently achieves state-of-the-art performance and evaluate its generalizability and data efficiency. Particularly, the model achieves whole brain segmentation task complete ROI with 133 tissue classes in a single network, outperforming prior state-of-the-art method SLANT27 ensembled with 27 networks. Our model performance increases the mean DSC score of the publicly available Colin and CANDI dataset from 0.7264 to 0.7444 and from 0.6968 to 0.7025, respectively. Code, pre-trained models, and use case pipeline are available at: https://github.com/MASILab/UNesT.

Indexed as

Imaging, Three-DimensionalAlgorithmsHumansImage Processing, Computer-AssistedMachine LearningMagnetic Resonance ImagingHierarchical transformerRenal substructure segmentationWhole brain segmentation

Identifiers

PMID37725868
PMCPMC11229077
OpenAlexW4386160755

What OpenQuestion holds

Textmetadata
LicenceTDM
Read underepoch 390

Registered trials

None linked

Read under generation 80e0d062 · epoch 390. Bibliography from PubMed, PubMed Central and OpenAlex; grants from NIH RePORTER; trial links from ClinicalTrials.gov; estimates, votes and beliefs from the OpenQuestion graph.