VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentation

Fabian Isensee, Constantin Ulrich, Yannick Kirchhoff, Klaus Maier-Hein, Maximilian Rokuss, Moritz Langenberg, Benjamin Hamm, Sebastian Regnery, Lukas Bauer, Efthimios Katsigiannopulos, Tobias Norajitra

cvpr2026-06-01

Analysis: not analyzed

Local PDF: stored

Source PDF: linked

Source PDF

Research metadataShow details

Affiliations: German Cancer Research Center, Division of Medical Image Computing, Germany · Faculty of Mathematicsand Computer Scienceand3Medical Faculty-Heidelberg University · Helmholtz Imaging, · Department of Radiation Oncology, Heidelberg University Hospital, Germany · HIDSS4Health, Heidelberg7Pattern Analysisand Learning Group, Heidelberg University Hospital · a) known prompts b) cross-modality c) unknown concept d) clinical language Real-World Dataset · Figure1. Vox Tellperforms3Dmedicalimagesegmentationdirectlyfromarbitraryfree-textprompts.Thefigureshowsprogressively · modalities,(c)novelconceptsneverencounteredduringtraining,and(d)clinicallanguageunderstandingfromrealradiologyreportswith · shownin(d),where Vox Telloutperformspriortext-promptablesegmentationmethods.
Published: 2026-06-01
Processed: 6/20/2026, 6:26:23 PM
Analysis model: Not available
Analysis status: not_analyzed
Local PDF artifact: papers/pdf/2026/voxtell-free-text-promptable-universal-3d-medical-image-segm.pdf

Abstract

We introduce VoxTell, a vision-language model for text-prompted volumetric medical image segmentation. It maps free-form descriptions, from single words to full clinical sentences, to 3D masks. Trained on 62K+ CT, MRI, and PET volumes spanning 1K+ anatomical and pathological classes, VoxTell uses multi-stage vision-language fusion across decoder layers to align textual and visual features at multiple scales. It achieves state-of-the-art zero-shot performance across modalities on unseen datasets, excelling on familiar concepts while generalizing to related unseen classes. Extensive experiments further demonstrate strong cross-modality transfer, robustness to linguistic variations and clinical language, as well as accurate instance-specific segmentation from real-world text. Code is available at: https://github.com/MIC-DKFZ/VoxTell

Stored markdown is an ingestion scaffold, so it remains excluded until substantive methodology, result, and limitation sections are available.

Analysis unavailable

Only an ingestion scaffold is stored; substantive analysis is not available yet.

Additional figures

Figure 1. VoxTell performs 3D medical image segmentation directly from arbitrary free-text prompts. The figure shows progressively challenging scenarios: (a) known anatomical structures seen during training, (b) generalization of learned concepts to other imaging modalities, (c) novel concepts never encountered during training, and (d) clinical language understanding from real radiology reports with spatially grounded descriptions. The bar chart (right) reports Dice scores on a held-out radiotherapy cohort using report-derived prompts, shown in (d), where VoxTell outperforms prior text-promptable segmentation methods.

Figure 2. Overview of VoxTell. Left: A 3D image volume is encoded into latents, while a free-text prompt is first embedded and then processed by a prompt decoder to produce multi-scale text features that guide image decoding. Right: The decoder performs multi-stage vision–language fusion: at each resolution, text embeddings modulate volumetric features, extending MaskFormer-style query–image fusion to multiple scales with deep supervision.

Figure 4. Free-Text Segmentation on ReXGroundingCT. Eval- uation on the ReXGroundingCT benchmark [19] (validation set), which links radiology report findings to 3D segmentations in CT- RATE [68] chest CTs, assessing instance-level localization and segmentation from text. Following the benchmark protocol, both

Figure 3. Prompt Stability. Dice score distributions of all meth- ods across multiple textual prompts for the same anatomical struc- ture. Competing methods exhibit high variability, often failing on certain synonyms or misspellings, while VoxTell maintains consis- tently high performance, even on prompts not seen during training.

Figure 5. Qualitative comparison of text-prompted segmentation across varying prompt complexity. (a) Known anatomical concepts, (b) unseen pathological structures, and (c) sentence-level clinical descriptions from in-house radiology reports. VoxTell produces accurate segmentations across all prompt types, while competing methods struggle on in-distribution prompts and fail on unseen or complex queries. tial and semantic descriptions, e.g. ”spiculated carcinoma in the left upper lobe with pleural contact.” We attribute Vox-