This paper introduces EccoMamba, a U-shaped encoder-decoder architecture for medical image segmentation. EccoMamba addresses limitations of existing Mamba-based models, such as their inability to capture hierarchical anatomical features and spatial dependencies due to fixed-direction sequence modeling. The proposed architecture enhances Mamba by incorporating a Hierarchical Aggregation Enhancement (HAE) module for multi-scale feature extraction and a Structural Continuity Orthogonal (SCO) module to mitigate directional bias and improve spatial continuity. Empirical results on four benchmark datasets demonstrate that EccoMamba consistently outperforms state-of-the-art methods in segmentation accuracy and structural fidelity.
Problem
The paper identifies several limitations in existing Mamba-based architectures for medical image segmentation:
Disruption of Spatial Structures: Mamba models flatten images into 1D sequences, which disrupts the intrinsic 2D or 3D spatial structures essential for medical imaging, hindering the capture of fine anatomical details.
Directional Bias: The unidirectional sequence processing in Mamba introduces directional bias, reducing its ability to symmetrically perceive horizontal and vertical structures.
Lack of Multi-Scale Mechanisms: The absence of explicit multi-scale mechanisms limits Mamba's capacity to model semantic variations across organs of different sizes, which is crucial for accurate segmentation.
Suboptimal Segmentation Performance: These limitations collectively lead to suboptimal segmentation performance, particularly in anatomically complex regions.
Key Contributions
A novel Hierarchical Aggregation Enhancement (HAE) module is proposed to address Mamba's limitations in modeling hierarchical anatomical features by integrating multi-scale convolutions and hierarchical attention.
The Structural Continuity Orthogonal (SCO) module is introduced to overcome Mamba's directional bias and spatial discontinuity by applying orthogonal Axial Shifts (AS) and feature fusion.
Extensive experiments on four benchmark datasets (ISIC 2018, ISIC 2017, Synapse, and ACDC) demonstrate the effectiveness and strong generalization capability of EccoMamba across diverse evaluation metrics.
Method
EccoMamba is a U-shaped encoder-decoder architecture that integrates selective state-space modeling with structural enhancement modules.
Architecture Overview: The architecture consists of a four-stage encoder that performs hierarchical feature extraction and progressive downsampling, followed by a decoder for upsampling and segmentation. Each encoder stage includes a Hierarchical Aggregation Enhancement (HAE) module and a Vision State Space (VSS) block. Skip connections are augmented with a Structural Continuity Orthogonal (SCO) module.
Hierarchical Aggregation Enhancement (HAE) Module: This module aims to capture hierarchical anatomical features across different scales while maintaining spatial coherence. It splits the input feature map into two parts: one for a multi-scale convolutional branch and another for a hierarchical attention branch. The convolutional branch uses parallel pathways with 1x1 pointwise convolution and depthwise separable 3x3 convolution for fine-grained features, and a 3x3 max-pooling operation for coarse-scale context. The attention branch processes features through channel attention and spatial attention mechanisms.
Structural Continuity Orthogonal (SCO) Module: This module mitigates directional bias and spatial discontinuity by explicitly modeling orthogonal dependencies across spatial axes. It processes the input feature map through distinct Axial Shifts (AS) branches, which involve shifting features horizontally and vertically, followed by convolution and fusion.
Loss/Objective: Hybrid loss of Dice loss and cross-entropy loss.
Optimization: AdamW optimizer with an initial learning rate of 1 × 10-3 and a weight decay of 1 × 10-2. Cosine learning rate decay schedule.
Augmentation: Not stated in the paper.
Data Handling: Batch size is uniformly set to 24 for all datasets. Sliding-window strategy used for inference on the Synapse dataset. Mixed-precision training (AMP) is used.
Implementation Details: Implemented in PyTorch on a single NVIDIA RTX 4090 GPU. Trained for 300 epochs. Best checkpoint selected based on validation DSC.
Results
Datasets: ISIC 2018, ISIC 2017, Synapse, and ACDC.
ACDC: Achieved highest DSC of 91.83%, outperforming VM-UNet (91.04%).
Synapse: Achieved highest average DSC (81.51%) and competitive HD95 of 22.02.
Ablation Study (ISIC 2018): Base+HAE improved DSC to 90.42%, Base+SCO to 90.22%, and EccoMamba (Base+HAE+SCO) to 91.18%.
Runtime/Training Cost: EccoMamba requires 38.30M parameters, 6.22 GFLOPs, and 0.63 GB of memory, offering a favorable trade-off between accuracy and complexity.
Weaknesses / Limitations
No explicit paper-reported limitations were found.
Evidence
Method evidence: "To address these limitations, we propose the Enhanced Cross-hierarchical Continuity Orthogonal Mamba (EccoMamba), a U-shaped encoder-decoder architecture tailored for medical image segmentation. EccoMamba enhances the Mamba backbone by improving its ability to model local spatial continuity, multi-scale anatomical structures, and orthogonal directional dependencies." (section: Introduction)
Result evidence: "Extensive experiments on four benchmark datasets (ISIC 2018, ISIC 2017, Synapse, and ACDC) demonstrate the effectiveness and strong generalization capability of EccoMamba across diverse evaluation metrics." (section: Key Contributions)
Limitation evidence: Not stated in the paper.
Additional Figures
Figure 1: (a) Overview of the proposed EccoMamba architecture. (b) Hierarchical Aggregation Enhancement (HAE) module. (c) Structural Continuity Orthogonal (SCO) module. (d) Axial Shifts (AS) applies orthogonal shifts (horizontal and vertical) and dual convolutional pathways to model fine-grained spatial dependencies. medical images. The input image is first projected into fea-
Figure 2: Visual comparison of EccoMamba and other mod- els on the ISIC 2017 dataset. The ground truth is shown in blue contours, while the segmentation results are shown in red.
Figure 3: Visual comparison of EccoMamba and other models on the Synapse dataset. Red boxes highlight regions where EccoMamba yields more accurate segmentations. Fig. 3 presents visual comparisons between EccoMamba and other competing models on the Synapse dataset. Red
Fig. 4 shows visual comparisons on the ACDC dataset. Even in regions with unclear boundaries, complex tissue morphology, or tightly packed structures, EccoMamba accu- rately delineates cardiac structures such as the LV, RV, and MYO. In contrast, other models exhibit issues such as dis-
Figure 5: Stage-wise placement analysis of the HAE mod- ule. Stage-wise Placement of the HAE Module