SEED: A semantic endoscopic encoder duo for unified ROI segmentation
Authors: Huang, D., Zhu, Y., Tang, W., Liu, J.
Journal: Neurocomputing
Publication Date: 21/01/2027
Volume: 710
eISSN: 1872-8286
ISSN: 0925-2312
DOI: 10.1016/j.neucom.2026.135370
Abstract:Endoscopic image segmentation is critical but challenging, as it suffers from blurred boundaries and a lack of object-level semantic understanding, which together limit fine-grained segmentation in complex anatomical scenes. To address these issues, we propose SEED, a Semantic Endoscopic Encoder Duo for Unified Region of Interest (ROI) Segmentation. Concretely, SEED introduces an asymmetric, hierarchy-preserving dual-encoder framework that assigns complementary roles to spatial and semantic representation learning. A trainable hierarchical encoder preserves fine-grained spatial information, while a frozen semantic encoder provides object-level priors. Rather than directly combining their outputs, SEED coordinates the two branches through two purpose-specific mechanisms: a Structure-Guided Multi-Scale Adapter (SG-MSA) propagates deep structural context to high-resolution features for boundary refinement, whereas the Dense Glue Layer selectively injects dense semantic information into the high-resolution decoding pathways without altering the principal image representation used by the mask decoder. In addition, we construct the Endo-MultiSeg dataset, the first endoscopic ROI segmentation dataset covering diverse lesion types and anatomical structures, providing a resource for multi-category annotation in this field. Extensive experiments on Endo-MultiSeg and two public benchmarks demonstrate that SEED achieves the best overall performance among the evaluated methods. On Endo-MultiSeg, SEED outperforms SAM2 by 1.04 and 1.63 percentage points in Dice and IoU, respectively. Our code and dataset will be made available on request.
Source: Scopus