Object-Centric State-Guided SAM for Temporally Coherent Multi-Object Segmentation in Complex Underwater Environments

Abstract

Robust perception in complex underwater environments is critical for marine resource exploration, ecological monitoring, and robotic operations, yet remains challenging due to low visibility, color distortion, dynamic backgrounds, and long-term occlusions. Although foundation models such as SAM 2 demonstrate strong segmentation performance on static imagery, their lack of explicit temporal modeling limits reliability in underwater video scenarios. To address this limitation, we propose AquaSlot-SAM, a hybrid object-centric temporal segmentation framework that integrates Slot-based State Space Models (SlotSSMs) with prompt-guided refinement in SAM 2. SlotSSMs explicitly maintain object-level temporal states, which are converted into adaptive prompts to guide SAM 2 toward spatially precise and temporally coherent segmentation. A sparse Slot Mixer enhances inter-object interaction, while confidence-aware mask fusion balances global stability and local boundary refinement. Extensive experiments on three public underwater benchmarks demonstrate that AquaSlot-SAM consistently outperforms strong contemporary baselines, achieving substantial improvements across multiple metrics and validating the effectiveness of structured memory-guided prompting for dynamic underwater segmentation.

Qiang Li

Professor and header of AG

My research interests include underwater robot, collaborative robots, humanoid robots.