Learning Spatial-Semantic Features for Robust Video Object Segmentation [pdf][code]

Xin Li, Deshui Miao, Zhenyu He, Yaowei Wang, Huchuan Lu, Ming-Hsuan Yang

• In this paper, we propose a robust video object segmentation framework equipped with spatial-semantic features and discriminative object queries to address the above issues. Specifically, we construct a spatial-semantic network comprising a semantic embedding block and spatial dependencies modeling block to associate the pretrained ViT features with global semantic features and local spatial features, providing a comprehensive target representation. In addition, we develop a masked cross-attention module to generate object queries that focus on the most discriminative parts of target objects during query propagation, alleviating noise accumulation and ensuring effective long-term query propagation.

Spatial-Temporal Multi-level Association for Video Object Segmentation [pdf][code]

Deshui Miao, Xin Li, Zhenyu He, Huchuan Lu, Ming-Hsuan Yang

European Conference on Computer Vision (ECCV) 2024

• we propose a spatial-temporal memory to assist feature association and temporal ID assignment and correlation. We evaluate the proposed method by conducting extensive experiments on numerous video object segmentation datasets, including DAVIS 2016/2017 val, DAVIS 2017 test-dev, and YouTube-VOS 2018/2019 val. The favorable performance against the state-of-the-art methods demonstrates the effectiveness of our approach.

Context-Guided Black-Box Attack for Visual Tracking [pdf]

Xingsen Huang, Deshui Miao, Hongpeng Wang, Yaowei Wang, Xin Li

IEEE Transactions on Multimedia (TMM)

• We propose a context-guided black-box attack method to investigate the robustness of recent advanced deep trackers against spatial and temporal interference.

Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th LSVOS [pdf] [code]

Learning Spatial-Semantic Features for Robust Video Object Segmentation [pdf] [code]

1st Place Solution for MOSE Track in CVPR 2024 PVUW Workshop: Complex Video Object Segmentation [pdf] [code]

2nd Place Solution for the LSVOS Challenge 2023: Video Object Segmentation [pdf]