Skip to Main content Skip to Navigation
New interface
Conference papers

Incorporating depth information into few-shot semantic segmentation

Yifei Zhang 1 Désiré Sidibé 2 Olivier Morel 1 Fabrice Meriaudeau 3 
1 VIBOT - Equipe VIBOT - VIsion pour la roBOTique [ImViA EA7535 - ERL CNRS 6000]
CNRS - Centre National de la Recherche Scientifique : ERL 6000, ImViA - Imagerie et Vision Artificielle [Dijon]
Abstract : Few-shot segmentation presents a significant challenge for semantic scene understanding under limited supervision. Namely, this task targets at generalizing the segmentation ability of the model to new categories given a few samples. In order to obtain complete scene information, we extend the RGB-centric methods to take advantage of complementary depth information. In this paper, we propose a two-stream deep neural network based on metric learning. Our method, known as RDNet, learns class-specific prototype representations within RGB and depth embedding spaces, respectively. The learned prototypes provide effective semantic guidance on the corresponding RGB and depth query image, leading to more accurate performance. Moreover, we build a novel outdoor scene dataset, known as Cityscapes-3i, using labeled RGB images and depth images from the Cityscapes dataset. We also perform ablation studies to explore the effective use of depth information in few-shot segmentation tasks. Experiments on Cityscapes-3i show that our method achieves excellent results with visual and complementary geometric cues from only a few labeled examples.
Complete list of metadata

Cited literature [31 references]  Display  Hide  Download
Contributor : Désiré Sidibé Connect in order to contact the contributor
Submitted on : Wednesday, July 1, 2020 - 11:55:33 PM
Last modification on : Thursday, August 4, 2022 - 5:07:03 PM
Long-term archiving on: : Friday, September 25, 2020 - 10:15:56 AM


Files produced by the author(s)


  • HAL Id : hal-02887063, version 1


Yifei Zhang, Désiré Sidibé, Olivier Morel, Fabrice Meriaudeau. Incorporating depth information into few-shot semantic segmentation. 25th International Conference on Pattern Recognition (ICPR 2020), Jan 2021, Milan, Italy. pp.3582--3588. ⟨hal-02887063⟩



Record views


Files downloads