Skip to Main content Skip to Navigation
Conference papers

Incorporating depth information into few-shot semantic segmentation

Abstract : Few-shot segmentation presents a significant challenge for semantic scene understanding under limited supervision. Namely, this task targets at generalizing the segmentation ability of the model to new categories given a few samples. In order to obtain complete scene information, we extend the RGB-centric methods to take advantage of complementary depth information. In this paper, we propose a two-stream deep neural network based on metric learning. Our method, known as RDNet, learns class-specific prototype representations within RGB and depth embedding spaces, respectively. The learned prototypes provide effective semantic guidance on the corresponding RGB and depth query image, leading to more accurate performance. Moreover, we build a novel outdoor scene dataset, known as Cityscapes-3i, using labeled RGB images and depth images from the Cityscapes dataset. We also perform ablation studies to explore the effective use of depth information in few-shot segmentation tasks. Experiments on Cityscapes-3i show that our method achieves excellent results with visual and complementary geometric cues from only a few labeled examples.
Complete list of metadatas

Cited literature [31 references]  Display  Hide  Download

https://hal-univ-evry.archives-ouvertes.fr/hal-02887063
Contributor : Désiré Sidibé <>
Submitted on : Wednesday, July 1, 2020 - 11:55:33 PM
Last modification on : Thursday, July 9, 2020 - 3:49:01 PM
Long-term archiving on: : Friday, September 25, 2020 - 10:15:56 AM

File

ICPR_2020_YZ_DS_OM_FM.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02887063, version 1

Citation

Yifei Zhang, Désiré Sidibé, Olivier Morel, Fabrice Meriaudeau. Incorporating depth information into few-shot semantic segmentation. 25th International Conference on Pattern Recognition (ICPR 2020), Jan 2021, Milan, Italy. ⟨hal-02887063⟩

Share

Metrics

Record views

52

Files downloads

71