Incorporating depth information into few-shot semantic segmentation
Abstract
Few-shot segmentation presents a significant challenge
for semantic scene understanding under limited supervision.
Namely, this task targets at generalizing the segmentation
ability of the model to new categories given a few samples.
In order to obtain complete scene information, we extend the
RGB-centric methods to take advantage of complementary depth
information. In this paper, we propose a two-stream deep neural
network based on metric learning. Our method, known as RDNet,
learns class-specific prototype representations within RGB and
depth embedding spaces, respectively. The learned prototypes
provide effective semantic guidance on the corresponding RGB
and depth query image, leading to more accurate performance.
Moreover, we build a novel outdoor scene dataset, known as
Cityscapes-3i, using labeled RGB images and depth images
from the Cityscapes dataset. We also perform ablation studies
to explore the effective use of depth information in few-shot
segmentation tasks. Experiments on Cityscapes-3i show that our
method achieves excellent results with visual and complementary
geometric cues from only a few labeled examples.
Origin : Files produced by the author(s)
Loading...