Multiscale Attention-Based Prototypical Network For Few-Shot Semantic Segmentation

Yifei Zhang (1) , Désiré Sidibé (2) , Olivier Morel (1) , Fabrice Meriaudeau (1)

1 VIBOT - Equipe VIBOT - VIsion pour la roBOTique [ImViA EA7535 - ERL CNRS 6000]
2 IBISC - Informatique, BioInformatique, Systèmes Complexes

Deep learning-based image understanding techniques require a large number of labeled images for training. Few-shot semantic segmentation, on the contrary, aims at generalizing the segmentation ability of the model to new categories given only a few labeled samples. To tackle this problem, we propose a novel prototypical network (MAPnet) with multiscale feature attention. To fully exploit the representative features of target classes, we firstly extract rich contextual information of labeled support images via a multiscale feature enhancement module. The learned prototypes from support features provide further semantic guidance on the query image. Then we adaptively integrate multiple similarity-guided probability maps by attention mechanism, yielding an optimal pixel-wise prediction. Furthermore, the proposed method was validated on the PASCAL-5 i dataset in terms of 1-way N-shot evaluation. We also test the model with weak annotations, including scribble and bounding box annotations. Both the qualitative and quantitative results demonstrate the advantages of our approach over other state-of-the-art methods.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Yifei Zhang, Désiré Sidibé, Olivier Morel, Fabrice Meriaudeau. Multiscale Attention-Based Prototypical Network For Few-Shot Semantic Segmentation. 25th International Conference on Pattern Recognition (ICPR 2020), Jan 2021, Milan, Italy. pp.7372--7378. ⟨hal-02977830⟩