Skip to Main content Skip to Navigation
New interface
Journal articles

A central multimodal fusion framework for outdoor scene image segmentation

Yifei Zhang 1 Olivier Morel 1 Ralph Seulin 1 Fabrice Mériaudeau 1 Désiré Sidibé 2 
1 VIBOT - Equipe VIBOT - VIsion pour la roBOTique [ImViA EA7535 - ERL CNRS 6000]
CNRS - Centre National de la Recherche Scientifique : ERL 6000, ImViA - Imagerie et Vision Artificielle [Dijon]
Abstract : Robust multimodal fusion is one of the challenging research problems in semantic scene understanding. In real-world applications, the fusion system can overcome the drawbacks of individual sensors by taking different feature representations and statistical properties of multiple modalities (e.g., RGB-depth cameras, multispectral cameras). In this paper, we propose a novel central multimodal fusion framework for semantic image segmentation of road scenes, aiming to effectively learn joint feature representations and optimally combine deep neural networks with statistical priors. More specifically, the proposed fusion framework can automatically generate a central branch by sequentially mapping multimodal features into a common space, including both low-level and high-level features. Besides, in order to reduce the model uncertainty, we employ statistical fusion to compute the final prediction, which leads to significant performance improvement. We conduct extensive experiments on various outdoor scene datasets. Both qualitative and quantitative experiments demonstrate that our central fusion framework achieves competitive performance against existing multimodal fusion methods.
Document type :
Journal articles
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03160284
Contributor : Frédéric Davesne Connect in order to contact the contributor
Submitted on : Friday, March 5, 2021 - 9:52:16 AM
Last modification on : Thursday, August 4, 2022 - 5:07:03 PM

Identifiers

Citation

Yifei Zhang, Olivier Morel, Ralph Seulin, Fabrice Mériaudeau, Désiré Sidibé. A central multimodal fusion framework for outdoor scene image segmentation. Multimedia Tools and Applications, 2022, 81, pp.12047-12060. ⟨10.1007/s11042-020-10357-y⟩. ⟨hal-03160284⟩

Share

Metrics

Record views

146