Skip to Main content Skip to Navigation
Journal articles

Improving Image Description with Auxiliary Modality for Visual Localization in Challenging Conditions

Abstract : Image indexing for lifelong localization is a key component for a large panel of applications, including robot navigation, autonomous driving or cultural heritage valorization. The principal difficulty in long-term localization arises from the dynamic changes that affect outdoor environments. In this work, we propose a new approach for outdoor large scale image-based localization that can deal with challenging scenarios like cross-season, cross-weather and day/night localization. The key component of our method is a new learned global image descriptor, that can effectively benefit from scene geometry information during training. At test time, our system is capable of inferring the depth map related to the query image and use it to increase localization accuracy. We show through extensive evaluation that our method can improve localization performances, especially in challenging scenarios when the visual appearance of the scene has changed. Our method is able to leverage both visual and geometric clues from monocular images to create discriminative descriptors for cross-season localization and effective matching of images acquired at different time periods. Our method can also use weakly annotated data to localize night images across a reference dataset of daytime images. Finally we extended our method to reflectance modality and we compare multi-modal descriptors respectively based on geometry, material reflectance and a combination of both.
Complete list of metadata

Cited literature [95 references]  Display  Hide  Download
Contributor : Valérie Gouet-Brunet Connect in order to contact the contributor
Submitted on : Monday, October 26, 2020 - 11:05:22 AM
Last modification on : Thursday, August 4, 2022 - 5:07:03 PM
Long-term archiving on: : Wednesday, January 27, 2021 - 6:27:54 PM


Files produced by the author(s)



Nathan Piasco, Désiré Sidibé, Valérie Gouet-Brunet, Cédric Demonceaux. Improving Image Description with Auxiliary Modality for Visual Localization in Challenging Conditions. International Journal of Computer Vision, Springer Verlag, 2021, 129 (1), pp.185-202. ⟨10.1007/s11263-020-01363-6⟩. ⟨hal-02912239⟩



Record views


Files downloads