Florian Fervers

Fraunhofer IOSB

Karlsruhe, Germany

Welcome! I am a PhD student working at Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB) in cooperation with the Computer Vision for Human-Computer Interaction Lab (CV:HCI) at Karlsruhe Institute of Technology (KIT) under supervision of Prof. Rainer Stiefelhagen. I completed my Masters in Computer Science at KIT in 2020 with an emphasis on machine learning and robotics. My current research focuses on GNSS-free self-localization by exploiting globally available aerial imagery.

News

Jul 1, 2024	Our work Statewide Visual Geolocalization in the Wild has been accepted at ECCV 2024.
Dec 14, 2023	Published einx and weightbridge.
Jun 28, 2023	I have been invited to hold a talk on Cross-view Geolocalization at the 18th Dortmunder Autotag.
Apr 16, 2023	I have been invited to hold a talk on Cross-view Geolocalization at the CVPR 2023 tutorial A Comprehensive Tour and Recent Advancements toward Real-world Visual Geo-Localization.
Feb 27, 2023	Our work Uncertainty-aware Vision-based Metric Cross-view Geolocalization has been accepted at CVPR 2023.

Selected Publications

Uncertainty-Aware Vision-Based Metric Cross-View Geolocalization

Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner, Michael Arens, and Rainer Stiefelhagen

CVPR 2023

Abstract arXiv Code Video Project page

This paper proposes a novel method for vision-based metric cross-view geolocalization (CVGL) that matches the camera images captured from a ground-based vehicle with an aerial image to determine the vehicle’s geo-pose. Since aerial images are globally available at low cost, they represent a potential compromise between two established paradigms of autonomous driving, i.e. using expensive high-definition prior maps or relying entirely on the sensor data captured at runtime. We present an end-to-end differentiable model that uses the ground and aerial images to predict a probability distribution over possible vehicle poses. We combine multiple vehicle datasets with aerial images from orthophoto providers on which we demonstrate the feasibility of our method. Since the ground truth poses are often inaccurate w.r.t. the aerial images, we implement a pseudo-label approach to produce more accurate ground truth poses and make them publicly available. While previous works require training data from the target region to achieve reasonable localization accuracy (i.e. same-area evaluation), our approach overcomes this limitation and outperforms previous results even in the strictly more challenging cross-area case. We improve the previous state-of-the-art by a large margin even without ground or aerial data from the test region, which highlights the model’s potential for global-scale application. We further integrate the uncertainty-aware predictions in a tracking framework to determine the vehicle’s trajectory over time resulting in a mean position error on KITTI-360 of 0.78m.
Continuous self-localization on aerial images using visual and lidar sensors

Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner, Michael Arens, and Rainer Stiefelhagen

IROS 2022

Abstract arXiv Project page

This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle’s sensor information with aerial imagery of an unseen target region. Geo-tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used for this purpose. The proposed geo-tracking method aligns data from on-board cameras and lidar sensors with geo-registered orthophotos to continuously localize a vehicle. We train a model in a metric learning setting to extract visual features from ground and aerial images. The ground features are projected into a top-down perspective via the lidar points and are matched with the aerial features to determine the relative pose between vehicle and orthophoto. Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos. It exhibits strong generalization, is robust to changes in the environment and requires only geo-poses as ground truth. We evaluate our approach on the KITTI-360 dataset and achieve a mean absolute position error (APE) of 0.94m. We further compare with previous approaches on the KITTI odometry dataset and achieve state-of-the-art results on the geo-tracking task.
Improving Semantic Image Segmentation via Label Fusion in Semantically Textured Meshes

Florian Fervers, Timo Breuer, Gregor Stachowiak, Sebastian Bullinger, Christoph Bodensteiner, and Michael Arens

VISAPP 2022

Abstract arXiv Code

Models for semantic segmentation require a large amount of hand-labeled training data which is costly and time-consuming to produce. For this purpose, we present a label fusion framework that is capable of improving semantic pixel labels of video sequences in an unsupervised manner. We make use of a 3D mesh representation of the environment and fuse the predictions of different frames into a consistent representation using semantic mesh textures. Rendering the semantic mesh using the original intrinsic and extrinsic camera parameters yields a set of improved semantic segmentation images. Due to our optimized CUDA implementation, we are able to exploit the entire c-dimensional probability distribution of annotations over c classes in an uncertainty-aware manner. We evaluate our method on the Scannet dataset where we improve annotations produced by the state-of-the-art segmentation network ESANet from 52.05% to 58.25% pixel accuracy. We publish the source code of our framework online to foster future research in this area. To the best of our knowledge, this is the first publicly available label fusion framework for semantic image segmentation based on meshes with semantic textures.