MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments From a Single Moving Camera

MCML Authors

Felix Wimbauer

→ Group Daniel Cremers
 Computer Vision & Artificial Intelligence

Daniel Cremers

Prof. Dr.

Director

Computer Vision & Artificial Intelligence

Abstract

In this paper, we propose MonoRec, a semi-supervised monocular dense reconstruction architecture that predicts depth maps from a single moving camera in dynamic environments. MonoRec is based on a multi-view stereo setting which encodes the information of multiple consecutive images in a cost volume. To deal with dynamic objects in the scene, we introduce a MaskModule that predicts moving object masks by leveraging the photometric inconsistencies encoded in the cost volumes. Unlike other multi-view stereo methods, MonoRec is able to reconstruct both static and moving objects by leveraging the predicted masks. Furthermore, we present a novel multi-stage training scheme with a semi-supervised loss formulation that does not require LiDAR depth values. We carefully evaluate MonoRec on the KITTI dataset and show that it achieves state-of-theart performance compared to both multi-view and singleview methods. With the model trained on KITTI, we furthermore demonstrate that MonoRec is able to generalize well to both the Oxford RobotCar dataset and the more challenging TUM-Mono dataset recorded by a handheld camera.

inproceedings

---

CVPR 2020

IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual, Jun 14-19, 2020.

---

Authors

F. Wimbauer • N. Yang • L. von Stumberg • N. Zeller • D. Cremers

---

Links

DOI GitHub

---

Research Area

 B1 | Computer Vision