University of Oulu

L. Huynh, P. Nguyen, J. Matas, E. Rahtu and J. Heikkilä, "Boosting Monocular Depth Estimation with Lightweight 3D Point Fusion," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 12747-12756, doi: 10.1109/ICCV48922.2021.01253

Boosting monocular depth estimation with lightweight 3D point fusion

Saved in:
Author: Huynh, Lam1; Nguyen, Phong1; Matas, Jiri2;
Organizations: 1University of Oulu
2Czech Technical University in Prague
3Tampere University
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 7.2 MB)
Persistent link:
Language: English
Published: IEEE Computer Society, 2021
Publish Date: 2022-03-04


In this paper, we propose enhancing monocular depth estimation by adding 3D points as depth guidance. Unlike existing depth completion methods, our approach performs well on extremely sparse and unevenly distributed point clouds, which makes it agnostic to the source of the 3D points. We achieve this by introducing a novel multi-scale 3D point fusion network that is both lightweight and efficient. We demonstrate its versatility on two different depth estimation problems where the 3D points have been acquired with conventional structure-from-motion and Li-DAR. In both cases, our network performs on par with state-of-the-art depth completion methods and achieves significantly higher accuracy when only a small number of points is used while being more compact in terms of the number of parameters. We show that our method outperforms some contemporary deep learning based multi-view stereo and structure-from-motion methods both in accuracy and in compactness.

see all

Series: IEEE International Conference on Computer Vision
ISSN: 1550-5499
ISSN-E: 2380-7504
ISSN-L: 1550-5499
ISBN: 978-1-6654-2812-5
ISBN Print: 978-1-6654-2813-2
Pages: 12747 - 12756
DOI: 10.1109/ICCV48922.2021.01253
Host publication: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Conference: IEEE/CVF International Conference on Computer Vision Workshop
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: Infotech project Vision-based 3D perception for mixed reality applications (Prof. Janne Heikkilä).
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.