Depth Super-Resolution Meets Uncalibrated Photometric Stereo

Abstract : Input HR RGB images and LR depth maps Output HR albedo and depth maps Relighting Figure 1: Given an RGB-D sequence of n ≥ 4 low-resolution (320 × 240 px) depth maps and high-resolution (1280 × 1024 px) RGB images acquired from the same viewing angle but under varying, unknown lighting, high-resolution depth and reflectance maps are estimated by combining super-resolution and photometric stereo within a variational framework. Abstract A novel depth super-resolution approach for RGB-D sensors is presented. It disambiguates depth super-resolution through high-resolution photometric clues and, symmetrically , it disambiguates uncalibrated photometric stereo through low-resolution depth cues. To this end, an RGB-D sequence is acquired from the same viewing angle, while illuminating the scene from various uncalibrated directions. This sequence is handled by a variational framework which fits high-resolution shape and reflectance, as well as lighting , to both the low-resolution depth measurements and the high-resolution RGB ones. The key novelty consists in a new PDE-based photometric stereo regularizer which implicitly ensures surface regularity. This allows to carry out depth super-resolution in a purely data-driven manner, without the need for any ad-hoc prior or material calibration. Real-world experiments are carried out using an out-of-the-box RGB-D sensor and a hand-held LED light source.
Type de document :
Communication dans un congrès
Liste complète des métadonnées

Littérature citée [32 références]  Voir  Masquer  Télécharger

https://hal-normandie-univ.archives-ouvertes.fr/hal-02118570
Contributeur : Yvain Queau <>
Soumis le : vendredi 10 mai 2019 - 16:00:42
Dernière modification le : vendredi 21 juin 2019 - 10:04:12
Document(s) archivé(s) le : lundi 30 septembre 2019 - 21:24:27

Fichier

ICCVW17.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Songyou Peng, Bjoern Haefner, Yvain Quéau, Daniel Cremers. Depth Super-Resolution Meets Uncalibrated Photometric Stereo. 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), Oct 2017, Venice, Italy. pp.2961-2968, ⟨10.1109/ICCVW.2017.349⟩. ⟨hal-02118570⟩

Partager

Métriques

Consultations de la notice

23

Téléchargements de fichiers

30