Modeling Indirect Illumination for Inverse Rendering

Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is r...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) S. 18622 - 18631
Hauptverfasser: Zhang, Yuanqing, Sun, Jiaming, He, Xingyi, Fu, Huan, Jia, Rongfei, Zhou, Xiaowei
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.06.2022
Schlagworte:
Computational modeling
Image sensors
Lighting
Pipelines
Rendering (computer graphics)
Shape
Three-dimensional displays
Vision + graphics; 3D from multi-view and sensors; Image and video synthesis and generation; Physics-based vision and shape-from-X
ISSN:1063-6919
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is rarely modeled in previous methods, as it requires expensive recursive path tracing which makes the inverse rendering computationally intractable. In this paper, we propose a novel approach to efficiently recovering spatially-varying indirect illumination. The key insight is that indirect illumination can be conveniently derived from the neural radiance field learned from input images instead of being estimated jointly with direct illumination and materials. By properly modeling the indirect illumination and visibility of direct illumination, interreflection- and shadow-free albedo can be recovered. The experiments on both synthetic and real data demonstrate the superior performance of our approach compared to previous work and its capambility to synthesize realistic renderings under novel view-points and illumination. Our code and data are available at https://zju3dv.github.io/invrender/.
ISSN:1063-6919
DOI:10.1109/CVPR52688.2022.01809