Modeling Indirect Illumination for Inverse Rendering

Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is r...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 18622 - 18631
Main Authors: Zhang, Yuanqing, Sun, Jiaming, He, Xingyi, Fu, Huan, Jia, Rongfei, Zhou, Xiaowei
Format: Conference Proceeding
Language:English
Published: IEEE 01.06.2022
Subjects:
Computational modeling
Image sensors
Lighting
Pipelines
Rendering (computer graphics)
Shape
Three-dimensional displays
Vision + graphics; 3D from multi-view and sensors; Image and video synthesis and generation; Physics-based vision and shape-from-X
ISSN:1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is rarely modeled in previous methods, as it requires expensive recursive path tracing which makes the inverse rendering computationally intractable. In this paper, we propose a novel approach to efficiently recovering spatially-varying indirect illumination. The key insight is that indirect illumination can be conveniently derived from the neural radiance field learned from input images instead of being estimated jointly with direct illumination and materials. By properly modeling the indirect illumination and visibility of direct illumination, interreflection- and shadow-free albedo can be recovered. The experiments on both synthetic and real data demonstrate the superior performance of our approach compared to previous work and its capambility to synthesize realistic renderings under novel view-points and illumination. Our code and data are available at https://zju3dv.github.io/invrender/.
ISSN:1063-6919
DOI:10.1109/CVPR52688.2022.01809