Alert button

Relighting4D: Neural Relightable Human from Videos

Jul 14, 2022
Zhaoxi Chen, Ziwei Liu

Figure 1 for Relighting4D: Neural Relightable Human from Videos
Figure 2 for Relighting4D: Neural Relightable Human from Videos
Figure 3 for Relighting4D: Neural Relightable Human from Videos
Figure 4 for Relighting4D: Neural Relightable Human from Videos

Share this with someone who'll enjoy it:

Human relighting is a highly desirable yet challenging task. Existing works either require expensive one-light-at-a-time (OLAT) captured data using light stage or cannot freely change the viewpoints of the rendered body. In this work, we propose a principled framework, Relighting4D, that enables free-viewpoints relighting from only human videos under unknown illuminations. Our key insight is that the space-time varying geometry and reflectance of the human body can be decomposed as a set of neural fields of normal, occlusion, diffuse, and specular maps. These neural fields are further integrated into reflectance-aware physically based rendering, where each vertex in the neural field absorbs and reflects the light from the environment. The whole framework can be learned from videos in a self-supervised manner, with physically informed priors designed for regularization. Extensive experiments on both real and synthetic datasets demonstrate that our framework is capable of relighting dynamic human actors with free-viewpoints.

* ECCV 2022; Project Page https://frozenburning.github.io/projects/relighting4d Codes are available at https://github.com/FrozenBurning/Relighting4D  
View paper onarxiv icon

Share this with someone who'll enjoy it: