Neural Textures
Neural textures optimized over a video of a single person via backpropagation. Code is available in the Neural Textures repository.
We propose a new type of full-body human avatars, which combines parametric mesh-based body model with a neural texture. We show that with the help of neural textures, such avatars can successfully model clothing and hair, which usually poses a problem for mesh-based approaches. We also show how these avatars can be created from multiple frames of a video using backpropagation. We then propose a generative model for such avatars that can be trained from datasets of images and videos of people. The generative model allows us to sample random avatars as well as to create dressed avatars of people from one or few images.
Our generative architecture is based on the combination of StyleGANv2 and neural dressing. The StyleGAN part is used to generate neural textures, which are then superimposed on SMPL-X meshes and rendered with a neural renderer. During adversarial learning, the discriminator considers a pair of images of the same person.
Neural textures optimized over a video of a single person via backpropagation. Code is available in the Neural Textures repository.
A generative model that samples random neural textures and fits the latent code to one or a few images of a person. Code is available in the StylePeople repository.
For this project we collected a new dataset, TEDXPeople.
@InProceedings{Grigorev_2021_CVPR,
author = {Grigorev, Artur and Iskakov, Karim and Ianina, Anastasia and Bashirov, Renat and Zakharkin, Ilya and Vakhitov, Alexander and Lempitsky, Victor},
title = {StylePeople: A Generative Model of Fullbody Human Avatars},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2021},
pages = {5151-5160}
}