Ayush Tewari - Self-Supervised 3D Digitization of Faces

Описание: March 9th, 2021. MIT CSAIL

Abstract: Photorealistic and semantically controllable digital models of human faces are important for a wide range of applications in movies, virtual reality, and casual photography. Recent approaches have explored digitizing faces from a single image using priors commonly known as 3D morphable models (3DMMs). In this talk, I will discuss methods for high-quality monocular 3D reconstruction by learning 3DMMs from 2D data such as videos. Learning from 2D data allows for better generalization compared to training on limited 3D scans. I will also talk about methods for photorealistic editing of portrait images using these 3D models. Our method learns a mapping between the latent spaces of a 3DMM and StyleGAN, enabling semantically meaningful control over portrait images. Through these methods, I will demonstrate that a single 3D face scan, combined with image and video datasets can enable disentangled editing of the head pose, scene illumination, and facial expressions in portrait images at photorealistic quality.

Bio : Ayush Tewari is a Ph.D. student working with Prof. Christian Theobalt at the Max Planck Institute for Informatics in Saarbruecken, Germany. He received his M.Sc. in Computer Science from Grenoble INP, and B.Tech. in Computer Science and Engineering from IIIT Hyderabad. His research interests are in computer vision, computer graphics, and machine learning, with a focus on self-supervised 3D reconstruction and synthesis problems.