FaceCraft4D: Animated 3D Facial Avatar Generation from a Single Image
TL;DR: Our model can create full-head 4D avatars from a single image.
Overview of FaceCraft4D. Our approach begins with estimating the geometry of a single input image using a Geometry Prior.
This geometry guides the synthesis of personalized multiview images, providing 360° views and varied expressions, with support from both 2D image and video priors.
Since the synthesized data often exhibit inconsistency across views, we propose COIN optimization for robust 4D optimization.
By fitting the model to the multiview data, we achieve our final 4D avatar.
Consistent-Inconsistent (COIN) Training. To address geometric and color inconsistencies between the views,
we train two 3D representations: a view-consistent base model (GaussianAvatar) and an MLP,
encoding inconsistencies between the views. The MLP with inconsistencies lets us robustly
reconstruct the high-quality 3D base model.