🎨 Video diffusion becomes a controllable 3D avatar from a single photo
Matthias Nessler's team at TU Munich released Video Diffusion Avatars (VDA): the WAN2.2-Animate-14B model is fine-tuned into a 3D avatar from a single photo (6.5 GB adapter, 34 hours on A100). The 'camera-triplet' approach (a moment captured from three cameras) teaches the model to think about the head in 3D and eliminates the dependence of facial expressions on the viewing angle.
🌍 Video diffusion models are now fine-tuned into 3D avatars instead of 3DMM and NeRF pipelines: VDA autonomously creates shadows, reflections, refractions, and hair dynamics. Creating characters for virtual speakers, VFX, and avatars is now easier.
👤 From a single photo, the camera can orbit the head along any trajectory, while facial expressions (down to the tongue) and body pose are controlled independently. It also works on drawn characters. Code and weights are open under CC BY-NC-SA 4.0 (non-commercial), requiring ~65 GB of disk space and a 48–80 GB GPU.
Source 1: https://simongiebenhain.github.io/video-diffusion-avatars/ Source 2: https://arxiv.org/abs/2610.23772