🎨 Viggle-Animate: character replacement in video from a single frame
Replaces a character in an existing video from a single repainted frame. Weights are on Hugging Face, and there is a browser demo. You only need the video and one of its frames with the new hero. 124 frames render in 26 seconds on an NVIDIA B200 — 6.1 times faster than Wan2.2-Animate-14B.
🌍 Auxiliary models (pose, segmentation, face tracking) are not needed, and sampling is reduced from ~30 to 3 passes — replacement is reduced to a single GPU without orchestration. Large video models are being adapted to niches through open fine-tuning.
👤 Locally, bf16 requires ~96 GB of VRAM, and offloading reduces this to ~12 GB at the cost of speed. Weak lip-sync, degradation in scenes with multiple heroes, and audio is not preserved. For commercial use, check the MiniMax H3 Community License.
Source 1: https://viggle.ai/blog/viggle-animate-character-replacement-from-a-repainted-frame Source 2: https://huggingface.co/Viggle/Viggle-Animate