🤖 LTX-Video: Faster-than-real-time video generation
Lightricks has published a technical article and an arXiv paper (arXiv:2501.00103) on the LTX-Video transformer diffusion model. The Video-VAE and denoising transformer are designed together: the VAE compresses video by a factor of 192 (one token represents a 32x32x8 pixel block), and the decoder immediately reconstructs pixels and cleans up noise without an upscaler. On an H100, the model creates 5 seconds of 768x512 video at 24 frames per second in approximately 2 seconds.
🌍 The unified design of the VAE and transformer with 1:192 compression drastically reduces generation costs: real-time video is now feasible for interactive applications, live content, and API services where latency is critical.
👤 The code and weights are available on GitHub and HuggingFace — the model can be run locally or via the LTX API and LTX Playground to independently verify the speed.
Source 1: https://ltx.io/technical-blog/ltx-video-realtime-video-latent-diffusion Source 2: https://arxiv.org/abs/2501.00103