Speculative Sampling: Real-Time Interactive Video

The open-source, training-free Speculative Sampling method (arXiv:2610.20556) generates a large chunk with noise padding for interactive video, caches predictions, and recalculates only the tail when the input changes, removing the latency lower bound tied to chunk size; the code is published on GitHub under the Apache 2.0 license.

Compiled by Sergey KostenchukPublished 2026-10-26Updated 2026-10-26

2026-10-26 Research NVIDIA

🤖 Speculative Sampling: Real-Time Interactive Video

A Speculative Sampling method for interactive video generation has been released (arXiv:2610.20556), with the code open-sourced under the Apache 2.0 license. This training-free approach involves the model generating a large chunk with noise padding and caching predictions, which are then output as playback progresses; when the input changes, only the tail is recalculated.

🌍 Bidirectional video models with large windows can be served in real time without post-training in the style of Self-Forcing: the focus shifts from reducing chunk size to accelerating sampling. This is a path toward interactive games and simulations using existing checkpoints.

👤 You can run the demo yourself: the repository is at github.com/Yubo-Shankui/SpeculativeSampling and the script is h3_live_demo.py (requires 8 NVIDIA B300 GPUs). Scenarios include free TI2V interaction and a boss fight with Margit from Elden Ring.

Source 1: https://arxiv.org/abs/2610.20556 Source 2: https://yubo-shankui.github.io/speculative_sampling/

Sources