The Venus team (Ant Group), in collaboration with Tsinghua University, has released Realtime-Venus — a full-duplex interaction system: the model listens and watches while it speaks, and can delegate background tasks to an external runtime without interrupting the conversation. Under the Apache 2.0 license, two 9B-parameter checkpoints are open — Realtime-Venus-Omni for audio and video processing and Realtime-Venus-Audio for voice conversations — as well as the Harness runtime, a demo, and a beta version of an Android app.

image
image

What happened

The Venus team (Ant Group), together with Tsinghua University, published Realtime-Venus — a full-duplex interaction system in which the model simultaneously listens, watches, and speaks. Two 9B-parameter checkpoints based on MiniCPM-o 4.5 (Omni-Flow) with a Qwen3-8B language backbone and a 40,960-token context have been released. Realtime-Venus-Omni handles audio and video and decides when to join a conversation on its own; Realtime-Venus-Audio is designed for voice conversations and synthesizes speech via discrete S3 tokens and a streaming flow-matching decoder. Background tasks are delegated through -queries on a shared causal timeline: the external Realtime-Venus-Harness runtime executes them — for example, searching for tickets in a city seen through the camera — without blocking the conversation, and returns the result to the same chat. Long-video memory works without fine-tuning. According to the arXiv:2609.13814 technical report, the Omni variant leads in 6 out of 8 video benchmarks among online models: 70.2% on StreamingBench and 81.3% on Daily-Omni; the Audio variant shows 78.0% on MMAU. On Full-Duplex-Bench v1.5, the system continues speech after "backchannels" in 97% of cases (C_RESUME metric) and reacts to real interruptions in 75% of cases (C_RESPOND metric).

Context

Full duplex is a mode in which a voice agent does not "hand over the microphone" in turns, but continuously listens and observes, even when it is speaking itself: this is how humans hold conversations. Until now, such a mode was mainly offered by cloud realtime APIs — GPT-4o Realtime is mentioned in the material as a comparative context — and live duplex behavior remained closed engineering of large labs. A separate complexity of this mode is that the agent must distinguish when the interlocutor is interrupting to redirect a thought, and when they are merely "backchanneling" without requiring a response; these scenarios are measured on the Full-Duplex-Bench v1.5 benchmark. The technical assembly of Realtime-Venus is built from known components: speech output via discrete S3 tokens and a streaming flow-matching decoder, the multimodal foundation MiniCPM-o 4.5, and the language backbone Qwen3-8B. The scientific interest here lies in the integration and training of duplex behavior, not in a new class of architectures: the architectural novelty is concentrated in the protocol — -queries on a shared causal timeline plus the external Realtime-Venus-Harness, which executes background tasks and returns the result to the same conversation.

Why this matters for the industry

For the industry, Realtime-Venus is an architectural template for an interactive agent that can be downloaded and run, not just another "talking" model. Two cycles — live duplex dialogue and asynchronous Harness — allow an LLM assistant to hold a conversation and execute external tasks in parallel: search, tool calls, working with what the camera sees, without pauses for "wait, let me check." The distinction between interruptions, "backchannels," and redirects, measured on Full-Duplex-Bench v1.5, sets a measurable standard for voice assistants: if independent measurements confirm the C_RESPOND 75% and C_RESUME 97% figures, the "duplex plus asynchronous delegation" pattern will likely be copied in open voice assistants, and Full-Duplex-Bench metrics may become a reporting standard for realtime models. Open 9B weights under Apache 2.0 move interactive voice agents from the category of cloud APIs to the category of locally deployable products: for startups, this is simultaneously pressure on API-wrapper margins and a window for their own products. The liveliness of interest is confirmed by the already-appeared third-party quantized variant of the Realtime-Venus-Omni-heretic-ARA checkpoint.

Why this matters for users

For researchers of multimodal and speech models, this is a rare open object for studying turn-taking, interruptions, and backchannels in full duplex. Developers of voice products can already download both 9B checkpoints from Hugging Face (Apache 2.0), deploy Realtime-Venus-Harness, and build a prototype of a duplex assistant: Audio — for voice lines and call scenarios, Omni — for video agents that decide for themselves when to join a conversation. Everything is available for independent verification: you can run the models on Full-Duplex-Bench v1.5 and StreamingBench, compare them with the figures from the arXiv:2609.13814 technical report, and evaluate the protocol on your own tasks. For quick evaluation without deployment, there is an online demo and a beta Android APK, although the mobile production case is still at an early stage.

What is still unknown / limitations

Mainly: the word "realtime" remains a claim by the authors, not a verified engineering fact — there are no published measurements of latency, throughput, and production-serving cost in the sources. The requirement of at least one NVIDIA A100 for the online demo does not in itself refute duplex, but means a high entry barrier: prototypes and pilot evaluations can be run today by teams with their own GPU infrastructure, and the descent to consumer-GPU depends on the appearance of quantizations — the signal is already there in the form of the third-party Heretic-ARA variant, but the barrier has not yet been lowered. All benchmark figures are taken from the authors' technical report and await independent evals, including on Full-Duplex-Bench v1.5 and StreamingBench. The Android APK exists only in beta, so the local mobile scenario cannot yet be considered a product.

Sources

Author

Look at AI, editorial team