Tencent has released Hy3, a highly efficient model featuring a Mixture-of-Experts (MoE) architecture. It possesses 295 billion total parameters, with only 21 billion active parameters per token. The development is focused on solving complex reasoning tasks, writing software code, and powering autonomous AI agents with support for context lengths up to 256K tokens.
What Happened
As part of its new release, Tencent introduced the Hy3 model, which combines a massive total parameter count (295B) with low computational load per token by utilizing only 21B active parameters. Through post-training optimization, developers significantly improved performance: hallucination rates were reduced from 12.5% to 5.4%, and tool-calling stability increased substantially. The model supports a long context of up to 256K tokens and is optimized for FP8 format.
Context
The Mixture-of-Experts (MoE) architecture allows for scaling model capabilities while maintaining high inference speeds. The release of Hy3 is a significant step in the democratization of such efficient systems, enabling the creation of advanced agentic solutions without the need for ultra-powerful monolithic models that require excessive computational resources.
Why It Matters for the Industry
For the industry, the emergence of Hy3 means expanded possibilities for creating complex autonomous agents and workflow tools. The efficiency of the MoE architecture lowers the barrier to entry for developing advanced AI systems by reducing inference costs while maintaining high performance. This could trigger a mass transition from dense models to sparse architectures in industrial applications.
Why It Matters for Users
Users gain access to a more reliable and powerful AI assistant that excels at programming and analyzing large volumes of data. Thanks to the reduced hallucination rate and long context support, the AI is less likely to make factual errors and follows complex instructions more accurately in extended dialogues, making it an effective tool for working with large codebases and documentation.
Sources
Author
Look at AI, Editorial Team