Tencent (the Tencent Hy / Hunyuan team) has released its flagship MoE model Hy4 preview in open source: 770B total parameters, 49B active per token, native 1M token context, and an Apache 2.0 license that permits commercial use. Weights in BF16 and FP8 formats are already available for download, the model is supported by official vLLM and SGLang images, and it effectively invites users to verify the claimed frontier-class characteristics themselves. The main caveat is the preview status: reasoning mode is enabled by default and slows down operation, and vendor benchmarks have not yet been confirmed by independent runs.

image
image
image

What Happened

Tencent published the Hy4 preview weights in two formats, BF16 and FP8, simultaneously on four platforms: Hugging Face, ModelScope, GitCode, and CNB. The architecture features 78 layers and 256 routed experts plus one shared expert, of which eight are activated per token; attention is built on the Gated DeepSeek Sparse Attention (Gated DSA) mechanism with an IndexCache, the checkpoint includes an iHC residual mechanism with four residual streams, and a separate 10B-parameter MTP layer (0.7B active) for speculative decoding. Claimed metrics: GPQA Diamond 92.3, SWE-bench Multilingual 82.9, SWE-bench Pro 65.7, and Deep SWE 64.3. In an internal blind comparison by 163 Tencent experts on 203 engineering tasks, Hy4 preview slightly outperformed GLM 5.3.

Context

The point of the Hy4 architecture is separating scale and cost: like other MoE models, not all 770B parameters work on each token, only 49B, so inference is cheaper than a dense model of that size. The bottleneck for long context is usually attention, and here the bet is on Gated DSA: sparse indices computed in some layers are reused in others via IndexCache, which reduces the cost of processing contexts up to 1M tokens. The MTP layer for speculative decoding and iHC show co-design of architecture and inference: speed optimizations are built into the checkpoint itself, not bolted on from the outside. A separate feature of the release is that the architecture is disclosed in enough detail for reproduction, which is atypical for the frontier class and makes the model a full object for independent ablations.

Why This Matters for the Industry

This is the second major open frontier flagship of the MoE generation in a short time under the Apache 2.0 license that permits commercial use, so access to a strong model is no longer a moat, and product value shifts to data, workflow integration, and distribution. Ready-made images, deployment recipes, and quantization via AngelSlim mean the model is ready for self-hosting right away, not just on paper: companies can run a pilot and compare its cost to their current closed API bill. If the IndexCache mechanism is confirmed in practice and adopted by the ecosystem, reusing sparse indices between layers could become a standard part of the open-source inference stack for long context, and the MTP layer sets an example of optimizations built into the checkpoint. If the numbers are confirmed by third-party runs, an open frontier model will become real pressure on closed API prices and a base for fine-tuning.

Why This Matters for Users

Weights can be downloaded from Hugging Face (as well as ModelScope, GitCode, and CNB) and the model can be run on 8 GPUs via vLLM or SGLang following official recipes, calling it via an OpenAI-compatible API from existing clients with minimal changes. Reasoning mode is enabled by default and can be disabled with the reasoning_effort=no_think parameter. A realistic plan right now is piloting on non-interactive tasks like batch processing, offline analytics, and code generation, as well as long-context analytics prototypes, private coding assistants, and agent pipeline PoCs. For interactive scenarios, the model is currently inconvenient: it spends extra time on reasoning and tends to re-verify its own results, though the team promises quick iterations.

What Is Still Unknown / Limitations

The claimed metrics are vendor numbers without disclosed methodology in the provided materials, and no one has independently reproduced them yet. The internal blind test by 163 Tencent experts contains a conflict of interest, and the margin over GLM 5.3 is minimal. The preview status means some early-version limitations are promised to be fixed, but no timeline is given. The 770B-parameter model requires serious infrastructure, which narrows the circle of those who can verify it themselves, and its behavior on real long-context tasks outside benchmarks remains an open question until third-party runs.

Sources

Author

Look at AI, editorial team