IFM has released the open MoE model K2-Horizon-MoVA-36B-A4B under the Apache 2.0 license. The model has 36B total parameters, but only 4B are activated per token. It uses a proprietary Mixture-of-Values (MoVA) attention mechanism instead of standard attention, and features a native context length of 512K tokens. The final checkpoint has been published and is available for download.

image
image
image

What Happened

According to the official model card, the model scores 58.6% on Terminal-Bench 2.1, 26.8% on tau3-Banking, 80.8% on GPQA Diamond, and 38.9% on SciCode. IFM claims that K2-Horizon-MoVA-36B-A4B outperforms open dense models in the ~30B class and MoE models up to 15 times larger, and competes with closed frontier models. Along with the release, the company promises to publish intermediate checkpoints, data, the training recipe, and code. The larger model in the family is named K2-Horizon-375B-A23B.

Context

The architecture is built on MoE: the total capacity of 36B parameters is distributed among experts, and inference cost is determined by the number of active parameters. This means a configuration with 4B active parameters fits on hardware where a dense model of comparable quality would not. MoVA replaces standard attention with a mixture of values instead of a single value per attention query, which is claimed to provide higher expressiveness at the same cost. The native context length of 524,288 tokens was achieved during the midtraining stage, well before the final release.

Why This Matters for the Industry

An open MoE model with 4B active parameters and claimed frontier-class results narrows the gap in inference costs for agentic products and coding tools. If the self-reported benchmarks are confirmed, MoVA attention will become a concrete architectural alternative for agent workloads and coding. The promised release of data, the recipe, and training code will allow independent teams to verify the recipe and make K2-Horizon-MoVA-36B-A4B a reference point for open MoE models in the ~30B class. The key test for the entire architecture is the larger model in the family, K2-Horizon-375B-A23B: if it confirms the effectiveness of MoVA at scale, the family could set a new cost/capability benchmark for open agentic models.

Why This Matters for Users

The model can be downloaded and run right now: the model card includes working recipes for vLLM (tensor-parallel 2, --reasoning-parser k2_horizon, --enable-expert-parallel) and SGLang, validated on 2x H200. Additionally, 4/6/8-bit quantizations and MLX builds are available on Hugging Face. This is a ready-made option for experiments, internal automation, and prototyping agentic and coding products on your own hardware, where the long native context covers extended agentic chains and large documents without truncation, and deployment does not require waiting for an API or negotiating license terms.

What Is Still Unknown / Limitations

All benchmark figures are taken from IFM's model card and are self-reported: there are no independent reproductions, and public comparison tables with specific base models (which exact dense ~30B models and which MoE models up to 15 times larger) are not included in the sources, nor are latency/throughput data. The results are uneven: 80.8% on GPQA Diamond versus 26.8% on tau3-Banking. No paper or ablations showing the contribution of MoVA separately from MoE have been published. The promised data, recipe, and code have not yet been released. The 512K context, custom MoVA attention, and special launch flags create operational risks for production that need to be addressed with your own measurements.

Sources

Author

Look at AI, editorial team