News

IFM releases MoE model K2-Horizon-MoVA-36B-A4B under Apache 2.0

Compiled by Sergey KostenchukPublished 2026-09-04Updated 2026-09-18

2026-09-04 Research HuggingFace

Expanded analysis for this story

Open the longform version with context, source trail, and what changed.

🤖 IFM releases MoE model K2-Horizon-MoVA-36B-A4B under Apache 2.0

The final checkpoint is already on Hugging Face: 36B parameters, 4B activated per token, its own Mixture-of-Values (MoVA) technique instead of standard attention, native 512K token context.

🌍 If the benchmarks claimed in the model card (58.6% on Terminal-Bench 2.1, 80.8% on GPQA Diamond) are confirmed, MoVA attention will become a concrete architectural alternative for coding and agentic tasks, compressing inference costs.

👤 You can run it right now: Hugging Face has 4/6/8-bit quantizations and MLX builds, the model card has working vLLM and SGLang recipes; the senior model in the family is K2-Horizon-375B-A23B.

Source 1: https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B Source 2: https://huggingface.co/IFM/K2-Horizon-375B-A23B

Sources

Story

IFM has published the final checkpoint of the open MoE model K2-Horizon-MoVA-36B-A4B: 36B parameters, 4B active per token, its own Mixture-of-Values (MoVA) technique instead of standard attention, native 512K token context, and Apache 2.0 license. An open MoE with 4B active parameters and claimed results comparable to closed flagship models narrows the gap in inference cost: if the claimed benchmarks are confirmed, MoVA attention becomes a concrete architectural alternative for agentic workloads and coding, and the promised release of data, recipe, and training code will allow for independent