Moonshot AI releases Kimi K3.1 Pro weights

Moonshot AI has released the open weights for Kimi K3.1 Pro, a 2.8-trillion parameter MoE model (104B active) with a 1,048,576-token context, where Gated DSA sparse attention replaces full Gated MLA attention in 24 of 93 layers.

Compiled by Sergey KostenchukPublished 2026-11-04Updated 2026-11-04

2026-11-04 Research HuggingFace

🤖 Kimi K3.1 Pro weights released

Moonshot AI has released the weights for the first model in the K3.1 series, post-trained on the Kimi K3 base using large-scale joint RL across hundreds of tasks. It is a 2.8-trillion parameter MoE (104B active), with 896 experts (16 per token), a 1,048,576-token context, and MXFP4 for expert weights. In 24 of 93 layers, Gated MLA has been replaced with Gated DSA (DeepSeek Sparse Attention), which accelerates long context.

🌍 Sparse attention reduces KV-cache costs and makes 1M-context practically usable, while ready-made vLLM/SGLang recipes lower the barrier to entry for self-hosting.

👤 The weights are on Hugging Face: you can download and run the model (requires a serious cluster, 8+ B300/GB300-level GPUs) or try it via the kimi-k3.1-pro API. It is one of the strongest open models for agentic tasks and code.

Source 1: https://huggingface.co/moonshotai/Kimi-K3.1-Pro

Source 2: https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro

Sources