Yandex has published the base model AliceAI-Foundation-80B-A3B-Base on Hugging Face under the Apache 2.0 license — its first pretrain fully trained from scratch without external weights. With 80 billion parameters, only about 3 billion are active per token, and the context reaches 262,144 tokens. According to the technical report, the model outperforms larger open pretrains DeepSeek-V4-Flash-Base, Nemotron-3-Super-120B-A12B-Base, and GLM-4.5-Air-Base in Russian factual knowledge and a number of technical benchmarks. At the same time, the company released two new Russian-language factual benchmarks, WikiWebFacts and HardMultiQA, and a detailed technical report with ablations. This is a base model without chat instructions, so it still needs to be fine-tuned independently for conversational products.

image
image
image

What happened

Yandex has made the base model AliceAI-Foundation-80B-A3B-Base publicly available under the Apache 2.0 license: 80 billion parameters, of which about 3 billion are active per token, a MoE of 512 experts with top-10 routing plus one shared expert, a hybrid architecture of 36 layers of KDA linear attention and 12 layers of full attention, and a context of up to 262,144 tokens. The model was fully trained from scratch, without using external weights. According to the published tables, it outperforms larger open pretrains DeepSeek-V4-Flash-Base (284B-A13B), Nemotron-3-Super-120B-A12B-Base, and GLM-4.5-Air-Base (106B-A12B) in Russian factual knowledge — WikiWebFacts 86.5, HardMultiQA 67.9, EGE CoT 90.5 — as well as in MATH-500 (91.1), LiveCodeBench v5-6 (50.5), and AIME 2026 pass@32 (96.7). It also claims superiority over the previous closed Alice AI LLM 235B Base with about seven times fewer active parameters. Along with the weights, Yandex released two new Russian-language factual benchmarks, WikiWebFacts and HardMultiQA, and a technical report with ablations on KDA, Attention Residuals, Muon, and MoE stabilization via Z-loss.

Context

Last year's Alice AI LLM 235B was a pretrain based on Qwen3-235B-A22B, meaning it relied on external weights; AliceAI-Foundation-80B-A3B-Base is Yandex's first pretrain completed independently from initialization to checkpoint. The bet on a sparse MoE with 512 experts and a hybrid where 36 of 48 layers are allocated to KDA linear attention is a calculation for cheap long context: active parameters per token are several times fewer than in comparable open models. The description of the history of MoE activation stabilization via Z-loss indicates that the technical report includes real pretraining problems, not just final victories. The release of weights with ablations is a rare case of a reproducible pretraining recipe in the industry outside of China and the US, and the WikiWebFacts and HardMultiQA created by Yandex give the community what was previously unavailable in open form — a tool for measuring Russian factual knowledge. According to the company, this base model is intended to become the foundation for a unified reasoning model for Alice.

Why this matters for the industry

The Apache 2.0 license allows commercial use, so for teams building Russian-language AI products, the foundation has become free and legally clean. The economics are also changing: about 3 billion active parameters per token versus 12–13 billion in the pretrains the model competes with — if the numbers are confirmed, this is an argument in favor of data quality and training recipe, not brute scale. For product teams, this is a rare complete set of "building blocks": a pretrain with strong Russian factual knowledge, a 262,144-token context, open benchmarks for evaluation, and a technical report with ablations that can be reproduced. On such a base, it is logical to expect a new class of Russian RAG systems and SFT projects, as well as a reduction in the dependence of Russian-language products on external weights. The precedent is no less important: Yandex's open pretrain goes up against models from Chinese open labs on equal terms with fewer active parameters.

Why this matters for users

The weights can already be downloaded on Hugging Face and run on your own infrastructure: in transformers (reference — version 5.16.1, KDA layers will require flash-linear-attention 0.5.0) or in vLLM via a ready-made Docker image with tensor-parallel on 4 GPUs. Real scenarios right now — completion pipelines, few-shot, data generation for distillation, eval runs, SFT experiments on your own Russian corpora, and RAG on long documents relying on 262k context. Licensing costs are zero, and prototyping long-context Russian-language applications becomes significantly cheaper. The limitation is significant: this is a base model without chat instructions, Yandex has not published a ready-made instruct version, so a conversational or agentic product will require its own fine-tuning — the model cannot be taken into production without modifications.

What is still unknown / limitations

All comparisons with DeepSeek-V4-Flash-Base, Nemotron-3-Super-120B-A12B-Base, and GLM-4.5-Air-Base were performed by the model's author, including two new benchmarks of his own authorship, so until independent runs, the claimed superiority remains a plausible hypothesis, not a confirmed fact. The internal comparison with the closed Alice AI LLM 235B Base is the hardest to verify from the outside. Yandex has not published latency, throughput, and cost per token, and the maturity of the vLLM stack for KDA layers still needs to be tested in practice. The timing of the appearance of an instruct or reasoning version on this foundation is also unknown.

Sources

Author

Look at AI, editorial team