A roundup of five fresh GGUF/MLX builds has appeared on Hugging Face, the main one being bsaleh03/Qwen3.8-27B-ASCII-Condensed. The vocabulary of this Qwen3.8-27B quant has been reduced from 248,320 to 129,006 lines: all non-ASCII tokens were removed, the freed video memory was allocated to the KV cache, and on a 16GB RTX 5070 Ti the context grew from 114,688 to 135,168 tokens without any fine-tuning. The price of the technique is significant: Cyrillic and CJK are now encoded with several tokens per character, and the vision module has been removed. Alongside it in the roundup are an uncensored coding agent LFM2.5-2.6B via Heretic and a 4-bit MLX version of the same Qwen3.8-27B for Apple Silicon.

image
image
image

What happened

The author of the GreenNeuralRobots Telegram channel compiled a roundup of five fresh GGUF/MLX builds published on Hugging Face. The main artifact is bsaleh03/Qwen3.8-27B-ASCII-Condensed: a GGUF quant of Qwen3.8-27B in UD-IQ4_XS format, 13.6 GB in size, under the Apache 2.0 license, from which all non-ASCII tokens were removed from the embedding table and output head. The vocabulary was reduced from 248,320 to 129,006 lines, and the freed VRAM was allocated to the KV cache: the maximum context grew from 114,688 to 135,168 tokens, i.e., by 20,480 tokens, 1.18 times, on a 16GB RTX 5070 Ti. Structural verification showed that 864 out of 866 tensors are bit-for-bit identical to the base quant. The downside is measurable: Cyrillic now costs about three tokens per character, CJK and accented Latin about two, and the vision module has been removed from the build, so the model works only with text. In addition to it, the roundup includes saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic — a 2.6B LiquidAI LFM2.5 coding agent, uncensored via Heretic v1.4.0, reducing the refusal rate from 96 to 7 out of 100 with a KL divergence of 0.014, and released in 14 GGUF quants, and mlx-community/Qwen3.8-27B-4bit — a 4-bit MLX version of the same Qwen3.8-27B, 16.1 GB in size.

Context

The mechanics of the technique follow from the architecture of transformer inference. Embedding is a gather operation over the vocabulary, the output head is a GEMV over the vocabulary, so removing non-ASCII lines from both tables reduces VRAM consumption predictably and precisely, and the freed memory can be entirely allocated to the KV cache, expanding the context. Fine-tuning is not required: the weights remain the same, only the size of the vocabulary tables changes. The author has published reproducible pruning tools on GitHub, so the technique is repeatable on other models and other vocabulary domains. In parallel, the Heretic pipeline has become established in the abliteration ecosystem: uncensored versions of fresh models appear in the first days after their release, as happened with LFM2.5-2.6B. The roundup as a whole shows the maturity of the local scene: domain-specific GGUF builds, MLX quantization for Apple Silicon, and abliteration exist as debugged, ready-to-use routes.

Why this matters for the industry

For the industry, the technique demonstrates that the context of local LLMs can be expanded without fine-tuning by redistributing VRAM between the vocabulary and the KV cache: long context on 16GB cards turns from a feature into a commodity, and the unit economics of local AI products improve almost for free. The technique has no defensive position: the pruning tools are public, the Qwen3.8-27B weights are distributed under Apache 2.0, so the competitive advantage shifts from the method to the product around it. If independent checks confirm the preservation of quality, vocabulary pruning may become a standard step when building domain-specific GGUFs for a specific language or task, and further — an option in inference stacks and quantizers, where "model editing for the vocabulary" is set by configuration, not by separate fine-tuning. A separate signal for developers and distributors: the Heretic pipeline releases uncensored versions of fresh models in the first days after release, and this is already a stable, not a one-off, practice of the ecosystem.

Why this matters for users

If you have a 16GB card and work with code or English text, ASCII-Condensed gives 135,168 tokens of context in llama.cpp with one command: embeddings go to system RAM via -ot "token_embd.weight=CPU", with a speed loss of about 1%. For Russian, the build is not suitable — about three tokens per character will eat up all the gain, for Cyrillic it is better to take the original build. The uncensored lfm2.5-2.6b-fable5-coding-agent-heretic in Q4_K_M quant weighs 1.56 GB and runs even on a 6GB card via Ollama or LM Studio — an option for experiments with a local coding agent on weak hardware. The set is completed by parakeet-tdt-0.6b-v3 in Q8_0: a compact offline speech recognizer weighing 740 MB with Russian support and a WER of 6.54% on FLEURS.

What is still unknown / limitations

The structural identity of 864 out of 866 tensors is an integrity check, not a quality assessment: there are no perplexity and long-context benchmarks after pruning in the sources, nor latency measurements. Even for purely ASCII prompts, the output distribution is not bit-for-bit identical to the base quant, because softmax normalization now goes over the reduced vocabulary of 129,006 lines. The stated Heretic metrics (refusal rate and KL divergence) measure the intervention and proximity to the base, but not the preservation of the agent's coding capabilities. Predictions about vocabulary pruning becoming a standard option in inference stacks are an interpretation, not an established fact. Finally, the vision module is absent in the build, so multimodal scenarios are unavailable.

Sources

Author

Look at AI, editorial team