Qwen3.8-27B with ASCII vocabulary: 135K context tokens on 16 GB VRAM

A GGUF build, bsaleh03/Qwen3.8-27B-ASCII-Condensed, has been published on Hugging Face. In this build, the vocabulary of the Qwen3.8-27B quant (UD-IQ4_XS, 13.6 GB, Apache 2.0) has been reduced from 248,320 to 129,006 lines by removing all non-ASCII tokens, freeing up VRAM for the KV cache and increasing the context from 114,688 to 135,168 tokens on a 16 GB RTX 5070 Ti.

Compiled by Sergey KostenchukPublished 2026-09-17Updated 2026-09-18

2026-09-17 Coding Meta

🛠 Qwen3.8-27B with ASCII vocabulary: 135K context on 16 GB VRAM

In a selection of five fresh GGUF/MLX builds on Hugging Face, the main one is bsaleh03/Qwen3.8-27B-ASCII-Condensed: the vocabulary of the UD-IQ4_XS quant (13.6 GB) has been reduced from 248,320 to 129,006 lines — all non-ASCII tokens have been removed, the freed-up VRAM has been allocated to the KV cache, and the context has grown to 135,168 tokens on an RTX 5070 Ti.

🌍 This technique expands the context of local LLMs without fine-tuning: the embedding is a gather, the output head is a GEMV over the vocabulary, and trimming both tables frees up VRAM exactly for the KV cache. The pruning tools are on GitHub.

👤 On 16 GB — 135K context in llama.cpp with a single command (speed loss ~1%), but the build is not suitable for Russian: ~3 tokens per character. The uncensored LFM2.5-2.6B via Heretic v1.4.0 in Q4_K_M weighs 1.56 GB and runs on 6 GB VRAM.

Source 1: https://huggingface.co/bsaleh03/Qwen3.8-27B-ASCII-Condensed Source 2: https://huggingface.co/saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic

Sources