The VIDRAFT team has introduced POCKET-35B—a highly efficient Mixture-of-Experts (MoE) model with 35 billion parameters, optimized to run on standard CPUs without the use of graphics cards.
What Happened
Developed based on Darwin-36B-Opus, the POCKET-35B model supports the GGUF format and is designed for use with tools such as llama.cpp, Ollama, or LM Studio. Thanks to its MoE architecture, it achieves a generation speed of 27.0 tok/s on Xeon processors, which is nearly 2.7 times faster than existing analogs. The model size ranges from 8.2 to 21 GB, allowing it to run on most modern PCs and smartphones with sufficient RAM.
Context
The development of On-Device AI requires architectures that can efficiently distribute computational resources. The use of GGUF quantization and MoE architecture allows complex language tasks to be moved from cloud servers to local consumer hardware (Edge Computing).
Why It Matters for the Industry
The emergence of efficient MoE models of this scale, capable of running on consumer hardware without a GPU, lowers the barrier to entry for local deployment of powerful AI. This stimulates the development of the On-Device AI industry and allows developers to prototype local agents and new UX patterns that are independent of cloud APIs.
Why It Matters for Users
Average users can now run advanced 35B-parameter language models directly on their smartphones or home computers without the need to purchase expensive graphics cards. This ensures high privacy and reduces dependence on paid cloud service subscriptions.
What Is Not Yet Known / Limitations
For full enterprise-level implementation, issues regarding security and data management still need to be addressed.
Sources
Author
Look at AI, Editorial Team