The Stage.AI team has introduced the edge-lm project, which allows running compressed Gemma 4 checkpoints on Apple Silicon devices (Mac and iPhone) via MLX. By utilizing PLE architectures and vector quantization (AQLM-style), model sizes have been reduced by 6.4–7x while maintaining high accuracy in key tasks.
What Happened
Developers from Stage.AI have implemented extreme compression technology for the Gemma 4 model family. For example, the E2B version now occupies only 1.44 GB instead of the original 9.26 GB (in BF16 format). Despite such a significant reduction in weights, the models maintain high efficiency in IFEval (instruction following), τ²-Bench (tool use), and MMLU-Pro (general knowledge) benchmarks.
Context
The optimization is based on the application of PLE architectures and AQLM-style vector quantization methods. The edge-lm project is aimed at working within the Apple ecosystem, using the MLX framework to maximize the efficient use of Apple Silicon chip resources.
Why It Matters for the Industry
This technology demonstrates an effective trade-off between extreme compression and functional preservation. This is critical for the development of local on-device assistants, as it allows for reduced dependence on cloud computing and lowers inference costs.
Why It Matters for Users
iPhone and Mac users gain the ability to use the powerful capabilities of Gemma 4 directly on their devices with minimal RAM consumption. This ensures high-quality responsiveness and tool functionality without the need for a constant internet connection.
Sources
Author
Look at AI, Editorial Team