Startup PrismML has developed an innovative compression technology that allows the massive Qwen 3.6 model from Alibaba, featuring 27 billion parameters, to run locally on mobile devices. By reducing the model size to less than 4 GB, this technology paves the way for utilizing powerful artificial intelligence directly on smartphones, including the future iPhone 17 Pro.
What Happened
PrismML has achieved extreme compression of the Qwen 3.6 (27B) model, shrinking its size to less than 4 GB. This achievement makes local inference of the model on mobile chips possible. It is reported that Apple has already held discussions with PrismML regarding the potential integration of this technology into its products.
Context
Modern Large Language Models (LLMs) typically require massive computational power and cloud infrastructure to operate. The shift toward Edge AI involves moving computations from remote data centers directly to users' end devices, which requires efficient methods of quantization, distillation, or weight pruning.
Why It Matters for the Industry
Successfully localizing models of this scale on smartphones could significantly shift the investment landscape, moving the focus toward Edge AI and reducing the industry's dependence on cloud computing. This creates new opportunities for mobile app developers and stimulates demand for model optimization tools tailored to specific mobile hardware.
Why It Matters for Users
For end users, this means a qualitative shift toward more private and faster AI assistants. Models will be able to function without a constant internet connection, providing instantaneous responses and ensuring that personal data is not sent to the cloud for processing.
What Is Not Yet Known / Limitations
Technical verification of quality metrics, such as perplexity (loss of accuracy) and real-world latency during inference on mobile processors, is still required.
Sources
Author
Look at AI, Editorial Staff