🚀 Colibrì: Running GLM-5.2 on Consumer Hardware
A pure C engine that allows running the GLM-5.2 model (744B parameters, MoE) on devices with ~25 GB of RAM. The core idea is streaming experts directly from an NVMe drive to memory.
🌍 The project demonstrates a method for optimizing MoE models by offloading weights to fast storage. This expands the accessibility of frontier-class models on hardware that is dozens of times cheaper than server-grade H100s.
👤 You can run a powerful neural network on a standard PC provided you have a fast SSD and ~370 GB of free space. The generation speed is approximately 0.1 tokens per second.
Source 1: https://github.com/JustVugg/colibri Source 2: https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4