New PSM Paradigm to Accelerate LLMs

🚀 New PSM paradigm to accelerate attention mechanisms in LLMs

The Active State-machine Memory Architecture (ASMA) based on the Persistent State Machine (PSM) has been developed. It allows computations to be moved directly into memory cells, potentially reducing system bus traffic by 99.47% and energy consumption by 99.0% compared to GPUs when using INT4 precision.

🌍 This solution addresses the fundamental "memory wall" problem, where transferring the KV cache between DRAM and the processor consumes thousands of times more energy than the computations themselves. The technology paves the way for creating ultra-efficient chips for inference with massive contexts.

👤 This theoretical breakthrough in AI accelerator architecture could lead to the emergence of devices capable of running powerful models locally with minimal battery consumption and massive generation speeds.

Source 1: https://zenodo.org/records/21679919