A new computing paradigm, Persistent State Machine (PSM), has been introduced, designed to radically accelerate attention mechanisms in large language models (LLMs). Utilizing the Active State-machine Memory Architecture (ASMA) based on PSM allows computations to be moved directly into memory cells, which could reduce energy consumption by 99% compared to traditional GPUs.

What Happened

The Active State-machine Memory Architecture (ASMA), based on the Persistent State Machine (PSM) paradigm, has been developed. This solution allows attention mechanism computations to be performed directly in memory, potentially reducing system bus traffic by 99.47% and energy consumption by 99.0% when using INT4 precision. The work includes mathematical proofs, software simulation results, and synthesizable Verilog RTL code for hardware implementation.

Context

Modern LLMs face the "memory wall" problem, where the process of moving the KV cache between DRAM and the processor requires thousands of times more energy than the mathematical computations themselves. This creates a critical limitation for scaling context and the efficiency of neural networks on current architectures, such as NVIDIA GPUs.

Why It Matters for the Industry

The technology proposes a shift from the classical von Neumann architecture to the concept of In-Memory Computing. The availability of Verilog RTL code allows for the design of specialized prototypes at the FPGA and ASIC levels. This paves the way for creating a new class of ultra-efficient inference chips capable of radically reducing the cost of running models with long contexts.

Why It Matters for Users

This theoretical breakthrough in accelerator architecture could lead to the emergence of mobile and edge devices capable of running powerful LLMs locally. This would ensure high text generation speeds and the ability to work with massive amounts of data without significant battery drain or the need for constant connection to cloud servers.

What Is Not Yet Known / Limitations

At this stage, this is fundamental research based on simulations. The technology requires a transition from software models to real silicon and is not applicable to the current production stack based on NVIDIA H100.

Sources

  • [Persistent State Machine: A Formal Computational Paradigm for High-Sparsity LLM Attention Acceleration [Version 6.0]](https://zenodo.org/records/21679919)

Author

Look at AI, Editorial Staff