A sharp decline in the cost of using large language models may signal a transition of AI technologies into the category of standard, cheap resources, questioning the return on investment for massive infrastructure expenditures.
What Happened
The Silicon Data LLM Token Expenditure Index has fallen nearly 20% compared to the peak recorded in May 2026. Prior to this, the indicator had shown twofold growth since its launch in December.
Context
Amid falling token prices in the industry, colossal capital expenditures (capex) exceeding $700 billion persist. There is a growing gap between the decreasing cost of inference and the massive investments in building computational infrastructure.
Why It Matters for the Industry
The decline in token costs indicates intensifying competition and the commoditization process of base AI models. This could lead to lower margins for companies investing in infrastructure and will require startups to find new ways to build business moats beyond simply using APIs.
Why It Matters for Users
For developers and users, this means lower operating expenses (OPEX) for running RAG systems, agentic architectures, and other LLM services. Simultaneously, the price drop lowers the barrier to entry for creating new AI-based applications.
What Remains Unknown / Limitations
It is unclear how long the current trend of declining prices will last and whether the industry can maintain growth rates amidst the declining profitability of base models.
Sources
Author
Look at AI, Editorial Team