Nolan Lawson, creator of PouchDB and engineer at Socket, published a benchmark titled "The AI cost of JavaScript frameworks": the Claude Opus 5.5 agent (High Effort) built the same Pokémon catalog application on 11 JavaScript frameworks and meta-frameworks, and the API build cost varied by almost half — from $2.67 on Hono+htmx to $4.97 on Angular. The author's main conclusion: in the era of agentic development, the volume of code generated by the stack directly translates into token expenses, not only for generation but also for future maintenance.

image

What happened

On October 28, Nolan Lawson launched the Claude Opus 5.5 agent in High Effort mode in an isolated sandbox and gave it the same task: build a standard Pokémon catalog application using PokéAPI data. The experiment was repeated on 11 JavaScript frameworks and meta-frameworks, excluding CSS frameworks and HTML pre-generation to eliminate extraneous factors, with the cost of each build calculated based on actual API calls. The Hono+htmx combination was the cheapest: $2.67, 8.6 minutes, and 4.26M tokens; Astro was close behind at $2.76 and 8.2 minutes. Angular was the most expensive: $4.97, 17.2 minutes, and 9.76M tokens, which is approximately 1.9 times more expensive in terms of money and 2.3 times more in terms of tokens than Hono+htmx; in two out of five runs, the Angular result did not meet the specification.

Context

Nolan Lawson is the creator of PouchDB and currently an engineer at Socket, so the benchmark is backed by a practitioner rather than a research lab or marketing team. The point of the experiment rests on a shift in how code is created: when an AI agent writes the application, the verbosity of a framework is no longer just an aesthetic complaint, as every imposed abstraction and every line of boilerplate code is paid for in tokens — the agent must both read and reproduce this volume. Hence the author's central thesis: the size of the generated code determines future token costs for maintenance, turning stack selection from a matter of taste into a matter of economics.

Why this matters for the industry

For the industry, there are three implications here. First: a measurable variable, "AI cost of the framework," is added to stack selection matrices — how much money, tokens, and time the agent spends on building, and how many runs meet the specification, shifting the argumentation from feelings to metrics. Second: in content products with low interactivity, the pattern favors MPA approaches like Hono+htmx and Astro, which showed the best results, while heavy SPA stacks are forced to prove that their volume is justified by interactivity. Third: the methodology is ready for reuse — one reference task, a sandbox, several runs, and calculation based on API calls give teams a practical eval template for agentic coding. If replications on other models confirm the picture, frameworks will likely be expected to be compact and have a predictable structure for agents, and the next step could be CI gates that assess the AI cost of the codebase; this is a prediction, not a measured fact.

Why this matters for users

If you are choosing a stack for a new content site or reviewing your current one, you should first reconsider your usual comparison metrics: in the benchmark, Lighthouse Performance (97–100) and Accessibility (100) scores were saturated for all participants and did not differentiate them, making the overall Lighthouse score an unreliable selection criterion. The real differences lay in the JS bundle size and Total Blocking Time: Astro had 0 KiB of client-side JavaScript compared to 142 KiB for Next.js, and htmx and Astro options showed 0 ms TBT compared to 103 ms for Angular. The second practical step is a cheap audit of your own codebase: measure the volume of code the agent reads and writes during a typical edit, because this is the volume that converts into token spend on each iteration.

What is still unknown / limitations

The benchmark has strict boundaries of applicability: one model, Claude Opus 5.5 in High Effort mode, was tested on one type of task — a content catalog with low interactivity. The findings cannot be transferred to highly interactive SPAs like dashboards, other models, effort levels, or framework versions — replications are needed, which are not yet available. The author's thesis that code size determines future token costs for maintenance is plausible, but the methodology for measuring the maintenance itself is not described in the published data, so this is a hypothesis, not a measured fact. The absolute sums are currently small: the difference between the cheapest and most expensive stack was about $2.3 for building a simple application. Finally, two failed runs out of five for Angular indicate significant variance within a single framework, so conclusions about generation reliability require a larger number of runs.

Sources

Author

Look at AI, editorial team