Subframe co-founder Adam Ginsberg launched Real Design Benchmark — an open benchmark in which Claude Opus 5.5 and GPT-6.1 Sol in medium effort mode design a real product screen based on a text brief, screenshots of adjacent screens, and a set of fonts, icons, and colors, without seeing the original. In a blind comparison, 57% of participants preferred the AI version versus 33% for the real screen, and correctly identifying the model's work was possible in only 58% of cases, compared to 50% random guessing.

image

What happened

The benchmark works as a game on the realdesignbenchmark.com website: visitors are shown two screens of the same product, first choosing the best option, then guessing which of the two was made by AI. The tasks are set up as follows: the model receives a text brief, four screenshots of adjacent screens, and a set of fonts, icons, and colors, after which it generates HTML for a new screen without seeing the real design; both options are rendered identically to eliminate differences in presentation. By October 12, 2026, the project had accumulated 116 tasks from 52 products and several thousand responses: 57% of participants preferred the AI version, 33% — the real screen, and another 10% considered the options roughly equal. The model's work was correctly recognized in 58% of cases, which is only slightly above the 50% random guessing level.

Context

Without one semantic detail, the numbers can easily be misread: the models are not proposing an open design from scratch, but generation within a specific product's design system, essentially restoring the brand style based on indirect signals. Therefore, the result measures strong adherence to context and style, not the ability to come up with new interface solutions. The methodological hygiene is above average for self-published benchmarks: blind pairwise comparison with a double question, identical rendering of both options via HTML and screenshot, manual checking of prompts for leakage of the target screen, test accounts with dummy data, and an open methodology version 1.1, published on October 14, 2026. The project's status is also important: this is a well-executed measurement of an already existing capability of frontier models, not a breakthrough in the models themselves, and a manual experiment by one author, not a commercial product.

Why this matters for the industry

For the industry, this is a public, measurable proof that frontier models already generate UI that people in the context of a real product not only cannot distinguish from a designer's work, but often prefer. The recipe itself is reproducible: a pipeline of product context assembly, model call, HTML generation, screenshot, and blind comparison can be deployed today as offline generation of draft screens and variations from brand assets, without strict latency requirements. The benchmark also offers a new quality criterion for LLMs in subjective tasks, not just in code and math. For now, the effect is reputationally-signaling: a verifiable argument that "UI generation works in a real context" has appeared, which can be referenced in pitches and roadmaps, but it does not yet translate into direct changes in buyers' budgets. A forecast, not a completed fact: if traffic is maintained and breakdowns by models, versions, and product categories appear, the project could become a regular leaderboard of "subjective" LLM capabilities, which would be referenced at releases.

Why this matters for users

Everything can be checked by hand: the game itself is available on realdesignbenchmark.com — you choose the best of two screens and guess where the AI work is — the methodology and the HTML screens generated by the models for real products are published alongside. For designers and developers, this provides a visual assessment of how well the model fits a new screen into someone else's system of fonts, icons, and colors without manual adjustment to the brand, and for everyone following LLM capabilities beyond programming — a concrete idea of how far generative UI has come.

What is still unknown / limitations

Caution is needed for several reasons. Outperforming random guessing by only 8 percentage points is a point estimate based on several thousand responses without published confidence intervals, control for repeat votes, and accounting for audience self-selection in the original material, so it is not yet possible to separate the real signal from noise. "Prefer" in a blind comparison is a metric of immediate aesthetic judgment: usability, completeness of screen states, edge cases, and accessibility are not measured at all by this protocol. The benchmark also does not disclose inference cost and generation delay, model runs are non-deterministic and consistency guarantees of results are not verified, and validation remains entirely human — slow and not cheap.

Sources

Author

Look at AI, editorial team