Serge Bulaev's experiment showed that most AI agents tested for Claude Code create only a visual imitation of PPTX presentations, which is unsuitable for professional editing.
What Happened
During the testing of 11 different AI sub-agents, it was revealed that most third-party skills for Claude Code do not know how to work with native OOXML format objects. Instead of creating full tables and presentation elements, the agents generate "visual illusions"—sets of disconnected text blocks and graphic shapes that only vaguely resemble the required structure. Only the official skill from Anthropic and the slides_maker tool demonstrated full editability and correct object handling.
Context
The problem lies in the gap between the ability of Large Language Models (LLMs) to generate text or code and their ability to manipulate complex, structured file formats. Most current solutions focus on visual similarity, ignoring the internal file specifications, which is critical for workflow integration.
Why It Matters for the Industry
For AI tool developers, this is a signal of the need to move from simple "chat-to-file" interfaces to agents that operate at the document object model level. The future of agentic systems requires strict adherence to format specifications (e.g., OOXML) and the implementation of new quality evaluation metrics, such as the "structural integrity" of the output file.
Why It Matters for Users
Users employing AI to automate content creation may find that instead of a ready-to-edit presentation, they receive a collection of dozens of unrelated rectangles. This forces them to spend time manually reworking "agent-generated" files, which effectively negates the benefits of automation. When choosing tools, preference should be given to solutions that work with native objects, such as anthropics/skills.
What Is Not Yet Known / Limitations
The focus of the problem is shifting from purely engineering constraints to risks for professional content management and legal aspects of data control.
Sources
Author
Look at AI, Editorial Team