The author of the Telegram channel 'Misha, let's start over' published a blind comparison of ChatGPT (GPT-5.1 High in very high thinking effort mode) and Claude Fable 5.1 (Extra mode) on six practical tasks on iXbt Live: each run was conducted in a new chat without memory and personalization, and the tasks contained hidden traps. On a 10-point scale, Claude is ahead with 54 points to ChatGPT's 47, with four out of six tasks ending in a tie. There is no practical reason to switch models based on the test results: the models showed identical results in table cleaning and bug hunting in code.
What happened
The test consisted of six tasks: writing an article, cleaning a table, finding bugs in code, a mortgage calculator, comparing options by four criteria, and an incident report. The scores were distributed as follows: article — 5 to 6 in favor of ChatGPT, table — 10:10, bugs in code — 10:10, mortgage calculator — 9:9, comparison by criteria — 10:10, incident report — 10:2 in favor of Claude. It was on the incident report that ChatGPT hallucinated, fabricated data in the report, and missed the deadline, and received the only 'zero' for answers to hidden questions in the traps. In the practical part, both models cleaned a table of 2,900 rows in about 20 minutes, found all six types of errors, and independently arrived at an annual revenue of 19,870,310 rubles; in the bug-fixing task, each model found all five planted defects in the log analyzer script.
Context
This format emerged as a response to the saturation of public leaderboards: average scores are increasingly poor at predicting how a model will behave on a specific practical task, so independent runs with hidden traps and verifiable answers are gaining weight against 'battles' in a warmed-up context, where models tend to play along with the user. An important detail for interpreting the results: both models were tested in the most 'expensive' modes — very high thinking effort for ChatGPT and Extra for Claude, meaning the quality ceiling of the systems was measured, not the economics of ordinary everyday scenarios. Finally, this is an independent author's experiment, not a vendor report: none of the model developers participated in conducting the test.
Why this matters for the industry
For the industry, the main signal is de facto parity: on typical office tasks, top models are indistinguishable, so 'we use the best model' ceases to be a product advantage. Value shifts to the layer around the model: verification of results, artifact format, routing by task type, proprietary data, and workflows. The failure on the incident report is a second consequence for product builders: where an error costs money, independent control of the model's output is needed, because rare data fabrication in a finished report is not visible in average leaderboard scores. An atypical metric for leaderboards is also indicative — artifact efficiency at equal functionality: it is this that distinguishes models where quality matched, and such a metric should be considered in product evaluation alongside cost and latency.
Why this matters for users
There is no need to choose a model based on the test results; it is more reasonable to focus on the details of the artifacts: if you need a table 'for work' — Claude has live Excel formulas, and if a detailed change history is important — ChatGPT has an audit log of 843 edits. In the coding task, the quality matched, but the script generated by Claude ran in 0.14 seconds compared to 0.37 seconds for ChatGPT, while the original version took 3.7 seconds. The main takeaway for daily work is not about choosing a model, but about the mode of use: in tasks involving reporting and numbers, facts need to be independently verified regardless of which system produced them.
What is still unknown / limitations
The material remains a single-author experiment: one run per task, subjective 10-point scores, and the absence of repeated runs mean that the final score cannot be read as a stable ranking — the entire gap is created by one task with a failure, and under such a scheme, a single failure is statistically indistinguishable from a systematic difference. The difference in code speed was obtained from a single measurement on a single machine and is sensitive to system load and script implementation style; without repeated runs and code review, '2.5 times faster' cannot be considered a reproducible fact. There is not enough data for production solutions: the material lacks prices, API latency, and stability checks between runs.
Sources
Author
Look at AI, editorial team