Chatbots Make Financial Mistakes in 57% of Cases

A report by fintech company Saturn, “Artificial Authority,” found that 18 popular AI models make errors in an average of 57% of answers to real financial questions, with the error rate reaching 88% on complex topics.

Compiled by Sergey KostenchukPublished 2026-09-21Updated 2026-09-21

2026-09-21 Research OpenAI

🤖 Chatbots Make Financial Mistakes in 57% of Cases

London-based fintech company Saturn tested 18 models — ChatGPT, Claude, Copilot, Grok, and Gemini — on 121 financial questions about pensions, taxes, debts, and loans in its “Artificial Authority” report. Over 10,000 responses: an average of 57% errors, up to 88% on complex topics, and 99% for some. Bots don’t know current tax rules and make up their own.

🌍 A 57% error rate means an advisor loses their license, while a model just gets a version update. The FCA has warned that AI blurs the line between “information” and “advice,” and agentic payments (Worldline, ING, Mastercard) are being implemented faster than rules are being created.

👤 Don’t trust bots with final financial decisions — mortgages, pensions, loans. They explain general principles adequately, but verify figures with official sources: 57% don’t check, and 23% have already received incorrect data.

Source 1: https://www.saturnos.com/report/artificial-authority

Sources