🤖 AI Agents and Prediction Markets Threaten the Independent Internet
The movie data resource TheNumbers.com suffered an attack due to asymmetric crawling: while Google might scan 5 pages per visitor, Anthropic can request up to 38,000 pages. This creates a DDoS-like effect, exhausting website resources.
🌍 The problem requires a revision of protection and monetization models. The emergence of new audiences, such as AI agents and Polymarket participants, is forcing the implementation of new protocols, such as paid scanning models from Cloudflare.
👤 The independent internet is becoming more fragile. AI agents searching for data to facilitate instant trades can inadvertently disrupt the operation of useful services.
Source 1: https://stephenfollows.com/p/what-just-happened-to-thenumberscom-should-worry-us-all