Bots now account for 31% of all HTTP traffic, with AI agents the fastest-growing segment. AWS and Cloudflare are rebuilding infrastructure to handle the burst-pattern load AI agents create.
The web was built as a document system for humans to read. AI agents don't read. They query.
The internet was built for humans. That statement is becoming less accurate every year.
A TechCrunch analysis drawing on Cloudflare traffic data puts bot traffic at 31% of all HTTP requests over the last six months. AI crawlers, search engines and assistants account for roughly a quarter of all bot requests. Cloudflare's projection: non-human traffic will exceed human traffic sometime in the first half of 2027.
Share of all internet HTTP traffic that is now non-human, per Cloudflare's traffic analysis
This isn't only a volume story. AI agents behave fundamentally differently from human users. A single agent can spin up multiple sub-agents that query hundreds of databases and search thousands of documents in seconds, then go dormant entirely. The traffic pattern isn't a slow stream. It's intermittent bursts of high-intensity activity.
Infrastructure built for consistent human browsing patterns can't handle this efficiently. AWS responded this week by launching a next generation of OpenSearch Serverless designed specifically for agentic workloads. The core change is meaningful: compute decouples from storage, allowing systems to scale to zero when agents are idle and spike in seconds when they're active. Pay-per-burst infrastructure built for non-human demand.
The implication for marketers and publishers is more than technical. If AI agents are increasingly how information gets discovered and synthesised, the optimisation target is shifting. You're not optimising for a human reader to find your page. You're optimising for an AI agent to retrieve your answer when queried.
That's a different craft. The SEO principles that dominated the last decade were built around search engines indexing documents and humans choosing from a list of results. An agent-native web retrieves a single answer. Position one becomes the only position.
The infrastructure is already being rebuilt for machines. The question for marketers is whether their content strategy is.