Why we're rethinking cache for the AI era

Cloudflare, working with ETH Zurich researchers, says AI crawler traffic behaves differently from human browsing and can make web caches far less effective.

The company argues that current CDN cache designs are poorly suited to AI-era access patterns, especially for scraping and retrieval-augmented generation workloads.

As cache hit rates fall, origin servers can face more traffic and higher costs, forcing site owners to balance human performance against automated AI access.

Cloudflare says CDNs may need new cache architectures to better manage AI crawlers and preserve efficiency.