95% of GPU capacity goes unused in Kubernetes clusters

Cast AI’s research across tens of thousands of Kubernetes clusters found average GPU utilization at just 5%.

CPU usage averaged 8% and memory 20%, indicating that a large share of paid capacity sits idle.

The findings point to major inefficiency in AI infrastructure spending as cloud costs keep rising.

They also suggest one-time rightsizing is not enough, because workloads and traffic continue to change over time.