Veltrix and the Day the Trace Loops Broke

Veltrix 2.4.1’s original autoscaling layer overprovisioned workers and triggered observability problems plus major latency spikes.

After repeated fixes failed, the team removed that layer and deployed veltrim, a Kubernetes admission controller with Lua rules.

Veltrim blocks risky scale-downs using CPU and cache-miss checks, helping protect warm pods and LLM caches.

The new setup reduced worker churn, stabilized pod behavior, and lowered p95 latency, though it added some kube-apiserver CPU overhead.