Scale before the spike: Predictive autoscaling for GPU workloads on Kubernetes
The 3 AM Call We got paged one Tuesday morning. A critical production service had crashed under traffic—not gradually degraded, but crashed. Hundreds of pending pods. Users were seeing 15–20% error rates. The incident postmortem was…
Building an AI factory on Kubernetes
An AI factory is not just a model or a cluster. It is a pool of GPUs that many teams draw from at once: one team fine-tuning, another serving inference, a third running evaluations, all on…
The lazy developer’s guide to observing your own code
It’s no secret that developers are increasingly being asked to shift left. It seems there’s always something new to shift left on. And now developers are being asked to shift left on observability. This means that…
How to turn slow queries into actionable reliability metrics with OpenTelemetry
Slow SQL queries degrade user experience, cause cascading failures, and turn simple operations into production incidents. The traditional fix? Collect more telemetry. But more telemetry means more things to look at, not necessarily more understanding. Instead…
A practical guide to solving when zero+zero=two in mesh observability
A Service Mesh like Istio, together with Kiali gives you a lot on day one. You install the mesh, point Prometheus at it, and suddenly you have request rate, latency, error rate, and a fairly good…
Does Kubernetes DRA Replace HAMi?
Projects that want to share a GPU on Kubernetes have to work around an API instead of with it. The device plugin interface could count devices, and that was the whole vocabulary: nvidia.com/gpu: 1. It meant one…
You can’t debug what you can’t see — Observability for AI Agents
This article reflects practical experience building and operating production AI agent systems. Traditional APM can’t tell you why your agent spent far more than usual asking the same question three times. We’ve been running AI agents…
Your Kubernetes health checks are accidentally waking your services. Here’s the fix.
Scale-to-zero breaks when health checks scale you back up. Learn how KubeElasti’s ProbeResponse lets Kubernetes services stay genuinely idle — while keeping load balancers and uptime monitors happy. Scale-to-zero sounds perfect on paper. Idle service. No…
Why goodput matters more than throughput for LLM serving
When we benchmark an LLM serving setup, the number almost everyone reaches for first is throughput: how many requests per second the system can push through. It is easy to measure, easy to compare, and it…
Challenge: Scaling Infrastructure Without Linear Team Growth Operating Kubernetes at large scale required more than deploying clusters. Manual provisioning and ad-hoc operations would not scale, and planned maintenance could not be disruptive. As internal adoption increased,…