#observability

I built an LLM-as-judge eval on my own blog and got a suspiciously perfect 16/16. Here's the three-round test I ran before trusting that number: single-variable corruption, and a self-consistency check the research says most teams skip.

The most expensive AI agent failures don't throw an error, they hide. One ran at a six-figure-a-year rate for days while every dashboard stayed green, because the signals that catch it, per-session cost and anomalies, are the ones nobody watches. Why agent loops run away, and the two cost instruments your monitoring is missing.

Three findings from a real customer engagement on AWS Bedrock: what a load test was actually doing, why p95 latency was 45 seconds, and the prompt-caching default that costs every team money. Plus the three CloudWatch metrics that catch all three.