https://www.chenk.top/en/tags/cloudmonitor/Recent content in CloudMonitor on Chen Kai BlogHugoenMon, 04 May 2026 09:00:00 +0000https://www.chenk.top/en/aliyun-fullstack/07-observability/Mon, 04 May 2026 09:00:00 +0000https://www.chenk.top/en/aliyun-fullstack/07-observability/<p>The worst production outage I ever caused took three hours to diagnose. A Node.js service was returning 502s intermittently — maybe 5% of requests — and I had nothing. No centralized logs (each ECS instance had its own <code>/var/log/</code> and I was SSH-ing into them one at a time). No metrics dashboards (I was running <code>top</code> and <code>df -h</code> in terminals). No tracing (I was adding <code>console.log</code> timestamps to try to figure out which downstream call was hanging). Three hours later, I found the issue: a connection pool to RDS was exhausting under load because a forgotten cron job was holding connections open. The fix was two lines of code. The diagnosis took three hours of misery because I had zero observability.</p>https://www.chenk.top/en/terraform-agents/07-observability-and-cost-control/Tue, 24 Mar 2026 09:00:00 +0000https://www.chenk.top/en/terraform-agents/07-observability-and-cost-control/<p>Agents are non-deterministic, multi-step, and call expensive APIs. This combination means you can&rsquo;t debug them after the fact unless you instrumented them from the start. This article sets up three pipelines through Terraform — logs, traces, and metrics — into a unified dashboard, adds six SLS queries to solve real incidents, and sets up four alarms that have actually fired and saved my projects in production.</p> <p>By the end, you&rsquo;ll have a DingTalk channel that alerts you before the bill explodes, latency increases, the error rate spikes, or an agent starts looping on itself — plus SLO budgets that turn operational feelings into data.</p>