The problem
On-call was drowning in low-signal alerts and incidents took hours to diagnose. There was no shared definition of healthy, so reliability work stayed reactive.
What CloudForge built
- 1Instrumented services with OpenTelemetry for traces, metrics and logs.
- 2Defined SLIs, SLOs and error budgets with product and engineering together.
- 3Rebuilt dashboards and alerting around symptoms rather than causes, which killed the noise.
- 4Wrote runbooks and ran incident game-days to build muscle memory.
The outcome
MTTR dropped 58% and alert noise fell 70%. On-call became sustainable, and reliability decisions are now driven by error budgets instead of gut feel.
Why the program started with user impact
Reducing alert volume is not the same as improving reliability. The useful starting point is a critical user journey, a measurable service-level indicator and an objective that reflects the business consequence of failure. Alerts can then focus on material error-budget consumption rather than every infrastructure symptom.
Runbooks and incident exercises complete the system. A dashboard can show that a service is failing, but recovery improves only when ownership, decision authority, communication and a tested response are clear before the next incident begins.
Related CloudForge guidance
Want an outcome like this?
Send CloudForge the project context and the company will scope what it would take for your stack.
Contact CloudForge →