I’ll take a chance and say that if you review your last ten incident wash-up (post-incident review) reports, you’ll see something along the lines of better monitoring and alerting at least seven times. I say this because I’ve been studying post-incident reviews for the last ten years. The bad news is that no matter how many times you change your monitoring tools or how many millions you spend on the newest best AI-enabled tool, monitoring will keep coming up on incident wash-ups. The good news is that most of the time, you do not need to change your monitoring tool to fix the problem.

In this talk, I’ll share lessons I learnt through experience on how to solve this problem.