> ## Content Index
> Fetch the complete content index at: https://sudoreboot.me/llms.txt
> Use this file to discover other available public pages before exploring further.

# My monitoring cried wolf until I stopped listening, so I fixed the wolf
- URL: https://sudoreboot.me/alert-fatigue/
- Published: 2026-10-26T12:00:00.000Z
- Updated: 2026-10-30T15:47:18.000Z
- Description: Keyword checks, heartbeats, retries, and the flaky Wi-Fi camera that had to stop paging me.
- Author: Jon
- Tags: Homelab, monitoring

Monitoring only works if you still read the alerts. Mine come from three places: an uptime monitor checking every service, a network scanner that notices devices coming and going, and the server's own health alerts. All of it lands as push notifications on my phone, and for a while that phone buzzed so often that I'd stopped looking at it. That's the real failure mode, and it's sneakier than any outage.

## What actually works

- **Keyword checks, not just status codes.** A proxy will happily return `200` for an error page. Checking that the response contains a word you expect catches that instantly.
- **Heartbeats for scheduled jobs.** The nightly sync pings the monitor when it finishes. If the ping doesn't show up, *that's* the alert, and it catches the job that silently never started.
- **Retries before alerting.** Two failed checks in a row before anyone gets notified wipes out most of the noise from brief network blips.
- **Let sleepy devices sleep.** Battery and Wi-Fi gadgets that doze to save power get 30 minutes to vanish before they count as down.

## What got muted

One Wi-Fi camera drops off the network many times a day and comes back a minute later. Every drop meant a "down" notification *and* a "reconnected" notification, and within a week I was ignoring both, along with everything else. Down alerts for that one device are now off. It still shows online or offline on the dashboard; it just doesn't get to wake me up.

Connect and disconnect alerts for phones are off too. People coming and going isn't an incident.

> Rule of thumb: if an alert never makes you do anything, fix it or delete it.