Skip to content

Concept map· 6 connected terms

Incident Management

Also called: Incident response · Outage management · On-call incident process

In one line
The process a team follows to detect, triage, fix and learn from a live outage, with severity levels, named roles, customer updates and blameless postmortems.

Reviewed

What is incident management?

Incident management is the structured way a team handles a live failure, from the first alert to the last follow-up action. It defines how an incident is declared, how severe it is (typically SEV1 to SEV4), who is in charge, how customers are kept informed and how the team learns from it afterwards. In a WhatsApp integration, typical incidents are a key template stopped by Template Pausing after its quality drops, a webhook endpoint returning errors while Meta's retries pile up, an access token that has expired or been revoked so every send fails with Error 190 (Access Token Invalid), or a Meta platform disruption outside your control.

The core idea is that an outage is a coordination problem as much as a technical one. A named incident commander makes decisions and keeps the timeline. An operations lead, or a subject-matter expert, works on the fix. A communications lead updates the status page, support agents and affected customers on a fixed rhythm. Separating these roles stops the most experienced engineer from trying to debug, decide and write customer updates all at once.

Incident management starts where Alerting ends: an alert raises the signal, and incident management is what people do once it fires. It also drives the reliability numbers. Mean Time to Recovery measures how quickly the process restores service, each incident spends part of the Error Budget, and the loss of a whole region escalates into Disaster Recovery. In India some incidents carry reporting duties too: the CERT-In Directions require specified cyber security incidents to be reported within six hours.

Key takeaways

  • Incident management is the structured process for handling a live outage, from the first alert to the last follow-up action.
  • A written severity scale (SEV1 to SEV4) decides who is paged and how often customers hear from you.
  • Separate roles: an incident commander decides, an ops lead fixes, a communications lead keeps customers and support informed.
  • Mitigate first and find the root cause later, and declare recovery only when customer-facing signals are normal again.
  • Every SEV1 and SEV2 ends with a blameless postmortem whose action items have owners and dates.

Walk an incident from first alert to postmortem

6 / 6
  • Detected to Triaged: declare
  • Triaged to Mitigating: page the roles
  • Triaged to Closed, no impact: no impact
  • Mitigating to Monitoring: fix deployed
  • Monitoring to Mitigating: relapse
  • Monitoring to Resolved: signals healthy
  • Resolved to Postmortem: within 5 days
  1. Detected
  2. Triaged
  3. Mitigating
  4. Monitoring
  5. Resolved
  6. Postmortem

Step 6 of 6 · end of walk

Postmortem

Blameless review of the timeline, impact and contributing causes. Every action item gets an owner and a date.

Arrived from Resolved via within 5 days.

Can move to

Nothing — this is a final state.

The lifecycle of an incident as states. Play the usual path, then switch to a relapse after a premature fix, or a false alarm closed at triage, and read what each state demands of the team.

Happy path, step 6 of 6: Postmortem (via within 5 days). Blameless review of the timeline, impact and contributing causes. Every action item gets an owner and a date.

Why does incident management matter?

Every minute a WhatsApp flow is down has a visible cost. OTP logins fail and users give up, order updates stop and calls flood the support line, and Meta keeps retrying undelivered webhooks for up to seven days, so a slow recovery also leaves a long backlog behind it. Teams with a rehearsed incident process restore service faster, tell customers the truth sooner, and turn each outage into a fix that prevents the next one, instead of repeating the same scramble every quarter.

How does incident management work?

  1. Detect and declare

    An alert, a Synthetic Monitoring check or a spike in support tickets sets it off. Anyone on call can declare an incident. Downgrading a false alarm is cheap; finding a real outage an hour late is not.

  2. Assign a severity

    Use a short written scale. SEV1 is a customer-facing outage such as OTPs failing for everyone, SEV2 is partial impact such as one template paused, SEV3 is degraded but working, and SEV4 is minor. Severity sets who is paged and how often updates go out.

  3. Name the roles

    An incident commander coordinates and decides, an ops lead investigates, and a communications lead handles the status page and support. Keep one shared channel and one running timeline.

  4. Mitigate first, diagnose later

    Restore service before you hunt for the root cause: roll back the deploy, switch OTPs to a backup authentication template or SMS, or scale the webhook workers. Confirm recovery from customer-facing signals, not just green dashboards.

  5. Communicate on a schedule

    Post updates at a fixed interval, for example every 30 minutes during a SEV1, even when nothing has changed. Tell support agents what to say before customers start asking.

  6. Run a blameless postmortem

    Within a few days, write up the timeline, the impact, the contributing causes and the action items, each with an owner and a date. Ask what in the system allowed the failure, not who made the mistake.

Example: Saturday night, every OTP failing

At 19:40 on a Saturday, a Delhi quick-commerce app's login success rate halves. A synthetic check confirms that OTP sends are failing, and the logs show error 190 on every Graph API call. The on-call engineer declares a SEV1, takes the incident commander role and pulls in a backend engineer and a support lead. A secrets rotation earlier that evening had saved a truncated access token. The team restores the correct token within 14 minutes, and the support lead posts status-page updates and gives agents a holding reply for customers. The postmortem on Tuesday adds a deploy check that calls the token debug endpoint, plus an alert on any spike in error 190.

Common wrong turns

Common mistakes with incident management

  • Debugging before declaring

    Engineers often investigate quietly for 40 minutes before telling anyone. Declare early: closing an incident costs little, while a late declaration delays customer updates and help.

  • One person doing every job

    The engineer fixing the problem should not also be writing status updates and briefing management. Name an incident commander and a communications lead, even in a small team.

  • Declaring recovery when the fix ships

    After a webhook outage, Meta keeps re-delivering missed events and queued sends still have to go out. Resolve only when customer-facing signals are normal and the backlog has drained.

  • Postmortems that name a culprit

    Blame teaches people to hide information next time. Ask what made the mistake easy (missing alerts, unsafe deploys, no runbook) and fix that.

  • Action items without owners

    A postmortem whose actions have no owner and no date changes nothing. Track them like any other work and review them at the next incident review.

Incident management in code

# incident-policy.yaml: read by the paging bot, linked from every runbook
severities:
  SEV1:
    meaning: "Customer-facing outage: OTP or order messages failing for most users"
    examples: ["every send failing with error 190", "webhook endpoint returning 5xx"]
    page: [primary-oncall, secondary-oncall, eng-manager]
    status_page: required
    update_every: 30m
  SEV2:
    meaning: "Partial impact: one template paused, one number limited, one flow broken"
    page: [primary-oncall]
    status_page: if_customers_notice
    update_every: 60m
  SEV3:
    meaning: "Degraded but working: delayed statuses, rising queue age"
    page: []
    ticket: true
roles: [incident_commander, ops_lead, comms_lead, scribe]
postmortem:
  required_for: [SEV1, SEV2]
  due_within: 5 business days
  blameless: true

Write the scale down before you need it: severity decides paging, status-page duty and update cadence.

Key facts at a glance

Incident management: key facts and limits

Common severity scale
SEV1 (outage) to SEV4 (minor)
Core roles
Incident commander, ops lead, comms lead, scribe
Typical SEV1 update cadence
Every 30 minutes
Meta webhook retries
Up to 7 days, so backlogs follow outages
Template pause escalation (Meta)
3 hours, then 6 hours, then disabled
CERT-In reporting window
6 hours for specified incidents
Postmortem
Blameless, within a few working days

Good practice

Best practices for incident management

  • Keep a one-page runbook for each critical flow (OTP, order updates, the inbound inbox) listing the Graph API checks to run first.
  • Keep a backup authentication template and a backup utility template approved in advance, so mitigation never waits for template review.
  • Check Meta's platform status page during triage so you do not spend an hour debugging an upstream disruption.
  • Rehearse with a game day each quarter. Your first real SEV1 should not be the first time anyone has used the roles.
  • Write customer updates in plain language: what is affected, what to do meanwhile, and when the next update will come.

Incident Management FAQs

What is the difference between incident management and alerting?+
Alerting raises the signal; incident management is the human process that follows. An alert pages someone, and incident management then decides severity, roles, mitigation, communication and the postmortem.
What do SEV1, SEV2 and SEV3 mean?+
They are severity levels that rank an incident by customer impact, with SEV1 the worst. On a typical scale SEV1 is a full customer-facing outage, SEV2 partial impact, SEV3 degraded service and SEV4 a minor issue. Each level sets who is paged and how often updates go out.
What should a WhatsApp incident runbook check first?+
Check the things that stop every message: token validity, template status and number health. Look for error 190 on sends, check whether key templates are APPROVED or PAUSED, read the number's quality rating and health_status, then check your webhook error rate and Meta's platform status.
Do Indian businesses have to report incidents to anyone?+
Some do. Under the CERT-In Directions, specified cyber security incidents must be reported to CERT-In within six hours. The DPDP Act adds personal-data breach duties as its rules phase in, so check with your compliance team which duties apply to you.

See InfiQ pricing

Try it on the official WhatsApp Business API

InfiQ is an official Meta Business Partner. Start a 7-day free trial — no card needed — and see this running on your own number.

Start free trialBrowse all 550 terms