Skip to content

Four Golden Signals

Latency, traffic, errors and saturation: the four measurements SRE practice recommends watching first for any user-facing service.

Also called: Golden signals · LETS metrics · Latency, traffic, errors, saturation

Reviewed

Interactive lab

Hands on: use the controls to explore. Anything that plays on its own pauses while you are working inside it.

Four signals across a WhatsApp stack

4 / 4
Four signals across a WhatsApp stack. 4 rows by 4 columns. Each cell is a button: activate it to read its note, and use the arrow keys to move between cells.
Send API (yours)Graph API callsWebhook receiverQueue & workers
Latency
Traffic
Errors
Saturation

Hover, tap or arrow-key through the cells — a dot in the corner means there is a note.

Saturation by Send API (yours)

CPU, pool useDashboard only

Rarely the first limit for I/O-bound services, but watch database pool waits.

Legend — press one to highlight its cells

Each row is a golden signal, each column a part of the stack. Tap a cell to see what to measure there and whether it should page, open a ticket or stay on a dashboard.

Row 4 of 4. Saturation × Send API (yours): CPU, pool use. Rarely the first limit for I/O-bound services, but watch database pool waits.

01What is the four golden signals framework?

The four golden signals are latency, traffic, errors and saturation: the measurements the "Monitoring Distributed Systems" chapter of Google's Site Reliability Engineering book (2016) recommends as the minimum for any user-facing system. The idea is that if you can watch only four things, these four will reveal almost every problem customers would notice, and they give every service the same starting dashboard.

Latency is how long requests take, measured separately for successes and failures. Traffic is the demand on the system, such as requests or messages per second. Errors are the rate of failed requests, whether explicit (HTTP 500, a Meta error code), implicit (a 200 with the wrong content) or by policy (slower than promised). Saturation is how full the system is: the most constrained resource, such as a queue, a connection pool or a throughput limit, and how close it is to its ceiling.

For a WhatsApp integration the signals map cleanly. Latency: P99 latency of your send API, of the Graph API POST /messages call, and of the time your webhook receiver takes to return 200. Traffic: messages per second per phone_number_id against the number's throughput (MPS), and inbound webhooks per second. Errors: Graph API error codes such as Error 130429 (Rate Limit Hit), 131026 and 190, failed statuses in webhooks, and non-200 webhook responses. Saturation: queue depth and age (consumer lag), worker and database pool usage, and headroom against the throughput cap and the portfolio's messaging limit.

The golden signals are a starting set of dashboards, not targets. A Service Level Indicator (SLI) is the handful of these measurements you attach an objective to and alert on. Sibling checklists exist too: RED (rate, errors, duration) for request-driven services and USE (utilisation, saturation, errors) for resources such as CPUs and disks. All of them are building blocks of observability.

Key takeaways

  • The four golden signals are latency, traffic, errors and saturation, the starting set Google's SRE book recommends for user-facing services.
  • For a WhatsApp stack they map to send and webhook latency, messages per second, error codes and failed statuses, and queue depth and headroom.
  • Measure latency as percentiles, and track successful and failed requests separately so fast failures do not flatter the numbers.
  • Traffic falling to zero is a failure too: alert on webhook silence, not only on errors.
  • The signals are dashboards; SLIs are the few of them you attach targets to and page on.

02Why does the four golden signals framework matter?

Teams new to monitoring either watch too little (CPU and a ping) or drown in hundreds of charts nobody reads. The golden signals give every service the same four questions, so on-call engineers know where to look in any system at 3 am. For WhatsApp they also catch the failures that infrastructure metrics miss: a broadcast pinned at the throughput cap, OTPs waiting behind marketing in the queue, or a webhook stream that has silently stopped arriving.

03 · How it works, step by step

How does the four golden signals framework work?

  1. Step 1

    Instrument latency as histograms

    Record request durations into histogram buckets labelled by outcome, so you can compute P50, P95 and P99 and compare successes with failures. Averages hide the slow tail customers feel.

  2. Step 2

    Count traffic where demand enters

    Count sends per phone_number_id and category, and inbound webhook notifications per field (messages, statuses), so you can compare demand against each number's throughput.

  3. Step 3

    Break errors down by cause

    Label failures by HTTP status and Meta error code. 130429 is a pacing problem, 131026 an undeliverable recipient and 190 an expired token: three different responses for three different owners.

  4. Step 4

    Pick the constrained resource for saturation

    For messaging, the binding constraint is rarely CPU. Track queue depth and the age of the oldest job, connection pool usage, and messages per second as a share of the throughput cap.

  5. Step 5

    Decide what pages and what only charts

    Good alerting pages a human only for the few signals tied to customer-facing SLOs (webhook acknowledgement, OTP send success, webhook silence); keep the rest on dashboards and tickets.

04The four golden signals framework: key facts and limits

The four signals
Latency, traffic, errors, saturation
Source
Google, Site Reliability Engineering (2016), Monitoring Distributed Systems
RED method
Rate, errors, duration: for request-driven services
USE method
Utilisation, saturation, errors: for resources
WhatsApp traffic ceiling
Up to 80 mps per number by default, 1,000 after upgrade
Throughput exceeded
Error 130429

05The four golden signals framework in code

import client from "prom-client";
 
// Latency: histogram, labelled by outcome so failures don't flatter the percentiles
const sendSeconds = new client.Histogram({
  name: "wa_graph_send_seconds",
  help: "Latency of POST /{PHONE_NUMBER_ID}/messages",
  labelNames: ["phone_number_id", "outcome"],
  buckets: [0.1, 0.25, 0.5, 1, 2, 5],
});
// Traffic and errors: one counter, labelled by Meta error code ("none" on success)
const sends = new client.Counter({
  name: "wa_graph_sends_total",
  help: "Send attempts",
  labelNames: ["phone_number_id", "error_code"],
});
// Saturation: how much work is waiting
const queueDepth = new client.Gauge({ name: "wa_send_queue_depth", help: "Jobs waiting to send" });
 
export async function send(phoneNumberId, payload) {
  const end = sendSeconds.startTimer({ phone_number_id: phoneNumberId });
  const res = await fetch(`https://graph.facebook.com/v26.0/${phoneNumberId}/messages`, {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.ACCESS_TOKEN}`, "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  const body = await res.json();
  end({ outcome: res.ok ? "ok" : "error" });
  sends.inc({ phone_number_id: phoneNumberId, error_code: body.error ? String(body.error.code) : "none" });
  return body;
}
 
setInterval(async () => queueDepth.set(await queue.count()), 5_000);
app.get("/metrics", async (_req, res) => {
  res.set("Content-Type", client.register.contentType);
  res.end(await client.register.metrics());
});

One histogram, one counter and one gauge cover all four signals for the send path.

06 · Common mistakes, and the fix

Common mistakes with the four golden signals framework

  • Averaging latency

    A mean of 300 ms can hide 2% of sends taking eight seconds. Use histograms and look at P95 and P99 per number and per message category.

  • Letting fast failures improve the numbers

    A send rejected in 20 ms pulls average latency down just as things get worse. Label latency by outcome and chart successful requests separately.

  • Watching CPU as saturation

    Messaging workers are I/O-bound and hit the throughput cap, the queue or the connection pool long before CPU. Measure the resource that actually runs out.

  • No alert for silence

    If Meta stops reaching your webhook (an expired certificate, a changed callback URL), error rates stay at zero because nothing arrives. Alert when inbound webhook traffic drops far below its usual level for the hour.

  • Paging on all sixteen cells

    Four signals across four components is sixteen candidate alerts, and paging on all of them creates fatigue. Page on the few tied to SLOs, and ticket or chart the rest.

07Example: The night the dashboards stayed green

A Kolkata food-delivery platform's CPU and error charts looked normal during a Friday rush, yet riders complained that order updates were late. The golden-signals dashboard told the story in one row: traffic per number was flat at its throughput ceiling, Graph API latency was normal, errors were low because the shared limiter was doing its job, and saturation was the red flag, with the oldest job in the send queue 14 minutes old. Splitting order updates into their own priority queue, and spreading campaign traffic across a second number, brought queue age back under ten seconds.

08 · Tips from the bench

Best practices for the four golden signals framework

  • Put the four signals for one component on one row of the dashboard, components stacked below, so a problem reads left to right.
  • Label everything with phone_number_id: limits apply per number, and averages across numbers hide the one that is stuck.
  • Chart send-to-delivered time from status webhooks as a separate latency signal; it is what customers actually wait for.
  • Use queue age, not just depth, as the saturation signal for time-sensitive messages like OTPs.

09Four Golden Signals FAQs

What are the four golden signals?+
Latency, traffic, errors and saturation. Google's SRE book recommends them as the minimum to monitor for any user-facing service: how long requests take, how much demand there is, how often requests fail and how full the system is.
What is the difference between golden signals and the RED and USE methods?+
They are overlapping checklists for different targets. RED (rate, errors, duration) covers request-driven services, USE (utilisation, saturation, errors) covers resources, and the golden signals add saturation to the request view, which matters for queues.
Are golden signals the same as SLIs?+
No. The golden signals are what you watch; SLIs are the few measurements you attach objectives to. Usually one or two golden-signal metrics per journey, such as OTP send success and latency, become SLIs.
Which golden signal matters most for a WhatsApp integration?+
Saturation is the one teams most often miss. Throughput caps and queues fill up quietly while errors stay low, so watch queue age and messages per second against each number's limit alongside the others.

See InfiQ pricing

From definition to your first message

InfiQ is an official Meta Business Partner. Start a 7-day free trial — no card needed — and see this running on your own number.

Start free trialBrowse all 550 terms