Description
What is observability?
Observability is how well you can understand what is happening inside a system from the outside, using only the data it emits. A system is observable when an engineer can answer a new question (why did this campaign's delivery rate fall from 96% to 71% at 11:20?) without shipping new code to find out. The word is often shortened to o11y: an o, eleven letters, then a y.
The raw material is telemetry, traditionally grouped into three signals. Metrics are cheap numeric time series (sends per second, failed statuses by error code, queue age) that show something changed. Logs are timestamped records of individual events (this send, this webhook, this Graph API error with its FB Trace ID) that show what happened. Traces follow one request across services as linked spans, showing where time and errors accumulated. Structured Logging, Distributed Tracing and the Four Golden Signals are the building blocks, and OpenTelemetry is the common standard for producing them.
Observability is often contrasted with monitoring. Monitoring checks known failure modes against thresholds and pages someone when one trips; observability is what lets you investigate the failures nobody predicted. Both depend on correlation. If the log line, the trace and the Message Status webhook for one message all carry the same wamid and trace ID, an engineer can move from a dip on a dashboard to the exact failing message in a few clicks.
Key takeaways
- Observability is how well you can explain a system's behaviour from the telemetry it emits, without shipping new code to investigate.
- Metrics show that something changed, logs show what happened to individual events, and traces show where time and errors built up.
- Correlate every signal with the same identifiers: wamid, phone_number_id, trace ID and Meta's fbtrace_id.
- Keep metric labels low-cardinality: template, status and error code, never wamid or phone numbers.
- Monitoring checks known failure modes; observability lets you answer questions nobody predicted.
Example
Example: A delivery dip explained in ten minutes
At 11:20 a Pune skincare brand sees its campaign delivery rate fall from 96% to 71%. Its metrics show failed statuses rising, and grouping them by error code shows almost all are 131049. Logs for a sample of failed wamids confirm that every one is a marketing template to users who had already received several marketing messages that week, while traces show the API calls themselves were fast and successful. Within ten minutes the team knows it is Meta's per-user marketing limit, not an outage. They stop the retry job that was making it worse, and support gets a clear explanation for customers.
Mechanics
How does observability work?
step 1 — Emit structured events
Log each send, status and webhook as JSON with stable fields:
wamid,phone_number_id, template name, Meta error code,fbtrace_idand trace ID. Mask phone numbers before they reach the log store.step 2 — Count what matters as metrics
Sends, statuses by state, failures by error code, webhook acknowledgement latency, queue depth and age. Keep labels low-cardinality: template and error code, never
wamidor phone number.step 3 — Trace across the asynchronous gaps
Carry trace context through queues and workers, so one message's journey (API call, queue, Graph API request) is a single trace, and link status webhooks back to it.
step 4 — Correlate everything
Use the same identifiers in logs, metric exemplars and traces, so you can pivot from a graph to the events behind it without guessing.
step 5 — Ask questions first, then alert
Build dashboards for the delivery funnel and the golden signals, investigate with ad-hoc queries, and turn the questions you find yourself asking every week into alerts.
Usage
Observability in code
{"ts": "2026-10-19T11:21:07.412Z","level": "warn","event": "whatsapp.status","status": "failed","wamid": "wamid.HBgMOTE5ODEyMzQ1Njc4FQIAERgSQTFC","phone_number_id": "106540352242922","recipient": "91******5678","template": "festive_offer_v2","error_code": 131049,"campaign_id": "cmp_2026_10_festive","trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"}
One event, every identifier you need to pivot: the wamid, the number, the template, Meta's error code and the trace ID. The phone number is masked.
Properties
Observability: key facts and limits
- Three core signals
- Metrics, logs and traces
- WhatsApp correlation IDs
- wamid, phone_number_id, trace ID, fbtrace_id
- Never use as metric labels
- wamid, phone numbers, user IDs
- Status states to count
- sent, delivered, read, failed
- Common telemetry standard
- OpenTelemetry (OTLP)
- Shorthand
- o11y
Caveats
Common mistakes with observability
High-cardinality metric labels
Putting
wamid, phone numbers or user IDs on metrics creates millions of time series and a huge bill. Keep those in logs and traces, and label metrics with template, status and error code.Logging personal data
Full phone numbers and message bodies in logs are a privacy liability. Mask numbers, leave message content out, and set retention deliberately.
Three tools that cannot talk to each other
Logs in one place, metrics in another, traces nowhere and no shared IDs turn every investigation into guesswork. Put the same trace ID and
wamidin every signal.Dropping Meta's identifiers
Graph API errors include an
fbtrace_id, and failed statuses carry an error code. Discard them and you have nothing to give Meta support, so log both on every failure.Dashboards instead of questions
Fifty panels nobody reads do not make a system observable. Start from the questions on-call engineers actually ask, and keep the data needed to answer new ones.
Why it matters
Why does observability matter?
In a WhatsApp stack many failures happen outside your own code: a template paused by Meta, a number's quality dropping, per-user marketing limits returning error 131049, a webhook endpoint quietly timing out. Without good telemetry these all look the same ("delivery went down") and teams guess. With it, support can tell a customer why their message failed, developers find the cause in minutes, and the business can tell a Meta-side limit from its own bug. Good observability is what makes Alerting, incident response and capacity planning possible at all.
Notes
Best practices for observability
- Start with the delivery funnel: sent, delivered, read and failed, by template and error code.
- Sample traces of normal traffic, but keep every trace that contains an error.
- Alert when webhook traffic goes silent, not only when errors rise.
- Give support a read-only view that looks a message up by its
wamid.
Questions
Observability FAQs
- What is the difference between observability and monitoring?
- Monitoring watches for known problems; observability lets you investigate unknown ones. Monitoring checks thresholds set in advance and pages when they trip, while observability means your telemetry is rich enough to answer questions you did not anticipate.
- What are the three pillars of observability?
- Metrics, logs and traces. Metrics show that something changed, logs record what happened in individual events, and traces show how a single request moved across services.
- Why did my WhatsApp delivery rate suddenly drop?
- Group failures by Meta error code first, because the code usually names the cause. In failed status webhooks, 131049 (per-user marketing limits) and 131026 (message undeliverable) are common; on the send call itself, look for 130429 (throughput exceeded), paused templates and token errors, then check your own webhook endpoint.
- What does o11y mean?
- o11y is shorthand for observability: the letter o, eleven letters, then y. It follows the same pattern as i18n for internationalisation and turns up in tool names, job titles and hashtags.