Description
What is load shedding?
Load shedding is the deliberate choice to refuse or postpone some work when a system is overloaded, so that the most important work still completes on time. Instead of letting every request slow down together until everything times out, the system ranks traffic by priority and drops or defers the lowest-priority requests first. (In software it has nothing to do with scheduled power cuts. It borrows the name because the idea is the same: cut some load to keep the whole grid up.)
In a WhatsApp stack the priorities are usually clear. An Authentication Template carrying an OTP is worthless if it arrives after the code expires, and order and payment updates are time-sensitive. A marketing Broadcast can go out twenty minutes late with little harm, and analytics events or read-receipt fan-out to a CRM can be sampled or dropped. A shedding policy tags each request with a priority class at the edge. When a saturation signal crosses a threshold (queue age, worker concurrency, CPU, event-loop lag), it rejects the low classes with 503 or 429 and a Retry-After header, or parks them in a deferred Message Queue that drains once the pressure falls.
Load shedding sits beside three related patterns. Backpressure slows producers down so queues do not grow without bound. Graceful Degradation keeps serving every request, with fewer features. Rate Limiting is a fixed cap set in advance, whereas shedding reacts to actual saturation and admits everything when there is headroom. Load shedding is the one that drops or delays whole requests by priority.
Key takeaways
- Load shedding refuses or postpones the least important work during overload so the most important work still finishes on time.
- Rank traffic before the overload: OTPs and inbound messages first, then order updates, then marketing, then analytics.
- Trigger on saturation signals such as the age of the oldest queued job, not on raw request rate.
- Shed early and cheaply, with a 503 or 429 and a Retry-After header, before any expensive work is done.
- Never shed Meta's webhooks with error codes: Meta retries them for up to seven days. Acknowledge, then shed the processing.
Usage
Load shedding in code
// Priority classes, highest first.const PRIORITY = { otp: 0, inbound: 0, order_update: 1, marketing: 2, analytics: 3 };// The class comes from the route the gateway owns, never from what the caller claims.const ROUTE_CLASS = {"/webhooks/whatsapp": "inbound", // P0: never shed (Meta would only retry it)"/v1/otp": "otp","/v1/orders/notify": "order_update","/v1/campaigns/send": "marketing","/v1/events": "analytics",};// Thresholds on the age (ms) of the oldest job waiting in the send queue.const SHED_AT = [{ minAgeMs: 20_000, shedFrom: 3 }, // drop analytics fan-out{ minAgeMs: 45_000, shedFrom: 2 }, // defer marketing sends{ minAgeMs: 120_000, shedFrom: 1 }, // last resort: defer order updates too];// oldestJobAgeMs() reads a value cached and refreshed every second, not a live query.export function shedder(oldestJobAgeMs) {return (req, res, next) => {const priority = PRIORITY[ROUTE_CLASS[req.path]] ?? 2; // unknown routes are not criticalconst rule = SHED_AT.findLast((r) => oldestJobAgeMs() >= r.minAgeMs);if (rule && priority >= rule.shedFrom) {metrics.increment("requests.shed", { priority });res.set("Retry-After", "60");return res.status(503).json({ error: "overloaded", shed_priority: priority });}return next();};}
Decide with a cached signal before any parsing or database work, so a rejection costs almost nothing.
Properties
Load shedding: key facts and limits
- Typical shed signals
- Oldest-job age, in-flight requests, event-loop lag
- Reject response
- 503 (or 429) with a Retry-After header
- Protect first
- OTPs, inbound messages, order and payment updates
- Shed first
- Analytics and CRM fan-out, then marketing sends
- Authentication message TTL (Meta)
- Default 10 minutes, settable from 30 s to 15 min
- If you answer Meta non-200
- The webhook is retried for up to 7 days
Mechanics
How does load shedding work?
step 1 — Classify every request
Assign a priority at the edge: P0 for OTPs and inbound customer messages, P1 for order, payment and delivery updates, P2 for marketing and bulk sends, P3 for analytics and internal fan-out. Derive it from the route or template category, and carry it on the job so workers can shed too.
step 2 — Measure saturation, not just traffic
Use signals that predict missed deadlines: the age of the oldest queued job, in-flight requests per worker, event-loop or garbage-collection lag. Request rate alone says little about whether you are keeping up.
step 3 — Shed from the bottom up
When the first threshold trips, refuse P3. At the next, defer P2 to a later queue. Protect P0 and P1 for as long as possible. Separate thresholds make shedding gradual rather than all-or-nothing.
step 4 — Reject early and cheaply
Shed at the load balancer or the first handler, before parsing, database calls or Graph API requests. A rejection that costs as much as serving the request saves nothing.
step 5 — Drop work that has already expired
An OTP past its validity, or a reminder for an event that has started, should be discarded rather than sent late. Check deadlines when a worker picks up a job, not only when it was queued.
Caveats
Common mistakes with load shedding
Shedding inbound webhooks with error codes
Answering Meta with
503does not drop the event: under Meta's Webhook Retry Policy it comes back for up to seven days, which means duplicates and a bigger backlog later. Acknowledge webhooks with 200, queue them durably, and shed the processing behind the queue instead.Treating every request as critical
If every caller marks itself P0, the policy sheds nothing. Assign classes centrally from the message type or template category, not from what the caller asks for.
Shedding too late and too expensively
Rejecting after authentication, parsing and a database lookup still burns most of the cost. Decide at the edge, using cheap cached signals.
Dropping marketing without telling anyone
A silently shed broadcast looks like a delivery failure to the marketing team. Defer rather than drop where you can, and report what was shed, when and why.
Clients that retry instantly
A shed client that retries immediately doubles the load. Send
Retry-After, and make your own producers respect it with backoff and jitter.
Example
Example: A cricket final, a broadcast and 40,000 logins
On the night of a cricket final, a Mumbai fantasy-sports app starts a marketing broadcast just as 40,000 users try to log in. The age of the oldest job in the send queue passes 20 seconds and trips the first threshold, so analytics fan-out is dropped. At 45 seconds, marketing batches move to a deferred queue that resumes once the backlog drains. OTP sends keep a p99 under five seconds and logins succeed. The broadcast finishes 25 minutes late, which nobody notices.
Why it matters
Why does load shedding matter?
Overload rarely arrives at a convenient time: a Diwali broadcast, a flash sale and a webhook retry storm after an outage can all land in the same ten minutes. Without shedding, the OTP for a customer trying to pay waits behind 200,000 promotional sends and expires, which costs far more than a late promotion ever could. A shedding policy turns overload from a total outage into a controlled, explainable delay of the least important work, and it gives the business a clear, agreed rule about what is protected.
Notes
Best practices for load shedding
- Agree the priority order with the business before an overload, not during one.
- Trip each threshold on purpose in a load test and confirm that OTP latency holds.
- Count every shed request by class, and alert as soon as P1 shedding starts.
- Give each priority class its own worker pool so one class can never starve another.
Questions
Load Shedding FAQs
- What is the difference between load shedding and rate limiting?
- Rate limiting enforces a fixed cap whatever the system's state; load shedding reacts to real overload and drops the least important work first. A rate limit can reject traffic on a quiet day, while shedding acts only when the system is saturated.
- Is load shedding the same as backpressure?
- No. Backpressure slows producers so work waits; load shedding refuses or defers work so the rest finishes on time. Many systems use both: backpressure first, and shedding when queues keep growing anyway.
- Which WhatsApp messages should never be shed?
- OTPs and inbound customer messages usually come first, followed by order and payment updates. Marketing broadcasts, analytics and internal fan-out are shed first, because a short delay rarely harms them.
- Should I return 429 or 503 when shedding load?
- Return 503 Service Unavailable for overload, and 429 Too Many Requests when one client exceeds its share, both with a Retry-After header. Clients and proxies treat both as retryable, so the header is what prevents an instant retry storm.