Skip to content

Canary Release

Also called: Canary deployment · Canary rollout · Progressive delivery · Canary

A canary release sends a small slice of live traffic to a new version, compares its errors and latency with the stable one, and rolls back if it degrades.

Reviewed

What is a canary release?

A canary release is a deployment strategy in which a new version of a service — your send API, webhook processor or Chatbot — receives a small slice of production traffic, such as 1% or 5%, while the stable version serves the rest. The canary's error rate, latency and business metrics are compared with the stable version's; if it holds up, its share grows in steps (5%, 25%, 50%, 100%), and if it degrades, traffic shifts back automatically. The name comes from the canaries miners once carried to detect gas before it could harm people.

What makes it a canary rather than a slow push is the gate. Each step waits long enough to collect a meaningful sample, then compares like with like: good practice runs a fresh baseline of the old version at the same size as the canary, so both see the same traffic mix and the same warm-up effects, and the decision rests on the difference between them. Blue-green deployment is the contrast — two full environments with all traffic switched at once, quick to roll back but with every user exposed at the moment of the switch.

Messaging systems add their own wrinkles. Webhooks for one conversation should be routed consistently — by wa_id or conversation ID — so a customer does not get the new bot flow for one message and the old one for the next. Outbound side effects are real: a canary that sends duplicate templates sends them to real customers, so idempotency and a small first step matter. And the Error Budget policy decides how much risk a canary may take: when the budget is spent, risky releases wait.

Key takeaways

  • A canary release exposes a new version to a small share of real traffic first, and widens it only while its metrics stay healthy.
  • Compare the canary with a baseline of the same size started at the same time, not with the whole long-running fleet.
  • Gate each step on error rate, latency and a business signal, and roll back automatically when the canary is measurably worse.
  • Unlike blue-green deployment, which switches all traffic at once, a canary limits a bad release to a few percent of users.
  • Webhook-driven systems need sticky routing by conversation, so one customer's messages do not bounce between versions.

See InfiQ pricing

Interactive stop

Should v42 go to 25%, or roll back?

14 / 14

Metric · run 1

Webhook events acknowledged within 500 ms

Visitors per variant4,000 / 4,000

  • ABaseline v41Winner93.4%± 0.8 pts, 95% interval 92.7% to 94.2%
  • BCanary v4285.5%± 1.1 pts, 95% interval 84.4% to 86.5%
  • Observed rate
  • 95% interval (± 1.96 × standard error)
  • True rate (unknown in a real test)
  • Where the leaders overlap

A (Baseline v41) wins

Observed 93.4%. Its 95% interval no longer overlaps the other one, so the gap is very unlikely to be chance.

  • ABaseline v41Control

    The stable bot on a 5% slice the same size as the canary, started at the same moment.

    3,737 of 4,000 converted

  • BCanary v42

    New build that adds a synchronous policy lookup to every reply, on a sticky 5% of conversations.

    3,418 of 4,000 converted · −8.5% vs A

Baseline and canary each collect 4,000 webhook events (illustrative). Watch the 95% intervals narrow: once the baseline clearly beats the canary on fast acknowledgements, the rollout stops. Run it again to see the noise.

4,000 visitors per variant. A (Baseline v41) wins. A: 93.4%, 95% interval 92.7% to 94.2%; B: 85.5%, 95% interval 84.4% to 86.5%.

How does a canary release work?

  1. Deploy the new version beside the old

    Run the canary as separate instances behind the same load balancer or queue, plus a baseline of the stable version at the same size, both pointing at the same dependencies.

  2. Route a small, sticky slice

    Send 1–5% of traffic to the canary, hashed on a stable key such as wa_id or tenant ID, so each customer consistently sees one version for the whole rollout.

  3. Compare against the baseline

    Measure the Four Golden Signals that matter most here — error rate (5xx, timeouts, Cloud API rejections) and P99 Latency — plus a business signal such as bot completion rate, for canary and baseline over the same window, using the same Service Level Indicator definitions.

  4. Promote in steps, or roll back

    If the canary is not measurably worse after the minimum sample, widen it: 5%, 25%, 50%, 100%. If it is worse, shift its traffic back automatically and alert the team.

  5. Finish and clean up

    At 100%, retire the old version and the baseline, and record the outcome so the next rollout's thresholds are tuned on real data rather than guesses.

Example: A claims bot update at a Chennai insurer

An insurer in Chennai ships version 42 of its claims bot, which adds a synchronous policy lookup to every reply. The pipeline sends 5% of conversations, hashed by wa_id, to v42, with a v41 baseline of the same size. After 4,000 webhook events each, 93% of baseline events are acknowledged within 500 ms but only 86% on the canary, and the confidence intervals no longer overlap. The rollout halts and the 5% returns to v41 within a minute. The team moves the lookup to a background job, and v43 passes every step to 100% the next morning.

— A claims bot update at a Chennai insurer

Why does a canary release matter?

Most outages begin with a change. A canary turns "we deployed and everything broke" into "we deployed to 5% and rolled back automatically in four minutes". For a WhatsApp platform the stakes are customer-visible: a bot regression that misroutes refund requests, a payload change that makes the Cloud API reject sends with error 132000, or a slower webhook handler that triggers Meta's retries. Catching those on a small slice protects both customers and the month's error budget.

A canary release in code

import { createHash } from "node:crypto";
 
const CANARY_PERCENT = Number(process.env.CANARY_PERCENT ?? 5);
 
// Same wa_id -> same bucket -> same version for the whole rollout
export function pickVersion(waId) {
  const bucket = createHash("sha256").update(waId).digest().readUInt16BE(0) % 100;
  return bucket < CANARY_PERCENT ? "canary" : "stable";
}
 
// Webhook router: verify, pick the version, hand off, acknowledge
app.post("/webhooks/whatsapp", verifySignature, async (req, res) => {
  const value = req.body.entry?.[0]?.changes?.[0]?.value;
  const waId = value?.contacts?.[0]?.wa_id ?? value?.statuses?.[0]?.recipient_id ?? "unknown";
  await queues[pickVersion(waId)].add("event", req.body); // split batched events first in production
  res.sendStatus(200);
});

Hashing on the customer's WhatsApp ID keeps every conversation on one version while the slice grows.

A canary release: key facts and limits

Typical first step
1–5% of traffic
Typical steps
5% → 25% → 50% → 100% (illustrative)
Compare with
A same-size baseline of the stable version
Gate metrics
Error rate, p95/p99 latency and a business signal
Routing key for WhatsApp
wa_id or tenant ID, so routing is sticky
Contrast
Blue-green switches 100% of traffic at once

Common mistakes with a canary release

  • Comparing with the whole fleet

    Long-running instances have warm caches and a different traffic mix, so a brand-new canary looks worse for reasons unrelated to the code. Compare it with a fresh baseline of the stable version at the same size.

  • Deciding on too little traffic

    At 1% of a quiet hour a canary may see 40 requests — far too few to tell 1% errors from 3%. Set a minimum sample per step and wait for it, or raise the slice.

  • Non-sticky routing

    Random per-request routing sends one customer's messages to both versions, mixing bot states and hiding conversation-level regressions. Hash on wa_id or conversation ID.

  • Gating only on HTTP errors

    A bot that answers 200 and passes every Health Check but sends customers into the wrong flow looks perfect to infrastructure metrics. Add a business signal — completion rate, handoff rate, template rejections — to the gate.

  • Forgetting shared state

    A canary that writes a new database schema or queue message format can break the stable version beside it. Keep changes backward-compatible during the rollout and migrate in separate, reversible steps.

Best practices for a canary release

  • Automate the rollback; a canary that waits for a human to notice is just a slow full rollout.
  • Hold at least one step through a traffic pattern that matters — a broadcast send or a peak hour.
  • Pause canaries while the error budget is exhausted, so risky releases wait for a healthier month.
  • Ship bot copy and flow changes behind a feature flag as well, so content can roll back without a redeploy.
  • Keep Load Testing before release: it proves capacity, while the canary proves behaviour on real traffic.
  • Review every automatic rollback through your incident management process, even when only 5% of users saw the bad version.

Canary Release FAQs

What is the difference between a canary release and blue-green deployment?+
A canary shifts traffic gradually and gates each step on metrics; blue-green switches all traffic at once between two full environments. Blue-green rolls back quickly, but every user meets the new version at the moment of the switch.
What percentage of traffic should a canary get?+
Start with 1–5%, enough to gather a meaningful sample within minutes to an hour. Low-traffic services may need a larger first slice or a longer hold, because the decision needs enough events to separate a real regression from noise.
How long should each canary step last?+
Long enough to reach your minimum sample and cover a representative traffic pattern — commonly 15 to 60 minutes per step. Steps that include a broadcast or a peak hour catch load-related regressions that quiet periods hide.
Can I canary a WhatsApp chatbot?+
Yes — route a sticky share of conversations, hashed by `wa_id`, to the new bot version and compare completion and handoff rates as well as errors. Keep each conversation on one version so a customer never switches flows mid-chat.

Talk to InfiQ

From definition to your first message

InfiQ is an official Meta Business Partner. Start a 7-day free trial — no card needed — and see this running on your own number.

Start free trialBrowse all 550 terms