Skip to content

A glossary story9 chapters · 5 min read

Structured Output (JSON Mode)

Constraining an LLM to reply in JSON that matches a schema you define, so your code can turn it into buttons, template variables or CRM fields.

Also called: Structured outputs · JSON mode · Schema-constrained generation · Constrained decoding

Reviewed

Follow along

Read the schema that keeps reply buttons valid

5 / 5
{
"type": "object",
"properties": {
"body": { "type": "string", "maxLength": 1024 },
"buttons": {
"type": "array",
"minItems": 1,
"maxItems": 3,
"items": {
"type": "object",
"properties": {
"id": { "type": "string", "enum": ["slot_am", "slot_pm", "slot_eve"] },
"title": { "type": "string", "maxLength": 20 }
},
"required": ["id", "title"],
"additionalProperties": false
}
}
},
"required": ["body", "buttons"],
"additionalProperties": false
}

Step 5 of 5 · lines 15–21

Required fields and no extra keys, for each button and for the whole reply: exactly the shape the send code expects.

Jump to
Two files from a delivery-slot bot: the JSON Schema that encodes WhatsApp's button limits, and the code that validates the model's reply and maps it to a Cloud API interactive message.

slot-reply.schema.json, step 5 of 5, lines 15–21: Required fields and no extra keys, for each button and for the whole reply: exactly the shape the send code expects.

Chapter 1 · The scenario

Example: A delivery-slot picker that stopped breaking

A Bengaluru grocery bot used to ask the model for "a friendly message with three slot options" and split the text into buttons. Titles like "Tomorrow morning (7–9 am, express)" ran past 20 characters and the send failed, leaving customers with no reply. The team switched to a schema: a body of up to 1,024 characters, one to three buttons, each id drawn from that day's real slot IDs and each title capped at 20 characters. Button-related send errors stopped, and because the IDs are real, a tap maps straight to the booking API.

— A delivery-slot picker that stopped breaking

Chapter 2 · What is structured output (JSON mode)?

Structured output — often loosely called JSON mode — is a way of making a large language model (LLM) answer in a machine-readable format — almost always JSON — that matches a schema you define, instead of free-form prose. The schema names the fields, their types, which ones are required, the allowed values (enums) and length limits. Your code can then parse the reply straight into a WhatsApp interactive message, the values for each message template variable, or the fields of a CRM record.

It comes in two strengths. JSON mode, the older and looser option, only guarantees that the output parses as JSON; the model can still omit fields, invent keys or use the wrong types. Schema-constrained output, often called structured outputs, restricts generation token by token so the result always conforms to the schema, within the subset of JSON Schema the provider supports. Either way the content can still be wrong: a well-formed object with a made-up delivery date is still made up.

Structured output shapes the model's answer; tool calling (function calling) asks your code to do something. Both use JSON Schemas, and some providers implement one with the other, but the intent differs: a structured reply is the end product ("here are three reply buttons"), while a tool call is a request mid-conversation ("run check_order_status for order 48213 and tell me the result"). Entity extraction is a common use of structured output — pulling an order ID, a date and a PIN code out of a message into typed fields.

Key takeaways

  • Structured output constrains an LLM to return JSON that matches your schema instead of free prose.
  • Plain JSON mode guarantees valid JSON; schema-constrained output also guarantees the fields and types you asked for.
  • Put WhatsApp's limits in the schema: at most 3 reply buttons, each title up to 20 characters.
  • Validate every response in code anyway — refusals, truncation and business rules are not caught by the schema.
  • Structured output shapes the model's answer; tool calling asks your code to take an action.

Chapter 3 · How it unfolds

How does structured output (JSON mode) work?

  1. Design the schema from the destination

    Start from the payload you will send. For reply buttons: a body string and a buttons array of at most 3 items, each with an id from an enum and a title with maxLength 20. Mark fields required and disallow extra properties.

  2. Request structured output

    Pass the schema with the request using your provider's structured-output or JSON-mode option, and describe each field — in the system prompt or in the schema's description keywords.

  3. Validate in code

    Parse and validate the reply with a schema library even when the provider promises conformance. Then check the business rules a schema cannot express, such as whether a slot is still free.

  4. Handle the edge cases

    Treat refusals, truncated output and validation failures as normal paths: retry once with the validation error, then fall back to a plain text reply or a fixed menu.

  5. Map to the WhatsApp payload

    Convert the validated object into the Cloud API body — an interactive message, or the parameters for each template variable — and send it. Log the raw model output next to the wamid for debugging.

Chapter 4 · In code

Structured output (JSON mode) in code

from typing import Literal
from pydantic import BaseModel, Field
 
class Button(BaseModel):
    id: Literal["slot_am", "slot_pm", "slot_eve"]    # only real slot IDs
    title: str = Field(max_length=20)                # WhatsApp reply-button limit
 
class SlotReply(BaseModel):
    body: str = Field(max_length=1024)
    buttons: list[Button] = Field(min_length=1, max_length=3)
 
raw = llm.generate(
    system="Offer the customer today's delivery slots. Be brief and friendly.",
    user=customer_text,
    response_schema=SlotReply.model_json_schema(),   # option name varies by provider
)
reply = SlotReply.model_validate_json(raw)           # raises if anything is off

One set of types drives both the schema sent to the model and the validation of its reply.

Chapter 5 · Where it goes wrong

Common mistakes with structured output (JSON mode)

  • Assuming every schema keyword is enforced

    Providers support different subsets of JSON Schema; length limits, patterns or array bounds may be ignored in strict mode. Enforce WhatsApp's limits again in your own validation.

  • Parsing JSON-mode output without validation

    JSON-mode output can parse cleanly and still miss a required field or use a string where you expected an array. Validate against the schema before you build a payload.

  • Letting the model invent IDs

    If button or list-row IDs are free strings, the model will make some up and taps will route nowhere. Constrain IDs to an enum built from the real options for this conversation.

  • No fallback path

    Refusals, timeouts and truncated output happen. Keep a fixed menu or plain-text reply ready so the customer still gets an answer when validation fails.

  • Newlines in template variables

    Template body parameters cannot contain newlines or tabs, and long values push a template past its character limits. Strip, trim and length-check generated values before you send them.

Chapter 6 · The facts

Structured output (JSON mode): key facts and limits

JSON mode guaranteesSyntactically valid JSON only
Schema-constrained output guaranteesJSON matching your schema (supported keywords only)
Reply buttonsMax 3; title ≤ 20 characters; ID ≤ 256 characters
List messagesMax 10 rows; row title ≤ 24, description ≤ 72 characters
Text message bodyMax 4,096 characters
Template body variablesNo newlines, tabs or more than 4 consecutive spaces

Chapter 7 · Why it matters

Why does structured output (JSON mode) matter?

WhatsApp is strict about message shapes. A reply-button message allows three buttons with titles of up to 20 characters; a list message allows ten rows with 24-character titles; a template body variable cannot contain a newline. Free-text LLM output breaks those rules unpredictably, and the send fails with an API error, usually while a customer is waiting. Structured output turns the model into a component that returns exactly the object your send code expects, so failures become rare, typed and catchable.

Chapter 8 · What good teams do

Best practices for structured output (JSON mode)

  • Generate the schema from the same types your send code uses, so the two never drift apart.
  • Keep schemas small and flat — deep nesting raises error rates and latency.
  • Add a description to every field; models follow field descriptions closely.
  • Log validation failures with the raw output — they are the best guide to fixing the prompt.

Chapter 9 · Structured Output (JSON Mode) FAQs

Structured Output (JSON Mode) FAQs

What is the difference between JSON mode and structured outputs?+
JSON mode guarantees valid JSON; structured outputs also guarantee it matches your schema. With JSON mode you must check fields and types yourself; with schema-constrained output the shape is enforced during generation, within the keywords your provider supports.
Is structured output the same as function calling?+
No, though both use JSON Schemas. Structured output shapes the final answer the model gives you; function calling is the model asking your code to run an action and return a result before it answers.
Do I still need to validate structured output?+
Yes. Validation catches unsupported schema keywords, refusals, truncated responses and business rules a schema cannot express, such as whether an order belongs to the customer.
Can an LLM fill WhatsApp template variables directly?+
Yes, with care. Generate the values as structured output, validate their length and characters, and send them as template parameters. The template text itself must already be approved, so the model only supplies the variable values.

See InfiQ pricing

From definition to your first message

InfiQ is an official Meta Business Partner. Start a 7-day free trial — no card needed — and see this running on your own number.

Start free trialBrowse all 550 terms