What is a large language model (LLM)?
A large language model (LLM) is a neural network — almost always a transformer, the architecture introduced in 2017 — trained on very large amounts of text to predict the next token. A token is a piece of a word: in English one token averages roughly four characters, and Indian scripts usually need more tokens for the same meaning. Generating a reply means repeating one step: read everything so far, predict a probability for every possible next token, pick one, append it, and go again until the model emits a stop token or hits its output limit.
That simple loop explains the four things every WhatsApp bot builder has to plan for. Cost: providers bill per input and per output token, and the input includes the system prompt, tool definitions, retrieved knowledge and the whole conversation history on every turn. Latency: the reply streams token by token, so long answers take longer. Non-determinism: tokens are sampled, so the same question can get differently worded answers. Limits: the context window caps how much text the model can consider at once.
An LLM is the engine, not the agent. On its own it only produces text: it does not know today's order status, cannot call your API and remembers nothing between requests except what you send again. An AI agent wraps the model with a system prompt, retrieval-augmented generation (RAG) over a knowledge base, tool calling, conversation state and guardrails — and that surrounding system is where most of the reliability of a WhatsApp bot is won or lost.
Key takeaways
- A large language model (LLM) predicts the next token of text, over and over, to write a reply — that is the whole mechanism.
- Every request resends the instructions, history and retrieved knowledge, so input tokens usually dominate the bill.
- LLMs are non-deterministic and have no memory between requests: your system supplies the context every time.
- An LLM is the engine, not the agent — tools, retrieval, state and guardrails around it make an AI agent.
- Meta has charged Business Agent messages per token since 1 August 2026, at USD 2.00 per million tokens.
Figure 1
Where the tokens go in a six-turn chat
- System prompt + tool definitions
- History + retrieved knowledge
- Generated reply
Stacked · tokens per request (illustrative)
Largest total: Turn 6 at 8,330 tokens per request (illustrative).
Hover, focus or tap a column for exact values; press it to pin.
| Category | System prompt + tool definitions | History + retrieved knowledge | Generated reply | Total |
|---|---|---|---|---|
| Turn 1 | 2,200 | 1,800 | 140 | 4,140 |
| Turn 2 | 2,200 | 2,600 | 180 | 4,980 |
| Turn 3 | 2,200 | 3,400 | 120 | 5,720 |
| Turn 4 | 2,200 | 4,300 | 160 | 6,660 |
| Turn 5 | 2,200 | 5,100 | 150 | 7,450 |
| Turn 6 | 2,200 | 6,000 | 130 | 8,330 |
Why it matters
Why does a large language model (LLM) matter?
Understanding the token loop turns vague AI decisions into numbers. It tells a business what a conversation will cost and why long chats cost more per turn, tells a developer where latency comes from and why answers vary between test runs, and explains why a model stated a refund policy you never wrote — it predicted plausible text because nothing in its context said otherwise. Those facts decide model size, prompt design, how much history to keep, and whether to use Meta's hosted Business Agent or your own stack.
How does a large language model (LLM) work?
Everything becomes tokens
The system prompt, tool definitions, retrieved knowledge, conversation history and the customer's new message are concatenated and tokenised. Nothing persists between requests, so all of it is sent again each turn.
Attention scores the context
The transformer relates every token to every other token in the context window and produces a probability distribution over the next token.
One token is sampled
Settings such as temperature and top-p decide how adventurous the pick is. Lower values make answers more consistent, but repeated runs can still differ.
The loop repeats and streams
Each new token is appended and the step runs again, until a stop token or the maximum output length. Output is generated sequentially, so reply length drives latency.
Your code takes over
The text (or a structured output) is checked, trimmed to WhatsApp's 4,096-character text limit and sent through the Cloud API — or turned into a tool call your code executes.
Example: Pricing a course-enquiry bot in Ahmedabad
A coaching institute in Ahmedabad pilots an LLM bot for course enquiries. Logging one week shows a typical turn sends about 2,200 tokens of instructions and tool definitions, 2,000–6,000 tokens of retrieved course details and history, and gets back only 120–180 tokens. By turn six, history alone has tripled the input. The team summarises older turns instead of resending them verbatim, retrieves three course chunks instead of eight, and caps replies at 80 words — cutting input tokens per conversation by about 40% and making replies arrive noticeably faster. For scale, Meta quotes its own Business Agent at about 20,000–25,000 tokens per message.
A large language model (LLM) in code
// Any LLM provider SDK works here; request and response shapes vary by provider.import { llm } from "./llm-client.js";export async function replyWithLlm({ waId, phoneNumberId, history, question, kbChunks }) {const messages = [{ role: "system", content: SYSTEM_PROMPT }, // resent every turn{ role: "system", content: "Facts:\n" + kbChunks.join("\n") }, // retrieved knowledge (RAG)...history.slice(-8), // trim history: input tokens cost money{ role: "user", content: question },];const out = await llm.generate({ messages, maxOutputTokens: 300, temperature: 0.2 });const text = out.text.slice(0, 4096); // WhatsApp text body limitlog.info({ waId, inputTokens: out.usage.input, outputTokens: out.usage.output });await fetch(`https://graph.facebook.com/v26.0/${phoneNumberId}/messages`, {method: "POST",headers: { Authorization: `Bearer ${process.env.ACCESS_TOKEN}`, "Content-Type": "application/json" },body: JSON.stringify({ messaging_product: "whatsapp", to: waId, type: "text", text: { body: text } }),});}
The model call is the middle of the function; trimming history, logging token usage and enforcing WhatsApp limits are the parts that keep it affordable and safe.
Common mistakes with a large language model (LLM)
Treating the model as a database
An LLM produces plausible text, not looked-up facts, so it will state prices or policies it was never given. Ground answers in retrieved knowledge and refuse when the knowledge is missing.
Budgeting only for the reply
The reply is a sliver of the tokens; the resent prompt, tools and history are most of them. Summarise old turns, retrieve fewer and better chunks, and measure tokens per conversation.
Expecting identical answers
Sampling means wording changes between runs, so a single manual test proves little. Use evaluation sets with several runs per case before changing a prompt or model.
Assuming English quality carries over
Accuracy, tone and token counts differ in Hindi, Tamil and Hinglish. Test the languages your customers use and watch how many tokens they consume.
Letting replies run long
Long answers are slower to generate and harder to read on a phone. Cap output length and ask for short, specific replies with one next step.
Best practices for a large language model (LLM)
- Log input and output tokens per conversation from day one — it is the only reliable cost model.
- Keep stable instructions at the start of the prompt; many providers discount repeated prefixes through prompt caching.
- Use a smaller, faster model for routing and classification, and a larger one only where reasoning is needed.
- Ask for structured output when the reply feeds code, such as template variables or CRM fields.
LLM FAQs
- Is an LLM the same as an AI agent?
- No. The LLM is the text-generating engine; an AI agent is the system around it that adds instructions, retrieval, tool calling, memory of the conversation and guardrails so it can actually look things up and act.
- Why does the same question get different answers from an LLM?
- Because each token is sampled from a probability distribution. Lower temperature makes answers more consistent, but wording can still vary, so judge quality across many runs rather than one.
- How are LLMs charged?
- Usually per token, counted separately for input and output. Meta's Business Agent uses one global rate of USD 2.00 per million tokens covering both, and has done since 1 August 2026.
- Do LLMs understand Hindi and other Indian languages?
- Most current models handle Hindi and major Indian languages, with quality varying by language and model. Romanised Hinglish and code-mixed text need explicit testing, and Indian scripts often use more tokens per word.
- Does an LLM remember previous WhatsApp conversations?
- Not by itself. Each request is independent, so any memory comes from your system resending history or a stored summary. Keep conversation state in your database, not in the model.