Chapter 1 · The scenario
Example: Teaching trade shorthand, not prices
A Surat textile wholesaler's WhatsApp bot must turn messages such as "50 than rayon print 44 inch, same as last time, dispatch by Friday" into an order draft. A prompted general model got 81% of a 400-message test set right (illustrative figures), stumbling on trade shorthand — "than" for a bolt of cloth, width codes, repeat-order phrasing. The team fine-tuned a small open-weight model with LoRA on 1,800 reviewed past orders, kept 300 aside for testing, and reached 95% on the same test set at a fraction of the per-message inference cost. Prices and stock were deliberately left out of the training data: the bot still looks them up through retrieval and the inventory API, because they change every week.
Chapter 2 · What is fine-tuning?
Fine-tuning is the process of taking a pretrained large language model (LLM) and training it further on a smaller set of your own examples, so its behaviour shifts towards what those examples show. Supervised fine-tuning (SFT) uses pairs of input and ideal output — a customer message and the reply your best agent would send — while preference tuning uses pairs of better and worse answers. Either way the result is a new set of model weights, served under its own model ID.
Full fine-tuning updates every weight and needs serious GPU capacity. Parameter-efficient methods such as LoRA (low-rank adaptation) train a small set of extra adapter weights on top of a frozen base model, which makes tuning cheaper and faster and lets several adapters share one base. Hosted model platforms usually expose fine-tuning as an upload of JSONL training files and a training job, then bill the tuned model at its own rate.
What fine-tuning changes is behaviour: tone of voice, reply length, a strict output format, how to classify or route messages, the vocabulary of a trade. What it does not reliably do is store facts. A model tuned on last quarter's price list will repeat stale prices fluently and cannot say where they came from. That is why retrieval-augmented generation (RAG) — which fetches current documents into the prompt at query time — is usually the first choice for factual business knowledge, and fine-tuning is kept for gaps that a good system prompt and RAG cannot close.
Key takeaways
- Fine-tuning continues training a pretrained model on your own input–output examples, producing new model weights.
- It is good at teaching style, format and narrow tasks; it is a poor way to teach facts that change.
- For prices, policies and stock, use retrieval-augmented generation (RAG): it changes the prompt, not the model.
- Expect to need hundreds to a few thousand clean, reviewed examples plus a held-out test set.
- Every base-model upgrade or data change means retraining and re-evaluating the tuned model.
Chapter 3 · Why it matters
Why does fine-tuning matter?
Teams often reach for fine-tuning to "teach the bot our business" and end up with an expensive model that still invents prices. Knowing what it is actually for saves months: most WhatsApp bots get further with a clear system prompt, a well-kept knowledge base and RAG. Fine-tuning earns its cost when you need consistent behaviour at scale — a small model that classifies intents cheaply, replies in a house style in Hinglish, or always produces exact JSON — and you have the data and evaluation discipline to prove it helps.
Chapter 4 · How it unfolds
How does fine-tuning work?
Exhaust prompting and retrieval first
Write the system prompt, add a few worked examples and ground answers in your knowledge base. Measure on an evaluation set. Fine-tune only if a specific, repeatable gap remains.
Collect and clean examples
Gather real conversations that show the target behaviour, remove personal data, correct wrong answers and balance the cases. A few hundred excellent examples beat thousands of noisy ones.
Format and split the data
Write chat-format JSONL — one example per line with system, user and assistant turns — and hold back 10–20% as a test set the model never trains on.
Train, usually with an adapter
Run a supervised job: LoRA on an open-weight model or a hosted tuning job. Watch training and validation loss and stop before the model starts memorising the training set.
Evaluate against the baseline
Compare the tuned model with the prompted base model on the held-out set and on safety cases — accuracy, format validity, tone, refusals and escalations. Ship only on a clear win.
Version, monitor and retrain
Pin the model ID, keep the dataset under version control, and plan a retrain when the base model is retired or upgraded or when live behaviour drifts.
Chapter 5 · In code
Fine-tuning in code
{"messages":[{"role":"system","content":"Turn wholesale textile orders into JSON. Reply with JSON only."},{"role":"user","content":"50 than rayon print 44 inch, dispatch by Friday"},{"role":"assistant","content":"{\"items\":[{\"fabric\":\"rayon print\",\"width_in\":44,\"qty\":50,\"unit\":\"than\"}],\"dispatch_by\":\"2026-10-23\"}"}]}{"messages":[{"role":"system","content":"Turn wholesale textile orders into JSON. Reply with JSON only."},{"role":"user","content":"same as last order but 20 than cotton 58 inch"},{"role":"assistant","content":"{\"items\":[{\"fabric\":\"cotton\",\"width_in\":58,\"qty\":20,\"unit\":\"than\"}],\"repeat_of_last\":true}"}]}
One reviewed example per line, with the same system prompt the bot uses in production. No prices, no personal data.
Chapter 6 · The facts
Fine-tuning: key facts and limits
| What it changes | Model weights (RAG changes the prompt instead) |
|---|---|
| Typical dataset | Hundreds to a few thousand reviewed examples |
| Held-out test set | Commonly 10–20% of examples, never trained on |
| Common file format | Chat-style JSONL, one example per line |
| Parameter-efficient method | LoRA: small adapter weights on a frozen base |
| Retrain triggers | New base model, new behaviour or measured drift |
Chapter 7 · Where it goes wrong
Common mistakes with fine-tuning
Fine-tuning to teach facts
Prices, policies and stock change; a tuned model keeps repeating what it learnt, with no source to cite. Put facts in a knowledge base and retrieve them, and tune only for behaviour.
Training on unreviewed transcripts
Real chats contain wrong answers, curt replies and personal data, and the model learns all of it. Review, correct and mask before training — personal data inside weights cannot be selectively erased.
No baseline and no held-out set
Without comparing against the prompted base model on unseen data, you cannot tell whether tuning helped or just memorised. Evaluate both on the same test set.
Overfitting a small dataset
Too many epochs on too few examples and the model parrots training replies and loses general ability. Watch validation loss and test on new phrasings.
Forgetting the maintenance bill
A tuned model is tied to its base. When the base is retired or a better one ships, you retrain and re-evaluate — budget for it from the start.
Chapter 8 · What good teams do
Best practices for fine-tuning
- Start with the smallest model that might work; tuning often makes a small model competitive on one narrow task.
- Include examples of what the bot should refuse or escalate, not only happy paths.
- Combine the two: fine-tune for format and tone, retrieve for facts.
- Use exactly the same system prompt in training and in production.
Chapter 9 · Fine-Tuning FAQs