Language Detection for Hindi, Hinglish & Indian Bots

Chapter 1 · The scenario

Example: Marathi customers in Nagpur

An appliance-repair company in Nagpur notices that customers writing in Devanagari are all receiving Hindi replies — including the many who write Marathi. The detector was inferring language from script alone. The team adds a language identifier that separates Hindi from Marathi within Devanagari, registers Marathi translations (mr) of its booking and visit-reminder templates, and asks low-confidence customers once with two reply buttons, "हिंदी" and "मराठी", storing the answer on the contact. Escalations to agents for "wrong language" drop, and reminder replies arrive in the customer's own words.

— Marathi customers in Nagpur

Chapter 2 · What is language detection?

Language detection, also called language identification (LID), is working out which language a message is written in and which script it uses, so a bot can answer in the same way. In India the two questions come apart constantly: a customer may write Hindi in Devanagari ("मेरा ऑर्डर कब आएगा?"), the same sentence in Latin letters ("mera order kab aayega?"), English, Tamil in Tamil script, or a code-mixed blend of all of these within one message.

Script is the easy half. Every character belongs to a Unicode block — Devanagari U+0900–U+097F, Bengali U+0980–U+09FF, Tamil U+0B80–U+0BFF and so on — so counting characters tells you the script deterministically. Language is the statistical half: Devanagari is shared by Hindi, Marathi, Nepali and Konkani, and Latin script carries English, romanised Hindi (Hinglish), romanised Tamil and more. That needs a trained identifier or an LLM, and it comes with a confidence score.

The output drives everything downstream: which template translation to send, which WhatsApp Flow variant to open, which instructions to give the large language model, which agent queue to route to, and which words natural language understanding (NLU) should expect. This page covers detecting the language; translating into it, and judging translation quality, are separate problems.

Key takeaways

Language detection identifies both the script and the language of a message, so a bot can reply the way the customer writes.
Script is not language: Devanagari is shared by Hindi and Marathi, and Latin letters carry English as well as Hinglish.
Messages like "ok", "hi" or an emoji carry no language signal — keep the conversation's last confident language.
WhatsApp does not tell you a user's app language, so detection from the text, plus an explicit language choice, is what you have.
Map the result to the template translations you actually registered, such as hi, ta or en_US, before sending.

Chapter 3 · Why it matters

Why does language detection matter?

Customers answer faster and more fully when a business writes back the way they wrote. A bot that replies in formal English to "bhai refund kab milega", or in Devanagari to someone who types Hinglish, feels like a form letter and gets more "agent please" escalations. For marketers it decides which template translation a segment receives; for developers it decides which model prompt, Flow and knowledge-base language to use — and getting it wrong on a two-word message can flip a whole conversation into the wrong language.

Chapter 4 · How it unfolds

How does language detection work?

Find the dominant script
Count characters per Unicode block and take the majority, ignoring digits, punctuation and emoji. This is exact and costs microseconds.
Identify the language within the script
Use a character n-gram language identifier or an LLM to choose between languages that share a script — Hindi versus Marathi in Devanagari, English versus Hinglish in Latin — and keep its confidence score.
Treat romanised text as its own case
General identifiers trained on standard text often label Hinglish as English. Use a model or word list built on code-mixed chat data, and record the result as a language plus script, such as Hindi in Latin script.
Apply thresholds and memory
If the message is too short or confidence is low, keep the conversation's last confident language or the customer's saved preference, and offer an explicit "Change language" option instead of guessing.
Act on the result
Pick the matching template language.code, Flow and knowledge-base articles, and tell the LLM explicitly how to reply — for example "answer in Hindi written in Latin script".

Chapter 5 · In code

Language detection in code

1import re
2 
3SCRIPTS = {
4    "Devanagari": (0x0900, 0x097F), "Bengali": (0x0980, 0x09FF),
5    "Gurmukhi": (0x0A00, 0x0A7F),   "Gujarati": (0x0A80, 0x0AFF),
6    "Tamil": (0x0B80, 0x0BFF),      "Telugu": (0x0C00, 0x0C7F),
7    "Kannada": (0x0C80, 0x0CFF),    "Malayalam": (0x0D00, 0x0D7F),
8}
9HINGLISH = {"kab", "kya", "hai", "nahi", "mera", "meri", "karo", "bhejo",
10            "chahiye", "kaise", "aayega", "kitna", "abhi", "bhai"}
11 
12def dominant_script(text: str) -> str | None:
13    counts: dict[str, int] = {}
14    for ch in text:
15        cp = ord(ch)
16        if ch.isascii() and ch.isalpha():
17            counts["Latin"] = counts.get("Latin", 0) + 1
18        for name, (lo, hi) in SCRIPTS.items():
19            if lo <= cp <= hi:
20                counts[name] = counts.get(name, 0) + 1
21    return max(counts, key=counts.get) if counts else None
22 
23def detect(text: str, last: str | None) -> tuple[str, float]:
24    """Returns (language tag, confidence). Low confidence -> keep 'last'."""
25    script = dominant_script(text)
26    words = re.findall(r"[a-z]+", text.lower())
27    if script is None or len(text.strip()) < 6:
28        return (last or "en", 0.0)          # "ok", "hi", an emoji: no signal
29    if script == "Tamil":
30        return ("ta", 0.95)
31    if script == "Devanagari":
32        return ("hi", 0.6)                  # could be Marathi: confirm with a model
33    if script == "Latin":
34        share = sum(w in HINGLISH for w in words) / max(len(words), 1)
35        return ("hi-Latn", 0.75) if share >= 0.25 else ("en", 0.7)
36    return (last or "en", 0.3)

A deterministic first pass. Production systems add a trained identifier for languages that share a script and for code-mixed text.

Chapter 6 · Where it goes wrong

Common mistakes with language detection

Equating script with language
Devanagari does not mean Hindi and Latin letters do not mean English. Add a language step within each script, or you will answer Marathi speakers in Hindi and Hinglish speakers in English.
Switching language on every message
A customer who has written Tamil all along and then sends "ok" should not get an English reply. Make language sticky per conversation and change it only on a confident signal or an explicit choice.
Answering Hinglish in formal Hindi
Customers who type Hindi in Latin letters often find Devanagari replies harder to read. Mirror the script as well as the language unless the customer asks otherwise.
Detecting a language you cannot serve
Sending a template with a language code that has no approved translation fails. Keep the detection output mapped to the translations, Flows and articles that actually exist, with a defined fallback.

Chapter 7 · The facts

Language detection: key facts and limits

Scheduled languages of India	22 (Eighth Schedule of the Constitution)
Devanagari Unicode block	U+0900–U+097F — Hindi, Marathi, Nepali and more
Tamil Unicode block	U+0B80–U+0BFF
Template language codes (examples)	hi, mr, ta, te, bn, en_US
User's app language in webhooks	Not provided — detect from text or ask
SMS fallback in Indian scripts	UCS-2: 70 characters per single SMS

Chapter 8 · What good teams do

Best practices for language detection

Store the detected language and script on the contact, with the date and how it was decided.
Offer a one-tap language switch in the menu — explicit choice always beats inference.
Run detection on voice-note transcripts too, after speech-to-text.
Report bot accuracy per language, not only overall, so a weak Tamil flow is visible.

Chapter 9 · Language Detection FAQs

Language Detection FAQs

How do I detect Hinglish in a WhatsApp message?+

Check that the text is in Latin script, then use a model or word list trained on code-mixed chat to separate romanised Hindi from English. General language identifiers trained on standard text often mislabel Hinglish as English.

Is language detection the same as translation?+

No. Detection decides which language and script a message is in; translation converts text between languages. A bot can detect Tamil and reply from pre-written Tamil content without translating anything.

Does WhatsApp tell me the customer's language?+

No. The messages webhook gives you the text, the sender's profile name and WhatsApp ID, but not the app's language setting. Detect it from what the customer writes, or ask once and store the answer.

What should a bot do when a message is too short to detect?+

Keep the language it already had. Single words, greetings, numbers and emoji carry little signal, so fall back to the conversation's last confident language or the saved preference instead of switching.

See InfiQ pricing

	Script	Language	Confidence	Reply with
मेरा ऑर्डर कब आएगा?
mera order kab aayega bhai
என் ஆர்டர் எப்போது வரும்?
When will my order arrive?
माझी ऑर्डर कधी येणार?
ok 👍

Language Detection

Same question, six ways to write it

Example: Marathi customers in Nagpur

Chapter 2 · What is language detection?

Why does language detection matter?

How does language detection work?

Find the dominant script

Identify the language within the script

Treat romanised text as its own case

Apply thresholds and memory

Act on the result

Language detection in code

Common mistakes with language detection

Equating script with language

Switching language on every message

Answering Hinglish in formal Hindi

Detecting a language you cannot serve

Language detection: key facts and limits

Best practices for language detection

Language Detection FAQs

See how this works on InfiQ

From definition to your first message

See how this works on InfiQ