Building Multilingual WhatsApp Bots with AI Memory

Conversation memory allows your WhatsApp AI to remember what products a customer asked about, what issues they faced, and what preferences they expressed.

How conversation memory works

Every conversation turn is stored with metadata:

• Timestamp
• User ID
• Topic

WhatsApp conversations can span dozens of messages in a single session. Managing context efficiently prevents the LLM from losing track of earlier parts of the conversation while controlling API costs.

Sliding window approach

• Keep the most recent 8–10 messages in full
• Maintain a compressed summary of earlier messages
• Preserve the thread of longer conversations while keeping token usage predictable

Topic-based segmentation

When a conversation shifts topics (e.g., from order inquiry to product question), the system:

• Detects the topic transition
• Compresses the previous topic's context into a short summary
• Gives full attention (full messages) to the new topic

Token budget management

For each LLM call, allocate tokens across three main pools plus output:

• System prompt and instructions: 500–800 tokens (fixed)
• Retrieved context (memory + RAG): 1000–2000 tokens (variable)
• Conversation history: 1000–1500 tokens (sliding window)
• Response generation: 500–1000 tokens (output)

This keeps total token usage under 4000–5000 tokens per turn, balancing response quality with API cost.

At Rs 0.15–0.30 per LLM call, this architecture supports 10,000+ daily conversations within a manageable budget.

India's linguistic diversity means your WhatsApp AI must handle multiple languages naturally. Multilingual support for Indian users is far more complex than simple translation.

The real challenge — code switching

Indian users rarely speak pure Hindi or pure English. They mix languages mid-sentence:

"Mujhe ek blue color ka kurta chahiye under 2000 rupees."

This code-switching (Hinglish) breaks traditional NLU systems that expect a single language per message.

Our approach to multilingual WhatsApp AI

1. Language detection per message

• Use a lightweight classifier (fastText or custom model)
• Detect the dominant language of each message
• Target 95%+ accuracy for Hindi, English, Tamil, Telugu, and Hinglish

2. Code-switch aware processing

• Avoid blindly translating everything to English
• Process code-switched text directly using multilingual LLMs (Claude 4, GPT-4 handle Hinglish well)
• Use multilingual embeddings (e.g., Cohere embed-v3) for RAG retrieval
• Prevent translation errors, especially for product names, brand names, and numbers that should remain unchanged

3. Response language matching

• AI responds in the language the customer uses
• If a customer writes in Hindi, the response is in Hindi
• If they switch to English mid-conversation, the AI switches too
• No need for the customer to manually select a language preference

4. Regional language support

Beyond Hindi and English, Indian businesses need:

• Tamil
• Telugu
• Kannada
• Malayalam

WhatsApp isn't just text. Customers send images, voice notes, documents, and location pins. An advanced AI agent must handle all of these.

Image processing

• Product photos:
• Example: "Is this available?"
• AI uses vision models to identify the product and check inventory
• Defect photos:
• Example: "My order arrived damaged"
• AI classifies the defect type
• Triggers the appropriate return/replacement flow
• Screenshots of errors:
• Example: "I'm getting this error during checkout"
• AI reads the screenshot and troubleshoots the issue

Voice note processing

Many Indian customers, especially in Tier 2–3 cities, prefer voice notes over typing.

Pipeline:

1. Receive voice note via WhatsApp webhook
2. Transcribe using Whisper (supports Hindi, Tamil, Telugu, and code-switched audio)
3. Process transcribed text through the standard NLU/LLM pipeline
4. Respond with text (or optionally generate a voice response)

Voice note support dramatically increases accessibility and adoption in non-metro markets where typing in regional scripts is cumbersome.

Document handling

Customers frequently send:

Beyond simple conversation memory, advanced WhatsApp AI uses user profile embeddings for deep personalization.

Building user profiles

Every interaction updates a structured user profile, including:

• Purchase history:
• Products bought, categories, price ranges, frequency
• Browsing interests:
• Products asked about but not purchased
• Communication style:
• Formal vs casual, language preference, emoji usage
• Support history:
• Past issues, resolution satisfaction, escalation patterns
• Engagement patterns:
• Active hours, response speed, preferred message types

Profile embeddings for recommendations

• Convert user profiles into vector embeddings
• Compare them against product embeddings
• Enable "customers like you also bought" recommendations that are genuinely relevant

Example:

• A customer who consistently buys organic skincare products will not receive recommendations for synthetic alternatives.

Segment-based campaigns

User profile vectors enable intelligent segmentation for WhatsApp broadcast campaigns:

• Move from one-size-fits-all blasts to micro-segments based on behavioral similarity
• Send targeted messages with significantly higher conversion rates

Privacy and consent

All personalization respects user consent and regulatory requirements:

• Customers can request their data via WhatsApp (e.g., "Show me my data" or "Delete my history")
• Compliant with India's DPDP Act 2023
• Profile data is encrypted at rest and in transit
• Access is restricted to the AI processing pipeline only

Boolean & Beyond builds WhatsApp AI agents with production-grade memory systems, multilingual intelligence, and media processing capabilities.

Our systems:

• Handle Hindi–English code-switching with 95%+ accuracy
• Maintain conversation context across sessions using Redis + vector databases + PostgreSQL
• Process voice notes and images natively

We deploy memory-enabled WhatsApp agents for Indian enterprises across:

• D2C and e-commerce
• Healthcare
• Financial services
• Education

Explore more from our AI solutions library:

• How to Build a Multilingual Voice AI Agent for Indian Languages — Extend your multilingual AI capabilities from WhatsApp to voice with Hindi, Tamil, and Kannada support.
• What is Model Context Protocol (MCP) and Why It Matters — Connect your WhatsApp bot to CRM, inventory, and internal tools using MCP for real-time data access.