The Challenge of Mixed-Language Commands: Hinglish, Tamilish & Code-Switching in Voice AI

Author: Reji Adithian, Sr. Marketing Manager | Published: March 2026

Introduction: The Reality of Multilingual Voice Interaction

Imagine this scenario: A user in Bangalore speaks to their smart home device, naturally flowing between English and Hindi—"Alexa, mera ghar thandi hai, please increase the temperature" (Alexa, my home is cold, please increase the temperature). Or a Tamil Nadu professional giving voice commands to their business application: "Update the meeting with Ananya to 3 PM, aana weekend-kku nalla irukku?" (Would Sunday work for her?).

This isn't scattered English peppered with occasional Hindi or Tamil words. This is code-switching—the natural linguistic phenomenon where bilingual or multilingual speakers fluidly mix languages within the same sentence or conversation. Studies show that over 70% of Indian users naturally code-switch between languages in daily voice conversations, yet most commercial voice AI systems are built on monolingual assumptions. The result: significant accuracy degradation, frustration, and missed market opportunities for voice product builders.

For companies building voice AI for India and similar multilingual markets, understanding and handling code-switching isn't a nice-to-have—it's foundational. This blog explores the technical and business challenges of mixed-language voice recognition, why traditional ASR systems fail, and how modern approaches are reshaping the landscape of multilingual voice AI.

What is Code-Switching and Why It Matters for Voice AI

Understanding Code-Switching

Code-switching (or code-mixing) is the linguistic practice of alternating between two or more languages or language varieties in a single conversation. It's not broken English or poor language fluency—it's a sophisticated, rule-governed behavior exhibited by highly proficient multilingual speakers. In India, code-switching is ubiquitous across urban and semi-urban populations, spanning professions from software engineering to retail, from corporate meetings to family dinners.

There are three primary types of code-switching relevant to voice AI:

• Intra-sentential: Mixing languages within a single sentence ("Mera laptop ka battery 5% mein hai, plug in karo")—My laptop battery is at 5%, plug it in.
• Inter-sentential: Alternating languages between sentences ("Main kal Delhi ja raha hoon. I have a meeting with the product team tomorrow.")—I'm going to Delhi tomorrow.
• Tag-switching: Inserting tags or emotional phrases from one language ("You need to finish this report, haan? It's quite urgent.")—You need to finish this report, right?

Why Code-Switching is Critical for Voice AI

From a business perspective, code-switching represents both a challenge and an opportunity. Voice AI products that ignore code-switching alienate the majority of their potential users in multilingual markets. In India alone, this affects hundreds of millions of potential users. From a technical standpoint, code-switching disrupts every layer of traditional speech recognition pipelines:

• Language detection: Standard systems cannot determine which language to decode when code-switching occurs within a single utterance.
• Acoustic modeling: Speech acoustics change mid-sentence, violating the assumptions of single-language acoustic models.
• Language modeling: N-gram language models trained on monolingual corpora assign near-zero probability to perfectly valid code-switched sequences.
• Intent recognition: NLU systems struggle to extract intent when mixing languages disrupts expected token sequences.

For voice product teams, this means lower word error rates (WER) on real-world data, higher user frustration, and reduced adoption in multilingual regions.

The Hinglish Phenomenon: India's Dominant Voice Pattern

Hinglish in Everyday Commands

Hinglish—the blending of Hindi and English—is the most prevalent code-switching pattern in India. It dominates voice interactions, from casual queries to professional instructions. Here are representative examples of real-world Hinglish voice commands:

Hinglish Command (Transliteration)	English Translation	Linguistic Pattern
"Paani ki bottle lao, thandi garmi mein piyas lag gayi"	"Bring water bottle, I'm thirsty in this heat"	Intra-sentential; verbs in Hindi, nouns mixed
"Kal ka meeting 3 PM par reschedule kar do"	"Reschedule tomorrow's meeting to 3 PM"	Intra-sentential; English nouns (meeting, PM), Hindi frame
"Mujhe 50 rupees ka recharge karna hai, Jo aaj available hai"	"I need to buy a 50-rupee recharge available today"	Intra-sentential; numbers and units in Hindi, product names in English
"Check karo ki mera pending invoice kab payment ho gaya"	"Check when my pending invoice got paid"	Intra-sentential; English verbs (check, payment), Hindi frame
"10 baje se 2 PM tak meeting schedule hai, uske baad free ho jaana"	"I have meetings scheduled from 10 AM to 2 PM, then I'm free"	Inter-sentential; both languages for time expressions

Why Hinglish is Linguistically Consistent

Hinglish isn't random. Speakers follow consistent patterns: English technical terms, numbers, and product names coexist naturally with Hindi grammatical structures and everyday vocabulary. This consistency is crucial for ASR system design—models can learn these patterns if trained appropriately. However, traditional systems trained exclusively on English or Hindi fail precisely because they don't capture these predictable mixing patterns.

Research into Hinglish patterns reveals fascinating consistency rules. English nouns predominate in technology, commerce, and formal domains ("laptop," "invoice," "meeting"), while Hindi verbs, pronouns, and prepositions carry grammatical structure. This division of linguistic labor makes Hinglish highly learnable—a well-trained model can predict with reasonable confidence which language component will appear in which syntactic position. For instance, action verbs predominantly follow Hindi conjugation patterns ("kar do," "de do," "ho gaya"), while the objects and details are often English ("Check karo ki mera pending invoice kab payment ho gaya"). Understanding these patterns enables ASR systems to build probabilistic expectations about language mixing, dramatically improving transcription accuracy for predictable code-switched utterances.

Beyond Hinglish: Tamilish, Benglish, and Regional Code-Switching

The Diversity of Indian Language Mixing

While Hinglish dominates discussions, code-switching patterns vary across India's linguistic regions. Each major Indian language exhibits its own mixing patterns with English, creating distinct challenges for ASR systems. The linguistic heterogeneity of India—with 22 official languages and hundreds of dialects—means that a one-size-fits-all approach to code-switching fundamentally fails. A Hinglish-optimized system performs poorly for Tamil-English or Kannada-English speakers. Effective voice AI for India requires understanding regional code-switching preferences and building language-pair-specific models.

Tamilish: Tamil-English Code-Switching

Tamil Nadu and Tamilish-speaking communities exhibit distinctive code-switching patterns. Examples include:

• "Inga vandhiya customer-ku discount du, adi 20% ah irukkanum" (Give this customer a discount, it should be 20%)—mixing Tamil verbs with English nouns.
• "Meeting-nu scheduled aagirukku 4 PM ku, nee attend pannu" (The meeting is scheduled for 4 PM, you attend)—English temporal markers with Tamil grammar.
• "Enna recipe-la coconut oil use pannratha?" (Do you use coconut oil in this recipe?)—Tamil sentence frame with English noun.

Other Major Language Mixing Patterns

• Benglish (Bengali-English): Common in Kolkata and Bengali-speaking regions; follows subject-object-verb structure of Bengali with English technical terms.
• Kannaglish (Kannada-English): Dominant in Bangalore tech hubs; heavily influenced by IT industry vernacular.
• Telugish (Telugu-English): Growing in Hyderabad and tech centers; follows Telugu agglutinative patterns with English service nouns.
• Marathish (Marathi-English): Prevalent in Mumbai's financial and trading communities.

Each regional pattern reflects the unique grammar, phonology, and cultural context of the underlying Indian language. For voice AI systems serving India, supporting multiple code-switching patterns isn't a luxury—it's essential for meaningful market penetration.

Why Traditional ASR Systems Fail at Code-Switching

The Monolingual Bottleneck

Most commercial ASR systems—even those claiming "multilingual" support—are fundamentally monolingual. They operate on a language-detection paradigm: identify which language is being spoken, then apply the appropriate language-specific acoustic and language models. This approach catastrophically fails for code-switching utterances.

Key Technical Failure Points

• Acoustic Mismatch: Single-language acoustic models are trained on speech patterns specific to that language. When a speaker code-switches, the acoustic characteristics change mid-utterance. A traditional Hindi ASR system encounters English phonemes for which it has no trained acoustic patterns; English systems encounter Hindi sounds. The result is transcription errors or complete system failure. For example, the English consonant cluster "str" (in "stream") doesn't naturally occur in Hindi phonotactics, so Hindi-only systems fail to recognize such clusters accurately.
• Language Detection Ambiguity: Forced-choice language detection breaks down when both languages are present in equal measure within a single utterance. Some systems delay language detection until after observing sufficient context, but by then, errors have accumulated. This creates a catch-22: commit early to a language choice and miss code-switching cues, or delay decision-making and accumulate acoustic recognition errors.
• Language Model Probability Collapse: N-gram language models trained on monolingual text assign near-zero probability to valid code-switched sequences. A Hindi language model has never seen certain mixed patterns ("laptop ka battery"), so it assigns probability close to zero, causing ASR systems to misrecognize or reject perfectly valid utterances. The system's decoder considers the utterance so improbable that it actively "corrects" what the acoustic model heard, forcing a monolingual interpretation.
• OOV (Out-of-Vocabulary) Explosion: Words from the secondary language are typically out-of-vocabulary in monolingual models. Even if the acoustic model successfully decodes the sound, the language model cannot score it, creating cascading errors. In worst cases, OOV word sequences consume most of the model's probability mass, preventing valid transcriptions.
• Transliteration Confusion: Indian language speakers often use Roman script transliteration for their native languages in text-based contexts. Code-switched utterances may contain transliterated words, adding another layer of ambiguity. A system trained only on Devanagari Hindi text cannot score Roman transliterations like "namaskar," even though they're phonetically identical and semantically clear to native speakers.

The combined effect: real-world code-switched utterances see word error rates 30-50% higher than their monolingual equivalents, even on modern commercial systems. For multilingual applications like voice assistants, this translates to dramatically reduced utility and user satisfaction in code-switching-heavy markets.

Technical Approaches: Building Code-Switching-Aware ASR

Language-Agnostic Acoustic Modeling

Modern approaches abandon strict language separation. Instead, they train acoustic models on multilingual speech data, allowing the model to learn language-independent acoustic representations. The key insight: phonemes across different languages sometimes share similar acoustic characteristics. A phoneme shared between Hindi and English can be learned from either language's data, enabling generalization.

Techniques include:

• Multilingual Phoneme Sets: Create a unified phoneme inventory spanning multiple languages, trained jointly on code-switched and monolingual speech.
• Language-Agnostic Features: Modify feature extraction to emphasize acoustic properties relevant across languages, de-emphasizing language-specific nuances.
• Domain-Specific Training: Collect and label code-switched speech from the target domain (e.g., smart home commands, automotive voice control), then fine-tune acoustic models on this data.

Multi-Encoder Architectures

Instead of forcing the system to identify one language, multi-encoder approaches maintain parallel processing streams for different languages. Each encoder specializes in a language, and a gating mechanism learns to weight contributions from each encoder based on acoustic context. When code-switching occurs, both encoders activate, and their outputs are weighted and fused.

This architecture aligns with how multilingual speakers process mixed-language input—their brains don't rigidly switch between language modes; instead, both languages remain partially activated.

Transliteration-Aware Tokenization

Since many Indian language speakers write their languages using Roman script, ASR systems must recognize and normalize transliterated text. A single word might appear in Devanagari or Roman script depending on context. Transliteration-aware tokenization normalizes these variations, reducing OOV issues.

Code-Switched Language Models

The most critical component is a language model trained on actual code-switched text. This requires:

1. Data Collection: Gather transcribed code-switched speech from real users in the target domain. Generic code-switched datasets often don't reflect the specific mixing patterns of commercial voice applications. Academic corpora (like SEAME for Southeast Asian multilingual speech) provide useful baselines, but production systems require domain-specific data. An automotive voice control system needs hundreds of hours of real driver commands mixing language pairs; a banking IVR system requires distinct financial transaction language mixing patterns. The cost of data collection is substantial, but essential for accuracy.
2. Probabilistic Weighting: Train language models that assign reasonable probability to code-switched sequences. Mixed-language n-grams should receive probability mass similar to monolingual n-grams, based on their frequency in the training data. This requires careful balancing: over-weighting code-switched patterns (while scarce in training data) introduces noise; under-weighting them causes the old monolingual collapse problem. Techniques like interpolation (combining monolingual and code-switched models) or explicit code-switching tags help manage this balance.
3. Domain Adaptation: Fine-tune general code-switched models on domain-specific data (e.g., automotive commands, banking IVR phrases) to capture the specific code-switching patterns of that application. A fintech app's code-switching patterns differ substantially from a smart home system's—financial terminology, transaction types, and user urgency levels all influence language choice. Domain adaptation brings WER improvements of 10-30% depending on domain specificity.

The Role of Context in Disambiguating Mixed-Language Intent

Contextual Language Modeling

Code-switching creates genuine ambiguity. The phrase "battery" in English sounds similar across languages, yet carries different connotations and grammatical implications in context. Resolving this requires understanding broader context and building models that maintain state across utterances:

• Conversation History: If the user has been speaking primarily in English, their next code-switched phrase might skew toward English interpretations. Models that remember recent language preferences perform better. Conversational context systems maintain running estimates of the user's language preference distribution and weight language model probabilities accordingly. This simple technique—called "language preference tracking"—improves WER on subsequent turns by 5-15% on average.
• Domain Context: In a banking app, certain phrases are more likely in one language than another. Language models fine-tuned on banking code-switched examples learn these domain-specific patterns. For instance, "account balance check kar de" (check account balance) follows predictable patterns—"account balance" stays English (technical term), while the action verb "check kar de" mixes languages. Domain models capture these patterns; generic models do not.
• Named Entity Context: Company names, product names, and proper nouns often remain in English even in code-switched utterances. Recognizing that "Airtel ka SIM" keeps "Airtel" and "SIM" in their native English/brand form helps the system assign correct pronunciation and segmentation. Named entity recognition modules specialized for code-switched speech identify these anchors and provide valuable context for downstream components.

Semantic Understanding Beyond Words

Advanced systems leverage semantic understanding to resolve code-switching ambiguities. If an utterance's translated meaning violates logical intent, the system can reparse using alternative language interpretations. This requires integration of ASR with NLU (Natural Language Understanding) components.

On-Device vs. Cloud Processing for Multilingual Voice

The Trade-offs

Building multilingual, code-switching-aware ASR creates a critical architectural decision: process on-device or in the cloud?

Cloud-Based Processing

Advantages: Larger, more complex models; access to real-time language model updates; continuous learning from user data; ability to leverage multiple specialized models.

Disadvantages: Network latency (critical for voice—users expect ~100ms response); privacy concerns (audio sent to external servers); cost scalability; dependence on connectivity.

On-Device Processing

Advantages: Sub-100ms latency (essential for natural voice interaction); privacy-by-design (no audio leaves the device); works offline; lower operational cost at scale; user data never leaves the device.

Disadvantages: Model size constraints (on-device models must fit in device memory); reduced model complexity; harder to update models; limited access to cloud-scale training data.

The Multilingual Consideration

Multilingual, code-switching-aware models are inherently larger than monolingual models. Supporting 5-10 languages or multiple code-switching patterns requires more parameters. For on-device deployment in resource-constrained contexts, this creates engineering challenges. Solutions include:

• Model Compression: Quantization, pruning, and knowledge distillation reduce model size while maintaining accuracy.
• Language Packaging: Users download only the language packs they need, reducing initial device footprint.
• Hybrid Approaches: Fast on-device ASR for common utterances, with cloud fallback for complex or ambiguous cases.

For Indian markets, where device diversity is high, on-device processing with intelligent cloud fallback represents the pragmatic choice.

Industry Applications: Where Code-Switching Matters Most

Automotive Voice Control

Indian automotive OEMs (Tata, Mahindra, Maruti, Hero MotoCorp) are rapidly integrating voice control as a safety and convenience feature. Drivers naturally code-switch during navigation and commands: "Mere ghar ke liye navigation on kar de" (Turn on navigation for my home) or "Temperature 22 degrees set kar, AC full blast mein de" (Set temperature to 22, AC on full blast). Accurate code-switching support dramatically improves user experience and safety—drivers can issue commands without taking eyes off the road or switching to a specific language mentally. Vehicles with poor code-switching support frustrate users and drive market share to competitors. For OEMs, code-switching support is increasingly a competitive differentiator in the Indian market.

Call Centers and Customer Service

India's massive BPO and call center industry—a 200+ billion-dollar sector handling customer service for global enterprises—relies heavily on voice interaction. Support agents naturally mix languages with customers, especially in tier-2 and tier-3 cities where customers have lower English proficiency. Code-switching handling improves call transcription quality, enables better sentiment analysis (understanding emotional undertones across mixed languages), and supports automatic quality assurance scoring. A call center agent saying "Aapka issue resolve ho gaya haan, kya aap satisfied hain?" (Your issue is resolved, are you satisfied?) requires code-switched sentiment analysis to capture both content and emotional intent. Call centers implementing code-switching-aware analytics report 15-25% improvements in quality scoring accuracy and agent performance insights.

Smart Home and IoT Devices

Smart speakers in Indian homes encounter constant code-switching. Users command lights, temperature, music, and information retrieval in their natural speech pattern, often within a single sentence: "Lights dim kar de, aur bedroom ka AC 20 degrees par set kar" (Dim the lights, and set the bedroom AC to 20 degrees). Devices that handle this naturally gain significant user satisfaction advantages and higher voice interaction adoption rates. Market research shows that smart speaker adoption in India is heavily constrained by language and voice recognition accuracy; code-switching support is a key growth lever.

Banking and Financial Services IVR

India's financial inclusion efforts rely heavily on voice-based banking, particularly for the 200+ million unbanked and underbanked users accessing mobile banking via voice. IVR (Interactive Voice Response) systems must robustly understand code-switched queries: "Mera account balance check kar de" (Check my account balance), "3000 rupees ka Airtel recharge karani hai" (I need an Airtel recharge for 3000 rupees), or "Last transaction mujhe samjha de, kab hua tha?" (Explain my last transaction to me, when was it?). Poor code-switching support forces users to repeat queries multiple times, creating frustration and reducing financial inclusion impact. Banks and fintech startups that prioritize code-switching ASR achieve significantly higher voice banking adoption and transaction success rates.

E-Commerce and Marketplace Voice Search

India's e-commerce boom—projected to reach $200+ billion by 2030—increasingly drives voice search adoption, particularly in smaller cities where users prefer voice search to typing. Code-switching support in voice search directly impacts search relevance and conversion. A user searching for "Black color ka formal shirt dijiye, size XL mein" (Show me black formal shirts in size XL) expects the system to understand mixing English adjectives ("Black formal") with Hindi quantity markers and size specifications. Marketplaces like Flipkart, Amazon India, and Meesho that implement code-switching-aware voice search report 20-40% increases in voice search engagement and improved conversion rates from voice searches.

Benchmarking Mixed-Language Accuracy: WER and Beyond

Word Error Rate (WER) Challenges

WER—the standard metric for ASR accuracy—becomes problematic for code-switched speech. If the system transcribes a phrase slightly incorrectly due to grammar variations, is this one error or many? Traditional WER creates confusion for code-switched metrics.

Better Metrics for Code-Switching

• Code-Switched WER: Variants of WER that weight errors by language boundaries, recognizing that errors within code-switched segments are more costly.
• Intent Accuracy: Measure whether the system extracted the correct user intent, regardless of minor transcription errors. This is more user-relevant than perfect transcription.
• Language-Specific WER: Report separate WER for each language component, showing system performance on Hindi words vs. English words within mixed utterances.
• Coverage and Naturalness: Measure what percentage of real-world code-switched utterances the system handles without requiring users to rephrase, and how natural the user experience feels.

How Mihup Handles Code-Switching with On-Device Multilingual AI

Building production-grade code-switching support requires deep expertise in Indian languages, speech processing, and on-device optimization. Mihup approaches this challenge through several integrated strategies:

• Multilingual Training Data: Thousands of hours of labeled code-switched speech across Hindi, Tamil, Telugu, Kannada, Marathi, Bengali, and English, collected from real users across diverse domains.
• Language-Agnostic Acoustic Models: Custom acoustic models trained on unified phoneme inventories spanning Indian languages, with fine-tuning on code-switched speech.
• On-Device Optimization: Quantized, pruned models that fit on-device while maintaining accuracy, enabling private, low-latency code-switched ASR without cloud dependency.
• Domain-Specific Models: Pre-trained models for automotive, smart home, banking, and call center domains, with further fine-tuning capability for specific applications.
• Contextual NLU: Intent and entity extraction systems that understand code-switched queries, resolving language ambiguities through semantic context.

For product teams building voice AI for India, exploring multilingual AI's growth potential reveals that code-switching support directly correlates with user adoption and retention in Indian markets.

Best Practices for Implementing Code-Switching Support

Data Collection and Annotation

Collect real-world code-switched utterances from your target domain and user base. Academic datasets may not reflect your specific use case. Ensure annotations include detailed language segmentation (marking which language each token belongs to) for training language-aware models. Best practices include: hiring native speakers from your target regions for accurate annotation; implementing inter-annotator agreement metrics to ensure consistency; and collecting data across diverse contexts (age groups, socioeconomic backgrounds, education levels) to capture the full diversity of code-switching patterns. Budget 2-3x more for annotation of code-switched data compared to monolingual data—the complexity is significantly higher.

Iterative Domain Adaptation

Don't assume general multilingual models work for your domain. Deploy initial models, collect real user utterances, identify error patterns, and iteratively fine-tune on domain-specific code-switched data. A production workflow typically includes: baseline evaluation on a held-out test set, A/B testing new models against current production systems, incremental model updates (weekly or bi-weekly), and continuous monitoring of real-world WER and user satisfaction metrics. Companies following this iterative approach see consistent 3-5% WER improvements per iteration for the first 4-6 iterations, with diminishing returns thereafter.

User Testing with Native Speakers

Test with native speakers of the code-switched language pair. They instinctively recognize unnatural or incorrect handling of code-switched speech. Native speaker feedback is invaluable for identifying edge cases and improvements. Formal usability testing with diverse age groups (teenagers, working professionals, elderly users) reveals different code-switching patterns and system pain points. Some users code-switch heavily; others prefer monolingual speech. Systems must gracefully handle both extremes.

Privacy-First Architecture

Prioritize on-device processing where possible. Code-switched speech often contains sensitive, personal, or financial information; keeping audio on-device is both a privacy benefit and a competitive advantage. Users are more comfortable with voice banking, health queries, and personal commands if they know audio never leaves their device. On-device processing also eliminates network latency, improving user experience for time-sensitive applications like automotive voice control.

Graceful Degradation

Build fallback mechanisms. If the system is uncertain about a code-switched segment, it's better to ask for clarification than to guess incorrectly. Clear error messaging helps users adapt their speech patterns to system capabilities. For example, if a system is uncertain about a particularly complex code-switched phrase, it might respond: "I understood 'schedule meeting,' but I'm not sure about the time. Could you say the time again?" This is far preferable to silently misrecognizing the time and scheduling the meeting incorrectly.

The Future of Code-Switching in Voice AI

As voice AI matures, code-switching support will transition from a differentiator to a baseline expectation. Emerging trends include:

• End-to-End Neural Models: Monolithic neural models that process raw audio to text without intermediate components, learning code-switching naturally without explicit language routing.
• Multilingual Foundation Models: Large pre-trained models covering many languages and code-switching patterns, with efficient fine-tuning for specific languages and domains.
• Real-Time Adaptation: Systems that detect and adapt to individual user code-switching patterns in real-time, personalizing language handling.
• Cross-Lingual Transfer: Learning from code-switching data in one language pair to improve handling of others through transfer learning.

Conclusion

Code-switching isn't a linguistic edge case or a fringe phenomenon—it's the dominant speech pattern for hundreds of millions of multilingual users globally, with particularly high prevalence in India. Voice AI systems that ignore code-switching fundamentally fail to meet user expectations in these markets.

Building production-grade code-switching support requires investment in multilingual training data, domain-specific model adaptation, contextual understanding, and on-device optimization. The technical challenges are substantial, but the payoff is enormous: voice products that feel natural, intuitive, and genuinely integrated into users' daily multilingual lives.

For product teams, engineers, and leaders building voice AI for multilingual markets, the message is clear: code-switching support isn't a future feature to bolt on—it's a foundational requirement for meaningful user adoption and market success.

Frequently Asked Questions

What's the difference between code-switching and code-mixing?

In linguistics, the terms are largely interchangeable, though some scholars distinguish between them. Code-switching typically refers to alternating between languages at structural boundaries, while code-mixing can include more fluid, intra-sentential blending. For voice AI purposes, both involve handling mixed-language utterances, so the distinction is academic rather than practical.

Why don't existing multilingual ASR systems (like Google Speech-to-Text) handle code-switching well?

Most commercial systems use language detection followed by language-specific processing. This paradigm breaks down for code-switching. Providers have improved multilingual support, but dedicated code-switching training on real user data remains limited because such data is challenging to collect and label at scale. Specialized providers with focused investment in code-switching patterns tend to achieve better results.

How much code-switched training data is needed to build a production system?

At minimum, several thousand hours of labeled, code-switched speech across your target domain and language pairs. Starting with 100-500 hours of domain-specific data and iteratively expanding based on real-world error patterns is a practical approach. Data quality (accurate transcription and language labeling) matters more than raw quantity.

Can on-device models really support multiple languages and code-switching with acceptable accuracy?

Yes, but with trade-offs. Modern quantization and compression techniques allow deployment of reasonably complex multilingual models on modern smartphones. Accuracy will be slightly lower than cloud models, but on-device latency and privacy benefits often outweigh the accuracy cost. Hybrid approaches (on-device for common cases, cloud for edge cases) provide good balance.

How do I measure code-switching ASR accuracy if standard WER doesn't work well?

Use a combination of metrics: code-switched WER (weighting errors by language boundaries), intent accuracy (did the system extract the correct user intent?), and qualitative feedback from native speakers. Segment results by language to identify which language pair is problematic. For production systems, track real-world metrics like user correction rates and explicit error reporting.