Sarvam Models Guide

TL;DR

• Sarvam AI (Bengaluru, founded August 2023) ships open-weight text models plus separate speech and document products. This guide covers the text LLMs in the model registry: Sarvam-M, Sarvam 30B, and Sarvam 105B.

• Sarvam-M (23 May 2025) is a 24B Apache-2.0 post-train of Mistral-Small-3.1-24B-Base. API id sarvam-m. Thinking mode is reasoning_effort set to low, medium, or high on an OpenAI-compatible API at https://api.sarvam.ai/v1.

• Sarvam 30B and Sarvam 105B were open-sourced on 6 March 2026. Both are Mixture-of-Experts transformers trained from scratch in India on IndiaAI compute, under Apache-2.0. Sarvam states a June 2025 knowledge cutoff for both.

• 30B activates about 2.4B parameters (128 experts, top-6, grouped-query attention). It powers Samvaad, Sarvam’s conversational agent platform. 105B activates about 10.3B parameters (128 experts, top-8, multi-head latent attention, 128K context via YaRN). It powers Indus.

• Do not route speech or document jobs to these checkpoints. Saaras is ASR, Bulbul is speech synthesis, and Sarvam Vision is document intelligence. They are not aliases of 30B or 105B.

Quick Decision Guide

Who this guide is for

• Best for: AI engineers · platform engineers · teams deploying Indian-language assistants
• Difficulty: Intermediate
• Estimated time: 45 min

Learning Path

Large Language Models → Transformers → Sarvam Models → Mistral Models → DeepSeek Models → Cost Optimization

On this page

• Why This Matters
• The Problem Sarvam Models Solve
• How We Got Here
• Architecture
• Step-by-Step Flow
• Real Production Example
• Design Decisions
• Comparisons
• Common Mistakes
• Where It Breaks Down
• When NOT to Use Sarvam
• Running in Production
• Related Guides
• Interview Questions
• FAQs
• References

Why This Matters

Most open-weight shortlists in 2026 are still Llama, Mistral, Qwen, and DeepSeek. Those families are strong general models. They are not, by default, tokenizers and training mixes built around the 22 scheduled languages of India. Fertility — tokens per word — on Odia, Santali, or Manipuri can dominate latency and cost even when the English benchmark looks fine.

Sarvam’s text models exist for that gap. The 30B and 105B release post says the tokenizer targets all 22 scheduled languages across 12 scripts, and that a large share of pre-training tokens went to the 10 most-spoken Indian languages. The same post says both models were trained from scratch in India, including pre-training, supervised fine-tuning, and reinforcement learning, on compute from the IndiaAI mission.

For an engineering team, that changes the bakeoff, not the evaluation method. You still measure your own traffic. You add a candidate whose active compute (2.4B on 30B, 10.3B on 105B) and license (Apache-2.0) are explicit, and whose product mapping is also explicit: 30B on Samvaad, 105B on Indus.

Engineering Insight

Vendor benchmark tables are a starting point. Sarvam publishes MMLU, GPQA Diamond, LiveCodeBench, SWE-Bench Verified, and Tau2 numbers. Re-run the slice that matches your language mix and tool schema before you pin a model id.

The Problem Sarvam Models Solve

Indian-language production systems fail in three repeatable ways:

1. Tokenizer waste. A general multilingual tokenizer splits Indic words into long token sequences. The same user utterance costs more, fills the context window faster, and lowers tokens per second.

2. Translated-English behavior. A model that is strong on English MMLU can still mishandle numerals, names, and tool arguments inside Hindi or Tamil dialogue. Sarvam’s Samvaad write-up calls out numeral handling and tool calls during multilingual conversations as the reason 30B was tuned for that platform.

3. No from-scratch open checkpoint. Adapting a Western base model is faster, and Sarvam did that with Sarvam-M. It does not prove you can train the stack — data, kernels, routing, and inference — inside the country. The 30B and 105B release is that proof point, with weights on Hugging Face (sarvamai/sarvam-30b, sarvamai/sarvam-105b) and on AI Kosh.

The models do not solve speech or document layout. Those are other Sarvam products. Mixing them into the same client as sarvam-30b is a routing bug.

How We Got Here

Sarvam was founded in August 2023 by Vivek Raghavan and Pratyush Kumar. The company describes about $41 million raised, with Lightspeed, Peak XV Partners, and Khosla Ventures among the backers on its about page.

The first text model in this registry, Sarvam-M, shipped in May 2025. Hugging Face lists it as a finetune of mistralai/Mistral-Small-3.1-24B-Base-2503, Apache-2.0, text-only, with languages including English and ten Indian languages on the model card. Serving notes say vllm>=0.8.5 and an OpenAI-compatible endpoint. reasoning_effort turns thinking on. Any of low, medium, or high enables thinking; the card does not describe three different thinking budgets.

On 6 March 2026 Sarvam open-sourced 30B and 105B. The blog is the primary spec: both are reasoning MoEs, 128 experts, trained on 16T tokens (30B) and 12T tokens (105B). Pre-training had a long-horizon phase, a mid-training phase, and a long-context extension. Routing uses sigmoid scores plus an expert-bias term rather than a plain softmax gate. Reinforcement learning is an asynchronous GRPO setup. Sarvam says it omits KL regularization against a reference model and uses a group-relative objective inspired by CISPO.

Later 2026 posts are adjacent products, not new text checkpoints. Sarvam Epoch (7 August 2026) summarized inference, Indus agents, and Sarvam Code. Sarvam Vision 2.1 (24 September 2026) is document intelligence. Keep those out of the LLM router.

timeline
  title Sarvam text-model timeline
  2023-08 : Lab founded in Bengaluru
  2025-05 : Sarvam-M Apache-2.0 post-train of Mistral Small
  2026-03 : Sarvam 30B and 105B open-sourced
  2026-08 : Sarvam Epoch product summary

Architecture

Sarvam-M and the 2026 MoEs do not share a backbone. Treat them as two generations.

Sarvam-M. Dense 24B class, initialized from Mistral Small 3.1. You inherit that base’s architecture and then Sarvam’s post-training. There is no MoE routing to configure. Context length is whatever the Mistral Small 3.1 base and Sarvam’s serving config expose; confirm it on the model card before you set max_tokens.

Shared 30B / 105B design. Sparse expert feed-forward layers, 128 experts, one shared expert, rotary embeddings, and RMSNorm. A routed scaling factor of 2.5 appears on both Hugging Face architecture notes. Expert capacity differs, so “128 experts” does not mean the same active FLOPs.

Sarvam 30B. Grouped-query attention with 4 key-value heads. The model card describes 19 layers, a dense FFN intermediate size of 8192, MoE intermediate size of 1024, and top-6 routing. rope_theta is very high (8e6) for long-context stability without RoPE scaling. Active compute is about 2.4B non-embedding parameters. Sarvam positions it for throughput and for multilingual voice-agent tool use on Samvaad. The model itself is text; audio is outside the checkpoint.

Sarvam 105B. Multi-head latent attention. The card describes a head dimension of 576, query head dimension 192 split into RoPE and no-position parts, value head dimension 128, hidden size 4096, intermediate size 16384, MoE intermediate size 2048, and top-8 routing. YaRN scaling with factor 40 is how they state 128K context. Active parameters are about 10.3B. This is the Indus model.

flowchart TD
  req[Request] --> lang{Language and task}
  lang -->|Short Indic dialogue or tight GPU| m30[Sarvam 30B]
  lang -->|Hard reasoning coding or agents| m105[Sarvam 105B]
  lang -->|Legacy sarvam-m client| mm[Sarvam-M]
  m30 --> serve[vLLM or SGLang or API]
  m105 --> serve
  mm --> serve
  serve --> out[Text completion]

Published scores are Sarvam’s, not an independent DataAIHub eval. On the 105B card, examples include MMLU 90.6, MMLU Pro 81.7, GPQA Diamond 78.7, LiveCodeBench v6 71.7, SWE-Bench Verified 45.0, and Tau2 average 68.3, compared by Sarvam with models such as GLM-4.5-Air and GPT-OSS-120B. On the 30B card, examples include MMLU 85.1, HumanEval 92.1, GPQA Diamond 66.5, and LiveCodeBench v6 70.0. Use them to decide which public suite to re-run, not as a guarantee on your traffic.

Step-by-Step Flow

1. Write the workload down. Language mix, average input tokens, whether the model must call tools, and the latency budget. A Hindi voice agent and a Python repair agent should not share a default.

2. Drop speech and vision out of this choice. If the job is transcription, use Saaras. If it is document extraction, use Sarvam Vision. If it is text reasoning, continue.

3. Pick a generation. New work starts at 30B or 105B. Sarvam-M stays for clients that already send model="sarvam-m".

4. Pick a size inside the 2026 pair. Start with 30B when active compute and concurrency dominate. Move to 105B when 30B misses tool-use or hard reasoning cases you care about. Sarvam’s own split is Samvaad on 30B and Indus on 105B.

5. Choose host. OpenAI-compatible API (https://api.sarvam.ai/v1, key from the dashboard) or self-host. Self-host docs on the model cards cover Transformers, vLLM, and SGLang. Sarvam-M specifically calls for vLLM 0.8.5 or newer.

6. Pin ids and timeouts. Do not use a floating alias if you can pin sarvam-30b or sarvam-105b as documented on the Hub. Set a deadline that includes thinking tokens. 105B reasoning traces are longer than a short 30B chat reply.

7. Evaluate on your set. Include at least one Indic set and one tool-calling set if that is production. English-only MMLU will hide tokenizer and numeral failures.

8. Ship behind a router. Keep the other model id as a fallback. Log language, model id, and tool-call success. Revisit when Sarvam ships a newer text checkpoint; Vision 2.1 was not one.

sequenceDiagram
  participant App
  participant Router
  participant API as Sarvam API or vLLM
  App->>Router: messages plus language
  Router->>API: model id 30B or 105B
  API-->>Router: text or tool call
  Router-->>App: response and model id

Real Production Example

A state helpline already runs speech in and speech out. The language model only sees a transcript and must return either a spoken answer or a tool call (account lookup, ticket create). Sarvam’s Samvaad section describes this shape: 30B fine-tuned for conversational agents, with tool calls inside Hindi and Tamil calls.

A practical split:

• ASR stays on Saaras (or another ASR). Do not ask 30B to transcribe.
• Policy and tools go to Sarvam 30B because the active footprint is 2.4B and the platform Sarvam itself uses for this job is Samvaad.
• Escalation goes to Sarvam 105B only when the tool trace fails twice or the intent is a multi-step reasoning task (eligibility that spans several documents). 105B’s published Tau2 and BrowseComp numbers are why it is the escalation tier, not the first hop on every call.
• Speech out uses Bulbul or another TTS. The LLM returns text.

The same pattern maps to a text-only support bot: 30B for the common path, 105B when the conversation needs web search or a long tool chain, which is how Sarvam describes Indus.

Self-hosting that bot means one vLLM (or SGLang) process per checkpoint, not one process loaded with both if memory is tight. 105B is the memory risk. 30B is the concurrency play. Sarvam’s blog reports higher tokens per second per GPU for 30B versus a Qwen3 baseline on H100-class machines, and an MXFP4 path for Apple Silicon local trials. Those are vendor measurements. Confirm them on your GPU and your sequence lengths.

Design Decisions

API versus weights. The API is the fastest way to test Indus-like behavior. Weights are Apache-2.0, so a regulated or air-gapped deployment can leave the API. Budget MLOps for MoE serving; top-6 and top-8 routing are not a dense 7B load.

30B versus 105B. Decide with active parameters and your latency SLO, then confirm with a small labeled set. 30B is not “a small dense model.” It is a 30B-total MoE with a small active path. 105B is not a 105B dense model either. Comparing either to a dense 70B on parameter count alone will mis-size the GPU.

Sarvam-M versus the MoEs. Sarvam-M is the right artifact if you already depend on Mistral Small’s shape and only need Sarvam’s post-train. It is the wrong default for a new agent. The later models were trained from scratch, include agentic traces in SFT, and have a newer tokenizer story.

Thinking on or off. For Sarvam-M, thinking is a boolean in practice: any reasoning_effort enables it. For 30B and 105B, the training included reinforcement learning aimed at reasoning. You still cap completion tokens. An unbounded think trace will blow the latency SLO even when the answer is short.

Indic tokenizer versus translation pipeline. Translating to English, calling a general model, and translating back adds two failure points and hides numerals. Prefer the Sarvam checkpoint when the user language is Indic, and only fall back to translation if eval says the checkpoint loses.

Comparisons

Against other families:

• Mistral. Sarvam-M is downstream of Mistral Small. Mixtral and Mistral Large are different products with their own licenses and EU residency story. Do not assume a Mistral Regional Endpoint applies to Sarvam weights.
• DeepSeek. Also an open reasoning MoE story, with a much larger global install base. Choose DeepSeek when your eval and ecosystem (existing vLLM flags, community quants) already favor it. Choose Sarvam when Indic fertility, India-trained data, or the Apache-2.0 30B/105B pair wins the same eval.
• Llama. Larger ecosystem and more fine-tunes. License terms differ by Llama generation. Sarvam’s Apache-2.0 is simpler to read; Llama may still win on English-only coding agents. Measure both.

Sarvam’s comparison tables include GLM, Qwen3, Gemma, and GPT-OSS variants. Those tables are useful as a map of what Sarvam optimized for. They are not a substitute for your set.

Common Mistakes

• Sending audio bytes to sarvam-30b or sarvam-105b. Those checkpoints are text. Put ASR in front.
• Treating Sarvam Vision 2.1 as a 105B upgrade. It is a document model announced in September 2026.
• Assuming reasoning_effort values on Sarvam-M select different depths. The model card says each value enables thinking mode.
• Sizing GPUs off the “105B” name as if all experts were dense. Active compute is about 10.3B, but the full expert set still has to be stored unless you use an offload design you have actually tested.
• Copying Mistral Small context or chat-template settings onto 30B/105B. Different tokenizers and different attention (GQA versus MLA).
• Quoting Sarvam’s SWE-Bench or Tau2 numbers as if they used your harness. The 105B blog labels SWE-Bench Verified with the SWE-Agent harness. Your agent harness will not match that number automatically.
• Forgetting the June 2025 knowledge cutoff on 30B and 105B. The blog uses a February 2026 Codeforces contest as evidence the models had not memorized those problems. Live facts still need tools or retrieval.

Where It Breaks Down

English-only frontier agents. If the entire corpus and tool API are English and you already have a tuned Llama, DeepSeek, or closed flagship path, Sarvam’s Indic investment does not pay for itself. 105B’s published SWE-Bench Verified score (45.0) is behind several larger models in Sarvam’s own larger-model table.

Very long tool traces on 30B. 30B is the efficient tier. Sarvam’s 30B Tau2 average in the release post is lower than 105B’s. Escalating hard traces is the design, not a failure.

Multimodal inputs. No vision encoder on these three checkpoints. A screenshot workflow needs a different model or Sarvam Vision, then a text handoff.

Unreviewed quants. Apache-2.0 allows community quantizations. A 4-bit build that was not in Sarvam’s H100 or Apple Silicon notes can lose numeral accuracy in Indic text even when English perplexity looks unchanged. Test the quant on your language mix.

API terms versus weight license. Apache-2.0 covers the weights. The hosted API has its own terms and pricing. Read the dashboard before you promise customers a particular retention story.

When NOT to Use Sarvam

• You need native image or audio understanding in one model call. Use a multimodal model, or compose Saaras / Vision with a text model on purpose.
• Your acceptance set is English coding-agent benchmarks and a closed model already clears it inside the latency budget. Switching for the logo is not an architecture decision.
• You cannot run MoE serving and you cannot call the API. A dense small model you already operate is more honest than a 105B checkpoint that does not fit.
• You need a contractual EU-only region that another vendor already sells as a product. Sarvam’s public story is India-trained weights and an API, not a drop-in copy of Mistral Regional Endpoints.
• You are about to fine-tune Sarvam-M because a blog from 2025 is what the team remembers. Check whether 30B already covers the task. Fine-tuning the older dense model can be the expensive path.

Running in Production

API. Point an OpenAI SDK at https://api.sarvam.ai/v1 with a dashboard key. Pin the model id. For Sarvam-M, set reasoning_effort only when you want thinking, and cap max_completion_tokens. Log the id on every response so a later router change is visible.

Self-host. Follow the Hugging Face card for the checkpoint you picked. The 30B and 105B cards list Transformers, vLLM, and SGLang. Load one revision you have hashed. Do not track main in production.

Capacity. 30B is the default for concurrent conversations. 105B is a smaller pool for hard requests. Autoscale them separately. A single queue that always prefers 105B will idle the efficient pool and miss the SLO the efficient pool was for.

Tokenizer. Use the tokenizer shipped with the checkpoint. Swapping in a Mistral or Llama tokenizer changes both cost and quality, especially on Indic text. Sarvam’s fertility charts are the reason.

Observability. Track input tokens, output tokens, think tokens if the API exposes them, tool-call error rate, and language. Alert on a sudden mix shift toward 105B; that usually means the router’s confidence rule is wrong, not that users became harder overnight.

Safety. The 30B/105B post describes an India-specific safety fine-tune and red-team prompts. That is not a waiver. Keep your own policy filter for your domain, especially if the assistant gives financial, medical, or government-process answers.

Change control. Subscribe to the Sarvam blog RSS. A Vision or ASR launch can sit next to a text-model post. Read the title before you roll a new checkpoint into the LLM service.

Related Guides

• Large Language Models
• Mistral Models — Sarvam-M’s base family
• DeepSeek Models — the other open reasoning MoE comparison
• Llama Models
• Cost Optimization
• Context Windows

Company and model pages: Sarvam AI, Sarvam 105B, Sarvam 30B, Sarvam-M.

Interview Questions

What is the difference between Sarvam-M and Sarvam 105B? Sarvam-M is a 24B post-train of Mistral Small from May 2025. Sarvam 105B is a from-scratch MoE from March 2026 with about 10.3B active parameters and a 128K context window. They do not share serving settings.

Why can a 30B model be cheaper to serve than its name suggests? Only about 2.4B non-embedding parameters are active per token. You still store the expert pool. The win is compute and memory traffic per token, not a 2.4B download.

When would you refuse to put Sarvam 30B behind a voice API directly? When the client is sending audio. 30B consumes text. ASR and TTS are separate models. The LLM should see a transcript and return text or a tool call.

What does Apache-2.0 change operationally? You can self-host and modify the weights without a vendor runtime license. You still have to comply with the hosted API contract if you use the API, and you still need to eval safety for your domain.

FAQs

Which model id should a new app call? Start with the 30B or 105B id documented on the Hugging Face cards and the Sarvam dashboard. Use sarvam-m only to keep an existing client working.

Are the weights actually downloadable? Yes. Sarvam’s March 2026 post points at Hugging Face and AI Kosh for both 30B and 105B. Sarvam-M is sarvamai/sarvam-m.

Does 105B see images? No. Sarvam Vision is a different line. The 105B model card is text generation.

What is the knowledge cutoff? Sarvam’s 30B/105B post says June 2025. Use tools or retrieval for anything newer.

References

• Sarvam, “Open-Sourcing Sarvam 30B and 105B,” 6 March 2026, https://www.sarvam.ai/blogs/sarvam-30b-105b
• Hugging Face model card, sarvamai/sarvam-105b, https://huggingface.co/sarvamai/sarvam-105b
• Hugging Face model card, sarvamai/sarvam-30b, https://huggingface.co/sarvamai/sarvam-30b
• Hugging Face model card, sarvamai/sarvam-m, https://huggingface.co/sarvamai/sarvam-m
• Sarvam, “Sarvam-M,” https://www.sarvam.ai/blogs/sarvam-m
• Sarvam about page, https://www.sarvam.ai/about-us
• Sarvam blog RSS, https://www.sarvam.ai/rss.xml

Source: Sarvam AI blog and Hugging Face model cards, reviewed 25 September 2026.