Self-Hosted LLM vs API: Cost, Privacy & Performance for Indian Enterprises

By Rajesh Kannan, Co-Founder & CTO, Gramosoft · Last updated: November 10, 2026 · Prices and rules checked on November 9–10, 2026

Every Indian enterprise building with AI reaches the same fork: keep paying per token to a hosted API, or run an open model on GPUs you control. Vendors on each side will tell you their route is cheaper and safer. The honest answer depends on your volume, your data and who will keep the servers running.

This guide starts with how Gramosoft sets up both routes for clients, then works through cost in rupees with GST, a break-even calculator you can run with your own prices, what the DPDP Act means for prompts that leave India, GPU sizing and speed, and the hybrid setup most teams end up with.

Swipe the table sideways on mobile →

How Gramosoft runs LLMs for Indian enterprises

Gramosoft sets up LLM systems three ways: open models on the client's own servers or an Indian GPU cloud, hosted APIs where they are cheaper or smarter, and a hybrid gateway that decides per request. We start from the data and the monthly token volume, not from a preferred vendor, and give the running cost in rupees.

• Self-hosted setups: open models served on the client's GPUs or an Indian GPU cloud, with access control, logging and a private endpoint your applications call. Our Ollama setup guide shows the simplest version.
• Products built for both routes: GdoczAI's document pipeline is built on open OCR models such as MinerU, DeepSeek OCR and Chandra OCR that run on your own GPUs, while its cloud version uses hosted APIs. GsearchAI is open source and runs on Neo4j and pgvector inside your network. GcrawlAI has an MIT-licensed core you can self-host.
• Around the model: AI development, cloud setup, data protection and cloud security reviews, so the private model is actually private.

Where we fall short: we don't sell GPUs or run a data centre, and we don't train foundation models from scratch. If your volume is low and your data can leave India, we will tell you to use an API and skip the servers.

"Clients often ask us to self-host to save money. At most volumes it doesn't. The reason to self-host is control over where the data goes, and once that is the reason, the cost question gets much easier to answer honestly."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

What each route costs in India (November 2026)

An API bill grows with every token and drops to almost nothing when idle. A self-hosted model costs the same every hour the GPU is rented, busy or not, plus the time of whoever runs it. Below are published list prices converted to rupees, with 18% GST added to both sides for a fair comparison.

Swipe the table sideways on mobile →

*75% input and 25% output tokens, at the RBI reference rate of ₹93.8579 per dollar on November 9, 2026. Prices from the Claude API pricing page, read November 10, 2026; the Haiku 5.5 rate shown applies to prompts up to 100,000 tokens. Batch processing halves these figures and cached input costs less. Other providers price differently.

Swipe the table sideways on mobile →

Hourly rates from E2E Networks' pricing page, generated November 4, 2026 and read November 10; monthly plans are quoted per deal. We use one Indian provider as a worked example, not a recommendation.

• Costs the API price hides: retries, long system prompts sent with every call, and usage that grows once staff like the tool.
• Costs the GPU price hides: engineer time for updates, monitoring and on-call, a second GPU for failover, storage, and idle hours at night.
• Quality isn't free either: an open model that needs more retries or human review than the API it replaces costs more than its GPU bill.

Break-even: when self-hosting pays for itself

The break-even point is the monthly token volume at which the API bill equals the fixed cost of the server. Divide the server's monthly cost, including GST and engineer time, by the API's blended price per million tokens. Below that volume the API is cheaper; above it, the server is, if it can keep up.

A cost comparison needs arithmetic you can rerun with your own quotes, so here is the small script behind the chart. It uses only Python's standard library. Change the exchange rate, prices and engineer cost to yours.

break_even.py

"""Self-hosted LLM vs API: monthly cost and break-even volume in INR.

List-price arithmetic, not a benchmark. Replace every input with your own
quotes, exchange rate and measured token volumes before deciding anything.
"""
from dataclasses import dataclass

HOURS_PER_MONTH = 730          # always-on server
GST = 0.18                     # applied to both sides; registered firms may claim input credit


@dataclass
class Api:
    name: str
    usd_in: float              # USD per million input tokens
    usd_out: float             # USD per million output tokens

    def inr_per_mtok(self, usd_inr, in_share=0.75, batch=False):
        blended = in_share * self.usd_in + (1 - in_share) * self.usd_out
        if batch:
            blended *= 0.5     # batch processing discount
        return blended * usd_inr * (1 + GST)


@dataclass
class Gpu:
    name: str
    inr_per_hour: float        # per GPU, before GST
    count: int = 1

    def monthly_inr(self, ops_inr):
        return self.inr_per_hour * self.count * HOURS_PER_MONTH * (1 + GST) + ops_inr


def break_even_mtok(gpu, api, usd_inr, ops_inr, **kw):
    """Million tokens per month at which the API bill equals the server bill."""
    return gpu.monthly_inr(ops_inr) / api.inr_per_mtok(usd_inr, **kw)


if __name__ == "__main__":
    USD_INR = 93.8579          # RBI reference rate, 9 Nov 2026
    OPS_INR = 50_000           # your engineer time per month: an example input, not a quote

    apis = [Api("Claude Sonnet 5.5", 2.00, 10.00), Api("Claude Haiku 5.5", 0.10, 0.50)]
    gpus = [Gpu("1x L4 24GB", 49.00), Gpu("1x H100 80GB", 379.05), Gpu("2x H100 80GB", 379.05, 2)]

    for g in gpus:
        print(f"{g.name:<14} server: Rs {g.monthly_inr(OPS_INR):>10,.0f} a month incl. GST and ops")
    print()
    for a in apis:
        print(f"{a.name}: Rs {a.inr_per_mtok(USD_INR):,.2f} per million tokens (75% input, incl. GST)")
        for g in gpus:
            mt = break_even_mtok(g, a, USD_INR, OPS_INR)
            print(f"  vs {g.name:<14} break-even {mt:>9,.0f} M tokens/month (~{mt / 30:,.0f} M a day)")
        print()

Output (run on November 10, 2026)

1x L4 24GB     server: Rs     92,209 a month incl. GST and ops
1x H100 80GB   server: Rs    376,514 a month incl. GST and ops
2x H100 80GB   server: Rs    703,027 a month incl. GST and ops

Claude Sonnet 5.5: Rs 443.01 per million tokens (75% input, incl. GST)
  vs 1x L4 24GB     break-even       208 M tokens/month (~7 M a day)
  vs 1x H100 80GB   break-even       850 M tokens/month (~28 M a day)
  vs 2x H100 80GB   break-even     1,587 M tokens/month (~53 M a day)

Claude Haiku 5.5: Rs 22.15 per million tokens (75% input, incl. GST)
  vs 1x L4 24GB     break-even     4,163 M tokens/month (~139 M a day)
  vs 1x H100 80GB   break-even    16,998 M tokens/month (~567 M a day)
  vs 2x H100 80GB   break-even    31,739 M tokens/month (~1,058 M a day)

• Against a Sonnet-class API: one H100 breaks even near 850 million tokens a month, about 28 million a day. A small model on an L4 breaks even near 208 million.
• Against a Haiku-class API: the break-even is in the billions of tokens a month. On cost alone, cheap small-model APIs are very hard to beat.
• The catch the script can't check: whether one GPU can actually serve your break-even volume at the speed users expect, and whether the open model is good enough. Measure both on your own prompts before you sign a GPU contract.

Privacy and DPDP: what changes when prompts leave India

With a hosted API, every prompt and any personal data in it leaves your network for the provider's servers. India's DPDP Act allows that by default, but sector rules and client contracts may not. A self-hosted model keeps the prompt in India and under your control, which is often the real reason to choose it.

• DPDP section 16: personal data may be processed outside India unless the government notifies a country as restricted. No country had been notified as of Mondaq's March 2026 analysis; check again before you decide.
• Sector rules still apply: section 16 doesn't override stricter laws, such as the RBI requirement to store payment data in India. Banks, insurers, hospitals and government work often carry their own conditions.
• Deadlines: the DPDP Rules were notified on November 13, 2025, and core obligations apply from May 2027, per AZB & Partners.
• Provider options: the Claude pricing page lists a US-only inference option at 1.1 times the price, with global routing as the default. Read your provider's data-retention and training terms, not just its prices.
• Self-hosting isn't compliance: you still need consent, access control, encrypted storage, prompt logs with retention limits, patching and a breach process.

This is general information, not legal advice. Have your counsel confirm what applies to your sector.

Performance: GPU memory, speed and answer quality

Self-hosted performance starts with memory. A model's weights need roughly its parameter count times the bytes per parameter: two at 16-bit, one at 8-bit, half at 4-bit. Add about 20% for the KV cache and runtime, more for long contexts. That number picks your GPU, and your GPU sets your monthly bill.

• Speed: a local model skips the internet round trip, which helps short, frequent calls such as autocomplete or classification. A busy shared GPU can still be slower than an API, so test under realistic load.
• Throughput: serving software that batches requests, such as vLLM, gets far more tokens per hour from the same GPU than a single-user setup. Our guide to running models with Ollama covers the simpler end.
• Quality: frontier API models usually lead on long, multi-step reasoning. Open models close much of the gap on narrow tasks such as extraction, classification and search answers, especially with retrieval. Our list of the best local LLMs compares current options.
• Quantisation: 4-bit models fit smaller GPUs but can lose accuracy on some tasks. Compare against the 16-bit version on your own test set.

The hybrid setup most enterprises end up with

Most Indian enterprises don't pick one route. They put a gateway in front of both: requests carrying personal or regulated data, and high-volume routine tasks, go to a local open model; hard reasoning with no sensitive data goes to a hosted API. One audit log covers both, and switching a task is a config change.

• Classify before you route: tag requests by data type (personal, financial, health, public) and task type at the gateway, not inside each app.
• Redact on the way out: strip names, phone numbers and account details before anything goes to an API, even when routing allows it.
• Keep one interface: expose both routes through the same API shape so apps don't care which model answered. This also makes it easy to move a task when prices change.
• Log both routes: who asked, which model answered, token count and cost. That log is your break-even data for next year.

Self-hosted or API: a decision checklist

Four questions settle most cases. Ask whether data must stay in India or inside your network, whether one task runs at hundreds of millions of tokens a month, whether an open model passes your own quality test, and whether someone can run GPUs. Mostly yes means self-host; mostly no means API.

1. Data: list what will appear in prompts and which law, regulator or client contract governs it.
2. Volume: measure tokens per month per task from real usage, not estimates. Run the break-even script with that number.
3. Quality: build a test set of 100 to 200 real prompts with expected answers. Score the API and two open models on it.
4. People: name who patches, monitors and is on call for the GPU server. If nobody, start with an API or a managed setup.
5. Exit: keep prompts and evaluation sets portable so you can move between routes as prices change.

Frequently asked questions

Related guides

• Best Local LLMs in 2026: Open Models You Can Run on Your Own Servers
• How to Run LLMs Locally with Ollama: A Setup Guide for Businesses
• ChatGPT vs Claude vs Gemini for Business (2026): Which AI Should Your Company Use?
• 7 Free LLM APIs in 2026: Providers, Limits & What You Can Build
• RAG vs Agentic RAG vs GraphRAG: Which One Fits Your Business Data?
• Best Embedding Models for RAG (2026): Qwen3, Voyage 4, BGE-M3, OpenAI and More
• Best AI Agent Frameworks in 2026: LangGraph vs CrewAI vs OpenAI Agents SDK vs Google ADK
• What Is MCP (Model Context Protocol)? A Plain-English Guide