Bring your own LLM, explained: cost math, vendor lock-in, and why hosted chat platforms mark up AI 8-15x

What “bring your own LLM” actually means

Most customer-support platforms ship with an AI option these days. Look a layer down and almost all of them resell a model the platform vendor rented from OpenAI or Anthropic, then they bill you per “resolution”, per “AI message”, or per seat, whichever metric makes the invoice look smallest. They get a margin. You get a black box.

Bring your own LLM (BYO LLM) is a different architecture. The chat platform never sees your business logic running through a model it owns. You paste an API key for the provider you already use (Google Gemini, OpenAI, Anthropic, OpenRouter, an Ollama endpoint behind your VPN), the platform stores the key encrypted, and every AI inference call goes directly from the platform code to the provider you chose, billed to your account, in your name, at the provider's public list price.

Three things change when you flip that switch:

• The AI bill stops being someone else's margin. You pay Google or OpenAI directly. Tokens are tokens. There is no per-resolution or per-message markup hidden inside the chat subscription.
• You can switch models without changing platforms. Gemini 2.5 Pro got cheaper this quarter? Switch. Claude shipped a better long-context model for your refunds policy? Switch. The chat platform is just orchestration; it should not lock you in.
• You keep the relationship with the model vendor. Your prompts, your logs, your enterprise agreement, your data residency. The chat platform never even sees the raw model traffic.

The cost math on a real customer-support ticket

Let's grade two architectures on one realistic ticket. The visitor types “Hi, what's your refund policy if the item arrived damaged?” The AI does a RAG lookup against your knowledge base, retrieves three chunks (about 1,800 input tokens), and produces a 4-sentence answer (about 220 output tokens). One ticket. Real numbers.

On Gemini 2.5 Flash at Google's public price (around $0.30 per million input tokens and $2.50 per million output tokens in 2026), one ticket costs roughly:

Input :  1,800 tokens × $0.30/MTok  ≈ $0.00054
Output:    220 tokens × $2.50/MTok  ≈ $0.00055
Embedding: 90 tokens   × $0.15/MTok ≈ $0.000014
Total per ticket                    ≈ $0.0011

Round it up to ~$0.0011 per ticket on Gemini Flash. Switch to GPT-4o-mini, you're in the same order of magnitude. Even Claude Sonnet 4.x lands under a cent for this shape of ticket.

Now look at the hosted-AI billing. Intercom's “Fin” agent prices around $0.99 per resolution at their published rate. Tawk.to's AI Assist add-on bills per response after a quota. Crisp's AI tier is a per-seat add-on with usage caps.

Same one ticket, two architectures:

Even if you assume Intercom internally pays $0.05 per ticket to a frontier model, the markup is still ~20x. They run a margin business on AI tokens. There's nothing villainous about that, but you deserve to know what the line item is paying for.

At 10,000 monthly tickets handled by AI, the gap is striking:

• BYO LLM on Gemini Flash: ~$11/month on tokens.
• Hosted Fin: ~$9,900/month in resolution charges.

That is the entire commercial argument for BYO LLM, before we even talk about lock-in, residency, or which model you actually prefer.

The lock-in problem nobody talks about

When the AI is bundled into the chat platform, switching providers is not a checkbox. The platform's prompts, tool definitions, RAG retrieval, and post-processing are all tuned around the one model it ships with. Even if a competitor model is cheaper or smarter for your domain, you're stuck on the rails the platform laid.

BYO LLM platforms (OpenAgent included) expose the provider behind a simple interface: name, model id, API key, base URL for self-hosted endpoints. The orchestration layer is the same regardless. That means running Gemini for the cheap bulk tickets and Claude for the long thread your tier-2 humans are escalating to is just a routing rule. It's not a migration project.

Who actually wins on each provider

Security and residency: where the keys live

BYO LLM is only an honest pitch if the keys you paste are actually protected. The threat model has three layers:

1. At rest: Keys land in the database in ciphertext, encrypted with a per-tenant key. The application server holds the unwrap key in memory, never on disk in plaintext.
2. In transit: Outbound calls from the platform to your chosen provider go over TLS. The platform never proxies them through a shared egress that could log raw bodies.
3. In logs: No request to a model provider lands in a searchable log with its Authorization header intact. The provider itself logs whatever its enterprise agreement specifies. That's your contract with them, not the chat platform.

You should be able to revoke a key from your provider dashboard at any time and have the chat platform gracefully fall back to live-chat-only, not hang. That's table stakes for “your keys, your data” to mean anything.

When BYO LLM is the wrong choice

Honest take: BYO LLM costs you something in convenience. Three cases where the hosted approach genuinely wins:

• You handle five tickets a week. If you're running a side project and total AI usage is two cents a month, the operational overhead of managing a provider account is real and the math is moot.
• You need a single bill for procurement reasons. Some enterprise purchasing teams will sign one $20K invoice but refuse to handle a separate $20 invoice from Google. Talk to procurement before you architect.
• You don't want to think about model choice ever. Some teams want one button labeled “AI on” with no options. Hosted is fine for that.

For everyone else (teams scaling past 1,000 tickets a month, teams with a data-residency requirement, teams who already pay an LLM vendor for something else), BYO LLM is the architecturally honest choice. The chat platform should be software, not a token reseller.

How to switch a provider in OpenAgent

The mechanics are deliberately boring. Three steps from the dashboard:

1. Settings → Providers. Add a new provider row: pick Gemini, OpenAI, Anthropic, OpenRouter or “OpenAI-compatible endpoint”. Paste the key. The platform encrypts on submit.
2. Agents → Edit. Open the agent that should run on the new model. Change the “Model” dropdown. Save.
3. Smoke test. Open the embedded widget, ask the agent anything. The agent header shows the active provider; the audit log records the switch with a timestamp.

You can repeat per agent, so “Sales” runs on Claude (tone) and “Support” runs on Gemini Flash (cost). One workspace, two models, two separate provider bills, zero chat-platform AI charges.

Bottom line

BYO LLM isn't a feature. It's a stance on who profits from your AI usage. If you're comfortable paying a chat platform an 8-15x markup on tokens for the convenience of one invoice, hosted AI is fine. If the math has stopped working, or if you have a model vendor relationship you'd rather grow than abandon, the platform fee belongs on the chat product and the model fee belongs with the model vendor. That's OpenAgent's entire premise.