AI customer support that runs on your own LLM keys

Two years ago, “AI customer support” meant a clunky chatbot that recognised five intents and gave up. Today it means a live chat agent that reads your knowledge base, answers in your tone, escalates cleanly when it is unsure, and runs on a model you can swap any time. The technology stopped being the bottleneck. What replaced it is architecture, billing, and the rather boring question of whose AI bill you are paying.

This guide covers the whole stack. What the modern AI support agent actually does, what each layer costs, how to ship it without locking yourself into a per-resolution invoice that scales with usage, and the decisions that matter once it is live. If you are evaluating a platform or planning to build, treat this as the executive briefing.

What an AI customer support agent actually does

Pull back the cover on a current-generation AI support agent and you find five components glued together by orchestration code.

1. A chat widget on your site. One <script> tag, renders inside a Shadow DOM so it cannot collide with your CSS, loads lazily after first paint so it does not tank Lighthouse scores. The visitor never installs anything.
2. An LLM doing the inference. In 2026 the practical defaults are Google Gemini Flash for cost (effectively free at low volume), OpenAI gpt-4o-mini for balanced quality, Anthropic Claude Sonnet for the harder reasoning tasks, OpenRouter as a switching layer if you want one key that fronts several models, or a self-hosted Ollama model behind your VPN for regulated workloads.
3. A retrieval layer. Without grounding, the model confidently invents pricing tiers that do not exist. With grounding, every reply pulls a handful of chunks from your help center, policies, and product docs, stuffs them into the prompt, and cites the source inline. Done right, the visitor sees clickable references for every claim.
4. Tools and escalation. The agent does not just talk. It can look up an order id, file a refund ticket, fetch the visitor's journey, and most importantly, hand off to a human when it knows it is over its skis. The handoff carries the full conversation forward so the human starts with context.
5. An operator dashboard. Live inbox of every conversation, the ability to jump in mid-thread, routing rules so the right team sees the right message, alerts to Slack or Google Chat when no operator is at their desk, analytics on what the AI is getting right and where it is escalating.

That is the stack. None of it is novel anymore. What still varies across platforms is the billing model, which we are coming to next.

The cost math: AI customer support per ticket

Run a median customer support conversation through the stack today and you can put real numbers on every layer. Assume an average ticket of 8 messages, mixed visitor and agent, roughly 600 input tokens and 400 output tokens per AI reply, 4 AI replies per ticket.

The model inference per ticket lands roughly here. Gemini 2.5 Flash: about $0.001. OpenAI gpt-4o-mini: about $0.003. Anthropic Claude Sonnet: about $0.03. Self-hosted Llama 3.1 70B on a single H100: about $0.005 amortised at moderate traffic. Numbers move with model version and provider pricing, but the order of magnitude is stable.

Now compare to what hosted AI support platforms charge. Intercom Resolve runs about $0.99 per resolution. Zendesk Advanced AI prices out around $0.50 to $0.75 per resolved conversation depending on the plan. Tawk.to's AI Assist add-on uses a credit pack that works out to roughly $0.15 to $0.40 per AI message.

The gap is not a rounding error. The hosted-AI vendor is paying the same $0.001 to $0.03 to the same providers, then adding a margin of 15x to 300x. They can do this because the customer has no visibility into the underlying call. Bring-your-own-LLM architecture removes the opacity. You pay the provider list price, billed to your account, in your name, and the platform charges you a flat fee for the software itself.

Why your own LLM keys matter

Beyond cost, three things change when you bring the keys:

• Model choice is yours. When Google releases Gemini 3.0 and you want to test it, you flip a setting. You do not file a ticket asking the platform vendor to add support.
• Data trust is one relationship instead of two. Your customers' messages reach the model provider you already vetted (Google, OpenAI, Anthropic). The chat platform never sees the raw inference or your billing relationship with the provider. Fewer third parties, smaller surface area for compliance review.
• Cost ceiling is yours to set. You can put a hard monthly cap on your provider account. A hosted-AI platform billing per resolution can blow through a budget on a Reddit hug-of-death.

For a longer treatment of this argument with the full cost spreadsheet and switching playbook, see Bring your own LLM, explained.

How the agent stays grounded

A language model with no context will happily invent a return policy. The fix is retrieval-augmented generation (RAG). You feed your help center URLs, PDFs, or pasted text into the platform. It chunks the content, embeds each chunk into a vector database, and on every visitor turn it pulls the four or five most relevant chunks and stuffs them into the prompt as context.

The model is then told to ground its reply in those chunks and to cite them inline. The widget renders those citations as clickable links so the visitor can verify the source in one click. If the retrieval pass finds nothing relevant, a properly configured agent says so rather than guessing.

The piece operators care about most is that retrieval is automatic. You do not write intents, decision trees, or training examples. You drop your docs in, the platform indexes them, and the agent searches on every turn. When you update a doc, you reindex and the new content is searchable in seconds. For the deeper mechanics see How RAG works inside OpenAgent.

When the AI is wrong, what happens

Every honest evaluation of AI customer support has to answer this. Modern agents fail in three predictable ways:

1. The visitor asks something not in your docs. The agent should detect low retrieval confidence and offer to connect a human. A good platform makes this a one-line escalation policy.
2. The visitor is angry. Refunds, complaints, anything where empathy beats accuracy. The agent recognises emotional cues (or you set a simple rule on certain phrases) and routes to a human immediately.
3. The visitor is asking the model to do something operational (issue a refund, change a subscription, cancel an account). The agent should not be writing to your production systems. It should collect the intent, file an internal ticket, and tell the visitor a human will follow up.

The dashboard should make all three of these visible. You watch the first hundred conversations on your real traffic, you tune the thresholds and routing rules, and within a week the agent is handling the long tail of repetitive questions cleanly while the human team gets only the cases worth their time.

Shipping AI customer support in an afternoon

Concretely, this is what the first afternoon looks like on a bring-your-own-LLM platform:

1. Sign up and add your site as an allowlist host. The widget will only mount on domains you have registered, which stops competitors from embedding your script.
2. Paste one <script> tag on the site. The bubble appears bottom right within seconds.
3. Open the LLM tab and paste your provider API key. Default to Gemini Flash if you want the cheapest baseline, OpenAI gpt-4o-mini for balanced quality, or your own self-hosted endpoint URL if you have one.
4. Add your help center to the knowledge base. Paste a URL and the platform crawls and indexes it. Repeat for any PDFs of policies, pricing, or product docs.
5. Test in a private window. Ask the questions your real visitors ask. Read the citations. Tune the agent prompt and the retrieval grants if anything reads off.
6. Turn on Slack or Google Chat alerts so you get a ping the moment a visitor messages outside dashboard hours. Enable browser notifications on the dashboard itself for in-hours.
7. Go live and watch the first ten conversations.

From signup to first real visitor reply, the wall clock is usually under 90 minutes. The refinement loop after that runs for a couple of weeks as you watch the data.

What to look for when picking a platform

A short checklist that filters out the worst marketing. Demand answers to each:

• Can I bring my own LLM API key, or does the platform meter me? (If they meter, every conversation has a $0.50 to $1 markup baked in.)
• Can I swap models without contacting support? (Should be a dropdown, not an SLA.)
• Is retrieval automatic from URLs and uploads, or do I need to write intents? (Intents are 2018 architecture.)
• What does the human handoff look like? (One click to a real dashboard, or a 12-step approval flow.)
• What does the export look like if I leave? (Your messages, your contacts, your KB, in a standard format, one click.)
• Where is my data stored, in encrypted form, with what retention policy? (DPA on request, period.)

If a platform cannot answer those six confidently, you are looking at someone else's margin, not your platform.

Where OpenAgent fits

OpenAgent is a bring-your-own-LLM live chat and AI support platform. Flat fee per site: $5 per site per month billed monthly, or $36 per site per year billed annually ($3 per site per month). The agent works with Gemini, OpenAI, Claude, OpenRouter, and any OpenAI-compatible endpoint including self-hosted Ollama. Retrieval is automatic from URL crawls and pasted text. Slack and Google Chat webhooks and OS-level browser push are wired in. You export everything any time, no lock-in.

If you want to see the cost math for your own ticket volume, the AI vs human support ROI calculator does the breakdown in seconds. If you want to read the deeper architecture decision behind the bring-your-own-LLM stance, the BYO LLM guide is the next click.