Your AI feature is live. Nobody owns the bill, the drift, or the bad week.
Shipping was the easy part. The hard part is knowing what the model costs per customer, when answers changed, and who gets paged when quality drops.
You shipped the AI feature. Now who is watching it?
Launch day feels like the finish line. It is actually the starting line for three problems that show up quietly, weeks later. The feature flag is on for everyone. Support is quiet for forty-eight hours. Then finance asks why the OpenAI invoice tripled. A power user finds a jailbreak that emails competitors' pricing. The model provider ships a silent update and your summarizer starts writing twice as much text overnight.
That is post-launch AI ops: the work almost nobody budgeted for in 2025, and the work every team with a live feature needs in 2026.
The three problems that arrive together
1. The bill grows on its own
Every user query costs money behind the scenes. Token usage scales with users, not with headcount. More traffic means a bigger invoice even if your team stays the same size.
A viral workflow or a badly cached RAG pipeline can turn a demo into a five-figure monthly line item without a single new customer. Without tracking, you find out when finance forwards you the bill.
2. The AI changes without telling you
The models you rely on get updated by their providers. Prompts rot. Your document corpus ages. Someone edits the wiki and nobody re-ingests.
Answers that were accurate last month can quietly get worse. Nobody notices until a customer complains. When they say "it was better last month," you need to answer what changed in one query, not a week-long investigation.
3. Something breaks and no one is watching
AI providers have outages and sudden policy changes, like any vendor. One rate limit, one safety filter update, and your flagship feature becomes a spinner.
If nobody owns "watching the AI," your team hears about problems from an angry user, not a dashboard alert. Engineering learns about it from the CEO's screenshot.
These are not edge cases. They are the steady state of production AI.

What actually protects a business after launch
Here is what teams that survive put in place the same week they turn the feature on:
Know what each customer costs you in AI spend, not just the total bill. Finance will not accept "AI is strategic" as a budget line forever. They want cost per active user per month, cost per successful task, top ten expensive tenants or workflows, and cache hit rate on embeddings and completions.
Instrument at the request level: model, tokens in and out, feature name, tenant ID, cache status. Then set budgets and brakes: soft cap warnings, hard cap degradations (shorter answers, cheaper model tier), admin overrides with logging. Without brakes, your best customer can accidentally burn your margin by uploading a thousand PDFs on a Friday.
Keep a record of every change to your prompts and models, so you can undo a bad one. Treat prompts and models like dependencies:
- Pin model IDs in config, not in someone's Notion page
- Changelog when you bump a model or edit a system prompt
- Re-run eval suites before you promote a change
- Track corpus version with each answer trace

Test the AI's answers regularly, not just before launch. Pre-launch benchmarks lie kindly. Production needs:
- Golden questions per feature, re-run nightly
- User thumbs tied to traces
- Regression alerts when abstention rate or average length moves
- Red team prompts on a schedule
A one-point drop in exact-match on refunds policy might be a legal issue, not a metric nit.
Assign a real person to own it, the same way you would own a server or a bank account. Healthy orgs assign explicit owners:
- Product: what the feature should do, priority of fixes
- Platform: keys, rate limits, failover, cost dashboards
- Domain experts: corpus quality, policy updates
- On-call: pager when error rate or latency breaches SLO
An AI feature with no owner does not crash. It just slowly gets worse while everyone assumes someone else is checking on it.
Retainer mindset vs project mindset
AI features are not "done" at handoff like a static landing page. They need:
- Weekly spend review until stable
- Monthly eval review
- Quarterly prompt and corpus audit
- Incident playbooks (provider down, key leaked, toxic output)
Teams that sold a fixed build without a run plan discover this in the first invoice cycle.
Playbooks that prevent panic
Provider outage: Fail over model or show a graceful degradation message. Never an infinite spinner.
Key leak: Rotate, invalidate sessions, audit usage spike.
Toxic or PII output: Block send, log trace, route to human review queue.
Cost spike: Identify tenant, enable stricter limits, notify account manager.
Run these as tabletop exercises before they happen live on a holiday.
What to instrument on day one (minimum)
- Request latency p95
- Error rate by step (retrieve, generate, tool)
- Tokens per request
- Abstention or "I don't know" rate
- Human override rate
- Model and prompt version on every trace
If you cannot filter a bad afternoon by version, you cannot fix it fast.
Unit economics: a worksheet finance will actually use
Build a simple model before launch and refresh it monthly:
- Average tokens per user session by feature
- Share of sessions that hit retrieval vs pure completion
- Cache hit rate on embeddings and repeated prompts
- Human review minutes per hundred AI tasks (often forgotten cost)
- Provider price per million tokens per model tier you use
Multiply by realistic peak daily active users, not demo traffic. Add 30% headroom for retries and abuse. If the margin story only works at zero logging and the cheapest model, you have a research project, not a product.
Teams in emerging markets should also model FX and payment friction on provider billing. A dollar invoice is not a dollar cost line for every entity.
When to downgrade models (on purpose)
Downgrade is a feature, not a failure:
- Tier 1: Short classification, routing, extraction. Small fast model.
- Tier 2: Customer-facing drafts with review. Mid model with temperature cap.
- Tier 3: Rare reasoning tasks. Expensive model with strict budget.
Route at the gateway. Log which tier served each request. Product marketing hates this table. Finance loves it.
Hosted inference and data residency (2026 buyer reality)
Enterprise and government buyers increasingly ask where inference runs. That affects:
- Whether customer content can leave a region
- Whether you need a dedicated agreement with the provider
- Whether on-prem or VPC-hosted options are on the roadmap
Even if you are not selling to government yet, design traces and configs so you can swap providers without rewriting product logic. Hard-coding one vendor's SDK through every screen is how migrations become rewrite projects.
Bottom line
Shipping AI is a product launch. Operating AI is a subscription to reality: prices change, models change, users adapt, attackers probe.
The teams winning in 2026 assigned owners, meters, and evals the same week they turned the feature on. Everyone else is learning why "we launched AI" and "we run AI" are different sentences.

