The Bill After the Build: What an AI System Really Costs to Run

Most AI budgets stop at launch. The proposal covers discovery, build, integration, and a go-live date. Then the system runs for two years, and nobody wrote down what those two years cost. Here is what actually lands on the invoice.

The model bill is the part you can compute

Model fees are metered per token — roughly per word in and per word out. That makes them predictable if you do the arithmetic once.

Take an internal support assistant handling 8,000 conversations a month, six turns each. That is 48,000 calls to the model. Each call carries your instructions, the retrieved documents, and the conversation so far — call it 5,000 tokens in — and returns about 400 tokens out. You are buying 240 million input tokens and 19 million output tokens a month. At mid-tier model pricing, that lands near $1,000 a month, and prompt caching on the repeated instructions can cut it meaningfully.

Two things surprise people. First, input dominates: you pay for everything you send, every turn, and long context is the main cost lever you control. Second, retries, evaluation runs, and internal testing add 15–30% on top of production traffic. Budget the gross number, not the clean one.

Model prices per token have fallen consistently. Total spend usually rises anyway, because usage grows and newer models get used for harder work. Plan for volume, not for rates.

The bills nobody put in the proposal

Model fees are rarely the largest line. The rest of the running cost is ordinary operational work that AI systems happen to need more of:

• Human review. Someone reads a sample of real outputs every week and decides whether quality holds. For a customer-facing system, assume a few hours a week, permanently.
• Infrastructure around the model. Search indexes, vector stores, logging, and the pipeline that keeps your documents current. Modest individually, constant in aggregate.
• Change work. New products, new policies, new edge cases the system handles badly. Expect a small, steady stream of adjustments rather than occasional big releases.
• Vendor changes. Models get deprecated on published timelines. Migrating to a successor means re-testing everything the old one did well, typically once a year.
• Compliance and records. Retention, access control, and the audit trail that proves what the system told a customer in March.

Drift is the line item that surprises people

Software fails loudly. AI systems degrade quietly, and they stay confident the whole way down.

Your business drifts first. Prices change, a product retires, a policy gets rewritten. The system keeps answering from the version it was given, fluently and wrongly, until someone notices.

Your users drift next. Once people learn what the system handles, they push into adjacent territory it was never scoped for. Traffic that looked like FAQ answering becomes contract interpretation, and the accuracy you measured at launch no longer describes what is happening.

Your content drifts too. Retrieval that worked over 400 documents gets vaguer over 4,000, because more near-matches crowd the right answer out of the results.

None of this triggers an alert. It shows up months later as rising escalations, falling usage, or a complaint from a customer who acted on something the system said. By then the fix costs more than the monitoring would have.

The defence is unglamorous and cheap: sample 50 to 100 real interactions a month, score them against what a good employee would have said, and track the score over time. One person, half a day, every month. That single habit catches nearly every failure mode above while it is still small.

A budget that survives contact with reality

Plan annual running cost at 30–50% of build cost for the first two years. Below 20%, the system is being neglected and the quality curve will show it. Above 60%, either the build left too much unfinished or the use case is genuinely more volatile than it looked.

Split that number into three buckets: usage (model fees and infrastructure, scaling with volume), custody (review, monitoring, records — roughly fixed), and change (adapting to your business and your vendor's roadmap). The buckets behave differently, so a single blended figure will mislead you the first time volume doubles.

Then name an owner. An AI system without a named owner and a standing budget line does not get maintained; it gets tolerated until it is quietly abandoned. That is the real cost of skipping this exercise — not overspending, but paying for a build twice.

Work the run-rate out before the build, not after. The number often changes what you decide to build, which is exactly what it is for. It is the first calculation we do with a client, and occasionally it is the reason we recommend a smaller system than the one they came in asking for.