Every company that put AI into production in the last three years started the same way: an API key, a metered bill, and a model someone else runs. That was the right call. Cloud AI APIs collapse months of infrastructure work into an afternoon, and no sane team builds a serving stack before proving the product works. But “right first choice” and “right permanent choice” are different claims — and for a specific class of workloads, the bill, the data boundary and the dependency eventually start arguing for infrastructure you own. The interesting question is not whether to move. It is which workloads, when, and how to move them without betting the company on a rack of hardware.
The three pressures
The first pressure is cost shape, not cost size. Metered pricing means the bill is a straight line through your usage: every customer you add, every feature that calls the model, every background job you automate moves you along it. Success makes it worse — the bill grows precisely because the product works. The absolute number matters less than the shape: a business whose unit economics contain a per-call fee paid to a third party has outsourced a slice of its margin, permanently, at a price it does not set. Owned inference inverts the shape — capital cost plus a comparatively flat operating cost — so beyond some utilisation, each additional call is nearly free. Where that crossover sits is an empirical question you answer with your own traffic, not a number from anyone’s blog post.
The second is the data boundary. Every prompt is an export: customer records, contract clauses, medical notes, code — whatever the workload touches leaves your infrastructure and transits a processor you audit through paperwork rather than access. Contracts, retention clauses and regional hosting soften this; for many workloads they soften it enough. But some data is radioactive enough — regulated, privileged, competitively sensitive — that the cleanest compliance story is the shortest one: the data never leaves. That argument gets stronger every time a workload moves deeper into the company’s core.
The third pressure is the quietest: someone else’s decisions. A metered API is a dependency whose roadmap you don’t control. Models get deprecated on the provider’s schedule, and the replacement — however capable — behaves differently against the prompts and evals you tuned for its predecessor. Rate limits, capacity and pricing change by announcement. None of this is malice; it is what being downstream means. A system that quietly became core infrastructure deserves the same scrutiny you would give any other single vendor sitting in the critical path.
The bill grows precisely because the product works. Cost shape, not cost size, is what forces the conversation.
The honest counter-case
Owning infrastructure is the wrong move more often than it is the right one, and the exceptions are worth stating plainly. If your volume is modest, metered pricing is cheaper than the ops salary you would add — the crossover argument cuts both ways. If your workload genuinely needs frontier-model capability, open-weight alternatives may not match it yet, and no hardware purchase fixes a quality gap. And if nobody on the team can own GPUs, drivers, serving software and capacity planning as a first-class operational responsibility, then buying hardware converts an invoice into an outage generator. The move makes sense at the intersection of three conditions: sustained volume, workloads that open-weight models handle at your required quality, and the operational maturity to run what you buy — or a partner who brings it.
What owning it actually involves
The work is concrete and mostly unglamorous. Sizing comes first: inference hardware is chosen against your real traffic — model size, context lengths, latency targets, peak concurrency — not against a spec sheet. Model selection is an evaluation exercise, not a leaderboard read: candidate open-weight models run against an eval set built from your actual workload, because the only benchmark that matters is yours. The serving stack — inference server, batching, quantisation, failover — is mature open tooling now, but it is a system to run, with the monitoring, alerting and upgrade discipline any production system demands. And quality telemetryjoins the usual dashboards: owned models don’t silently improve under you, but they don’t silently regress either — you own the pace, which means you also own noticing.
One discipline holds the whole thing together: the cloud baseline. Before any workload moves, its current behaviour is measured — quality on the eval set, latency, cost per unit of work. The owned stack earns the workload by matching or beating that baseline, demonstrably, before cutover. Without the baseline, migration is a leap of faith followed by an argument. With it, migration is a benchmark result followed by a decision.
Migrate like you deploy: one workload at a time
The pattern that works is the same one that works for every risky replacement: incremental, reversible, measured. Workloads move one at a time, lowest-risk first — internal tooling and batch processing before anything customer-facing. Each workload runs shadow traffic on the owned stack while the API remains authoritative; when the benchmarks hold through real load, the workload cuts over, with the API kept as fallback until confidence is earned rather than assumed. It is the strangler pattern applied to inference, and it means the company is never one bad day away from having bet everything on new hardware. This is the shape of an engagement we are running right now: custom inference hardware specified against measured workloads, migration proceeding workload by workload, every cutover gated on its cloud baseline.
A side effect of doing it this way deserves mention: the eval sets, baselines and routing you build for the migration permanently upgrade how you run AI. You end the project knowing what each workload requires, what it costs on any substrate, and how to move it again — which means the next provider price change or model deprecation is a spreadsheet exercise instead of a fire drill. Optionality, once built, keeps paying.
The end-state is usually hybrid
It is worth saying explicitly, because the debate is usually framed as a binary: almost nobody ends up fully off the cloud, and that is not a failure of the migration. The stable end-state for most companies is a routing decision per workload. High-volume, well-understood, data-sensitive workloads run on owned infrastructure, where the flat cost shape and the short compliance story do their work. Experimental features, spiky workloads and anything that genuinely needs a frontier model stay on metered APIs, where elasticity and capability are worth the per-call price. The infrastructure you own becomes the baseline; the API becomes what it always should have been — a premium option you reach for deliberately, not a default you drifted into. The routing layer that makes this possible is thin, but it changes the negotiating position entirely: every workload has a home it can move to, and every vendor knows it.
The decision, compressed
Stay on metered APIs while volume is small, while you need capabilities only frontier models provide, or while nobody can own the ops. Start measuring the moment the bill becomes a line item leadership asks about: build the eval set, capture the baselines, price the crossover with your own numbers. Move when a workload clears all three tests — sustained volume, achievable quality on open-weight models, an owner for the infrastructure — and move it the boring way, one benchmark-gated cutover at a time. The companies that get this right don’t treat owned AI infrastructure as a statement. They treat it the way they treat every other build-versus-rent decision: with a baseline, a benchmark and a reversible path.