AI / ML topic Should You Rent AI or Run Your Own?

Every team using AI ends up asking this. You can pay per token to use someone else’s model through an API. Or you can download an open-weights model and run it yourself, on GPUs you rent or buy.

Either way, a question goes in and an answer comes out. Which is cheaper, faster or safer?

The answer is “it depends”, and this post is about what it depends on. Prices are as of September 2026 and they move fast. Treat the numbers as a snapshot and the reasoning as the point.

The cost: a meter or a lease

An API is a meter. Anthropic charges $2 per million input tokens and $10 per million output for Claude Sonnet 5. Hosted open models are much cheaper: Together AI serves Meta’s Llama 3.3 70B for $1.04 per million, in or out. You pay for exactly what you use, and nothing when you don’t.

Self-hosting is a lease. Lambda rents H100s from $4.19 an hour each in a two-GPU machine, and a 70-billion-weight model at 16 bits needs two of them just to hold its 140 GB of weights. Leave them on all month and that’s about $6,100, whether they answer ten questions or ten million. So what decides the cost per token is GPU utilization: a GPU earns its keep only while it’s busy.

That break-even point is sobering. Hosted open-model prices are so low that two rented GPUs have to stay busy around the clock to compete. Big, steady workloads can get there, especially with heavy batching and quantization. Many teams can’t, and they’d be paying for idle GPUs overnight.

Buying the box

Owning changes the math: you pay once, plus power. NVIDIA’s RTX 5090 launched at $1,999 with 32 GB of memory, enough for a 30-billion-weight model at 4 bits. At its full 575 watts around the clock, and the US average home rate of about 18 cents per kWh in July 2026 (EIA), the card alone would burn roughly $77 a month in electricity.

For bigger models, Apple’s Mac Studio now takes up to 512 GB of unified memory, and NVIDIA’s DGX Spark packs 128 GB into a desktop box. None of these match a data-center GPU’s throughput, but for one person or a small team, they don’t have to.

Nobody else’s queue

An API’s speed is out of your hands. You share the provider’s servers, and every account has a rate limit that rises as you use the service more (Anthropic’s tiers are one example). Your own GPU has no rate limit, only a capacity limit, and at low load it can be very quick: in NVIDIA’s benchmarks, a small model on one H100 has a time to first token of about 10 milliseconds for a short prompt with a single user. Load it up and each user waits longer, which is the latency-versus-throughput trade that runs through all of serving.

It never left the building

This is often the real reason teams self-host. With your own hardware, prompts never leave your network, which may be the only acceptable answer for some data (the government cloud post is all about that).

APIs have closed some of the gap. OpenAI says API data isn’t used to train its models by default, and Anthropic says the same for its commercial products. Some providers also offer zero data retention on request. But “they promised” and “it never left the building” aren’t the same thing to an auditor.

The best models aren’t for sale

As of September 2026, the top of the LMArena text leaderboard is all closed models: their weights aren’t published, so you can only reach them through someone else’s service. Open models have closed much of the distance. OpenAI’s own open-weights gpt-oss-120b runs on a single 80 GB GPU under the Apache 2.0 license.

For well-defined tasks like classification, extraction and summarizing your own documents, an open model is often enough. For the hardest reasoning, the frontier APIs usually still lead. Either way, test both on a few hundred of your own real questions before deciding. A leaderboard can’t tell you that.

Who gets paged?

Price comparisons leave out one line: with an API, when the model is down, the provider’s engineers are on it. With your own GPUs, it’s you. Drivers, inference-server upgrades, security patches, new model versions, failed hardware and capacity planning are all yours now. That’s a real cost, and it’s paid in engineers’ time rather than in the hourly GPU price.

So, rent or run?

Rent by default. Per-token pricing is cheap, elastic and someone else’s pager. Run your own when your data can’t leave, when your volume is big and steady enough to keep GPUs busy, or when you need control over the exact model and its latency. Start on the meter; sign the lease when the meter runs all night.

Many teams end up with both: an API for the hard questions and a small self-hosted model for the steady, sensitive, repetitive work. Whichever you pick, re-run the numbers every few months, because they won’t hold still.

References & further reading

Prices first (all as of September 2026), then the rest.