An open-weight model is free to download, but running it is not: one Nvidia H100 left on for a month costs about $2,900 to $6,500 in cloud rent at public on-demand prices checked on 5 November 2026, before you pay anyone to deploy and look after it. At the same date, a billion tokens a month on a small hosted model such as Claude Haiku 5.5 costs about $180, so self-hosting only pays when you keep the GPUs busy or a data rule leaves you no choice.
This guide is for a UK CTO weighing up two routes. Route one: download an open-weight model such as gpt-oss, Qwen, DeepSeek, Mistral or Llama and serve it on your own GPUs. Route two: pay a provider per token. I'll list every cost line, run a worked example with real published prices, and show the one number that decides it, which is utilisation.
Every price below was checked on the provider's own page on 5 November 2026. GPU clouds and AI labs bill in US dollars, so I've kept dollars. Prices exclude VAT and change often.
What "free" leaves out
The licence is the only free part. gpt-oss-120b, for example, ships under Apache 2.0, has 117 billion parameters and, according to its model card on Hugging Face, fits on a single 80GB GPU such as an H100. That's a real gift. But the model card doesn't run itself.
To serve it you need a GPU with enough memory, rented by the hour or bought outright. You need storage for the weights, logs and any fine-tuned versions. You need a serving stack, monitoring, alerts, patching, and someone on call when it falls over at 2am. You need to test quality every time you change the model or the prompts. And you pay for every hour the GPU sits idle, which with business-hours traffic is most of them.
With an API, all of that sits inside the per-token price. With self-hosting, it sits on your payroll.
GPU compute: what one H100 costs to rent
GPU rent is the biggest line and the easiest to check. The hyperscalers charge the most but give you UK regions, enterprise contracts and the rest of their cloud next door. Specialist GPU clouds charge less.
| Provider and instance | GPUs | Region | Price per hour | Per month at 730 hours |
|---|---|---|---|---|
| AWS p5.4xlarge | 1x H100 80GB | EU (London) | $8.944 | $6,529.12 |
| AWS p5.48xlarge | 8x H100 80GB | EU (London) | $71.552 | $52,232.96 |
| Azure NC40ads H100 v5 | 1x H100 NVL | UK South | $8.725 | $6,369.25 |
| Azure ND96isr H100 v5 | 8x H100 | UK South | $122.90 | $89,717.00 |
| Lambda 1x H100 SXM | 1x H100 80GB | Not UK-specific | $4.29 (effective 16 Nov 2026) | $3,131.70 |
| Lambda 1x A10 | 1x A10 24GB | Not UK-specific | $1.29 (effective 16 Nov 2026) | $941.70 |
| RunPod H100 SXM, Secure Cloud | 1x H100 80GB | Varies | $3.99 | $2,912.70 |
| RunPod H200, Secure Cloud | 1x H200 | Varies | $5.29 | $3,861.70 |
| RunPod L40S, Secure Cloud | 1x L40S 48GB | Varies | $1.09 | $795.70 |
Two things stand out. First, the same H100 costs more than twice as much on AWS or Azure in the UK as on RunPod or Lambda. Second, the region matters: Azure's 8-GPU ND96isr H100 v5 is $122.90 an hour in UK South against $98.32 in East US, and AWS's p5.48xlarge is $71.552 in London against $55.04 in North Virginia. Keeping data in the UK costs a premium.
Commitments bring the hyperscaler price down. Azure lists a one-year reservation for the NC40ads H100 v5 in UK South at $57,323, which works out at about $6.54 an hour, around 25% below on-demand. But a reservation is a bet that you'll keep the GPU busy for a year.
Owning the hardware is the other option. Nvidia and the server makers don't publish list prices for H100 systems, so I haven't put a number on it. If you go this way, budget for depreciation over three years or so, power and cooling, rack space in a UK data centre, spares, and the risk that a newer GPU generation makes yours look slow. Buying only makes sense if you can keep the cards busy for their whole life.
The cost lines most budgets miss
Storage is small but real. RunPod charges $0.07 per GB a month for standard network storage under 1TB, so 200GB for weights and logs is about $14 a month. Networking can surprise you on the hyperscalers, where data leaving the cloud is billed, while Lambda says on its pricing page that it charges no egress fees.
Engineering time is the big hidden line. Someone has to choose a serving framework, set up autoscaling, wire in monitoring, apply security patches, upgrade drivers and CUDA, and swap in new model versions. I won't quote a salary I can't source, so here is an illustration using our own published day rate of £950 to £1,500: two days a month of a specialist's time on care and feeding costs £1,900 to £3,000 a month. Your in-house cost may be lower or higher, but it is never zero.
Evaluation is the line people forget entirely. Every model upgrade, quantisation change or prompt change needs a test set and a run against it, or quality drifts without anyone noticing. With an API you still evaluate, but you don't also have to prove the serving stack is behaving.
Security and compliance come next. Your own endpoint needs access control, logging, a data protection impact assessment and a place in your incident plan. If your auditors ask who can see prompts, the answer has to be on paper. Our guide to private LLMs for UK businesses covers what that involves.
Then redundancy. One GPU is a single point of failure. A production service usually needs two, which doubles the compute line before a single extra token is served.
| Cost line | Self-hosting | Hosted API | Example figure |
|---|---|---|---|
| Model licence | Free for open-weight models such as gpt-oss (Apache 2.0) | Included in token price | $0 |
| GPU compute | Paid per hour, busy or idle | Included in token price | $2,912.70 to $6,529.12 a month for one H100 |
| Redundancy | Second GPU for failover | Provider's problem | Doubles the compute line |
| Storage | Weights, logs, fine-tunes | Not needed | About $14 a month for 200GB on RunPod |
| Networking | Egress billed on hyperscalers | Usually included | Varies by provider |
| Engineering and on-call | Deploy, monitor, patch, upgrade | Integration only | £1,900 to £3,000 a month for two days at £950 to £1,500 a day |
| Evaluation | Every model or prompt change | Every prompt or model change | Team time |
| Security and compliance | Your endpoint, your controls | Vendor terms plus your controls | Team time |
| Idle capacity | Paid for | Not paid for | Depends on traffic shape |
A worked example: one billion tokens a month
Here are the assumptions, so you can swap in your own. The workload is 1 billion tokens a month: 800 million input tokens and 200 million output tokens, an 80/20 split that's typical for document and support work where prompts carry a lot of context. Prompts are under 100,000 tokens. No batch discounts, no prompt caching. API prices are standard list prices from each provider's pricing page on 5 November 2026.
| Option | Input price per 1M | Output price per 1M | Monthly cost |
|---|---|---|---|
| Claude Haiku 5.5 (Anthropic API) | $0.10 | $0.50 | $180 |
| gpt-6-luna (OpenAI API) | $0.10 | $0.50 | $180 |
| gpt-oss-120B (Together AI serverless) | $0.15 | $0.60 | $240 |
| gpt-oss-120B (Groq) | $0.15 | $0.60 | $240 |
| Qwen3 Next 80B A3B (Amazon Bedrock, Europe London) | $0.23 | $1.86 | $556 |
| Claude Sonnet 5.5 (Anthropic API) | $2 | $10 | $3,600 |
| gpt-6.1-sol (OpenAI API) | $2 | $10 | $3,600 |
| Self-host, 1x H100 on RunPod Secure Cloud | n/a | n/a | $2,912.70 plus people |
| Self-host, 1x H100 on AWS London | n/a | n/a | $6,529.12 plus people |
At this volume, the API wins clearly. Running gpt-oss-120b yourself on one RunPod H100 costs about twelve times what Together AI charges to run the same model for you. Add a second GPU for failover and a couple of days a month of engineering, and the gap gets wider.
The only row self-hosting beats on GPU rent alone is the frontier tier, Claude Sonnet 5.5 or gpt-6.1-sol at $3,600. But that's not a fair fight. If an open model is good enough for the job, the real alternative is the same open model on a managed endpoint, not a frontier model. If it isn't good enough, self-hosting it doesn't help.
Break-even: utilisation is everything
The break-even formula is simple. Take your monthly self-host cost and divide it by the blended price per million tokens of the API you'd otherwise use. At an 80/20 mix, gpt-oss-120B on Together AI blends to $0.24 per million tokens, Claude Haiku 5.5 to $0.18, and Claude Sonnet 5.5 to $3.60.
So one RunPod H100 at $2,912.70 a month breaks even with Together AI's gpt-oss-120B at about 12.1 billion tokens a month. That's about 4,600 tokens every second, every hour of every day. Against Claude Haiku 5.5 it's about 16.2 billion tokens a month. Against Claude Sonnet 5.5 it drops to about 809 million. Rent the H100 from AWS London instead and the break-even against Together AI rises to about 27.2 billion tokens a month.
Whether one GPU can sustain that rate depends on the model, prompt length, output length and how many requests you batch together. Measure it on your own prompts before you decide. And remember the hours. A GPU that's busy only during UK office hours, say 50 hours a week, is idle for about 70% of the month, so your real cost per token is more than three times the figure at full load.
That's why the two numbers that matter are your sustained tokens per second and the share of hours you keep the GPU busy. Get those from a pilot, not a vendor slide.
When self-hosting wins
Data residency and control. If a contract or regulator says prompts must never leave infrastructure you run, an API is off the table whatever it costs. Note that hosted APIs are improving here: Anthropic offers US-only inference at a 1.1x price multiplier, and OpenAI lists a 10% uplift for its regional processing endpoints. Check whether the region you need is on the list before you assume you must self-host.
Very high, steady volume. Overnight batch jobs, classification over millions of records, or an always-busy internal service can keep GPUs near full. That's where the break-even maths turns in your favour.
Small fine-tuned models. A model of a few billion parameters, tuned for one narrow task, can run on a single A10 or L40S at about $800 to $950 a month on Lambda or RunPod, and may do that one job as well as a much larger general model.
Offline and air-gapped sites. Defence, some health settings and remote operations can't call out to the internet. There, self-hosting is the only choice.
When it doesn't
Spiky or modest traffic, a small team with no one to carry a pager, or a need for the best available model quality all point to an API. So does speed: you can be live on an API in days, while a reliable self-hosted service takes weeks to build and test.
The middle route: managed open-model endpoints
You don't have to choose between a frontier API and your own racks. Managed services run open-weight models for you and charge per token. Together AI listed gpt-oss-120B at $0.15 input and $0.60 output per million tokens, and also rents dedicated H100s at $5.49 per GPU hour. Amazon Bedrock prices open models by region, including Europe (London): Qwen3 Next 80B A3B at $0.23 input and $1.86 output, and Ministral 8B 3.0 at $0.23 for both. Groq, which runs models on its own inference chips, lists the same gpt-oss-120B price as Together AI and gpt-oss-20B at $0.075 input and $0.30 output; its Llama models are now sold to enterprise customers through sales only. Azure AI Foundry also offers open models, though I haven't priced it here.
This route keeps your options open. Build against an open model on a managed endpoint now, keep your prompts and evaluation set portable, and move the same model onto your own GPUs if volume or a data rule later justifies it. Our piece on model-agnostic infrastructure explains how to build so that switching is a configuration change, not a rewrite.
How we can help
If you're not sure which side of the break-even line you're on, the first step is measuring your real token volume and traffic shape. Our free AI readiness score takes two minutes and shows where to start.
Sources
All checked on 5 November 2026. API prices: Anthropic pricing documentation, Claude pricing page, OpenAI API pricing. GPU prices: AWS EC2 on-demand pricing and AWS P5 instances, Azure retail prices API, Lambda pricing, RunPod pricing. Managed open models: Together AI pricing, Amazon Bedrock pricing, Groq supported models and prices. Model details: gpt-oss-120b model card on Hugging Face.