Fine-tuned open models or the biggest API? How we decide
The cheapest model is the smallest one that does the job well.
Every AI project we take on reaches the same fork early. Do we call a frontier model through an API, or do we fine-tune an open-weight model and run it ourselves?
No rule settles it. A few questions usually do.
How much traffic will it see?
API pricing is simple and it scales in a straight line. That is ideal for a prototype, or for a tool a dozen people use a few times a day. It gets expensive when the same task runs thousands of times an hour, every hour.
Published estimates of where self-hosting starts to pay off vary a lot, from around 5 million tokens a day to 30 million, depending on the model, the hardware and how well the serving setup is tuned. The exact number matters less than the shape. Below a certain volume you are paying for convenience. Above it, you are paying a premium for capacity you could own.
How narrow is the task?
Frontier models are generalists. They can write a poem, debug a query and summarise a contract. Most business workloads need one of those things, done the same way, every time: classify this ticket, pull these fields out of this invoice, draft this reply in our tone.
A smaller open model, fine-tuned on a few thousand good examples of that one task, often matches a general model on it. It is also faster, because it is smaller, and more predictable, because it has been trained on exactly what you want. On our projects, that combination is where the big savings come from: inference costs up to 70% lower than calling a general-purpose model for the same work.
Where does the data have to live?
For many companies in the region, this question decides everything. Customer records, contracts and health data often cannot leave a particular country or a particular cloud. Open models can run on AWS, Azure or Google Cloud in the region you choose, or on your own hardware. You control the logs, how long they are kept and who can see the prompts.
Who will maintain it?
This is the honest cost of going open. Someone has to test new model releases, retrain when the task drifts, watch quality and keep the serving infrastructure patched. If nobody owns that work, the savings disappear into firefighting. An API provider does all of it for you, and that is worth real money.
What we usually recommend
Start on an API. It is the fastest way to learn what the task really is and what good output looks like. Log every request and response.
Once the workload is stable and the volume is real, use those logs to fine-tune a smaller open model. Run both side by side for a few weeks, compare quality and cost, and switch when the numbers say so.
Some work should stay on a frontier model for good: open-ended reasoning, rare requests, anything where the question changes every time. The point is not to avoid the big models. It is to stop paying for them on jobs a much smaller model can do.