If you’ve followed the earlier tracks, you now know three useful ways to shape what a language model does for you: prompting (talking to it well), RAG (giving it your own knowledge to draw from), and fine-tuning (actually adjusting the model itself). Each is popular. Each has passionate advocates. And honestly, a lot of teams pick between them based on excitement rather than fit, which quietly wastes a lot of money and time.
This article is the compass. Before we spend the whole track diving into how fine-tuning works, I want to help you decide whether fine-tuning is the right tool for a given problem in the first place. Because for most tasks, it isn’t. And knowing when it is saves you from either underusing a great technique or reaching for it when a simpler tool would have worked far better.
What each technique actually changes
Let me put the three side by side, because the difference between them is easier to see than you might think.
Prompting changes how you ask. The model stays as it came out of the factory. You get better output by writing clearer, richer instructions. It’s fast, cheap, and always available.
RAG changes what the model reads before answering. Again, the model itself is unchanged. You give it the right pages of your own knowledge base at the moment of the question. Answers stay grounded in real, up-to-date material.
Fine-tuning changes the model itself. Its weights are adjusted through further training, so the model’s default behaviour, style, and habits shift. This is a much heavier move, permanent (until you fine-tune again), and the only one that changes the underlying artefact.
Figure 1 puts this contrast in one picture.
Figure 1: Prompting changes how you ask. RAG changes what the model reads at answer time. Fine-tuning is the only one that actually changes the model’s weights. The three techniques operate at completely different layers.
Read that figure carefully, because it’s the mental model the whole rest of this article depends on. The three techniques aren’t competing versions of the same thing. They’re different tools operating at different layers of the system.
The single question that decides most cases
Here’s the framing that will save you the most time and money. When a model isn’t giving you what you want, ask yourself: is the problem that the model doesn’t know something, or that it doesn’t behave the way I want?
That single question resolves the majority of real-world decisions:
- If the problem is knowledge (facts, documents, current information, anything about your data), the answer is almost always RAG. Fine-tuning is a terrible way to teach a model facts, because facts change constantly and re-training is slow and expensive. RAG updates the moment you edit a document, and it lets the model cite its sources.
- If the problem is behaviour (tone, style, format, following a specialised pattern, sounding like your brand), you have a real fine-tuning candidate on your hands. This is what changing the model’s weights is good for.
- If the problem is neither of those and just needs clearer instruction, the answer is better prompting, and you should try that first, always.
That’s it. That’s the single question that resolves most of the noise around this decision.
When fine-tuning wins
Let me be more specific about the cases where fine-tuning is truly the right call, because they’re narrower than the hype suggests. Fine-tuning shines when:
- You want a very specific style or voice consistently, and prompting alone can’t hold the line reliably. A brand voice, a house writing style, a particular way of formatting responses.
- You have a specialised, narrow task the model doesn’t do well by default, but that you have thousands of good examples of. Classifying medical notes into your particular taxonomy, extracting fields from a particular form layout, generating responses that follow your organisation’s specific conventions.
- You need lower cost or latency at scale. A smaller, fine-tuned model can sometimes match a much larger general model on your specific task, at a fraction of the cost per query. This one matters most for high-volume applications where the API bill is real.
- You need to shift what the model refuses or accepts. Adjusting a model’s defaults for a specialised domain (say, security research or medical content) is a legitimate fine-tuning use case.
Notice what’s not on that list: teaching the model facts. That’s important enough to say twice.
When fine-tuning is the wrong answer
Just as important is knowing when not to fine-tune, because this is where a lot of money and effort quietly vanishes:
- When your knowledge changes often. Every change means re-tuning. RAG updates instantly.
- When you want the model to cite sources. Fine-tuning bakes information into weights; there’s nothing to cite.
- When you don’t have a lot of good training examples. Fine-tuning without enough data usually makes things worse, not better.
- When you haven’t seriously tried prompting first. Many “we need to fine-tune” projects turn out to be “we need to prompt better” once someone sits down with the anatomy of a prompt.
- When you’re chasing a slight quality bump. Fine-tuning is expensive and adds long-term maintenance cost. It should solve a real, named problem, not shave a percentage point off a benchmark.
There’s a pattern in that list: fine-tuning is the heaviest, most permanent, most expensive of the three techniques, so it should earn its place, not be the default.
The decision framework, step by step
Figure 2 puts it all together as the flow I’d honestly recommend anyone follow.
Figure 2: Start with prompting. If the problem is that the model needs your knowledge, move to RAG. Only if it’s about behaviour (style, format, specialised patterns) and you have the training data to support it, move to fine-tuning. These techniques compose; they’re not exclusive.
Walking through it:
- Start with prompting. For most tasks, a well-structured prompt with the right role, format, and examples solves the problem. It’s free, fast, and reversible. Never skip this step.
- If the model needs knowledge it doesn’t have, move to RAG. Facts, documents, your own data, anything that changes over time. This is nearly always the right answer for knowledge problems.
- If the problem is truly about behaviour and you have solid training data, then consider fine-tuning. Style, tone, a specialised format, a narrow task. Even here, try prompting hard first, because a great prompt often gets you 80% of the way there without the cost and maintenance of fine-tuning.
A crucial thing to notice: these techniques compose. You can fine-tune and use RAG and use good prompting all at once, and mature systems usually do. A fine-tuned model that follows your house style, retrieves from your knowledge base, and sits behind a good prompt can combine the strengths of all three. The decision isn’t which one to pick; it’s which one to add next to solve the problem you actually have.
A concrete example, to make it real
Imagine you’re building a customer support assistant.
- First move: prompting. Design a great system prompt for the persona and a template for the answer format. This alone will get you a surprisingly capable assistant.
- Second move: RAG. Wire it up to your product documentation and help articles so it answers from your content, not the model’s fuzzy memory. Now the answers are grounded and current.
- Third move (only if needed): fine-tuning. If, after all the above, you notice the model still doesn’t quite sound like your brand, or repeatedly formats things wrong in a way you can’t fix with prompting, then consider a lightweight fine-tune to lock in the tone and format.
If instead you’d started with fine-tuning, you’d have spent weeks and thousands of dollars adjusting a model to memorise your product docs, only to have to redo it every time the docs changed. The order matters.
The mindset for this whole track
The rest of this track is going to teach you fine-tuning properly: how it actually works, how to prepare data, how to use it efficiently, how to evaluate it. But everything in it rests on this framing article, and I want you to keep it close as you go. Fine-tuning is a useful, sometimes irreplaceable tool. And it’s also the technique most often reached for when it isn’t needed. The teams that get the most out of it are the ones who use it where it fits, and use prompting and RAG everywhere else. Be one of those teams, and this track will be worth far more than the sum of its techniques.
We’ll start the real work in the next article by looking under the hood at what fine-tuning actually does to a model, so the rest of the track has a solid mechanical foundation to build on.
Next in the series: How Fine-Tuning Works, the mechanics of adjusting a model’s weights.