Now the article that quietly changed how fine-tuning is done in the real world. Until a few years ago, fine-tuning a large language model was expensive, and mostly the preserve of teams with serious hardware budgets. Then a small family of techniques appeared with the funny names LoRA and QLoRA, and suddenly fine-tuning became practical for the rest of us. Understanding why ties together a lot of what we’ve built.
This article explains the shift. What “full fine-tuning” costs, what LoRA does instead, why it works so much better than it has any right to, and how QLoRA squeezes even more out of the trick. Once you see it, the choice between them for real projects becomes obvious.
The cost problem with full fine-tuning
Let’s start honest. When we say “fine-tune a model,” the most straightforward version, called full fine-tuning, adjusts every weight in the model. All of them. For a modern language model with, say, 7 billion parameters, that means 7 billion little dials, all being nudged during training.
The trouble isn’t just that it takes time. It’s that for training to work, the machine has to hold several copies of every weight in memory: the weight itself, the gradient telling it which way to nudge, and some additional bookkeeping the optimiser needs to make good updates. In practice, this often works out to needing memory for something like four times the number of parameters. That’s the “cost of full fine-tuning” in a nutshell, and it’s why it used to require serious GPU hardware and serious money.
Which raises a useful question: do we actually need to touch every weight to get the behaviour change we want?
The insight: most of the shift is small
Recall from the earlier article that fine-tuning nudges an already-good model toward your task. It doesn’t rebuild it. The base model already has all the language ability, and we’re just bending it a little. So intuitively, the change we’re making is small, even though we’re changing many weights.
This is the insight that LoRA exploits. If the overall change to the model is small, why not represent it as a small change explicitly, instead of writing new values for billions of individual weights?
LoRA: fine-tune a tiny adapter, not the whole model
LoRA stands for Low-Rank Adaptation, and here’s the idea. Instead of changing the base model’s weights, you freeze them entirely. Set in stone. Untouchable. Then, at certain layers of the model, you add a small pair of little matrices that get added on top. Those small matrices, and only those, are what get trained. The original model stays as it was. Figure 1 shows the trick.
Figure 1: In LoRA, the huge base model weights are frozen (nothing about them changes). Small pairs of adapter matrices are inserted at certain layers, and only those tiny adapters are trained. At inference, the frozen weights and the trained adapters combine to produce the fine-tuned behaviour.
Now here’s the number that makes people sit up. The trained adapter is often less than 1% of the size of the full model. For a 7-billion-parameter model, LoRA might train only 30 to 50 million parameters. That’s a hundredfold or more reduction in what you’re actually training, and yet, for most tasks, the resulting behaviour is nearly identical to what you’d get from full fine-tuning. Nearly identical, at a fraction of the cost.
Why does this work so well? Because the actual behavioural shift needed is usually small. A tiny adapter has enough room to represent it. And because you’re not touching the huge base weights, the general capability from pretraining stays intact, with less risk of accidentally degrading the model’s general skills.
There’s a practical operational bonus too. Because the adapter is so small, you can train many different LoRA adapters for the same base model, each for a different task or style, and swap them at runtime. Same 7-billion-parameter base, different tiny adapters for different jobs.
QLoRA: same trick, even less memory
LoRA already reduced the training cost sharply. QLoRA takes the same idea and adds a second money-saver, this one focused on the base model’s memory footprint during training.
Recall that even though LoRA doesn’t train the base model’s weights, those weights still have to be held in memory during training, so the model can run its forward pass. And a 7-billion-parameter model, stored the usual way, takes a lot of memory. QLoRA’s contribution is to store the frozen base model in a compressed form, called quantisation, that uses roughly a quarter of the memory (or less), while still being accurate enough for training the adapter.
That’s the whole idea. Freeze and compress the base model, then train a tiny LoRA adapter on top. The base model’s quality is barely affected by the compression during training, and the adapter still learns effectively.
The practical difference this makes is enormous. Fine-tuning a large model that used to require a data-centre-class GPU can now, with QLoRA, be done on a single consumer or workstation GPU. This is the technique that democratised fine-tuning, and it’s why “I fine-tuned a 7-billion-parameter model on my laptop over the weekend” is now a plausible sentence.
Which one should you use?
For almost everyone, almost every time, the answer is LoRA or QLoRA, not full fine-tuning. Figure 2 lays out the choice.
Figure 2: For almost every practical project, start with LoRA (or QLoRA when memory is the bottleneck). Full fine-tuning is the specialist choice, only when you have a very unusual behaviour shift to make and the budget for it.
The rough guidance:
- Start with LoRA. For nearly every practical fine-tuning project (style, format, tone, narrow tasks, most everything covered in article 1’s “when fine-tuning wins” list), plain LoRA gets you most of the benefit of full fine-tuning at a small fraction of the cost. It’s the sensible default.
- Use QLoRA when memory is your bottleneck. If you’re trying to fine-tune a big model on modest hardware (a single GPU, a consumer machine, or a low-cost cloud instance), QLoRA is what makes it possible. The quality difference from LoRA is usually small in practice, and the cost savings can be transformative.
- Consider full fine-tuning only for unusual cases. Very large behavioural shifts across many layers, deeply specialised domains where LoRA proves insufficient, or when you have both a serious budget and a real, measured need. In most projects, if you find yourself reaching for full fine-tuning, it’s worth asking whether a bigger or cleaner dataset, or a stronger base model, would have solved the problem more cheaply.
The category label you’ll see for LoRA and QLoRA in the literature is parameter-efficient fine-tuning (often abbreviated PEFT). If you see that term, know it’s referring to this family of techniques.
Why this matters
Let me zoom out for a moment, because the impact of these techniques is easy to underappreciate. Before them, fine-tuning was something serious teams did occasionally, with serious hardware, at serious expense. After them, fine-tuning became something small teams and individuals could do routinely, on ordinary machines, at ordinary cost. That’s the shift that turned fine-tuning from a rare tool into a standard part of the AI toolkit.
Which loops back to the article that opened this track. Fine-tuning is now cheap enough that the barrier to using it isn’t cost anymore; it’s judgement. Knowing when to use it is still the whole game, and that judgement is what article 1 was for. If you’re using fine-tuning correctly, LoRA and QLoRA are what let you use it affordably.
Bridging to the practical
You now understand fine-tuning conceptually (article 2), how to prepare the data that makes it work (article 3), and the techniques that make it practical (this article). The obvious next step is to see how this all comes together in a real, hands-on fine-tune. That’s what the next article is: a practical, guided walk-through of running a LoRA fine-tune on an open model, from data to trained adapter, so you can go from understanding to actually doing it.
Next in the series: A Practical Guide to Fine-Tuning an Open Model, from data to trained adapter, hands-on.