I want to start with a small confession: for a long time, the phrase “machines that learn” sounded like marketing to me. Machines follow instructions. They do what they’re told. So how could they learn anything?
The answer turned out to be simpler than I expected. Once it clicked, a lot of the mystery around AI fell away. So let me try to give you that same click.
The old way: spell out every rule
Imagine I ask you to write instructions for a computer to tell apart a photo of a cat from a photo of a dog. Not for you, since you do it instantly, but as a list of precise rules a machine can follow.
You might start with “cats have pointy ears.” But so do some dogs. “Dogs are bigger.” Not always. “Cats have whiskers.” So do dogs. Every rule you write has a hundred exceptions, and pretty soon you have a tangled mess of rules that still gets it wrong half the time.
This is the old way of making computers do things, and it works wonderfully for problems you can fully describe: adding up a column of numbers, sorting names alphabetically, checking if a password matches. You write the rules; the computer follows them.
But “is this a cat or a dog?” can’t really be written as rules. Neither can “is this email spam?” or “what will this house sell for?” The world is too messy. And that messiness is exactly where machine learning comes in.
The new way: show examples, let the machine find the rule
Here’s the flip that drives most of machine learning. Instead of writing the rules, we show the machine thousands of examples and let it figure out the rules on its own.
We don’t tell it what a cat looks like. We show it ten thousand photos labelled “cat” and ten thousand labelled “dog,” and we let it discover the patterns that separate them. Figure 1 shows this flip, which is the main idea to keep in mind.
Figure 1: The old way feeds in rules and data to get answers. Machine learning feeds in data and answers to get the rules.
Read Figure 1 slowly, because it is the core idea:
- Old way: you give the computer rules + data, and it produces answers.
- Machine learning: you give the computer data + answers, and it produces the rules.
That’s it. Machine learning is software that derives rules by studying examples. When people say a model was “trained,” this is what they mean: it was shown lots of examples and slowly adjusted itself until its rules matched them.
A tiny, concrete example
Let me make this real with the simplest example I can think of. Forget cats and dogs; let’s predict something with a clear pattern.
Suppose I have data on how many hours people studied and the score they got on a test:
| Hours studied | Score |
|---|---|
| 1 | 52 |
| 2 | 60 |
| 3 | 71 |
| 4 | 78 |
| 5 | 90 |
You can feel the pattern: more hours, higher score, roughly ten points per hour. A machine can find that same pattern automatically. Here it is in a few lines of Python:
import numpy as np
hours = np.array([1, 2, 3, 4, 5])
scores = np.array([52, 60, 71, 78, 90])
# "Learn" the straight-line relationship between hours and scores.
slope, intercept = np.polyfit(hours, scores, 1)
print(f"Learned rule: score = {slope:.1f} * hours + {intercept:.1f}")
# Now predict a score for someone who studied 6 hours, data it never saw.
predicted = slope * 6 + intercept
print(f"Predicted score for 6 hours: {predicted:.0f}")Running this, the machine discovers a rule close to score = 9.4 × hours + 43, and predicts about 99 for six hours of study. Nobody wrote that rule. The machine found it by looking at the five examples.
If you want the one line of maths behind this, it’s just the equation of a straight line:
score = (slope × hours) + intercept“Training” simply means nudging that slope and intercept until the line sits as close as possible to the real data points. That’s the basic trick. More advanced AI systems, including the ones behind ChatGPT, are much bigger versions of this same idea: adjust the knobs until the examples are explained well.
The part that actually matters: generalisation
Now here’s the subtle bit, and it’s the difference between a machine that’s genuinely useful and one that’s secretly useless.
We don’t care whether the machine can repeat the examples we showed it. We care whether it does well on things it has never seen before. Predicting the score for six hours, a row that wasn’t in our table, is the real test. This ability to handle new, unseen cases is called generalisation, and it’s the whole point.
Think about how you studied for exams. There are two ways to prepare. One is to understand the subject; then you can answer any question, even ones you’ve never encountered. The other is to memorise last year’s answers word for word, which works brilliantly right up until the questions change, and then you’re stuck.
Machines can fall into the same trap. A model that “memorises” its training examples instead of learning the underlying pattern looks like a genius on the material it studied and falls apart on anything new. This failure has a name: overfitting. Figure 2 shows what it looks like.
Figure 2: Too simple and it misses the pattern (underfitting). Just right and it captures the trend (good generalisation). Too eager and it contorts itself to hit every single point (overfitting), then fails on anything new.
In Figure 2, the middle picture is what we’re always aiming for: a model that captures the real trend without obsessing over every last dot. The one on the right has technically “learned” the examples perfectly; it passes through every point. But it has memorised the noise, and it will make terrible predictions on new data. A student who memorised the answer key, not the subject.
Why this matters for everything that follows
Almost every AI system you’ll ever hear about is built on the three ideas in this article:
- Data: the examples we learn from. More and better examples usually means better learning.
- Training: the process of adjusting the model until it explains those examples well.
- Generalisation: the goal of doing well on new, unseen cases, not just repeating the old ones.
When you read later that a language model was “trained on a huge amount of text,” you now know what that really means: it was shown an enormous number of examples and slowly adjusted itself until it captured the patterns in human language, hopefully well enough to generalise to sentences nobody ever showed it.
Everything from here builds on this. Next, we’ll look at what makes language such a hard thing to teach a machine, because words, it turns out, are trickier than test scores.
Next in the series: What Is NLP? Teaching Machines to Work with Language. If you’re just joining, start with Fundamentals of AI, ML, Deep Learning & Gen AI.