In the last article we met attention: the ability of a word to look at other words and focus on the ones that matter. I described it a little loosely on purpose, to get the intuition across. Now I want to sharpen it into the precise form used in modern language models. It has two parts: self-attention, where every word in a sentence attends to every other word at once, and multi-head attention, where the model does this several times in parallel to catch different kinds of relationships. Together, they form the core of the transformer, the architecture behind essentially every large language model.
This is the same simple idea from last time, just made more precise. Let me build it up carefully.
Self-attention: every word looks at every word
Last time, we focused on one word (“it”) looking at the others. Self-attention simply does this for every word in the sentence, all at the same time. Each word looks at every other word (and at itself), works out which ones are relevant, and builds an updated, context-aware version of itself.
Take “The cat sat on the mat.” Under self-attention, “cat” looks around and pays attention to “sat” (what it’s doing); “sat” pays attention to “cat” (who’s doing it) and “mat” (where); “mat” attends to “on” and “sat.” Every word ends up enriched by the words that give it meaning. Figure 1 shows this web of all-to-all connections.
Figure 1: In self-attention, every word attends to every other word in the sentence at once. Each word builds a new, context-aware version of itself by blending in the words most relevant to it (thicker lines = more attention).
Query, Key, and Value: attention as a search
To make self-attention work precisely, each word takes on three roles. These have slightly technical names, Query, Key, and Value, but an everyday analogy makes them easier to follow: searching for something.
Imagine looking for a video online. You type into a search box. That’s your Query, what you’re looking for. Every video has a label or title describing it. That’s its Key, what it’s about. And every video has its actual content. That’s its Value, what you get if you pick it. Searching means matching your query against all the keys, then retrieving the content (values) of the best matches.
Self-attention does exactly this, with every word playing all three roles at once:
- Each word forms a Query: “here’s what I’m looking for in the other words.”
- Each word offers a Key: “here’s what I’m about, so others can decide if I’m relevant.”
- Each word offers a Value: “and here’s the actual content I’ll contribute if you attend to me.”
For each word, the model matches its Query against every other word’s Key to get relevance scores, softmaxes those into weights (just like before), and then blends the Values according to those weights. The result is a new version of that word, informed by whichever other words its query matched. Here’s the skeleton in simple code:
import numpy as np
def softmax(x):
e = np.exp(x - np.max(x, axis=-1, keepdims=True))
return e / e.sum(axis=-1, keepdims=True)
def self_attention(Q, K, V):
scores = Q @ K.T # match every Query against every Key
weights = softmax(scores) # turn scores into focus weights (each row sums to 1)
return weights @ V # blend the Values by those weights
# Three words, each already turned into small Query/Key/Value vectors:
Q = K = V = np.array([[1, 0], [0, 1], [1, 1]], dtype=float)
print(np.round(self_attention(Q, K, V), 2))
# each word comes out as a context-aware blend of all the wordsThe Query, Key, and Value for each word aren’t hand-made. They’re produced by weights the model learns during training. In other words, the model teaches itself how each word should search, how it should describe itself, and what it should contribute. (One footnote for the curious: the scores are scaled down a little before the softmax to keep the maths stable, a small detail that doesn’t change the idea.)
Multi-head attention: several perspectives at once
Now the second half. A single round of self-attention captures one way that words relate. But words relate in many different ways at the same time. In “The tired cat that chased the mouse finally slept,” one kind of relationship is grammatical (which noun goes with “slept”?), another is about actions (who chased whom?), another about description (what was the cat like?). Asking a single attention step to capture all of these at once is a lot.
So instead, the transformer runs several attention operations in parallel, each called a head. Each head has its own learned Queries, Keys, and Values, so each is free to focus on a different kind of relationship. Then the model combines what all the heads found into a single richer result. Figure 2 shows the idea.
Figure 2: Multi-head attention runs several attention operations side by side. One head might track grammar, another might track which words describe which, another might track references, then their findings are combined into one richer understanding.
The analogy I like: it’s a bit like handing the same paragraph to several careful readers, each told to watch for something different, one for grammar, one for who-did-what, one for tone, and then pooling everyone’s notes. You end up understanding the paragraph more fully than any single reader could manage alone. That’s multi-head attention: many perspectives, gathered in parallel, then merged.
Why this matters
Step back and look at what self-attention plus multiple heads gives us. Every word gets to look at every other word directly (solving the fading-memory problem). It all happens in parallel (solving the slowness problem). And several heads capture many kinds of relationships at once. This bundle, multi-head self-attention, is the core mechanism in the transformer, the reason it can read a whole paragraph and track how its parts hang together.
But the core mechanism isn’t the whole model. Multi-head self-attention still needs a few more pieces around it: a way to remember word order (attention alone treats a sentence like a bag of words), some extra processing after each attention step, and, crucially, the trick of stacking many of these layers so understanding builds up in stages. Assembling all of that into the complete machine is the transformer, and putting it together, end to end, is what we’ll do next.
Next in the series: The Transformer Architecture, End to End: assembling every piece we’ve built into the machine behind modern AI.