How a Transformer Predicts
the Next Token

Before any math can happen, the sentence has to stop being text. Step 1 splits "The cat will sit on the ___." into tokens; step 2 swaps each token for an integer ID. From this point on, the model works only with numbers.

Step 1: split the text into tokens

The tokenizer splits our sentence into six tokens and leaves position 7 empty, because that is the slot the model must fill. The position numbers simply show reading order (like every number on this page, this is the simplified teaching view).

As a plain sequence, the model's working input is:

[The] [cat] [will] [sit] [on] [the]

• Tokens are not always whole words. A tokenizer breaks longer or less common words into smaller reusable pieces: unbelievable -> un + believ + able. Our sentence is simple, so each token here happens to be a full word.

Step 2: tokens become token IDs

Every token in the model's vocabulary has its own integer ID, so this step is a mechanical lookup. The IDs below are simplified teaching examples, not values taken from any particular tokenizer or model.

1. Look up. Find each token in the vocabulary.
2. Replace. Swap the token for its integer ID.
3. Hand off. Pass the ID list forward so the next step can turn each ID into a vector.

So the sentence the model actually receives is:

[791, 8415, 690, 2503, 389, 279]

• The crucial point. IDs are only labels. The number 8415 does not itself contain the meaning of "cat": it is the label the model uses to find the meaning, and the meaning arrives in the next step.

Next: how each of those six IDs becomes a vector of numbers that can actually carry meaning, in the Embeddings tab.

Step 2 turned the sentence into IDs: [791, 8415, 690, 2503, 389, 279]. But an ID is just a label, the way a defect ID points at a bug without describing it. The number 8415 does not contain the meaning of "cat". The embedding is where the features actually live.

Step 3: token IDs become embeddings

• What an embedding is. For every token ID, the model looks up an embedding: a list of numbers where each number represents a feature the model learned during training.
• Read it like a scorecard. In the toy table below, "cat" scores 0.94 on Animal-related and "sit" scores 0.96 on Action-related: the numbers describe what each token is about.

Here is the running sentence, "The cat will sit on the ___.", in a simplified 4-dimensional embedding space. Every value in this table is a simplified teaching example: real models learn thousands of dimensions.

A real embedding does not come with four tidy columns. It looks like this:

cat -> [0.018, -0.246, 0.774, ..., -0.091]   (thousands of numbers)

• Honesty check. Real dimensions are learned during training and do not normally have human-readable labels like "Animal-related". The column names here are teaching props, nothing more.

Step 4: add positional information

Embeddings say what a token is, not where it sits. And order changes everything:

"The cat chased the dog."   ->  the cat does the chasing
"The dog chased the cat."   ->  same words, opposite meaning

• The fix. The model combines two signals: token embedding + position information. Each vector now carries identity and location together.
• Why it matters for the blank. Position 6 reading "a noun probably comes next" is one of the strongest hints about what should fill the gap.

Position by position, here is what location adds in our sentence. The readings are simplified teaching interpretations, not literal labels inside the model.

Every token now knows what it is and where it stands: open the Attention tab to watch those vectors start asking each other for help.

Attention is how the final position asks the rest of the sentence for help. On its own, the last "the" only knows that a noun phrase has started: to set up the prediction it has to pull in who is acting (cat), what the action is (sit), and what kind of word fits the blank.

Step 5: every token gets a Query, a Key, and a Value

• Three vectors per token. From each token's order-aware vector, the model computes three new vectors using matrices it learned during training (WQ, WK, WV).
• Three roles. Each vector answers a different plain-English question, summarized below.

Q = Embedding x WQ
K = Embedding x WK
V = Embedding x WV

Conceptually, the Query of the final the behaves like: "I need information that helps identify the noun that should come next."

And here is what each token's Key advertises. Real Keys are vectors of numbers: these paraphrases are simplified teaching examples.

Step 6: self-attention scores the sentence

One formula runs the whole conversation:

Attention(Q, K, V) = softmax(QK^T / sqrt(d)) x V

1. Compare. Match the current token's Query against every Key in the sentence.
2. Score. Each comparison produces a relevance score.
3. Normalize. Softmax converts the scores into percentages that add up to 100%.
4. Collect. Pull information from the Value vectors, weighted by those percentages.

For the final position in "The cat will sit on the ___.", a simplified attention pattern could look like this. The percentages are teaching examples, not real model weights.

Blend the Values with those weights and the final position now holds an idea close to: "An animal is going to sit on a particular surface or object."

• Learned, never hand-written. Attention weights come from training data. No engineer programmed a rule giving "on" a 34% share: the model learned which tokens matter by seeing patterns in text.

Step 7: many heads, one richer picture

• Attention runs in parallel. The model does not run one attention calculation: several heads run at the same time, and different heads may learn to track different relationships.
• Each head has a specialty. In our simplified picture: a grammar head reads "will" and "the" and expects a noun, an action head checks the noun is compatible with sitting, a relationship head treats "on" as a surface signal, a subject head keeps the cat in view, a phrase head recognizes common continuations of "sit on the", and another head can watch long-range grammar.
• Combined output. All head outputs are merged into one richer representation that carries every one of those readings at once.

The sentence has now been read from several angles at once: the Layers tab shows how repeated transformer blocks refine that blended reading into a prediction-ready vector.

Attention decided where each token should look. The machinery that follows does the refining: a feed-forward network sharpens what each position now knows, residual connections and normalization keep everything intact and stable, and then the whole block repeats, layer after layer.

Step 8: the feed-forward network refines the picture

After attention has gathered context from across the sentence, each token's representation passes through a small network:

linear transformation -> activation -> linear transformation -> improved contextual representation

• Attention collects, feed-forward processes. Attention pulled in the right information from the other tokens; the feed-forward network works on that combined meaning and turns it into a sharper, more usable representation.
• It runs on each position. Every token's representation goes through this chain. For next-token prediction, the position we care about most is the final one (the second "the").
• What the final position now encodes. After this refinement, the last position carries a checklist-like picture of what should come next.

These strengths are simplified teaching examples: inside the model this is a pattern of numbers, not a labeled checklist.

Step 9: residual connections and normalization

Two safety rails wrap around attention and the feed-forward network so useful information survives and the numbers stay well behaved:

new representation = old representation + processed information

• Residual connections prevent information loss. Each step adds its processed information on top of the old representation instead of replacing it, which helps preserve useful information learned earlier.
• Layer normalization keeps the numbers stable. After each addition, the values are rescaled into a steady range so they do not drift out of control as they pass through many layers.

Put together, one transformer block has this shape:

1. Input. The token representations enter the block.
2. Multi-head self-attention. Every token gathers context from the tokens that matter to it.
3. Add and normalize. The attention output is added to the input, then normalized.
4. Feed-forward network. Each position gets refined.
5. Add and normalize. Added back on top and stabilized once more.
6. Output. A more contextual representation, ready for the next block.

Step 10: repeat through many layers

That block does not run once. It repeats, and a model may stack dozens or even hundreds of these layers, with the representation becoming more contextual at every pass.

• Each pass adds context. The same sentence flows through block after block, and the final position keeps absorbing more of the sentence's structure and meaning.
• By the final layer. The last position approximately expresses one instruction:

  "predict a noun describing something a cat can sit on"

• Honesty check. That instruction is stored as numbers, not as an English sentence hiding inside the model. The quote is our human translation of what the numbers encode, written for teaching.

The stack has done its thinking; the Decoding tab shows how those final numbers become one score per vocabulary token, then a probability table, then the word on your screen.

The stack of layers ends with one score for every token in the model's vocabulary. Decoding is the final stretch: raw scores become probabilities, a selection rule picks one token from the table, and that pick is the word you actually see.

Step 11: the final vector becomes logits

The refined vector at the final position is multiplied by an output matrix, and out comes one raw score, called a logit, for every token the model knows:

final vector x output matrix -> one raw score per vocabulary token (a logit)

These logits are simplified teaching numbers; a real model scores every token in its vocabulary in one go.

• Higher means more suitable. "mat" at 8.2 fits this context better than "moon" at 0.7, and far better than "running" at -1.2.
• They are not probabilities yet. Logits can be any size, even negative, and they do not sum to anything meaningful. One more step turns them into percentages.

Step 12: softmax turns scores into probabilities

Softmax converts the whole set of logits into a clean probability table:

Probability(token i) = e^(logit i) / sum of e^(all logits)

Again, simplified teaching numbers: the exact values do not matter, the shape of the table does.

• Every token gets a share. Each logit is exponentiated and divided by the total, so every share is positive and together they sum to exactly 100%.
• Small gaps get stretched. "mat" leads "floor" by only 0.6 in logits, but by 16 percentage points in probability: the exponential rewards even a modest lead.
• Honesty check. The model does not know "mat" is correct. It predicts that "mat" is the most probable next token for this context, and those are two different claims.

Step 13: pick one token

One probability table, several ways to pick from it. The whole finish in miniature:

1. Score. Every vocabulary token gets a logit.
2. Normalize. Softmax turns the logits into a table that sums to 100%.
3. Select. A decoding rule chooses exactly one token from that table.

• Greedy decoding. Always take the highest probability. Here that is "mat" at 36%, every single time.
• Sampling. Draw randomly according to the probabilities. "mat" wins most often, but "sofa" comes up too (about 7 times in 100), and it makes a perfectly plausible sentence.
• Temperature. A dial that controls how adventurous the draw is.

Treat the effect descriptions as simplified rules of thumb for teaching.

Greedy (always the top token):  The cat will sit on the mat
Sampling (one possible run):    The cat will sit on the sofa

That is the full journey from prompt to token; the QA angle tab turns it into what this mechanism means for the people who test it.

You now know the machine. Text became tokens, tokens became vectors, attention mixed them, and a probability table picked mat. This last tab is about what that machine means for anyone who has to test it: why generation is a loop, why LLM features feel flaky, and the whole pipeline recapped on one screen.

Step 14: generation is a loop

Picking mat did not finish the job, it produced exactly one token. The selected token is appended, the input becomes "The cat will sit on the mat", and the entire stack you just walked through, tokenization all the way to softmax, runs again from the top on that longer text.

• The table is rebuilt. The appended token changes the context, so the model produces a fresh probability table for the new, longer sentence. Nothing is stored anywhere as "the answer".
• A fresh table appears. On the new pass the strongest candidates are punctuation and connectors, because the sentence now looks complete.
• One token at a time. There is no plan for the whole sentence stored anywhere in the model. Each cycle answers exactly one question: what is the most probable next token, right now.

As with every number on this page, these values are simplified teaching examples, not real model output. In pseudocode, the whole generator is a short loop:

while not stopped:
    tokens     = tokenize(text)
    probs      = full_forward_pass(tokens)   # steps 1 to 12
    next_token = pick(probs, temperature)    # step 13
    text       = text + next_token           # append, repeat

Why testers should care

Everything above was mechanics. Read the same story as a tester and each mechanical fact turns into a testing consequence.

• Every output comes from a probability table. With sampling enabled and temperature above 0, the decoder draws from that table, so the identical prompt can legitimately return different answers on different runs. That is why LLM features feel flaky in a way normal code does not, and why assertions on exact strings keep breaking. For deterministic checks, run at temperature 0. Where you cannot control temperature, assert on properties of the answer (required fields present, format valid, forbidden content absent) instead of exact wording.
• Hallucination is this mechanism working as designed. The model emits the most probable continuation, and most probable is not the same as true. A confident wrong answer is not the pipeline malfunctioning, it is the pipeline doing its job on unhelpful odds. That is exactly why eval suites exist: you measure answer quality across many cases instead of assuming correctness.
• No database was consulted. At no point in steps 1 to 14 did the model look anything up. It computed a continuation from learned statistical patterns. So when an answer includes a price, a version number, or an API endpoint, that value was generated, not fetched, and it deserves verification.

The whole pipeline on one screen

The full journey from raw text to the next token, one row per stage. Every numeric value here is a simplified teaching example; real models use thousands of dimensions, billions of parameters, and many transformer layers.

If you keep one line from this page, keep the chain: text -> tokens -> token IDs -> vectors -> attention -> transformer layers -> vocabulary scores -> probabilities -> selected next token. The transformer does not retrieve a fixed answer from a database. Training taught it statistical and structural patterns, roughly "cat" plus "sit on the" points toward mat, floor, chair, sofa, bed, and it calculates which continuation is most probable for the current context.

Where to go from here

A short ladder for a QA reader who wants to turn this theory into day-job skills, in the order that builds best.

1. Separate the LLM from the agent. A next-token engine on its own only predicts text; an agent wraps it with tools, memory, and goals. See where the line sits in LLM vs AI agent.
2. Score outputs with evals. Turn "most probable is not always true" into numbers you can gate a release on with the DeepEval masterclass.
3. Ground answers with RAG. Cut hallucination by letting the model draw from your own documents in the RAG tutorial for QA.
4. Set up the machine end to end. Build a working local AI testing environment with the AI Tester Blueprint setup guide.
5. Pick your lane. Work out where a QA background lands among the roles in ML vs AI vs DL engineer.

You have now watched a transformer predict a single token end to end, and you know why testing one is a different sport from testing regular code. The next practical step is to get the machine running on your own laptop: start with the AI Tester Blueprint setup guide.