The Transformer architecture in plain language

Short answer: a sentence entering a model is like a worksheet passed down a row of teachers, each adding a layer of pencilled notes. At the very end the model does only one thing — guess what the next word should be.

A student asked a hard question

A Secondary 2 computer studies teacher was demonstrating an AI tool when a student put up a hand: "Sir, how does it know what I'm talking about?"

He answered that it has a lot of data. The student pressed: "Does it think first and then answer?" He was stuck. It is an excellent question, and the answer upends most people's mental picture: the model does not compose an answer and then write it out. It appends one word at a time, and before each word it re-reads everything that came before.

To see why, follow a sentence through the machine.

Step one: pass it down a row of teachers

Picture a stack of worksheets passed along a row of teachers in a staffroom. The first teacher reads it and pencils an observation in the margin. The second receives the original worksheet plus the first teacher's note, and adds a layer of her own. And so on, through several dozen teachers.

Those are the layers of a Transformer. A typical model has dozens of them, each receiving the previous layer's output and adding its own processing. Lower layers handle surface matters — what part of speech a word is, how it pairs with its neighbours. Higher layers handle abstractions — the intent of the sentence, its register, what it takes for granted.

No single layer "understands" the sentence. It is the accumulated stack of annotations that finally gives the model enough to do the last step.

Step two: every teacher is deciding where to look

There is one crucial detail about how each teacher in that row works. Before writing anything, each one decides which other words in the sentence matter most for the word currently being processed.

Take "Ming borrowed Wah's notes, and he was very grateful." When processing "he", the teacher has to decide whether it refers to Ming or Wah. The way to do that is to look back across the sentence and put attention on the clues "borrowed" and "grateful", which points to Ming.

That mechanism is self-attention. The details belong to lesson 5, which uses the analogy of a student underlining the important words in an exam question. For now the only thing to hold on to is that every layer re-decides, for every word, which parts of the sentence to look at.

Before Transformers arrived in 2017, the standard approach read a sentence strictly left to right, like passing a note along a row, so information from the start had to survive the whole journey to reach the end. That was both slow and prone to losing things in long sentences. The Transformer breakthrough was to let the whole class look at the whiteboard at once: the entire sentence is laid out in front of every position, and any position can look directly at any other.

Step three: guess the next word

By the final layer, the model holds a great deal of information about the sentence. The only thing it does with that information is score every possible next word in its vocabulary.

Given "The weather today is really", the model might come up with "hot" at 0.31, "cold" at 0.18, "nice" at 0.12, "humid" at 0.09, and so on, then draw one word from that distribution. The drawn word is appended, the whole sentence goes back in, and the next word is guessed.

That single fact explains several things teachers meet daily:

• Why answers differ between runs. It is a draw, not a pick of the top score.
• Why long outputs drift off topic. Each step only cares whether the next word follows smoothly, so a long piece can wander.
• Why it states wrong facts so confidently. The phrase "I am not sure" is far rarer in training text than a confident assertion, so its probability is naturally low. Lesson 9 takes this apart properly.

An example from the classroom

A Primary 5 Chinese teacher needed a reading comprehension worksheet for a unit on public transport in Hong Kong. Her first prompt was one line: "Write me a Primary 5 Chinese reading comprehension worksheet."

The passage was decent, but the questions were too hard and two of them asked for a two-hundred-character response, which is plainly wrong for Primary 5.

Her second attempt: "Write a 350-character expository passage for Primary 5 students on public transport in Hong Kong. Then set five comprehension questions: two retrieval, two inferential, one open-ended. Use Hong Kong Traditional Chinese conventions. Do not include answers." That version was usable immediately.

Seen through the mechanism just described, what changed? In the first attempt, the only clues available at each guessing step were "Primary 5, Chinese, comprehension, worksheet". In the second, every step had a length, a text type, a question-type distribution and a language convention to work with. Every concrete condition in a prompt narrows the range the model draws from at each step. That is the underlying reason prompt engineering works, and the prompt engineering lesson covers the techniques.

Try it yourself

To see word-by-word generation with your own eyes, paste this into any AI tool:

Continue this in one sentence: "During recess this morning, out on the playground"
Then, on a new line, list four other words you considered for the first word of your continuation, and say why you did not choose them.

A model's self-report is not a precise record of its internals, but you will see very clearly that the unit it works in is "the next word", not "the whole paragraph".

What this means for your classroom

1. Put the important conditions in the prompt instead of hoping it infers them. Year level, length, genre, language conventions, things to avoid — each one written down changes the output materially. The model has no common-sense knowledge that a Primary 5 pupil cannot write two hundred characters.
2. Do not expect identical output twice. Submit the same essay for marking twice and the marks may differ. Marking is therefore a way to generate draft feedback, while the mark itself stays with the teacher. That is the technical reason we insist on teacher review in the platform.
3. Ask for long outputs in stages. Because the model keeps appending, asking for a whole lesson plan in one go invites internal inconsistency. Asking for the learning objectives first, confirming them, then asking for the activity sequence noticeably improves quality.
4. The architecture is shared, so your skills travel. Whether the school runs OpenAI, Claude or an open model on campus, the underlying design is a Transformer. Prompting skills learned today survive a change of provider.

Next

This lesson talked about "a sentence entering the model", but the model never sees words. It sees fragments. How the cutting is done, and why Chinese produces so many more fragments, is the next lesson.

Continue with tokenisation, and why Chinese costs more than English. Previous lesson: why teachers need to understand LLMs.