What parameters store: what 10B to a trillion actually means

At an IT committee meeting the principal pointed at "a trillion parameters" on a slide and asked: "The other vendor says seven hundred billion. Is that thirty per cent worse? Should we take the bigger one?" The IT coordinator started to answer and realised all he had was "the vendor says it's faster" — which is exactly the gap this lesson fills. Parameter count is not a score. It is how many adjustable dials sit inside the model.

Lessons two and three covered generation and tokens. This is module one, lesson four: where the ability to predict the next token actually lives. After this lesson you should be able to tell the principal why a larger parameter count is not automatically the right fit for your school, and where school documents belong instead.

---

Twenty years of marking instinct

Picture a Chinese language teacher who has marked essays for twenty years. She may not quote the DSE marking criteria from memory, yet within thirty seconds of picking up a Secondary 3 argumentative script she knows whether the paragraphs drift and the claims are hollow. That is not because she stores PDFs in her head, but because countless rounds of marking built a sense of pattern.

Model parameters are the mathematical version of that instinct: billions of tiny numbers, meaningless alone, that together determine what token is most likely next. Training keeps nudging those numbers until prediction is good enough. Lesson 6 covers how the dials are set; this lesson covers what they store.

"Ten billion parameters" means ten billion dials. More dials can hold finer patterns, but whether the settings are any good still depends on data and training method.

Compressed patterns, not a filing cabinet

This is the central distinction.

Parameters store patterns compressed from enormous text, not item-by-item archives. Therefore:

• It knows how argumentative essays are usually structured — the pattern was learned well.
• It knows Hong Kong secondary schools generally run from Secondary 1 to Secondary 6 — common enough to become a pattern.
• It does not reliably know the third recommendation in your school's 2018 external review report — almost absent from training, so it could not be compressed in.

When you ask the third kind of question, the model usually will not say it lacks data. It assembles something plausible from the first two kinds of pattern — the hallucination mechanism in lesson 9.

Parameters are marking instinct, not the school archive — school facts belong in a retrievable, citable knowledge base, not in a hope that the model remembers.

A primary IT coordinator's blind test

A primary school IT coordinator wanted a model on the school server so teachers could draft circulars without going to the internet. Two options: seven billion parameters on a machine the school already had with a dedicated graphics card, or seventy billion parameters needing about a hundred thousand dollars of new hardware.

He took twenty real circulars from the past year, generated versions with both models, and had three panel heads score them blind.

The 7B model matched the 70B on format and register. Gaps showed up in occasionally loose logic in long paragraphs and more errors on uncommon Education Bureau policy names. Once both models had retrieval over school documents, the second gap almost disappeared.

His conclusion: start with 7B plus a knowledge base and put the saved budget into teacher training. For most school tasks, putting the right documents in front of the model beats tripling its size — practical detail is in small language models and local deployment.

Reading the labels

• 7B, 13B, 70B. B is billion. A 7B model is in the "one graphics card" class; 70B needs a workstation or several cards.
• MoE, mixture of experts. Total parameters may run into hundreds of billions, but only a fraction activates per answer — like a school with sixty teachers where each question routes to two specialists. Marketing quotes the total; the active count per answer is much smaller.
• Undisclosed closed flagships. Parameter figures quoted for comparison are often outside estimates, not hard specs.

Hardware needs also tie to quantisation — the same model can be squeezed into less memory.

Where people go wrong

• The drift: procurement meetings compare parameter size only, without asking about Traditional Chinese, on-campus data, or teacher monitoring. Self-check: Does your selection sheet have columns for Chinese/HK usage, deployment location and audit logs? If it only has parameter count, rewrite the sheet.

• The drift: assuming everyday chat writes school rules into the model. Pasted documents do not permanently change parameters. Self-check: Start a fresh chat with no documents and ask about an internal rule. If it still answers as though it knows, you need a knowledge base, not a larger model.

• The drift: deciding school AI is impossible because 70B will not run, while ignoring 7B plus quantisation plus RAG. Self-check: What share of your task list is drafting circulars, questions and comment first drafts? If over seventy per cent, 7B is often enough; escalate only for the remainder.

Try it once

Time: about twelve minutes. You need: any AI tool.

Do this: paste the following:

From memory, write out in full the complete marking criteria for the Chinese Language paper of the Hong Kong Diploma of Secondary Education. Do not summarise or paraphrase.
If you cannot, explain instead which kinds of thing you do remember and which kinds you do not, with one example of each.

Deliverable: the model's admission of what it remembers versus what it cannot — usually general structure and common criteria, not verbatim wording.

Done when: you can explain to the principal or a parent, using the marking-instinct analogy, that the model knows writing patterns but not item three of your 2018 review report — so school documents must live in a knowledge base, not in parameters.

Next lesson: embeddings, positional encoding and self-attention. Previous lesson: tokenisation and why Chinese costs more.