Where hallucination and bias come from, and how teachers can intercept them

Friday break time, and the geography teacher is frowning at her screen in the staff room: "It says Hong Kong's per-capita electricity use rose 37% between 2018 and 2023, and cites the IEA 2024 report. I've searched for half an hour. That figure isn't there."

Hallucination is not the model being broken. It is designed to fill every blank. Bias is the same system faithfully copying the world in its training data. This is the last lesson in module 2 after alignment, tying the generation mechanics you have already studied to the misfires and skew you meet every week.

---

The generator has no blank key

Lesson 2 established that each step scores possible next words and draws one. Lesson 4 established that parameters store patterns, not encyclopaedia entries. Lesson 6 established that pre-training rewards smooth continuation, not verified truth. Lesson 8 established that alignment rewards answers that feel helpful and plausible.

Put those together and the outcome is blunt. When the model meets a fact with no reliable basis, it does not stop. It keeps predicting the next most plausible word. The more fluent the paragraph, the more we trust it — yet fluency and correctness are different things. The geography teacher's 37% is a textbook case.

Four ways output goes wrong

Naming the cause tells you which remedy to reach for, instead of telling pupils simply not to trust AI.

Empty shelf

Your board minutes, one pupil's support plan, last year's anniversary timetable — none of it was in the training data. The model assembles the closest public pattern it knows. For school facts the fix is retrieval from your own files first, covered in the school knowledge base and RAG.

Thin shelf

Obscure Education Bureau circular numbers, specialist vocabulary, formal names of local bodies — the model has seen a little, so it mixes true and false. That is more dangerous than knowing nothing, because verification cost rises when lies sit inside accurate material.

Stale shelf

Curriculum changes, examination timetables, policy updates — the model remembers an older version. That connects directly to the knowledge cutoff in the next lesson: it did know, but only up to a date.

The question assumes what is not there

Ask for a summary of chapter three of a 2019 circular that was never issued and most models will still produce one, because your sentence presupposed it exists. Ask whether such a circular exists, what its number is, and to say plainly if unsure, and the answer usually shifts.

Three channels where bias enters

Ratios in the training text. If "nurse" mostly co-occurs with women and "engineer" with men, generated career profiles carry that ratio forward. The model has no opinion; it is copying statistics.

Annotators in demonstration and ranking data. The exemplars of lesson 7 and the rankings of lesson 8 were written and scored by a particular group. Their sense of what is appropriate or helpful is not necessarily a Hong Kong classroom's.

Language and regional imbalance. English data vastly exceeds Chinese, Simplified exceeds Traditional, and Hong Kong usage is only part of Traditional. Local curriculum context is handled less reliably.

In materials, bias most often appears as stereotyped gender and occupation pairings, examples that assume one family structure or income level, and over-marking of ornate but empty writing. A fuller ethics discussion is in privacy, bias and copyright ethics.

A scene from the staff room

A Primary 5 form teacher asked for ten eighty-word career profiles for a reading unit. In the first draft, seven professional roles were men and three caring roles were women. Four characters lived on Hong Kong Island's mid-levels while nineteen pupils in the class live in public housing.

She did not edit ten profiles one by one. She rewrote the prompt: roughly balanced gender, at least three jobs that do not require a degree, family backgrounds that include public housing, subsidised sale flats and subdivided units, and a ban on inventing statistics. The second draft was far more usable. She saved the prompt in the panel shared folder and spent eight minutes of co-planning comparing the two versions. Colleagues later found the same pattern in English role-play materials.

Hallucination is stopped by verification. Bias is stopped by a checklist. Neither disappears because you upgraded the model tier.

Five school-level gates

1. Give a way out. End prompts with: if unsure, write "teacher to verify" rather than filling in.
2. Require citations. Where a knowledge base exists, ask for the paragraph source. Treat numbers, dates and names without a citation as unverified.
3. Split facts from prose. AI handles structure and sentences. School history, policy numbers, statistics and names come from a person.
4. Ask backwards. After an answer, start a fresh turn: "Which items above are you actually unsure about? List them."
5. Double-read high-stakes output. External circulars, comments and anything touching grades get a second colleague's read.

Where people go wrong

• The mistake: treating fluency as accuracy. Pupils say "but it sounds so clear". Clarity is not correctness. Self-check: can every number, date and regulation name be matched to an official page or a school document?
• The mistake: banning pupils while skipping your own checks. Teachers are fooled by fluency too. Self-check: what was the last AI paragraph you used without verifying, and what kind of fact was it?
• The mistake: thinking "do not make things up" is enough. There is no fabrication switch, only a continue-writing mechanism. Self-check: does your prompt include an explicit way out and a verifiable citation rule?

Try it once

Time: about 12 minutes. Materials: any AI tool.

Do this: paste prompt A, then prompt B, and compare:

A: Explain in detail the main content of chapter three of the Education Bureau's 2019 "Guidelines on Artificial Intelligence Teaching in Primary and Secondary Schools".

B: Did the Hong Kong Education Bureau issue a document in 2019 titled "Guidelines on Artificial Intelligence Teaching in Primary and Secondary Schools"? If not, say so plainly. If you are unsure, say you are unsure. Do not describe contents while unsure.

Hand in: three lines noting whether A produced a plausible-sounding answer, whether B was more cautious, and one "way out" sentence you would add to a shared panel prompt.

You got it right if: you can explain to a colleague why A triggers hallucination more easily than B, and name one verification step you will add to your next school prompt.

Continue with knowledge cutoffs and model versioning. Previous lesson: alignment with RLHF and DPO. Related reading: LLM safety, hallucination and prompt injection risk.