The school knowledge base and retrieval-augmented generation

At the start of term a newly posted secondary Liberal Studies teacher asks on WhatsApp: "How many marks off for a late long assignment?" Three colleagues give three different answers. She tries a general AI instead and gets a fluent, authoritative paragraph that does not match the school's Assessment Policy at all. The vice-principal later says: "The AI is not stupid. You asked in the wrong place. It has never opened our filing cabinet."

Retrieval-augmented generation is a pivotal applications lesson. The previous lesson covered job-ticket prompts; this one covers what material goes into the prompt, so the AI reads school documents before it speaks rather than improvising from training memory.

---

How a school office clerk answers

A general AI is like a well-read consultant who has never crossed your school gate. On school-specific detail it fills gaps with "what schools usually do", and it often sounds sure.

RAG changes the sequence: look up first, write second. The name retrieval-augmented generation puts retrieval before generation, and the order is the point. Picture a school office clerk handling a parent enquiry. She does not answer from memory. She opens the parent circulars binder, finds the relevant page or two, replies from the wording, and says which paragraph it came from.

Three steps: file, match, write

File (chunk and index). When the school uploads documents, the system cuts each one into passages and gives every passage a meaning fingerprint, an embedding explained in embeddings, positional encoding and self-attention. The intuition you need: matching is by meaning, not identical wording. Asking what proof to submit after sick leave can still find the passage about a medical certificate.

Match (retrieve). When someone asks a question, the system fingerprints the question too, finds the closest passages, often a dozen at first, then narrows to the best three or four (reranking is in the next lesson).

Write (generate). The selected passages go to the model with the question and an instruction to answer only from them and mark where each point came from. Documents never enter model training. They stay in the school database, where they can be added, removed or exported.

Only an answer with a source can be checked. Being checkable is the fundamental difference between a school platform and a general chat tool.

Why you should not "train" it instead

The instinct is understandable: if it does not know us, train it on our data. That path is costly, risky and aimed at the wrong kind of problem.

A school knowledge base with retrieval suits facts: homework limits, leave procedures, marking rubrics, teaching schedules. Update a policy by uploading a new version. Delete a document and citations stop. Answers can cite passage by passage. An ordinary server is enough.

Fine-tuning suits a way of speaking: fixed feedback tone, a particular table layout, the house style of a specialist task. Once data is folded into parameters it is hard to recover piece by piece. You cannot tag every sentence with "from paragraph three". You need a GPU and specialist staff. Proper use of fine-tuning is in LoRA and QLoRA on one GPU.

One line covers it. For facts, retrieve. Only for voice should you consider fine-tuning.

What a pastoral head chose

At one primary school the pastoral panel weighed two routes: fine-tune a "school assistant" on three years of circulars, parent letters and leave procedures, or build a knowledge base with retrieval. IT support priced it out. Fine-tuning needs cleaned corpora, GPU time and another run whenever policy wording changes. Retrieval means upload PDFs, chunk by heading, and panel heads can see in the admin panel which documents get cited most. The head chose retrieval for a practical reason: "We need policy Q&A that is updatable, deletable and checkable, not a tone machine that memorised an old circular."

A grounded prompt: only these pages

Even without a full platform you can simulate retrieval by hand. Put extracts inside angle brackets and write hard rules:

Inside the angle brackets below are extracts from our school's documents. Answer using only these extracts.
Rules: after each conclusion, mark in square brackets which extract it came from; if the extracts do not contain the answer, reply "not mentioned in the documents" and do not fill the gap from your own knowledge.
Question: what are the arrangements for a make-up test when a pupil misses a test through illness?
<<<
[passage 1] ...
[passage 2] ...
>>>

Run the control experiment. Ask something the documents genuinely do not cover, such as missing a test for a family holiday. With the "say not mentioned" rule the model admits the gap. Remove that line and you get a beautifully constructed invention, which is what a general AI does closed-book. The fuller pattern is in grounded prompts with citations for RAG.

Where people go wrong

• The drift: asking "what is our homework policy?" in a general chat tool and declaring AI untrustworthy. Self-check: should this question be asked where a knowledge base exists? Without a filing cabinet, plausible invention is the only outcome.

• The drift: uploading dozens of scanned PDFs and expecting perfection without maintaining versions or headings. Self-check: do documents have clear section titles? Vague old and new versions mixed in one folder retrieve vaguely.

• The drift: treating a citation as automatic endorsement and never opening the source. Self-check: pick three AI answers at random. Can you open the original and match them in thirty seconds? If not, the problem may be chunking or retrieval, not "a smarter model".

Try it once

Time: about 15 minutes. Materials: any AI tool; one real school text (a section of homework policy, a circular template or a panel rubric, roughly 300–500 words, no pupil personal data).

What to do:

1. Cut the text into three passages and label them [passage 1], [passage 2], [passage 3].
2. Paste the grounded prompt above with a question answered in the text (for example late submission penalties).
3. Ask a question not answered in the text and compare whether the model honestly says "not mentioned".

What to hand in: both Q&A pairs, noting whether the model cited passage numbers and whether you could find the matching sentence in the original.

How you know it worked: you can explain to a colleague the difference between RAG and training the model, and demonstrate that checkability requires citations, and what happens when off-limits rules are missing.

Next lesson: chunking, reranking and lost in the middle. Previous lesson: prompt engineering in one lesson.