AI guardrails with Llama Guard, NeMo and enforced structure

In March, a Secondary 1 class teacher received a screenshot from a parent. Her son had asked a free AI app about self-harm, and the reply included specific methods. The parent asked one thing: "Could the one the school uses do that too?" The teacher could not say at which step it would be stopped, and that afternoon she went to the IT team for a sample audit log — this lesson is written for that kind of question.

A guardrail is not a filter but a chain of checks running in both directions. This is the intermediate lesson in the safety module, following the privacy discussion in lesson 23; the next lesson covers attacks designed to get around guardrails.

---

The seven-second delay line

Live radio often uses a delay line: the presenter's words enter a buffer, and the producer has a few seconds to cut anything that should not go to air. AI guardrails do the same job, and both directions need the delay — the message the student sends, and the reply the model returns.

Watching only the exit and not the entrance is a producer who monitors the guest but not the phone-in lines. A student can type a classmate's phone number, their own identity card number, or "forget your earlier rules". Half the risk sits at the door, not at the exit.

Two common industry components

A school does not need to build guardrails, but at procurement it should know which layers the vendor stacks.

Llama Guard is a small model whose only job is review. It does not answer questions; it reads a conversation and decides whether it falls into a risk category — violence, self-harm, sexual content, hate, privacy, criminal instructions and so on — and it can check both the user's input and the model's output. Think of it as a producer's assistant who only classifies: the category list can be adjusted per organisation, and school settings normally run stricter thresholds than a consumer product.

NeMo Guardrails takes a different approach. It is not a model but a layer of rules wrapped around the application, defining where a conversation is allowed to go. For example: in student mode, do not discuss certain topics; when answering a school policy question, always consult the knowledge base first; in certain situations, always hand over to a human. It is the flow card in the producer's hand — deciding which line this call takes, not judging every sentence.

In practice the two are used together: NeMo sets the flow, Llama Guard sets the content. Neither is sufficient alone.

A third guardrail: lock the format

This is the most underrated of the three, and it needs no extra model.

When output must match a fixed structure — a rubric must have a criterion name, a maximum score and four level descriptors — software can validate each field before anything is displayed. Output that fails is caught and retried rather than handed straight to a teacher or student.

Why does this count as safety? Because it limits the AI's freedom. Free-form text is hard to check automatically; output with fixed fields can be checked by the dullest and most reliable means available — are all the fields present, are the values in range, do the marks add up. For the technique, see the prompt course lesson on structured output.

The value of a guardrail is not making the AI say "no" forever, but letting the handful of cases that need a person float to the surface while the system handles the rest.

Borderline cases from a stress test

Before switching on the student AI tutor, a primary school ran a stress test. The discipline and guidance team worked with the IT team, and three teachers played students, deliberately submitting 40 messages in four categories: clearly inappropriate, borderline distress, personal data, override instructions.

The most valuable finding was category two. A Primary 6 pupil saying "I have been very unhappy lately and do not want to come to school" and receiving a cold refusal is about the worst outcome available. The school adjusted the configuration: messages of this kind are not blocked, they are flagged to the class teacher immediately, and the reply includes routes to help.

The numbers were concrete. Of 40 messages: all 12 clearly inappropriate were blocked; 6 of 8 borderline cases were flagged correctly; all 10 personal-data cases were masked; all 10 override instructions were refused and logged. The remaining work is not chasing 100% but routing misjudgements through a teacher appeal channel — Chinese History on wartime casualties, Biology on the reproductive system, guidance conversations about distress can all be caught by a crude filter.

Human review is not a patch. It is part of the design: the system finds the cases a human should see; the human decides what to do.

Edor.ai's four layers

Edor.ai protects what students send and receive with four-layer gating:

1. Input gating. Before a message reaches the model, it is checked for inappropriate requests (stricter thresholds in student mode), personal data (masked before sending with a reminder to the student), and override instructions (refused and logged).
2. The system prompt. Every call injects school policy, the age level and subject context, scaffolded tutoring rules that give hints rather than answers, and a prohibition on requesting or emitting personal data. Administrators maintain it centrally, so teachers do not rewrite it each time.
3. Output review. Responses are classified for content safety as safe, needs attention or blocked. "Needs attention" content is masked in the sensitive part with a notice shown; "blocked" content is not displayed and is flagged to the administrator.
4. Audit and quotas. Every AI interaction is written to ai_audit_logs with the user, module, model, usage and safety action. Daily request and monthly usage quotas can be set per role.

Above those four layers, the teacher monitoring panel lets a class teacher view class conversations, pause an individual student and flag messages for follow-up after class. Flagged content enters a human review queue, where a person decides whether it was a misjudgement, needs follow-up, or should be handled under school rules. Enforced structured output applies to rubrics, tiered worksheets and draft circulars; missing fields trigger a retry.

On personal data, student fields are encrypted at rest in the database and only learning content is sent when calling the AI, while conversations and AI logs are purged on the retention schedule the school sets. The full architecture is in the student AI safety net.

Where people go wrong

• Asking at procurement only "do you have a content filter". A complete answer covers input, system prompt, output and records; a missing answer means a missing layer. Self-check: ask the vendor for one flow diagram showing the four steps between a student pressing send and text appearing on screen.
• Treating a flag as punishment. If students believe a flag equals a demerit, they learn to evade the system rather than ask for help. Self-check: does the handbook say plainly who reads a flagged message, whether parents are told, and whether conduct grades are affected?
• Conflating guardrails with injection attacks. Guardrails police content safety; jailbreaks and prompt injection route around instructions — see lesson 25. Self-check: does your security briefing separate "inappropriate content" from "forged instructions" as two threats?

Try it once

What to do: Work with two colleagues. Each writes five test messages (15 in total): three clearly inappropriate, three borderline emotional, three with fictional personal data, three override instructions, three normal teaching questions. Send them one by one on your school's AI platform (or a test environment) and record whether each was allowed, blocked, flagged or masked.

What to hand in: A 15-row log: summary of the test sentence, actual result, whether you think it was correct, and if it was a mis-block, one suggested threshold change.

How you know it worked: Clearly inappropriate and override messages should be blocked or refused 100% of the time. Borderline distress should not be coldly refused — it should receive a caring reply and a flag. Personal data should be masked. Normal teaching questions (such as "explain the discriminant of a quadratic") should not be blocked. If something was mis-blocked, you can name a concrete adjustment (for example, "whitelist the reproduction unit for Secondary 3 Biology") rather than only "loosen it a bit".