A prompt debugging checklist

The problem: seventh rewrite, output still wrong

A Secondary 5 Citizenship and Social Development teacher needed a source comparison table. The first attempt had the wrong columns, so she rewrote it. The second was too academic in register, so she rewrote it again. By the seventh she could no longer remember what the original had said, and the output was still unusable.

She had not written badly. The trouble was that every attempt changed three or four things at once — a new role, a reworded task, two added constraints, one example deleted. Whether the output got better or worse, she had no idea which change did it. This is the standard way prompt debugging fails.

Treat it like marking instead: find which step went wrong, then fix that step. This lesson is an ordered checklist. The order matters, because earlier problems disguise themselves as later ones. An ambiguous task looks like a formatting problem; an over-long context looks like the model not being clever enough.

One principle runs through everything: change one thing, then rerun with the same input. That is the difference between debugging and guessing.

Why this works

A model does not misunderstand you. It computes the most likely continuation of the text you gave it. So when the output is wrong there is almost always an identifiable gap in the input: something you thought you said but did not, something you said in the wrong place, or something you said twice in two conflicting ways.

Picture a relief teacher working from the notes you left, and the lesson going wrong. You would not rewrite the whole set of notes. You would ask — did I fail to say which page? Did I say it but bury it in the third paragraph? Does the example I attached actually contradict what I asked for? Debugging is those questions, asked in a fixed order.

Check 1: is the task ambiguous

Symptom: the output is broadly in the right area but not the thing you wanted, and the emphasis shifts between runs.

How to check: read your task sentence to a colleague unfamiliar with the work and ask what they would hand you. If they have to ask you a question first, the model needed that answer too.

The three most common gaps are no audience (for students or for parents), no purpose (class discussion or assessment), and no scope (a whole unit or one lesson).

Role: you are a prompt diagnostic assistant.
Task: examine the prompt below and identify anything ambiguous or open to more than one reading.
Output format: a three-column table — the ambiguous text (quoted verbatim), two possible readings, and one sentence that would resolve it. Below the table, one line naming what to clarify first.
Constraints: do not rewrite the prompt; diagnose only. Do not introduce teaching requirements the original did not have. Do not list as ambiguous anything the prompt already settles.
Prompt:
[paste your prompt]

Check 2: is the output format specified

Symptom: the content is broadly usable but the structure changes every run and needs tidying by hand.

How to check: does your prompt contain a sentence that states what the output is? "Explain in detail" is not a format. "Three points of one sentence each, plus a table" is.

The full treatment is in structured output as tables and JSON — name every column, close the list of permitted values, forbid additions. Any time output will be merged or filed, this check cannot be skipped.

Check 3: is the context too long

Symptom: you pasted a lot of material and the output reflects only the beginning and the end, or the model starts ignoring constraints you set earlier.

How to check: count what you pasted. A thirty-page policy in full, a term's teaching materials, twenty pieces of student work in one go — all too long.

Three fixes, easiest first: paste only the relevant passages, move the most critical constraints to the end, and retrieve rather than paste (see grounded prompts with citations). The mechanism behind the neglected middle is in chunking, reranking and lost in the middle.

Check 4: is the instruction buried

Symptom: you definitely wrote a requirement and the output simply does not contain it.

How to check: where in the prompt does that sentence sit? Wedged in the middle of a long background passage, its chances of being overlooked rise sharply.

Fix: pull the three to five most critical constraints out, number them, put them at the very end, and add a line to the output specification asking the model to confirm each.

Role: you are a prompt structure review assistant.
Task: examine the prompt below and identify which requirements are positioned where they may be overlooked.
Output:
1. "Requirements list" — every actionable requirement extracted individually, with its position in the prompt (opening / middle / end).
2. "High risk" — the requirements sitting in the middle or surrounded by background material.
3. "Suggested reordering" — list the order only; do not rewrite the content.
Constraints: do not add, remove or reword any requirement; extract and reorder only. Any sentence you cannot classify as a requirement goes in a list marked "teacher to judge".
Prompt:
[paste your prompt]

Check 5: do the examples contradict each other

Symptom: you added examples and the output became less consistent, not more.

How to check: hold each example against your constraints line. The usual contradictions are a constraint saying twenty words maximum next to an example running to forty; a constraint forbidding answers next to a sample reply containing one; and two examples using different column orders.

Examples carry a lot of weight. When an example fights a constraint, the model usually follows the example. So examples must be samples of the output you actually want, not throwaway illustrations. See few-shot examples and output consistency.

Role: you are a consistency checker for prompt examples.
Task: compare the constraints line of the prompt below against the examples attached to it and find where the two disagree.
Output format: a four-column table — the requirement in the constraints line, the corresponding part of the example, the verdict (consistent / inconsistent / not covered by the example), and whether to change the example or the constraint. Below the table, one line naming the disagreement most likely to be causing unstable output.
Constraints: compare only the prompt text and the examples; do not run the prompt or speculate about what the model would produce. Do not rewrite the examples — identify the difference and suggest a direction. Where the examples contradict each other, such as using different column orders, list those in a separate section. Mark anything you cannot judge as "teacher to decide".
Prompt and examples:
[paste your prompt together with all its examples]

Check 6: is the model wrong for the task

Symptom: the first five checks pass, the prompt is precise, and the output still skips steps or miscalculates partway through multi-step reasoning.

How to check: how many reasoning steps does this task need? A five-criterion rubric, each criterion quoting the student's own text, then a total — that is multi-step. Writing a circular is single-step.

Single-step work runs perfectly well on a cheap, fast tier and gains nothing from a flagship. Multi-step reasoning is worth escalating. But note: wrong format, wrong register and ignored requirements will not improve at all with a different model. Those are prompt problems. For how the tiers differ see the model database and the pages for OpenAI, Anthropic Claude and Google Gemini.

All six check out and it is still wrong

At that point the problem is usually not the prompt but the task. Three common cases:

• The task needs the school's own documents. You are asking about your policy, your scheme of work or your format, and the model has never read them. That needs a knowledge base, not a prompt.
• The task needs human judgement. Anything touching an individual student's circumstances, home-school relations or conduct — AI can organise the material but should not draw the conclusion.
• The task asks for too much at once. One prompt wanting a lesson plan, questions, a worksheet and a rubric should be four prompts, which is far more effective than rewriting one.

Common mistakes

1. Changing several things at once. Even when it improves, you do not know which line did it, so the next similar problem starts from scratch. Fix: one change, then rerun with the same input.
2. Judging on a single run. Models are probabilistic. Fix: run the same input three times before concluding, and treat it as a prompt problem only when all three fail.
3. Solving everything by adding words. The prompt grows until it triggers checks 3 and 4. Fix: delete before you add — removing repetition and background that does not affect the output often works better than another requirement.
4. Skipping check 1 and going straight to format. With an ambiguous task, a perfect format specification only arranges the wrong thing neatly. Fix: keep to the order.

Going further

• Panel debugging together: turn the six checks into a sheet and run every template through it at co-planning. It saves far more time than each teacher experimenting alone, and the conclusions go straight into the template's known limitations field — see building a panel prompt template library.
• A student version: Secondary 5 Information Technology can turn checks 1 and 5 into a classroom activity, finding three readings of one ambiguous prompt.
• An extra check for marking prompts: does the output quote the student's own text as evidence? Comments with no quotation usually mean the model is marking on impression.
• An extra check for SEN-related prompts: does the output contain any diagnostic or labelling language? Treat this as the highest priority check of all.
• Primary subjects: pay particular attention to the audience field in check 1. The vocabulary gap between Primary 4 and Primary 6 is wide, and a vague year level is the single most common cause of failure in primary prompts.

Course conclusion

Eighteen lessons in, the core comes down to a few things. Write the role, the task, the output and the constraints clearly. Use examples to make output stable. Where facts are involved, require citations and verification. Where student work is involved, forbid rewriting. Then keep the prompts that work with the school rather than with an individual.

The syllabus will take you back to any lesson. To go one layer deeper and understand why models behave as they do — why they hallucinate, why they lose earlier context, how guardrails work — go to the LLM Classroom. The two courses are two sides of the same set of techniques. To see how all of this runs inside a school platform, see platform features and security and privacy.