Debug prompts with six ordered checks
When output fails, do not rewrite the whole prompt. Work through task clarity, format, context length, buried instructions, example conflicts and model fit — one change at a time.
以繁體中文閱讀On Tuesday after school, Mr Ng was on his fourth rewrite of a Primary 5 maths word-problem differentiation prompt. Run one listed final answers. Run two dropped the SEN adaptation note. Run three added a table the spreadsheet could not import. Run four pasted the entire term overview. "Every fix broke something else," he told the ICT coordinator. She handed him the six-check order sheet and said: change output format only, same class data, run three times. Run five: columns aligned — tone and SEN notes became a short queue of single fixes.
Debugging is not another draft. It is marking: find which step failed, fix that step only. This is lesson 18, closing module four of the prompt-engineering course. Earlier lessons on the syllabus each target one skill; this lesson puts them in a fixed order so you stop guessing.
Change one thing at a time
Changing role, task, constraints and examples together means you never learn which line mattered — the standard way debugging fails. Fixed habit: change one item → rerun the same input → note the difference. Run three times before concluding; treat it as a prompt problem only when all three fail. One run in three drifting is stability — tighten format and examples rather than rewriting the task from scratch.
First two checks
Check 1 — task, audience and purpose
Signs: broadly the right area but emphasis shifts between runs, or it reads for the wrong reader — pupils, parents or panel head.
Self-check: read the task line to an uninformed colleague and ask what they would hand you. If they must ask who it is for and what it is for, the model needed those answers too. Common gaps: audience, purpose (discussion / assessment / filing), scope (whole unit or one lesson).
You can use a diagnostic prompt to list ambiguity and one resolving sentence — but do not rewrite the whole prompt in this step.
Check 2 — output format nailed down
Signs: content usable, structure different every run, tidying by hand.
Self-check: are column names, order, permitted values and "output the table or JSON only, no text before or after" all stated? "Explain in detail" is not a format. For merging or import see structured output as tables and JSON.
Mr Ng's run three failed check 2: the model invented column headers each time because the prompt never fixed them. Naming five columns and forbidding extras fixed import before he touched tone or SEN wording.
Checks three and four
Check 3 — context too long
Signs: you pasted a lot and the output reflects only the start and end, or earlier constraints are ignored.
Self-check: whole policy, a term's materials, twenty pieces of work at once — all too long. Fix: paste relevant passages only, move critical constraints to the end, retrieve instead of paste (grounded prompts with citations). Mechanism: chunking, reranking and lost in the middle.
Context too long
Run four pasted twelve weeks of scheme-of-work notes. The model remembered the first learning objective and the last assessment date but dropped the SEN line in the middle. Trimming to the current unit's three objectives removed the drift without touching the task sentence.
Check 4 — critical requirements buried
Signs: you definitely wrote a constraint and the output simply omits it.
Self-check: where does that sentence sit? Wedged in long background, it is easily overlooked. Fix: extract three to five numbered constraints at the very end and require confirmation in the output. A structure review prompt can list high-risk positions.
Buried instructions
The SEN note sat on line eleven between class context and differentiation rules. Moving it to "Constraint 2" at the bottom with a confirmation line in the output spec made omissions visible on the next run.
Earlier problems disguise themselves as later ones — with an ambiguous task, a perfect format only arranges the wrong content neatly.
Checks five and six
Check 5 — examples fighting constraints
Signs: examples added and output got less stable.
Self-check: a twenty-word limit with a forty-word example; no answers allowed with an answer in the sample; two examples with different column orders — the model usually follows the example. See few-shot examples and output consistency.
Example conflicts
Run one included a sample problem with the worked solution visible, while the constraints forbade giving answers. Removing the solution from the example stabilised hints faster than adding another prohibition.
Check 6 — model fit for step count
Signs: the first five checks pass, multi-step reasoning still skips or miscalculates.
Self-check: writing a circular is single-step; five criteria each quoting pupil text then a total is multi-step. The latter may benefit from a stronger reasoning tier; format, register and ignored requirements gain nothing from swapping model. Tier differences: model database, OpenAI, Anthropic Claude, Google Gemini.
Model mismatch
Differentiating three parallel word problems with reading-level variants is mostly format and clarity — check 6 rarely helps. Asking the model to derive three new problems from a textbook chapter while matching each to a specific LO code is multi-step; if checks 1–5 pass and working still jumps, try a stronger reasoning tier.
All six pass
Three common cases remain: needs school documents (knowledge base, not more constraints); needs human judgement (individual pupils, home-school relations, conduct); asks for too many deliverables at once (split the prompts). For marking, also ask: does output quote the pupil's own words? For SEN-related work: any diagnostic or labelling language?
Places that go wrong
- Changing several things at once, or judging on one run. Self-check: one change only? Three runs on the same input?
- Adding words without deleting redundant background. Triggers checks 3 and 4. Self-check: can you remove paragraphs that do not affect output?
- Skipping check 1 and fixing format. Self-check: does the task line state audience, purpose and scope?
Try it once
Time: 10–15 minutes.
Do this: Take a recent unsatisfactory prompt and its input. Tick every check that might apply, in order. Change only the earliest ticked item, then rerun the same input three times.
Deliver: Before and after prompts, a one-line summary of each of the three runs, and one sentence on which check you changed and whether three runs improved consistently.
Done when: You can explain to a colleague why you tackled that check first rather than another; if all six are ruled out, you can say whether the next step is knowledge base, split tasks or human judgement — not a seventh full rewrite.
What to take from this lesson
- When output fails, change one element at a time and rerun on the same input instead of rewriting the whole prompt.
- Work six checks in order — ambiguous task, unspecified format, over-long context, buried instruction, inconsistent examples, wrong model.
- Run the same input three times to separate prompt problems from random variation.
- When all six pass, the task may need school documents or human judgement rather than more prompt edits.
Questions that keep coming up
You skipped the core habit — one change, same input, three runs. Roll back to the last version that was closest, tick the six checks in order, and fix only the earliest item that still fails.
Usually check 4 (buried instruction) if the constraint sits in the middle, or check 3 (over-long context) if you pasted whole policies. Move three to five numbered constraints to the very end before you add more words elsewhere.
Only after checks 1–5 pass. Long worked solutions, cross-document comparison and rubrics that quote pupil text then total scores may need a stronger reasoning tier. Wrong columns, register and ignored limits will not improve by swapping model alone.