Multimodal prompts: transcribe first, judge second

At break on Wednesday, Mr Lam photographed a Secondary 1 visual arts shading assessment sheet and asked the AI to score composition, tone and proportion out of five each. The reply said too much empty space on the left and an unfinished corner bottom-right. On the paper, the gap was on the right and the unfinished area top-left. The AI had flipped the layout yet wrote a confident comment. "If I had not checked the image," he said, "this would have gone to the parent."

Multimodality is not a different model. It is the same "most plausible continuation" logic applied to images and sound — and when something is unclear, the model still guesses. This is lesson 16 in module four; fixed fields are in the previous lesson, a template library in the next.

---

Why one step is especially risky

Ask for marking or analysis in one step and a recognition error arrives dressed in reasonable commentary, and the teacher cannot see which step failed. Reading a 7 as a 1 in maths or mirroring left and right in art — same pattern: the input was wrong but looks like bad judgement.

The core design is one sentence: have the AI state what it saw or heard, let the teacher confirm, then mark or feed back. For the mechanism see where hallucination and bias come from and multimodality, reasoning models and embodied AI.

Two steps for handwritten work

Step one: transcribe only

Role: you transcribe handwritten work. You are not marking.
Task: copy every line the student wrote in the image.
Output: line-by-line transcription, plus a "low recognition confidence" section listing uncertain words and line numbers.
Constraints: transcribe only — no calculation, no judgement, no extra steps. Write "illegible" rather than guessing. Transcribe incorrect characters exactly as written, never auto-correct.

The teacher spends thirty to sixty seconds checking against the book, then passes confirmed text to step two.

Step two: mark from confirmed text only

Role: you are a [subject] marking assistant.
Task: from the checked text below, identify where the work goes wrong.
Output: for each item — item number, where the problem is, error type (conceptual / calculation / copying / cannot determine), one hint for the student (no final answer).
Constraints: do not reinterpret the image. Where you cannot tell, set error type to "cannot determine" and write "teacher to check the original book".
Checked text: [paste]

Why handwritten Chinese is weakest

Printed English is best, printed Chinese next, handwritten English after that, and handwritten Chinese weakest — more strokes, more similar characters, corrections and writing outside the grid. The model will not volunteer uncertainty, so "low recognition confidence" must sit in the output specification.

The expensive part of multimodal work is not the API call. It is the minute checking against the original. Skip that minute and you often spend a lesson repairing trust.

Graphs and diagrams

Most used in Science and General Studies. Four parts in the output: "read from the image" with positions, "inferred" with basis, "not legible", and three classroom questions — do not answer for students. Never report a familiar trend or textbook conclusion as data on this image; when unsure, give a range and write "teacher to verify".

Reading aloud and oral practice

Compare a recording to the passage. Output "differences heard", "low recognition confidence" and "three things to practise". No scores, grades or percentages; do not comment on accent itself, only departures from the text; tone encouraging and specific. Edor.ai oral practice is positioned as practice; formal assessment stays with the teacher. For timetabling see AI oral practice; for marking workflow see AI marking and instant feedback.

Privacy and platform

Cover or crop name, class and student number before shooting. Do not upload pupil work to consumer tools with no no-training terms, no encryption and no ai_audit_logs. Work inside the school platform, or de-identify first.

Where this goes wrong

• One step: "mark this page for me". A misread becomes a wrong analysis. Self-check: is there a separate transcription step?
• Oral feedback with a percentage for parents. The same recording can differ on a rerun. Self-check: do the constraints forbid scores and grades?
• Silent correction when transcribing compositions. Error statistics and writing assessment skew. Self-check: are wrong characters required verbatim?

Try it once

Time: 10–15 minutes.

Do this: Photograph one anonymised handwritten page with identity fields covered. Run once asking for marking. Run the transcribe-only template, check the transcription yourself, then run mark-from-checked-text.

Deliver: Both final comment sets. List items flagged under low recognition confidence in step one, and which comment in the one-step run would have been wrong if you had skipped checking.

Done when: You corrected at least one line in the transcription, step two quotes only checked text, and it does not state unconfirmed visual details as fact.