Context caching and cost control: the same slides do not need reloading every time

In the last week of the summer holiday a senior secondary history panel head pasted an entire new-year curriculum document into an AI and planned to ask, unit by unit, about core concepts, overlap with junior forms, suggested assessment and linkable current affairs. On the sixth question she paused: "Do I have to paste this PDF again every time? It feels wasteful." That question is the best entry point into how AI costs actually work.

Retrieval and chunking taught you to feed less. This lesson teaches you to compute less: why repeat questions on the same long material get cheaper and faster, and how prompt order supports that.

---

Re-reading the same material is expensive

Every time a model processes an input it must turn tokens into internal state. That step consumes a large share of computation and is why longer input means a higher bill (Chinese uses more tokens; see tokenisation and why Chinese costs more).

If you ask fourteen questions about the same three-thousand-word curriculum document and paste the full PDF at the start every time, you ask the system to compute the same opening fourteen times. Providers therefore offer context caching, or prompt caching: when the opening is byte for byte identical, the processed result is stored briefly and reused, and you mostly pay for the new lines at the end.

Slides already up, change only the question

Picture a group task in class. Three pages of reading material stay projected. Each group gets a different discussion question.

The wasteful way: every time the question changes, reload the entire deck from slide one through three, then hand out the new question. Six groups, six questions, eighteen full play-throughs.

The sensible way: leave the three pages on screen. Hand out only a slip with the new question.

Caching does the second thing. Google's Gemini enables implicit caching on newer models by default, OpenAI enables prompt caching by default on supported models, and Anthropic lets developers mark where the cacheable section ends. Schools do not need vendor trivia. One layout rule covers it.

Caching cares whether the opening is identical. Change one character at the top and everything from that character onward must be recomputed.

Unchanging first, changing last

Caching compares from the first line forwards. An identical opening can be reused. Change anything in the opening and the match breaks from that point.

Recommended order, the same as lost in the middle recommends for accuracy, one reason about cost and one about attention:

1. First: the long stretch that is identical every time, curriculum document, full policy text, panel question-writing conventions, exemplar article.
2. Middle: task description, if it is also fixed.
3. Last: the question unique to this request, plus the most important off-limits and format rules.

Never put "today is 12 September" before the document. One date line can void the cache for the whole file. If you need the date, put it last.

Six conversations for a reading bank

A secondary English panel needed a Secondary 4 reading question bank: six articles, four question types each, vocabulary, comprehension, inference and extended writing.

Teacher A opened twenty-four new conversations, one article and one question type each. It took an afternoon and the styles had to be harmonised afterwards. Teacher B used one conversation per article: paste "article plus panel conventions" as the fixed opening, then ask for all four question types in that same thread. Six conversations covered the job, noticeably faster, and the four types for each article matched because they shared context. She saved the fixed opening as a panel template. Caching had become a working method, not just a technical detail.

Annual subscription versus token billing

Where a vendor charges by token, twenty questions on one long document can cost vastly more than one. That is precisely why a fixed annual school fee matters. Edor.ai charges per school per year (roughly 800 students and 100 staff) with AI usage included and model price rises absorbed by the platform. Deployment can align with Hong Kong EDB "AI for Education" funding of up to HK$500,000 per school. See pricing.

Even on an annual fee it is worth understanding caching: it explains response speed, prompt order, and why consecutive questions on one document beat a fresh conversation per question.

One extra rule: consecutive follow-ups on one document stay in one conversation. When the topic has drifted or the thread is very long, start again, so irrelevant text does not build a new middle amnesia zone.

Where people go wrong

• The drift: opening a new conversation and re-pasting the full text for every question, then complaining the AI is slow and expensive. Self-check: from the second question onward, are you only changing the last few lines? Is the fixed opening byte for byte identical?

• The drift: adding greetings, dates or "please and thank you" before the fixed material. Self-check: move them to the final line and compare response time or token use if your tool shows it.

• The drift: pasting pupil personal data into a free tool to chase cache hits and save tokens. Self-check: is the data inside a gated platform? Cheaper is not the same as appropriate. See security and privacy first.

Try it once

Time: about 10 minutes. Materials: any AI tool; one passage of roughly 400 words of teaching material or policy you can reuse.

What to do: with layout A, ask two different questions in the same conversation, changing only the last two lines. Close the thread. With layout B, ask the same two questions in new conversations each time, with the question before the material.

Layout A (recommended):

Below is the full teaching material for Secondary 5 History unit 2 at our school.
<<<
[paste material]
>>>
Task: set questions based on the material above.
This request: write two cause-and-effect questions.
Final requirements: set questions only from the material; add one marking point per question, under 25 words.

Layout B (not recommended): put "write two cause-and-effect questions" and the date before the material, and start a new conversation for each question.

What to hand in: notes on how the second question felt under each layout (speed, whether style continued from the first question), and whether you only edited the end of the prompt.

How you know it worked: you can explain why caching wants fixed material first and changing material last, and link that to putting off-limits rules last for lost in the middle, two reasons, one layout.

Next lesson: from chatbot to AI agent. Previous lesson: chunking and reranking.