Context caching and cost control, or why asking about the same document again is cheaper

A panel head did something clever over the summer. She pasted an entire senior secondary curriculum document into an AI and then asked fourteen questions in a row: the core concept of each unit, which topics overlap with junior forms, suggested assessment approaches, current affairs that could be linked in, and so on.

Afterwards she asked us: "Do I have to paste that document again for every question? It feels wasteful."

Her question is the best available entry point into how AI costs actually work.

Leave the worksheet on the desk instead of reprinting it

Picture a group activity. Every group works from the same three pages of source material, with a different discussion question each.

The wasteful way: every time the question changes, reprint the three pages and hand them out with the new question. Thirty groups means ninety pages.

The sensible way: print the three pages once and leave them on the desk, then hand out only a slip with the new question.

Context caching does the second thing. When you send an AI several requests whose opening stretch is byte for byte identical, the provider can keep the result of processing that opening and reuse it, instead of computing it again. You pay for the slip of paper, not the ninety pages.

Why it is cheaper: it skips repeated work

Every time a model reads an input, it must turn each token into internal state, which is where a large share of the computation goes. It is also why longer inputs cost more.

The logic of caching is that this opening was processed a minute ago and the result is still there, so reuse it. Every major provider offers some version of this, and the discount is considerable, with cached input typically billed at a small fraction of the normal rate. Google's Gemini enables implicit caching by default on newer models, OpenAI enables prompt caching by default on supported models, and Anthropic lets developers mark explicitly where the cacheable section ends.

Schools do not need to remember which vendor does what. One practical rule covers it.

The rule: unchanging first, changing last

Caching compares from the beginning forwards. As long as the opening is identical, that part can be reused. Change anything in the opening and the match breaks, so it all has to be processed again.

So a prompt should be ordered like this:

1. First, the long material that is the same every time: the curriculum document, the full policy text, the marking rubric, the exemplar work.
2. Then the description of the task.
3. Last, the question unique to this request, plus the most important constraints and format requirements.

Sharp-eyed readers will notice this is exactly the order recommended in chunking, reranking and lost in the middle: long material first, crucial instruction last. Two entirely different reasons pointing at the same layout. One is about cost and one about accuracy, and they happen to reinforce each other, which is rare enough in practice to be worth remembering.

The most useful consequence for schools

Many teachers habitually open a new conversation for each new question and paste the material again. That is the worst option on both cost and speed.

The better habit is one set of material, several consecutive questions, all in the same conversation. This buys three things: caching applies, so it is faster and cheaper; the context is consistent, so answers do not contradict each other; and you can follow up with "quote the original text for your third point".

There is one opposite case to watch. If a conversation has become long and the topic has drifted, start again. A long conversation accumulates irrelevant material, which is exactly the high-risk zone from the previous lesson. The rule is: consecutive follow-ups on one document stay in one conversation, and a change of topic starts a new one.

An example from the classroom

A secondary English panel needed a question bank of reading materials for Secondary 4. The source was six articles, each requiring four question types: vocabulary, comprehension, inference and extended writing.

The first teacher opened twenty-four new conversations, pasting one article and one question type each time. It took her an afternoon, and she then found the question styles differed between conversations and had to be harmonised afterwards.

The second teacher opened one conversation per article, pasting the article and the panel's question-writing conventions first (both identical every time), then asking for all four question types in that same conversation. Six conversations covered the whole job, noticeably faster, and the four question types for each article came out stylistically consistent because they shared the same context.

The interesting part was her third step. She saved that fixed opening, the article plus the conventions, as a panel template in the shared folder. The next teacher only replaces the article. At that point caching has stopped being a technical mechanism and become a panel working method.

What this means for your classroom

1. Ask all your questions about one document in one sitting. It saves the most time and money and produces the most consistent style.
2. Turn fixed conventions into a template that goes first. Question-writing formats, marking rubrics and vocabulary conventions are identical every time and deserve to be saved as a reusable opening block.
3. Never put anything variable before the document. One line reading "today is 11 September" at the top is enough to void the cache for the whole document. If you need the date, put it last.
4. Understanding cost structure is what lets you read a quotation. Where a vendor charges by token, twenty questions on one long document costs vastly more than one, which is precisely the point of a fixed annual fee. See pricing.
5. Cost must never override privacy. Cheaper is not the same as appropriate. Where student personal data is processed is decided by security and privacy first and price second.

An ordering exercise to try now

Same material, two layouts. You will feel the difference in response speed:

(Layout A, recommended)
Below is the full teaching material for Primary 5 Chinese unit 3 at our school.
<<<
[paste the material, long passages here]
>>>
Task: set questions based on the material above.
This request: write three inference questions.
Final requirements: set questions only from the material and introduce nothing beyond it; add one marking point per question, under 25 words.

(Layout B, not recommended)
Write three inference questions, only from the material, with a marking point each. Today is 11 September.
Here is the full material:
<<<
[paste the material]
>>>

In layout A the fixed part comes first and only the last two lines change between requests, so the cache hits and the most important constraint lands where attention is highest. In layout B something changes at the very top every time, so the cache almost never hits, and the constraints are buried in the opening.

What comes next

So far the AI has only been reading what you hand it. The next lesson takes on a fundamentally different question: what if it could look things up, calculate, and take actions itself? See from chatbot to AI agent, with tool calling and MCP. To revisit why Chinese uses more tokens, see tokenisation and why Chinese costs more. To compare real provider pricing, see the model database.