The rise of small language models, from Phi and Gemma to the 8B class

At a staff workshop a teacher asked a very direct question: "If the strongest model costs a few hundred dollars a month, why are we bothering with small ones? Shouldn't we use the best available?"

One question back makes it clear: would you engage a university professor to mark a Primary 3 dictation?

The primary generalist and the professor

An experienced primary generalist teacher knows the curriculum, the ability range of her pupils, how to talk to parents, and how things get done in your school. She will not publish in an international journal, but she handles nine tenths of daily school work quickly and accurately, and she is in the building whenever you need her.

A university professor has extraordinary depth in a specialist field and can tackle the hardest problems. But you must make an appointment, you must pay, and he does not know how your school works.

That is the relationship between large and small models. The question was never which is cleverer, but how clever this particular task needs someone to be.

A small model is not a cut-down version

A common misunderstanding is that small models are large ones with layers removed. The main reason small models have improved so much recently is the quality of the training material.

Take Microsoft's Phi family. Its design idea is to train on large volumes of carefully filtered and synthesised "textbook-like" material rather than a copy of the whole web. The result is a 14B parameter model that approaches or beats far larger rivals on mathematics and reasoning benchmarks. In education terms it is a pupil taught from excellent textbooks rather than one who read widely but indiscriminately.

Google's Gemma family takes a different route, turning the research behind the flagship Gemini models into downloadable open weights deliberately built to run on ordinary hardware. Recent versions accept text, image and audio input and cover more than 140 languages with a long context window.

The 7B to 9B models from Meta, Alibaba Qwen and Mistral are what schools most often deploy locally, because Chinese support and community resources are comparatively mature there. Current line-ups are in the model database.

One caution: every model card states its language support. The Phi family's card, for example, says explicitly that it targets English and is not designed for multilingual use, which is a significant warning for Chinese-medium school work. How to read those pages is in reading a model card and what GGUF and Safetensors mean.

Tasks where a small model is enough

From what we see in schools, the following are practically indistinguishable on an 8B class model:

• turning a transcript of a meeting into a list of decisions;
• converting a teacher's paragraph of instructions into a worksheet table;
• rewriting one comprehension passage at Primary 4, 5 and 6 difficulty;
• classifying a hundred open-ended survey answers from pupils;
• checking whether a circular has omitted the date, venue and reply deadline;
• building a vocabulary list from an English article with difficulty labels.

What they share is this: the answer comes mainly from the material you supplied, not from the model's memory.

When a flagship is still worth it

• Problems needing several steps of reasoning, such as a Secondary 5 mathematics proof or a full physics solution.
• Finding internal contradictions inside a long document.
• Outward-facing writing where tone matters enormously, such as replying to a complaint from a parent.
• Complex judgements that combine images and text.

What these need is depth of reasoning rather than breadth of knowledge, which is covered in multimodality, reasoning models and embodied AI.

An example from the classroom

A secondary school's Liberal Studies and careers team had a survey to process: nine hundred pupils, three open-ended questions each, about 2,700 pieces of text. Sorting that by hand would have taken one teacher a full week.

They worked in two layers. The first layer used an 8B model running on the school server to file each answer under further study, employment, emotional support, extracurricular activities or other, and to pull out keywords. That handled all 2,700 answers, and the pupils' own words never left the campus network for a second.

Only the second layer sent the de-identified statistical summary to a cloud flagship model to draft an analysis for senior management, because that report needed careful tone and inferences across categories.

The result: one teacher finished a week's work in two free periods, and the most sensitive raw text stayed in school throughout. Small models for volume, large models for judgement, is currently the most practical school design.

What this means for your classroom

1. Do not judge the capability you need by the price tag. Ask first whether the answer lies in the material you are supplying. If it does, a small model is usually enough.
2. Sensitive material should default to the in-school model. Compositions, counselling notes and SEN-related documents should be processed in school wherever that is possible.
3. Fix a small model's factual gaps with a knowledge base, not with fine-tuning. Many schools assume they must train the AI to know their school. The correct route is retrieval, covered in the school knowledge base and retrieval-augmented generation.
4. Speed is pedagogical value in itself. When you need three difficulty levels of an example sentence mid-lesson, a local small model usually answers faster than a cloud flagship, and a class cannot wait ten seconds.

A small experiment to run now

Give the same prompt to a small model and a flagship. The two will perform almost identically, because everything needed is in the material:

Below are five pupils' answers to "what extracurricular activity would you most like the school to add". Sort them into no more than three groups, name each group, and list the answer numbers in it.
Classify only from the answers I have supplied. Do not add your own ideas and do not evaluate the pupils.
1. A photography club, because I like taking pictures.
2. More ball game competitions.
3. I want to learn cooking.
4. A badminton team.
5. I want to learn video editing.

Now change the prompt to "analyse how adding these activities would affect Other Learning Experiences under the current Hong Kong senior secondary curriculum", and the gap appears immediately. That is the dividing line for how clever a task needs someone to be.

What comes next

That closes module 3. The next module moves into the core concepts of building applications, starting with the most basic and most underrated of them, prompt engineering in one lesson. To revisit model file formats, return to reading a model card and what GGUF and Safetensors mean. To see why small models are cheap to fine-tune, read LoRA and QLoRA on one GPU.