Open Weights or Closed API? Real School Maths on Self-Hosting, and When Privacy Flips the Answer

On a Wednesday after school, an ICT panel head slides a GPU quote across the staff-room table: "Buy a machine, run open weights, stop paying API fees." We run his school's weekday usage through September 2026 published pricing. The annual cloud API bill lands between US$99 and US$556 — roughly HK$800 to HK$4,300. He goes quiet for two seconds. "So before we even switch it on, the hardware has already outspent what we meant to save."

For a typical Hong Kong secondary school in 2026, the open-versus-closed argument is no longer mainly about which model is cleverer. It is about where data may live, and whether the school can carry the maintenance and safety work that follows.

Weights, renting and where data sits

Open weights mean the parameters can be downloaded and inference runs on the school's own hardware. A closed API opens only the window: questions in, answers out, weights never leave the vendor's data centre. Many 2026 models use mixture-of-experors design — only a fraction of parameters activate per token, which keeps cloud per-token pricing low. Schools self-hosting must remember something different: VRAM still has to hold the full weight file, not just the active slice. That miscalculation ruins hardware budgets every term.

Self-hosting buys location inside the campus network. It does not buy a contemporary flagship's reasoning, fewer hallucinations, the vendor's content-safety classifiers, or someone to call when the box falls over at 10 p.m.

Open models that actually fit a server room

The consumer-GPU circle

Near the 24GB VRAM line, these models are realistic for an ordinary school: Qwen3.8-27B (27.8B dense vision-language, Apache 2.0, 1M context, documented for 24GB); Muse Glimmer 30B (30B dense multimodal, Apache 2.0, 131k context, official Ollama / LM Studio / llama.cpp support); Ministral 3 at 3B, 8B and 14B (Apache 2.0, single-GPU or even laptop); qwen2.5:14b on Edor.ai's Ollama suggestions; llama3.1 as the on-premises default.

Server-class territory

Mistral Large 3 (675B total / 41B active MoE, Apache 2.0, 256k context) needs server hardware. DeepSeek-V4.1-Flash (552B MoE, MIT, 1M context) is similar unless you use a distilled or quantised build. GLM-5.2 (744B MoE, MIT) and Hunyuan Hy4 Preview (770B total / 49B active, Apache 2.0, 1M context) have clean licences and high bars. Kimi K3 (2.8T total / 104B active, bespoke licence, 1M context) is not a school-server conversation.

The practical line: sub-30B lands in a server room; above that you are in data-centre territory. Licences must be read model by model — Qwen's open flagship uses a bespoke licence without image input, and GLM-5.3 flagship weights were still unpublished when this article was updated.

Renting the bill versus self-hosting the hidden costs

Assumptions stated plainly: 800 students and 100 staff; 600 AI requests per school day; 3,000 input and 700 output tokens per request; 190 school days per year. These are estimating assumptions, not vendor quotes.

At September 2026 published rates, renting runs roughly: flagship GPT-6 Astra / Claude Fable 5.1 about US$7,410 a year; mid-tier Claude Sonnet 5 about US$1,482; value Gemini 3.8 Flash (introductory) about US$556; cheapest DeepSeek-V4.1-Flash (off-peak) about US$99. Self-hosting Qwen3.8-27B or Muse Glimmer 30B has zero token fees but four real costs — hardware, power and facilities, IT time for install / update / backup / incidents, and the opportunity cost of a weaker model teachers notice in daily use. Add them up against those rental figures and, at typical secondary volumes, hardware rarely pays back on saved API fees alone.

With Edor.ai the school pays one fixed annual fee covering roughly 800 students and 100 staff, AI usage included with no token charges, and cloud and on-premises deployments share the same features and price structure — so procurement should return to "where does the data live", not "spend or save".

The moment data policy flips the answer

Money often points to renting. Written data policy can reverse the answer overnight. Self-hosting is reasonable or necessary when: a sponsoring body or special school rule says data must not leave the campus network; the material is in the most sensitive category — SEN assessments, counselling cases, social-work referrals, suspected abuse — and the school will not put it through a third party even with contract protection; or the school needs a sentence parents and the management committee can understand — "on that machine in the server room" beats a contractual summary.

Under the six Data Protection Principles of Hong Kong's Personal Data (Privacy) Ordinance, the school remains the data user throughout, and the security obligation under DPP4 does not transfer through outsourcing. Self-hosting does not remove that duty; it narrows it to systems the school controls. Many schools route sensitive tasks locally and everyday planning and tutoring to the cloud.

Six real costs behind self-hosting

Self-hosted models hallucinate too, usually more than a contemporary flagship — teacher review stays mandatory. Commercial APIs sit behind content classifiers, abuse detection and continuously updated safety policy; self-hosting hands that back to the school, and pupil-facing use needs its own gating and monitoring — see student AI safety, gating, PII and monitoring. Speech recognition and synthesis for oral practice still largely come from OpenAI or Azure OpenAI; cutting the cloud off entirely means giving up some features. INT4 / INT8 quantisation runs on a small GPU at the cost of precision — test exact-calculation tasks on your own material first. In April 2026 Meta moved its frontier line from Llama to the closed-weight Muse Spark, making Llama legacy — a three-year plan cannot assume today's open weights stay open; confirm already-downloaded weights remain usable. LoRA / QLoRA fine-tuning teaches tone and format, not reliable facts; for curriculum schedules and school policies, use school knowledge-base retrieval, not fine-tuning alone.

School decision path and checks

Answer in order: Is there a written rule that data must not leave the campus network? Yes → on-premises, skip the cost comparison. Is there staff who will maintain servers long term? No → cloud, with data protections in contract. Both possible → route sensitive and everyday tasks; for a school with IT capacity that is usually the best position. Does the contract guarantee identical cloud and on-premises features, one-click full export, and future switchability?

Checks you can run this week: a machine with 24GB or more VRAM that can stay powered on; trial qwen2.5:14b or llama3.1 and ask whether Chinese output meets the panel standard; list tasks that must stay local and write them into the school AI policy; confirm gating, teacher monitoring and audit logging are configured; verify whether the chosen model is MIT, Apache 2.0 or bespoke and whether commercial use is restricted. For deployment and ownership, see on-premises versus cloud deployment; for model reference, see the model database and the 2026 all-model roundup.

Common misreadings

• The claim: Open source is free, so self-hosting must be cheaper. What actually happens: Weights may download at no charge, but hardware, power, IT time and capability gap together often exceed a US$99–556 annual cloud API bill at ordinary school volumes. The honest reason to self-host is usually data location, not a zero token line on the invoice.

• The claim: MoE models only activate part of the parameters, so self-hosting must be light. What actually happens: Inference activates a slice, but loading weights still requires VRAM for the full parameter file. A model such as Kimi K3 at 2.8T total parameters is not solved by buying a consumer GPU.

• The claim: A few weeks of fine-tuning makes the model "know our school". What actually happens: LoRA / QLoRA adjusts vocabulary and format, not reliable storage of school facts. Schemes of work, rubrics and notice formats belong in a knowledge base with retrieval and teacher review.

Three things to try on Monday

1. Take last month's AI request count, compare it to the assumption here (600 per school day), estimate annual tokens, then look up September 2026 official pricing for DeepSeek-V4.1-Flash and Gemini 3.8 Flash — put both numbers in front of the management committee as the renting baseline.
2. Ask IT whether the server room has a host with 24GB VRAM that can run continuously; if not, take self-hosting off the agenda until the premise is true.
3. Spend fifteen minutes each with counselling and SEN leads listing data types that must never pass through a third-party cloud; a list becomes routing rules and concrete contract language.