Open Weights or Closed API? The Real Cost Maths of Self-Hosting, and Where Privacy Flips the Answer

An IT coordinator asked us: "If we buy a machine and run an open model, we stop paying API fees, right?" We put his school's actual usage through September 2026 published pricing. The annual cloud API cost came to somewhere between US$99 and US$556 — roughly HK$800 to HK$4,300. In other words, before the machine is even switched on, the hardware has already outspent what it was meant to save. Self-hosting may still be the right decision. Saving money is just not the reason.

This article works through both sets of numbers, then identifies the point at which privacy and data policy reverse the answer.

Plain-language mechanics: weights, renting, and where the model lives

Open weights means publishing the recipe. A model's weights are the several hundred billion trained numbers that determine how it answers. Publish them and anyone can download and run the model on their own machine. Closed means only the window is open: you send a question, you get an answer, and you never see the recipe.

Parameter count is the brain-cell count, but in 2026 look at active parameters. Most new models use a mixture-of-experts design: a model may hold 744B parameters and activate only a fraction of them per token. That is how cloud vendors push prices down, but it is bad news for a school self-hosting, because the VRAM still has to hold all the parameters, not just the active ones. This is the single most common miscalculation we see.

What self-hosting actually buys is location, not capability. Data stays inside the campus network and passes through no third party. What it does not buy: a cleverer model, fewer hallucinations, the cloud vendor's content-safety classifiers, or someone to call when it breaks.

Head-to-head: which open models are realistic for a school?

One sentence summarises that table: what genuinely lands in a school server room is the sub-30B group; above that it is a data-centre conversation. Licences also have to be read model by model rather than vendor by vendor — the open build of Qwen's flagship uses a bespoke licence without image input, and the GLM-5.3 flagship weights were still unpublished when this article was updated.

The real cost maths, both sides on one page

Start with renting. The conversion below uses clearly stated assumptions: 800 students and 100 staff; 600 AI requests per school day; 3,000 input and 700 output tokens per request; 190 school days per year. These are estimating assumptions, not vendor figures.

The self-hosting side has no token fees but four costs, two of them non-cash:

1. Hardware — a 24GB-class GPU and a machine that can stay powered on;
2. Power and facilities — continuous running, cooling, uninterruptible power;
3. IT staff time — installation, updates, backups, incident handling. This is the item most often omitted and usually the most expensive;
4. Opportunity cost — a self-hosted model is generally weaker than a contemporary flagship, and teachers notice the difference.

Add those four up, set them against the right-hand column above, and the conclusion is fairly stark: at typical secondary school volumes, the annual cost of renting a mid-tier or value model is too low for hardware to pay back on saved API fees alone. For self-hosting to win on pure cost you usually need one of: usage far above the assumptions here (a district-wide shared deployment, say), spare server capacity the school already owns, or a sponsoring body with a central data centre across which the cost can be spread.

One point matters particularly for schools: with Edor.ai the school pays a single annual subscription covering roughly 800 students and 100 staff, with AI usage included and no token fees, and cloud and on-premises deployments have identical features and the same price structure. So the school's real question is not "spend or save" but "where does the data live" — which is exactly where the question belongs.

The moment privacy flips the answer

The arithmetic above counts only money. Put data policy in and the balance can tip immediately. In three situations self-hosting is reasonable or even the only option:

One, school policy states in writing that data must not leave the campus network. Some sponsoring bodies and special schools hold stricter internal rules on student data, and in that case the cost comparison never needs to be run.

Two, the material is in the most sensitive category. SEN assessment records, counselling cases, social worker referrals, suspected abuse cases — many schools will not put these through a third party even with contractual protection. The workable pattern is routing: sensitive tasks to the local model, everything else to the cloud.

Three, the school needs a simple answer for parents and the school management committee. "It is on that machine in the server room" is an answer anyone understands, and it costs far less to communicate than a summary of contractual terms.

Under the six Data Protection Principles of the Personal Data (Privacy) Ordinance, the school remains the data user throughout, and the security obligation under DPP4 does not transfer through outsourcing. Self-hosting does not make that responsibility disappear, but it narrows it to territory the school actually controls.

Limits and blind spots: five real costs of self-hosting

One, self-hosted models hallucinate too — generally more than a contemporary flagship. Open weights fix data location, not correctness. Every teacher review step stays exactly as it was.

Two, you lose the cloud vendor's safety net. A commercial API sits behind content classifiers, abuse detection and a continuously updated safety policy. Self-hosting hands all of that back to the school. Student-facing use especially needs its own gating and monitoring; see student AI safety, gating, PII and monitoring.

Three, not every feature can be localised. The speech recognition and synthesis behind oral practice currently come from OpenAI or Azure OpenAI. Cutting the cloud off entirely means giving up some features or sourcing them elsewhere.

Four, quantisation is a trade, not a free win. INT4 / INT8 quantisation lets a model run on a small GPU at the cost of some precision. That is usually fine for Q and A and summarising, but anything requiring exact calculation should be tested on the school's own material first.

Five, open-source commitments change. In April 2026 Meta moved its frontier line from Llama to the closed-weight Muse Spark, making the Llama family legacy. A three-year plan cannot assume that what is open today is open in three years. Confirming that already-downloaded weights remain usable is more practical than trusting a roadmap.

Six, fine-tuning is not how the AI learns about your school. LoRA / QLoRA fine-tuning teaches vocabulary, format and tone, but it does not reliably store facts. For the AI to cite the school's curriculum schedule and policies, use retrieval — see the school knowledge base.

Actionable guidance: a decision path and a checklist

Decision path, answered in order:

1. Is there a written rule that data must not leave the campus network? Yes → go on-premises and skip the cost comparison.
2. Does anyone maintain servers? No → go cloud, with data protections written into the contract.
3. Both are possible? → route: sensitive tasks local, everyday tasks cloud. For a school with IT capacity this is usually the best position.

On-premises checklist you can run this week:

• Confirm hardware: is there a machine with 24GB or more of VRAM that can stay powered on?
• Pick a model: start with qwen2.5:14b or llama3.1 and check whether Chinese output meets the panel's standard;
• Define routing rules: which tasks must stay local? Write the list into the school AI policy;
• Add the safety layer: gating, teacher monitoring and audit logging do not appear automatically just because the model is local;
• Confirm the exit: can data be exported in full in one click, and what would moving back to cloud involve?
• Check licences: is the chosen model MIT, Apache 2.0 or bespoke, and are there commercial-use restrictions?

For the full discussion of deployment and data ownership, see on-premises versus cloud deployment. For the complete open and closed model reference, see the model database and the 2026 all-model roundup.

Takeaway

For schools in 2026, open versus closed is no longer an argument about cost. Renting has become cheap enough that hardware struggles to pay back, and the value of self-hosting concentrates in one thing: the school decides where the data lives. Whether that is worth the hardware and staff time is not answered on a quotation — it is answered in the school's data policy.

The good news is that this need not be a binary. One platform, identical features across cloud and on-premises, sensitive tasks routed locally and everyday tasks to the cloud, with the whole dataset retrievable at any time. What the school keeps is optionality, and that matters far more than picking a side.