What LangChain and LlamaIndex actually do

A secondary school received three AI proposals. Two of them stated in the technical section that they were "built on LangChain and LlamaIndex". The principal handed the documents to the IT coordinator and asked: "What are these two? Are they impressive?"

He answered honestly: "I have heard of them, but I am not sure what they are responsible for in this proposal."

This lesson exists so that a school can ask the right questions in that situation. Not one line of code required.

Picture your school's standard forms and procedures

A school organises an outdoor visit. The real educational design is the teacher's work: where to go, what pupils should observe, how it will be assessed. But making it happen needs a whole set of connecting parts: the application form, the parental consent template, the insurance declaration process, the finance reimbursement format, the transport booking procedure, the incident response guidance.

None of those parts has educational value in itself. Without them, though, every trip has to be reinvented and every teacher does it differently.

A development framework is that set of parts for an AI application. The model is the core capability, equivalent to the educational design, while the framework supplies every fitting needed to connect it to a real process: reading PDFs in, chunking them, storing vectors, attaching tools, logging each step, retrying when the model fails, and switching to another provider without a rewrite.

Two names, two emphases

LangChain centres on orchestrating process and tools. It handles what happens first, what happens next, how to receive the model's request to use a tool, how to retry a failure, and where to insert a point at which a human intervenes. The agent loop from the previous lesson is, in real development, usually assembled from this kind of framework. See from chatbot to AI agent.

LlamaIndex centres on turning your documents into something retrievable. It handles reading in PDFs, Word files, slide decks and web pages, chunking them, building the index, and reranking the dozen passages retrieved. In other words it is the engineering counterpart of the school knowledge base and retrieval-augmented generation and chunking, reranking and lost in the middle.

The two are not competing choices and are frequently used together, with LlamaIndex handling documents and retrieval while LangChain orchestrates the process and the tools.

The real value of a framework is substitutability

If you remember one thing, remember this.

Without a framework, code is written directly against one provider's interface, and the consequence is that moving from OpenAI to Anthropic, or to a school-hosted Ollama, means rewriting the system. This has genuinely stranded schools, not because it was technically impossible but because nobody had time to do it twice.

A framework provides one common interface. Changing model should, in principle, mean changing a line of configuration and nothing else.

For a school the implication is direct: bargaining power and long-term flexibility. A sound technical proposal should leave the school able to change provider in three years without starting over. This is the same argument made about MCP in the previous lesson, at a different layer. MCP standardises how tools connect to models, while a framework standardises how models connect to your application.

So should a school build its own?

Usually not, and the reasons have nothing to do with technology.

Building an AI application in-house makes the school permanently responsible for security updates, provider interface changes, model retirements, data backups, student safety gating, audit records, and the most concrete risk of all: who takes over when the teacher who wrote it transfers to another school. We have seen schools invest a whole year in an excellent prototype and then leave it idle after one staffing change.

The sensible division is that the platform handles this layer while the school puts its effort into what only the school can produce: the content of the knowledge base, the panel's prompt templates, the pedagogical design and the assessment criteria. No vendor can substitute for those three, and they are the assets a school actually accumulates.

An example from the classroom

Back to those three proposals. Having read this lesson, the IT coordinator asked four questions at the evaluation meeting.

First: "You use LlamaIndex for document retrieval, so will answers cite the source passage? Can a teacher open the original?" The first proposal could not answer, because they had built retrieval without returning sources to the interface.

Second: "How much would we change when we switch provider?" The second proposal explained plainly that model configuration sits in one place and can be swapped. The third was evasive.

Third: "Is every AI answer logged? Who can see it? How long is it kept?" That eliminated one proposal with no audit design at all.

Fourth: "At the end of the contract, can we take our documents and index away as a complete set? At what cost?"

Not one of the four questions requires coding knowledge, yet together they exposed the whole gap between the three proposals. The one selected was not the proposal with the most technical vocabulary but the one that answered all four specifically.

That is the point of this lesson. A teacher's expertise lies not in choosing a framework but in checking whether the design protects the school's long-term interests.

What this means for your classroom

1. Do not defer to technical vocabulary. "Built on framework X" is not a quality guarantee, and you are entirely entitled to ask which part it handles and what problem it solved.
2. Put the four questions in your procurement checklist. Citations, provider substitutability, audit records and data export deserve to be standard fields when a school evaluates an AI proposal.
3. Do not spend school resources rebuilding this layer. A school's scarce resources are teacher time and school-based knowledge, not engineering capacity.
4. Whatever you accumulate must be portable. Knowledge base documents, prompt templates and rubrics belong to the school. Confirm that any proposal supports a full export, see security and privacy.

A prompt for scrutinising a proposal

Paste the technical section of a vendor proposal into an AI with the prompt below and you get a question list you can take to the evaluation meeting:

You are a consultant helping a Hong Kong school evaluate an IT procurement. Inside the angle brackets below is the technical section of an AI proposal.
Task: set out what this proposal does and does not say on four matters. One, whether answers cite their source so a teacher can verify them. Two, the cost of changing AI provider. Three, the audit record for each AI answer and its retention period. Four, the arrangements for exporting the school's data and index at the end of the contract.
Format: a table with columns for the matter, the supporting quotation from the proposal, what is missing, and the specific question to put to the vendor.
Final requirements: judge only from the text I supplied; write "not stated" for anything the proposal does not mention and do not infer on its behalf; every piece of supporting evidence must be a direct quotation.
<<<
[paste the technical section]
>>>

Note how the constraints come last and the long material comes at the end. Those two habits come respectively from prompt engineering in one lesson and chunking, reranking and lost in the middle.

What comes next

That completes module 4. Throughout it we have said that fine-tuning is only for teaching a model a way of speaking. So how is fine-tuning actually done, and why has it recently become cheap enough for a single GPU? The next module starts at LoRA and QLoRA on one GPU. To see how the whole course fits together, return to the LLM Classroom syllabus.