LangChain and LlamaIndex: what to ask when those names appear in a proposal

At the start of term a secondary principal laid three AI proposals on the table, each cover stating "built on LangChain and LlamaIndex". He turned to the IT coordinator: "I don't know these names—do they mean it's powerful?" The coordinator answered: "I know they're developer tools, but I can't say which wires they actually run in our school."

This closes module 4. The previous lesson covered agents and tool calling. In practice the loop is rarely handwritten; frameworks wire retrieval, tools and logging together. Teachers need not pick a framework, but they must ask design questions in review—otherwise jargon becomes a blank cheque.

---

Front of house and backstage

On performance night the audience sees ten minutes on stage: lights, sound, projection, the MC's links. Backstage another crew works in parallel: HDMI paths, radio channels, backup playback, a log of which cue failed and must rerun.

The language model is the performer on stage—generating text, answering prompts. Frameworks such as LangChain and LlamaIndex are backstage wiring—reading PDFs in, chunking them, building indexes, attaching tools, retrying failures, writing each step to a log. Without backstage rigging every show means new cables; with it, changing the performer (changing model) should not mean ripping out the lighting desk.

Two backstage roles

The names appear together, but the emphasis differs.

LangChain runs the sequence

It answers what happens first and next: how to receive the model's tool request, how the agent loop turns, where a human confirmation interrupt sits. The "read the result and schedule the next step" pattern from the previous lesson is usually assembled here.

LlamaIndex runs document intake

It answers how files become searchable passages: ingest PDFs, Word, web pages, chunk, build vector indexes, rerank after retrieval. That is the engineering counterpart of the school knowledge base and RAG and chunking and reranking.

In production the two often combine: LlamaIndex for documents and retrieval, LangChain for process and tools—like a show needing both sound and lighting, not either/or.

A framework's real value is not fame but whether backstage wiring survives a change of model—worth taking to the review meeting.

Substitutability belongs in the contract

Code written directly against one provider's API often means moving from OpenAI to Anthropic, or to a school-hosted Ollama, equals a rewrite. We have seen schools spend a year on a prototype and then lack the capacity to migrate when a contract ends.

A framework supplies a shared interface: ideally only a small configuration block changes when the model changes, while process, audit and UI stay put. That aligns with MCP from the previous lesson—MCP for tools, frameworks for models—both protecting long-term flexibility.

Edor.ai already embeds this layer: switch among OpenAI (default), Azure OpenAI, Anthropic, Poe and local Ollama, and receive finished tools rather than a box of parts.

Four questions at the review meeting

Back to those three proposals. After this lesson the IT coordinator asked only four things—no coding required:

1. Sources: "Will answers cite which school document passage they came from? Can a teacher open the original?" The first proposal built retrieval but never returned passages to the UI and could not answer.
2. Substitutability: "If we change AI provider in three years, how much must we change?" The second named a single configuration location; the third was evasive.
3. Audit: "Is every AI reply logged? Who can see it? How long is it kept?" One proposal had no audit design and dropped out.
4. Portability: "At contract end, can we export documents and indexes as a complete set? At what cost?"

Those four questions exposed the whole gap. The winner was not the proposal with the most jargon but the one with specific answers on all four. A teacher's expertise is not choosing a framework but checking that design encodes data ownership, audit and human review.

When the coordinator fed the winning proposal's technical appendix into an AI with the exercise prompt below, the gap list took under three minutes to produce—time well spent before the panel voted.

Where people go wrong

• Wrong turn: seeing framework names and assuming reliability without asking about citations and audit. Self-check: run the exercise prompt below on the technical appendix before the meeting.
• Wrong turn: spending teacher time and school budget maintaining the framework layer in-house. Self-check: ask who owns security updates when the subject teacher leaves; if nobody, choose a platform.
• Wrong turn: comparing model brands only, not whether data exports in one click. Self-check: demand a demo export including indexes; see security and privacy.

Try it once

Time: about 12 minutes. Materials: any AI tool plus a (shareable) technical appendix or product spec excerpt.

1. Paste:

You are a consultant helping a Hong Kong school review an IT procurement. Between <<< and >>> is an AI proposal technical section.
For four matters list what the proposal states, omits, and what to ask the vendor: one, whether answers cite sources; two, cost of changing AI provider; three, audit and retention for each reply; four, export of data and indexes at contract end.
Judge only from the supplied text; write "not stated" rather than inferring; quote the proposal when you cite evidence.
<<<
[paste the technical section]
>>>

2. Hand in: the four-part list plus one sentence from you marking the most dangerous omission and why.

You got it right if the list contains honest "not stated" entries and your sentence points to audit or export. Module 4 ends here; module 5 opens at LoRA and QLoRA on one GPU. Course map: LLM Classroom.