A 5-Step Examiner Doesn't Need LangGraph

I spent an evening working out how askStudent would fit into LangChain.js before I gave up on it. The tool needed to halt the agent mid-run, hand a question to a React component, and then sit there — for thirty seconds or five minutes — until a human typed something back. Every approach I found came down to a callback firing at the wrong moment, or splitting the run into two halves and ferrying state between them. Both were more code than the thing I was trying to avoid writing.

So I wrote it myself. The examiner that shipped is a system prompt and a while loop. The prompt names five steps in order. The loop only knows how to call one tool, wait, and stop. Once those requirements were clear, every framework I looked at was solving a problem I didn't have, and making the one I did have harder.

What the System Does

The public tree is v0-defense-system: an in-browser oral-exam bench for an intro AI course. The main loop is unremarkable. The system shows a Python problem (implement K-Means, implement KNN), the student writes code in a CodeMirror editor, the code runs locally through Pyodide, and when a run succeeds they click "Submit for Defense."

Problems can be generated too. A teacher gives a topic and a type; a separate path — not the defense loop — asks the model for JSON, syntax-checks the scaffold in Pyodide, and retries up to three times. That path has no tools. If the JSON still fails the check, it comes back anyway.

The defense Agent only appears after submit. It plays a humorless examiner through a fixed sequence, declared in the system prompt as five required steps:

1. readStudentCode
2. runPython (baseline)
3. injectTestCase (at least once)
4. askStudent (a conceptual question, not yes/no)
5. finishDefense (verdict, score, comment)

There is a sixth tool, message, for one-way notes. It does not wait, and the prompt tells the model not to use it as a question.

The product is a coding practice bench. The Agent is the right-hand panel of that bench, not a platform. That is why the rest of this looks the way it does.

Why I Didn't Use a Framework

The two requirements that decided it

Two things made up my mind.

The first is where tools execute. runPython and injectTestCase have to run inside the page's live Pyodide instance — the same interpreter the student just used. runPython keeps globals across calls. injectTestCase appends a probe after the student's current code and runs the combination. Neither is a remote sandbox. Both reach into a WASM runtime the rest of the page already owns.

The second is suspension. When the Agent calls askStudent, the loop has to stop — not poll, not time out, stop — until a human reads the question, types an answer, and hits send. That pause might last thirty seconds or five minutes.

Both cross the boundary between the Agent and the UI, which is exactly what a framework abstracts away. Whichever one I picked, I'd have spent the effort reaching back through its abstractions to get at things it had helpfully hidden.

The runtime constraint is real, but smaller than it looks

It's tempting to say frameworks can't run in a browser. That isn't true — LangChain.js supports browser and edge runtimes, and the Vercel AI SDK is built for the client.

The honest version is weight and depth. This app already ships Pyodide, a multi-megabyte WASM payload that loads before a single line of my code does, and I wasn't eager to put a framework's dependency tree on top of that. There is no backend. Everything a framework does happens in the same tab the student is typing in. When something goes wrong I'm reading a stack trace through three layers of abstraction, in a browser console, on a machine I don't control, during a live lab. Thin layers were worth a lot to me here.

The pipeline is a straight line

Five steps, in order. No branches, no concurrency, no second agent in the conversation. LangGraph is built to manage state machines considerably more interesting than that.

The sequence is also not a graph in my code. It lives in XML in the system prompt. The engine does not check that injectTestCase ran before finishDefense. A model that skips a step still gets a score. That is a real gap. It is also the reason a graph library would have been theatre: I would have been drawing edges for a checklist I had already written as text.

I should be fair, because LangGraph's most relevant feature is the one I reimplemented badly. Its interrupt plus a checkpointer is the "stop, wait for a human, resume" pattern I needed — and unlike my version, it survives the process going away. I traded that off knowingly. The bill for it is further down.

How It's Built

Three layers. Invocation, the task loop, and the tool contract are small. The six tools, the prompt, compaction, persistence, and the UI are not in that number. The file that actually decides what a "defense" is is the prompt, rebuilt before every LLM call.

Layer 1: the LLM invocation layer

This layer sends messages and streams responses back. That's the whole job.

No SDK — just fetch against an OpenAI-compatible /chat/completions endpoint, with a hand-written SSE parser. The parser exists because three streams arrive interleaved and all three matter: ordinary text (content), reasoning tokens (reasoning_content, which DeepSeek and friends emit), and tool call JSON arriving a few characters at a time (tool_calls).

// Concatenating three streams simultaneously
if (delta.content) {
  content += delta.content;
  onText?.(delta.content);
}
if (delta.reasoning_content) {
  thinking += delta.reasoning_content;
  onThinking?.(delta.reasoning_content);
}
if (Array.isArray(delta.tool_calls)) {
  for (const tcDelta of delta.tool_calls) {
    // Concatenate tool name and argument JSON increments
  }
}

The layer also swallows one piece of vendor trivia: some models reject temperature outright with a 400. When the invocation layer sees that specific error, it drops the parameter and retries. Callers never find out.

Layer 2: the tool layer

Six tools. Each is a file exporting { definition, handler } and registering itself into a global registry as an import side effect.

Handlers take parameters and return data. They don't touch engine state. What comes back is a fixed shape:

return {
  content: "String result for the LLM to read",
  asAiMessage: true/false,  // Show this in the UI as the examiner speaking
  waitForReply: { question }, // If present, halt until the student answers
  finalResult: "...",       // If present, the task is complete
  shouldBreak: true/false   // Force the loop to exit
};

Those fields are the entire vocabulary between a tool and the engine. A tool can say "wait for a reply" or "we're done"; the engine decides what that means. The engine never learns what a tool did internally.

The compromise is worth flagging, since it's the part of the design I'm least sure about. asAiMessage and waitForReply aren't data — they're presentation and control-flow instructions hitching a ride on the result, which is a funny thing to find in a return value from a function that supposedly only returns data. A stricter design would have the engine derive both from the tool's identity and keep the payload pure. I went with the loose version because it keeps each tool's behaviour readable in one file, and because six tools is small enough that the coupling hasn't cost me anything yet. At twenty I'd probably regret it.

Layer 3: the task loop

task-loop.js, and it's what it sounds like: build messages, call the LLM, run a tool, handle the result, repeat.

The system prompt is rebuilt before every call, with the current problem and a fresh snapshot of the student's code. If a student edits their solution halfway through the defense, the examiner sees the edit on the next turn instead of grading a stale copy. The prompt also tells it to do that.

Single-step execution is enforced. If the model returns three tool_calls, the engine runs the first, discards the rest, and reminds it to call one tool per turn. Letting a model fan out across tools it picked in one breath was, in testing, the fastest route to an incoherent transcript. Single-step is the one part of the five-step protocol the engine actually guarantees. The rest is hope plus a reminder.

Suspension is the part I like. When askStudent returns a waitForReply payload, the loop doesn't return — it awaits a Promise whose resolve has been handed out to a variable the UI can reach:

if (result.waitForReply) {
  setState(TaskState.WAITING_FOR_STUDENT, { question: result.waitForReply.question });

  const studentReply = await new Promise((resolve) => {
    askResolver = (text) => {
      askResolver = null;
      resolve(text);
    };
  });

  appendUser(studentReply);
  setState(TaskState.THINKING);
}

agent.submitStudentMessage(text) calls askResolver(text), the await unblocks, and the loop carries on with the student's answer in the message history. It's ordinary JavaScript. The only trick is being willing to let a resolve escape its closure.

Then the guardrails, because a loop like this will misbehave in front of a class if you let it:

Safeguard	Trigger Condition	Response
Maximum iterations	Exceeds 30 rounds	Force-terminate the task
No-tool-call penalty	2 consecutive rounds with no tool call	Force-terminate the task
Dead-loop detection	Same tool + same args appear ≥ 4 times in a 10-step window	Intercept execution, inject a prompt forcing the LLM to try a different approach
LLM call failure	Network error or API error	Exponential backoff retry, up to 3 attempts

None of these were designed in advance. The dead-loop detector exists because a model decided a student's KNN implementation ought to be tested against an empty input, got an exception back, and then injected the identical test case four more times — apparently expecting a different answer on the fifth. The no-tool-call penalty exists because another run announced its grade in plain prose instead of calling finishDefense, and then carried on chatting, quite pleasantly, with a student who had already closed the tab.

The thresholds themselves — four repeats inside ten steps, two silent rounds — are just where I happened to get burned. There's no theory behind the numbers.

What This Design Gives Up

Rolling your own means you own the parts nobody puts in a demo.

The transcript survives a refresh. The wait does not. Entries are written to localStorage. The Promise holding askStudent open is not. Reload while the examiner is waiting, and the next thing the student types is liable to start a free-QA turn instead of answering the question. A checkpointed framework, or even a beforeunload flush plus a resume path, would fix this. I have the former half by accident and the latter not at all.

There's no observability worth the name. The session stores a running token count. I do not have traces or replay. When a defense goes strange, I reconstruct it from the transcript.

That guardrail table is a small framework. Iteration caps, backoff, loop detection — they're in every mature framework because everyone who writes an agent loop eventually needs them. I didn't avoid that work. I deferred it until each item hurt enough to fix. That's a defensible strategy. It isn't the same as the work being unnecessary, and I'd be lying if I called those four rows anything other than reinvention.

Nothing here is secure, and for an exam that matters. The whole system is client-side. The prompt, the rubric, the tool definitions and the API key all live in a bundle the student can read. The probe that injectTestCase just ran is visible in the tool card, so a student who wants to can patch the editor before the next turn — and the prompt tells the examiner to re-read if they do. A student motivated enough can get a passing grade without writing K-Means at all. I built this for a lab where that isn't the threat I'm defending against. If it were carrying real assessment weight, the Agent would have to move behind a server, the traces would have to disappear from the student view, and a good half of the reasoning in this post would change with it.

When I'd Reach for a Framework

These are the cases where I'd have lost this argument:

• The Agent is the product rather than a panel inside one.
• Runs have to survive a crash or a refresh, including a wait for a human. Checkpointing is unpleasant to retrofit, and my wait-state currently doesn't.
• Branching, retries over subgraphs, or more than one agent in the conversation.
• Several people maintaining the agent code, where a shared vocabulary beats a tailored one.
• You need traces and evals from day one — anything whose output somebody has to audit later.

Frameworks are good at orchestration. They are worse at reaching into a Pyodide instance and a React input box they didn't design. A five-step oral exam, declared in a prompt, is not orchestration.

Closing

A while loop, a Promise held open across a UI boundary, and one small contract between tools and the engine covered the job. The work was getting the job right: five tools in a fixed order, running in the same tab as the student's editor, stopping when a person had to speak.

LangGraph would have given me a graph I didn't have, and a checkpointer I still kind of want. For this examiner, the prompt was the graph.