← AI Catalyst C3All programsHomeSearch
AI Catalyst C3·Core Session - Week 16·2:12:07

Build Reliable AI Apps Using LangGraph: The Travel Agent That Changes Its Mind — State, Nodes, Cycles, Interrupts, and Why the Vocabulary Is the Skill

Akhil Alampally Returning Catalyst trainer (taught the loop-engineering day of the Hermes sprint). Enterprise-leaning practitioner - describes keynoting to 500-800 people at a manufacturing company weeks earlier. Frames the session as an introduction by choice: 'I would not dive deep into the detail, primarily to keep the conceptual understanding... much more clearer.' · Shivani Outskill community manager - opens, handles the Week 16 network drops, relays the town-hall and Bangalore immersion logistics, posts the CSAT · Rajesh Cohort member - built a working version of the same app during the session; his demo is what the class actually gets to see, including an LLM-wired variant · Praful Thakkar Cohort member - asks the LangGraph-versus-loop-engineering question that produces the session's sharpest comparison

Session map

WHY A GRAPHTHE SEVEN WORDSHOW YOU BUILD ITFour attemptsprompt, pipeline, framework, DIYWhen not todecision, retry, or humanGraph vs looptrust vs traceabilityStateone shared recordNodes and edgesyou draw the mapFan-out, reducerwho writes firstConditional edge, cyclethe backwards arrowCheckpoint, interruptauto-save, then freezeVocabularytwelve words, right buildScreenshot to builddiagram, plan mode, implementLangSmithan eye for the agentEval setten cases, one number
Why a graphThe seven wordsHow you build it
click a node — its card pops up (drag it anywhere, × to close)
Concept

The map reads left to right — why a graph flow into the seven words, then into how you build it. Click any node to open that idea here; every timestamp jumps into the recording.

The short version

  1. THE WHOLE SESSION IS ONE WORKED PROBLEM: a travel agent that can change its mind. One messy sentence in — 'two of my family want 5 days in Bali in mid-November, we like nature and food, nothing too touristy, budget about 1.2 lakh INR, flying from Bangalore' — and out the other end an itinerary that searched flights, stays and activities at once, checked the total against the budget, went back for cheaper options when it was over, and stopped to ask before handing anything over. Nothing in the input sentence is structured: no dates, no airport codes, no clean numbers, and four or five jobs hiding inside it.
  2. THE FOUR-ATTEMPT ESCALATION IS THE ARGUMENT. Ask a model once: you get a confident, well-written five-day plan with no real prices, no availability, no budget check, and no way to tell whether the numbers were invented. Build it in n8n: it works right up until the total comes in at 1,35,000 against a 1,20,000 budget — and n8n runs left to right on a trigger, so its only move is to hand you a plan you cannot afford. Use an autonomous agent framework: impressive demos, very little wiring, and you need to know what it will do before it does it. Write it all yourself: you are now building cycles, memory, pauses, interrupts, state and tracing by hand.
  3. SEVEN WORDS ARE THE ACTUAL CURRICULUM. State is one shared booking record everyone reads and writes — 'nobody emails anybody'. A node is a specialist that does exactly one job. An edge is an arrow you draw: 'the AI does the thinking inside every step, but it never gets to rewrite the map.' Parallel execution fans out independent searches. A reducer is the rule for combining simultaneous writes — replace or append. A conditional edge makes the next arrow depend on a test. A cycle is LangGraph's word for a loop, and 'it is the single reason LangGraph exists'. A checkpointer auto-saves the state after every step. An interrupt is the freeze before money moves.
  4. KNOWING WHEN NOT TO USE IT IS PART OF THE SKILL. Straight line with no decisions, cycles or retries: use n8n, it is faster to build and easier to change. One prompt is enough: just ask the model. Someone else's ops team has to edit it: keep it as linear automation, even at a time cost, because the learning curve is the real overhead. 'Reach for LangGraph only when there is a decision, a retry, or a human in the middle.' Asked whether a marketing pipeline — script, send, feedback, edit, approve — fits, his answer is blunt: no, that is a pipeline, it would be over-engineering.
  5. LANGSMITH IS THE OTHER HALF, AND IT IS THE ENTERPRISE HALF. Nine steps, one breaks — which one? Without a recording you add print statements, rerun, change something, and half an hour later still do not know. LangSmith records every action, every tool call, every retry and its reasoning: 'an eye for the agent'. He is explicit that observability is what makes guardrails possible at all — you cannot stop an agent storing personal data if you cannot see what it does. One hard gotcha: the data region is chosen at ACCOUNT level and cannot be changed afterwards.
  6. THE VOCABULARY IS THE PROMPT. 'The vocabulary is how you give your orders to Claude Code.' Ask for 'a travel agent that can try again if it is too expensive' and the tool picks any framework it likes. Ask for 'a conditional edge from the budget check; if over budget, go to a relax step that loops back to the three searches, capped at three rounds' and you get the build you wanted in twelve words. His analogy: talk to a civil engineer in civil-engineering terms and they do more with fewer words. You do not need the Python — you need the nouns.
  7. THE BUILD HALF FAILED LIVE, AND THAT IS ITS OWN LESSON. He screenshotted the architecture diagram, attached it to Codex, described the problem in LangGraph terms, asked for mock data, a Mermaid diagram in the README and a restrained Apple-style UI with a 'Sherlock mode' toggle for per-node visibility. Codex interrogated him through plan mode — at which point he noted 'I should have started this in plan mode' and, later, that he should have said 'go with your recommended answers for all your questions you're about to ask'. Then his Codex limits ran out, twice, on two accounts. A cohort member's working version was demoed instead, and the honest closing note is that the mock version revises nothing because 'the AI part is missing — the moment you plug in an LLM, you should be very good to go.'
  8. THE HANDOVER IS A FOUR-ITEM TO-DO LIST that doubles as the production checklist: connect an LLM after the UI mock is done; add Firecrawl and Skyscanner for live data; add LangSmith as the tracer; and create ten edge cases to test it — 'this is what we call an eval set'. If the agent passes 80 of 100 cases, the benchmark is 80%, and that number is what you hand back to a model to ask what to optimise. For anything customer-facing, guardrails come first: does it store customer data or fail to anonymise identifiers, which he frames against GDPR.

Outskill's post-read

The written recap Outskill published after the session (build-reliable-ai-apps-using-langgraph.docx). Where it is sharper than the live session, those points are here; the full concepts below are from the recording.

  1. Restates the seven core terms with the same analogies used on air: state as a booking record, node as a specialist, edge as an arrow you draw, reducer as replace-or-append, cycle as the reason LangGraph exists, checkpointer as auto-save, interrupt as the pause before spending.
  2. 'You decide the map. The AI only thinks inside each node.' - the written version of the session's central constraint.
  3. The graph in six ordered steps: read the request; fan out to three parallel searches; check the budget; if over, loop back capped at three rounds; if it fits, interrupt for approval; hand over or go again.
  4. The LangChain family, disambiguated in four lines: LangChain the parent, LangGraph the open-source graph framework, LangFlow the no-code drag-and-drop tool, LangSmith the tracer.
  5. The state field list, which the live session assembled by asking the cohort and never wrote down cleanly: request, departure, destination, dates, travelers, budget, things to do, flight/stay/food preferences, date-shift tolerance, flight type.
  6. The build order for Codex or Claude Code: attach a screenshot of the graph; describe it in LangGraph terms; list what state must hold; start with mock data; start in plan mode and review; ask for a plain UI with a behind-the-scenes toggle; then connect the LLM, live data and LangSmith.
  7. Four 'same shape' applications with the loop and pause named for each: lead qualification (loop under score 70, pause before emailing the founder), RFP response (redraft any section missing a requirement, pause before the client), competitor watch (retry if the page changed, pause before alerting), support triage (loop when not confident, pause before promising a refund).
  8. The LangSmith account-level data-region warning is repeated in writing, which suggests it is a known support issue rather than an aside.

Its headings Overview · Why a single prompt or a linear workflow is not enough · What LangGraph is · Core concepts · The travel agent graph · LangSmith: tracing the agent · When to use LangGraph · Where the same shape applies · Building it with Codex or Claude Code · Testing before production · Key takeaways

At a glance, three clicks deep

Skim here first: the closed row is the glance, open is the study card with the key points and timestamps, and the ↓ link drops to that concept's full write-up below.

01The four-attempt escalation: why a prompt, a pipeline, or a framework each hits a wallA graph framework earns its complexity exactly when the process must branch on a result, revisit an earlier…0:16

A graph framework earns its complexity exactly when the process must branch on a result, revisit an earlier step, or stop for a human — the three things a linear pipeline structurally cannot do.

The messy sentence hides 4-5 jobs and contains no structure (0:16)

Model alone: confident plan, no real prices, no availability, no budget check, no self-stop (0:17)

n8n: left to right, trigger-driven, linear; looping needs multiple triggers and workflows (0:21)

'A line of steps has exactly one move' - carry on with an unaffordable plan (0:21)

Autonomous frameworks: impressive demos, little wiring, unpredictable (0:40)

Roll-your-own means hand-building cycles, memory, pauses, interrupts, state and tracing (0:41)

LangGraph is free and open source, made by LangChain (0:22)

↓ Full write-up of this concept

02State: one shared booking record, and nobody emails anybodyState is a single shared, mutable record scoped to one job;0:24

State is a single shared, mutable record scoped to one job; every node reads from it and writes to it rather than passing data to the next node directly.

One record per job; search, pricing, payment and support all read the same one (0:25)

'Nobody emails anybody, they all read and write the same record' (0:25)

The kitchen analogy: a reservation ID whose status every chef updates (0:26)

The thinking pane in ChatGPT or Claude Code is the same mechanism made visible (0:26)

Travel-agent state accumulates: flights found, stays found, activities searching, budget check waiting (0:25)

The post read lists the full field set the live session never wrote down cleanly

↓ Full write-up of this concept

03Nodes and edges: the AI thinks inside the step but never rewrites the mapA node is a single-purpose step that reads and writes the shared state;0:27

A node is a single-purpose step that reads and writes the shared state; an edge is a human-authored arrow determining which node runs next. Model discretion is confined inside nodes.

Node = a specialist that does only one job really well (0:28)

Four nodes: read request, search flights, search stays, add up cost (0:27-0:28)

The cost node has no AI in it - 'pure arithmetic' (0:28)

Edge = one arrow; you draw it, you decide what runs next, not the model (0:29)

'The AI does the thinking inside every step... but it never gets to rewrite the map' (0:29)

Humanise the nodes: a person at each one, coordinating to hit the budget (0:24)

'Nodes are nothing but a bunch of functions working in conjunction' (1:23)

↓ Full write-up of this concept

04Fan-out and the reducer: who gets to write first when three nodes finish at onceFan-out runs independent nodes concurrently;0:30

Fan-out runs independent nodes concurrently; a reducer is the declared rule for merging simultaneous writes to the same state field, either replacing the existing value or appending to it.

Independent searches fan out and run at once, then rejoin to rebalance (0:30)

Sequential execution in a pipeline costs time for no benefit (0:30)

The race question: 'who gets to write into the booking first?' (0:32)

Reducer = the rule for combining two results: replace or append (0:33)

He flags it explicitly as an edge case to remember (0:33)

Targeted replan later depends on retaining valid results - i.e. on appending (1:20)

↓ Full write-up of this concept

05The conditional edge and the cycle: the backwards arrow that is the whole pointA conditional edge routes on a runtime test;0:33

A conditional edge routes on a runtime test; a cycle is an edge that returns to an earlier node, capped at a fixed number of rounds so a retry cannot run indefinitely.

Conditional edge = an arrow made dependent on a condition (0:34)

Budget fits -> forward to human approval; does not fit -> backwards, search differently (0:33-0:34)

Human rejection is also a condition that triggers another round (0:34)

Cycle is LangGraph's word for a loop - 'we don't use the name loop here' (0:35)

'It is the single reason why LangGraph exists' (0:35)

Cap the cycle - three rounds, by convention (0:39)

Cap semantics matter: initial plan as attempt one, so at most two replans (1:21, 1:24)

A user rejection resets the attempt counter (1:24)

↓ Full write-up of this concept

06Checkpointer and interrupt: auto-save after every step, and the freeze before money movesA checkpointer persists state after every step, enabling cross-session memory, crash recovery and rewind;0:35

A checkpointer persists state after every step, enabling cross-session memory, crash recovery and rewind; an interrupt is a deliberate pause that yields control to a human before a consequential action.

Checkpointer saves state after every step - auto-save in a game (0:35)

Remembers a user across sessions; survives a mid-run server restart (0:36)

Lets you rewind to an earlier round because that round was saved (0:37)

This is what lets one agent serve 100-150 users and scale with infrastructure (0:37-0:38)

The ChatGPT resume-after-five-days example is the same mechanism (0:36-0:37)

Interrupt: 'it stops before it spends money' (0:38)

Waits 5 seconds or 5 days; nothing is lost (0:38)

On approval the interrupt releases, payment runs, checkpoint and state update (0:38)

In the generated plan, an interrupt is also used to ASK for a missing state field (1:23)

↓ Full write-up of this concept

07When NOT to use LangGraph: three disqualifiers and one signalLangGraph is warranted when a process requires a runtime decision, a retry that changes something, or a hum…0:42

LangGraph is warranted when a process requires a runtime decision, a retry that changes something, or a human pause — and is over-engineering for linear pipelines, single-prompt tasks, and anything a non-technical colleague must maintain.

Straight line, no decisions/cycles/retries -> n8n, faster to build and easier to change (0:43)

One prompt is enough -> ask the model once (0:43)

Someone else must edit it -> linear automation, to avoid the learning-curve overhead (0:43)

'Reach for LangGraph only when there is a decision, a retry or a human in the middle' (0:44)

Signal shape: a loop, a pause, and parallel independent work to combine (2:08)

A marketing script pipeline gets a flat no - 'over-engineering' (2:06)

CrewAI solves the same problems differently; both are 'languages' for orchestration (2:10-2:11)

Named fit: finance, because of the human-in-the-loop element (2:10)

↓ Full write-up of this concept

08LangSmith: the agent tracer, and why guardrails depend on itAgent tracing records every step, tool call and retry with its reasoning;0:45

Agent tracing records every step, tool call and retry with its reasoning; it is the precondition for debugging a multi-step agent and for making any verifiable claim about what it does with data.

Without tracing: print statements, reruns, and still not knowing (0:45)

LangSmith = 'an eye for the agent' - thinking, reasoning, tool calls, steps, retries (0:48)

'LangGraph builds the agent. LangSmith records what it does' (0:48)

Observability named as the pillar every enterprise is standing towards (0:48)

Guardrails depend on visibility - you cannot stop what you cannot see (0:49)

PII/GDPR framed as the concrete case (0:49, 2:09)

DATA REGION IS ACCOUNT-LEVEL AND UNCHANGEABLE - repeated in the post read (2:03-2:04)

APAC recommended for India (2:06)

The LangChain family: LangChain parent, LangGraph framework, LangFlow no-code, LangSmith tracer (1:04-1:05)

↓ Full write-up of this concept

09The vocabulary is the prompt: twelve words that replace a framework argumentFramework fluency for agent-assisted building means knowing a framework's domain vocabulary well enough to…0:49

Framework fluency for agent-assisted building means knowing a framework's domain vocabulary well enough to specify in it; the coding agent supplies the syntax.

'The vocabulary is how you give your orders to Claude Code' (0:49)

Vague prompt lets the tool pick any framework; jargon prompt lands the build (0:49)

The worked replacement: 'add a conditional edge from the budget check... capped at 3 rounds' (0:49)

'We don't have to necessarily understand the Python programming... but we need to know what LangGraph consists of' (0:24)

The civil-engineer analogy - jargon buys efficiency with fewer words (1:45)

Codex asks 9 clarifying questions; the vocabulary is what answers them (1:19-1:21)

Stated takeaway: understand the concepts, learn the jargon, build a simple agent (1:39)

↓ Full write-up of this concept

10Screenshot to plan to build: the workflow, and the two things he wished he had doneA diagram-first build loop: screenshot the architecture as context, specify in framework vocabulary with mo…1:06

A diagram-first build loop: screenshot the architecture as context, specify in framework vocabulary with mock data, request plan mode, review the plan's assumptions, then implement.

Screenshot the diagram as context - 'the fastest way to build' (1:06)

Mock data first; Firecrawl and Skyscanner named as the later live sources (1:09)

Ask for a Mermaid diagram in the README (1:16)

Restrained UI plus a per-node 'Sherlock mode' visibility toggle (1:17)

'I should have started this in a plan mode' (1:16)

The meta-instruction he wished he had used: go with your recommended answers (1:21)

Nine plan-mode questions, all answerable from the vocabulary (1:19-1:21)

Targeted replan chosen: rerun only affected searches, retain valid results (1:20)

Two Codex accounts exhausted live; the app never ran (1:40-1:41, 2:01)

The mock cannot revise - the LLM is the missing part (1:57)

↓ Full write-up of this concept

11The handover to-do list: live data, a tracer, and ten edge cases that become your eval setAn eval set is a fixed list of edge cases run against an agent to produce a pass percentage;2:03

An eval set is a fixed list of edge cases run against an agent to produce a pass percentage; that percentage is the benchmark against which optimisation is measured. Guardrails - data handling and anonymisation - are checked first and separately.

Four-item to-do list: connect the LLM, add live data sources, add the tracer, create 10 edge cases (2:03-2:05)

'Creating a bunch of edge cases where the model can break is an eval set' (2:05)

100 cases, 80 passing = an 80% benchmark (2:09)

Hand the score and the set back to a model and ask what to optimise (2:10)

Guardrails FIRST for anything customer-facing: storage and anonymisation of personal data (2:09)

GDPR named as the standard (2:09)

Use AI to generate the edge cases in the first place (2:05)

'Sherlock mode should say the same things as the LangSmith tracer, if not better' (2:04-2:05)

↓ Full write-up of this concept

12LangGraph versus loop engineering: wiring specialists, or trusting one good nodeGraph orchestration wires single-purpose specialists with explicit routing and is traceable;0:27

Graph orchestration wires single-purpose specialists with explicit routing and is traceable; loop engineering delegates a goal to a capable node and is cheaper and simpler but harder to attribute. The difference is model capability, not chronology.

Graph = nodes as specialists, directions you wire, loops you draw (0:27)

Loop engineering = more abstract; describe a goal, skip the node decomposition (0:27, 1:58)

Higher-tier models are what make the abstraction viable (1:58)

One loop-engineering node may do 15 things, and you are trusting it (1:59)

Tracing is the cost - 'no way for us to trace their activity' (1:59)

Loop engineering is cheap - 'you're just defining when to loop and how to loop' (1:59)

'Don't think about loop engineering as the next evolution' - it's model evolution (1:59-2:00)

Predicts LangGraph docs will evolve toward loops (1:58-1:59)

CrewAI as a peer implementation; both are orchestration 'languages' (2:10-2:11)

↓ Full write-up of this concept

The concepts in full

01

The four-attempt escalation: why a prompt, a pipeline, or a framework each hits a wall

0:16

One messy sentence, four honest attempts, four different walls.

The input is deliberately realistic: "two of my family want 5 days in Bali in mid-November, we like nature and food, nothing too touristy, budget is about 1.2 lakh INR in total, flying from Bangalore." Nothing in it is structured — no dates, no airport codes, no clean numbers — and four or five separate jobs are hiding inside it.

Attempt one, ask the model. ChatGPT or Claude returns a lovely, confident, instant five-day plan. What it does not return is real flight prices, real availability, or any check that the total fits 1.2 lakh. "There is no way for me to know if it did not simply invent the numbers." A model on its own is brilliant but has no hands, cannot go and look, and will not stop itself.

Attempt two, build it in n8n — the tool the cohort already knows. Read the request, find flights, find a hotel, add it up, send it back. Which works until the total comes in at 1,35,000 against a 1,20,000 budget. And here is the wall: n8n is always left to right, always driven by a trigger or an event, always a pipeline. "A line of steps has exactly one move: carry on, and hand you a plan that you cannot afford." To loop you would need multiple triggers and separate workflows watching for branch events.

Attempt three, an autonomous agent framework: "I've seen a lot of impressive demos, but there's very little wiring made between them — you need to know exactly what it will do before it does it." Attempt four, write it yourself: now you are building the cycles, the loops, the memory management, the pauses, the interrupts, the state handling and a tracer, by hand. LangGraph is the framework that gives you those parts so you can fast-track to the agent.

Worked example · from the session

He asks the cohort what an n8n workflow does when it lands over budget. The answers come back: revise, repeat, budget — and Caroline's 'get angry', which he accepts as the honest first reaction.

Why it matters

It names the boundary condition. Every automation decision Paul makes is really the question 'does this ever need to go backwards?'

People get this wrong

A model with web search solves this, because now it can look things up.

Lookup is one missing piece. It still will not check the total against your constraint and stop itself, and you still cannot see what it did.

A line has one move. A graph can go back. The same job, the same failed budget check, two different outcomes. LINE (n8n, Make, Zapier) Read the request Find flights Find a hotel Add it up 1,35,000 vs 1,20,000 Only move: carry on. Hands you a plan you cannot afford. GRAPH (LangGraph) Read the request Flights Stays Things to do Budget check Ask the traveller fits over budget: go back, search differently (cap 3) The red arrow is the entire difference.
A line has exactly one move when it hits a wall: carry on. A graph can go back. The green arrow is the whole difference.
A line of steps has exactly one move: carry on, and hand you a plan that you cannot afford.0:21
For your projects

Paul builds in n8n-shaped tools and in Lovable. This concept is the clearest statement in the corpus of the exact point at which a linear automation stops being the right answer - which is a scoping decision he makes repeatedly.

  • Audit the agency's existing n8n and Zapier workflows for any that silently 'carry on' when a check fails - those are the graph candidates
  • Add a 'does this need to go backwards?' question to the intake checklist for any automation scope
Try it now
Try it now

Take one workflow you own and describe what it does when its final check fails. If the answer is 'nothing, it finishes anyway', it is the wrong shape.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

What exactly can't n8n do in this example, and why?

Go back. It executes left to right from a trigger, so when the budget check fails there is no arrow backwards — it can only continue and deliver a plan the user cannot afford.

What is wrong with asking ChatGPT once, given it produces a good-looking plan?

You get no real prices or availability, no verification against the budget, and no way to distinguish researched numbers from invented ones. Confidence is not grounding.

02

State: one shared booking record, and nobody emails anybody

0:24

Every booking system you have ever used already works this way: one record per booking.

In LangGraph, state is the single shared record for the job. His analogy is a booking: search writes to it, pricing writes to it, payment writes to it, and the support agent on the phone reads the same record. "Nobody emails anybody. They all read and write the same record." For the travel agent the record holds two travellers, Bangalore to Bali, mid-November, 1.2 lakh budget — and then accumulates: flights, three options found; stays, two options found; things to do, searching; budget check, waiting.

His second analogy is a kitchen. Your table has a reservation ID, and that ID's status travels into the kitchen; every chef updates it — boiling done, grill done — until the recipe is complete, so the waiter knows exactly when to collect. He also points out you have already seen this: when you open the thinking pane on ChatGPT or Claude Code, the running commentary you read is the same mechanism, a record being updated as work completes.

The consequence is the one that matters operationally. Because there is one record rather than a chain of payloads, every step's contribution is visible in one place, and a later step can read what an earlier one found without anyone having to plumb it through.

Worked example · from the session

Asked later how the checkpointer relates, he makes the connection concrete: the state is what gets saved. Rajesh's working demo shows the state changing visibly at each node in the 'Sherlock mode' panel - planning attempt marked as 1, requirements validated with missing fields none, three matching flights found, itinerary combined at 43,500 against a 60,000 budget.

Why it matters

It is the difference between an automation you can debug and a chain of steps whose intermediate values are gone.

People get this wrong

State is just a variable bag for convenience.

It is the coordination mechanism. Parallel nodes, retries and checkpointing all only work because there is exactly one record for them to agree on.

Nobody emails anybody. They all read and write the same record.0:25
For your projects

Every multi-step automation Paul has built passes data between steps. This is the argument for a single shared record instead - and it is the pattern that makes a run inspectable after the fact.

  • Refactor one multi-step workflow so every step reads and writes a single record rather than passing payloads along
  • Use the same shape for the KB pipeline itself - one state record per session being extracted, rather than per-script files
Try it now
Try it now

Sketch the state record for one process you automate. If you cannot list its fields, the process does not yet have a shape.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

What does 'nobody emails anybody' mean in engineering terms?

No node hands a payload to another node. All communication is through the shared record, so a step's output is available to every later step without explicit plumbing.

Why is a single state record easier to debug than a chain?

Because the whole run's accumulated findings live in one inspectable place, rather than existing only transiently inside the step that produced them.

03

Nodes and edges: the AI thinks inside the step but never rewrites the map

0:27

You draw the arrows. The model is only allowed to think inside the boxes.

A node is a specialist that does exactly one job well. In the travel agent: one node reads the request and turns a messy sentence into dates and airport codes; one searches flights and knows nothing about hotels; one searches stays; one adds up the cost, which is "pure arithmetic, no AI anywhere in it". Each updates the state when it finishes. He also offers a humanising frame that makes the parallelism obvious: "assume at every node there is a human sitting to do that particular exact job" — so when flights come back expensive and hotels cheap, those two people coordinate to land the budget.

An edge is one arrow between two nodes, and you draw it. "This matters more than it seems... because you are the person to decide what runs next, not the model." Then the line that carries the whole design philosophy: "the AI does the thinking inside every step, for sure, but it never gets to rewrite the map." He calls it a blueprint — how data must flow between nodes is your decision, exactly as it would be on an n8n canvas; what changes is that the map can contain arrows going backwards.

Nodes, he clarifies during the build, "are nothing but a bunch of functions working in conjunction, to put it in a simple fashion."

Worked example · from the session

The pure-arithmetic node is the tell. The budget adder has no model in it at all, which is what makes the budget check trustworthy - a model asked to add prices could be wrong, and this cannot.

Why it matters

It is the concrete version of 'designed autonomy': the model's discretion is bounded by a map you authored.

People get this wrong

Using a graph framework means handing more decisions to the model.

The opposite. The graph is where you take routing decisions back and leave the model only the reasoning inside each step.

The AI does the thinking inside every step, for sure, but it never gets to rewrite the map.0:29
For your projects

This is the control principle Paul cares about most in agent work - the thing that separates a system you can predict from one you hope about. It is also the honest answer to 'how autonomous is this?'

  • Draw the map before writing any prompt, for the next agentic thing the agency builds
  • Use the node-per-job rule as a code-review test: any node doing two things is a design smell
Try it now
Try it now

List the nodes for a process you know. If any node's description needs the word 'and', split it.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Why does he insist the cost node contains no AI?

Because the budget decision depends on it. Arithmetic done by code is correct; arithmetic done by a model is probable. The node boundary is where that choice gets made.

Who decides execution order in LangGraph, and why does it matter?

You do, by drawing the edges. It means the system's possible behaviours are bounded by a map you authored, rather than by what the model decides at runtime.

04

Fan-out and the reducer: who gets to write first when three nodes finish at once

0:30

Three people finish their research in the same minute. Who writes into the record first?

Sequential execution is slower for no reason when the steps are independent. Flights, stays and things to do do not depend on one another, so the graph fans out and runs all three at once, then rejoins to rebalance. "While I'm doing it in a sequential form, just the way I'm doing it in n8n, then it is going to take more amount of time."

Then he stops on the question that parallelism creates, using cohort members' names to make it physical: Praful searching flights takes two minutes, Sara searching stays three, Caroline searching activities four — fine, they land in sequence. But what if all three finish simultaneously? "It's like three people are talking to the same phone call. Who gets to answer it first?" Does the new result replace what is already there, or sit alongside it?

The rule for combining two results is called a reducer, and it does one of two things: replace the value in state, or append to a list in state. He flags it as an edge case you simply have to remember, and it is exactly the kind of thing that is invisible until three nodes race and one result vanishes.

Worked example · from the session

The flip side shows up in his own to-do list later: 'targeted replan' means rerunning only the affected searches while retaining valid results. That only works if the reducer appends rather than replaces the results you wanted to keep.

Why it matters

Parallelism is the performance win; the reducer is the correctness condition attached to it.

People get this wrong

Running steps in parallel is purely a speed optimisation with no design consequence.

It creates a write-ordering problem. The reducer is the design decision parallelism forces you to make.

For your projects

The concurrency question most no-code automation never has to ask, and therefore the one most likely to produce a silent bug when Paul does build something parallel.

  • Identify the independent lookups in an existing workflow and run them concurrently - the cheapest latency win available
  • For any parallel step, decide replace-or-append explicitly rather than by accident
Try it now
Try it now

For a parallel step you already run, write down whether each result replaces or accumulates. If you have to guess, you have a latent bug.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

What problem does a reducer exist to solve?

Simultaneous writes from parallel nodes to the same state field. Without a declared rule, one result silently overwrites another.

Why fan out at all rather than just running the searches in order?

Because the searches are independent, so sequential execution only adds latency. The rejoin is where the results get compared.

05

The conditional edge and the cycle: the backwards arrow that is the whole point

0:33

'It is the single reason why LangGraph exists.'

A conditional edge is an arrow whose destination depends on a test. After the budget check: 1,20,000 fits, so go forward and ask the traveller to approve; 1,35,000 does not, so go backwards, search again, and do it differently. "The edge, or the checkpoint, where you make an arrow dependent on a condition is called a conditional edge." And rejection at the approval step is itself a condition — if the human does not like it, round again.

The backwards arrow is a cycle, which is LangGraph's own word: "we don't use the name loop here, but they both mean the same thing." Round one came in at 1,35,000 and was over budget; round two shifted the dates, accepted a stopover, and came in under. He caps it at three rounds "just for the sake of convention", and both the session and the post read repeat the cap — an uncapped cycle is how a retry loop becomes a bill.

This is the capability that the whole four-attempt escalation was built to motivate. A pipeline cannot draw this arrow. "It is much more prominent for building agents left, right, centre."

Worked example · from the session

The cap has a definitional wrinkle Codex catches him on during the build: does three retries mean three total plans or an initial plan plus three replans? He chooses initial-plan-as-attempt-one, and the generated plan records it as the initial itinerary plus at most two replans - with a user rejection treated as new information that resets the counter.

Why it matters

It is the one capability that justifies the framework, and the cap is the guardrail that keeps it affordable.

People get this wrong

A retry means running the same step again.

It means running it differently — shifted dates, an accepted stopover, only the affected searches. A retry that changes nothing is just a slower failure.

The travel agent that changes its mind Seven pieces assembled: read, fan out, check, loop or pause, hand over. 1. Read the request 2a. Search flights 2b. Search stays 2c. Things to do messy sentence in, dates and airport codes out 3. Budget check pure arithmetic, no AI fan-out: independent, so they run at once. A reducer decides replace-or-append when they land together. 4. Interrupt: ask the human freeze; 5 seconds or 5 days fits Relax the constraints shift dates, allow a stopover over budget CYCLE, capped at 3 rounds an uncapped retry is a bill 5. Hand over the itinerary reject A checkpointer saves the state after every one of these steps - which is what makes the rewind, the resume and the 5-day pause free.
The seven pieces assembled: read, fan out to three searches, budget check, conditional edge either back into the searches (capped at three rounds) or forward to the human interrupt.
It is the single reason why LangGraph exists.0:35
For your projects

The retry-with-different-parameters pattern. Paul's site-health and audit work is full of checks that currently just report a failure; this is the shape that acts on one.

  • Add a capped retry-with-different-parameters step to one agency automation that currently just reports failure
  • Pick the cap deliberately and log every round, so a three-round failure is diagnosable rather than mysterious
Try it now
Try it now

Take a check that currently fails loudly and decide what a second attempt would change. If nothing, you do not need a cycle.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

What is the difference between a conditional edge and a cycle?

A conditional edge chooses a destination based on a test; a cycle is what you get when one of those destinations is a node that already ran. The condition is the test, the cycle is the shape.

Why cap the number of rounds?

An uncapped retry runs until it succeeds or until the bill stops it. The cap converts an open-ended loop into a bounded one with a defined failure state.

06

Checkpointer and interrupt: auto-save after every step, and the freeze before money moves

0:35

'You never had a card charged without tapping confirm. Same idea.'

A checkpointer saves the state after every step. His analogy is a game's auto-save at each milestone. Three consequences follow, and they are all operational. It remembers a user across sessions — "if someone comes back next month, it still knows that they had 4am departures, Sara hates 4am departures." It survives a server restart mid-run, so the agent resumes rather than starting over. And it lets you rewind: if something went wrong at round two, you can go back to round two, because round two was saved. He notes this is also what lets one well-built agent serve a hundred or 150 users at once, and lets you scale infrastructure to meet demand.

His illustration is one everyone in the cohort has lived: ask ChatGPT to search, let it work for five or ten minutes, cut the conversation, come back five days later and say resume. It does not lose the context it gathered. That is a checkpointer.

An interrupt is the deliberate pause that puts a human in the loop. "One more jargon — the one your finance team cares about: it stops before it spends money. And that's how it has to be, because we don't want it to spend money on our behalf." The agent does every bit of the work, then freezes and waits — five seconds or five days, nothing is lost. The human approves or rejects; on approval the interrupt releases, the payment confirmation runs, and both the checkpoint and the state are updated.

Worked example · from the session

In the generated plan the interrupt appears twice: once on a 'collect missing information' node, which freezes and asks when the state lacks a field it needs, and once at the approval step before the itinerary is handed over. The first is the more interesting placement - an interrupt as a way to ask rather than guess.

Why it matters

It is the concrete answer to 'how do I let an agent act without letting it act unsupervised?'

People get this wrong

Human-in-the-loop means the agent stops and you lose the run if you are slow to answer.

The state is checkpointed, so the pause is free. Five seconds or five days is the same to it — which is what makes the gate practical rather than theoretical.

The seven words that are the actual curriculum Everything is a rule about how one shared record changes. STATE one shared booking record "nobody emails anybody" NODE a specialist, exactly one job EDGE an arrow you draw the AI never rewrites it REDUCER replace or append when writes collide CONDITIONAL EDGE the next arrow depends on a test CYCLE the arrow that goes back. Cap it. CHECKPOINTER auto-save after every step INTERRUPT the freeze before money moves 5 seconds or 5 days PARALLEL independent nodes run at once then rejoin to rebalance You do not need the Python. You need the nouns - they are what make a build prompt unambiguous.
The seven words, arranged around the one record they all act on. State is the centre; everything else is a rule about how it changes.
It stops before it spends money. And that's how it has to be.0:38
For your projects

The interrupt is the mechanism behind every 'ask me first' rule Paul has written into his own preferences and standards. This session names it, prices it at zero, and shows where it goes.

  • Add an explicit interrupt before any irreversible action in the agency's automations - send, publish, pay, delete
  • Use checkpointing to make a long extraction run resumable, so a crash costs minutes rather than a restart
Try it now
Try it now

Name the one action in your automation that must never happen unattended. That is where the interrupt goes.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Name three distinct things a checkpointer buys you.

Memory of a user across sessions, recovery from a crash without restarting the run, and the ability to rewind to a specific earlier round for inspection.

What is the difference between an interrupt and a failure?

An interrupt is a designed pause with the work complete and the state saved, waiting on a human decision. A failure loses the work. The interrupt can wait five days at no cost.

07

When NOT to use LangGraph: three disqualifiers and one signal

0:42

'Being honest with you, when you should not use LangGraph is also an important skill to know.'

Three disqualifiers. If the problem is a straight line with no decisions, no cycles, no retries — use n8n, "it is faster to build and easier to change." If one prompt to ChatGPT or Claude is enough, just ask the model once. And if someone else on the team has to be able to edit it, keep it as a linear automation even at a time cost: "it helps you to maintain with no overhead of the learning curve for different teams and different people involved." That third one is a maintainability argument rather than a technical one, and it is the one most often ignored.

The positive signal is compact: "reach for LangGraph only when there is a decision, a retry, or a human in the middle." Later he adds the shape to watch for: "wherever you see a loop and a pause and a parallel processing requirement, that's a signal for you to start with LangGraph."

And he applies it against a cohort member's real case without hedging. Rahul's marketing workflow — write scripts for videos, send to the business, get feedback, create the videos, send them, approve, edit — gets a flat no: "this is a pipeline. You are just saying step 1, step 2, step 3, step 4, step 5. Anything that is related to pipeline, I believe you should not do it in LangGraph, because it's, like, over-engineering. You can just do that in n8n."

Worked example · from the session

He is equally direct that LangGraph is not the only answer where it does fit: CrewAI solves the same problems with a different implementation. 'Think of these LangGraph or CrewAI as more like languages, programming languages, to create agents and manage agents, or orchestrate them.'

Why it matters

A trainer naming the cases where his own subject is the wrong tool is the most reusable thing in the session.

People get this wrong

More steps means you need a graph.

More steps means you need an automation. You need a graph when the steps have to be revisited, chosen between, or paused.

Reach for LangGraph only when there is a decision, a retry or a human in the middle.0:44
For your projects

The over-engineering guard. Paul's instinct is to reach for the more capable tool; this is a trainer with a commercial stake in LangGraph telling the cohort when to reach for n8n instead.

  • Write the three disqualifiers into the agency's scoping checklist so the cheap answer is checked first
  • For each existing automation, note which of the four cases it is - the ones with no loop and no pause are correctly built already
Try it now
Try it now

Take the last automation you scoped and check it against the three disqualifiers. If all three fire, you already chose correctly.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Which disqualifier is about people rather than technology?

The third. If an ops or non-technical team has to edit the automation, a linear tool wins on maintainability even where a graph would be technically better.

Why does a five-step marketing workflow not qualify, despite having several steps?

Step count is not the test. It runs forward through every step with no branch, no retry that changes anything and no pause — so it is a pipeline.

08

LangSmith: the agent tracer, and why guardrails depend on it

0:45

Nine steps. One breaks. Which one?

Without a recording you add print statements everywhere, run it again, change something, and "hope half an hour later you're still not sure." With a recording you know exactly which node, which tool call, which attempt. He walks the flights node as an example: it went to Skyscanner, looked at an itinerary, found the cost high, shifted the dates by three days, found it moderate, then did the same across Google Flights — and none of that reasoning exists anywhere unless it was traced. "If the agent goes rogue, I need to find it accountable and see where exactly it went rogue."

LangSmith is "an eye for the agent": a microscopic view of its thinking, reasoning, tool calls, steps, and why it retried. The division of labour is one line: "LangGraph is going to help you build a graph and build this agent, while LangSmith is going to keep a recording or keep a trace of what this agent built, the LangGraph, is going to do."

The part worth carrying beyond the tool is his reason. "This is the most important observability pillar every single enterprise is actually standing against." And crucially: "only when you know how the agent is working or thinking behind the lines, you'll be able to take much more steps towards guardrailing." You cannot guarantee an agent is not reading or storing personal data if you cannot see what it does. The hard gotcha, repeated in the post read, is that the data region is chosen at account level — "this can't be changed for that LangSmith account itself. This is an account-wide enforcement, not a project-wide" — because everything the agent does is sent to a cloud and stored there for observability.

Worked example · from the session

He never actually wires it. He types 'also integrate LangSmith' into Codex mid-build, offers the API key later in a .env file, and the run dies on usage limits. The to-do list he hands over carries it as item three, with the note that the built-in 'Sherlock mode' timeline should say the same things as the LangSmith trace 'if not better'.

Why it matters

It is the difference between an agent you can operate in production and one you can only demo.

People get this wrong

Tracing is for debugging during development and can be added later.

It is also the compliance surface, and the region decision that comes with it is permanent at account level — so it is a setup decision, not a later addition.

For your projects

The corpus's first proper treatment of agent observability, and it arrives with a concrete, expensive gotcha - the data region is chosen once per account and cannot be changed.

  • Add a tracing decision to the agency's standard agent-build checklist, alongside the guardrails question
  • If any client agent is ever traced, choose the data region deliberately - it is a one-way door at account level
  • Reuse his framing when explaining to a client why observability is a compliance requirement, not a nice-to-have
Try it now
Try it now

For any agent you run, ask where its decisions are recorded. If the answer is nowhere, you cannot debug it and cannot make compliance claims about it.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Why does he treat observability as a prerequisite for guardrails rather than a separate concern?

Because a guardrail is a claim about behaviour. Without a trace you cannot verify that the agent is not reading or storing data it should not, so the guardrail is an assertion rather than a control.

What is the one-way decision in LangSmith setup?

The data region, chosen at account level. Everything the agent does is sent to that region's cloud and stored there, and the choice cannot be changed for that account afterwards.

09

The vocabulary is the prompt: twelve words that replace a framework argument

0:49

'The vocabulary is how you give your orders to Claude Code.'

He is explicit that the first half of the session was not a programming lesson, and says so twice. "We don't have to necessarily understand the Python programming and all the other things around it, but we need to know what LangGraph consists of primarily." And at the end: "the idea of the session is to understand how LangGraph works, what are the typical jargons used around the ways of agents in LangGraph, and build a simple agent that gets the job done using the things that we have learned. That is the takeaway."

The mechanism is prompt specificity. Ask Claude Code to "build a travel agent that can try again if it is too expensive" and it can choose any framework and any shape, and may not get the job done. Ask instead: "add a conditional edge from a budget check; if it is over budget, go to a relaxed step that loops back to the three searches, capped at three rounds." Same request, same model, twelve words of the right vocabulary, and the build lands. "Now, we are using the actual jargon that LangGraph needs, or the Claude Code needs to understand in the context of a LangGraph."

His analogy answers the cohort member who asked what to do if you are not familiar with the terminology: "if you're talking to a civil engineer, they will appreciate if you use civil engineering jargon, and they will do the work much more efficiently with a limited number of words." The awareness is the deliverable — is it state, a checkpoint, a loop, an edge, a conditional edge, an interrupt — "because I'm sure everyone is proficient enough to talk to Claude Code and Codex of the world. This session is primarily to teach you how to talk with LangGraph kind of projects."

Worked example · from the session

He also shows the reverse asymmetry in his own build: Codex asks him nine clarifying questions in plan mode - Python or TypeScript, real LLM or deterministic mock, what V1 delivers, chat or editable fields, targeted or full replan, what Sherlock mode exposes, which LLM, what the cap means, what happens on rejection. Each answer is a term he already had. Without the vocabulary those nine questions are nine guesses.

Why it matters

It reframes framework learning: you need the nouns, not the syntax, and the nouns are an afternoon.

People get this wrong

You have to learn a framework properly before a coding agent can build in it.

You have to learn its vocabulary. The agent supplies the implementation; the nouns are what make your specification unambiguous.

The vocabulary is how you give your orders to Claude Code.0:49
For your projects

This is the session's actual thesis and the most portable idea in it: learning a framework's nouns is what lets a coding agent build in that framework, without learning the framework's code.

  • Build a short glossary skill for each framework the agency prompts into, so the right nouns are always in the prompt
  • Test the claim directly: prompt the same build twice, once vaguely and once in framework terms, and compare what comes back
Try it now
Try it now

Pick a framework you keep avoiding and learn only its seven core nouns. Then prompt with them.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

What does the vocabulary actually buy you, mechanically?

Specificity. It removes the framework and shape decisions from the model's discretion, so the build you get is the one you intended rather than one of many valid interpretations.

Does this mean you never need to understand the code?

Not quite — his claim is narrower: you do not need to write the Python to specify the design. You do need to know what the parts are called and what each one does.

10

Screenshot to plan to build: the workflow, and the two things he wished he had done

how-to1:06

Screenshot the diagram, attach it, describe it in the right nouns. Then watch the plan mode ask you nine questions you can now answer.

The method is fast and unglamorous. Screenshot the architecture diagram you have just been taught, attach it to Codex or Claude Code as context, and describe the build in framework terms — which is where the vocabulary pays off. He asks for mock data first and names the live sources for later, requests a Mermaid diagram in the README, and specifies a plain interface with a per-node visibility toggle he names Sherlock mode: "a toggle to see behind the scenes, which will unlock the audit for Sherlock Holmes mode of looking at every step."

Plan mode then interrogates him: Python or TypeScript (Python, for the clearest learning path); real LLM plus mock travel data or fully deterministic; local learning app or deployable demo (local, defer the cloud); chat or editable fields for the trip requirements; targeted replan or full replan (targeted — rerun only the affected searches while retaining valid results, with the audit explaining which constraint drove the retry); what Sherlock mode exposes; which LLM; what the cap means; and what happens if the user rejects the plan. He answers all nine, notes he should have started in plan mode, and then names the better instruction: tell it to go with its recommended answers for everything it is about to ask.

The plan that comes back is worth reading as an artefact: nodes to interpret the request, validate requirements, collect missing information with an interrupt, plan the searches, search flights, stays and activities, compose an itinerary and evaluate the plan; conditional routing for revise-strategy and request-approval; the state and interface definitions; mock providers; the UI; a test plan; and an assumptions section that hands his own ambiguities back to him — budget as a hard ceiling, currency conversion out of scope, three retries meaning the initial itinerary plus at most two replans, a rejection treated as new information that resets the counter, and the app recommending but never reserving or purchasing.

Worked example · from the session

The build fails and a cohort member's succeeds. Rajesh had built a working version - including an LLM-wired variant - and his demo is what the class actually sees: click Create My Plan, and Sherlock mode narrates planning attempt 1, requirements validated with no missing fields, a strategy prepared, parallel searches fanned out, flights and stays retained, the itinerary combined at 43,500 against a 60,000 budget, and the budget evaluator deciding whether to approve or retry. Then Revise Itinerary, which asks what should change.

Do it in this order

Gotchas["He did not start in plan mode and said so mid-build: 'I should have started this in a plan mode.'", "Nine clarifying questions later he identified the meta-fix he wished he had used: 'I should have said, go with your recommended answers for all your questions you're about to ask.'", 'His app never ran. Two separate Codex accounts hit usage limits during the session; he switched model and effort level and still could not finish.', "The mock version cannot revise anything, and he says so: 'nothing will ever be revised, because this is just a mock data... the AI part is missing. The moment you plug in an LLM, I think you should be very good to go.'", "'It's burning tokens like anything, yes' - agreed with a cohort member. A visible audit UI over a graph agent is not a cheap build."]

Why it matters

It is the whole session's payoff method, including the two corrections he makes to it on air.

People get this wrong

Plan mode is an optional slow path for cautious people.

It is where your instruction's ambiguities get named. He skipped it, noticed, and said so — and the assumptions it produced are the most useful part of the transcript's build half.

For your projects

A build method Paul can copy directly - and a live demonstration of what running out of model credits mid-demo looks like, which is the risk his own credit discipline exists to manage.

  • Adopt the screenshot-as-context habit: diagram first, screenshot, attach, then describe
  • Add 'go with your recommended answers for every question you are about to ask' as a standing opener for plan-mode builds
  • Never demo a build live on a single account near its usage ceiling - he burned two in one session
Try it now
Try it now

Screenshot a diagram of something you want built and attach it before writing a word of the prompt.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Why attach a screenshot rather than describe the architecture in prose?

It is faster and less lossy — the diagram already is the specification. The prose then only has to supply the vocabulary and the constraints the picture cannot carry.

What did the plan's assumptions section actually do for him?

It surfaced the ambiguities in his own instructions — what the retry cap counts, whether a rejection resets it, whether currency conversion is in scope — before any code was written.

11

The handover to-do list: live data, a tracer, and ten edge cases that become your eval set

2:03

Ten edge cases where it can break. That is an eval set, and it costs an afternoon.

Rather than finish the build, he writes the handover as a to-do list and shares the document. Four items. Connect the LLM to the app after the UI mock is done — the mock revises nothing without it. Add Firecrawl and Skyscanner APIs for real-life ingest, replacing the mock providers. Add LangSmith as the tracer, with the API key from the LangSmith console and the data-region caveat. And: "create 10 edge cases for testing the app. Edge cases are where the app can break, where the human in the loop can be assumed — using AI, of course, using ChatGPT or anything."

Then the definition, stated plainly: "essentially, you are creating an instruction set or an eval set. This is what we call as an eval set. Creating a bunch of edge cases where the model can break is an eval set to essentially give a benchmark of your entire agent."

And the loop that follows, given in answer to a question about testing a customer-support bot before production. First, guardrails: "does your app have guardrails? What if it is taking customer data and storing it somewhere? What if it is not anonymising a social security number, but just storing it as is? Because these are against the GDPR standards." Second, the eval set: "if you have 100 edge cases that you have tested, and your AI is performing 80%, your benchmark is 80%." Then take that number back to a model — here is my set, I am getting 80%, how do we optimise, what are the places we can optimise, what better decisions can we make so we can scale it. "Eventually, you'll get an understanding or suggestions from AI itself in this process."

Worked example · from the session

The deliberate ordering is the teaching point: guardrails come before the benchmark. A support agent that scores 95% while storing unredacted identifiers has failed a test the eval set was never going to catch.

Why it matters

It is the lowest-friction definition of evaluation in the corpus, and it converts 'is it good enough?' into a number you can act on.

People get this wrong

Evaluation means a formal framework and a dataset you do not have.

It means ten edge cases you wrote down, run repeatably. The percentage is the whole apparatus.

For your projects

A four-item production checklist and, more usefully, the cheapest possible definition of an eval set - ten edge cases and a percentage. It is directly applicable to the agency's own skills library.

  • Write ten edge cases for one existing agency automation and score it - the first real benchmark it will have had
  • Hand the failing cases back to a model and ask what to optimise, which is exactly his loop
  • Add the guardrails question - does it store or fail to anonymise personal data - to the pre-launch checklist for anything customer-facing
Try it now
Try it now

Write ten ways your automation could break, run them, and count. You now have a benchmark you did not have this morning.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Why does the guardrails check come before the eval set rather than inside it?

Because they test different things. An eval set measures whether the agent does its job; a guardrail check asks whether it mishandles data while doing it. A high score does not clear the second question.

What makes a percentage more useful than a judgement about quality?

It is comparable. You can hand the number and the failing cases to a model, change something, and re-score — which is an optimisation loop rather than an opinion.

12

LangGraph versus loop engineering: wiring specialists, or trusting one good node

0:27

Asked point blank whether loop engineering does this better, he declines to call it the next evolution.

A cohort member asks the question twice — early, and again at the end with cost and speed attached. His answer is consistent. A graph divides work into nodes where each node is a specialist agent and you wire the directions and the loop-backs. Loop engineering is "a little bit of more advanced and abstract form of graph, where we just worry about the loops because the agents themselves have become more prominent and more better. I don't have to divide them into sub-agents or nodes to connect and talk to each other."

So the trade is specificity against trust. "Given the access to higher tiers of model, you don't have to worry about the nodes anymore, you just have to describe a goal. That is the context or premise of loop engineering." But: "here, one node inside the LangGraph is a specialist agent. In loop engineering, one node can do 15 different things, and you're assuming that the node will do that exactly as is. And there is no way for us to trace their activity also. Hence it becomes a little complex." On cost he is relaxed — "loop engineering is not super expensive, I think it's one of the easiest ways to do, because you're just defining when to loop and how to loop."

Then the correction that makes the concept worth keeping: "don't think about loop engineering as the next evolution. ChatGPT 3.5 can only do 10 things. Now ChatGPT Astra can do 100 things. Does that mean we have increased loops or cycles? It's just the inherent nature of the model that has just evolved, to become much more goal-oriented by just a mere description of an abstract problem statement." Loop engineering is not a newer technique; it is the same technique made viable by better models. He also predicts convergence — LangChain improving its docs for loops, "or they will call it LangLoop, whatever it is."

Worked example · from the session

He places CrewAI alongside LangGraph rather than against it: the same problems, a different implementation, and both best understood as 'languages, programming languages, to create agents and manage agents, or orchestrate them.'

Why it matters

It stops a false progression narrative between two sessions in the same corpus, and names the real axis: how much you need to trace.

People get this wrong

Graph frameworks are the older, heavier way and loops are the modern one.

They sit on a trust-versus-traceability axis. The graph is the choice when you need to explain a failure; the loop is the choice when the node is good enough and you do not.

Don't think about loop engineering as the next evolution. It's just the inherent nature of the model that has evolved.1:59
For your projects

Session 15 taught loop engineering and this session teaches graphs, and Paul would reasonably wonder which supersedes which. The trainer answers it directly, and the answer is 'neither'.

  • Keep both shapes in the toolkit and pick by traceability need, not by recency
  • Watch the LangChain docs for loop-oriented guidance, which he predicts is coming
Try it now
Try it now

For a job you would currently give to one capable agent, ask whether you would be able to explain a failure afterwards. If not, the graph is buying you something.

Check yourself

Answer from memory first — the recall attempt is what makes it stick. Then reveal.

Is loop engineering a replacement for graph orchestration?

No. He explicitly rejects the progression framing: the shift is that models got capable enough to be given goals instead of steps, which changes what is viable, not which technique is correct.

What do you give up by describing a goal instead of wiring nodes?

Traceability and attribution. One node doing fifteen things is one node you cannot inspect step by step when it goes wrong.

Pull quotes

I asked them one question: how many of you have heard of the startup Lovable? The answer is zero.1:49

Tools referenced

ToolCoverageMomentContext
codexdemonstratedThe build environment. GPT-5.6 on medium, then Saul Light, then a data model at extra-high effort. Screenshot attached as context, plan mode used belatedly, and TWO accounts exhausted on usage limits during the session.
LangGraphexplainedThe session's subject. Explained end to end - state, nodes, edges, parallel fan-out, reducers, conditional edges, cycles, checkpointers, interrupts - and prompted into Codex, but never actually running by the end of the session. Python chosen over LangGraph JS on air.
LangSmithexplainedAgent tracer. Explained thoroughly and typed into the build prompt; never wired, no key supplied. The account-level data-region constraint is the one hard operational fact captured.
langchainexplainedNamed as the parent company of LangGraph, LangFlow and LangSmith, and disambiguated for the cohort in four lines.
n8nexplainedThe counter-example throughout: left-to-right, trigger-driven, cannot loop back. Named as the correct choice for linear work and for anything an ops team must maintain.
chatgptexplainedThe 'attempt one' counter-example - confident plan, no real prices, no budget check - and also the recommended tool for generating the ten eval-set edge cases.
LangFlowmentionedThe no-code drag-and-drop sibling for building similar graph-based agents - 'consider this is more like an n8n for building agents'.
CrewAImentionedNamed as a peer that solves the same problems with a different implementation; both framed as 'languages' for orchestrating agents.
claude-codementionedOffered as the alternative to Codex - 'the concept is same, the tool might be different'. The cohort chose Codex; he later regrets it ('I should have started with Claude Code').
firecrawlmentionedNamed as the live-data source to add after the mock stage, for scraping flight and hotel information.
SkyscannermentionedNamed alongside Firecrawl as the live flight-data source for a later iteration; also used as the worked example of an untraced node's reasoning.
claudementionedNamed alongside ChatGPT as attempt one, and as an option for analysing eval-set failures.
MermaidmentionedRequested in the README - 'a fancy way of saying a flow chart'.
AgentGridmentionedSuggested by a cohort member as a way to continue when one model's tokens run out; he had not tried it.
JEVmentionedA new speed-optimised model discussed at length during the build wait - waitlist, benchmark table, token pricing. He is sceptical of the benchmarks and of the speed/accuracy trade. Name as-heard; spelling unverified.

Session materials

Archived locally on V: — click to open. Companion pages link to the LMS.

Action items

Resources mentioned

Resources
  • docThe vocabulary-versus-vagueness prompt pair (the session's central demonstration) 0:49
  • docThe live Codex build prompt, assembled on air - NOT verbatim 1:08
  • docThe meta-instruction he wished he had opened with 1:21
  • docThe four-item handover to-do list (shared as a document with the cohort) 2:03
  • docSlide deck: 'Build a Travel Agent That Changes Its Mind — LangGraph & LangSmith' 0:14
  • docLangSmith console (data region is chosen here, at account level, permanently) 2:03
  • docRajesh's working demo — the build the class actually saw 1:53

Extraction notes

This page was built from an auto-generated transcript, which garbles product and people's names. Those were corrected silently in everything above and logged here for transparency. The warnings flag claims that were true on the recording day but change fast.

Transcript corrections applied

The transcript saysThe trainer actually means
Langraph / LangGroup / Lang graphLangGraph
Langsmith / Lang SmithLangSmith
LangflowLangFlow
Anaton / NATIN / NATOM / Nate in / NA10 / Annator / NHL / Enhanced workflown8n (systematically garbled throughout)
codecs / CodexCodex
Cloud Code / clot / CloudClaude Code / Claude
red user / reduce a particular statereducer
Virtual lock mode / Sherlock Holmes modeSherlock mode (his name for the per-node audit toggle)
Saul / Saul Light / SOLa Codex model tier, name as-heard and unverified
Astra / GPT 5.64 / 5.6V / GPT-6 AstraOpenAI model names as-heard; versions unverified
Crue AI / Krue AI / Cruel AICrewAI
JEF / JEV / Jav / Jab / J-E-B / GIFJEV (a new model discussed during the build wait; spelling unverified)
Fire crawlFirecrawl
Prothal / Prithel / Prithu / Truffle / Profil / ProfflePraful (the cohort member); the ASR is especially unstable on this name
Bengaluru / BangaloreBengaluru (both used interchangeably)
1,35,000 / 1,20,000 / 1.2 lakhIndian numbering; 1.2 lakh = 120,000 INR
Nano BananaNano Banana (Google's image model - correctly heard)

True on recording day — verify before relying