Module 1 · Part B of 3

The ReAct Loop

An agent built on a language model runs the same perceive-reason-act loop as anything else, with one detail that gives the pattern its name: the model reasons in words between the actions it takes. Yao et al. (2022) called this ReAct, for reasoning and acting. They found that interleaving short reasoning traces with tool use handles many tasks more reliably than planning everything first or acting with no reasoning step.

A tool is code your harness can run for the model: a named function with a description, an input schema, and a return value. A tool call is the model's structured request for that code. The harness parses the request, runs the function, appends the result, and calls the model again.

This loop is the foundation that most general-purpose agent frameworks are built on, LangChain's createReactAgent among them, and the rest of this module works up to it.

The loop is unchanged. With a language model in the reasoning step, the agent reads tool results and acts by calling tools.

The one field that tells the loop what to do next

Every chat-completions response carries a finish_reason field that tells you why the model stopped generating. Inside an agent loop, this is the only signal you need to decide what to do next: run a tool, return the answer to the user, or handle an error. Before reading what its values mean, make two calls and watch the field change.

The first call answered on its own and came back "stop". The second was offered a tool and asked for something training data cannot supply, so instead of stopping it came back "tool_calls", handing your code the name of the function to run. Those two values cover the common branching. The rest signal trouble that routes the loop into a fallback harness, where the agent might retry the call or hand the conversation off to a human.

"stop" means the model has written its final answer and the loop can return content to the user.
"tool_calls" means the model is asking your code to run a named function and append the result before calling back.
"length" means the token budget ran out before the answer finished.
"content_filter" means a safety classifier stopped the output.

In modern ReAct, a short router reads that field. If the model asked for a tool, the router runs your function, appends the result, and calls the model again. If the model stopped, the router returns the answer.

The arrangement is strong because the model is never bound to the plan it started with. Whenever a tool returns something it did not expect, that result joins the next round of reasoning, and the model can reconsider where it stands before deciding what to do next.

Applied · AnatomyMap the model, memory, tools, and router?

The pieces an LLM agent is built from

Zoom in on the agent and you find the decomposition Lilian Weng drew in 2023. A language model sits at the center. Around it sit memory, planning, tools, and the actions that reach the world.

This course's minimal agent fills each slot with the simplest thing that works:

Memory is the messages[] array.
Planning is the finish_reason router.
Tools are the functions your harness exposes.
Action is the tool call the harness executes.

None of that lives inside the model. It is the harness you build around a single text-in, text-out call.

After Lilian Weng (2023): the model is the agent's core controller, with memory, planning, tools, and action wired around it.

The modules are independent. You can replace the model or the tools or the router on its own, and as long as each one still honors the contract the others expect of it, the rest of the loop keeps working.

That independence also matters for how you read agent code you didn't write. Whatever framework wraps it, any LLM-based agent must contain these four pieces, and when something breaks the cause is usually in one of them:

the model returning the wrong finish_reason,
a missing turn in the messages array,
the tool executor handing back an unexpected value,
or the router misreading the signal.

The four-piece frame gives you a checklist that applies regardless of how the surrounding code is organized.

Building the loop from scratch with one tool

As an exercise, we can try to create a simple loop from scratch. All we need is a single tool named get_current_time plus one system prompt and one while loop. Run it with "What time is it in UTC?" to start, and see what else you can ask:

The flow is short: the model asks for the tool, the executor appends the timestamp as a tool-role message, the model reads that timestamp, and the loop exits when finish_reason reads "stop".

Think, act, observe, repeat. The turn exits the moment finish_reason is "stop".

What just happened inside the messages array.
The loop pushed four things into state.messages:

the system prompt (turn 0),
the user question (turn 1),
an assistant message with a tool_calls field (turn 2, the request),
and a tool-role message holding the function's return value (turn 3).

The second LLM call read all four of those including the new tool result and wrote the final answer. The growing array is the only "memory" this agent has. If you saved that array to disk and reloaded it next week, the model would resume the conversation without noticing the gap.

Applied · OperateNeed context-rot evidence and mitigations?

Context rot: why a longer loop reads worse

This issue becomes more likely as the context grows. Context rot is the tendency for contradictions, stale observations, and poor conventions to accumulate in the prompt history until the model's next decision gets worse. The effect also has a positional component: Liu et al. (2023) showed that models can underuse information buried in the middle of a long context. The further an important signal drifts from the current turn, the easier it is for the loop to ignore or contradict it.

Redrawn from Liu et al. (2023). As the loop appends turns, the answer you need drifts into the weak middle.

Common mitigations keep each decision close to the evidence it needs:

plan before the loop accumulates observations,
delegate sub-tasks so each sub-agent works on a short context,
or summarize earlier turns before they become a liability.

Each of those costs something in latency or fidelity or complexity, and each is worth weighing against the task at hand.

The same loop, prebuilt and pointed at this course

The loop above hard-wires one tiny tool. LangChain's createReactAgent generalizes the same machinery to support more tools, standard interfaces, and observability integrations. An instance of this class is wired below to a single tool, read_course_page, which hands the agent any page of this course as markdown. The model decides which pages to read for your question, so give it a try.

The cell produces an artifact. When you run it, you get a working chat panel rather than a wall of logs, rendered from the agent's own code, that you can hold a conversation with. The word itself is older than its current use, since build pipelines and machine-learning training runs have produced "artifacts" and "model artifacts" for years. Anthropic's recent live artifacts sharpened the idea for agents, where an artifact becomes a persistent, interactive HTML surface that the agent builds and that you keep using long after the reply which produced it. The panel below builds one on a small scale, and an agent that can hand back a full interface rather than plain text is a thread that runs through the rest of this course.

Run the cell to build the panel, then chat with it. Pin one or more with the Pages chips to ground every answer in them, or pin nothing and let the agent choose what to read. The reply renders as markdown, with a chip for each source the agent read so you can expand it to see the text it pulled back. Alongside the answer, you also get the reasoning trace and the token usage. It remembers the conversation per chat. Edit the cell and rerun to rebuild the artifact.

Before you continue

Each question is answerable by editing inputs in the canvas above and rerunning.

Force the model to call the tool even when it does not need to. Try a question the tool genuinely cannot answer ("capital of France?") and watch finish_reason on step 1. Was it "stop" or "tool_calls"? Now edit the system prompt to require a tool call before answering ("You must always call get_current_time once, then answer."). Does the model comply? What does that tell you about how strongly the system prompt steers a model that has training-data confidence in another direction?
Count the steps on a genuinely tool-needing question. "How many minutes until 17:00 UTC?" takes more than one tool call. Run it a few times. Did the model call get_current_time once, twice, or three times? Why would a model re-call a tool whose output it already has, and how would you discourage that in the system prompt?

Deep · SourcesTrace these mechanisms to primary sources?

References

Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022). The original alternating thought / action / observation pattern; the figure above is redrawn from this paper.
Wei et al., Chain-of-Thought Prompting (2022). The "reason" half of ReAct; what the model is doing inside reasoning_content when it deliberates before acting.
Lilian Weng, LLM Powered Autonomous Agents (2023). The decomposition the figure above adapts: a language model as the agent's core controller, surrounded by memory, planning, and tool use.
Xu et al., ReWOO: Decoupling Reasoning from Observations (2023). A deliberate contrast to ReAct: the model writes the whole plan up front and a separate executor runs it without re-querying the model between steps. Comparing the two reveals the trade-off between reactive replanning and pre-committed execution.
Schick et al., Toolformer (2023). Covered again in 1c's references; revisit here for the underlying mechanism the ReAct loop exploits.

Comprehensive list at Going Further · References.

← Module 1a: The Agent Module 1c: Tools at Scale →