ReAct, the Cut String, and Outsized Expectations of a Technology Still Too Young

The previous part ended at a basic limit: a prompt can steer the answer, and RAG can put more documents into the context, but both mostly change the condition of the same prediction:

A prompt writer arranges the words by hand. A RAG pipeline finds the words and inserts them by machine. The methods differ, but the direct effect still lands on $h_t$, the context the model sees before it generates the next token.

At this point an idea becomes almost unavoidable: if the context decides the next step, why not let the model build its own context? Let it write a plan, pick a tool, read the result, revise the plan and repeat. The context is then no longer a static block prepared by a human in advance. It becomes a tape that keeps growing over time.

That is the moment the agent walks on stage.

The vision was not small. An agent could observe, plan, call tools, check its own work and coordinate with other specialised agents. If it worked, software would no longer just wait for a human to press each button; it could organise work into a goal-directed sequence of actions.

The problem was never the vision. The problem was that the vision arrived while LLMs were still very young, and the tool interfaces and runtimes that agents need had barely begun to form. In the 2022 and 2023 wave, most systems had to assemble that future from very raw materials: a model that generates text, a parser that cuts a piece of text into a command, and a runtime that calls a function and pastes the result back into the context.

ReAct and the first multi-agent systems are therefore best seen as architectural sketches drawn ahead of their substrate. They saw the future quite accurately: reasoning must connect to action; action must receive feedback; large pieces of work must be divided into roles. But they had to realise that vision with models that were unstable about format, contexts that were still short, tool calling that was not yet a standard, and a recovery mechanism that mostly consisted of asking the model to say it again.

This essay dissects that historical gap: from a model proposing actions in language to a system executing actions that are structured, permissioned, stateful and backed by evidence.

That boundary once sat on a regular expression.

I. Before ReAct, there was a router

In May 2022, Karpas and colleagues published the MRKL architecture, short for Modular Reasoning, Knowledge and Language.¹ At the same time AI21 Labs introduced Jurassic-X, its own MRKL implementation.

MRKL's intuition is clear: do not force a language model to do everything. A system can consist of many specialised modules, such as a calculator, a database, a knowledge base or a reasoner. A router receives a natural-language request, chooses the right module and passes it the arguments it needs. Only when no expert fits does the request go to the general language model.

What matters is the division of labour. The calculator is responsible for arithmetic. The database is responsible for returning records. The language model does not need to pretend to be an unreliable calculator when the system already has a real one.

MRKL also points straight at the hardest part: from a human sentence, how do you extract exactly the arguments the specialised module needs? For an arithmetic question the system must not only pick the calculator; it must extract the right operation and the right operands. For an API it must identify the right function name, the right fields and the right type for each value.

The AI21 team did not treat this as something a well-phrased instruction could solve. In their operand-extraction experiment they used J1-Large 7B with prompt tuning: the pretrained weights stay frozen and only ten prompt tokens are learned. An appendix of the paper puts prompt tuning next to few-shot prompting with ten examples and concludes that the few-shot approach's performance "is limited": when the numbers are longer than those in the examples, few-shot drops sharply while prompt tuning stays highly accurate.

MRKL therefore holds an important intuition that the later agent wave kept chasing: the router must be a real component of the system, not a tone of voice. To turn words into commands, you have to design and evaluate the interface between the model and the executing module. That is exactly the missing substrate that early experiments were forced to simulate with prompts.

In late 2022 LangChain appeared and quickly popularised building applications out of chains of model calls, tools and memory. MRKL-style and zero-shot ReAct agents in LangChain used a very pragmatic solution: the prompt asked the model to write in the Thought, Action, Action Input format, and then a regex living directly inside the agent code read that string to build an AgentAction. Only in mid-April 2023 was this string-cutting pulled out into a separate output-parser class.

The approach was cheap, flexible and easy to change. It also moved the burden from a separately trained and evaluated router onto a contract written in prose. The architecture still had a router and tools, but the routing decision and the model's explanation were generated on one text channel and only separated at the end.

That choice laid the foundation for the first agent era: an architecture aimed at the future, propped up with the text interface of its day.

II. ReAct: when context becomes a loop

In October 2022, Yao and colleagues posted ReAct, short for Reasoning and Acting, to arXiv.² Instead of only generating a reasoning chain like Chain-of-Thought, or only issuing commands like acting-only methods, ReAct interleaves three kinds of step:

• Thought: the model records reasoning, a plan or a change of strategy.
• Action: the model chooses a valid action in the environment.
• Observation: the environment returns a result for the model to use in the next step.

The Thought here is nothing new. It is the very Chain-of-Thought the previous part dissected: a scratchpad the model writes into its own context, not the state of the system. What ReAct adds lies in the other two beats.

We can describe a ReAct loop with the following conceptual model. Let $h_t$ be the context at step $t$, $\tau_t$ the string the model generates, $\Pi$ the parser and $\mathcal E$ the environment:

The symbol $\oplus$ here is not necessarily raw string concatenation. It stands for however the runtime folds the generated step and the environment's result into the new context, ideally with clear boundaries and metadata.

Read the figure clockwise. The model reads the whole tape below and writes $\tau_t$, a piece of text with three labels. The scissors $\Pi$ cut out what follows Action: and Action Input:, and only that thick rust arrow reaches the environment. The other two arrows both pour back into the tape: the dashed one is text the model wrote, the blue-grey one is what the world returned. On the tape the two kinds of cell sit side by side, and to the eye almost nothing tells them apart except the label.

Seen this way, ReAct makes an important change: the context gains a time axis. The model does not just receive information and answer once. It can use an Observation to revise a hypothesis, change a query or choose a different action.

Still, Thought and Observation enter the next inference as representations in the context. A Thought is a note the model wrote itself. An Observation is a result supplied by the environment, after the runtime has encoded it to feed back to the model. The part that actually touches the environment is $a_t$, and that action exists only when the parser and the runtime accept the model's proposal.

This does not make ReAct "just text". ReAct really can interact with Wikipedia, move through ALFWorld and operate in WebShop. But an action takes effect not because the string Action: carries any power of its own. It takes effect because an external program interprets that string and calls the environment.

ReAct's experimental results are also subtler than the story usually told. On HotpotQA with PaLM-540B, ReAct scores 27.4 exact match, below CoT's 29.4. The often-quoted 35.1 belongs to a strategy that switches from ReAct to CoT-SC when ReAct gets stuck, not to ReAct alone. On FEVER the direction flips: ReAct reaches 60.9 against CoT's 56.3. The paper reads these results candidly: retrieval helps the model invent fewer facts, but the rigid Thought, Action, Observation format also makes it less flexible when it has to chain several reasoning steps. The paper also records a failure mode specific to ReAct: the model repeats its previous Thought and Action and cannot break out of the loop.

On the other side, ReAct makes a clear leap on interactive benchmarks. On ALFWorld it beats the imitation-learning baseline by 34 percentage points of success rate while using only one or two in-context examples; on WebShop the absolute improvement is 10 points. These are significant results, and there is no need to talk them down to defend this essay's thesis.

What needs separating is the scope of consequences. ALFWorld is a text-simulated world; WebShop is a simulated shopping environment; the Wikipedia API mostly serves search and reading. Actions in those benchmarks have state and can make a task succeed or fail, but they are not equivalent to moving real money, deleting production data or emailing a customer. A wrong step in ALFWorld costs one attempt; a wrong step in production may not be recoverable.

ReAct proved that reasoning traces and environmental feedback can work together effectively. It did not prove that a command interface written in prose is safe enough for every environment with side effects.

III. Toolformer and the fork in tool use

In February 2023, Schick and colleagues published Toolformer.³ Where ReAct mostly teaches tool use through the prompt and in-context examples, Toolformer intervenes in training.

The team used a handful of demonstrations per API so that the model itself proposed where and with what content to insert tool calls across a large corpus. Those calls were executed and the results spliced into the text; the system then kept only the calls that reduced the loss on the following tokens, by at least a threshold compared with both not calling and calling without the result. Finally, GPT-J 6.7B was fine-tuned on the dataset augmented with tool calls.

Toolformer still represents API calls as tokens. The tool's input and output are linearised and placed in the text between delimiters; in practice the authors used perfectly ordinary characters such as [, ] and -> so they would not have to change the vocabulary. So at deployment time there still has to be a mechanism that recognises a call, executes the API and returns the result. The difference is that the grammar of tool calls has become part of the fine-tuning data rather than a convention described in the prompt.

In other words, ReAct and Toolformer do not represent "text" versus "system". Both need a runtime. They differ in where the tool-calling behaviour is learned:

• ReAct shows the model the action format in context.
• Toolformer fine-tunes the model on strings containing API calls filtered by a self-supervised signal.

In early 2023, the ReAct style of implementation was more attractive to most product teams. It did not require the right to fine-tune a frontier model, tools could be changed by editing a prompt, and demos came together very quickly. LangChain and many other frameworks turned the pattern into a few lines of configuration.

The price of convenience was a fragile interface. The model was asked to reason in prose and to emit a command with exact syntax, in the same generation. The parser had to find the boundary between the two after the string had already been produced.

When the parser catches it, the system looks as if it has a pair of hands. When the parser slips, you finally see what those hands are wired to.

IV. The parser: the cut between words and actions

Early ReAct-style agents typically ran a simple loop:

1. Put the request, the tool list and the history of Thought, Action and Observation into the prompt.
2. Call the model for the next step.
3. Parse the output into an AgentAction or an AgentFinish.
4. If there is an action, call the tool and feed the result back into the loop.
5. If there is a final answer, stop.

LangChain's documentation of the time describes AgentExecutor almost exactly this way.⁴ The custom-agent example also uses the stop sequence \nObservation: and explains why quite bluntly: without stopping there, the model "may hallucinate an observation for you".

In LangChain's MRKL code in early March 2023, the output was split with the regex Action: (.*?)\nAction Input: (.*). A commit on 9 March, meant only to accept blank lines between the two fields, had to edit that very expression, changing \n into [\n]*.⁵ The detail is small, but it shows where the contract between model and runtime lived at the time: in how many newline characters a regular expression was willing to skip.

We can abstract that interface as:

where $\Sigma^*$ is the set of all strings that can be generated, $\mathcal A$ is the structured action space, and $\bot$ denotes a parse failure.

For a context $h$, the interface's syntactic error rate can be written:

This is a conceptual model, not a claim that every agent framework uses the same parser or that one $\varepsilon$ can be measured directly across them. It helps separate two questions that are often merged into one:

• Did the model choose the right action?
• Can the parser read the model's string as an action at all?

A system can fail at both layers. Valid JSON can still call the wrong tool. An action with the right intent can still break because of a missing quote. A parser failure is an interface error; a semantic failure is a decision error. Fixing the first does not fix the second.

The conceptual model also shows why a parser that looks "pretty good" in a demo breaks on long tasks. If each step has syntactic error rate $\varepsilon$ and we treat the steps as independent for a moment, the probability of getting through $T$ steps without a single slip is

With $\varepsilon=5\%$, a one-step demo succeeds 95% of the time. But on a twenty-step task, $0.95^{20}\approx 0.36$: nearly two runs in three hit at least one $\bot$ from the parser. And this estimate is optimistic, because it assumes $\varepsilon$ stays still. The next section shows that inside an agent loop, $\varepsilon$ can climb with the very failures that came before.

A stop sequence is not an authority boundary either. It is a mechanism that halts generation when a token pattern appears. If the model uses a different variant, or the backend does not apply the stop as expected, the model can go on to write a line labelled Observation: itself. Formally, that line belongs to $\tau_t$, not to the result $o_t=\mathcal E(a_t)$.

This is the dangerous point: two strings can look identical to a reader and have completely different provenance. One is the output of a real tool. The other is a completion the model imagined. If the runtime simply appends both to one transcript without trustworthy metadata, the next step can hardly tell "the world answered" from "the model just wrote something that sounds like the world".

The four boxes on the left all say the same thing: look up the steel price. A human reads all four the same way. The regex accepts only the first; adding numbering, bolding the labels out of markdown habit, or saying it as a natural sentence all fall into $\bot$. The bottom strip is the other half of the problem: the dashed cut line is where the stop sequence should have stopped generation. If it does not, the rust line Observation: $640/ton is written by the model itself, while wearing the world's label.

Temperature zero does not remove the problem. It can make a single call more stable on the same input, but the context inside the loop keeps changing. Every Observation, error message or new piece of history creates a different generating condition. Deterministic decoding does not turn a contract written in prose into a type system.

V. When a parser error comes back as a sentence

A common choice in agent frameworks is to send the parse error back to the model so it can fix itself. Around late April 2023, a few weeks after the regex commit above, LangChain added the handle_parsing_errors option: when enabled, an error from the output parser is turned into an Observation for the LLM on the next round.⁶ OutputParserException also lets you attach the broken output and a description so the model can try again.

This is a reasonable strategy. Many small format errors can be fixed on the very next attempt without stopping the whole task. AgentExecutor also does not let the loop run forever: by default it stops after fifteen iterations. But that cap counts steps; it does not distinguish a step that made progress from a step that was just one more apology. The problem appears when retries have no error classification and no exit per error type.

Let $\alpha_t$ be the parser message fed back into the context. The next step becomes:

No theorem says that

There is even reason to suspect the opposite. The previous part showed how strong in-context learning is: a few examples placed before the question are enough to pull the model into a pattern. That mechanism does not distinguish good examples from bad ones. Every failed round leaves one more off-format output on the tape, and the further the loop goes, the more the bad examples outnumber the single correct template in the original prompt.

The error message may help the model fix things. It may also lead the model to explain, apologise, wrap its output in a code fence, or change format in a new way. AutoGPT issue #623 from April 2023 records exactly this kind of failure.⁷ The user was running in GPT-3.5-only mode; the system sent the broken JSON to another call to repair, but the "repair" call came back with a prose apology saying it could not return a valid value without knowing the schema. The result was still missing the command field and failed again. The contract AutoGPT demanded at the time was a JSON block with thoughts and command, described entirely in words inside the prompt.⁸

That failure mode can be called the apology loop. The name does not claim that every RLHF-tuned model inevitably apologises, nor that it is a universal law of probability. It describes a specific technical loop:

1. The model returns output in the wrong format.
2. The parser turns the error into a natural-language message.
3. The model responds to that message as if it were in a conversation.
4. The new response still does not match the machine grammar.
5. The system repeats without getting any closer to acting.

The loop on the left is those five steps folded into four stations. Every arrow works; no station is broken in the technical sense, which is exactly why the loop never stops on its own. The right column is the tape after a few rounds: one correct template at the top, and below it broken output, parser errors and apologies stacking up in turn. The vertical arrow repeats the argument above: the further down you go, the more bad examples the model sees.

This is context engineering polluting its own context. Each failed round adds a broken output, an error message and possibly an explanation. If the whole history is kept, the model on the next turn has to rediscover the syntactic contract inside a transcript that grows longer and holds more and more bad examples.

By mid-2025, a more modern version of this failure mode surfaced publicly in reports about Gemini CLI. One user described the agent repeatedly adding and removing the same piece of code, apologising after each failure, then going back to the exact solution that had just not worked.⁹ That issue was closed as a duplicate of a family of infinite-loop bugs tracked in a parent issue. In the parent issue, a contributor later summarised that these loops mostly came from bad tool usage: wrong parameters, tool failures and the like.¹⁰ Another issue asked for Gemini to be "less apologetic" after each correction; a maintainer replied that this was a bug in the model, not in Gemini CLI.¹¹

A comment in that same parent issue shows what the loop looks like up close.¹²

The screenshot is only a slice of the loop; the poster says the real repetition ran more than three times longer than what fit in the frame. Read top to bottom, the cycle has four beats and repeats verbatim: a lead-in sentence, "I see, the previous replacement failed…", a ReadFile, a second lead-in, "I've reviewed the errors…", and then an Edit that fails with the same message, Failed to edit, could not find the string to replace. There is no apology here, but the shape is exactly the shape of this section: the error comes back as a line of text, the model narrates that error in prose, then replays the very same action. Two small details are worth noticing. Every Edit still carries a green tick, because the tool call itself completed, while the real failure sits in the text of the result: a runtime event demoted to a sentence. And the poster asks whether temperature defaults to 0. As section IV argued, the answer barely matters: this loop does not come from sampling randomness, but from a tape that keeps handing back the same condition.

The public sources do not support concluding that Gemini implements the exact ReAct algorithm from the paper, much less that ReAct directly causes the model to apologise. The more accurate reading is this: the apologising belongs to the model, while an agent loop shaped like ReAct can amplify it. When a tool call fails, the error and a self-critique are fed back into the context; if the model does not change strategy and the runtime has no loop breaker, the apology becomes new tokens on the tape rather than a recovery signal. ReAct does not create the tendency to apologise, but an uncontrolled feedback loop can turn that tendency into a repeating chain. That the same shape of failure reappears two years later, on a far stronger model, shows this is not only about early models being weak. It is about how the loop is designed.

The fix is not to ban retries. The fix is to put retries back where they belong in the runtime:

• distinguish syntax errors, schema errors, authorisation errors and execution errors;
• cap attempts per error type, not only by total step count;
• do not let broken output become a demonstration that steers later turns;
• use structured error codes instead of a long complaint;
• stop hard when an error cannot be recovered;
• record a trace so you know at which node the loop broke.

A runtime error must keep its status as a runtime event. If every error is turned into a sentence for the model to "please understand", the system has handed the recovery mechanism to the very component that produced the error.

VI. Virtual departments: a vision that came too early

In spring 2023 the agent idea quickly grew into multi-agent systems. AutoGPT tied planning, self-criticism, memory and commands into one loop. BabyAGI separated task creation, prioritisation and execution.¹³ CAMEL used role-playing to let two chat agents cooperate.¹⁴ HuggingGPT used ChatGPT to plan, select models on Hugging Face, execute each subtask and summarise the results.¹⁵

These works are not alike and should not be lumped into a single architecture. Some had their own tools, their own state, explicit workflows or specialised models. Yet together they popularised a very attractive vision: a system could organise work like a team of specialists, with a Planner, Coder, Reviewer, Manager or Researcher each taking different responsibilities.

This is not a naive idea. Division of labour is how complex real-world systems scale: each role has a local goal, its own tools, its own state and a clear hand-off contract. Bringing that structure into agent software is a valuable direction. Early role prompts were the cheapest way to test the hypothesis before the infrastructure for agents fully existed.

Suppose a role receives context $h$ and role prompt $r_i$. The output of role $i$ still has the form:

Changing $r_i$ can change behaviour considerably. With a strong enough model, specialisation through context can create real value. But early LLMs still shared the same $\theta$, usually the same data sources, the same tools and the same transcript. The line "you are a strict reviewer" can help the model switch modes, but it does not by itself supply a test runner, a static analyser, production data or an independent standard of right and wrong.

So the limits of the 2023 "virtual department" do not prove that the vision was wrong. They show the gap between a role that is described and expertise that the system guarantees. A prompt-based Reviewer could already spot contradictions, ask questions and add perspectives. To become a trustworthy reviewer, the role also needs a strong enough model, verification tools, separate state and acceptance criteria that can be executed.

The difference shows up clearly in the many-reviewers problem. Suppose each reviewer misses a given bug with marginal probability $q$. If their errors are independent, the probability that all $k$ reviewers miss it is:

But if they are perfectly correlated, that probability is:

In practice the result usually lies between these two extremes and cannot be inferred from $q$ alone. Reviewers that share a model, use near-identical prompts and read the same artifact easily share the same blind spot. Kim and colleagues (2025) measured this directly across more than 350 LLMs:¹⁶ on one dataset, when two models are both wrong, they pick the same wrong answer about 60% of the time. More strikingly, larger and more accurate models have more highly correlated errors, even across different architectures and providers. Switching models does not guarantee independent votes; reusing the same model with a different role prompt gives even less reason to assume independence.

The figure takes $q=30\%$. The blue-grey line is the independence dream: two reviewers leave 9%, three leave 2.7%, and it falls to nearly zero very fast. The top line is the case where every reviewer shares one blind spot: however many you add, it stays at 30%. The middle line is only an illustrative assumption, not a measurement: half of the misses come from a shared blind spot and the other half are independent, that is $\tfrac12 q+\tfrac12 q^k$. It hits a floor at 15% and goes no lower, whether you add a fifth reviewer or a sixth. That shared part only shrinks when a real source of disagreement arrives, not when more people nod.

This is not a reason to abandon multi-reviewer architectures. It is the blueprint for the next generation: for reviewers to create system-level value, you must bring in real sources of disagreement, such as executable tests, different data, a different model, a formal verifier, a different schema, a different tool, or the right to see state that the role producing the artifact cannot. The vision of role division stays intact; the substrate has to mature enough to realise it.

Multi-agent research moved in that direction early. MetaGPT describes cascading hallucinations from naively chaining LLMs, then encodes SOPs into prompt sequences and requires agents to hand off through intermediate structured outputs.¹⁷ ChatDev designs a chat chain to govern what to communicate and a communicative dehallucination mechanism to govern how.¹⁸ In other words, the vision of an agent organisation does not stop at naming roles. It demands process, artifacts, interfaces and hand-off standards, just like a real engineering organisation.

VII. The unsigned tape: when the substrate lags the vision

Throughout this essay the context has been called a "tape". The image is not accidental. It goes back to the Turing machine, the founding model of computation: a head moves along a strip of tape, reads the symbol in the current cell and writes a new one according to a table of rules. For a language model the tape is the token stream in the context window, and the autoregressive loop is surprisingly simple: scan everything already on the tape, compute the distribution $P_\theta(x_{t+1}\mid x_{\leq t})$, write one more token at the end, and repeat.

The analogy breaks in two places, and those two breaks explain most of what this essay has described.

The first break: a Turing machine's head can move back and overwrite an old cell, while a language model can only append. A broken output, an invented Observation or an apology, once on the tape, stays there for every later read unless the runtime actively cuts it out. That is why the apology loop in section V accumulates: the tape has no delete key, and every failed round leaves one more bad example for the next round to follow.

The second break is the ink. On the tape, everything is written in the same ink: tokens. The developer's instructions, the user's question, the model's Thought, the tool's result and the server's error log all become runs of tokens one after another. Attention still sees the word Observation: at the start of a line, and the model can perfectly well learn that whatever follows that label usually comes from the environment. What it lacks is a way to verify it. A line Observation: 404 Not Found that the runtime really wrote, and an identical line the model just continued because it fit the context, are the same sequence of tokens. Nothing on the tape says who is holding the pen.

Infrastructure like this can be described with the metaphor of an unsigned tape: a tape on which no line carries a signature, with no provenance and no authority boundaries enforced by the runtime. This is not a flaw in the idea of agents; it is a sign that the underlying technology of the time was not designed for the level of autonomy the vision demanded. The chat API of March 2023 already had roles for system, user and assistant, but tool results had no role of their own in the API until function calling in June, and were not tied to the ID of a specific call until the API moved to tool_calls at the end of that year. In that gap, agent frameworks had to render everything as prose themselves.

The "signature" here is not necessarily a cryptographic signature. It is a name for the set of metadata a machine can check: who created a message, what kind of message it is, what it is entitled to request, which schema its payload follows and which runtime confirmed the result.

In a simple agent loop, the context can contain at least five sources:

• the user's request;
• the developer's or system's instructions;
• the model's reasoning and proposed actions;
• data from tools or the environment;
• the runtime's errors, state and decisions.

If all of these are rendered as prose and concatenated, labels like System:, Observation: or Reviewer: can still help the model recognise structure. But they do not by themselves create an authorisation boundary. A web page does not become a system message just because it contains the sentence "ignore all previous instructions"; at the same time, the model can still be swayed by that sentence if the application puts untrusted data into the same stream as instructions.

That is the foundation of indirect prompt injection. Greshake and colleagues showed that LLM-integrated applications blur the line between data and instructions: an attacker plants a prompt in a data source the system is likely to retrieve, and that prompt enters the context and influences how the model calls APIs.¹⁹ The problem is not that attention "doesn't know" what a role token is. The problem is that untrusted data has been placed into a language interface that can steer the very model proposing actions.

Put differently, on an unsigned tape the model has no reliable way to separate a command, which is binding, from data, which is only for reference. Both are just text, and whichever text sounds more like an order stands a better chance of being obeyed.

The same flattening also explains the apology loop, this time from the runtime's side. When the parser fails or a tool returns an error, the runtime is holding a binding event: this step cannot be executed, stop or change course. But on an unsigned tape the only way to report that is to write more text. The event is demoted to a line of conversation, and the model, trained to answer conversation politely, answers with an apology instead of halting the flow. The problem is not that the model cannot read the error. The problem is that the system handed it the error in a form that can only be replied to.

A trustworthy transcript should therefore not be just a string:

The runtime needs to store each message as a typed object, for example:

On the left is the unsigned tape, read top to bottom the way the model reads it. The shaded region is a command hidden inside the web page just retrieved, wearing the Observation: label like every other result. Below it come an Observation about the steel price, a line saying "Reviewer: checked, looks good", and finally a command to send an email. Nothing on the tape tells you which Observation came from a real tool call, what the reviewer checked with, or who authorised the email. On the right are the same sources stored as typed messages, each card carrying a source, a role and an authority attached by the runtime. The web page is still read, but as data, and its authority is none; the steel price is tied to tool call 7f3a; and the model's proposal sits in a pending state.

The model can still receive content rendered from those messages. But permission to execute must not be inferred from a sentence that merely sounds like an order. The runtime must rely on metadata that it controls itself.

This distinction resolves four common confusions:

• A proposal is not an authorised command. The model can propose delete_record; the policy engine can still refuse.
• A string that looks like an Observation is not a verified Observation. Only a result tied to a valid tool execution ID has provenance from the environment.
• A runtime error is not an opinion in a conversation. It is a state with a code, a type and a recovery policy.
• A reviewer's approval is not evidence. Evidence has to come from a trace, a test, data or a verifier that can be checked.

The unsigned tape also explains why simply lengthening the context is not enough to fulfil the agent vision. A larger window holds more history, but that history can still contain conflicting instructions, malicious data, broken outputs and untrustworthily labelled messages. Lost in the Middle, as the previous part discussed, adds that information being present in the context does not guarantee it is used evenly.²⁰

To move from prototype to mature system, that tape does not just need to be longer. It needs typed messages, provenance, policy and boundaries the runtime can enforce. Leaving the unsigned tape behind, through schemas, roles and tool call IDs attached by the runtime itself, is the step that turns a text-generating machine into a disciplined software system. The next section tells how that step began, and why it only got halfway.

VIII. Function calling, Structured Outputs and what is still missing

On 1 March 2023, the ChatGPT API already used a message structure with roles such as system, user and assistant.²¹ So it would be inaccurate to say that function calling was the first time the API could tell speakers apart.

The turning point on 13 June 2023 was elsewhere. OpenAI introduced function calling for gpt-4-0613 and gpt-3.5-turbo-0613.²² Developers could describe functions with JSON Schema; the models were fine-tuned to decide when to call a function and to return the arguments as JSON. The tool call had its own field, instead of hiding inside a Thought / Action / Action Input passage and waiting for a regex to cut it out.

This was an important architectural improvement. It moved part of the contract from a prompt convention into an API contract. Applications no longer had to guess which line of an essay was the command. SDKs and backends could receive a tool-call object, check the tool name, parse the arguments and then apply their own policy. In other words, the substrate began to catch up with the vision ReAct and the agent frameworks had laid out earlier.

But even the API reference of the time stated plainly that the model does not always generate valid JSON, may hallucinate parameters not defined in the schema, and that developers should validate the arguments before calling the function.²³ The separate field existed, but what sat inside it was still text generated by the model. And even when the JSON was valid, 2023 function calling did not make the other classes of error disappear:

• the model could still pick the wrong function;
• arguments could be syntactically valid and semantically wrong;
• data could be missing, stale or prompt-injected;
• the tool could fail during execution;
• a valid call could still exceed the user's permissions;
• the runtime still had to decide on retry, rollback and audit.

In August 2024, OpenAI introduced Structured Outputs.²⁴ With strict: true, output is forced to follow a supported JSON Schema. OpenAI describes combining model training with constrained decoding, which allows only the tokens that remain valid under the schema at each generation step. On its internal eval of complex JSON Schema following, gpt-4o-2024-08-06 with Structured Outputs scored 100%, while gpt-4-0613, the model launched with function calling a year earlier, scored below 40%.

Those two numbers measure exactly the gap this essay is about: on that eval, more than three in five calls once failed to match the schema, even with a dedicated tool-call field. But the 100% must also be read within its scope. The announcement itself lists the limits: only a subset of JSON Schema is supported; output can be incomplete if the model refuses the request, or if generation hits max_tokens or another stop condition; at launch the mode was not compatible with parallel function calls; and it does not prevent the model's mistakes inside the values of each field. Structured Outputs can remove a large class of syntax and schema errors when a request completes normally on a supported path. It does not prove that the contents of the fields are correct, that the chosen function is appropriate, that the user is entitled to do it, or that the side effect succeeded.

In short:

does not imply:

The four doors in the figure are four kinds of error, and an action has to pass through all four. The first door already has a lock: Structured Outputs. The other three are only pushed to, and each needs something different to hold it: tests or a verifier for meaning, a policy engine for permission, and retry, rollback and audit for execution. One door locked tight does not make the other three any tighter.

The parser has not disappeared; part of the parsing has moved into constrained output-generation infrastructure. That is real progress, but it has only made one door solid. It has not decided who holds the key, which doors they may open, or what to do if the room behind is on fire.

A trustworthy agent runtime must therefore treat the model as the component that proposes, not the final source of authority. Let $a_t$ be the action the model proposes; the runtime must evaluate it against the identity acting (the principal), the policy and the current state:

where $V$ can allow, deny or request approval. Only an accepted action enters the environment:

Here $o_t$ is the returned result, $s_{t+1}$ the new state and $e_t$ an event for the trace or audit. The system then knows not only what the model said. It knows who asked, which policy was applied, which tool actually ran, which state changed and what evidence was kept.

The figure is the two formulas above, drawn as a path. The model proposes delete_record(42); the proposal stops at the hexagon $V$, where the runtime checks it against the principal, the policy and the state $s_t$. From there, three exits. If allowed, the action finally reaches $\mathcal E$, and the environment returns all three things: the result $o_t$, the new state $s_{t+1}$ and an event $e_t$ written to the audit trail. If denied, the model receives a coded error, DENIED: scope, not a reproach. The third exit is waiting for a human to approve. Compared with the first figure of this essay, the scissors have been replaced by a checkpoint.

That is the foundation of a harness.

Conclusion

Prompt engineering changes how the model is asked. Context engineering changes what the model gets to see. ReAct goes one step further: it lets the previous turn's output and the environment's result build the context for the next turn. Multi-agent systems then organise that process into many roles and many hand-offs.

From today's vantage point it is easy to dismiss the 2022 and 2023 prototypes as fragile. That judgement misses the most important thing: they recognised the right shape of the future system before the parts it needed had fully appeared.

ReAct saw the loop between reasoning, action and observation. Toolformer saw the need to teach models how to call APIs. AutoGPT, BabyAGI, CAMEL, HuggingGPT and the multi-agent systems saw the possibility of decomposing goals, specialising roles and coordinating over many steps. That vision was not wrong. What moved too fast was the expectation that young LLMs, running on regex parsers and text transcripts, could realise it reliably right away.

The following years added the missing pieces one by one: models that follow instructions better, longer contexts, function calling, Structured Outputs, tool protocols, state management, observability and policy engines. Each new layer does not replace the agent vision; it makes that vision depend less on luck.

The boundaries of a mature system still need to be kept very clear:

The model proposes. The runtime authorises. Tools execute. State records. Verifiers check. The audit trail keeps the evidence.

That is not a rebuttal of virtual departments. It is the condition for them to become departments with real capability inside a software system.

ReAct and multi-agent systems saw the future early. What was missing was not the vision, but models and runtimes mature enough to carry it.

The next part picks up from here: how a harness manages tools, permissions, state, retries, rollback and the audit trail, turning ReAct's sketch into a system that can be operated without losing control.

1. Ehud Karpas et al., MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning, AI21 Labs, 2022. Prompt-tuning setup in §3.2.2; comparison with few-shot in Appendix A. ↩
2. Shunyu Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023. HotpotQA and FEVER numbers in Table 1. ↩
3. Timo Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools, NeurIPS 2023. ↩
4. LangChain, notebook custom_llm_agent.ipynb at v0.0.200, June 2023. ↩
5. LangChain, commit 7eba828, "Harrison/update regex (#1534)", 9 March 2023 (UTC). ↩
6. LangChain, AgentExecutor.handle_parsing_errors. The option appeared between v0.0.150 and v0.0.160. ↩
7. AutoGPT, issue #623, "Memory Feature seems to not work on gpt3only mode", 9 April 2023. ↩
8. AutoGPT v0.2.2, promptgenerator.py, the response_format section. ↩
9. Gemini CLI, issue #4086, 14 July 2025. ↩
10. Gemini CLI, issue #1531, "[TOOL LOOP] CLI getting stuck in tool loop". The summary was added to the issue description in December 2025. ↩
11. Gemini CLI, issue #5165, "Make Gemini Less Apologetic", 30 July 2025. ↩
12. Gemini CLI, comment by nbs in issue #1531, 27 June 2025. Screenshot reproduced unaltered for commentary; copyright remains with its poster. ↩
13. Yohei Nakajima, BabyAGI, April 2023. ↩
14. Guohao Li et al., CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society, NeurIPS 2023. ↩
15. Yongliang Shen et al., HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, NeurIPS 2023. ↩
16. Elliot Kim, Avi Garg, Kenny Peng, Nikhil Garg, Correlated Errors in Large Language Models, ICML 2025. ↩
17. Sirui Hong et al., MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, ICLR 2024. ↩
18. Chen Qian et al., ChatDev: Communicative Agents for Software Development, ACL 2024. ↩
19. Kai Greshake et al., Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection, 2023. ↩
20. Nelson F. Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 12 (2024). ↩
21. OpenAI, Introducing ChatGPT and Whisper APIs, 1 March 2023. ↩
22. OpenAI, Function calling and other API updates, 13 June 2023. ↩
23. OpenAI, API reference for function_call.arguments, July 2023 version: "the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema". ↩
24. OpenAI, Introducing Structured Outputs in the API, 6 August 2024. ↩