Stop Prompting for the Parts of Your Workflow That Never Change

The first version of our content pipeline called a model at every step, because that was the easiest way to reason about it. It was slower than it needed to be, cost more than it needed to, and produced different results from identical inputs often enough to be irritating.

The fix was not a better prompt. Most of those steps were doing the same operation every single time, and none of them needed a model to work out what that operation was.

That is the shape of most business automation. Read the invoice. Pull out the total. Check it against the purchase order. Route it. The specifics change on every run; the procedure never does. Paying a model to rediscover the procedure each time buys you nothing except fresh variance.

What the research measured

Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation by Trooskens, Karlsberg, Sharma, De Brouwer, Van Puyvelde, Young, Thickstun, Alterovitz and De Brouwer, submitted April 2026 and revised July 2026.

The idea is in the title. Use the model once, during a compilation phase, to generate executable code. Then run that code deterministically, with no further model calls.

The 17-transaction break-even is the number worth remembering. Compiling costs more than one runtime call, because generating and validating the code is real work. But it is a fixed cost paid once, and after roughly seventeen runs you are ahead. Most business workflows run seventeen times in a week.

The cost saving is not the main benefit

A 57x token reduction is significant, and it is not what we would sell this on.

The larger benefit is that the same input produces the same output on every run, indefinitely.

When a model decides at runtime, you get a fresh sample from a distribution on every execution. Usually it lands in the same place. Occasionally it does not, and you get an invoice routed to the wrong department for no reason anyone can reconstruct, because the run that made the decision is gone.

Compiled code does not have that property. If it routes wrongly, it routes wrongly every time, which means you find it immediately and can fix it permanently. A bug you can reproduce is a vastly better problem to have than a bug that appears in two percent of runs.

This connects directly to what the agent reliability literature shows. As we cover in how to read an AI agent's success rate, agents tend to fail unpredictably rather than degrading smoothly, so a system that behaves identically on every run removes the failure mode that is hardest to test for.

You also get an auditable artifact. When a client asks why a document was processed a particular way, you can show them the code. "The model decided" is not an answer that survives a compliance conversation.

Where this does not apply

Compilation works when the procedure is stable and the inputs vary within a known shape. It does not work when each run needs genuine judgment about something new. If your workflow is "read this customer complaint and decide how to respond," there is nothing to compile, because the reasoning is the work.

The document intelligence numbers are also worth reading honestly. 80.0% on key fields matched a direct LLM approach rather than beating it. The claim is comparable accuracy at dramatically lower cost with deterministic behaviour, not better accuracy. And 80% means one field in five needs a human, which changes how you design the process around it.

The approach also trades flexibility for predictability by design. Change the procedure and you recompile. For a workflow that shifts weekly, that maintenance may cost more than it saves.

The pattern for a real workflow

In practice almost no workflow is entirely one or the other. The useful move is to split it.

• Write out the workflow as steps, then mark each one: does this need judgment about something new, or is it the same operation on different data?
• The 'same operation' steps are candidates for compiling into code. In most business processes this is the large majority of steps.
• The judgment steps stay as model calls, and are where your model budget should go.
• Count how often the workflow runs. Under seventeen or so runs, on this evidence compilation may not pay back.
• Build the failure path first. At 80% extraction accuracy the question of what happens to the other 20% is the design, not an edge case.

This is also the cheapest way to control cost, because it moves work off the expensive tier. As we cover in what actually falls in AI pricing, frontier model prices have shown no meaningful decline, so shifting steps off the model entirely is more reliable than waiting for the model to get cheaper.

How this shows up in our work

We run our own content pipeline on n8n, and it follows this shape without us having read a paper about it. The steps that never change are code. The model gets called at the two points where something genuinely has to be judged.

The version we built first called the model at every step because that was easier to reason about. It was slower, cost more, and produced different results on identical inputs often enough to be irritating. Pulling the model out of the fixed steps fixed all three at once.

We wrote about the tooling side of this in n8n or custom code for automation. This guide is the same argument from the cost and reliability side.

If you have an automation that costs more than expected or gives different answers to the same input, go and count how many of its steps genuinely require judgment. In our experience the answer is one or two, and everything else is a candidate for code. That is usually fixable without a rebuild, and it is standard AI and automation work. Tell us what yours does.