Agentic AI grows up: from impressive demo to system that holds
Most agent pilots stall in the gap between a demo that works once and a system that works ten thousand times. The difference is almost never the model.
- Step-level reliability compounds ruthlessly: 95% per step is 36% over twenty steps.
- Scope is the primary reliability lever — narrow agents that do one job well beat general ones.
- Tools, not prompts, are where most agent failures actually originate.
- Evaluate trajectories, not just final answers, and gate consequential actions behind humans.
Why the demo lied
An agent that books a meeting, drafts the follow-up, and updates the record is a genuinely impressive thing to watch. The demo is real. The problem is that it was run once, by the person who built it, on a case they chose.
Production is a different distribution. The calendar API times out. The record has two contacts with the same name. The user's request is ambiguous in a way nobody anticipated. A single-shot demo tells you the ceiling; production tells you the floor, and the floor is what your users experience.
The arithmetic nobody wants to do
Agents chain steps, and reliability multiplies. An agent whose individual steps succeed 95% of the time completes a five-step task about 77% of the time, and a twenty-step task about 36% of the time. Push per-step reliability to 99% and the twenty-step task reaches roughly 82%.
Two conclusions fall out immediately. Shorter chains are dramatically more reliable than longer ones, and marginal improvements in step reliability matter more than they feel like they should. Most teams try to fix disappointing agents by changing the model or elaborating the prompt. The higher-leverage moves are usually to cut the number of steps, make each step verifiable, and give the agent a way to recover when a step fails.
Design rule we apply: if a task can be done with a deterministic workflow and one model call, it should be. Reserve agentic looping for cases where the path genuinely cannot be known in advance. Agency is a cost, paid in reliability, and it should buy you something.
Where failures actually come from
Having debugged a fair number of these systems, the failure distribution is consistent and it is not what teams expect.
- Tool design. The largest single source. Tools with vague descriptions, overlapping responsibilities, silent failure modes, or error messages that do not say what to do next. Models handle a well-specified tool remarkably well and flail against an ambiguous one.
- Context management. Too little context and the agent invents; too much and the relevant instruction gets lost in the middle. Managing what is in the window at each step is now a first-class engineering concern.
- Missing recovery paths. Real environments fail constantly. An agent with no defined behavior for a failed call will improvise, and improvisation is where the alarming outputs come from.
- Scope creep. An agent asked to handle everything handles nothing dependably. Narrow scope is the most underrated reliability technique available.
- The model. Real, but well down the list — and usually the first thing blamed.
Evaluating something that takes a path
Traditional evaluation compares an output to an expected answer. Agents make that insufficient, because two runs can reach the same answer by very different routes — one sound, one lucky. If you only score endpoints, you will ship the lucky one.
What works better: score the trajectory as well as the outcome. Did it select appropriate tools? Did it recover from the failure it encountered? Did it stop when it should have stopped, or keep going? Did it ask for help at the right moment? Building a set of recorded real cases with graded trajectories is more work than an accuracy number, and it is the difference between knowing your agent works and hoping so.
The autonomy question
The most consequential design decision in an agentic system is not which model to use. It is which actions the system may take without a human.
A workable frame is to sort actions by reversibility and blast radius. Reading data, drafting text, and preparing a recommendation are cheap to get wrong and easy to undo — good candidates for autonomy. Sending an external communication, moving money, changing a production configuration, or making a decision about a person are none of those things, and they belong behind an explicit human gate for a long time, possibly permanently.
The systems that succeed in regulated environments are almost always the ones that drew this line early and conservatively, then earned autonomy incrementally with evidence. The ones that stall are the ones that tried to demonstrate full autonomy in a domain that was never going to accept it.
What maturity actually looks like
The organizations getting durable value from agents share a profile that has little to do with model choice: narrow, well-defined scopes; carefully designed tools with explicit failure semantics; trajectory-level evaluation running in CI; comprehensive tracing so any run can be reconstructed; conservative autonomy boundaries with documented thresholds; and a deliberate shadow-mode period before anything acts on its own.
None of that is exciting, and all of it is the actual work. The demo is the easy 20%. The remaining 80% is reliability engineering, and it is where agentic AI either becomes infrastructure or stays a prototype.
More perspectives.
How AI is redrawing market structure
Model capability is commoditizing faster than almost anyone predicted. That doesn't mean the value disappears — it means it moves. A map of where it is going, and what that implies if you are buying rather than building.
Read the perspective →The quiet shift to small, specialized models
The largest model is rarely the right model. Why serious work is moving down the size curve — and what enterprises gain besides a smaller bill.
Read the perspective →What AI actually changes about work
Not jobs replaced wholesale — tasks redistributed, judgment concentrated, and a set of organizational problems nobody has an org chart for yet.
Read the perspective →Bring us a hypothesis. Leave with a system.
Tell us what's eating your team's time. We'll give you an honest read on whether AI is the right tool — and if it is, a scoped v1 with a timeline and cost.