Ai agents is reshaping how we work. The AI agent demo is a familiar genre. A natural language interface that accepts a request, breaks it into steps, calls some tools, executes them in sequence and returns a tidy answer. The first time you see one work, it feels like magic. The second time, it crashes. In particular, the third time, it confidently produces a wrong answer.
This gap – between demo and production – is where most AI agent initiatives die. The demo proves what is possible in ideal conditions. Production demands what is possible in adversarial conditions, at scale, on data the agent has never seen, with consequences for getting it wrong.
Closing that gap is one of the most important pieces of work in modern AI engineering. It is also one of the most frequently underestimated.
What production-ready AI agents actually require
A production AI agent is one that:
- Performs reliably across the full distribution of inputs it will encounter, not just the curated examples used in development.
- Fails gracefully when it encounters something outside its competence, rather than hallucinating an answer.
- Operates within bounded cost, even when prompted to do extensive multi-step reasoning.
- Provides traceability – for any output, you can reconstruct exactly what the agent did and why.
- Integrates with downstream systems in a way that respects their reliability requirements.
- Is monitored continuously, with degradation detected automatically.
a demo needs to do none of these things. The work of getting from one to the other is mostly engineering, not prompting.
The architectural patterns that work
Across the AI agent systems we’ve shipped to production, a small number of patterns recur. They are worth knowing because they are the difference between a system that works and one that doesn’t.
Tool-calling over open-ended generation. Production agents rarely “think” their way to an answer. They identify what the user wants, then call structured tools to get it. The agent’s job is to understand intent and orchestrate. The actual work is done by deterministic functions whose behaviour is predictable. This is dramatically more reliable than asking the agent to do everything inside its own reasoning.
Constrained outputs over free-form text. Where the agent must produce structured data – JSON, database records, function arguments – use schema-enforced output (function calling, structured output APIs). Free-form text parsed downstream is a reliability disaster waiting to happen.
Multi-step decomposition with explicit checkpoints. Complex tasks are broken into stages, each with a clear input, output and validation. The agent moves through them like a state machine, not a stream of consciousness. When something fails, you know exactly which stage failed.
Human-in-the-loop for high-consequence actions. Anything irreversible (sending an email, making a payment, deleting data) gets a human confirmation step by default. The agent prepares, the human approves. The cost in user friction is dramatically less than the cost of a confidently wrong action.
Retrieval-augmented generation over model memory. Where the agent needs domain-specific knowledge, retrieve it at query time from an authoritative source rather than relying on the model to “know” it. This shifts the failure mode from confident hallucination to graceful “I don’t know.”
Conservative defaults with explicit escalation. Agents should err on the side of doing less when uncertain. A response of “I can’t confidently answer this – let me hand you to a human” is vastly preferable to a confident wrong answer.
The evaluation discipline
The single most important capability for production AI agents is not the agent itself. It is the evaluation infrastructure around it.
We build every production agent with three layers of evaluation:
Layer 1: Pre-deployment evaluation. A test set of representative inputs with expected behaviour. Every change to prompts, tools, models or context triggers a full evaluation run. Regressions are blocked before they ship.
Layer 2: Continuous production evaluation. A sample of production traffic is automatically scored against quality criteria. This catches drift, edge cases and unexpected user inputs that weren’t in the test set.
Layer 3: Human review and feedback loop. Outputs flagged as low-confidence (or randomly sampled) are reviewed by domain experts. Their feedback flows back into the test set and the prompts.
Without these, an agent that passes initial QA will quietly degrade in production and nobody will know until a customer complains. With them, the agent gets measurably better over time.
The operational realities
a few things that don’t show up in demos but matter enormously in production:
Cost control. Multi-step agents can be expensive. A single user query that triggers ten LLM calls and three retrievals can cost tens of pence. At scale, these add up fast. Budgets per user session, model tiering (cheaper models for easy tasks, premium models reserved for hard ones) and caching of common patterns are all essential.
Latency. Users tolerate AI latency much less than they tolerate other latency, because the wait feels meaningful. A 12-second agent response feels broken even if the answer is good. Streaming responses, parallel tool execution and aggressive timeouts all matter.
Versioning. When you change a prompt or swap a model, what was the agent doing yesterday? Can you reproduce it? Production agents need full versioning of prompts, tool definitions and model selections – not just code.
Observability. When something goes wrong, can you reconstruct exactly what the agent did? Tool calls, intermediate reasoning, model outputs at each step – all of it should be logged in a way that’s queryable. The first time a customer complains about an agent’s behaviour, you will be very glad you have this.
Safety boundaries. What is the agent allowed to do? What is it explicitly prevented from doing? These should be enforced in the orchestration layer, not relied on as instructions in a prompt. Prompt-only constraints are bypassable; orchestration-level constraints are not.
When to use an agent (and when not to)
Not every AI feature needs to be an agent. Often a single-call LLM response is the right answer. The case for an agent grows when:
- The task requires multiple steps that can’t be predetermined.
- The agent needs to interact with external systems based on what it learns.
- The user can express intent in natural language but the underlying execution is structured.
- The task benefits from iterative refinement rather than one-shot generation.
The case shrinks when:
- The task is well-defined enough that you could write a deterministic version.
- The cost of error is high.
- The user prefers a structured interface anyway.
- Latency requirements are tight.
The reflexive “let’s make it an agent” response to any AI feature request is often the wrong instinct. Sometimes the answer is “let’s make it one good LLM call wrapped in clean engineering.”
The strategic point
AI agents that work in production are not a technology problem – they are a discipline problem. The technology is real. The capability is mature enough for serious deployment. What’s missing in most failed attempts is the engineering rigour around the model: tool design, evaluation infrastructure, observability, cost control, safety boundaries.
this is unglamorous work, but it is what separates AI features that quietly compound value from AI features that get quietly turned off six months after launch.
Considering an AI agent for your product or operation? We design and build agents that survive contact with real users. Get in touch to discuss yours.