Agentic Systems in Production: Tool Contracts, Budgets, and Blast Radius

The demo is always the same: a model, five tools, a clever system prompt, and a loop that runs until the agent decides it is done. It files the ticket, books the slot, refactors the file. Then you point it at a real enterprise workflow and discover that 'until it decides it is done' is not a termination condition, and that a tool the agent can call is a tool the agent will eventually call with the wrong arguments at the worst possible moment.

I have shipped agentic systems into environments where a wrong tool call sends a real email to a real customer, or writes to a system of record that finance reconciles at month end. What separates those from the demo is not a better model or a cleverer prompt. It is treating the agent loop as a distributed system driven by untrusted input. Four controls carry almost all of the weight: tool contracts, budgets, blast radius, and trajectory evals.

Tool contracts are the real prompt

Teams spend three weeks tuning the system prompt and ten minutes writing tool schemas. That is backwards. The schema is what the model re-reads on every single step, and it is the only artifact that constrains what actually happens inside your infrastructure. A vague tool description is not a documentation problem, it is a reliability problem that shows up as 'the agent is flaky'.

The rewrite that fixed the most failures for me was mechanical: replace free-text parameters with enums and opaque handles, split overloaded tools into one tool per intent, and turn every error return into an instruction. An agent that receives 'ValidationError at line 42' will retry the same call forever. An agent that receives 'Invalid date format, use YYYY-MM-DD' fixes it on the next step.

• One tool per intent — an update_record with a mode flag is two tools wearing a trench coat.
• Enums and IDs over free text; every free-text parameter is somewhere the model can invent a value.
• Make dangerous arguments unguessable: pass an opaque handle returned by a previous read, never a raw customer ID the model can hallucinate.
• Error strings are prompts. Write them for the model that has to recover, not for the engineer reading a stack trace.
• Return the minimum the next step needs. Dumping a forty-field object into context buys drift, not capability.

Budgets, because loops do not know when to stop

An agent has no intrinsic sense of cost. If the task is under-specified it will happily spend forty steps and nine dollars convincing itself it is making progress. Termination is your responsibility, and it needs four independent ceilings: steps, tokens, wall clock, and spend. Whichever trips first ends the run.

def run_agent(task, tools, budget):
    history, steps = [], 0
    while steps < budget.max_steps and not budget.expired():
        step = model.next_step(task, history)
        if step.is_final:
            return step.answer

        tool = tools.get(step.tool_name)
        if tool is None:
            history.append(reject(step, 'unknown tool, choose from the listed tools'))
            continue
        if not tool.policy.allows(step.args, actor=task.actor):
            history.append(reject(step, 'not permitted for this user'))
            continue

        history.append(tool.invoke(step.args, timeout=tool.timeout))
        budget.spend(step.usage)
        steps += 1

    # Never silently truncate. Escalate with the trajectory attached.
    raise BudgetExhausted(task.id, steps=steps, history=history)

In production I have never once seen a runaway agent produce a good answer on step thirty that it was not already close to by step six. Cap steps low, fail loudly, and hand the human the full trajectory rather than a shrug. A clean escalation at step eight is a better product than a confident wrong answer at step twenty-five.

Blast radius: the agent inherits the user, not the service

The most common security flaw I find in agent deployments is a service account with god rights. Every tool call runs as the platform, so the only thing standing between a customer and another tenant's data is the model's judgement. That is a classic confused deputy, and it becomes exploitable the moment your agent reads content it did not author — a retrieved document, a support ticket, a web page — because that content can carry instructions.

Authorize every tool call with the requesting user's own permissions, evaluated server-side at call time. Separate read tools from write tools and give them different policies. Make irreversible actions two-phase: the agent proposes a diff, a human or a deterministic rule approves it, and only then does the write execute. Dry-run plus diff has caught more bad agent behaviour for me than any prompt hardening.

Design the loop so that the worst thing an injected instruction can achieve is wasting a step.

Evals run on trajectories, not answers

Single-turn evals tell you nothing about an agent, because the failure modes are sequential: it picks the right tool with the wrong argument, recovers badly from an error, or loops between two tools until the budget dies. Score the whole trajectory — task success, steps to success, tool error rate, recovery rate after a failed call, and cost per resolved task. Cost per resolved task is the number that survives contact with a CFO.

Record real trajectories, replay them against stubbed tools in CI, and diff the behaviour on every prompt or schema change. It is the same discipline as a RAG eval suite, one layer up: without it, every change to a tool description is an unmeasured experiment running against production users. Agents are not hard because the models are weak. They are hard because we deploy an unbounded loop into an environment that assumed bounded ones, and then act surprised.