Cost Controls and Runtime Budgets

Agentic systems spend money, time, context, tool quota, and human attention every time they think, retrieve, call a tool, delegate, retry, evaluate, or ask for approval. Runtime budgets make those costs explicit and enforceable.

Download the cost controls and runtime budgets review checklist before using this chapter for a production review.

This chapter is not about making agents cheap at any cost. It is about making cost, latency, and autonomy part of the control plane. A useful agent should know when to continue, when to degrade, when to ask for approval, and when to stop.

Intent

Keep agentic behavior bounded by task value, risk, user tier, and operational limits.

A budget is not only a finance number. It is a runtime contract: how many steps the system may take, how many tools it may call, how long it may run, how much context it may carry, how many retries it may spend, how much human attention it may request, and how much autonomy is justified before escalation.

Runtime Budget Readiness Questions

Use these questions before enabling autonomous loops, retries, retrieval, delegation, or write tools:

Budgeting is an architecture feature. If the budget is only a dashboard after the run, it is accounting, not control.

Use When

• Agent runs can loop, retry, delegate, retrieve, or call tools.
• Model, tool, retrieval, or evaluator calls have meaningful cost or latency.
• Different task classes deserve different autonomy levels.
• The system serves interactive users and must preserve responsiveness.
• Operators need clear stop reasons when a run cannot continue.

Use this pattern before cost surprises appear in production. Retrofitting budgets after a runaway loop is harder because the architecture has already hidden the counters.

Avoid When

• The task is a single deterministic function call.
• The system cannot observe model calls, tool calls, retries, or wall-clock time.
• The organization wants cost reduction without defining acceptable quality and risk.

Even then, keep simple counters. The first useful prototype often becomes the first production workflow.

Budget Types

The most important budget is often not money. In production, human attention and user patience are usually scarcer than tokens.

Budget Ownership

Budgets should be owned by the runtime, not by the prompt. The model can propose that more work is useful, but software should decide whether the system is allowed to spend more.

Budget ownership usually belongs in the same layer that owns state, policy, trace IDs, and stop reasons:

• routers load the budget for the task class;
• loop controllers decrement the budget after each step;
• tool gateways check tool and side-effect budgets;
• retrieval services enforce evidence and query budgets;
• workflow engines persist budget state across retries and approvals;
• observability records budget events as first-class trace events.

If every tool wrapper tracks its own counters, the system will drift. The runtime needs one budget view for the run.

Runtime Budget Object

Give every run a budget object that travels with the state:

type RiskClass = 'low' | 'medium' | 'high';
type BudgetDecision = 'continue' | 'degrade' | 'approval_required' | 'stop';

type RuntimeBudget = {
  riskClass: RiskClass;
  maxCostCents: number;
  maxModelCalls: number;
  maxToolCalls: number;
  maxWriteToolCalls: number;
  maxRetrievalQueries: number;
  maxDelegations: number;
  maxRetries: number;
  maxWallClockMs: number;
  approvalRequiredAboveCents: number;
};

type RuntimeUsage = {
  costCents: number;
  modelCalls: number;
  toolCalls: number;
  writeToolCalls: number;
  retrievalQueries: number;
  delegations: number;
  retries: number;
  startedAtMs: number;
};

type PlannedActionCost = {
  estimatedCostCents: number;
  modelCalls: number;
  toolCalls: number;
  writeToolCalls: number;
  retrievalQueries: number;
  delegations: number;
};

The budget object should be versioned. A production incident caused by a budget change should be replayable against the old and new budget policy.

Enforcement

Check budgets before every expensive or risky action, not after the run is already over.

function checkBudget(
  budget: RuntimeBudget,
  usage: RuntimeUsage,
  next: PlannedActionCost,
  nowMs: number
): { decision: BudgetDecision; reason: string } {
  if (nowMs - usage.startedAtMs >= budget.maxWallClockMs) {
    return { decision: 'stop', reason: 'wall_clock_budget_exhausted' };
  }

  const projectedCost = usage.costCents + next.estimatedCostCents;
  if (projectedCost > budget.maxCostCents) {
    return { decision: 'stop', reason: 'cost_budget_exhausted' };
  }

  if (projectedCost > budget.approvalRequiredAboveCents) {
    return { decision: 'approval_required', reason: 'cost_approval_required' };
  }

  if (usage.writeToolCalls + next.writeToolCalls > budget.maxWriteToolCalls) {
    return { decision: 'degrade', reason: 'write_tool_budget_exhausted' };
  }

  if (usage.modelCalls + next.modelCalls > budget.maxModelCalls) {
    return { decision: 'degrade', reason: 'model_call_budget_exhausted' };
  }

  if (usage.retrievalQueries + next.retrievalQueries > budget.maxRetrievalQueries) {
    return { decision: 'degrade', reason: 'retrieval_budget_exhausted' };
  }

  if (usage.toolCalls + next.toolCalls > budget.maxToolCalls) {
    return { decision: 'degrade', reason: 'tool_call_budget_exhausted' };
  }

  if (usage.delegations + next.delegations > budget.maxDelegations) {
    return { decision: 'stop', reason: 'delegation_budget_exhausted' };
  }

  if (usage.retries >= budget.maxRetries) {
    return { decision: 'degrade', reason: 'retry_budget_exhausted' };
  }

  return { decision: 'continue', reason: 'within_budget' };
}

The decision should become part of the trace. Notice that the check uses projected cost, not only current usage. A budget gate that runs after the model call or tool call has already spent the money is only accounting. A budget gate that runs before the next action is control.

Budget Decision Flow

Use this flow before each model call, retrieval query, tool call, delegation, retry, approval request, or memory write. It turns budget pressure into an explicit runtime decision.

Degraded Modes

When a budget is nearly exhausted, the system should not improvise. It should switch to a known degraded mode:

The degraded mode should be visible to the user or operator. Silent degradation creates false confidence.

Risk-Based Budgets

Not every task deserves the same budget.

Low-risk tasks can be cheap and fast. High-risk tasks may justify stronger models, more evidence, more policy checks, and human approval. The mistake is giving every task the same budget because the runtime has no task class.

Budgets are a way to encode judgment. They say which tasks are worth more reasoning, more tools, more review, or more time.

Budget Calculator Example

Start with a simple per-run estimate before building a complex dashboard.

Then map the estimate to a policy:

The exact numbers will differ by model and vendor. The useful habit is the same: estimate before the run, reserve for retries, enforce before actions, and record actual spend against the policy version.

Alert Thresholds

Budget alerts should distinguish one expensive run from a systemic problem.

Every alert should point to an operator action. A cost alert that only says “spend is high” is too vague to help during an incident.

Interaction With Circuit Breakers

Budgets and breakers are related but not identical.

A budget says how much the system is allowed to spend. A breaker says when the system must stop or change strategy because the pattern of behavior is unsafe or unproductive.

Examples:

• a cost budget stops the run after a spend limit;
• a no-progress breaker stops the run even if budget remains;
• a tool budget limits calls to a tool class;
• a tool failure breaker disables a tool after repeated bad results;
• a delegation budget limits handoffs;
• a handoff breaker stops agents from bouncing work with no owner.

Use both. A run can be within budget and still be unsafe. A run can be safe but no longer worth the spend.

Observability

Budget events should be visible in traces:

• budget policy version;
• task class and risk class;
• starting budget;
• remaining budget after each step;
• budget checks before model and tool calls;
• decision: continue, degrade, approval required, or stop;
• stop reason;
• user-visible degraded mode;
• operator-visible explanation.

If cost is only visible in the monthly bill, the architecture is already too late.

Evaluation Guidance

Test budget behavior like any other production control.

The eval should check both outcome and trajectory. A final answer that looks good but exceeded budget without approval is not a pass.

Failure Modes

• Budgets live only in prompts, so the model can ignore or reinterpret them.
• Budgets are checked only after expensive work has already happened.
• The same budget is used for low-risk classification and high-risk automation.
• Cost is optimized while human review work increases.
• Retry budgets hide repeated failure instead of changing strategy.
• Delegation budgets are missing, so multi-agent systems create coordination cost.
• Budget exhaustion is logged as a generic error with no stop reason.
• Operators cannot compare budget policy versions during incident review.

Production Checklist

• Does every run start with a task class, risk class, and budget policy version?
• Are budgets checked before model, tool, retrieval, memory, and delegation actions?
• Are budget decisions recorded as trace events?
• Are degraded modes defined before launch?
• Can high-risk runs request approval for more budget?
• Can low-risk runs be stopped without human review?
• Are budgets tied to evals, incidents, and release gates?
• Can budget policy changes be rolled back?
• Can operators answer where the cost went for a single run?

Related Chapters

• Resource-Aware Agent Design
• Circuit Breakers, Fallbacks, and Replay
• Agent Loop
• Context Budgets and Working Sets
• Observability and Evals
• Production Evaluation Feedback Loops
• Durable Workflows