Reference Architecture

This reference architecture combines the core patterns in the book into a production-ready agentic system. It is intentionally conservative: deterministic software owns state, policy, and side effects; model calls propose decisions inside bounded loops.

Use this chapter as the bridge from pattern selection to system design. The diagram is not a framework prescription; it shows the ownership boundaries a production system needs before agents handle private data, external actions, or long-running work.

Architecture

Read this diagram as an ownership map for production systems: deterministic services own identity, state, policy, retrieval, approvals, and traces around bounded model calls.

Download the reusable review artifact: reference architecture review checklist.

Boundary Flow

Use this flow to review one request from entry to final status. It separates the control plane that decides what is allowed from the execution plane that retrieves evidence, calls tools, and writes state.

The flow should fail closed. A model proposal can move forward only after state, policy, schema, evidence, approval, or verifier checks accept it.

What This Architecture Is For

Use this shape when the agent may touch private data, call tools, wait for approval, run longer than one request, or affect customers, money, infrastructure, compliance, or internal operations. It is heavier than a prompt chain, but the extra structure buys reviewability.

Do not start here for a read-only prototype, a one-shot summarizer, or a workflow whose steps are fully deterministic. For those systems, use the smallest pattern that solves the task and keep the production controls proportional to the risk.

Readiness Questions

Use these questions before a design review:

1. What can the system do that a normal chat answer cannot?
2. Which action creates the highest risk, cost, or user impact?
3. Which service owns the state that survives retries, crashes, and approval waits?
4. Which policy check runs before retrieval, before tool use, before memory write, and before final answer?
5. Which evidence must the model see, and which evidence must stay outside model context?
6. What exact event moves a run from draft to approved action?
7. What trace proves that the system followed the intended path?
8. What switch disables the risky behavior without disabling the whole product?

If a team answers these with prompt text alone, the design is still a prototype.

Layered View

The layers do not require separate services. A small system can implement several layers in one codebase. What matters is ownership: another engineer should be able to find where each decision is made, tested, logged, and rolled back.

Control Plane vs Execution Plane

Keep control decisions out of free-form model text. The model can propose, summarize, classify, or draft. Runtime-owned services decide whether the proposal may affect state, tools, memory, approvals, or users.

This split prevents a common failure: a model both proposes an action and implicitly authorizes it. Production architecture should make the authorization path visible and testable.

Maturity Ladder

Adopt the architecture in stages. Do not build every platform component before the system has proved value.

The ladder is not a status badge. It is a scope control. A level 1 system can be excellent for a read-only task. A level 3 system that cannot prove approval binding is not production-ready.

First Implementation Path

Start with one user workflow and one risk boundary. A useful first implementation should be small enough to review in one design meeting.

This path keeps architecture tied to evidence. Each added layer should close a named risk, not satisfy a diagram.

Request Flow

1. Authenticate the user or event source.
2. Create a goal record with constraints, user scope, and requested outcome.
3. Route the task to a direct answer, Agentic RAG, durable workflow, or multi-agent process.
4. Run policy checks before retrieval, tool use, or side effects.
5. Persist observations, decisions, tool calls, and outputs.
6. Verify evidence, output quality, and policy compliance.
7. Return an answer, request approval, refuse, or escalate.
8. Store traces for debugging and eval dataset improvement.

Example: Support Refund Request

A customer asks for a refund. The architecture should keep three decisions separate.

The model may draft a recommendation. It does not authenticate the user, choose its own tools, decide payment authority, write memory without policy, or mark the workflow complete without a stop reason. That separation is the point of the reference architecture.

Control-Point Mapping

Turn the architecture into implementation tickets by assigning every risky transition to a concrete control point.

If a row has no owner, test, or stored evidence, the architecture is still a drawing. The implementation is ready only when each transition can fail closed.

Control Points

• Before retrieval: enforce access control and source eligibility.
• Before tool use: validate schema, permission, budget, and approval needs.
• Before final answer: verify claims, citations, and policy.
• Before memory write: classify what kind of memory is being stored.
• Before release: run evals and regression checks.

Reference Implementation Slices

Build the architecture in slices. Each slice should produce one deployable proof, not a diagram-only promise.

This sequence keeps teams from building orchestration before they can prove state, policy, and evidence boundaries.

Deployment Shapes

The reference architecture does not require a large platform on day one. Use the smallest deployment shape that preserves the control points for the risk you are taking.

Compression is safe only when ownership stays explicit. A prototype can share code paths. It cannot share away the question of who owns state, policy, tool authority, evidence, and rollback.

Boundary Placement Guide

Use this guide when deciding whether a component belongs inside the agent runtime, the application service, or a separate platform service.

The split should follow ownership and risk, not fashion. A separate service without a stronger owner is just more distributed code.

Evidence Bundle

A design that claims to follow this architecture should attach these artifacts:

Run Trace Contract

A reference architecture is only real if one run can prove the boundaries worked. Capture the run as structured trace data, not as scattered logs.

type ReferenceRunTrace = {
  runId: string;
  tenantId: string;
  userOrEvent: string;
  goal: string;
  route: "direct_answer" | "rag" | "agent_loop" | "durable_workflow" | "multi_agent";
  stateVersion: string;
  context: Array<{
    source: string;
    accessDecision: "allow" | "deny";
    freshness: "current" | "stale" | "unknown";
    citationRequired: boolean;
  }>;
  modelDecisions: Array<{
    step: string;
    proposal: string;
    acceptedBy: "schema" | "policy" | "human" | "runtime";
  }>;
  policyChecks: Array<{
    boundary: "retrieval" | "tool" | "memory" | "approval" | "final_answer";
    decision: "allow" | "deny" | "escalate";
    policyVersion: string;
  }>;
  toolCalls: Array<{
    name: string;
    authority: "read" | "draft" | "write" | "execute";
    idempotencyKey?: string;
    result: "success" | "failed" | "denied";
  }>;
  finalStatus: "completed" | "refused" | "needs_approval" | "policy_blocked" | "evidence_missing" | "failed";
  stopReason: string;
  rollbackPath?: string;
};

The exact field names can change. The invariant should not: a reviewer can reconstruct who asked, what evidence entered context, what the model proposed, which policies ran, which tools executed, why the run stopped, and how to reverse or disable the risky path.

Architecture Acceptance Tests

Before a pilot, run at least these acceptance checks against the trace contract.

If these tests are hard to write, the architecture is probably still hidden in framework glue, prompts, or informal process.

Runtime Components

• Identity and tenant boundary
• Goal and state store
• Prompt and instruction registry
• Tool gateway
• Retrieval router
• Memory service
• Policy engine
• Approval service
• Workflow engine
• Evals service
• Trace and audit store

Ownership Matrix

Assign owners before production. Missing ownership is an architecture defect.

One person can own several cells in a small team. The table still matters because every production failure will ask who owned the boundary.

Design Review Questions

Ask these questions before a production pilot:

1. What decision does the model make, and what decision does software make after that?
2. Which state changes survive a crash, retry, or approval wait?
3. Which tool call has the highest cost, risk, or irreversibility?
4. What evidence is required before the system answers or acts?
5. What happens when evidence is missing, stale, conflicting, or unauthorized?
6. What gets written to memory, and who can correct or delete it?
7. Which eval blocks release if the system regresses tomorrow?
8. Can an operator reconstruct the failed run without guessing?
9. What can be disabled without redeploying the whole system?
10. What would make the team shut the agent off?

Minimum Production Checklist

• Explicit owner for goal and state
• Tool schemas and validation
• Human approval for high-risk actions
• Access-controlled retrieval
• Citation validation for grounded answers
• Evals for core tasks
• Traces for every run
• Budget, timeout, and cancellation controls
• Incident review path
• Rollback or disable switch

Reference Architecture Review Gate

Use the reference architecture review checklist before a pilot or production release. Block release when:

• the model owns authentication, authorization, policy, or side-effect permission;
• durable state has no schema, replay path, or stop reason;
• retrieval sources lack access control, trust, freshness, or citation checks;
• approvals authorize vague future behavior;
• evals inspect only final text and ignore tools, policy, evidence, and state transitions;
• operators cannot reconstruct a successful run and a failed run from traces.

Scaling Path

Start small:

1. Single agent with tool validation.
2. Add goals and state.
3. Add retrieval and citation checks.
4. Add durable workflow for long-running tasks.
5. Add human approval gates.
6. Add eval datasets and observability.
7. Add multi-agent decomposition only when one agent becomes a bottleneck.

Anti-Patterns

Avoid these designs:

• The model chooses from a broad tool list without route-level permission.
• Prompt text is the only policy boundary.
• Retrieval, memory, and tool results are merged into context without provenance.
• The system can act but cannot replay why it acted.
• Approvals authorize vague future behavior instead of one exact action.
• Evals check only final answer quality and ignore tool use, policy, evidence, and stop reason.
• Rollback requires deleting the whole deployment because prompts, tools, and policies cannot be disabled independently.

Related Chapters

• Agentic System Architecture
• Agentic RAG Systems
• MCP-first Tool Use
• Production Runtime Overview
• Durable Workflows
• Policy Enforcement
• Observability and Evals