Lab 07 - Package Agents, Tools, Workflows, Memory, and Evals

Download the Lab 07 runtime packaging guided exercise worksheet, lab completion worksheet, and lab production readiness worksheet before you start.

Objective

Use a Mastra-style TypeScript runtime shape to package the same responsibilities you built from scratch: agent decision, tool execution, workflow control, memory, trace events, and evals.

What You Will Use

• Language: TypeScript
• Framework/runtime: Mastra-style runtime packaging
• Framework-agnostic lesson: a framework can package primitives, but your application still owns state, policy, tool contracts, traces, and acceptance criteria.
• Pattern chapters: Mastra Runtime, Building a Minimal Agent Runtime, Observability and Evals
• Source files:
  • mastra-runtime-pattern/typescript/src/runtime_packaging.ts
  • mastra-runtime-pattern/typescript/src/run_demo.ts
  • mastra-runtime-pattern/typescript/test/runtime_packaging.spec.ts

Exercise Time Budget

These estimates assume dependencies are already installed.

Setup

From the repository root:

npm install

This lab is deterministic and does not require a model key. The point is runtime packaging, not model quality.

Run It

npm run mastra-runtime:demo
npm run mastra-runtime:test

Expected Result

The test command should print:

Mastra-style runtime packaging tests OK

The run should also prove these behavior signals:

• policy is read before drafting;
• the draft is created for review, not sent;
• workflow steps and tool results are traced;
• the eval fails if a direct send or refund-issue tool is used.

The demo command should include this evidence:

{
  "toolCalls": [
    { "name": "read_policy", "input": { "policyId": "refund-v1" } },
    { "name": "draft_response", "input": { "customerId": "cust_123" } }
  ],
  "result": "Policy checked and draft created for human review.",
  "evaluation": {
    "status": "pass"
  }
}

The exact JSON also includes memory and trace fields. Use the toolCalls, result, and evaluation.status fields as the quick check.

Use this flow as the lab’s acceptance model. A framework can package the runtime, but the application still owns tool order, side-effect policy, trace evidence, and release evals.

Native Mastra comparison point:

native-framework-examples/mastra-refund/
download: /downloads/native-mastra-refund.zip
agent: refundDraftAgent
workflow: refundDraftWorkflow
tools: refund_policy.retrieve, refunds.create_draft
eval gate: refund_draft_no_money_movement

Guided Exercises

Use these exercises to inspect what the runtime package owns and what product policy still owns.

Deployment And Rollback Exercise

Treat the lab as a production candidate. Before a real runtime ships, write the rollback action that disables the highest-risk capability without deleting the whole service.

Inspect The Code

Open mastra-runtime-pattern/typescript/src/runtime_packaging.ts and find these boundaries:

• Agent: owns instructions and proposes the next decision.
• Tool: owns typed capability execution.
• WorkflowStep: owns deterministic runtime sequence.
• RuntimeState: owns memory, traces, tool calls, and result.
• evaluateRuntime: checks trajectory, not only final text.

The local code is intentionally small, but the mapping follows the production shape: agents, workflows, tools, memory, traces, and evals belong in one runtime package.

Change One Thing

Add a new tool named send_message with a direct side effect, then update evaluateRuntime so the eval fails if that tool appears in toolCalls. The repository test also checks the refund side-effect case:

{ name: "refunds.issue_refund", input: { amount: 42 } }

Expected lesson: framework tool registration is not permission. Application policy still decides what is safe.

Verify

Run:

npm run mastra-runtime:test

Compare the output with the expected result above before moving to the production extension.

Lab Review Gate

Before moving on, verify the runtime package:

Record the command output, trace events, tool order, and eval result in the lab completion worksheet.

Production Extension

Before a real Mastra implementation ships, add:

• real tool schemas and input validation;
• workflow retry and compensation policy;
• memory retention, redaction, and deletion controls;
• scorer/eval fixtures tied to release gates;
• trace export to the observability backend;
• approval gates for write tools and external communication.

Production Bridge

Use this table when adapting the lab to a real Mastra runtime:

The first production milestone is a runtime slice that can be disabled, replayed, and evaluated without reading framework internals.

Native Framework Extension

After the deterministic lab passes, port one vertical slice into a real Mastra project. Use Real Framework Setup Notes for current setup commands and compare your work with the repository example at native-framework-examples/mastra-refund/.

Native porting steps:

1. scaffold a Mastra project;
2. create one agent for the refund draft decision;
3. create typed tools for policy lookup and draft creation;
4. create a workflow that calls policy lookup before draft creation;
5. add an eval that fails if a send or refund-issue tool is called;
6. export a trace with workflow, agent, tool, policy, and eval events;
7. document rollback for disabling the draft tool.

Keep these assets outside framework-only code:

Completion standard: the native project proves the same behavior as this lab and links to the Support Refund Agent capstone. A native Mastra run is not complete just because an agent responds; the workflow must preserve tool order, side-effect policy, trace evidence, and eval gates.

Troubleshooting

Cross-Framework Mapping

• In LangGraph, this becomes graph state, nodes, tools, checkpoints, and evals around graph runs.
• In Mastra AI, these concerns are packaged as agents, workflows, tools, memory, observability, and evals.
• In AutoGen-style systems, the runtime package is split across manager, workers, functions, messages, and transcript evals.
• In CrewAI, the equivalent packaging splits between flow state, crew tasks, agent roles, tools, and flow-level evaluation.

Related Chapters

• Mastra Runtime
• Framework Selection
• Building a Minimal Agent Runtime
• Production Evaluation Feedback Loops
• Support Refund Agent Capstone