AcademyOperate

Building and Running AI Agents in Production

Enterprise agents from design to production: when to use them, how they are built, how MCP and A2A connect them, which threats they introduce, how to give them their own identities with least-privilege access, and how to evaluate and operate them.

O301Advanced12 slidesAbout 10 minutes

Start the course Download the PDF
1 / 12
PDF
AxiareteAcademy

O301Operate · Advanced

Building and Running AI Agents in Production

Architecture, protocols, identity, evaluation and operations for enterprise AI agents.

FormatKnowledge pack · 12 slides
ForPlatform leaders and architects
LevelAdvanced
Axiarete Academy · O30102 / 12

Definition

An agent is a model, tools and context running in a controlled loop

An agent uses a model to choose an action, calls a tool, observes the result and continues until it reaches a goal or a limit. Readiness depends less on the model than on what the agent can access, what context it has and how its actions are checked.

AccessTools, data and permissions.
ContextRetrieved knowledge, memory and instructions.
ControlLimits, approvals, logs and evaluations.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30103 / 12

Anatomy

The anatomy of an enterprise agent

InstructionsThe goal, role, constraints and definition of done.
ModelReasoning and planning, chosen per task for quality and cost.
ContextRetrieved knowledge, conversation state and memory.
ToolsAPIs, MCP servers and sandboxed code execution, each with its own permissions.
ControlIdentity, budgets, approvals, logging and a kill switch.
Design ruleEnforce permissions and budgets outside the model; injected text can override prompts.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30104 / 12

Patterns

Choose the simplest pattern that works

PatternUse whenMain risk
Fixed workflow with model stepsThe process is known in advanceBrittle when cases vary
RouterRequests fall into distinct typesMisrouting
Tool-using agentSteps vary from case to caseUnbounded loops and cost
Orchestrator and sub-agentsTasks split into independent partsCompounding errors; about 15× chat tokens (Anthropic)
Human checkpointsActions are high-impact or irreversibleApproval fatigue
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30105 / 12

Protocols

Protocols are becoming the integration layer

Model Context ProtocolConnects agents to tools and data through MCP servers. Introduced by Anthropic in November 2024; hosted since December 2025 by the Linux Foundation’s Agentic AI Foundation.
Agent2AgentLets agents from different vendors discover each other and exchange tasks. Introduced by Google in April 2025; version 1.0 in March 2026; hosted by the same foundation since August 2026.
What it means for ITTreat every MCP server and agent endpoint as an integration: inventoried, owned, authenticated, version-pinned and reviewed on change.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30106 / 12

Threats

Threats that agents introduce or amplify

Mapped to the OWASP Top 10 for LLM Applications 2026 (LLM) and the OWASP Top 10 for Agentic Applications for 2026 (ASI).

Prompt injection (LLM01)Typed directly, or hidden in documents, emails, tickets or web pages.
Agent goal hijack (ASI01)The agent is redirected toward an attacker’s objective.
Tool misuse (ASI02)Legitimate tools used in harmful ways or sequences.
Identity and privilege abuse (ASI03)Inherited or cached rights used beyond the user’s authority.
Agentic supply chain (ASI04)Tampered tools, MCP servers or tool descriptions.
Memory and context poisoning (ASI06)Corrupted context that persists across sessions.
Cascading failures (ASI08)Errors that propagate between connected agents.
Human-agent trust exploitation (ASI09)Persuasive explanations that win approval for unsafe actions.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30107 / 12

Identity and access

Give every agent an identity and only the privilege it needs

MCP authorization is optional; over HTTP it follows the OAuth 2.1 draft, accepts only tokens issued for that server and forbids passing them upstream.

Its own identityA non-human identity per agent; never shared credentials.
Delegated authorityToken exchange (RFC 8693) issues tokens naming both user and agent, within the user’s rights.
Short-lived credentialsIssued per task and expiring quickly.
Scopes per toolPermissions granted tool by tool, never platform-wide.
Named approversHigh-impact actions need a named approver.
TraceabilityEach action linked to the requester, agent, tool and result.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30108 / 12

Evaluation

Evaluate before release and after every change

1Define successWhat a correct outcome is, task by task.
2Build the setReal failures, edge cases and adversarial ones.
3Test offlineGrade end states across repeated runs; fail unsafe steps.
4Red-teamInjection, data exfiltration and privilege abuse.
5Release in stagesShadow, then limited, then general.
6Re-test on changeEvery model, prompt or tool change reruns the set.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30109 / 12

Operations

Operate agents like production services

OpenTelemetry’s GenAI agent span conventions still have Development status, so pin a version and expect change.

Verified task successShare of tasks whose end state is verified correct by sampling.
Escalation rateShare of tasks handed to people, and the reasons.
Intervention rateShare of actions people stopped or reversed.
Cost per outcomeModel, tool and review cost per verified task.
Policy blocksActions prevented by guardrails, by type.
Evaluation regressionChange in evaluation scores after each release.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30110 / 12

Readiness

The production readiness checklist

Named ownerAccountable for outcomes, cost and incidents.
Identity and scopesReviewed like any privileged account.
Tool inventoryEvery MCP server and API reviewed and approved.
BudgetsLimits on tokens, steps, time and spend.
Approval gatesFor consequential or irreversible actions.
Tracing and logsPrompts, tool calls and results, within retention rules.
Evaluation in CIRegression cases gate every change and should pass at close to 100%.
Incident runbookIncluding how to stop the agent and revoke its access.
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30111 / 12

Pitfalls

Common failure patterns in agent projects

An agent where a workflow would do
Use deterministic steps where the path is known
One agent with broad access
Scope permissions per tool and per task
Judging readiness from demos
Keep a versioned evaluation set of real cases
No limits on cost
Set budgets per run, per user and per day
Trusting retrieved content
Treat documents and tool output as untrusted
Unversioned prompts and tools
Version, review and roll back like code
Building and Running AI Agents in ProductionOperate
Axiarete Academy · O30112 / 12

Summary

Key takeaways

  1. Prefer a fixed workflow when the path is known; add agency only where steps vary.
  2. Treat MCP servers and agent endpoints as integrations to inventory and secure.
  3. Every agent needs its own identity, scoped delegated authority and a full audit trail.
  4. Evaluate on real cases before release and after every change.
  5. Run agents with budgets, approvals, tracing and a kill switch.
Axiarete AcademyExplore every trackTen courses across five tracks
Building and Running AI Agents in ProductionOperate

Further reading

Continue learning

O201 · OperateAI in IT OperationsAxiarete AcademyAll courses and tracks

Bring this course to your leadership team

We run private working sessions that apply these frameworks to your own estate, data and pending decisions.

Request a Private BriefingExplore the Academy