Blogs

Key Takeaway

• Production AI agents require deliberate decisions across six areas: build, test, run, secure, observe, and govern. A gap in any one area is enough to stall a deployment.
• Testing must cover more than whether an agent gives correct answers. It needs to catch adversarial inputs, policy violations, and cumulative session risks that only appear under real conditions.
• Security for AI agents works at three layers: what content is permitted, what actions are authorised, and what the agent’s infrastructure is allowed to touch. All three need to be in place before the agent goes live.
• Observability for agents is different from standard application monitoring. The signals that matter are tool selection accuracy, guardrail block rate, tokens per session, and task completion rate.
• For businesses in Malaysia building on AWS, VSTECS KU can help structure agentic AI strategy across all six pillars, from initial architecture decisions to fleet-wide governance.

Introduction

Building a demo AI agent can take days. Getting it into production, reliably and with real consequences at stake, is a different challenge.

Pilots stall not because the agent cannot do the work, but because the structure around it is not ready: policy controls, adversarial testing, observability setup. These are the pieces that get deferred until after something goes wrong.

AWS’s six-pillar framework for production AI agents breaks that work into six areas, each targeting a failure mode the others cannot cover. This article walks through what each pillar requires and where gaps appear in practice.

The question is not whether your agent handles the expected case. Whether it handles the unexpected one is what matters.

Here is how those failure modes break down across all six areas.

How the Six Pillars Map to Real Deployment Risks

Here is what each pillar covers and the main risk it controls:

AWS has published a detailed walkthrough of the six-pillar framework for teams building production agents on Bedrock.

Pillar 1: Build — Model, Framework and Data Access Decisions

Three decisions made at the build stage determine what every other pillar can achieve.

Model Selection

Match the model to the task, not to the benchmark. Test three to four candidates against your own tools and data, covering tool-calling accuracy, reasoning depth, latency, and cost. Revisit the choice every quarter as models improve. Amazon Bedrock provides a unified API that makes swapping models straightforward once you have shortlisted them.

Framework Selection

Strands Agents is AWS’s open-source option. It supports Model Context Protocol (MCP) for tool access and lets agents communicate with each other directly. LangChain and LangGraph suit stateful graph orchestration. CrewAI suits multi-agent coordination where specialist agents hand work to each other.

Knowledge and Tool Access

Static documents feed a knowledge base for retrieval-augmented generation. Real-time data comes through MCP tools. Specialist agents connect via Agent2Agent protocol. Each pattern suits different scenarios. Use the one that matches how current your data needs to be. The wrong choice creates errors that are hard to trace back to the source.

Pillar 2: Test — Correctness, Safety and Evaluation Before Launch

Unlike an application that either returns the right output or does not, an AI agent can pass every test and still behave incorrectly in production. Real inputs look nothing like controlled test scenarios. Enterprise AI readiness in testing requires three gates.

Gate 1: Correctness

Measure at the session, trace, and tool levels. Use both LLM-based judges and code-based evaluators, and run batch evaluations against expected tool sequences, not just expected outputs. AWS’s example thresholds are a safety score of 0.99 and a correctness score of 0.90, calibrated to risk. The right thresholds for your deployment depend on what the agent does and what a mistake costs.

Gate 2: Safety

Run adversarial inputs through your guardrails before launch. The attack surface is wider than your team may expect: prompt injection, off-topic requests, PII extraction, indirect injection through tool output or retrieval content, and supplier manipulation are realistic vectors. Test for over-refusal too, because filters that are too strict create their own reliability problems.

Gate 3: Evaluation Loops

An inner loop runs quick checks during development. An outer loop runs in CI/CD and blocks promotion when the score drops below the threshold. A production loop runs A/B testing after launch, with automatic rollback when the new version underperforms. Set up all three before the agent goes live. Amazon Bedrock AgentCore Evaluations supports the scoring layer across all three.

Pillar 3: Run — Matching Your Deployment Pattern to the Use Case

AI deployment planning for the run pillar comes down to one question: does your agent need to hold a live conversation, coordinate specialist work, or run in the background without a user present?

Interactive Agents

These run in real time with persistent memory across conversational turns. They suit customer-facing tasks where the user expects a continuous conversation.

Multi-Agent Systems

A supervisor agent routes work to specialist agents. Sessions can run for up to eight hours on isolated cloud instances, or persist across several days for longer workloads. This pattern suits complex workflows where different agents handle different domains.

Autonomous Agents

These agents run in the background, triggered by a schedule, an external event, or a signal from another agent. No live user is needed.

Before committing to a custom build, check whether an existing AWS agent already covers your use case. AWS provides pre-built agents for developer operations, security tasks, and autonomous workflows. Adapting one avoids known failure modes and ships faster.

Once the deployment pattern is settled, the next question is what the agent is allowed to do — and what stops it from doing something it shouldn’t.

Pillar 4: Secure — Three Layers Every AI Agent Needs Before Going Live

Security for production AI agents works at three layers, and each one covers something the others do not.

Content Filtering

Amazon Bedrock Guardrails applies denied topics, PII detection, and prompt-attack detection at the content level. This layer controls what the agent is allowed to say and what inputs it is allowed to act on.

Action Authorisation

Cedar policies (the policy language that defines what each agent is allowed to do) are enforced at the AgentCore Gateway on every tool call. A well-designed policy is specific: it restricts actions by role, operating hours, dollar amount, or any other business rule that applies. AgentCore Temporal Policies add cumulative controls at the session level: rate limits, per-session token budgets, and prerequisite checks. A prerequisite check means one action must complete before another is allowed to run.

Infrastructure Access

Give each agent its own restricted IAM role with only the access it needs. Explicitly deny destructive actions: deleting data, changing IAM permissions, or provisioning infrastructure. This reduces the impact if an agent is manipulated or behaves unexpectedly.

Content filtering alone is not enough for an agent that can take real actions. All three layers must be active before the agent goes live.

Pillar 5: Observe — Tracking Whether Your Agent Behaves Correctly Over Time

Latency, error rate, and uptime are table stakes. The metrics that tell you whether a production AI agent is behaving correctly are different: tool selection accuracy, tokens per session, guardrail block rate, and task completion rate.

AgentCore Observability tracks every tool call, token count, and guardrail intervention, and outputs them in a format your monitoring stack can read.

Watch for gradual drift. This is where agentic AI strategy shifts from launch to long-term reliability. Agent degradation can be slow and easy to miss: tool selections get slightly worse week over week, more tokens are needed to reach the same output, and guardrails start blocking at a higher rate. Anomaly detection on weekly trends catches this before it becomes visible to users.

Set specific agent-level alarms. A CloudWatch alarm set at 500,000 tokens in a five-minute window catches a runaway loop (an agent stuck repeating calls) before the bill arrives.

Pillar 6: Govern — Fleet-Wide Policy and Emergency Response

Governance gets complicated when agents multiply across teams. Enterprise AI readiness at the governance layer means having a shared policy and a clear response plan before any single team runs ahead on their own. This mirrors the operational excellence principles in the AWS Well-Architected Framework, applied specifically to an agent fleet.

AgentCore Registry stores a record of every agent in your fleet: who owns it, what stage it is at, its risk level, and when it was last reviewed. Teams can search the registry before building something new, which reduces duplication and keeps your agent inventory auditable.

Fleet-wide enforcement runs through AgentCore Gateway. The same Cedar policies, token budgets, and access controls apply to every agent behind the gateway, regardless of which team built it.

Plan your emergency response before you need it. Three options can be combined:

• Update a Cedar policy at the gateway to suspend a specific agent or team
• Set an AWS Budgets action to trigger a policy cut-off when spend hits a ceiling
• Use CloudWatch alarms to run automated runbooks that restrict IAM access, log the event, and notify stakeholders

For industries with regulatory requirements such as finance, public sector, and healthcare, a documented emergency response procedure is part of AI deployment planning, not an optional extra.

Frequently Asked Questions

1. What makes production AI agents different from prototypes?

A prototype runs under controlled conditions to show capability. A production AI agent operates with real users, unpredictable inputs, broad tool access, and live business consequences if something goes wrong. Enterprise AI readiness means covering all six pillars. Skipping any one of them means the agent is still effectively a prototype, regardless of how well it performed in a test environment.

2. What should be in place before deploying production AI agents?

Cover all six pillars before your agent handles real workloads. In practice, teams address three areas too late: safety testing for adversarial inputs, temporal policy controls that enforce cumulative rules across a session, and fleet-level governance that applies consistent policy across all agents. Each is straightforward to set up during AI deployment planning and difficult to retrofit after an incident.

3. How is AI agent security different from standard application security?

Standard application security protects infrastructure and data. AI agent security adds two layers on top of that: content filtering (what the agent is allowed to say and act on) and action authorisation (what the agent can do through its tools). Both layers need separate configuration, because an agent that passes content filtering can still cause harm through its tool calls if action authorisation is not in place.

4. What does AI deployment planning look like in practice?

Effective AI deployment planning works through the six pillars in parallel rather than one at a time. Model selection, framework choice, and tool access get resolved during the build phase and inform what testing is needed. Design security, observability, and governance before launch, not added afterwards. In Malaysia, the planning process is also where businesses should map data residency requirements under PDPA, and sector-specific regulations such as Bank Negara Malaysia’s Risk Management in Technology policy to the deployment architecture.

5. When should a business bring in an AWS partner for agentic AI strategy?

Bringing in a partner at the planning stage means each pillar gets the right design from the start, rather than being fixed after a deployment problem. VSTECS KU’s AgentCore architecture and professional services team work across the full stack: architecture, AgentCore configuration, Cedar policy writing, and the governance layer your agent fleet needs to operate safely over time.

Getting production AI agents right means covering each pillar. Testing catches problems that only appear under realistic inputs. Security requires all three layers to be active before launch, and observability tells you whether the agent is behaving correctly, not just whether the servers are healthy. Governance ensures that what applies to one agent applies to all of them, across every team. VSTECS KU’s AWS cloud solutions team can help structure the architecture, configuration, and governance layer across all six pillars.