The shift from Large Language Models (LLMs) as stateless API endpoints to sophisticated, stateful AI agents represents a fundamental architectural change in how intelligent systems are developed and deployed. This blog defines the core components and features that distinguish an AI agent from a basic LLM wrapper.
Defining the AI agent
An AI agent can be fundamentally viewed as an object-oriented class that wraps and orchestrates calls to an underlying LLM, which serves as the decision-making engine. Unlike stateless LLM API calls, an agent is instantiated with some attributes that provide persistent functionality and context: Identity and Persona: The agent is given a specific role (e.g., architect, software developer), tasks, and objectives, defining its operational scope; and Context: A defined context, similar to a dynamic system prompt, is provided to orient the agent to its purpose and constraints.
Core agentic capabilities
To fulfill its objectives, an agent is augmented with Tool Use and Memory. These are described below.
Tool Use (Action): Tools are external capabilities that allow the agent to move beyond text generation and take actions in the real world. These are typically APIs, functions, or interpreters that enable deterministic execution. The agent utilises the LLM for reasoning and planning (Chain-of-Thought) to decide when and how to invoke these tools to achieve a given task. Tools can range from executing code and database queries to interacting with external applications and initiating search operations.
Memory: Memory enables an agent to maintain state and learn over time. This is categorised into:
- Short-term Memory: The session context (e.g., the current conversational history), which ensures that subsequent requests are aware of previous interactions within the same session.
- Long-term Memory: Persistent storage (e.g., profiles, interaction summaries, process logs) that allows the agent to continuously improve its performance, refine its process sequences, and build user-specific profiles. This mechanism enables agents to become incrementally smarter through experience.
These components transform the base LLM into a stateful, autonomous object capable of complex task execution, requiring new architectural protocols for interaction with external resources and other agents.
Agent-to-world interaction
An AI agent’s functionality depends on its capacity to interact with the external environment through the execution of specified, deterministic tasks, primarily through Tool Use. This requirement has driven an architectural transition from tightly coupled systems to standardised, distributed protocols.
Initial architecture: Local tool implementation
Early implementations of agent systems typically co-located the agent and its tools on the same application host. In this arrangement, tools were implemented as local functions or APIs running alongside the agent. This approach results in a monolithic solution offering no standardisation for tool definition and limiting the architectural separation of distinct components.
Standardisation through the MCP server protocol
The Model Communication Protocol (MCP) Server introduced a standardised interface for agent-tool interaction. This server acts as an intermediary, providing a consistent mechanism for the agent to access tool execution, data resources, and contextual prompts.
Local MCP: In this case, the MCP Server is deployed on the same application instance as the agent. The communication protocol used in this coupled environment is standard input/output (stdio). The agent transmitted a request to the local MCP Server via stdio, which performed the necessary tool logic and returned the result. The primary advantage of this stage is the creation of a framework-independent and standardised method for tool integration.
Remote MCP and Distributed Architecture: To support enterprise architectures, the design has progressed by separating the agent from the tool host, enabling distributed operations. This involves relocating the MCP Server to a dedicated, remote application instance, defining the Remote MCP Server.
Facilitating distribution with streamable HTTP
The complexity inherent in this distributed transition is managed by migrating the communication protocol from stdio to streamable HTTP. For the agent designer, the shift from a local stdio link to a remote streamable HTTP connection is abstracted. The agent interfaces with an MCP Client, which handles the network layer requirements.
The MCP Client and Server are engineered to manage the network challenges associated with remote execution. Streamable HTTP is employed to allow the remote tool execution, which may involve extended or compute-intensive processes, to deliver its results efficiently. This protocol permits the agent to receive responses incrementally, providing a unified interaction experience that mirrors the perceived responsiveness of a local stdio call.
This architectural evolution successfully achieves separation of concerns, enabling independent management and deployment of specialised tool servers while ensuring that agents maintain a uniform, standardised interface for external interaction.
Advanced agent-to-world protocols
The Model Communication Protocol (MCP) Server extends beyond basic tool execution to offer capabilities that enrich the agent’s contextual awareness and introduce dynamic interaction within workflows. These capabilities include resources, prompts, and the newly integrated features: elicitation and sampling.
The MCP Server provides resources, which allow an agent to query and fetch specific data from external sources. The interaction is a two-step process: the agent first submits a query to the MCP Server, which returns a list of matching data items or available schemas. The agent then selects the necessary item from that list to retrieve the actual data payload. Additionally, the MCP Server can supply dynamic prompts or context sequences. Instead of relying solely on internal agent context, the agent can request process sequences or operational constraints directly from the MCP Server, ensuring that it adheres to specific, externally defined workflows or logical steps provided by the tool owner. Security considerations are managed through integrated authentication and authorisation features, which are fundamental in distributed systems for controlling agent access to specialised tools and resources.
A recent addition to the MCP protocol involves new features implemented on the client side: elicitation and sampling. These functionalities enable the creation of interactive APIs, challenging the definition of stateless application programming interfaces (APIs). An external tool call, orchestrated via the MCP Server, can now function as a multi-step workflow.
Elicitation is the mechanism that facilitates a human-in-the-loop interaction within an ongoing workflow. If a tool running on the remote MCP Server reaches a point in its execution requiring clarification or confirmation, it initiates an elicitation request. This request is routed back to the MCP Client, which then communicates with the user to obtain the necessary input. Once the user provides the information, the MCP Client sends the data back to the MCP Server, allowing the tool workflow to resume and complete its original task.
Sampling provides a parallel functionality for managing LLM interaction and costs. If a tool on the MCP Server requires an LLM call during its execution, the MCP Server can initiate a sampling request, sending the prompt back to the MCP Client. This design abstracts the LLM dependency away from the MCP Server owner. The client application then handles the interaction with the chosen LLM provider—be it a proprietary or managed service—and is responsible for the associated computational costs and rate limits. Once the generated response is received, the MCP Client forwards it back to the MCP Server. This mechanism allows tool providers to offer advanced functionalities without being financially burdened by the LLM calls necessitated by their tools, while also giving the client full control over model choice and compliance requirements.
Furthermore, the data transmitted between the MCP Client and Server is no longer restricted to text. The protocol now supports rich data types, including image and voice payloads, enabling agents to process and respond to multimodal inputs.
The sequence diagram of these interactions are shown below.
Agent-to-agent (A2A) workflows
While agent-to-world interaction enables a single agent to take action, agent-to-agent interaction allows a collection of agents to collaborate on complex objectives. These workflows always used to be centralised, single-instance implementations. However, more recent frameworks such as Google’s Agent2Agent (A2A) framework and IBM’s Agent Communication Protocol (ACP) allow distributed implementation of these multi-agent systems. Since A2A is more widely used, we’ll limit our explanation to this framework. But before that, let’s quickly discuss the patterns that are used in agent interactions.
Agent interaction patterns
In multi-agent development, several core interaction patterns are used for task decomposition and execution. These are briefly explained below.
One common pattern is the deterministic sequence. A principal or router agent receives a user request and routes it to specialised agents in a predefined, sequential, or parallel order. This approach is suited for tasks with established procedures, mirroring a well-defined organisational workflow where the steps and responsibilities are known in advance.
Another essential pattern is reflection. After a task is completed by one agent, its output is passed to a second agent (also called the critic) which reviews the work, identifies errors, and provides targeted feedback. This creates an internal feedback loop, where the first agent iterates on its output until the critic approves the result. This pattern prioritises result accuracy over execution latency, making it valuable for processes where mistakes carry a high cost. We should mention here that sometimes this is called inline validation.
The most complex pattern involves non-deterministic interaction led by an expert agent. Upon receiving a complicated request, the expert agent performs planning, effectively generating a chain of thought to decompose the task into sub-tasks. It then delegates these sub-tasks to specialised agents. In this scenario, the expert does not dictate the granular steps but rather the overall strategy, allowing the specialised agents to interact and collaborate among themselves to complete their delegated parts. This method is used to model human-like problem-solving but introduces substantial complexity and reduced predictability.
Distributed Agent2Agent framework
To move A2A collaboration into enterprise environments, where different teams manage specialised agents, a distributed framework is required. The Agent to Agent (A2A) framework, developed by Google, provides a standardised protocol for this distributed interaction, though custom solutions using services like Amazon SQS can also be implemented.
The process begins with a client agent needing assistance for a task. It first queries potential remote agents for their capabilities. Each remote agent responds with an Agent Card, a JSON metadata structure that serves as a digital business card. This card details the agent’s identity, a list of its specialised skills or tasks, its communication protocols, endpoint URL, and required authentication details.
The client agent uses the information from the Agent Cards to decide which remote agent is best suited to handle the request. It then sends the task to the selected remote agent’s endpoint.
A core feature of the A2A framework is its use of asynchronous communication via queues. When the client sends a task, it enters a queue managed by the remote agent’s host. The remote agent can respond immediately with a status update, indicating whether the task is queued, currently running, or being declined. The client agent can then poll the remote agent for status updates. Once the task is completed, the remote agent notifies the client agent, which then retrieves the results.
This asynchronous queuing mechanism is crucial for managing system stability, particularly due to the computational demands and rate limits associated with the underlying large language models. While synchronous interactions are supported, the queue-based, asynchronous approach enables better resource control and horizontal scaling across specialised agent teams.
Productionising agent systems
Deploying complex, distributed agentic systems, which rely on orchestrating multiple models, tools, and protocols, requires robust and scalable infrastructure. Managed cloud services address the operational challenges of hosting, securing, and scaling these architectures.
The foundational requirement for any agent is access to the underlying computational engine. Amazon Bedrock serves as a key LLM provider platform, offering API access to a variety of foundation models. It centralises model access and provides essential governance layers, including responsible AI enforcement through guardrails, and secure context provision via knowledge bases used for Retrieval-Augmented Generation (RAG).
Building upon the LLM foundation, Agent Core is a managed service designed specifically for deploying and operating these complex agent architectures within the AWS ecosystem. Agent Core provides a serverless runtime environment, abstracting away the infrastructure management associated with running long-lived agents.
Key capabilities provided by Agent Core include automated memory handling, which manages the short-term session context and long-term memory retrieval required by stateful agents. Crucially for enterprise deployment, Agent Core integrates built-in identity and access control (IAM), ensuring that agents and their components adhere to strict security boundaries and that tool access is properly authorised.
For developers seeking an open-source option with native integration into this infrastructure, the Strands Agent Python SDK is available. This SDK offers a framework for building agents that align well with the MCP Server protocol for Agent-to-World interactions and includes support for the distributed Agent-to-Agent framework. By leveraging these managed services and frameworks, organisations can shift their focus from solving infrastructure challenges to implementing the specialised logic and capabilities of their agent fleets.
Conclusion
The evolution of AI agents has moved rapidly from simple, centralised LLM wrappers to sophisticated, distributed systems governed by standardised protocols. The Model Communication Protocol (MCP) Server defined how agents interact with the external world through tools, evolving from local connections to distributed streamable HTTP. New features like elicitation and sampling have transformed external interactions into interactive, cost-controlled workflows. Concurrently, the Agent to Agent (A2A) framework has established a structured methodology for orchestrating specialised agent teams using metadata structures like Agent Cards and asynchronous queues. By adopting managed services such as Amazon Bedrock and Agent Core, the technical complexity of deploying and governing these advanced, autonomous agent architectures is managed, accelerating their readiness for production use.
