Recent advancements in generative AI infrastructure have introduced enterprise-grade agentic retrieval capabilities within Amazon Bedrock. Unlike traditional, single-shot Retrieval Augmented Generation (RAG) systems, which perform one retrieval and one generation, these agentic architectures utilize reasoning agents to iteratively plan and retrieve information as needed. This approach addresses common RAG limitations, such as handling multi-corpus queries or questions requiring refinement after an initial look.
The shift to Managed Knowledge Bases
Amazon Bedrock now provides Managed Knowledge Bases (Type: MANAGED), which automate the ingestion, storage, indexing, and retrieval processes. This eliminates the need for users to manually provision, scale, or maintain vector databases. By using the AgenticRetrieveStream API, developers can access first-class agentic capabilities, including multi-turn planning and tool-driven retrieval. This managed approach integrates with the AgentCore Gateway, which exposes knowledge bases as Model Context Protocol (MCP) tools for the agent to consult.
Observability and evaluation as core requirements
As agentic systems introduce complex, loop-based reasoning, traditional debugging methods become insufficient. To address this, the current deployment model incorporates comprehensive observability and evaluation directly into the infrastructure. By utilizing the AWS Cloud Development Kit (AWS CDK), developers can provision a complete solution that includes two Amazon CloudWatch dashboards and seven layers of telemetry, covering areas such as ingestion metrics, token usage, retrieval quality, and evaluation scores.
Infrastructure as code with AWS CDK
Using AWS CDK for this architecture simplifies the management of interconnected stacks. The system consists of four distinct stacks—managing knowledge bases, gateways, agents, and observability dashboards—that are cross-referenced to derive a deployment sequence automatically. This approach replaces manual export/import bookkeeping, reducing the risk of configuration drift and ensuring that complex systems remain maintainable.
Evaluation is supported through both on-demand and continuous paths. The system enables automated scoring using built-in evaluators—such as faithfulness, correctness, and tool-selection accuracy—which provide continuous feedback on the agent’s performance. These evaluations are essential for maintaining groundedness and auditability in enterprise applications, ensuring that synthesized answers can always be traced back to their source passages.
Source: aws.amazon.com
Original article: https://aws.amazon.com/blogs/machine-learning/build-agentic-retrieval-on-a-managed-amazon-bedrock-knowledge-base-with-cdk/
Leave a Reply