Agentic Retrieval on Amazon Bedrock Managed Knowledge Bases: The AWS CDK Blueprint

Most retrieval-augmented generation (RAG) systems do one thing: embed the question, fetch the nearest chunks from one index, hand them to a model, and return the answer. An AWS Machine Learning blog post published October 26, 2026, Build agentic retrieval on a Managed Amazon Bedrock Knowledge Base with CDK, shows what changes when a reasoning agent sits in that loop, and ships the whole thing as one infrastructure-as-code app.

This guide summarizes the architecture, the trade-offs AWS states, and what we would copy even if you do not use AWS. The details are AWS's; where we add judgment, we say so.

TL;DR

Where ordinary RAG breaks

AWS names three failure patterns. Ask something whose answer spans two corpora, and one retrieval returns half of what the model needs, which AWS calls worse than returning nothing because the answer comes back fluent and incomplete. Ask something where the right source depends on the question, and a single index has no way to choose. Ask something that needs a second look after the first result, and there is nowhere for it to happen.

Agentic retrieval puts a reasoning agent in the loop to decide whether to retrieve, which knowledge base to consult, and whether to go back for more. The cost is operational: a single-shot pipeline is a function you can read, while an agent that reasons and retrieves in a loop takes a different path on every question. AWS argues observability stops being optional because it becomes the only way to know the system works. For our own take on why agent visibility matters, see AWS Trust in Depth.

Managed vs customer-managed knowledge bases

The post draws a clear line. In the managed type (Type: MANAGED), Bedrock owns ingestion, storage, indexing and retrieval, including built-in embedding and reranking at no extra cost per AWS, and there is no vector database to provision, scale or patch. In the customer-managed type, you choose and run the vector store yourself.

Customer-managed remains right, AWS says, if you want to reuse an existing OpenSearch Serverless collection or must meet specific data-residency, tuning or compliance requirements. That is the main decision for existing deployments: agentic features are tied to the managed type.

The path of one question

The example uses two corpora, financial and weather, each ingested from Amazon S3 into its own managed knowledge base. At deploy time Bedrock parses, chunks, embeds and indexes them. Then every question follows these steps:

1. The user's question reaches an agent on AgentCore Runtime, which instruments the loop with OpenTelemetry spans.
2. The agent's reasoning model plans and routes. Because each knowledge base is its own tool, it selects the tool whose topic matches the question instead of searching one blended index.
3. AgentCore Gateway brokers the call over MCP, authorized by the agent's IAM role and signed with SigV4, and invokes that knowledge base's AgenticRetrieveStream.
4. AgenticRetrieveStream decomposes the question into sub-queries, retrieves iteratively, and streams back a grounded, cited answer.
5. The agent checks whether the context is sufficient; if not, it loops. Otherwise it returns the cited answer.
6. Spans, token counts and metrics flow to CloudWatch and X-Ray.

The design choice worth copying is routing by tool: one tool per corpus makes the agent's decision explicit and visible in traces. It also keeps access control simple, since each tool can have its own permissions, in line with the least-privilege thinking in our AWS agent registry guide.

Why CDK here

AWS gives a specific reason for CDK. The four stacks are densely cross-referenced: the Gateway needs both knowledge base ids, the agent needs the Gateway URL and ARN, and the dashboards need the knowledge base ids, the Gateway ARN and the agent runtime id, seven values crossing stack boundaries. In raw templates, that is exports and lookups kept in sync by hand plus an ordered deploy script. In CDK you pass the construct, and the export and dependency edge are derived from that one act.

The wiring in app.py is four constructor calls: a knowledge-base stack, a gateway stack that takes the two knowledge bases, an agent-runtime stack that takes the gateway, and a dashboards stack that takes all of them. The stack table in the post lists what each creates, including the knowledge bases reaching ACTIVE and a gateway that, per AWS, does not need Cognito because it uses IAM authorization.

Observability and evaluation

AWS ships two CloudWatch dashboards covering seven layers of telemetry, plus on-demand and continuous evaluation from the same deployment. The post frames faithfulness and correctness scores as a way to catch answers drifting from their sources before production does, and says cited answers trace back to source passages so responses stay auditable.

We have not run the sample, so we cannot confirm dashboard quality or evaluation costs. Treat the sample as a reference architecture, and budget for evaluation calls and telemetry storage, which AWS does not price in the post.

What people are asking

Does this replace the vector database? For the managed type, yes: the post says the OpenSearch Serverless collection and index from AWS's 2024 design are no longer something you create, scale or pay for in OCUs. That is a real simplification, and also a lock-in consideration, since agentic retrieval is not available on customer-managed knowledge bases.

How is this different from adding a reranker to RAG? A reranker improves which chunks come back from one retrieval. Agentic retrieval changes the control flow: the system can choose among sources, retrieve more than once and decide when it has enough. AgenticRetrieveStream also reranks internally, so it does both.

What about citations? The service returns a synthesized answer with citations traced to source passages. That helps audit, though a citation shows where text came from, not that the model used it correctly, which is why the sample pairs it with faithfulness and correctness evaluation.

What does it cost? The post does not give a price, and we did not find one. Plan around three buckets: the managed knowledge base, model tokens for the agent's reasoning loop, and observability and evaluation. Check current pricing on the Amazon Bedrock page before committing.

Can I do this without AWS? The pattern, not the services, is portable: one retrieval tool per corpus, an agent that decides which to call and when to stop, grounded answers with citations, and traces for every step. Any stack with tool calling and tracing can imitate it.

When agentic retrieval is worth it

Our judgment, informed by the post's own framing:

• Use it when answers span corpora, when users ask questions whose best source varies, or when a first retrieval often needs refinement.
• Skip it when most questions are lookups in one corpus. AWS says single-shot is the right design for a large class of questions.
• Expect higher latency and token use per question, since the agent may loop, and more to monitor.
• Watch security: agents that call tools through a gateway should be scoped tightly. Our look at the AgentCorruption hijack of Bedrock AgentCore agents shows what happens when they are not.

How to try it

1. Read the original post and the sample repository linked from it for the exact prerequisites and Region availability.
2. Start with one small corpus per knowledge base and verify ingestion reaches ACTIVE.
3. Deploy the CDK app in a sandbox account and ask questions that need both corpora, then one that needs neither.
4. Inspect traces to see which tool the agent picked and how many loops it ran.
5. Run the evaluation, then change one thing (chunking, prompt, model) and compare scores.

If you are new to the concept, our guide to agent skills and MCP covers the building blocks the Gateway uses.

Related reading on explainx.ai:

• AWS Trust in Depth for autonomous agents
• AWS agent registry, runtime and tool discovery
• AgentCore Payments and x402 agents
• AgentCorruption: one prompt hijacked every AgentCore agent
• What is MCP
• What are agent skills

Based on the AWS Machine Learning blog post of October 26, 2026; service availability, API names and pricing can change.