Stop your RAG app from leaking data across tenants

Picture this. The CEO wants a unified company AI that can search everything. But Legal points out that their unannounced M&A documents absolutely cannot sit in the same RAG pipeline as the summer intern wikis. The tension here is immediate. You have infrastructure cost on one side and security risk on the other.

You really do not want to spin up, manage, and synchronize twelve different vector databases just to keep HR data separate from Engineering. But cramming all your sensitive company data together into a single, shared index feels like an inevitable GenAI data leak.

The smart way to stop that leak isn't trying to catch it at the LLM level. It is dynamically translating user permissions into vector database pre-filters exactly at retrieval time. Today we are looking at an AWS reference architecture that relies on a middleware Lambda mechanism to force document-level isolation.

Enforcing Boundaries at the Point of Query

How do you keep HR data separate from Engineering data without managing a dozen different vector databases? You use a single shared Amazon Bedrock Knowledge Base. But you enforce strict boundaries by tagging every single file with department metadata before it gets indexed.

The actual security enforcement happens at the point of query. The core mechanism is a middleware Lambda function. This function sits directly between your user and the vector search. When a request comes in, this Lambda intercepts it. It does not just forward the text string to the database. Instead, it pauses and makes a call to an external authorization engine.

In this architecture, that engine is Amazon Verified Permissions. The middleware Lambda asks the engine a specific question: Based on this user's group, which department documents are they permitted to query?

From Policy to Live Filter

Verified Permissions evaluates human-readable access rules written in a language called Cedar. If the user is in the Engineering group, the engine returns an allow-list for Engineering resources. The middleware Lambda takes that authorization decision and dynamically constructs a hard metadata filter.

It builds a JSON object specifying that the department key must equal engineering. It bundles that exact filter into the Amazon Bedrock RetrieveAndGenerate API call.

json{  "filter": {    "equals": {      "key": "department",      "value": "engineering"    }  }}

The vector database applies this metadata filter before the similarity search even runs. The LLM never even sees the HR documents. It cannot hallucinate or leak them because they simply do not exist in the retrieval set. If an Engineering user explicitly asks about an HR salary document, the retrieval set comes back empty. The LLM has no context, so it cannot answer.

Decoupling Access Rules from Application Logic

This completely decouples your access rules from your application logic. How do you update access rules dynamically without having to redeploy the application code? You just update the Cedar policy in the Verified Permissions console. Because the middleware Lambda makes a fresh evaluation on every single request, the change takes effect immediately.

If a department needs temporary access to another team's project, you write a new Cedar policy. You deploy it to the policy store. The middleware Lambda instantly starts building an expanded metadata filter on the very next query. No application code changes. No deployments.

Failing Closed: The 'Deny by Default' Principle

But any time you introduce an external authorization service into a critical path, you have to ask about failure modes. What happens if the authorization service goes down? Does the LLM fail open and leak everything?

The answer is a hard no. The architecture is explicitly designed to deny by default. If the middleware Lambda cannot reach Verified Permissions, or if the request times out, it assumes zero permissions. It generates an empty metadata filter that matches nothing. Or it simply throws a permission error and blocks the query.

The system fails closed. You are making a deliberate availability trade-off. An outage in your authorization tier results in application downtime. It does not result in a catastrophic cross-department data leak.

The Hidden Operational Cost

This all sounds incredibly clean in theory. But rolling this out is a massive commitment. You are not just installing a library. You are adopting a sprawling, event-driven reference architecture that spans over a dozen AWS services.

The Ingestion Pipeline

To guarantee that every document gets its metadata tag before being indexed, you have to build an entirely separate ingestion pipeline. You upload a document to S3. EventBridge fires an event. An SQS queue buffers that event. Finally, a custom Lambda function writes an immutable JSON sidecar file containing the department tag next to the document.

Operational Overhead and Race Conditions

There is a steep operational cost here. You save money by not provisioning separate Bedrock instances, but you pay for it by managing SQS queues, tagging Lambdas, and dead-letter queues. If the tagging Lambda fails and hits the dead-letter queue, the document is excluded from the index entirely. You now need operational runbooks for monitoring that DLQ. You need automated recovery mechanisms to clear the queue and re-trigger the indexing job.

There is also a known race condition in this ingestion design. There is a brief window between the document upload and the sidecar creation. The architecture relies on a default thirty-second SQS batch window to guarantee the tagging completes before the next scheduled Bedrock indexing job runs.

Critical Limitation: Not for Multi-Tenant SaaS

Relying on a thirty-second SQS batch window to prevent security indexing races is brittle. It is an operational headache waiting to happen. Decoupling your access rules from your application code also introduces friction for developers.

How do your engineers test these Cedar authorization policies locally before pushing them live? Because the logic lives in a managed AWS service, testing changes safely requires building a completely separate CI/CD pipeline just for Verified Permissions.

All of this brings us to the most critical limitation of the pattern. Is this logical isolation secure enough for a multi-tenant B2B SaaS product? Absolutely not. The authors of the architecture explicitly warn against this. This pattern provides logical, filter-level isolation. It does not provide physical, IAM-enforced isolation.

If you are building an internal company tool, a bug in the middleware Lambda means the accounting department might accidentally query the marketing team's strategy docs. That is embarrassing. But if you are building a B2B SaaS product, a failure in the middleware means Customer A searches Customer B's proprietary data. That is a catastrophic, company-ending breach.

For multi-tenant SaaS, or for environments with strict regulatory compliance bounds, you cannot rely on metadata pre-filters. You still need a dedicated vector database per tenant. You must use hard IAM resource boundaries to guarantee physical separation. You can use this metadata filtering pattern inside a single tenant to handle role-based access. But you must never use it as a substitute for hard infrastructure isolation.

The Takeaway: Security Happens at Retrieval

What this architecture teaches us is that data protection in a RAG application happens at retrieval. Guardrails and LLM prompt instructions are just output safety layers. Real security requires ensuring the foundation model never touches unauthorized text to begin with.

By intercepting the query, checking an external policy store, and forcing a metadata pre-filter, you get centralized, auditable access control. You get this without duplicating your infrastructure. Just be honest with your team about the complexity you are taking on. You are trading infrastructure cost for event-driven operational overhead.

TAKEYOURPILLS.TECH. Go ship something.