Permission-Aware Retrieval: What Enterprise Security Teams Should Actually Ask Before Trusting a RAG Vendor

pratibha00
Jul 20
19 min read

Every enterprise security review eventually reaches the same question: "How do you make sure User A can't retrieve User B's data?" And in almost every RAG vendor's response, the answer is some version of "we support RBAC," delivered with total confidence and almost no detail behind it.

That answer isn't usually a lie. It's usually a gap — between what a vendor's platform is designed to do in principle and what's actually enforced, line by line, at the moment a query runs. Most security questionnaires don't ask a vendor to prove it. They ask a vendor to check a box, and most vendors do, because the alternative is admitting they're not entirely sure themselves.

That gap matters more with RAG systems than with almost any other type of enterprise software. A traditional application queries a database that already knows who's asking and what they're allowed to see. A RAG system's retrieval layer, left unmanaged, doesn't work that way at all — it searches for whatever content is semantically closest to the query, with no inherent concept of identity or authorization. Permission enforcement has to be deliberately built into that path. If it isn't, the failure mode isn't a slow response or a wrong answer. It's a real user seeing content — a competitor's contract terms, another department's financials, another client's case file — that they were never supposed to have access to. That's not a bug report. That's an incident.

This post is written for exactly the moment this question gets asked — whether you're the security or procurement stakeholder asking it of a vendor, or the technical buyer trying to get your own security team comfortable signing off on a deal. We'll walk through why this problem is structurally different in RAG systems, what real enforcement looks like at the architecture level, and — because claims deserve proof — a genuine look under the hood at how this gets implemented at the database layer, not just described in a slide.

By the end, you'll have a clear framework for one thing: how to tell the difference between a vendor who has actually built this, and one who's just telling you they have.

Why This One Question Decides Enterprise Deals

In enough enterprise sales cycles, there's a predictable moment: the deal has cleared product evaluation, pricing is agreed in principle, and everyone is moving toward a signature — and then it lands on a security architect's desk for review. That's where a surprising number of otherwise-won deals quietly stall.

The reason is rarely that the product doesn't work. It's that the vendor can't answer, with real specificity, how permissions are enforced at the data layer — and for a RAG system handling sensitive documents, that single unanswered question is often enough to pause or kill the deal outright.

This isn't a minor technical objection — it's often the objection. For companies in legal, financial services, healthcare, or any regulated industry, "can this system leak data across users or tenants" isn't a nice-to-have follow-up question. It's frequently the first substantive question a security team asks, because the consequences of getting it wrong aren't hypothetical: a compliance violation, a client trust breach, or in the worst case, a reportable data exposure incident. Security teams have seen enough vague answers to develop a low tolerance for them.

Vague answers create doubt that spreads. When a vendor's response to "how do you enforce row-level permissions in retrieval" is a slide with the word "RBAC" and no further detail, it doesn't just fail to answer the question — it raises a second, worse question in the reviewer's mind: what else in this vendor's security posture is similarly unexamined? That doubt tends to generalize, slowing down or reopening scrutiny on parts of the deal that had already been settled.

The technical buyer becomes the one under pressure. Often, the person pushing for the deal internally — a product lead, a head of engineering, an innovation champion — is the one who has to go back to their own security team and either produce real answers or admit they don't have them. Giving that person something concrete to bring back internally is, in a very direct sense, a sales enablement function, even though it looks like a technical document.

For all these reasons, permission-aware retrieval isn't a "nice engineering detail" buried in an architecture doc. It's frequently the single technical capability standing between a completed evaluation and a signed contract — which is exactly why it's worth being able to demonstrate, not just describe.

The Hidden Risk in How RAG Actually Retrieves Data

To understand why this problem is specific to RAG systems — and not just a generic "access control" checkbox — it helps to understand what's actually happening when a retrieval query runs.

Traditional applications retrieve by identity first. A conventional enterprise application — a CRM, a document management system, an internal wiki — is typically built around the assumption that every query happens in the context of a known user. The database schema, the application logic, or both, are designed to check "who is asking" before deciding "what can they see." Permissions aren't bolted on; they're structurally embedded in how the system was built from the start.

RAG retrieval works differently by default. A RAG system's core retrieval mechanism — vector similarity search — doesn't ask "who is this user allowed to see content from." It asks "what content is semantically closest to this query." That's the entire job it's designed to do, and it does it well. But it means that, left unmanaged, a retrieval system has no inherent concept of authorization at all. It will happily return the most relevant chunk of text regardless of whether the person asking has any right to see it.

This creates a specific and dangerous failure mode. Imagine a legal platform where Client A's contracts and Client B's contracts are both indexed in the same vector store — a common and reasonable architecture choice for efficiency. If permission enforcement isn't built directly into the retrieval path, a query from a user at Client A could surface a highly relevant, semantically similar clause from Client B's contract, simply because it's the closest match in vector space. Nothing about that failure looks like a bug. The system did exactly what it's designed to do — find the most relevant content — it just did so without checking whether the requester was allowed to see it.

This is why "we have RBAC" is an incomplete answer. Role-based access control typically governs what a user can do inside an application's interface — which buttons they see, which pages they can navigate to. It doesn't automatically extend into what content a retrieval engine is permitted to surface from an underlying vector index, unless that enforcement has been specifically engineered into the retrieval query itself. A vendor can have a fully functional RBAC system at the application layer and still have a completely open, unenforced retrieval layer underneath it. Both things can be true at once, and from the outside, they look identical.

This is the structural reason why permission-aware retrieval needs to be evaluated as its own capability — not assumed as a side effect of general access control — and why the next section walks through the specific ways vendors tend to (incompletely) address it.

Three Ways Vendors Commonly — and Wrongly — Claim to Handle This

When a security team pushes past "we support RBAC" and asks for specifics, the answers tend to fall into one of a few recognizable patterns. Each one sounds reasonable on the surface. Each one has a real gap that only shows up under the right test — usually after the deal has closed, not before.

1. Post-Retrieval Filtering

The claim: "We filter results based on user permissions before returning them."

The gap: The critical question is when that filtering happens. In many implementations, the retrieval engine first finds the most semantically relevant chunks across the entire index — including content the user has no access to — and only afterward filters out what that specific user shouldn't see. The problem is that by that point, the content has often already been passed into the language model's context window to generate a response. Filtering the output doesn't undo the fact that unauthorized content was already processed.

Depending on the implementation, it may also mean the system silently returns fewer or lower-quality results for restricted users, without anyone tracking that the underlying retrieval was never actually permission-aware.

2. Application-Layer-Only Enforcement

The claim: "Our application enforces who can access what."

The gap: This is often true — and often irrelevant to the actual risk. Application-layer checks typically govern what a user sees rendered in the product's interface. They don't necessarily govern what happens if the retrieval layer or vector database is queried directly — by an internal tool, an API integration, a debugging script, or a future feature the security team wasn't in the room for. If permissions live only in the application layer and not in the data layer itself, the system has exactly one enforcement point, and every path that bypasses the application — intentionally or not — bypasses security with it.

3. Metadata Tagging Without Enforcement

The claim: "Every document is tagged with owner and permission metadata."

The gap: Tagging data with permission metadata is a necessary step, but it's not the same as enforcing it. It's entirely possible — and more common than it should be — for a system to carefully tag every document with the correct owner, role, or tenant information, and then never actually reference that metadata in the retrieval query itself. The tags exist. They're accurate. They're just not doing anything. This is often the hardest of the three gaps to catch in a demo, because the metadata genuinely is there — it just isn't connected to anything that checks it.

What these three patterns have in common is that each one can be described honestly and still fail to prevent the exact scenario a security review exists to catch. None of them are lies. They're incomplete implementations that sound complete when summarized in a sentence — which is exactly why they need to be examined at the level of "show me the query," not just "describe the policy."

The next section covers what a genuinely complete implementation looks like — enforcement built directly into the retrieval layer, before content is ever surfaced or passed to a model.

What Real Enforcement Looks Like: An Architecture Overview

If the previous section covered what doesn't work, this one covers what does — not as a specific product's implementation, but as the architectural principle that any genuinely secure RAG system needs to follow: permission checks belong in the retrieval query itself, not around it.

Concretely, that means enforcement needs to show up at three distinct points in the pipeline, not just one.

At ingestion: every piece of content is tagged with its access boundaries, not just its owner. When a document enters the system, it needs to carry structured metadata describing exactly who — or which roles, groups, or tenants — are authorized to access it. This sounds similar to the "metadata tagging" pattern described in Section 4, and it is similar — the difference is what happens to that metadata next.

At indexing: permission metadata is stored as queryable data, not descriptive labels. The access information needs to live in a form the retrieval engine can actually filter against at query time — not as a separate reference table that has to be checked afterward, but as part of the same data structure the similarity search itself runs against.

At query time: permission filtering happens before or during similarity search, not after. This is the architectural detail that actually closes the gap. Instead of asking "what's the most relevant content in the entire index" and filtering afterward, the query itself is scoped from the start: "what's the most relevant content among only what this user is authorized to see." The unauthorized content is never retrieved, never scored, and never passed anywhere near the language model's context window — because from the system's perspective, it was never part of the searchable set for that particular request.

Why this ordering matters more than it might seem. The difference between "filter after retrieval" and "filter as part of retrieval" can sound like a minor implementation detail. It isn't. It's the difference between a system where unauthorized content briefly exists inside the process before being discarded, and a system where unauthorized content is structurally excluded from ever being considered. The first approach depends on every downstream step remembering to enforce the filter correctly, every time, in every code path. The second approach makes the enforcement a property of the query itself — which is a much harder thing to accidentally get wrong or bypass.

This is also why the underlying database matters more in RAG security discussions than it does in most software evaluations. A system built on a database with native, enforceable row-level security — where a security policy can be defined once, at the data layer, and applied automatically to every query regardless of which application or process is asking — has a structurally stronger foundation than one relying entirely on application code to remember to apply the right filter every time.

That's not a small distinction to a security reviewer. It's usually the exact distinction they're trying to uncover when they ask the question this entire post opened with

The next section moves from architecture to evidence — a real look at what this enforcement actually looks like implemented at the database layer, using PostgreSQL and pgvector.

Proof, Not Promises: A Look Under the Hood

Everything up to this point has been architecture and principle. This section exists because principles are easy to claim and hard to verify — so here's what permission-aware retrieval actually looks like at the database layer, using PostgreSQL with the pgvector extension.

This isn't meant as a full implementation guide. It's meant as evidence: a concrete example of the difference between "we filter based on permissions" as a sentence in a security questionnaire, and an actual, enforceable database policy that does the work automatically, on every query, regardless of which application or process is asking.

The foundation: permissions live in the database, not just the application.

PostgreSQL supports row-level security (RLS) natively — policies that restrict which rows a query can return, enforced by the database itself, not by application code that has to remember to apply a filter correctly every time. Applied to a table storing document embeddings, this means access control becomes a property of the data, not a responsibility scattered across every service that happens to query it.

A simplified version of the underlying table might look like this:

CREATE TABLE document_chunks (
    id SERIAL PRIMARY KEY,
    content TEXT,
    embedding VECTOR(1536),
    tenant_id UUID NOT NULL,
    allowed_roles TEXT[] NOT NULL,
    document_owner_id UUID NOT NULL
);

Each chunk carries not just its content and embedding, but the tenant it belongs to and the roles authorized to access it.

The enforcement: a row-level security policy applied once, at the table.

ALTER TABLE document_chunks ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation_policy
ON document_chunks
FOR SELECT
USING (
    tenant_id = current_setting('app.current_tenant_id')::UUID
    AND current_setting('app.current_user_role') = ANY(allowed_roles)
);

Once this policy is active, it applies automatically to every query against this table — including the similarity search itself. There's no separate filtering step to remember, and no code path that can accidentally bypass it, because the restriction is enforced by the database engine, not by application logic sitting in front of it.

The retrieval query: permission-aware from the start, not filtered afterward.

SET app.current_tenant_id = 'tenant-uuid-here';
SET app.current_user_role = 'associate';

SELECT content, 1 - (embedding <=> query_embedding) AS similarity
FROM document_chunks
ORDER BY embedding <=> query_embedding
LIMIT 10;

Note what's absent here: there's no WHERE tenant_id = ... or WHERE role = ... clause written into this query at all. That's the point. The similarity search runs exactly as written, and the row-level security policy transparently restricts the rows it's even allowed to consider — before ranking, before scoring, before anything reaches the application layer. Unauthorized content isn't retrieved and then discarded. It's never part of the searchable set for this request in the first place.

Why this matters for a security review specifically. This is the kind of implementation a technical buyer can put in front of their own engineering or security team and get a real answer back — not "that sounds reasonable," but "yes, that's how row-level security actually works, and yes, that would hold up." It's verifiable, inspectable, and doesn't rely on trusting that every application developer, every internal script, and every future feature remembers to apply the same filter correctly, forever.

That last point matters more than it might seem — which is exactly what the next section addresses: what happens when permission models get more complicated than a single tenant and a single role.

Handling the Complexity Enterprises Actually Have

The example above was deliberately simple — one tenant, one role, one straightforward policy — because the goal was to show the mechanism clearly. Real enterprise environments are rarely that clean, and a permission model that only works in the simple case isn't one a security team can actually rely on. This section covers the complexity that shows up in practice, and how the same underlying approach extends to handle it.

Group and role hierarchies. Enterprise organizations rarely grant access purely at the individual level. A document might be accessible to "anyone in the Contracts team," "anyone at Manager level or above," or some combination of both. The row-level security policy shown earlier can be extended to check group membership and role hierarchy rather than a flat list of allowed roles — for example, resolving a user's effective roles (including inherited ones from group membership) before the policy evaluates access, rather than requiring every document to explicitly list every role that might ever need access to it.

Inherited permissions. In many document systems, access isn't just set at the individual file level — it cascades from a folder, a project, or a client matter. A new document added to an existing project should typically inherit that project's access rules automatically, without requiring someone to manually re-tag it. This is generally handled by resolving permissions through a hierarchy at ingestion time — checking not just a document's own metadata, but the access rules of its parent container — so the row-level security policy at query time still only has to check a single, already-resolved permission field, rather than recalculating a complex hierarchy on every search.

Multi-tenant isolation. For platforms serving multiple enterprise clients from shared infrastructure — a common and reasonable architecture for cost efficiency — tenant isolation has to be absolute, not just role-based. This typically means the row-level security policy checks tenant identity as a hard boundary before it ever evaluates role or group logic, ensuring that even a misconfigured role permission can't accidentally leak data across tenant boundaries. Tenant isolation and role-based access are related but separate checks, and a robust policy treats them that way rather than collapsing them into one condition.

Mid-session role or access changes. Enterprise access isn't static — employees change roles, contractors get offboarded, permissions get revoked. A genuinely secure system needs to reflect that immediately, not on the next login or the next cache refresh. Because the policy in Section 6 checks the user's current role and tenant context at query time, rather than relying on a permission decision cached at login, a revoked role takes effect on the very next query — not after a session expires or a token refreshes. This is a meaningfully different guarantee than systems where permissions are checked once and trusted for the duration of a session.

Why this matters to a security reviewer specifically. Any vendor can demonstrate correct behavior in the simple case. The harder — and more revealing — question is what happens when a user's role changes mid-session, or when a document lives three folders deep with permissions inherited from two different levels, or when two tenants' data sits in the same physical table. Asking a vendor to walk through one of these specific scenarios, rather than accepting a general "yes, we handle that," is usually the fastest way to find out whether a permission model was built for real enterprise complexity or just for a demo environment.

Audit Trails That Actually Satisfy an Auditor

"Full audit trails" is one of the most commonly listed — and most commonly under-specified — claims in a RAG vendor's security documentation. Like RBAC, it's a phrase that sounds complete and often isn't, because logging that something happened is a very different thing from logging enough detail to actually answer an auditor's questions after the fact.

What a real SOC 2 audit actually asks for. When an auditor reviews access controls, they're generally not satisfied by "we log access." They want to be able to answer, for any given piece of data, at any point in time: who accessed it, when, under what permission grant, and why the system believed that access was authorized. That last part — the why — is the piece most logging implementations skip, because it requires the system to record not just the event, but the permission state that justified it.

What this looks like at the database layer. Continuing with the pervious pgvector example from above, an audit-ready logging approach captures the permission context at the moment of retrieval, not just the fact that a query occurred:

CREATE TABLE retrieval_audit_log (
    id SERIAL PRIMARY KEY,
    user_id UUID NOT NULL,
    tenant_id UUID NOT NULL,
    user_role TEXT NOT NULL,
    query_text TEXT,
    retrieved_chunk_ids INTEGER[],
    permission_policy_version TEXT,
    "timestamp" TIMESTAMPTZ DEFAULT now()
);

The critical detail here is permission_policy_version alongside user_role and tenant_id at the time of the query — not just which chunks were returned. This means that months later, if a permission policy has since changed, the log still accurately reflects what access rules were in effect at the moment that specific retrieval happened. Without this, an audit trail can tell you what was retrieved but not whether that retrieval was actually authorized under the rules in place at the time — which is frequently the exact question an auditor is asking.

Logging needs to happen at the same layer as enforcement. If permission checks are enforced at the database layer via row-level security, the audit log should be populated as close to that same layer as possible — ideally via a trigger or logging mechanism tied directly to the query execution, rather than logged separately by application code that has to remember to record it correctly every time. This mirrors the same principle above: enforcement and logging are both more trustworthy when they're structural properties of the data layer, not responsibilities scattered across application code paths.

Why this matters beyond passing an audit. A complete audit trail isn't just a compliance checkbox — it's what allows a security team to actually investigate a suspected incident after the fact. If a client ever asks "did anyone outside our organization see our data," a system with genuine, permission-aware logging can answer that question definitively. A system with only surface-level access logs can only shrug and say "probably not."

That distinction — between a system that can prove what happened and one that can only claim it — is the difference an enterprise security team is ultimately trying to find in every question this post has walked through so far.

Does This Slow Things Down?

It's a fair question, and one enterprise buyers ask often enough that it deserves a direct answer rather than a dismissive one: does adding permission enforcement at the retrieval layer make the system slower?

The honest answer is yes, slightly — and it's worth understanding exactly where that cost comes from, why it's usually smaller than expected, and why the alternative is a much worse tradeoff for anything handling sensitive enterprise data.

Where the overhead actually comes from. A row-level security policy adds a filtering condition that the database has to evaluate as part of query execution. In the pgvector example from above, that means checking tenant and role conditions alongside the vector similarity search itself, rather than running a completely unfiltered search. This is real, measurable overhead — but it's overhead applied to a well-indexed, structured filtering condition, not an expensive or open-ended computation.

Why the cost is usually small in practice. With proper indexing — specifically, indexing the columns used in the security policy (tenant_id, role-related fields) alongside the vector index itself — the database can narrow the searchable set efficiently before or during the similarity search, rather than filtering a full result set afterward. This is a well-understood database optimization problem, not a novel one; it's the same category of performance work that's gone into row-level security in PostgreSQL for years, applied to a newer type of workload.

The comparison that actually matters. The relevant question isn't "does permission filtering add latency" — some amount, almost inevitably, is close to unavoidable. It's "what does the unfiltered alternative cost instead." An unenforced or post-retrieval-filtered system might return results a few milliseconds faster, but it carries the much larger, much harder to quantify cost of a potential data exposure incident, a failed security review, or a client relationship damaged by a permissions failure. For most enterprise buyers, particularly in regulated industries, that tradeoff isn't a close call.

What to actually ask a vendor. Rather than accepting either extreme — "there's no performance cost" (unlikely to be fully true) or being scared off by "there's some cost" (true but incomplete) — the more useful question for a technical buyer to ask is how a vendor has indexed and optimized their specific permission model at scale, and whether they can share real latency numbers under realistic document volumes and concurrent users. A vendor who has actually done this work will have real numbers to share. A vendor who hasn't will usually answer the question in the abstract.

Performance and security aren't inherently in tension here — they just both require the same thing this whole post has been arguing for: enforcement built deliberately into the architecture from the start, rather than bolted on and hoped to be fast enough later.

How to Verify a Vendor Isn't Just Saying This

Everything covered so far points toward one practical outcome: a way to actually test a vendor's permission-aware retrieval claims, rather than accepting them at face value. This checklist is meant to be vendor-neutral — a set of questions any security or procurement team can bring to any RAG vendor, regardless of which one they're evaluating.

Ask where enforcement actually happens. Don't ask "do you support RBAC" — ask "at what point in a retrieval query is permission checked: before similarity search, during it, or after?" A vendor who can answer this precisely, and explain why, is describing a real implementation. A vendor who answers with a general description of their access control philosophy is likely describing an aspiration.

Ask what happens if the retrieval layer is queried directly. A meaningful follow-up question: "If someone bypassed your application and queried the underlying vector store or database directly, would permissions still be enforced?" This distinguishes application-layer-only enforcement from true data-layer enforcement, and it's a question many vendors haven't actually had to answer before.

Ask for a specific, non-trivial scenario — not a general confirmation. Rather than "do you handle group permissions," ask the vendor to walk through a concrete case: a document inheriting permissions from a parent folder, a user whose role changes mid-session, or two tenants sharing infrastructure. A vendor with a real implementation can walk through the mechanics. A vendor without one tends to answer in generalities or pivot to a different topic.

Ask what the audit log actually captures. Request a sample (redacted, if necessary) of what an audit log entry actually contains. The presence of user_id and timestamp alone is a weak signal. The presence of the permission context that justified access at that moment — the specific detail covered in audit trail section above — is a much stronger one.

Ask for real performance numbers, not reassurance. A vendor who has genuinely built and optimized permission enforcement at the database layer will typically be able to share latency figures under realistic load. A vague "it doesn't really slow things down" without supporting numbers is worth following up on.

Ask how this has been tested. A reasonable final question: "How do you verify this actually works — do you have automated tests specifically for permission boundaries, or is this validated manually?" Automated, repeatable testing for access control boundaries is a meaningfully stronger signal than manual review, particularly as a system's permission model grows more complex over time.

None of these questions require deep technical expertise to ask. They require knowing that "we support RBAC" is the beginning of a conversation, not the end of one — and that the difference between a vendor who has actually built this and one who hasn't usually becomes clear within the first follow-up question.

Where This Leaves You

If you're a security or procurement stakeholder evaluating a RAG vendor, the checklist in the previous section is yours to use — with us or with anyone else you're evaluating. That's a deliberate choice. A vendor confident in their own implementation should welcome those questions, not deflect them.

If you're a technical buyer trying to get your own security team comfortable with a RAG deployment — whether that's ours or one you're building internally — the architecture and code in this post are meant to be a real reference, not a simplified summary. Permission-aware retrieval isn't an exotic capability. It's a well-understood database engineering problem, solved with tools like PostgreSQL row-level security that have existed for years — applied thoughtfully to a newer kind of workload.

And if you're currently in the middle of a security review that's stalled on exactly this question, that's precisely the situation this post was written for.

We work with teams to design and implement permission-aware RAG architectures — including the enforcement, audit logging, and multi-tenant isolation patterns covered here — built to hold up under a real security review, not just describe well in one. If that's a gap you're currently navigating, let's talk through what that would look like for your specific environment.