Permission-aware retrieval: the feature every internal AI search needs and most skip

Most internal "ask your documents" tools are built the same way: crawl everything, embed everything, search everything, answer. It works on a test folder. It fails the first time someone in sales asks about salaries and the agent helpfully summarises the HR spreadsheet.

We think about this as a simple rule: an agent may only retrieve what the person asking could already open themselves. Everything else in the Knowledge Search Agent follows from that rule.

Permissions are content, not configuration

The first design decision is that access rights are synced with the content, from the source system, on every sync. A Google Drive file carries the users and groups that can view it. A SharePoint library carries its tenant permissions. A Slack channel carries its membership. We store those alongside the chunks, and we keep them fresh — a permission change in the source shows up in the index within the same sync window as a content change.

The alternative — administrators re-declaring permissions inside the AI tool — always drifts. Somebody gets added to a folder and nobody updates the second system.

Filter before ranking, not after

The second decision is where filtering happens. Filtering the top results after ranking is tempting and wrong: if the ten best passages are all in documents the user can't see, you end up with nothing, or worse, with a leaky answer built from titles and snippets.

We apply the permission filter inside the retrieval query itself. The ranker only ever sees candidates the user is allowed to read. This also means the "I don't know" behaviour is honest: if the answer exists only in documents you can't open, the agent tells you it can't find it and suggests who to ask.

Groups, nested groups and the "everyone" problem

Real permission systems are messy. Google Groups nest. Microsoft 365 has security groups, Microsoft 365 groups and SharePoint groups. Confluence has space permissions layered on page restrictions.

We resolve group membership at sync time into a flat set of principals per document, and we resolve the asking user's own principals at query time. Both sides are cached with short TTLs. The special case that bites everyone is "anyone in the organisation" links — those are common in Drive and SharePoint and they are accessible to every employee, so we treat them as such, but we let administrators exclude specific sources from company-wide scope if a link-sharing policy is looser than the company wants.

Agents inherit the same rule

Enterprise Agents read from the same knowledge layer, so the question becomes: whose permissions does an agent use? Our answer: an agent has its own identity with explicitly granted knowledge bases and tools. An agent that processes invoices reads the finance knowledge base and nothing else. When an agent acts on behalf of a person — for example answering their question in Slack — the intersection of the agent's grants and the person's permissions applies.

What this costs

Permission-aware retrieval adds work: more metadata per chunk, group resolution, and a filter stage in every query. In our measurements it adds tens of milliseconds to a typical query. Compared with the alternative, that is free.

If you are evaluating any internal AI search, ask one question: what happens when I ask about something I'm not allowed to see? The right answer is "nothing".