The first question is never which tool

Firm AI evaluations usually begin with a comparison. Two or three vendors, a feature grid, a demonstration on documents the vendor chose, and a scoring sheet weighted towards capability.

The grid is not useless. It is just several decisions too late. By the time you are comparing features you have already implicitly answered the questions that will actually constrain you, and you have answered them by accepting whatever each vendor happens to do.

There are three, and they run in order.

One: whose tenancy

Where does the data physically come to rest, and under whose control.

The UK National Cyber Security Centre’s cloud security principles frame the same issue as technically enforced separation across compute, storage and data flows. A provider’s general assurance is not a substitute for knowing which boundary protects a particular firm’s data.

The distinction that matters is not “cloud or not”. Almost everything is cloud. The distinction is whether the processing happens inside a tenancy the firm controls — where the firm holds the keys, sets the retention, and can produce an account of what is stored — or inside a vendor’s, where the firm holds a contractual promise instead.

A contractual promise is not nothing. Firms operate on contractual promises constantly. But it is a different class of assurance from control, and the difference becomes concrete at exactly the moments you would least like it to: a client audit, a regulator’s question, a matter where the other side asks what happened to a document.

The question to ask a vendor is not “is our data secure”. Every answer to that is yes. The question is: if we terminate tomorrow, what exists, where, and who can produce a list of it.

Two: which sub-processors

Almost no legal AI product is a single system. It is an application layer over one or more model providers, frequently with a retrieval service, a vector store, an observability platform and an analytics tool alongside.

Each of those is a party that may see content. The firm’s obligation to its clients does not thin out as it passes through that chain — it applies to the whole of it. But the visibility does thin out, and it thins out fast. Two hops down, most firms cannot name who is processing what.

This is answerable. Sub-processor lists exist, and vendors who will not produce one on request have told you something useful. The question is whether anyone reads it before signing rather than after an incident.

Three: whose identity model

This is the one that gets skipped, and it is the one that causes the failure that is hardest to remediate.

A law firm’s document management system encodes something that took years to get right: who may see what. Ethical walls between matters. Client confidentiality boundaries. Restricted matters visible to four named people. Departed lawyers whose access was removed on a specific date. None of that is decoration. Several parts of it exist because of a specific past incident, and some of it exists because a regulator or a client required it.

When a retrieval layer is put over that corpus, it either inherits those permissions or it does not.

If it inherits them, every query is evaluated in the context of the person asking, and the system can only surface what that person could already have opened. If it does not — if it indexes the corpus once, as a privileged service account, and answers everyone from the same index — then the firm has built something that will, eventually and without any malice, summarise a walled matter to somebody on the wrong side of the wall.

It will not look like a breach when it happens. It will look like a helpful answer. That is what makes it hard to detect: there is no access-denied event to audit, because access was never checked. The output is fluent, it is correct, and it is a disclosure.

A retrieval layer that ignores document permissions is not a search tool. It is a confidentiality breach with a text box.

Permissions must be inherited from the identity model the firm already operates. Never rebuilt inside the AI product, because a rebuilt permission model is a second source of truth, and second sources of truth drift. The first time somebody’s access changes in the DMS and not in the AI layer, the firm has an inconsistency nobody is monitoring.

Why the order matters

These three decisions constrain everything after them.

Tenancy determines what deployment options remain available. Sub-processors determine what you can tell a client when they ask, and how quickly. The identity model determines whether retrieval can be turned on across the document estate at all, or only across a curated subset that somebody must maintain by hand for as long as the system runs.

Choose the tool first and these get decided by default — by whatever the vendor built, for reasons that had nothing to do with your obligations. The decisions still get made. They just get made by somebody else, and you find out what they were during the implementation, when changing them is expensive.

There is a version of this that sounds like caution and is really the opposite. Answering these three questions first is what makes it possible to move quickly afterwards, because the remaining choices are genuinely reversible. A tool can be swapped. A retention posture negotiated at signature cannot be retrofitted, and a permission model that was never wired to identity cannot be corrected without reindexing everything and re-answering every question about what was exposed in the meantime.

Cheap decisions first looks like speed. It is the most reliable way to arrive at a system nobody will sign off.