Point a vector index at a law firm's document management system and you will get a working demo in an afternoon and an ethics problem by the end of the quarter.
flowchart TD
A["User query"] --> B["Identity, matter context, screens"]
B --> C["Access filter applied at query time"]
C --> D["Hybrid retrieval: semantic plus keyword"]
D --> E["Defined term and cross reference resolution"]
E --> F["Version and authority check"]
F --> G["Generation with mandatory source citation"]
G --> H{"Risk tier of the ask"}
H -->|"High"| I["Lawyer review before anything leaves"]
H -->|"Low"| J["Released with citations attached"]
I --> K["Interaction trace captured"]
J --> K
K --> L["Drift and retrieval quality monitoring"]
L --> M["Knowledge base repair"]
M --> D
classDef input fill:rgba(180,83,9,0.08),stroke:#b45309,stroke-width:2px,color:#0a1628
classDef closed fill:rgba(225,29,72,0.08),stroke:#e11d48,stroke-width:2px,color:#0a1628
classDef agent fill:rgba(74,144,196,0.08),stroke:#4a90c4,stroke-width:2px,color:#0a1628
classDef flow fill:rgba(232,98,26,0.08),stroke:#e8621a,stroke-width:2px,color:#0a1628
classDef gate fill:rgba(180,83,9,0.08),stroke:#b45309,stroke-width:2px,color:#0a1628
classDef output fill:rgba(15,118,110,0.06),stroke:#0f766e,stroke-width:2px,color:#0a1628
classDef research fill:rgba(124,58,237,0.08),stroke:#7c3aed,stroke-width:2px,color:#0a1628
class A input
class B,C closed
class D,E,F agent
class G flow
class H gate
class I closed
class J output
class K,L research
class M flow
Law firms run ethical screens. A partner conflicted off a matter must not see that matter's documents, and the obligation is professional, not preferential. Corporate legal departments carry the same shape of problem with privilege and with regulated data.
Embed the whole corpus into one index and you have created a system that retrieves across every wall in the building. Filter the results after retrieval and you have still ranked, scored, and reasoned over material the user was never allowed to see. Worse, the model may summarize what it cannot show.
Access control belongs in the query, not in the cleanup. Chunks carry their source document's permissions, and the filter is applied before ranking. This is unglamorous, it slows retrieval, and it is the difference between a tool a firm can deploy and one it cannot.
Split a credit agreement into 500-token windows and something specific goes wrong. Section 1 defines Permitted Indebtedness across two pages. Section 47 says a covenant applies to Permitted Indebtedness. Retrieve Section 47 alone and the chunk is grammatically complete, semantically confident, and meaningless.
The same failure runs through incorporation by reference, exhibits and schedules, defined terms that get amended in a side letter, and provisions that say "notwithstanding Section 12." A retrieval system that does not resolve those references will hand a lawyer a clause that means close to the opposite of what it appears to say.
A document management system holds the draft, three redlines, the version somebody saved to their desktop and re-uploaded, and the executed copy. They are all similar enough to retrieve together and different enough to matter.
This is a metadata problem, and firm metadata hygiene is usually poor because for thirty years nothing depended on it. Now something does. Authority ranking has to be explicit: what is executed, what is current, what is superseded, and what should have been archived two years ago.
| Symptom the user reports | What people blame | What it usually is |
|---|---|---|
| "It made something up" | Hallucination, model choice | Retrieval returned nothing useful and the model filled the gap |
| "It gave me the wrong clause" | Poor reasoning | Chunking severed a defined term or a cross reference |
| "That contract was renegotiated" | Stale index | The superseded version was never archived |
| "It contradicted itself" | Model inconsistency | Two conflicting documents both live in the corpus and both got retrieved |
| "It surfaced something I should not see" | Prompt injection | Permissions were applied after retrieval instead of during |
Four of those five are content and architecture problems wearing a model costume. An audit of one production knowledge base surfaced fifteen distinct issues in a single day: conflicting guidance, superseded documents still live, and gaps the agent had been confidently filling in from nothing.