AESOP — Enterprise AI Agent Governance
Context Architecture
What AESOP knows is limited by what it was given to read. Here is what it collects, where the rest comes from, and what actually changes because of it.
01
The contextfour layers
Context arrives in four layers. You only fully control the first one.
Layer 01The agent
What you are testing The system instructions, the agent's name, and an endpoint if you want it exercised live.
Layer 02The policy layer
What it gets measured against Governance documents you pick from the shared library or upload on the spot. Pick none and your organization's own policy is used instead.
Layer 03The run frame
What this run is for Notes on the agent's real situation, the failure modes you already suspect, and the rubric, either generated or supplied.
Layer 04The audience
Who the agent acts on Who they are, what state they are in, how vulnerable they are, and whether they deal with it once or for months.
The six reviewers never see this layer. Only the human-impact scorers do, so a sympathetic audience cannot soften how the instructions read.
02
Where the rest comes fromthree ways in
You type some of it. The platform inherits some from your organization and your past runs. The rest it observes from production.
Sources
Typed, inherited, observed
→
One record
Resolved once, per run
→
Own slice
Each agent gets only what it needs
→
Report
Scored, cited, human gate
You type it
Entered for one run: the instructions, your notes, the audience fields, a rubric, any document you upload.
Your
organization
organization
The shared document library, so one person's governance packet serves the whole team. Plus eighteen slots describing the business itself: strategy, structure, brand, blockers, exposure posture. Governance falls back here when you pick nothing.
Your
past runs
past runs
Last run's context, offered back pre-filled. Reports red-team can aim at, failures it can target, and a rolling five-run history that labels each finding a regression, a repeat, or new.
Production
Real conversations, pasted or imported. JSONL, Q/A, Human/Assistant, or a format the parser works out. Duplicates are dropped.
Meetings
Transcripts from Teams, Zoom, Granola, or Otter. The platform pulls out risks, decisions, and changes, each carrying the quote it came from. Nothing counts until a person approves it.
The agent
record
record
The endpoint and its credentials are read from the agent, not typed into the run. A password entered once does not become the one that gets used.
03
What each mode asks forseven evaluation types
The mode you pick decides what the platform goes looking for. A mode that lacks its context is offered as unavailable, with the reason on the card, rather than running weak.
Review the
written instructionsNeeds instructions
written instructionsNeeds instructions
Quality Analysis. Six reviewers score the instructions, then synthesis weights them into one certification.
Reviewers Instructions → Ethos, Bias, Safety, Legal, InfoSec (parallel) → Synthesis
Test it against
a rubricNeeds instructions
a rubricNeeds instructions
Rubric testing. Builds a rubric from your instructions, or uses the one you supplied, then writes scenarios from it.
Pipeline Rubric → Test scenarios → Execute
Red-team
itNeeds endpointNeeds instructions or a prior report
itNeeds endpointNeeds instructions or a prior report
Adversarial Red-Team. Attacks built from what is already known, then run against the live agent.
Pipeline Briefing → Scenarios → Off-audience scenarios → Persona review → Execute
Discover what
it doesNeeds endpointNo instructions needed
it doesNeeds endpointNo instructions needed
Black Box Investigation. Interviews an agent you know nothing about, then reconstructs its instructions from the answers. This is how an undocumented agent gets evaluated at all.
Pipeline Interview → Reconstructed instructions
Check for
differential treatmentNeeds instructions
differential treatmentNeeds instructions
Bias Probe. Holds a scenario fixed, changes one attribute at a time, and reads whether the answers move.
Pipeline Attribute pairs → Paired runs → Divergence scoring
Check how
it soundsNeeds endpoint
it soundsNeeds endpoint
Sounds Like AI. Asks ordinary questions and reads how the answers are phrased. Instructions sharpen the questions but are not required.
Pipeline Prompt set → Live answers → Voice scoring
Measure its effect
on the personNeeds endpointNeeds the audience fieldsNeeds a scenario phase first
on the personNeeds endpointNeeds the audience fieldsNeeds a scenario phase first
Human Impact. Reads the conversations the agent actually had and scores what they did to the person in them.
A real dependency, not a preference. It reads transcripts nobody else produced, so a scenario phase has to run first. With nothing to read it reports nothing, and nothing looks exactly like a clean result.
04
What the context changesinput → effect
Collecting context only matters if it moves the answer. Here is what each piece actually does.
System
instructions
instructions
The only input that sets how confident the review is allowed to sound. Length alone decides it. Under 500 characters and the report says so.
Governance
documents
documents
Appended to the instructions, so every reviewer reads them. Legal and InfoSec are told to check the instructions against them specifically. They also make that text longer, which raises the confidence band.
Vital
context
context
Two different jobs. The six reviewers use it only to adjust confidence, it never reaches their prompt. Red-team uses it as the boundary every attack has to fall outside.
The
audience
audience
Human Impact will not run without it. The scorers read the conversations, not the description, so the score is what the agent did rather than what the reader expected.
Prior
runs
runs
Red-team stops rediscovering the agent. The second evaluation is cheaper and sharper than the first, because it aims at what is already known instead of starting over.
Trace
taxonomy
taxonomy
People code real traces by hand, the codes get consolidated, and a person approves the taxonomy. Only then is a judge built for each one. The result is traceability: every repair cites a named code with its evidence attached, so you can walk a fix back to the conversation that justified it.
The honest limit. Context raises confidence. It does not measure correctness. Instructions and governance both move the confidence band by length, which tells you how much the platform had to read, not how right it is. The judgments that rest on evidence come from traces and prior runs. Everything else is framing.