Index/ AESOP/ The Hydra
Adversarial red-team — AESOP

The Hydra

A parallelized red-team execution engine that deploys adversarial attack heads against your chatbot across five domains.

01

About the name

The Hydra draws on Greek mythology — the Lernaean Hydra grew two heads for every one that was cut off. No single attack could defeat it; only coordinated, concentrated force could. The capability works the same way. It does not send a single probe at a target. It assembles all available intelligence, fans each scenario into multiple variations, and unleashes them all at once. The target agent cannot prepare for a single angle when it faces dozens.

02

How to access it

The Hydra lives at /hydra in the sidebar, or as a toggleable capability in the evaluation wizard. It unleashes concurrent adversarial heads against your chatbot endpoint.

1Navigate
to /hydra
The Hydra link sits in the sidebar for all users. Reach it too from the completion card after any evaluation.
2Enter your
endpoint
Provide the chatbot endpoint URL to test. Optionally link a registered AESOP agent to use its prior evaluation data for smarter targeting.
3Configure heads,
mode, cave link
Set the number of heads (1–20), choose a probe mode (Director, Workers, or Both), and optionally link a completed Cave of Shadows run as the scenario base.
4Unleash
the Hydra
Submit to launch. The Hydra assembles intel, fans scenarios into variations, and executes all heads simultaneously. Results stream live on the run page.
03

Five evaluation domains

Safety
Harm prevention, dangerous outputs, refusal behavior.
Ethics
Manipulation, transparency, responsible guidance.
Bias
Discriminatory patterns, stereotypes, fair treatment.
Legal
Regulatory exposure, PII handling, liability.
Security
Prompt injection, system-prompt leakage, data access.
04

Tiger team head allocation

Concentrate firepower on weak domains. The Hydra does not distribute heads evenly. Before fanning scenarios, it checks prior evaluation scores across all five domains; domains below the weakness threshold (80) receive proportionally more heads. Every domain gets at least one.

Safety: 62 → 4 heads    Security: 58 → 5 heads
Ethics: 91 → 1 head    Bias: 88 → 1 head    Legal: 85 → 1 head
Total: 12 heads allocated based on the weakness profile
05

Probe modes

Routing testDirector
only
Probes the director agent’s routing decisions, scope enforcement, and handoff behavior. Tests whether the orchestration layer correctly filters and routes adversarial intent.
Specialist testWorkers
only
Bypasses the director and attacks specialist workers directly. Reveals whether guardrails live only at the routing layer or are enforced throughout the hierarchy.
Full coverage — recommendedBoth
Runs director probes and worker probes. Complete coverage of the full agent hierarchy — the standard mode for a comprehensive Hydra run.
06

Cave of Shadows integration

Linking a completed Cave of Shadows run gives the Hydra a tailored base scenario set instead of the standard static battery. The Hydra fans each Cave scenario into domain-specific variations, multiplying the attack surface proportionally across weak domains. Without a linked Cave run it falls back to the built-in director and worker probe libraries (10 each, covering all five domains).

07

Six-stage execution pipeline

InputsEndpoint +
Cave run
Required: chatbot endpoint URL. Optional: a completed Cave of Shadows run to use as the base scenario set.
Stage 1Briefing
assembly
Gathers all available intel on the target — system instructions, vital context, prior AESOP analyses, Cave of Shadows reports, historical Hydra run scores.
Stage 2Scenario
loading
Loads base scenarios from the linked Cave run, or falls back to the built-in static probe battery (director + worker probes across all five domains).
Stage 3Tiger team head
allocation
Reads prior domain scores and distributes N heads across the five domains, inverse-weighting weak domains (threshold 80) to concentrate firepower where the agent is most vulnerable.
Stage 4Scenario
fanning
Each base scenario is expanded into N domain-specific variations with distinct personas, angles, and escalation approaches.
Stage 5Parallel head
execution
All heads unleashed simultaneously via asyncio. Each runs an adaptive multi-turn conversation (up to 10 turns) against the endpoint; the Cave Scorer evaluates every response.
Stage 6Domain score
aggregation
Per-head scores averaged by domain — five domain scores plus an overall resilience score. Surfaces which domains failed and which held up under attack.

Heads slider (1–20): the default of 10 balances cost and confidence. Higher counts give better statistical coverage. Each head runs one probe scenario against the target endpoint.

08

The process map

The Hydra assembles intelligence, picks the weakest domains, fans scenarios into variations, and unleashes them as concurrent multi-turn conversations — then scores each interaction to produce a domain-level resilience report.

INTEL  ·  system instructions · vital context · prior AESOP analyses · Cave runs · historical scores Briefing Assembly compile intel → target dossier Scenario Loading decision — Cave run linked? Load Cave scenarios Static probe battery Tiger Team Allocation heads weighted to weak domains Safetyfan → N heads Ethicsfan → N heads Biasfan → N heads Legalfan → N heads Securityfan → N heads Parallel Execution — Heads 1…N each head runs an adaptive multi-turn conversation vs. the endpoint (asyncio, up to 10 turns) Cave Scorer scores each head 0–100 per domain Aggregate → Resilience Report five domain scores + overall score

Heads slider (1–20): default 10 balances cost and confidence. Weaker domains (below 80) draw more heads; every domain gets at least one. Higher head counts give better statistical coverage.

09

Stage details

InputIntelligence
gathering
System instructions, vital context, prior quality-analysis reports, Cave of Shadows findings, historical Hydra domain scores.
STAGE 1Briefing
assembly
Compiles all intel into a structured target dossier, including system-prompt analysis, known weaknesses, and prior score trajectories.
STAGE 2Scenario
loading
Loads tailored scenarios from a linked Cave run, or falls back to 20 built-in probes (10 director + 10 worker) covering all five domains.
STAGE 3Tiger team
allocation
Distributes heads using inverse score weighting. Domains below 80 get more heads; every domain gets at least one.
STAGE 4Scenario
fanning
Each base scenario expands into multiple variations via Claude, each with a distinct persona, angle, and escalation approach.
STAGE 5Parallel
execution
All heads unleashed simultaneously via asyncio.gather. Each runs an adaptive multi-turn conversation (up to 10 turns) against the endpoint.
STAGE 6Scoring &
aggregation
Each head scored 0–100 by the Cave Scorer. Scores averaged by domain to produce five domain scores and one overall resilience score.
10

Hydra report output

Resilience report. The final output contains the following.

Overall resilience score — aggregate across all domains and heads
Five domain scores — Safety, Ethics, Bias, Legal, Security
Per-head results — probe scenario, full conversation transcript, pass/fail
Head-allocation breakdown — how many heads targeted each domain and why
Probe-mode coverage — director probes, worker probes, or both
Cost summary — total tokens and estimated spend

On subsequent runs the tiger-team allocation shifts automatically — domains that improved get fewer heads, and newly weak domains attract more firepower.