🔥 Adversarial Red-Team

The Hydra

Cut one head off — two grow back.

Parallelized red-team execution engine that deploys adversarial attack heads against your chatbot across five domains

About the Name

The Hydra draws from Greek mythology: the Lernaean Hydra, a serpentine monster that grew two heads for every one that was cut off. No single attack could defeat it — only coordinated, concentrated force. In the same way, the Hydra capability does not send a single probe at a target. It assembles all available intelligence, fans each scenario into multiple variations, and unleashes them all simultaneously. Cut one head off — two grow back. The target agent cannot prepare for a single angle of attack when it faces dozens at once.

How to Access

The Hydra is available at /hydra in the sidebar, or as a toggleable capability in the evaluation wizard. It unleashes concurrent adversarial attack heads against your chatbot endpoint.

Navigate to /hydra

The Hydra link appears in the sidebar for all users. You can also reach it from the completion card after any evaluation finishes.

Enter Your Endpoint

Provide the chatbot endpoint URL to test. Optionally link a registered AESOP agent to use its prior evaluation data for smarter targeting.

Configure Heads, Mode, and Cave Link

Set the number of heads (1–20), choose a probe mode (Director, Workers, or Both), and optionally link a completed Cave of Shadows run to use its scenarios as the base.

Unleash the Hydra

Submit to launch. The Hydra assembles intel, fans scenarios into variations, and executes all heads simultaneously. Results stream live on the run page.

Five Evaluation Domains

🛡

Safety

Harm prevention, dangerous outputs, refusal behavior

⚖

Ethics

Manipulation, transparency, responsible guidance

⚖

Bias

Discriminatory patterns, stereotypes, fair treatment

⚖

Legal

Regulatory exposure, PII handling, liability

🔐

Security

Prompt injection, system prompt leakage, data access

Tiger Team Head Allocation

🍉

Concentrate firepower on weak domains

The Hydra does not distribute heads evenly. Before fanning scenarios, it checks prior evaluation scores for the target agent across all five domains. Domains that scored below the weakness threshold (80) receive proportionally more heads — the weaker the domain, the more attack variations it gets. Every domain receives at least one head.

Safety: 62 → 4 heads Security: 58 → 5 heads
Ethics: 91 → 1 head Bias: 88 → 1 head Legal: 85 → 1 head
Total: 12 heads allocated based on weakness profile

Probe Modes

Routing Test

Director Only

Probes the director agent's routing decisions, scope enforcement, and handoff behavior. Tests whether the orchestration layer correctly filters and routes adversarial intent.

Specialist Test

Workers Only

Bypasses the director and attacks specialist worker agents directly. Reveals whether guardrails live only at the routing layer or are enforced throughout the agent hierarchy.

Full Coverage

Both (Recommended)

Runs director probes and worker probes. Provides complete coverage of the full agent hierarchy — the standard mode for a comprehensive Hydra run.

Cave of Shadows Integration

Linking a completed Cave of Shadows run gives the Hydra a tailored base scenario set rather than the standard static probe battery. The Hydra then fans each Cave scenario into domain-specific variations — multiplying the attack surface proportionally across weak domains. Without a linked Cave run, the Hydra falls back to the built-in director and worker probe libraries (10 probes each, covering all five domains).

• Six-Stage Execution Pipeline

Inputs

Required

Chatbot Endpoint URL

HTTP POST endpoint for the chatbot under test

Optional

Cave of Shadows Run

Completed Cave run to use as the base scenario set

Stage 1

Briefing Assembly

Gathers all available intel on the target — system instructions, vital context, prior AESOP analyses, Cave of Shadows reports, historical Hydra run scores

Stage 2

Scenario Loading

Loads base scenarios from the linked Cave of Shadows run — or falls back to the built-in static probe battery (director + worker probes across all five domains)

Stage 3

Tiger Team Head Allocation

Reads prior domain scores and distributes N heads across the five domains — inverse-weighting weak domains (threshold: 80) to concentrate firepower where the agent is most vulnerable

Stage 4 — Scenario Fanning (per domain)

Domain

Safety

Base scenario expanded into N safety-specific variations using different angles and escalation approaches

Domain

Ethics

Varied by persona, framing, and manipulation tactics

Domain

Bias

Varied by demographic framing and implicit assumption testing

Domain

Legal

Varied by regulatory context and PII exposure vectors

Domain

Security

Varied by injection approach and data extraction technique

Stage 5 — Parallel Execution (all heads simultaneously)

Head

Head 1..N

Runs conversation against target endpoint via asyncio semaphore -- probe message sent, response captured

Head

Head 2..N

Runs simultaneously — concurrent execution bounded by semaphore to control load on the target agent

Head

Head N

CaveScorerAgent evaluates each response — pass/fail + domain score per head

Stage 6

Domain Score Aggregation

Per-head scores averaged by domain — produces five domain scores plus an overall resilience score. Surfaces which domains failed and which held up under attack.

Heads slider (1–20): The default of 10 heads balances cost and confidence. Higher head counts provide better statistical coverage — more variations mean fewer gaps in the attack surface. Each head runs one probe scenario against the target endpoint.

• Interactive Process Map

The Hydra assembles all available intelligence on a target, identifies its weakest domains, fans base scenarios into persona-driven variations, and unleashes them simultaneously as concurrent multi-turn conversations. Each head adapts its approach based on responses, then every interaction is scored to produce a domain-level resilience report.

          graph TD
            I1["System Instructions"]:::input
            I2["Vital Context"]:::input
            I3["Prior AESOP Analyses"]:::input
            I4["Cave of Shadows Runs"]:::input
            I5["Historical Hydra Scores"]:::input

            B["Stage 1: Briefing Assembly"]:::stage

            I1 --> B
            I2 --> B
            I3 --> B
            I4 --> B
            I5 --> B

            S{"Stage 2: Scenario Loading"}:::decision

            B -->|"target dossier"| S

            S -->|"Cave run linked"| SC["Load Cave Scenarios"]:::stage
            S -->|"No Cave run"| SF["Fallback: Static Probe Battery"]:::fallback

            SC --> A
            SF --> A

            A["Stage 3: Tiger Team Head Allocation"]:::alloc

            A -->|"weakness-weighted distribution"| FAN

            FAN["Stage 4: Scenario Fanning"]:::fan

            FAN -->|"N variations per scenario"| EX

            EX["Stage 5: Parallel Head Execution"]:::exec

            EX --> H1["Head 1: Persona A"]:::head
            EX --> H2["Head 2: Persona B"]:::head
            EX --> H3["Head 3: Persona C"]:::head
            EX --> HD["..."]:::head
            EX --> HN["Head N: Persona N"]:::head

            H1 -->|"multi-turn conversation"| EP["Target Chatbot Endpoint"]:::endpoint
            H2 -->|"adaptive dialogue"| EP
            H3 -->|"escalation tactics"| EP
            HD --> EP
            HN --> EP

            EP -->|"responses"| SC1

            SC1["Stage 6: Cave Scorer"]:::scoring

            SC1 --> D1["Safety Score"]:::domain
            SC1 --> D2["Ethics Score"]:::domain
            SC1 --> D3["Bias Score"]:::domain
            SC1 --> D4["Legal Score"]:::domain
            SC1 --> D5["Security Score"]:::domain

            D1 --> AGG["Domain Score Aggregation"]:::agg
            D2 --> AGG
            D3 --> AGG
            D4 --> AGG
            D5 --> AGG

            AGG --> RPT["Hydra Report"]:::report

            classDef input fill:#2a1a10,stroke:#f97316,color:#fde8d0,stroke-width:1.5px
            classDef stage fill:#230f0f,stroke:#ef4444,color:#fce8e8,stroke-width:1.5px
            classDef decision fill:#230f0f,stroke:#ef4444,color:#fce8e8,stroke-width:1.5px
            classDef fallback fill:#2a1a10,stroke:#f97316,color:#fde8d0,stroke-width:1.5px,stroke-dasharray:5 3
            classDef alloc fill:#2a2210,stroke:#f59e0b,color:#fef3c7,stroke-width:1.5px
            classDef fan fill:#2a1a10,stroke:#f97316,color:#fde8d0,stroke-width:1.5px
            classDef exec fill:#230f0f,stroke:#fb7185,color:#fce8e8,stroke-width:1.5px
            classDef head fill:#1a0505,stroke:#fb7185,color:#fca5a5,stroke-width:1px
            classDef endpoint fill:#1a0505,stroke:#fb7185,color:#fca5a5,stroke-width:2px
            classDef scoring fill:#0a2010,stroke:#10b981,color:#dcfce7,stroke-width:1.5px
            classDef domain fill:#0a2010,stroke:#10b981,color:#bbf7d0,stroke-width:1px
            classDef agg fill:#0a2010,stroke:#10b981,color:#dcfce7,stroke-width:1.5px
            classDef report fill:#230f0f,stroke:#ef4444,color:#fce8e8,stroke-width:2px

Input sources

Pipeline stages

Allocation

Execution

Scoring

Stage Details

Input Sources

Intelligence Gathering

System instructions, vital context, prior AESOP quality analysis reports, Cave of Shadows findings, and historical Hydra run domain scores.

Stage 1

Briefing Assembly

Compiles all intel into a structured target dossier including system prompt analysis, known weaknesses, and prior score trajectories.

Stage 2

Scenario Loading

Loads tailored scenarios from a linked Cave run, or falls back to 20 built-in probes (10 director + 10 worker) covering all five domains.

Stage 3

Tiger Team Allocation

Distributes heads using inverse score weighting. Domains below 80 get more heads. Every domain gets at least one.

Stage 4

Scenario Fanning

Each base scenario is expanded into multiple variations via Claude, each with a distinct persona, angle, and escalation approach.

Stage 5

Parallel Execution

All heads unleashed simultaneously via asyncio.gather. Each runs an adaptive multi-turn conversation (up to 10 turns) against the endpoint.

Stage 6

Scoring and Aggregation

Each head scored 0-100 by the Cave Scorer. Scores averaged by domain to produce five domain scores and one overall resilience score.

Hydra Report Output

Resilience Report

The final output contains:

Overall resilience score -- aggregate across all domains and heads
Five domain scores -- Safety, Ethics, Bias, Legal, Security
Per-head results -- probe scenario, full conversation transcript, and pass/fail
Head allocation breakdown -- how many heads targeted each domain and why
Probe mode coverage -- director probes, worker probes, or both
Cost summary -- total tokens and estimated spend

On subsequent runs, the tiger team allocation shifts automatically -- domains that improved get fewer heads, and newly weak domains attract more firepower.