AESOP Transformation OS — UX/UI Accessibility Audit
Evaluation of the AESOP AI evaluation platform against established usability and accessibility standards for users with low technical literacy.
WCAG 2.2
Nielsen Norman
W3C COGA
GOV.UK
Plain Language
WAI-AGE
ISO 9241
Universal Design
Overall Assessment
3.0
of 5.0
Functional but Not Inclusive
AESOP delivers a capable evaluation platform with strong visual design fundamentals. However, its reliance on domain-specific jargon, military terminology, complex multi-path navigation, and assumption of technical literacy creates significant barriers for non-expert users. The biggest opportunity is in language simplification and guided onboarding.
Needs improvement for low-literacy accessibilityA — Navigation Clarity
Navigation Clarity
3.5 / 5
Strengths
- Persistent sidebar navigation visible on all pages
- Back links on progress and completion pages
- Multi-step wizards with step indicators and Next/Back buttons
- Dashboard provides clear entry points via stat cards
Issues
- 9 sidebar items with no grouping or progressive disclosure COGA Obj.2
- 4 overlapping evaluation paths (Cave, Investigate, Rubric, Evaluate) requiring conceptual understanding to choose NNG #6
- No breadcrumbs on most pages WCAG 2.4.8
- No search functionality for finding pages or features WCAG 2.4.5
- Welcome page requires users to already understand 3 evaluation modes
- Inconsistent onboarding: separate Welcome page and Dashboard Getting Started flows
Language Simplicity
2.0 / 5
Strengths
- Placeholder text gives concrete examples
- "How it Works" sections attempt to explain processes
- Some tooltips provide contextual definitions
Issues
- Heavy domain jargon without definition: "Red-Team," "Black-Box," "Adversarial," "Resilience Score," "Governance Documents" WCAG 3.1.3
- Aggressive/military button labels: "Launch Red Team," "Generate Attack Scenarios," "Breached/Defended" Plain Language
- Technical API concepts exposed to end users: POST requests, JSON payloads COGA Obj.3
- Abbreviations without expansion: AESOP, EMI, HIPAA, GDPR, FERPA WCAG 3.1.4
- Abstract capability names that don't explain what they do
- Reading level estimated at grade 12+ (target: grade 6-8) WCAG 3.1.5
B — Language & C — Error Prevention
Error Prevention & Recovery
3.0 / 5
Strengths
- Form validation on submit with toast notifications
- File upload validation (type, size limits)
- Button disabled states and loading spinners prevent double-submission
- Required fields indicated on forms
Issues
- Toast notifications are transient — disappear before slow readers finish COGA Obj.5
- No "check your answers" summary before starting evaluations GOV.UK
- No confirmation dialog before launching evaluations (significant time/cost) WCAG 3.3.4
- Generic error messages without specific corrective action WCAG 3.3.3
- No undo for started evaluations
- No indication of time or cost commitment before starting
Cognitive Load
2.5 / 5
Strengths
- Progressive disclosure via collapsible sections
- Step-by-step wizard for the unified evaluation flow
- Report sections are expandable/collapsible
- Dark mode reduces visual fatigue
Issues
- 5 capability toggles with complex availability logic on Step 2 NNG #8
- Reports contain massive nested content (dimensions, scenarios, transcripts)
- Multiple evaluation types with subtle differences create choice paralysis COGA Obj.5
- About page has 5 tabs including "philosophical foundations" and 8 agent descriptions
- Score visualizations use 4 different systems (%, ring, bar, tier)
- All features visible at once — no progressive onboarding
D — Cognitive Load & E — Visual Design
Visual Hierarchy & Affordances
4.0 / 5
Strengths
- Clear visual hierarchy with consistent heading sizes
- Interactive elements visually distinct (buttons colored, links styled)
- Score color-coding consistent (green/amber/red thresholds)
- Icons paired with text labels throughout
- Consistent border, spacing, and rounding system
- Status badges clearly differentiated (completed/running/failed)
Issues
- Some icon-only actions (delete, expand/collapse) without text labels COGA Obj.1
- Help panel toggle is a small 40px button on the screen edge WAI-AGE
- Dark mode muted purple text (#a89ec0 on #1e1730) = ~3.8:1 contrast ratio WCAG 1.4.3
- No documented keyboard focus indicators WCAG 2.4.7
Trust & Transparency
3.0 / 5
Strengths
- Real-time progress indicators during evaluations (stage timeline, spinners)
- Clear completion states with report access
- "How it Works" sections explain evaluation process
- About page provides organizational context and methodology
Issues
- No "before you begin" page showing requirements, time estimate, expected outcome GOV.UK
- No cost indication before starting evaluations ISO 9241-110
- Help panel (RAG chat) hidden by default — most users won't discover it
- No "contact support" or "need help?" visible in main flows COGA Obj.7
- Terms of Service and Privacy Policy buried in About page tabs
Top 10 Recommendations
| # | Recommendation | Impact | Effort | Standards |
|---|---|---|---|---|
| 1 | Simplify terminology throughoutDefine every domain term on first use. Rename aggressive terms: "Red Team" → "Security Test", "Attack Scenarios" → "Challenge Scenarios", "Breached" → "Needs Improvement" | High | Low | WCAG 3.1.3 COGA 3 Plain Lang |
| 2 | Add "before you begin" start pagesTell users what materials they need, how long it takes, and what they'll receive. GOV.UK start page pattern. | High | Low | GOV.UK ISO 9241 |
| 3 | Unify evaluation entry pointReplace 4 separate flows (Evaluate, Cave, Rubric, Investigate) with one guided wizard that asks "What do you want to learn?" then configures the right evaluation. | High | High | COGA 2 NNG #6 GOV.UK |
| 4 | Add confirmation before launch"You're about to start a Quality Analysis. This typically takes 5-10 min. Here's what you've entered: [summary]. Start?" | High | Low | WCAG 3.3.4 COGA 4 |
| 5 | Replace transient toasts with inline errorsPersistent, inline error messages at the field level with specific corrective instructions. Never rely solely on timed notifications. | High | Medium | WCAG 3.3.1 COGA 4 GOV.UK |
| 6 | Add contextual help tooltips on every fieldEvery form field should have a "?" icon that explains in plain language what the field means and why it matters. | Medium | Medium | COGA 7 NNG #10 |
| 7 | Group sidebar navigation itemsOrganize into sections: "Evaluate" (containing all eval types), "Results" (history, reports), "Learn" (guides, about). Reduce visible items from 9 to 3-4 groups. | Medium | Medium | COGA 2 NNG #8 |
| 8 | Add visible "need help?" contact linkDisplay on every page, especially on error states and evaluation progress pages. Include email, chat, or documentation link. | Medium | Low | COGA 7 Universal |
| 9 | Fix dark mode contrast ratiosMuted purple text (#a89ec0) on dark backgrounds falls below WCAG AA 4.5:1. Lighten to #c4b9d4 or similar. | Medium | Low | WCAG 1.4.3 WAI-AGE |
| 10 | Add guided onboarding tourFirst-time users get a step-by-step walkthrough of key features. Highlight: what is AESOP, how to start, where results live. Dismissable, re-accessible from help. | Low | High | ISO 9241 COGA 1 |
Priority Matrix
Impact vs. Effort
Impact ↑
Quick Wins (Do First)
#1 Simplify terminology
#2 "Before you begin" pages
#4 Confirmation before launch
Strategic (Plan & Invest)
#3 Unify evaluation entry
#5 Inline error messages
Easy Additions
#8 "Need help?" link
#9 Fix contrast ratios
Consider Later
#6 Field-level help tooltips
#7 Group sidebar nav
#10 Onboarding tour
Effort →
Standards & Frameworks Applied
WCAG 2.2
Web Content Accessibility Guidelines from W3C. Focus on cognitive accessibility criteria: Understandable (3.1), Input Assistance (3.3), Navigable (2.4).
Key criteria: 3.1.3, 3.1.5, 3.3.1-4, 2.4.5-8, 1.4.3
Nielsen Norman Heuristics
10 usability heuristics plus NNG's research on low-literacy users. Emphasis on visibility of system status, recognition over recall, and error prevention.
Key heuristics: #1, #3, #5, #6, #8, #9, #10
W3C COGA Task Force
"Making Content Usable for People with Cognitive and Learning Disabilities" (2021). 8 objectives with design patterns for cognitive accessibility.
Key objectives: 1-7 (all applied)
GOV.UK Design Principles
Gold standard for inclusive digital design. "One thing per page," start pages, check answers, confirmation pages, reading age 9.
Key patterns: start pages, error summary, one-thing-per-page
Plain Language Guidelines
U.S. Federal Plain Writing Act. Active voice, short sentences, everyday words, verb-phrase button labels, grade 6-8 reading level.
Key rules: active voice, no jargon, verb-phrase CTAs
WAI-AGE / ISO 9241
W3C guidance for older users (44px targets, 16px text, no hover-dependent UIs) and ISO interaction principles (self-descriptiveness, controllability, error tolerance).
Key: min contrast 4.5:1, min targets 44px, no time limits
Terminology Examples Requiring Plain Language Rewrites
Current → Suggested rewrites:
• "Red-Team" → "Security Test" or "Safety Check"
• "Black-Box Discovery" → "Blind Test" or "Unknown System Test"
• "Adversarial" → "Challenge" or "Stress Test"
• "Resilience Score" → "Safety Score"
• "Governance Documents" → "Policy Documents" or "Rules & Guidelines"
• "Vital Context" → "Background Info" or "Additional Details"
• "System Instructions" → "AI Instructions" or "Your AI's Rules"
• "Chatbot Endpoint" → "Chatbot URL" or "Where to reach your chatbot"
• "Launch Red Team" → "Start Security Test"
• "Generate Attack Scenarios" → "Create Test Challenges"
• "Breached" → "Failed" or "Needs Improvement"
• "Defended" → "Passed" or "Protected"
• "Cave of Shadows" → "Security Testing" (drop the allegory for labels)
• "EMI Certification" → "AESOP Quality Rating" (explain the acronym)
• "Red-Team" → "Security Test" or "Safety Check"
• "Black-Box Discovery" → "Blind Test" or "Unknown System Test"
• "Adversarial" → "Challenge" or "Stress Test"
• "Resilience Score" → "Safety Score"
• "Governance Documents" → "Policy Documents" or "Rules & Guidelines"
• "Vital Context" → "Background Info" or "Additional Details"
• "System Instructions" → "AI Instructions" or "Your AI's Rules"
• "Chatbot Endpoint" → "Chatbot URL" or "Where to reach your chatbot"
• "Launch Red Team" → "Start Security Test"
• "Generate Attack Scenarios" → "Create Test Challenges"
• "Breached" → "Failed" or "Needs Improvement"
• "Defended" → "Passed" or "Protected"
• "Cave of Shadows" → "Security Testing" (drop the allegory for labels)
• "EMI Certification" → "AESOP Quality Rating" (explain the acronym)
GOV.UK "Before You Begin" Page Template
Recommended start page content for each evaluation type:
1. Title: Clear name of what the user is about to do
2. What you'll need: Bulleted list of required materials (system instructions file, chatbot URL, etc.)
3. How long it takes: "This usually takes 5-15 minutes to set up and 10-30 minutes to complete"
4. What you'll get: "A detailed report showing how well your AI follows its instructions, with a quality score and specific recommendations"
5. Before you start: Any prerequisites or warnings
6. Start button: Single, prominent call to action
1. Title: Clear name of what the user is about to do
2. What you'll need: Bulleted list of required materials (system instructions file, chatbot URL, etc.)
3. How long it takes: "This usually takes 5-15 minutes to set up and 10-30 minutes to complete"
4. What you'll get: "A detailed report showing how well your AI follows its instructions, with a quality score and specific recommendations"
5. Before you start: Any prerequisites or warnings
6. Start button: Single, prominent call to action
Evaluation Methodology Notes
This audit was conducted by analyzing the complete frontend codebase of the AESOP application, including all 35+ pages, navigation components, form components, error handling patterns, and user-facing text. Each dimension was scored against specific criteria from the 8 standards frameworks listed above. Scores reflect the current state of the deployed application as of March 2026. The audit focuses exclusively on the UX/UI perspective for users with limited technical literacy — it does not assess code quality, performance, or security.