Charlie Fuller · AESOP Transformation OS

AESOP Transformation OS

Safety, accountability, and scale. The governed platform that plans, builds, evaluates, and repairs agentic AI — mapped end to end.

Platform

Executive OverviewWhat the platform does for the business — safety, accountability, and scale.→ Full Platform ShowcaseComplete technical walkthrough of every stage, agent, and evaluation mode.→ Full Pipeline FlowAll 10 stages from Organization Context through Monitoring, with metrics.→ Platform OverviewHigh-level architecture and platform capabilities.→ Context ArchitectureWhat context the platform gathers, where it comes from, and what it changes.→ PipelineStage-by-stage pipeline mechanics and handoffs.→ OperationalizationLaunch package: RACI, Tiger Team brief, maintenance and adoption plans.→ Technical AgentsThe 30+ agents that power evaluation, building, and repair.→ Decision FlowHow decisions move through the platform.→ Platform OptionsAll 7 platform targets — enterprise, API agents, deployment — with scoring logic.→ Recursive Self-Help SystemThe help reads the running app and rebuilds itself on every change.→

Discovery

Two-Phase Authored DiscoveryFrom mapping what exists (1) to questioning structural assumptions (2) — before anything gets built.→ Project Requirements DiscoveryPRD generation pipeline — plan, gather, synthesize, export.→ Executive Intake → RoadmapHow executive input flows through intake to a roadmap process map.→ What Happens to User InputFull scoring trace from submission through triage and prioritization.→

Methodology

AI Maturity Model — ExecutiveFive levels of organizational AI maturity, scored and visualized.→ AI Maturity Model — FullComplete model with all dimensions and scoring criteria.→ Implementation CycleHow projects move through the full lifecycle.→ Leveraging DeterminismSplitting every decision in two: the AI reads and judges, code does the arithmetic. What it made more predictable, less wasteful and cheaper to run.→

Adoption Roadmaps

Example RoadmapSample AI Adoption Roadmap deliverable — Gulf Coast Marine & Yacht.→ SMB AI Readiness Roadmap — TechnicalTechnical readiness assessment and staged adoption path.→ SMB AI Adoption Roadmap — BusinessBusiness-focused adoption stages with organizational readiness gates.→

Evaluations

Red TeamRed-team adversarial testing with attack personas.→ Bias ProbeDifferential fairness testing across demographics.→ Sounds Like AIScores how much an agent's replies read as machine-written.→ Rubric testingDomain-specific weighted rubrics for custom evaluations.→ InvestigationBlack-box endpoint discovery and agent analysis.→ RepairExtended-thinking rewrite with change summary and rationale.→ Trace Analysis: OverviewWhat it is, how you reach it, and what it changes. Decision path inspection through qualitative failure coding.→ Trace Analysis: LoopThe mechanism. A person codes the traces, the taxonomy becomes failure judges, each judge is validated against that person, and the rate is re-measured after a repair.→ Humane EvalScores the effect an agent is likely to have on a person, across 28 research-backed behaviours. A rate reports only where a model's coding matches a person's. A sibling to Trace Analysis, not a step in it.→ build-eval + hillclimbFull flow of the claude-api skill's two commands: how build-eval collects data, selects and validates a grader, and runs a baseline; how hillclimb iterates one change at a time with train/test overfitting guards.→ AESOP → Hillclimb HandoffHow AESOP's Export + Hillclimb Pack turns a completed evaluation run into structured input for /claude-api hillclimb — cases.jsonl, baseline.json, grader.md, and failure clusters in one zip.→ KB Audit ProcessKnowledge base completeness and coherence audit.→

Monitor

Production MonitoringThe runtime loop for a deployed agent: capture interactions, extract deterministic and probabilistic signals, diff against a baseline, and close the repair loop when a failure mode drifts.→ Deterministic EngineWhy the same trace scores the same every time: pure functions over the retrieval trace, no LLM, each check naming the rule it broke. Reads the trace a LangChain + Databricks RAG already emits.→ Xcel Monitor BuildThe build behind the loop: 30 governed documents, a deliberately naive retrieval agent, and a monitor that reads provenance and scored 0.944 precision on 240 labeled calls.→ LangChain Retrieval LayerThe retrieval path for the Xcel demo: a question becomes a query with no jurisdiction or status filter, and the retriever emits the trace the monitor reads. The null filter is the diagnosis, not a bug.→ Databricks Index LayerHow 30 governed documents become a Delta table and a DELTA_SYNC Vector Search index, with the governance columns that let the monitor's checks fire. The columns are the wiring.→ Monitor Rules EngineWhat the monitor does with one retrieval record: five fixed rules, no AI judge, the same verdict every run, each finding routed to a named owner. The part you can audit.→ Audit–Monitor Feedback LoopHow the monitor becomes the KB audit's ground truth: a doc can score 90 and still be served to the wrong state. Production findings re-weight, re-scope, and re-rank the audit.→ Demo DeckFive pages walking the case: why the layer is missing, the approach, the loop, the drift it catches, and why it holds up. Arrow keys or the edge arrows move page to page. Start here.→

Research & Context

Organization Research ProcessFive-stage Deep Research pipeline: Scope → Search → Fetch → Verify → Synthesize.→ Strategic PortfolioPrioritization agent that scores AI project candidates against strategic goals.→ Organization ContextDeep research and org context capture — the foundation every agent is built against.→

Governance

Delegation & ReadinessAssesses organizational readiness for AI delegation and autonomy.→ Regulatory MappingMaps regulatory requirements to agent evaluation criteria.→

Onboarding

AESOP OnboardingEnd-to-end client onboarding flow — intake through first roadmap delivery.→

Consulting

Consulting PortfolioAESOP Transformation OS — AI agent governance and operationalization consulting.→ Consultant OfferAI transformation consulting — what's included, pricing, engagement model.→

Integrations

Monday.com IntegrationMonday.com as the work-management front-end, wired into the AESOP Transformation OS.→ Glean IntegrationBuild Glean agents end-to-end — workflow JSON, skills, and custom actions.→ Claude IntegrationBuild Claude agents end-to-end — system prompt, tools, and MCP.→ OpenAI IntegrationBuild OpenAI assistants — config, tools, and vector store.→ Full-Stack CloudBuild full-stack apps end-to-end — Next.js, FastAPI, and Supabase.→ GenericBuild a platform-neutral system prompt deployable to any LLM platform.→

Site Audit

UX AuditUser experience audit across the platform.→ Color AuditDesign system color consistency audit.→ Testing & CI/CD — OverviewHigh-level testing strategy and CI/CD pipeline design.→ Testing & CI/CD — DetailedDetailed test plans and CI/CD configuration.→

Plans

Clarity EngineThree-layer diagnostic map for organizational AI readiness.→ Governance ArchetypePortfolio governance pipeline — intake, approvals, research, materials, portfolio refresh.→