Simulation Results

Track A — the organic, evidence-driven method — produced better outcomes on 7 of 8 metrics that showed meaningful divergence. Lower costs. Fewer errors. More customers stayed. But the real story isn’t the score. It’s the mechanism.

Track A delivered better business results across 7 of 8 metrics.

These three form a causal chain: lower translation debt → higher first-pass accuracy → lower cost per claim.

Bottom line: If you’re leading an AI transformation, stop asking “what’s our adoption target?” and start asking “what just broke, and what did we learn?” The operating model isn’t something you design in a conference room. It’s something you discover through implementation — one bounded deployment at a time.

A controlled simulation compared two approaches to AI transformation inside a modeled U.S. property-and-casualty insurer over 12 quarterly sprints. Both tracks processed claims, underwriting, and subrogation with the same 13 psychologically differentiated agent profiles. The only difference was the transformation method.

Metrics that showed meaningful differences between tracks (averaged across all 12 sprints).

The operating model for AI is not something an organization should fully design upfront — it is discovered through implementation.

• Track A’s observe-learn-redesign loop catches and fixes meaning-loss at organizational seams. Agents surfaced 137 missing-information gaps (Type 2 emergence) — decisions made from incomplete information where the incompleteness wasn’t recognized at decision time. Track A fixed these at the seam. Track B routed equivalent findings through a steering committee.
• Track A hits fewer exceptions — its loop fixes the seams that force claims into human handling. Track B’s higher exception rate means more claims stall on escalation and rework, not that Track B surfaces more problems.
• The trade-off is real: organic discovery produces better operational outcomes (lower costs, fewer errors, higher retention), but traditional change management provides more predictable governance (fewer surprises, higher trust scores, more policies bound). The right choice depends on whether you need speed of learning or predictability of process.

Specific evidence from this 10,168-decision run.

• Agent behavior diverged. The same psychological profile produced different behavior across tracks because the organizational rules differed — e.g. Tommy used AI in 45% of Track A decisions vs 44% in Track B and overrode AI 13% vs 10%; Diana used AI in 4% vs 3%.
• Supplement requests: 20.5% vs 22.8% — a 2.2-point gap. Track A’s redesigns complete the seams, so its request rate declines as gaps are fixed; Track B never completes the seams, so its rate stays high.
• Exception rate: 5.9% vs 6.5%. Track A’s loop fixes the seams that force claims into human handling; Track B’s higher rate means more claims stall on escalation.
• Translation debt: meaning lost at handoffs in 14.0% of Track A decisions vs 23.4% in Track B. Track A addresses translation seams as they appear; Track B routes them through steering committee → roadmap → vendor assessment.
• THEARI discoveries: Track A has 137 discoveries, all validated by evidence across multiple sprints. Track B has 12 that auto-advanced on schedule — same count, zero evidence required.
• Sample size: across 10,168 total decisions (5,081 A, 5,087 B), the divergence patterns are consistent in direction across sprints — not artifacts of small samples.

Translated to a public-market lens, Track A’s approach is worth roughly $6.3B more in market capitalization than Track B — about $10.8 per share (5.1% on PGR’s $213 baseline). A directional estimate, not a forecast.

• Cost channel: Track A handles a claim for $258 vs $346 — $88 cheaper. At 5.5M claims/year that is $484M a year in savings, or 0.56 points of combined ratio.
• Top-line channel: Track A retains 4.5 more points of customers, compounding into ~1.4 points of premium growth and $184M more annual income.
• Combined: $668M/year more net income, $1.15 of EPS, and — at 9.4× — $10.8 per share ($6.3B market cap).
• Caveat: the simulation’s absolute cost scale is not calibrated to real dollars, so this is built from the A-vs-B gap (defensible), not absolute levels. Anchored to PGR’s actual Q2 2026 combined ratio of 87.3%.

All tracked metrics, A/B average and gap across the 12-sprint run. (Per-sprint time series for every metric live in metrics.csv, below; the headline translation-debt trajectory is charted on the Findings Report.)

LLM-driven pattern detection — finds both known patterns (18 categories) and novel patterns outside any predefined category. Novel patterns, 16 total: 13 social, 2 workflow, 1 unclassified.

Track A — discovered patterns (a register condensed by theme; every finding was evidence-validated across sprints):

Track B — discovered patterns (timeline-gated, no evidence required):

Tracks discoveries through five phases of increasing certainty. Track A is evidence-gated; Track B auto-advances on schedule.

Representative Track A findings (all high-confidence, evidence-validated): translation-debt spikes at the approval handoff; a severe trust bifurcation (high-trust, low-stress agents vs low-trust, high-exhaustion agents); a bottleneck migrating to a single human; cycle time reported as 0.0 (a metric defect the engine caught); repeated shadow-AI behavior among high-experience agents.

Each track’s Discovery Engine unlocks higher autonomy as it proves reliability.

Agent behavior divergence. Same agent profile, different behavior across tracks — the difference comes from track rules, not personality. The starkest: Diana’s translation incidents are 216 in Track A vs 404 in Track B (71% of her decisions).

The same claims, processed through both tracks — where and why they diverge.

Download the underlying data files from this run.