Index/ Architecture & Beta Readiness
Solomon Engine · Architecture & Beta Readiness

The voice front-door to a personalized executive AI.

A ~10–15 min voice interview → structured extraction → an admin-reviewed, custom AI-assistant configuration. This one-pager reconciles the as-built system with its April–May beta testing history to inform post-feedback next steps.

Beta in market — 10+ exec cohort Baseline locked 2026-05-15 Prepared by Charlie Fuller 2026-06-04
01

Snapshot

at a glance
Interview Segments
2 — ~6 min each, own agent
Coverage Threshold
60% — to advance / trigger Solomon
Integration Suite
5/5 — pipeline verified end-to-end
Test/Fix Cycles
~15 — Apr 8 → May 19
Dominant Open Bug
1 — Segment-2 early wrap-up
Scope. This documents the Solomon voice-interview engine in beta — not the wider platform’s chat / RAG / knowledge-base layer. Where deployed reality differs from earlier collateral, see §06 Documentation Drift.
02

Architecture

as-built, verified against code

Tech stack

Frontend
Next.js 16 · React 19 · TS · Vercel
Backend
FastAPI 0.115 · Python 3.11 · Railway
Voice
ElevenLabs Conversational AI (2 agents)
Scoring
Claude Haiku
Instructions
Claude Sonnet 4-6
Data
Supabase (Postgres + RLS)
Email
Resend

Two segments

S1 — Pain Points & Workflow
pain_points · impact_baseline · TRIPS capture
S2 — Work Style & Tools
comms · strategy · team · frameworks · doc storage · org context

Prompt wiring

Segment prompts live in prompts/*.md, injected per-session via the ElevenLabs SDK overrides field. Dashboard base prompts are fully replaced at session start — historically the single biggest source of bugs.

Reliability guardrails

  • 60% threshold · skips excluded from denominator
  • Silence 10s timeout · 45s auto-end · 90s min-age
  • English-only · no identity re-ask · repeat handling
  • Duplicate-webhook + late-fetch + autoplay fixes

Key tables

  • interview_sessions · interview_segments
  • interview_extractions (+ eval JSON)
  • solomon_reviews — pending admin gate
  • api_usage_logs — token / cost

Admin tooling

  • Solomon review & approval workflow
  • Interviews history + transcripts
  • API-usage page (ElevenLabs credits)
  • On-demand AI quality-evaluation report
03

Processing Pipeline

what happens after the voice call
graph TD
  COL[Columbus
lead capture] --> V
  V["Voice interview
ElevenLabs · 2 agents"] -->|WebRTC disconnect| WH["Webhook
/interview-complete
HMAC-verified"]
  WH --> SC["Segment scoring
Claude Haiku · 0–100
skips excluded"]
  SC --> G{"Both segments
≥ 60%?"}
  G -->|No| RD["Flag for redo
can_redo = true"]
  G -->|Yes| S1["Solomon Stage 1
Extraction · Haiku
12 structured fields"]
  S1 --> G2{"≥ 60%?"}
  G2 -->|No| NR["needs_review
admin notified"]
  G2 -->|Yes| S2["Solomon Stage 2
Instructions · Sonnet
top 5 of 10 functions"]
  S2 --> REV["solomon_reviews
status = pending"]
  REV --> AG{"Admin
approve?"}
  AG -->|Reject| NR
  AG -->|Approve| DEP["Deploy to
client profile"]
  DEP --> PRD["PRD interview
(Inject & Skip)"]

  classDef gold fill:#E9E7FB,stroke:#4F46E5,color:#241A12,stroke-width:1.5px;
  classDef teal fill:#E4F1EE,stroke:#0E7C6E,color:#241A12,stroke-width:1.5px;
  classDef gate fill:#FAF0DC,stroke:#A5650A,color:#241A12,stroke-width:1.5px;
  classDef stop fill:#FBEBEA,stroke:#C23A30,color:#241A12,stroke-width:1.5px;
  classDef done fill:#ECE6FB,stroke:#7C3AED,color:#241A12,stroke-width:1.5px;
  class V,WH gold;
  class SC,S1,S2 teal;
  class G,G2,AG gate;
  class RD,NR stop;
  class DEP,PRD,REV done;
  class COL,PRD gold;
Nothing auto-deploys. Generated instructions always land in solomon_reviews as pending and require human approval. The Apr 29 integration suite confirmed every stage after the voice call — including duplicate-webhook and stuck-session recovery — passes 5/5.
04

Testing & Results

Apr 8 – May 19, 2026
C1 · Completion
Both segments reach coverage; no premature wrap-up.
C2 · Scoring
Score reflects quality; no false “All Done”; skips not penalized.
C3 · Control
Silence, English-only, no re-ask, graceful exit, recovery.
C4 · UI State
Results always load; stable buttons; accurate timer; recap.
C5 · Infra
Mic/WebSocket, audio on start, webhooks, email, credits.

Testing & Results timeline

DateRound & key findingResolution
Mar 4Stuck at 50% — invalid model ID (404); backend defaulted to 50% on scoring failureFixed — Model ID corrected
Apr 816-item UX changelog (scripting, lag, End button, recap, rating)All verified — BE 69 · FE 99 · E2E 130/1
Apr 21Stress test (Menchie/Niharika): false green @2–35%, skip penalty, 11s silence, S2 ran S1 script, language drift, vanishing results→ Apr 22
Apr 22Post-stress-test hardening11 fixes — threshold, skip-exclude, silence, coverage req, guardrail, race
Apr 23–24S2 stuck @52% (never reached Q5/Q6); brand colorFixed — S2 prompt rewrite · Sun Gold palette
Apr 29Root cause: override prop conversationConfigOverride silently ignored → agents ran on placeholder prompt the whole timeFixed — → overrides; +8-item reliability pass
Apr 29Automated integration suite (Charlie)5/5 pass — 92% / 95% high-quality runs
Apr 30–May 1Hard block: “Couldn’t connect” everywhere — 3 bugs (allowlist typo, timeout race, override fields disabled →1008)All fixed
May 5–6Paige blocked @50% “Scoring error” — production Anthropic key had zero credits; audio silent-start; repeat-questionCredits + code — autoplay fix · repeat guideline
May 12Result capture + nav good; S2 premature wrap-up persists (28% despite good input)Open
May 18–19Beta go/no-go: closing screen broke; refresh → “interview closed” lock-outFixed — results-wait 30→90s + retry; Submit Anyway; auto-deploy
The pattern: nearly every blocking failure was infrastructure wiring, not UX logic — prompt overrides never reaching ElevenLabs, a depleted API key silently scoring 50%, an allowlist hostname typo. The UX fixes stuck; the infra failures kept recurring in new forms.
05

Open Items

entering / during beta
#ItemSeverityStatus / decision
O1Segment 2 premature wrap-up — processes after 2–3 Q, skips Tech Stack / Work Styles (~28%)High · UXBaseline-locked 05-15; deep fix deferred to avoid regression. Topic-checklist guardrail proposed
O2Anthropic credit depletion silently degrades scoring to 50% fallbackHigh · OpsNeeds proactive credit alerting; partial via API-usage page
O3Audio silent-start on Segment 2MediumAutoplay fix shipped; monitor for recurrence
O4Contradiction-flagging guardrail never fully verifiedLowSegment kept ending before it could be tested
O5Email by manual link onlyProcessDeliberate — avoids spam-filter risk during beta
O6TBG logo is styled text, not image assetCosmeticAwaiting logo file
O7STATS-02 E2E failureNonePre-existing test-infra CORS issue, unrelated
06

Documentation Drift

reconcile before external use
Older collateral claimsAs-built reality
  • Four 5–7 min segments
  • 70% completeness threshold
  • Voyage AI embeddings in interview pipeline
  • Extracts “20+ parameters”
  • Interview “15–20 min” / agent says “30 min”
  • Two ~6-min segments (no S3/S4 prompts exist)
  • 60% across all enforcement points
  • Voyage powers chat/RAG only — not Solomon extraction
  • Stage 1 outputs 12 structured fields
  • Actual ~10–15 min; agents now say “about 5 min/segment”
07

Next Steps

post-beta-feedback, prioritized

Immediate

1 · Immediate
Permanent fix for credit depletion (O2)
Automated low-credit alerting on the Anthropic key — the highest-leverage reliability gap; it silently breaks scoring for every user.
2 · Immediate
Instrument beta sessions
Capture per-segment scores, completion rate, drop-off point, and Help/bug submissions to quantify O1’s real field frequency.

Near-term

3 · Near-term
Fix Segment-2 wrap-up (O1)
Hard topic-checklist guardrail — but only after field data, in an isolated branch with the integration suite as a regression gate.
4 · Near-term
Verify contradiction-flagging (O4)
Run a fresh full Segment-2 session to confirm the guardrail fires.

Structural

5 · Structural
Reduce the “Double Interview”
Implement CIPHER “Inject & Skip” — Solomon JSON pre-populates / hides ~65% of PRD fields. Largest UX-fatigue lever; shapes the Solomon→PRD roadmap.
6 · Structural
Reconcile documentation (§06)
Align beta & sales collateral with the system before external sharing.

Roadmap

7 · Roadmap
Hybrid voice + clickable · 8 · Transcript-quality protocol
Lower ElevenLabs token cost; target 90% accuracy on core pain-point extraction. Re-evaluate late June–July.

SOURCE · docs/changelogs (Apr 8 – May 19) · docs/testing · SUPERASSISTANT_APPLICATION_SUMMARY.md · solomon-post-interview-pipeline.md · backend/services/{interview_segments, solomon_stage1, solomon_stage2}.py · prompts/*.md
AGENTS · solomon-1 agent_0001kmrzfqzwe5qtvjevrfcqa486 · solomon-2 agent_9101kms0edweeyqtgrw2n9n7h2bw  |  LIVE · app.thebradburygroup.net
THE BRADBURY GROUP · CONFIDENTIAL · companion to docs/SOLOMON_ARCHITECTURE.md