Index/ Thesis Backend Audit

Thesis Backend Audit

architecture review — march 2026 — FastAPI + 22 agents + 7 schedulers

7
Schedulers
22
Agents in Memory
28
Route Modules
512MB
Fly.io RAM
~5s
Cold Start
01

1 — Scheduler Chaos

Critical
7 independent APScheduler instances, zero coordination
  • Every 5 min
    sync_schedulerGoogle DriveDISABLED but still starts
  • Daily 3:00 AM
    graph_syncNeo4j sync
  • Daily 4:00 AM
    engagementStakeholder
  • Daily 5:00 AM
    discovery_scanDISCO
  • Daily 6:00 AM
    researchAtlas
  • Weekly Mon 7 AM
    manifesto_digestCompliance
  • Immediate
    vault_watcherObsidian
  • Sequential startup adds 2-5s to cold start on a 512MB Fly.io machine
  • Race condition: graph_sync (3 AM) and engagement (4 AM) both touch stakeholder/graph data — no locking
  • Dead scheduler: Google Drive sync_scheduler still initializes despite integration being disabled
  • No coordination: each scheduler is an independent BackgroundScheduler() instance with its own thread pool
Consolidate into a single unified scheduler with job dependency ordering
02

2 — Duplicate Service Pipelines

High
Task Extraction
task_extractortask_auto_extractortask_kraken
3 overlapping services, unclear which runs when
Stakeholder Extraction
extractorscannerdeduplicatorlinker
4 services that could be 1 pipeline with stages
Chat Services
chat_agent_serviceproject_chat
2 chat paths with no shared abstraction
Document Processing
Upload triggerObsidian sync triggerAuto-extraction trigger
3 entry points, no deduplication gate
Impact: Same document can be processed 2-3x through different pipelines. Task extraction may produce duplicates across the three extractors. Stakeholder data may be inconsistent depending on which service ran last.
03

3 — Cold Start Bottlenecks

Medium
Startup time breakdown on 512MB Fly.io
7 schedulers (sequential)
2-5s
22 agents (import)
~1-2s
28 route modules
~0.5-1s
Supabase init (module level)
~0.3s
CORS middleware setup
~0.1s
  • agent_factory.py imports all 22 agents at module level — each agent imports its own dependencies
  • No lazy loading: agents not used in a request still consume memory
  • Supabase client initialized at module level in multiple files (no singleton pattern)
Defer schedulers 30s post-startup. Lazy-load agents on first use. Use a singleton Supabase client.
04

4 — Request Flow Issues

Medium
Middleware Chain
CustomCORSMiddleware
Logs INFO on EVERY request
→
HTTPSRedirectFix
OK
→
RequestIDMiddleware
8-char UUID may collide
→
Route Handler
No request timing
  • CORS logging at INFO level — 2-3 lines per request including health checks. Clutters logs, costs money on log aggregators
  • Short UUID (8 chars) from uuid4().hex[:8] — 16^8 = 4B combinations, collision risk grows with volume
  • No request timing middleware — can’t identify slow endpoints without custom profiling
05

5 — Quick Wins

Easy
ConfigCORS logging to DEBUG level1 line change, immediate noise reduction
RemoveDelete sync_scheduler startupDisabled integration still starting
PerfDefer schedulers 30s post-startupCuts cold start by 2-5s
RemoveRemove resend package (unused)Saves install time
RemoveRemove opportunities.routerMigrated to projects
DockerExpand .dockerignoreExclude tests/, scripts/, docs/
DockerMulti-stage Docker buildSmaller image, faster deploys
DockerAdd tesseract + popplerOCR/PDF deps missing from image
PerfRequest timing middlewareIdentify slow endpoints
06

6 — Priority Matrix

Impact /
Effort
Low Effort
High Effort
High Impact
CORS logging to DEBUG
Remove dead sync_scheduler
Defer schedulers 30s
Expand .dockerignore
Multi-stage Docker build
Unified scheduler system
Consolidate task extractors
Consolidate stakeholder services
Agent lazy-loading
Low Impact
Remove resend package
Remove opportunities router
Fix short UUID (8 chars)
Add request timing middleware
Recommended sequence: Start with the gold quadrant (high impact, low effort) — these are config changes and removals that can ship today. Then tackle agent lazy-loading and the unified scheduler as the next sprint. The silver quadrant items (service consolidation) are architectural and should be planned as a dedicated refactor.