Two-stage AI pipeline. Router classifies. Evaluator scores across five dimensions. Programmatic scoring never trusts the LLM directly. Every decision auditable.
The LLM returns dimension scores and reasoning. The programmatic layer re-computes the weighted score and applies threshold rules. If the LLM hallucinates a score, the deterministic layer catches it. Conflict type gates score levels — a direct adverse conflict caps the overall score regardless of other dimensions.
Every evaluation stores the complete prompt sent to the LLM, including the router classification context, the rubric version, and the model configuration. If the rubric changes, you can trace which evaluations used which version.
The full LLM response is stored — not just the scores, but the reasoning behind each dimension. If a partner questions a conflict flag, the system shows exactly why the evaluator flagged it.
The programmatic scoring layer is deterministic. Given the same dimension scores, it produces the same overall score and risk level. The scoring formula is versioned and stored alongside each evaluation.
All evaluation events are logged as structured JSONL — machine-readable, queryable, and exportable. No parsing. No black-box "the system said so." Every field is typed and documented.