A local 4-Tier Cascading Decision Engine. Each request stops at the cheapest tier that can answer it with confidence. Only the hard cases reach the full LLM.
Accurate on complex policy, but 1,500–25,000 ms per call and $5–$40 per 1,000 decisions.
Answer in 100–350 ms, but break down on multi-hop policy, temporal math, and prompt injection.
Intelligence 84.62 vs Opus 83.90, at a 44 ms median, on hardware you own, with no API bill.
Pre-compiled regex and keyword rules from RoutingRules/*.json run in under 50 µs and hot-reload without restarting the model.
The query is embedded with EmbeddingGemma 2 (740M params, 768D). It then takes a random walk across domain landmarks until it lands in one of the candidate answers, and the landing probabilities are computed in closed form.
Gemma 4 26B-A4B (MoE, 4B active) reads the answer-letter logits directly from a single forward pass. The scores are auto-calibrated with Platt temperature scaling, so the confidence values can be trusted.
Only 52 tokens matter: ' A'…' Z' and 'A'…'Z'. So we patched the output head to project onto a 64-row slice instead of the full 262,144-token vocabulary.
When Tier 1 confidence is low, or a rule says FORCE_COT, the full LLM continues from the KV cache Tier 1 already filled. The prompt is never processed twice. It writes up to 256 tokens of reasoning (max_cot_tokens=256), then reads a calibrated answer.
The rulebook stays pinned in KV cache. The next turn is decided while Pac-Man is still in the hallway, so it's ready by the junction with 0 ms wait.
Apache 2.0. Runs on one local GPU. No API keys, no data leaving the building, no per-token bill. Decisions and streaming chat share a single loaded model, so you don't need a second copy in VRAM.