Under the hood
How On The Money answers a question, and how we know the answer is trustworthy.
1 · Grounded by construction
The agent never invents a number or a coordinate. It states only what a tool returned, and it draws maps by naming semantic ids (a district, a value, a color); the backend resolves those to real geometry. A map is provably correct or it does not render.
2 · Verify & calibrate
Before an answer ships, a verify step re-derives the district's true total from the ground-truth store and compares it to what the model claimed. If it cannot be verified, confidence is downgraded to insufficient evidence rather than guessing. That is why answers carry an honest High / Partial / Insufficient signal.
3 · Evaluated & gated
These are the deterministic recorded baseline — the reference replay the CI gate grades against ground truth. Real-model accuracy comes from the live eval run (run_live).
A caught regression
Seed a units bug that drifts every high-confidence total 10% low (e.g. case la01-funds: 15516512.00 → 13964860.80, across 16 cases), still reported confidently — and the gate flips from pass to fail:
| accuracy | Brier | gate | |
|---|---|---|---|
| clean | 100% | 0.058 | pass |
| with bug | 33% | 0.698 | fail |
Reproduce with make regression-demo.
4 · Operated, not just built
Every answer carries a live read-out: how many tools ran, wall-clock latency, the tokens spent, an estimated cost, and how many tool calls failed and recovered. A tool error is caught and counted, never a dead stream. Open the run-stats line under any answer to see it.