Seb-9B-v2 (9B): the same 'decision model first, LLM fallback' idea, with numbers
Since ClaimBench frames this as how much the decision model can take over before a fallback: Disclosure: I'm the author of Seb-9B-v2 (https://huggingface.co/ironbcc/Seb-9B-v2), a 9B decision model on Qwen3.5-9B, Apache-2.0. It returns a probability for every option in one forward pass, speaks /v1/systemone and a plain chat API, reads images, and takes inputs up to 32k tokens. These are self-run numbers, not independent ones: Decision Index 0.3 public index 59.16 (board numbers: Jev 1.13 57.96, Jev 1.14 60.70), ahead of Jev 1.13 on 22 of 37 benchmarks and clearly behind it on knowledge and reasoning. About 22 ms per decision on one RTX PRO 6000, about 88 ms on an Apple M5 Max. Write-up with the misses included: https://huggingface.co/blog/ironbcc/seb-9b-v2
On a ClaimBench-style set I built from public 10-Ks (not the Vals data, which isn't public; 300 items), Seb-9B-v2 reached 0.917 accuracy (hosted Jev 1.13 0.923, Jev 1.14 0.920) and automated about 48% of items at a 1% error budget with Claude Haiku as fallback (median over random 30/70 splits, wide range). Details in the write-up.