Well Done! 👏 - Speed comparison to few other 3.6 27B and 3.8 27B quants
I was initially skeptical about the quality, but there hardly seems to be any degradation, and the speed gains are quite evident.
💻 Test Environment
- Hardware: Apple M3 Max (40-core GPU)
- Metric: tok/s = completion tokens (including thinking) ÷ wall-clock time
📊 Detailed Per-Test Speed Comparison
Splash Run #89 vs. OMLX 27B Family
| Test | Splash (tok/s) | Best OMLX 27B (tok/s) | Next-Best (tok/s) | Ratio |
|---|---|---|---|---|
| A1 (cycle-sort) | 35.8 | 29.2 (ThinkingCap) | 24.9 (qwopus) | 1.23× |
| A2 (JSON parser) | 48.0 | 29.2 (ThinkingCap) | 28.9 (AWQ) | 1.64× |
| A3 (regex) | 42.0 | 28.9 (ThinkingCap) | 27.5 (qwopus) | 1.45× |
| B1 (singleton) | 38.7 | 29.2 (ThinkingCap) | 28.2 (qwopus) | 1.33× |
| B3 (ring buffer) | 41.6 | 30.8 (ThinkingCap) | 29.6 (oQ6e) | 1.35× |
| C4 (rules) | 40.2 | 29.3 (ThinkingCap) | 28.8 (qwopus) | 1.37× |
| C6 (Bertrand) | 34.0 | 29.7 (ThinkingCap) | 25.9 (qwopus) | 1.14× |
| Tough | 40.1 | 30.4 (Swift-oQ4e) | 29.9 (AWQ) | 1.32× |
💡 Key Takeaways
- Consistent Lead: Splash leads every single test, hitting 34–48 tok/s compared to the OMLX 27B cluster's ~21–30 tok/s. This results in a consistent 1.3× to 1.6× speedup. Note that it doesn't quite hit the 2× mark claimed in the README—achieving that likely requires a highly draft-friendly workload.
- Massive Wall-Clock Victory: The actual real-world time savings are even larger than the raw tok/s metrics suggest. A full 3-run standard benchmark took 32.5 minutes on Splash vs.
91 minutes on AWQ (2.8× faster). - Efficiency Gains: This wall-clock win is partly driven by smarter token usage. Splash simply thinks less to get the job done (e.g., in test A1, Splash used 46k tokens compared to AWQ's 99k tokens).
Thanks for sharing your results! Great to see those!
Thanks for the detailed M3 Max tests! Splash 1.0.2 is now out with Apple9 decode improvements, including your 40-core M3 Max. We'd love to see a rerun with the same prompts and settings. You can update with brew update && brew upgrade incoai/tap/splash.
Of course! Running my personal benchmark is the least I could do to support the great work that you guys do! And the speed gains are actually pretty significant, as you can see in this updated table:
Splash v1.0 (#89) & v1.02 (#92/#93) vs. OMLX 27B Family
Token-weighted tok/s = Σcompletion_tokens / Σelapsed. Best/Next = top two OMLX 27B models per test.
| Test | Splash v1.0 | Splash v1.02 | Best OMLX 27B | Next-Best | Ratio v1.0 | Ratio v1.02 |
|---|---|---|---|---|---|---|
| A1 (cycle-sort) | 35.8 | 54.3 | 29.2 (ThinkingCap) | 24.9 (qwopus) | 1.23× | 1.86× |
| A2 (JSON parser) | 48.0 | 68.9 | 29.2 (ThinkingCap) | 28.9 (AWQ) | 1.65× | 2.36× |
| A3 (regex) | 42.0 | 67.0 | 28.9 (ThinkingCap) | 27.5 (qwopus) | 1.45× | 2.32× |
| B1 (singleton) | 38.7 | 55.0 | 29.2 (ThinkingCap) | 28.2 (qwopus) | 1.33× | 1.89× |
| B3 (ring buffer) | 41.6 | 42.9 | 30.8 (ThinkingCap) | 29.6 (oQ6e) | 1.35× | 1.39× |
| C4 (rules) | 40.2 | 59.6 | 29.3 (ThinkingCap) | 28.8 (qwopus) | 1.37× | 2.04× |
| C6 (Bertrand) | 34.0 | 48.4 | 29.7 (ThinkingCap) | 25.9 (qwopus) | 1.14× | 1.63× |
| Tough | 40.1 | 68.0 | 30.4 (Swift-oQ4e) | 29.9 (AWQ) | 1.32× | 2.24× |
v1.02 widens the lead over the best OMLX 27B from ~1.3× to ~1.4–2.4× (median ~1.9×).