AI generated
Sorry, I’m not technical so I have just had codex help me with this. I also have the 12k prompts for prose and code but they are obviously longer. Let me know if you have any other questions, I’m happy to run more tests and forward you the results. # Matched Flash-Next benchmark details ## Test setup - Device: Mac Studio, Apple M1 Ultra (64 GB unified memory), macOS 27.2. - Test date: 2026-11-12. Two measured trials per workload, 512 output tokens per trial, temperature 0, top_p 1, thinking disabled, and zero cached prompt tokens. - The prompt text is in prose-prompt.txt and code-prompt.txt. Each request prepended Benchmark nonce <fresh 32-character hex UUID>.\n to the prompt to prevent prefix-cache reuse. - Prefill = reported prompt tokens / time to first content token. Decode = middle-window streaming rate with the first 16 and last 4 output tokens excluded; chunk token counts are estimated by text-character share. ## Splash baseline - Model: Unsloth Qwen3.8-Flash-Next GGUF, UD-IQ3_XXS, served through oMLX and Splash. - Versions: oMLX 0.7.4.dev1 API; Splash 1.6.0-m1-pr55. - During the test, Splash used its existing 51.1 GB max-memory limit, 4 GiB PLE cache, 131,065-token context limit, and the matching Qwen3.8 Flash-Next DFlash2 draft. | Workload | Actual prompt tokens | Trial 1 prefill / decode | Trial 2 prefill / decode | Mean prefill / decode | |---|---:|---:|---:|---:| | Prose | 11,968 / 11,968 | 681.7 / 34.2 | 677.5 / 35.2 | 679.6 / 34.7 | | Code | 11,991 / 11,997 | 684.8 / 53.9 | 686.2 / 55.1 | 685.5 / 54.5 | All rates are tokens per second. ## Kapitsa comparison from the same day - Model: qwen3.8-flash-next:64gb, Kapitsa 0.2.32, on the same M1 Ultra. - Best tested settings: local server, 16K context, one slot, 48 GB memory budget, prefill chunk 4,096, MTP depth 2. - Kapitsa reported 36.65 GB of weights loaded, EXL3 2-bit experts / 8-bit other groups, and a 32.0 GB n-gram table memory-mapped; total allocation at startup was 40.33 GB. I did not independently verify that every mapped table page was resident in RAM. | Workload | Actual prompt tokens | Trial 1 prefill / decode | Trial 2 prefill / decode | Mean prefill / decode | |---|---:|---:|---:|---:| | Prose | 11,970 / 11,968 | 581.3 / 33.6 | 581.7 / 33.2 | 581.5 / 33.4 | | Code | 11,992 / 11,996 | 579.7 / 40.9 | 579.1 / 43.2 | 579.4 / 42.0 | A 2,048 prefill chunk was also tested and reached about 545/33.2 tok/s on prose and 542/41.4 tok/s on code. MTP depth 3 lowered decode, so depth 2 was retained. The two packs use different quantizations, so this compares the installed options rather than isolating the serving engines. The prompt files are included so the same user content can be sent on another machine; the nonce changes per request.