AI generated

Quick update on tandem.cpp, my patch set for llama.cpp that runs MoE models too big for your VRAM faster. It keeps the hot experts on the GPU and lets the CPU help with the rest. Repo: https://github.com/ewhacc/tandem.cpp DeepSeek-V4.1-Flash IQ2_XS (ISTA GSQ-RCO, 139 GiB of experts) on normal PCs Decode tok/s, one user, GSM8K questions, greedy, exact (no expert skipping on any engine), same quant on every engine: |PC|llama.cpp|tandem.cpp|Strata| |:-|:-|:-|:-| |RTX 5070 Ti 16 GB, 64 GB RAM, i5-14600KF|7.4|14.2|10.4| |RTX 5070 Ti 16 GB, 64 GB RAM, i5-12400F, budget DRAM-less SSD|-|10.4|5.7| |RTX 5090 32 GB, 128 GB RAM, i5-14600KF|11.5|33.1|28.0| At 64 GB the model is about twice your RAM, so it streams from the SSD. Less RAM than the model: Qwen3.8-Flash-Next IQ3_S (ISTA GSQ-RCO, 78 GiB) + MTP on a 32 GB PC |PC|llama.cpp|tandem.cpp|Strata| |:-|:-|:-|:-| |RTX 5070 Ti 16 GB, 32 GB RAM, i5-12400F, budget DRAM-less SSD|14.9|34.3|30.3| When the model fits in RAM, Strata is a bit ahead on Qwen3.8 (RTX 4090 + 64 GB: 97 vs 104 tok/s). Where tandem.cpp wins is less RAM and other models: DeepSeek-V4.1, gpt-oss-120b, GLM-4.5-Air and Qwen3.6 all gain 1.4-2.1x over plain llama.cpp. AMD (ROCm) The patch now builds with HIP. Tested on an RX 7900 XTX (i7-8700, 64 GB DDR4, PCIe 3.0), Qwen3.8 IQ3_S + MTP, same day: |Engine|tok/s| |:-|:-| |tandem.cpp|68.6| |Strata 0.2.9|69.7| |Strata's RDNA3 fork|80.1| |Avalon (its own IQ3_KS quant)|83.6| Level with stock Strata. This week's kernel work is shared CUDA/HIP code, so it helps NVIDIA too: +2.5% on the 7900 XTX, +1.3% on a 5070 Ti. On an old 6-core CPU, the AMD-specific engines are still ahead. Vulkan isn't supported yet. Build steps for both are in the README. Questions and numbers from your boxes are welcome, especially 32 GB machines and AMD cards.