AI generated

We have merged a dedicated Vulkan Flash Attention prefill kernel for Intel Arc and Xe, delivering massive prefill acceleration on supported platforms like ARL-H and Xe2/Xe3. This implementation leverages customized shaders to utilize Intel's native matrix core shapes and SLM architecture, including m16n16k16/SIMD32 support. To ensure stability, the kernel is currently enabled for these specific architectures while remaining disabled for others, like the A770, due to inconsistent test results. Additionally, backend refinements now extend these benefits to single-KV-head models like MQA and MLA. # Platform Prefill Performance Comparison (pp8192) |Platform|Model|Baseline (t/s)|PR (t/s)| |:-|:-|:-|:-| |ARL-H Windows|qwen3 8B Q4_K|126.58 ± 0.48|415.31 ± 1.04| ||qwen35 27B Q4_K|83.38 ± 0.22|119.95 ± 0.07| ||qwen35moe 35B.A3B Q4_K|247.82 ± 0.65|394.44 ± 0.89| ||gpt-oss 20B Q4_K|478.14 ± 0.17|639.02 ± 0.70| |PTL-H Windows|qwen3 8B Q4_K|234.92 ± 0.39|911.60 ± 14.56| ||qwen35 27B Q4_K|194.93 ± 0.59|264.39 ± 0.38| ||qwen35moe 35B.A3B Q4_K|632.12 ± 17.56|914.80 ± 67.39| ||gpt-oss 20B Q4_K|804.70 ± 84.56|1357.34 ± 125.30| |B70 Pro Windows|qwen3 8B Q4_K|1064.67 ± 4.37|2955.83 ± 1.87| ||qwen35 27B Q4_K|659.10 ± 0.11|932.13 ± 0.63| ||qwen35moe 35B.A3B Q4_K|2088.50 ± 3.32|3005.11 ± 4.24| ||gpt-oss 20B Q4_K|2311.37 ± 2.56|3967.29 ± 6.77| |B70 Pro Linux|qwen3 8B Q4_K|783.67 ± 0.20|1518.19 ± 1.05| ||qwen35 27B Q4_K|497.22 ± 1.17|647.78 ± 2.44| ||qwen35moe 35B.A3B Q4_K|1060.55 ± 5.09|1311.62 ± 7.55| ||gpt-oss 20B Q4_K|1237.51 ± 2.16|1841.35 ± 3.78| # Intel Arc B580: Detailed Model Benchmarks (FA1) Comparison of f16 Prefill (pp16384) performance |Model|Baseline $rightarrow$ PR|$Delta$%| |:-|:-|:-| |Gemma4 E4B Q4_K_M|115.98 $rightarrow$ 1156.62|+897.3%| |Phi-3.5-mini Q4_K_M|384.45 $rightarrow$ 932.90|+142.7%| |Granite 4.2 8B Q4_K_M|282.67 $rightarrow$ 520.98|+84.3%| |DeepSeek-R1-Qwen3-8B Q5_K_M|306.88 $rightarrow$ 559.34|+82.3%| |Llama 3.1 8B Q4_K_M|347.79 $rightarrow$ 633.73|+82.2%| |Ministral-3-8B Q5_K_M|324.17 $rightarrow$ 590.91|+82.3%| |Qwen3.5-9B Q6_K|687.50 $rightarrow$ 979.49|+42.5%| |Gemma3 4B Q4_1|1414.61 $rightarrow$ 1948.23|+37.7%| |Granite 4.0-h-tiny Q4_K_M|2503.03 $rightarrow$ 2927.33|+16.95%| |Ling-3.0-tiny (hkv=1)|76 $rightarrow$ 1239|+1530.3%| |Gemma4 12B (hkv=1)|52 $rightarrow$ 649|+1144.2%|