77f8060ca6
* fix: FA vec kernels for D=256 quantized KV cache Three bugs prevented Flash Attention from working with quantized (q8_0) KV cache on models with head dimension 256, such as Qwen3-Coder-Next: 1. need_f16_K/V used dimension-based logic (Dk != 128) that forced q8_0→f16 conversion at D=256. The kernel then read f16 data as q8_0 blocks. Changed to type-based logic (type_K == GGML_TYPE_F16) matching mainline. 2. quantize_q8_1_to_shared output pointers were not advanced between loop iterations, so the second iteration overwrote the first half of Q data in shared memory. Added i0 offset to output pointers. 3. Q_i32 register array in vec_f32 kernel was sized to 1 instead of 2 for D=256 due to an errant comparison (Dk >= expr evaluates to boolean 1, not expr). Removed the comparison. Also adds extern template declarations for (256, Q8_0, Q8_0) so the kernel is compiled and linked. Tested on Pascal (GTX 1080, CC 6.1) with Qwen3-Coder-Next (D=256, n_head_kv=2) using --cache-type-k q8_0 --cache-type-v q8_0. * fix: use dimension-based need_f16 for D=256 vec kernels |
||
|---|---|---|
| .. | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||