ab6d816818
v_offset is used as a column index into the dequantized f16 buffer, but was computed as a half-pointer difference. That is only correct for f16 K/V. With q8_0 (34 bytes per 32 elements) it yields the wrong column, so the sparse attention path reads V from the wrong positions and generation degenerates. |
||
|---|---|---|
| .. | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||