nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml/src/iqk
History
Kawrakow 77396a74b5 Better FlashMLA (#243) 
* This is a better FA for TG

It should benefit MLA and GQA. Tested to work with
DeepSeek-Lite MLA, not yet for GQA.
For tg64@pp8192 it is ~13% faster than MLA without FA,
and 57% faster that the main branch FA.

* WIP

* Cleanup

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2025-03-07 09:46:58 +02:00
..
iqk_common.h Better FlashMLA (#243) 2025-03-07 09:46:58 +02:00
iqk_config.h Hopefully this really fixes the confusion between AVX512 and FANCY_SIMD (#216) 2025-02-21 15:33:25 +02:00
iqk_flash_attn.cpp Better FlashMLA (#243) 2025-03-07 09:46:58 +02:00
iqk_flash_impl.h Better FlashMLA (#243) 2025-03-07 09:46:58 +02:00
iqk_mul_mat.cpp Better FlashMLA (#243) 2025-03-07 09:46:58 +02:00
iqk_mul_mat.h Better FlashMLA (#243) 2025-03-07 09:46:58 +02:00
iqk_quantize.cpp Hopefully this really fixes the confusion between AVX512 and FANCY_SIMD (#216) 2025-02-21 15:33:25 +02:00
iqk_quantize.h Q8_KV: 8-bit quantization type targeting the KV cache (#208) 2025-02-19 11:47:07 +02:00