nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/github-data/pull_requests/252 - MLA-2_ Allow usage of...

344 B
Raw Permalink Blame History

🔀 #252 - MLA-2: Allow usage of q8_0 for KV cache on CUDA

Author ikawrakow
State ❌ Closed
Created 2025-03-12
Updated 2025-03-12

Description

Performance is slightly lower than f16 KV cache but not too bad.