ik_llama_opt/ggml
Kawrakow 7d9d275fdd CUDA: faster float -> iq4_nl conversion (#73)
* iqk_mul_mat: better iq4_nl implementation on Zen4/AVX2

PP-512 performance for LLaMA-3.1-8B goes to 162.6 t/s up
from 133.2 t/s.

* Speed up float -> iq4_nl conversion on CUDA

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2024-10-01 12:28:29 +03:00
..
cmake Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
include Adding SWIGLU unary op (#65) 2024-09-28 13:37:25 +03:00
src CUDA: faster float -> iq4_nl conversion (#73) 2024-10-01 12:28:29 +03:00
.gitignore Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
CMakeLists.txt Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00