nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml
History
Kawrakow a9fa3b1563 iq3_k: AVX512 iqk_mul_mat 
We get PP-512 = 180 t/s, TG-128(4 threads) = 16.35 on the Ryzen-7950X
for LLaMA-3.1-8B.
In comparison, iq3_s has PP-512 = 96 t/s, TG-128 = 7.6 t/s with
iqk_mul_mat, and PP-512 = 28 t/s, TG-128 = 6.8 t/s in mainline llama.cpp
 		2024-08-01 09:38:06 +02:00
..
cmake Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
include iq3_k: Basics 2024-08-01 09:38:06 +02:00
src iq3_k: AVX512 iqk_mul_mat 2024-08-01 09:38:06 +02:00
.gitignore Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
CMakeLists.txt Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00