nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/src
History
Kawrakow 61304f5c04 Q4_0_R4 (#119) 
* Adding iq4_0_r4 - q4_0 repacked

We get PP-512(LLaMA-3.1-8B) = 278 t/s on a Ryzen-7950X CPU,
so ~5-6% faster than iq4_nl_x4.

* q4_0_r4: NEON

Here we get 115.8 t/s, so also ~5% better than iq4_nl_x4.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2024-12-02 17:01:48 +01:00
..
CMakeLists.txt Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llama-grammar.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
llama-grammar.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llama-impl.h Time to fix replace_all (#68) 2024-09-28 17:59:47 +03:00
llama-sampling.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llama-sampling.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llama-vocab.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
llama-vocab.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
llama.cpp Q4_0_R4 (#119) 2024-12-02 17:01:48 +01:00
unicode-data.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-data.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00