Kawrakow 0f6621d410 Q2_K_R4 (#136) ... * q2_k_r4: Zen4 PP-512(LLaMA-3.1-8B) = 256 t/s * q3_k_r4: AVX2 * q2_k_r4: AVX2 We get PP-512(LLaMA-3.1-8B) = 287 t/s. Also cherry-picked the q3_k_r4 AVX2 adaptation that I somehow forgot to push upstream. * q2_k_r4: NEON We get PP-512(LLaMA-3.1-8B) = 106.2 t/s. TG-128 is 36.02 t/s, which is ~10% higher than q2_K_S. * Make sure rows per thread are a multiple of 4 --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>		2024-12-11 18:16:49 +01:00
..
ggml-cann	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-cuda	MMQ for Q6_0 (#115)	2024-11-21 07:12:11 +01:00
ggml-sycl	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
iqk	Q2_K_R4 (#136)	2024-12-11 18:16:49 +01:00
kompute@4565194ed7	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
kompute-shaders	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
llamafile	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
vulkan-shaders	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
CMakeLists.txt	Enable q6_0 for flash attention (#101)	2024-10-22 11:34:49 +02:00
ggml-aarch64.c	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-aarch64.h	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
ggml-alloc.c	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-backend-impl.h	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
ggml-backend.c	Bitnet changes (#106)	2024-10-25 13:08:43 +02:00
ggml-blas.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-cann.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-common.h	Q2_K_R4 (#136)	2024-12-11 18:16:49 +01:00
ggml-cuda.cu	Faster MoE inference (#112)	2024-10-31 12:05:27 +01:00
ggml-impl.h	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-kompute.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-metal.m	Faster MoE inference (#112)	2024-10-31 12:05:27 +01:00
ggml-metal.metal	Faster MoE inference (#112)	2024-10-31 12:05:27 +01:00
ggml-quants.c	Q2_K_R4 (#136)	2024-12-11 18:16:49 +01:00
ggml-quants.h	iq2_bn_r4: fastest Bitnet CPU implementation on the planet (#124)	2024-12-06 12:15:39 +01:00
ggml-rpc.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-sycl.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml-vulkan.cpp	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
ggml.c	Q2_K_R4 (#136)	2024-12-11 18:16:49 +01:00