nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml/src
History
Kawrakow 7c94c3da56 IQ1_M_R4: better 1.75 bpw quants (#187) 
* iq1_m_r4: basics (quantize/dequantize)

* iq1_m_r4: Zen4 gemm

* iq1_m_r4: neon gemm

* iq1_m_r4: switch to q8_0_x4 also on AVX2/Zen4

With the deltas being per group of 8, we cannot make use
of the q8 sums stored in q8_1, so we get a tiny gain by
using q8_0_x4.

* iq1_m_r4: rename mul_mat_iq1_m_r4_q8_1 to mul_mat_iq1_m_r4_q8_0

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2025-02-06 14:08:52 +02:00
..
ggml-cann Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cuda MMQ for Q6_0 (#115) 2024-11-21 07:12:11 +01:00
ggml-sycl Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
iqk IQ1_M_R4: better 1.75 bpw quants (#187) 2025-02-06 14:08:52 +02:00
kompute@4565194ed7 Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
kompute-shaders Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llamafile Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
vulkan-shaders Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
CMakeLists.txt Enable q6_0 for flash attention (#101) 2024-10-22 11:34:49 +02:00
ggml-aarch64.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-aarch64.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-alloc.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-backend-impl.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-backend.c Bitnet changes (#106) 2024-10-25 13:08:43 +02:00
ggml-blas.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cann.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-common.h IQ1_M_R4: better 1.75 bpw quants (#187) 2025-02-06 14:08:52 +02:00
ggml-cuda.cu Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-impl.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-kompute.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-metal.m Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-metal.metal Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-quants.c IQ1_M_R4: better 1.75 bpw quants (#187) 2025-02-06 14:08:52 +02:00
ggml-quants.h IQ1_M_R4: better 1.75 bpw quants (#187) 2025-02-06 14:08:52 +02:00
ggml-rpc.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-sycl.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-vulkan.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml.c IQ1_M_R4: better 1.75 bpw quants (#187) 2025-02-06 14:08:52 +02:00