nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml/src
History
Kawrakow e811de75e9 BF16_R16 - 16 interleaved bf16 rows (#142) 
* Not working bf16_r4

* Adding bf16_r8

Small performance gain compared to bf16 - 258 t/s vs 234 t/s.
I guess, this is still sub-obtimal.

* bf16_rx: Very slightly faster by interleaving 16 rows

258 t/s -> 263 t/s

* Rename bf16_r4 to bf16_r16

We are interleaving 16 rows now.

* Cleanup unused stuff

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2024-12-15 09:54:21 +01:00
..
ggml-cann Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cuda MMQ for Q6_0 (#115) 2024-11-21 07:12:11 +01:00
ggml-sycl Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
iqk BF16_R16 - 16 interleaved bf16 rows (#142) 2024-12-15 09:54:21 +01:00
kompute@4565194ed7 Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
kompute-shaders Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llamafile Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
vulkan-shaders Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
CMakeLists.txt Enable q6_0 for flash attention (#101) 2024-10-22 11:34:49 +02:00
ggml-aarch64.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-aarch64.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-alloc.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-backend-impl.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-backend.c Bitnet changes (#106) 2024-10-25 13:08:43 +02:00
ggml-blas.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cann.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-common.h Q8_K_R8: Fastest quantized matrix multiplications (#141) 2024-12-14 09:24:30 +01:00
ggml-cuda.cu Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-impl.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-kompute.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-metal.m Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-metal.metal Faster MoE inference (#112) 2024-10-31 12:05:27 +01:00
ggml-quants.c BF16_R16 - 16 interleaved bf16 rows (#142) 2024-12-15 09:54:21 +01:00
ggml-quants.h iq2_bn_r4: fastest Bitnet CPU implementation on the planet (#124) 2024-12-06 12:15:39 +01:00
ggml-rpc.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-sycl.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-vulkan.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml.c BF16_R16 - 16 interleaved bf16 rows (#142) 2024-12-15 09:54:21 +01:00