nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml/src
History
Kawrakow d47e1c63b3 Performance improvements for legacy quants on ARM_NEON (#37) 
* WIP: trying to improve legacy quants

* WIP: trying to improve legacy quants

With this commit PP-512 for LlaMA-3.1-8B goes from
72 t/s to 87.2 t/s for q4_0, and from 61.5 t/s to 73.9 t/s
for q4_1, so 20+% improvement for both.

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2024-09-04 07:24:04 +03:00
..
ggml-cann Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cuda Faster Gemma2 (#27) 2024-08-27 17:40:59 +03:00
ggml-sycl Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
iqk Performance improvements for legacy quants on ARM_NEON (#37) 2024-09-04 07:24:04 +03:00
kompute@4565194ed7 Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
kompute-shaders Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
llamafile Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
vulkan-shaders Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
CMakeLists.txt Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-aarch64.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-aarch64.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-alloc.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-backend-impl.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
ggml-backend.c Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-blas.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cann.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-common.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-cuda.cu Faster Gemma2 (#27) 2024-08-27 17:40:59 +03:00
ggml-impl.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-kompute.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-metal.m Faster Gemma2 (#27) 2024-08-27 17:40:59 +03:00
ggml-metal.metal Faster Gemma2 (#27) 2024-08-27 17:40:59 +03:00
ggml-quants.c AVX2 quantization for Q8_K (#22) 2024-08-19 15:33:27 +03:00
ggml-quants.h Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-rpc.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-sycl.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml-vulkan.cpp Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
ggml.c Zen4 Flash Attnetion 2 (#36) 2024-09-04 07:20:55 +03:00