Kawrakow 7e5af2073c Faster MoE inference (#112) ... * multi_sdd: WIP * multi_sdd: CPU works * multi_add: CUDA * multi_add: simplify * multi_add: Metal * Metal: speed up mul_mat_id For the Granite-1B MoE model PP-512 goes from 156 t/s to 890 t/s, so nearly a 6X speedup! --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>		2024-10-31 12:05:27 +01:00
..
cmake	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
include	Faster MoE inference (#112)	2024-10-31 12:05:27 +01:00
src	Faster MoE inference (#112)	2024-10-31 12:05:27 +01:00
.gitignore	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
CMakeLists.txt	Move to c++17 projectwide (#80)	2024-10-04 14:43:26 +03:00