Kawrakow 947a348990 Adding SWIGLU unary op (#65) ... * Adding GGML_UNARY_OP_SWIGLU This commit implements the ggml op and CPU compute forward. I see ~3-4% speedup of PP-512 for Phi-3.5-mini. * GGML_UNARY_OP_SWIGLU: CUDA implementation I observe ~12% speedup for PP-512(Phi-3.5-mini). * GGML_UNARY_OP_SWIGLU: Metal implementation We get ~2% speedup for PP-512(Phi-3.5-mini). * GGML_UNARY_OP_SWIGLU: minor improvement on Metal * GGML_UNARY_OP_SWIGLU: cleanup --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>		2024-09-28 13:37:25 +03:00
..
cmake	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
include	Adding SWIGLU unary op (#65)	2024-09-28 13:37:25 +03:00
src	Adding SWIGLU unary op (#65)	2024-09-28 13:37:25 +03:00
.gitignore	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
CMakeLists.txt	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00