nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/src
History
Kawrakow 82c4f27332
Fuse the attention gate in Step-3.5-Flash (#1244) 
* WIP

* This works but is slow

* Turn off the up / gate clamps for now

* OK we need the clamping

* Fuse the clamp (CUDA)

* Fuse the clamp (CPU)

* WIP

* Be able to use merged q, k, v

* Be able to use merged up/gate experts

* Fuse the clamp (CUDA mmvq)

* WIP: graph parallel for Step-3.5

* WIP

* This should be it

* Cleanup

* Fix merge

* Not working attempt to extend fused_mul_unary to the Step-3.5 case

* It works now, but performance gain is very minor
 		2026-02-07 07:56:58 +02:00
..
CMakeLists.txt Enable and clean up compiler warnings in src (#824) 2025-10-11 16:01:13 +03:00
llama-arch.cpp Step-3.5: llama.cpp compatibility changes (#1240) 2026-02-07 07:56:11 +02:00
llama-arch.h Step-3.5: llama.cpp compatibility changes (#1240) 2026-02-07 07:56:11 +02:00
llama-build-context.cpp Fuse the attention gate in Step-3.5-Flash (#1244) 2026-02-07 07:56:58 +02:00
llama-build-context.h Step-3.5-Flash support (#1231) 2026-02-05 08:13:22 +02:00
llama-context.h POC: CUDA tensor parallel (MoE models) (#1022) 2025-12-01 19:25:40 +01:00
llama-cparams.h Additional graph reduce types for split mode graph (#1154) 2026-01-18 08:02:49 +02:00
llama-grammar.cpp llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-grammar.h llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-hparams.cpp Step-3.5: llama.cpp compatibility changes (#1240) 2026-02-07 07:56:11 +02:00
llama-hparams.h Step-3.5: llama.cpp compatibility changes (#1240) 2026-02-07 07:56:11 +02:00
llama-impl.h server: stop processing the prompt when client disconnects (#1134) 2026-01-13 07:56:59 +02:00
llama-load-tensors.cpp Graph parallel for Step-3.5-Flash (#1236) 2026-02-06 06:56:51 +02:00
llama-mmap.cpp Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
llama-mmap.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
llama-model-loader.cpp Step-3.5-Flash support (#1231) 2026-02-05 08:13:22 +02:00
llama-model-loader.h Merge ffn_up and ffn_gate experts tensors (#1137) 2026-01-12 18:30:53 +02:00
llama-model.cpp Step-3.5-Flash support (#1231) 2026-02-05 08:13:22 +02:00
llama-model.h Graph parallel for Step-3.5-Flash (#1236) 2026-02-06 06:56:51 +02:00
llama-quantize.cpp Merge ffn_up and ffn_gate experts tensors (#1137) 2026-01-12 18:30:53 +02:00
llama-sampling.cpp llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-sampling.h Adaptive p: history update fix + temp as flag (#1213) 2026-02-03 07:36:12 +02:00
llama-vocab.cpp Server: refactor and rename functions (#1151) 2026-01-18 08:16:57 +02:00
llama-vocab.h Update mtmd to improve accuracy of M-RoPE (#993) 2025-11-29 07:27:15 +01:00
llama.cpp Fix graph parallel when ngl < n_layers (#1241) 2026-02-06 11:48:24 +02:00
unicode-data.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-data.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode.cpp Server: refactor and rename functions (#1151) 2026-01-18 08:16:57 +02:00
unicode.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00