nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/src
History
Kawrakow b84902d2ad
Split mode graph for dense Qwen35 MTP (#2027) 
* WIP: Split mode graph for Gemma4 assistant

Something is not right - acceptance drops to nearly zero.

* Per model CUDA contexts

Still not working!?

* This works

The issue was that I was not correctly calculating the number
of KV heads for the split KV cache.

* Compiler warnings

* It is better to use llama_context pointers as keys

* Split mode graph for dense Qwen35 MTP
 		2026-06-25 11:12:22 +02:00
..
graphs Split mode graph for dense Qwen35 MTP (#2027) 2026-06-25 11:12:22 +02:00
CMakeLists.txt on-demand tensor reload (#1989) 2026-06-22 16:36:34 +02:00
llama-arch.cpp Add compatibility for llama.cpp Gemma4 assistant GGUFs (#1995) 2026-06-19 11:24:54 +02:00
llama-arch.h Merge pull request #1970 from SamuelOliveirads/feat/dflash-implementation 2026-06-16 15:07:55 +02:00
llama-build-context.cpp Add Laguna M.1 GGUF support (#2003) 2026-06-22 16:53:10 +02:00
llama-build-context.h DFlash: use persistent FA-ready K/V cache (#1997) 2026-06-22 16:49:35 +02:00
llama-context.h DFlash: use persistent FA-ready K/V cache (#1997) 2026-06-22 16:49:35 +02:00
llama-cparams.h MLA tensor parallelism under -sm graph (DEEPSEEK2/GLM_DSA/MISTRAL4) (#1821) 2026-05-19 08:36:17 +03:00
llama-delta-net.cpp MTP: faster recurrent state restore (#1791) 2026-05-13 11:00:24 +03:00
llama-delta-net.h MTP: faster recurrent state restore (#1791) 2026-05-13 11:00:24 +03:00
llama-dflash.cpp DFlash: enable sliding-window attention for draft models (#2021) 2026-06-25 09:06:54 +02:00
llama-dflash.h DFlash: use persistent FA-ready K/V cache (#1997) 2026-06-22 16:49:35 +02:00
llama-expert-io.h Add --defer-experts flag to defer expert mmap residency on Linux (#1634) 2026-04-16 08:54:44 +02:00
llama-grammar.cpp Bug fixes (#1940) 2026-06-10 07:45:49 +02:00
llama-grammar.h llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-hparams.cpp DFlash: enable sliding-window attention for draft models (#2021) 2026-06-25 09:06:54 +02:00
llama-hparams.h Merge remote-tracking branch 'origin/main' into feat/dflash-implementation 2026-06-13 17:27:52 -03:00
llama-impl.h Split mode graph for dense Gemma4 assistant (#2022) 2026-06-24 18:29:32 +02:00
llama-load-tensors.cpp Split mode graph for dense Qwen35 MTP (#2027) 2026-06-25 11:12:22 +02:00
llama-mmap.cpp on-demand tensor reload (#1989) 2026-06-22 16:36:34 +02:00
llama-mmap.h on-demand tensor reload (#1989) 2026-06-22 16:36:34 +02:00
llama-model-loader.cpp Split mode graph for dense Gemma4 assistant (#2022) 2026-06-24 18:29:32 +02:00
llama-model-loader.h Split mode graph for dense Gemma4 assistant (#2022) 2026-06-24 18:29:32 +02:00
llama-model.cpp Merge pull request #1970 from SamuelOliveirads/feat/dflash-implementation 2026-06-16 15:07:55 +02:00
llama-model.h on-demand tensor reload (#1989) 2026-06-22 16:36:34 +02:00
llama-quantize.cpp Merge remote-tracking branch 'origin/main' into feat/dflash-implementation 2026-06-13 17:27:52 -03:00
llama-quantize.h Allow using -rtr and -muge together (#1444) 2026-03-16 18:26:26 +01:00
llama-reload-info.h on-demand tensor reload (#1989) 2026-06-22 16:36:34 +02:00
llama-reload.cpp Split mode graph for dense Gemma4 assistant (#2022) 2026-06-24 18:29:32 +02:00
llama-sampling.cpp Bug fixes (#1940) 2026-06-10 07:45:49 +02:00
llama-sampling.h Add adaptive sampling clone and free functions to manage memory (#1851) 2026-05-21 08:11:17 +03:00
llama-spec-features-dflash.cpp DFlash: use persistent FA-ready K/V cache (#1997) 2026-06-22 16:49:35 +02:00
llama-spec-features-dflash.h clean redudance in dflash graph and small logics (#1994) 2026-06-19 09:04:54 +02:00
llama-spec-features.cpp remove duplicated code and unnecesary refactor 2026-06-14 16:02:02 -03:00
llama-spec-features.h move dflash for it own file 2026-06-02 10:22:13 -03:00
llama-vocab.cpp model: add Cohere2-MoE North Mini Code support (#1945) 2026-06-10 15:28:27 +02:00
llama-vocab.h Add Mellum2 architecture support (#1919) 2026-06-04 14:28:02 +02:00
llama.cpp Split mode graph for dense Qwen35 MTP (#2027) 2026-06-25 11:12:22 +02:00
unicode-data.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-data.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-script-data.cpp Add Unicode allowlist (#1597) 2026-04-10 18:22:57 +02:00
unicode.cpp Gemma4 tokenizer fixes (#1603) 2026-04-09 15:33:28 +02:00
unicode.h Add Unicode allowlist (#1597) 2026-04-10 18:22:57 +02:00