ik_llama_opt/src
Samuel Oliveira Alves ea94afe777
Speculative checkpoints for recurrent models (#1669)
* server: spec checkpoints for recurrent models

* fix: save/restore sampler state during speculative checkpoint

When speculative decoding rejects draft tokens and restores the
recurrent state checkpoint, the sampler (RNG, grammar, prev tokens)
must also be restored to maintain consistency. Without this, the
sampler state reflects the rejected draft tokens, leading to
potential divergence.

Uses common_sampler_clone() to snapshot the sampler before the
speculative batch decode, and restores it on rejection.

* server: snapshot recurrent state in tensor

* reset ngram mod state for rejected tokens

* server: refactor checkpoint state logic

* speculative: fix sampler for checkpoints

* recurrent model: implement recurrent kernel checkpoint

* recurrent model: refactor api

* spec: free rbudget before overwriting
2026-04-24 09:59:30 +02:00
..
graphs Refactor model compute graphs (#1651) 2026-04-18 17:08:43 +02:00
CMakeLists.txt Refactor model compute graphs (#1651) 2026-04-18 17:08:43 +02:00
llama-arch.cpp Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
llama-arch.h Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
llama-build-context.cpp Refactor model compute graphs (#1651) 2026-04-18 17:08:43 +02:00
llama-build-context.h Refactor model compute graphs (#1651) 2026-04-18 17:08:43 +02:00
llama-context.h Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
llama-cparams.h Control compute buffer sizes for large batches (#1560) 2026-04-01 07:57:52 +02:00
llama-delta-net.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
llama-delta-net.h Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
llama-expert-io.h Add --defer-experts flag to defer expert mmap residency on Linux (#1634) 2026-04-16 08:54:44 +02:00
llama-grammar.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
llama-grammar.h llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-hparams.cpp Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
llama-hparams.h Graph parallel for Gemma4-31B (#1596) 2026-04-09 08:00:22 +02:00
llama-impl.h Full graph parallel for Qwen3.5 (dense and MoE) (#1388) 2026-03-10 09:08:24 +01:00
llama-load-tensors.cpp Fix Gemma4 partial offload (#1657) 2026-04-19 14:25:05 +02:00
llama-mmap.cpp Add --defer-experts flag to defer expert mmap residency on Linux (#1634) 2026-04-16 08:54:44 +02:00
llama-mmap.h Add --defer-experts flag to defer expert mmap residency on Linux (#1634) 2026-04-16 08:54:44 +02:00
llama-model-loader.cpp fix: use int8_t for GGUF bool array loading instead of platform-dependent bool (#1648) 2026-04-17 07:25:07 +02:00
llama-model-loader.h Add --defer-experts flag to defer expert mmap residency on Linux (#1634) 2026-04-16 08:54:44 +02:00
llama-model.cpp Fix Gemma4 partial offload (#1657) 2026-04-19 14:25:05 +02:00
llama-model.h Better routing for Gemma4-MoE (#1615) 2026-04-11 15:19:02 +02:00
llama-quantize.cpp Quantization options (#1677) 2026-04-23 09:05:39 +02:00
llama-quantize.h Allow using -rtr and -muge together (#1444) 2026-03-16 18:26:26 +01:00
llama-sampling.cpp Log probabilities on token sampling crash (#1519) 2026-03-26 14:34:41 +01:00
llama-sampling.h Adaptive P sampler: update review logic, delete old code comments, put prep stage after logit bias (#1386) 2026-03-14 12:34:12 +01:00
llama-vocab.cpp Gemma4 tokenizer fixes (#1603) 2026-04-09 15:33:28 +02:00
llama-vocab.h Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
llama.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
unicode-data.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-data.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-script-data.cpp Add Unicode allowlist (#1597) 2026-04-10 18:22:57 +02:00
unicode.cpp Gemma4 tokenizer fixes (#1603) 2026-04-09 15:33:28 +02:00
unicode.h Add Unicode allowlist (#1597) 2026-04-10 18:22:57 +02:00