nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/common
History
Samuel Oliveira Alves ea94afe777
Speculative checkpoints for recurrent models (#1669) 
* server: spec checkpoints for recurrent models

* fix: save/restore sampler state during speculative checkpoint

When speculative decoding rejects draft tokens and restores the
recurrent state checkpoint, the sampler (RNG, grammar, prev tokens)
must also be restored to maintain consistency. Without this, the
sampler state reflects the rejected draft tokens, leading to
potential divergence.

Uses common_sampler_clone() to snapshot the sampler before the
speculative batch decode, and restores it on rejection.

* server: snapshot recurrent state in tensor

* reset ngram mod state for rejected tokens

* server: refactor checkpoint state logic

* speculative: fix sampler for checkpoints

* recurrent model: implement recurrent kernel checkpoint

* recurrent model: refactor api

* spec: free rbudget before overwriting
 		2026-04-24 09:59:30 +02:00
..
cmake Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
jinja Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
CMakeLists.txt Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
base64.hpp llava : expose as a shared library for downstream projects (#3613) 2023-11-07 00:36:23 +03:00
build-info.cpp.in build : link against build info instead of compiling against it (#3879) 2023-11-02 08:50:16 +02:00
chat-auto-parser-generator.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-auto-parser-helpers.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-auto-parser-helpers.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-auto-parser.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-diff-analyzer.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-peg-parser.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat-peg-parser.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
chat.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
common.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
common.h Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
console.cpp check C++ code with -Wmissing-declarations (#3184) 2023-09-15 15:38:27 -04:00
console.h gguf : new file format with flexible meta data (beta) (#2398) 2023-08-21 23:07:43 +03:00
json-partial.cpp common : introduce composable PEG parser combinators for chat parsing and new jinja template engine (#1369) 2026-03-09 11:03:33 +01:00
json-partial.h Move minja and nlohmann/json to vendor (#802) 2025-09-27 09:12:35 +02:00
json-schema-to-grammar.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
json-schema-to-grammar.h common : introduce composable PEG parser combinators for chat parsing and new jinja template engine (#1369) 2026-03-09 11:03:33 +01:00
llguidance.cpp Tool calls support from mainline (#723) 2025-09-01 08:38:49 +03:00
log.cpp Refactor chat and server file (#1062) 2025-12-15 08:27:20 +01:00
log.h Server: refactor and rename functions (#1151) 2026-01-18 08:16:57 +02:00
ngram-cache.cpp spec : add self speculative decoding, ngram and refactor (#1261) 2026-02-13 19:04:55 +01:00
ngram-cache.h spec : add self speculative decoding, ngram and refactor (#1261) 2026-02-13 19:04:55 +01:00
ngram-map.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
ngram-map.h spec : add self speculative decoding, ngram and refactor (#1261) 2026-02-13 19:04:55 +01:00
ngram-mod.cpp spec : add self speculative decoding, ngram and refactor (#1261) 2026-02-13 19:04:55 +01:00
ngram-mod.h spec : add self speculative decoding, ngram and refactor (#1261) 2026-02-13 19:04:55 +01:00
peg-parser.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
peg-parser.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
reasoning-budget.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
reasoning-budget.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
regex-partial.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
regex-partial.h Tool calls support from mainline (#723) 2025-09-01 08:38:49 +03:00
sampling.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
sampling.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
spec-tuner.cpp Self-decoding: Adds support for suffix decoding (#1646) 2026-04-18 16:10:10 +02:00
spec-tuner.h Allow tuning of the best args for speculative decoding. (#1595) 2026-04-08 08:02:42 +02:00
speculative.cpp Speculative checkpoints for recurrent models (#1669) 2026-04-24 09:59:30 +02:00
speculative.h Add llama_context to MTP (#1601) 2026-04-09 15:33:56 +02:00
suffix-tree.cpp Self-decoding: Adds support for suffix decoding (#1646) 2026-04-18 16:10:10 +02:00
suffix-tree.h Self-decoding: Adds support for suffix decoding (#1646) 2026-04-18 16:10:10 +02:00
train.cpp Server: refactor and rename functions (#1151) 2026-01-18 08:16:57 +02:00
train.h sync : ggml (backend v2) (#3912) 2023-11-13 14:16:23 +02:00
unicode.cpp Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00
unicode.h Autoparser - complete refactoring of parser architecture (#1376) 2026-04-22 10:04:13 +02:00