ik_llama_opt/common
g2mt b6bc5eedad Port speculative decoding from upstream to llama-server (#645)
* server : integrate speculative decoding

* server: Fix field names

* server: fix include, whitespace

* fix compile errors in speculative.cpp

* add llama_sampling_sample_and_accept_n to sampling

* finish porting speculative decoding in server

* port functions from common/speculative, common/sampling

* remove arg

* fix function names

* init params_dft to none

* correct value for n_ctx

* prefix kv cache tensors with model name to avoid conflict

* fix call arguments

* fix spec decoding args

* correct slot.id

* use n_max

* port the rest of sampling funcs

* fix func arguments

* slot.id starts at 1?

* Revert "prefix kv cache tensors with model name to avoid conflict"

This reverts commit fbd5dfd8660ced64a05a23fe3d5526ded635eb4b.

* disable draft logging

* disable logging in speculative.cpp

in mainline, these would be LOG_DEBUG, but since ik_llama doesnt support
it, logging is disabled entirely

* add more draft model parameters

* fix

* pass flash_attn

* add speculative params for parity

* set speculative params in launch_slot_with_task instead
2025-08-16 07:26:44 +03:00
..
cmake Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
CMakeLists.txt Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
base64.hpp llava : expose as a shared library for downstream projects (#3613) 2023-11-07 00:36:23 +03:00
build-info.cpp.in build : link against build info instead of compiling against it (#3879) 2023-11-02 08:50:16 +02:00
chat-parser.cpp Fix for Deepseek r1 parsing (#676) 2025-08-08 13:56:44 +03:00
chat-parser.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
chat-template.hpp add jinja template support (#677) 2025-08-09 12:50:30 +00:00
chat.cpp Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
chat.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
common.cpp Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
common.h Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
console.cpp check C++ code with -Wmissing-declarations (#3184) 2023-09-15 15:38:27 -04:00
console.h gguf : new file format with flexible meta data (beta) (#2398) 2023-08-21 23:07:43 +03:00
grammar-parser.cpp Added support for . (any character) token in grammar engine. (#6467) 2024-06-06 06:08:52 -07:00
grammar-parser.h gguf : new file format with flexible meta data (beta) (#2398) 2023-08-21 23:07:43 +03:00
json-partial.cpp Function calling support for Kimi-K2 (#628) 2025-07-23 18:11:42 +02:00
json-partial.h Function calling support for Kimi-K2 (#628) 2025-07-23 18:11:42 +02:00
json-schema-to-grammar.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
json-schema-to-grammar.h JSON: [key] -> .at(key), assert() -> GGML_ASSERT (#7143) 2024-05-08 21:53:08 +02:00
json.hpp json-schema-to-grammar improvements (+ added to server) (#5978) 2024-03-21 11:50:43 +00:00
log.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
minja.hpp add jinja template support (#677) 2025-08-09 12:50:30 +00:00
ngram-cache.cpp Fixed lookup compilation issues on Windows (#6273) 2024-03-24 14:21:17 +01:00
ngram-cache.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
regex-partial.cpp Function calling support for Kimi-K2 (#628) 2025-07-23 18:11:42 +02:00
regex-partial.h Function calling support for Kimi-K2 (#628) 2025-07-23 18:11:42 +02:00
sampling.cpp Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
sampling.h Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
speculative.cpp Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
speculative.h Port speculative decoding from upstream to llama-server (#645) 2025-08-16 07:26:44 +03:00
stb_image.h examples: support LLaVA v1.5 (multimodal model) (#3436) 2023-10-12 18:23:18 +03:00
train.cpp train : change default FA argument (#7528) 2024-05-25 15:22:35 +03:00
train.h sync : ggml (backend v2) (#3912) 2023-11-13 14:16:23 +02:00