ik_llama_opt/src
Samuel Oliveira Alves 1f3e832cb3
Improve mtp acceptance rate (#1499)
* wip: port MTP architecture

Ports the Multi-Token Prediction (MTP) architecture to the older `llama.cpp` codebase used by `ikllama`.

Changes include:
- Updating `llama_batch` to support `mtp_params`.
- Modifying `llama_decode_internal` (and `encode`) to handle MTP operations (Warmup, Update, Draft).
- Adding public APIs for MTP state management (`llama_set_draft_input_hidden_state`).
- Adapting the embedding extraction logic to skip MTP update passes.

* Refactors `server_slot` to support generic speculative decoding (MTP or Draft Model).

* core: enable hybrid outputs (logits + embeddings) for MTP support

* fix(mtp): correct KV-cache slot finding for updates

* fix(mtp): persist hidden states to prevent context corruption during drafting

* refactor(mtp): clean unused code

* fix(mtp): update server to new functions name

* fix(mtp): fix graph and save hidden state

* mtp: refactor integration, context params and kv cache search

* mtp: fix hidden state extraction and speculative acceptance flow

* server: fix MTP warmup for long prompts and reset token buffer

* llama: refactor MTP operation state to context parameters

* server: fix n_past calculation in MTP acceptance

* llama: fix mtp enable flags

* speculative: refactor MTP to use common_speculative interface

* context: remove unused signatures

* clip: fix deprecated enum-enum conversion warning

* common: fix format string crash in help message

* context: fix mtp activation logic

* llamat: always use the extracted embedding

* llama: get all embeddings to kv cache

* llama: revert logit to not run mtp for not supported arch

* llama: allocate all the n_outputs for MTP

* wip

* server-context: get only the last embedding for hidden state

* ggml-backend: fix array of bounds in debug build

* server-context: run mt kv update to each prompt batch

* revert segmentation fault fixes

* glm-mtp(feat): optimize graph embedding and recursive drafting
2026-03-25 10:20:22 +01:00
..
CMakeLists.txt Factor out delta net (#1286) 2026-02-18 17:16:17 +01:00
llama-arch.cpp Mistral 4 support (#1450) 2026-03-18 07:32:39 +01:00
llama-arch.h Auto-fit offloaded tensors to available VRAM (MoE models) (#1501) 2026-03-25 07:29:29 +01:00
llama-build-context.cpp Improve mtp acceptance rate (#1499) 2026-03-25 10:20:22 +01:00
llama-build-context.h Improve mtp acceptance rate (#1499) 2026-03-25 10:20:22 +01:00
llama-context.h Improve mtp acceptance rate (#1499) 2026-03-25 10:20:22 +01:00
llama-cparams.h Bring back #1333 and #1335 (#1340) 2026-02-28 14:31:42 +01:00
llama-delta-net.cpp Qwen-3.5/Next tweaks (#1447) 2026-03-18 07:32:17 +01:00
llama-delta-net.h Full graph parallel for Qwen3.5 (dense and MoE) (#1388) 2026-03-10 09:08:24 +01:00
llama-grammar.cpp llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-grammar.h llama : add token matching support to llama-grammar (#1220) 2026-02-03 07:57:17 +02:00
llama-hparams.cpp Mistral 4 support (#1450) 2026-03-18 07:32:39 +01:00
llama-hparams.h Full graph parallel for Qwen3.5 (dense and MoE) (#1388) 2026-03-10 09:08:24 +01:00
llama-impl.h Full graph parallel for Qwen3.5 (dense and MoE) (#1388) 2026-03-10 09:08:24 +01:00
llama-load-tensors.cpp Cleaner log for adjusted splits (#1494) 2026-03-24 07:49:40 +01:00
llama-mmap.cpp Fix clang warnings on macOS (#1354) 2026-03-03 16:27:16 +01:00
llama-mmap.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
llama-model-loader.cpp Better --n-cpu-moe (#1464) 2026-03-19 06:57:01 +01:00
llama-model-loader.h Better --n-cpu-moe (#1464) 2026-03-19 06:57:01 +01:00
llama-model.cpp Auto-fit offloaded tensors to available VRAM (MoE models) (#1501) 2026-03-25 07:29:29 +01:00
llama-model.h Take into account layer sizes for setting GPU layers (cont) (#1476) 2026-03-23 17:46:53 +01:00
llama-quantize.cpp Better --n-cpu-moe (#1464) 2026-03-19 06:57:01 +01:00
llama-quantize.h Allow using -rtr and -muge together (#1444) 2026-03-16 18:26:26 +01:00
llama-sampling.cpp Adaptive P sampler: update review logic, delete old code comments, put prep stage after logit bias (#1386) 2026-03-14 12:34:12 +01:00
llama-sampling.h Adaptive P sampler: update review logic, delete old code comments, put prep stage after logit bias (#1386) 2026-03-14 12:34:12 +01:00
llama-vocab.cpp Qwen3.5-MoE support (#1288) 2026-02-21 08:33:06 +01:00
llama-vocab.h Qwen3.5-MoE support (#1288) 2026-02-21 08:33:06 +01:00
llama.cpp Improve mtp acceptance rate (#1499) 2026-03-25 10:20:22 +01:00
unicode-data.cpp Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode-data.h Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unicode.cpp Server: refactor and rename functions (#1151) 2026-01-18 08:16:57 +02:00
unicode.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00