nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

server: accept max_completion_tokens as alias for max_tokens (#2321) 

OpenAI-compatible clients (e.g. pi coding agent) send
max_completion_tokens for the output token cap on custom
openai-completions providers. The server only read n_predict and
max_tokens, so the cap was silently dropped and n_predict fell back
to -1 (unlimited). This allowed runaway generations of 40k+ tokens
on long agent sessions.

Matches upstream llama.cpp behavior where max_completion_tokens is an
alias of n_predict (tools/server/server-schema.cpp).
This commit is contained in:
Jun Yamog 2026-08-15 19:40:07 +12:00 committed by GitHub
parent 85a784505d
commit 8e703ddd8a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/server/server-context.cpp
View File

@ -1125,7 +1125,7 @@ bool server_context::launch_slot_with_task(server_slot& slot, server_task& task)
auto stream_opt = json_value(data, "stream_options", json::object());
slot.params.include_usage = json_value(stream_opt, "include_usage", false);
slot.params.cache_prompt = json_value(data, "cache_prompt", true);
slot.params.n_predict = json_value(data, "n_predict", json_value(data, "max_tokens", defaults.n_predict));
slot.params.n_predict = json_value(data, "n_predict", json_value(data, "max_tokens", json_value(data, "max_completion_tokens", defaults.n_predict)));
slot.saturate_predict = json_value(data, "saturate_predict", false);
slot.sparams.top_k = json_value(data, "top_k", default_sparams.top_k);
slot.sparams.top_p = json_value(data, "top_p", default_sparams.top_p);