docs: --cache-ram-similarity is a minimum, not a maximum (#2395)

Both the `-crs` help text and docs/parameters.md describe the parameter as a
"max of similarity ... that triggers prompt cache". It is the opposite: a
candidate cache entry is REJECTED when it falls below the value.

server_prompt_cache::load() takes it as `min_reusable_fraction` and applies it as

    const float f_keep_cur = float(lcp_cur.first) / tokens.size();
    if (f_keep_cur < min_reusable_fraction) {
        continue;
    }

so raising the value makes reuse stricter, while the current wording reads as
though raising it would admit more. The parameter name in the function signature
already says "min"; only the user-facing strings disagree.

Also states what the fraction is OF, which neither string did: the matched
prefix over the length of the CACHED entry, not over the new prompt.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mattafaak 2026-09-02 12:44:32 -04:00 committed by GitHub
parent 3c58ae373a
commit c2206b80da
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 2 additions and 2 deletions

View File

@ -3069,7 +3069,7 @@ void gpt_params_print_usage(int /*argc*/, char ** argv, const gpt_params & param
options.push_back({ "*", "-ctx-ckpt-t N, --ctx-checkpoints-tolerance N", "the number of tokens before the full prompt to create the checkpoint. (default: %d, <=0 disable)",params.ctx_checkpoints_tolerance}); options.push_back({ "*", "-ctx-ckpt-t N, --ctx-checkpoints-tolerance N", "the number of tokens before the full prompt to create the checkpoint. (default: %d, <=0 disable)",params.ctx_checkpoints_tolerance});
options.push_back({ "*", "-ctx-ckpt-e NAME, --ctx-checkpoints-eviction NAME", "Eviction strategy for checkpoint. Accepts fifo, variance and auto. Auto defaults to variance. Variance preserves coverage and maintains uniform interval. (default: variance)" }); options.push_back({ "*", "-ctx-ckpt-e NAME, --ctx-checkpoints-eviction NAME", "Eviction strategy for checkpoint. Accepts fifo, variance and auto. Auto defaults to variance. Variance preserves coverage and maintains uniform interval. (default: variance)" });
options.push_back({ "*", "-cram, --cache-ram N", "set the maximum cache size in MiB (default: %d, -1 - no limit, 0 - disable)",params.cache_ram_mib }); options.push_back({ "*", "-cram, --cache-ram N", "set the maximum cache size in MiB (default: %d, -1 - no limit, 0 - disable)",params.cache_ram_mib });
options.push_back({ "*", "-crs, --cache-ram-similarity N", "max of similarity of prompt tokens to cache tokens that triggers prompt cache (default: %.2f).",params.cache_ram_similarity }); options.push_back({ "*", "-crs, --cache-ram-similarity N", "minimum fraction of a cached entry that must match the new prompt for that entry to be reusable (default: %.2f).",params.cache_ram_similarity });
options.push_back({ "*", "-cram-n-min N, --cache-ram-n-min N", "minimum number of the cached tokens that triggers prompt cache (default: %d).", params.cache_ram_n_min }); options.push_back({ "*", "-cram-n-min N, --cache-ram-n-min N", "minimum number of the cached tokens that triggers prompt cache (default: %d).", params.cache_ram_n_min });
options.push_back({ "*", "-n, --predict N", "number of tokens to predict (default: %d, -1 = infinity, -2 = until context filled)", params.n_predict }); options.push_back({ "*", "-n, --predict N", "number of tokens to predict (default: %d, -1 = infinity, -2 = until context filled)", params.n_predict });
options.push_back({ "*", "-b, --batch-size N", "logical maximum batch size (default: %d)", params.n_batch }); options.push_back({ "*", "-b, --batch-size N", "logical maximum batch size (default: %d)", params.n_batch });

View File

@ -153,7 +153,7 @@ Note: When the available memory is very limited, turn this option off (`-cram 0`
| Parameter | Description | Default | Notes/Examples | | Parameter | Description | Default | Notes/Examples |
| - | - | - | - | | - | - | - | - |
| `-cram, --cache-ram N` | Set the maximum cache size in MiB | 8192 | -1 = no limit, 0 = disable Very useful when the variations of the same prompt are re-sent to the model (coding agents, etc.). [PR 954](https://github.com/ikawrakow/ik_llama.cpp/pull/954) | | `-cram, --cache-ram N` | Set the maximum cache size in MiB | 8192 | -1 = no limit, 0 = disable Very useful when the variations of the same prompt are re-sent to the model (coding agents, etc.). [PR 954](https://github.com/ikawrakow/ik_llama.cpp/pull/954) |
| `-crs, --cache-ram-similarity N` | Max similarity of prompt tokens to cache tokens that triggers prompt cache | 0.50 | | | `-crs, --cache-ram-similarity N` | Minimum fraction of a cached entry that must match the new prompt for that entry to be reusable. Candidates below this are skipped | 0.50 | |
| `-cram-n-min, --cache-ram-n-min N` | Minimum number of cached tokens that triggers prompt cache | 0 | | | `-cram-n-min, --cache-ram-n-min N` | Minimum number of cached tokens that triggers prompt cache | 0 | |
## Sampling ## Sampling