nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Reduce the indexer temporary buffer size (#2271)

This commit is contained in:
Kawrakow 2026-08-07 17:09:18 +03:00 committed by GitHub
parent 36925b86c1
commit 61b37f5f44
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ggml/src/ggml-cuda/indexer_topk.cu
View File

@ -146,7 +146,7 @@ void ggml_cuda_op_indexer_topk(ggml_backend_cuda_context & ctx, ggml_tensor * ds
}
constexpr int64_t k_max_work_buffer_elements = 1 << 28;
constexpr int64_t k_max_work_buffer_elements = 1 << 26;
int max_rows = k_max_work_buffer_elements / n_kv / n_head;
if (max_rows < 1) max_rows = 1;