nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Slightly better graph split strategy (#1026) 

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow 2025-12-02 18:50:52 +01:00 committed by GitHub
parent a719349982
commit 92410bbd1e
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
src/llama-build-context.cpp
View File

@ -1133,7 +1133,10 @@ llm_expert_gating_func_type gating_op,
results[id] = shared_out;
}
cur = ggml_add(ctx, results[0], results[1]);
cur->op_params[0] = 0xff;
if (cur->ne[1] > 32) {
// Force a graph split
cur->op_params[0] = 0xff;
}
cb(cur, "ffn_shared_combined", il);
for (int id = 2; id < int(results.size()); ++id) {
cur = ggml_add(ctx, cur, results[id]);