nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Ignore MTP layer(s) when computing required memory (#1516)

This commit is contained in:
Kawrakow 2026-03-26 10:42:51 +01:00 committed by GitHub
parent 9eaf105ae8
commit c06067fa14
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/llama.cpp
View File

@ -2064,6 +2064,9 @@ static std::pair<std::vector<double>, double> get_layer_sizes(const llama_model_
LLAMA_LOG_WARN("Oops: strange layer index %d for tensor %s\n", il, name.c_str());
continue;
}
if (!model.mtp && model.hparams.nextn_predict_layers > 0 && il >= n_layer - model.hparams.nextn_predict_layers) {
continue;
}
result[il] += size;
if (auto pos = name.rfind(".bias"); pos < name.size() && name.size() - pos == 4) {
// bias, we don't need to account for those