nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix KLD precision (#325) 

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow 2025-04-12 16:17:50 +02:00 committed by GitHub
parent a3b16affaf
commit 65fc77c285
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/perplexity/perplexity.cpp
View File

@ -133,7 +133,7 @@ static double log_softmax(int n_vocab, const float * logits, uint16_t * log_prob
max_logit = std::max(max_logit, logits[i]);
min_logit = std::min(min_logit, logits[i]);
}
min_logit = std::max(min_logit, max_logit - 16);
min_logit = std::max(min_logit, max_logit - 24);
double sum_exp = 0.0;
for (int i = 0; i < n_vocab; ++i) {
sum_exp += expf(logits[i] - max_logit);