History

Marvin 1dd0700988 hybrid-llama: merge ik_llama IQK CPU GEMM into TurboQuant fork Base: AtomicBot-ai/atomic-llama-cpp-turboquant @ cd5609390. IQK source: ikawrakow/ik_llama.cpp @ fe215a8c (ggml/src/iqk only). - GGML_IQK_MUL_MAT / GGML_IQK_FLASH_ATTENTION options (default OFF) - 57 IQK repacked types, blocks, traits; IQK hooks in ggml_compute_forward_mul_mat - ggml-cpu with IQK ON builds and links; IQK OFF build unaffected Assisted-by: opencode (Muse Spark)		2026-09-05 18:09:49 -03:00
..
README.md	hybrid-llama: merge ik_llama IQK CPU GEMM into TurboQuant fork	2026-09-05 18:09:49 -03:00
llama-eval.py	hybrid-llama: merge ik_llama IQK CPU GEMM into TurboQuant fork	2026-09-05 18:09:49 -03:00
llama-server-simulator.py	hybrid-llama: merge ik_llama IQK CPU GEMM into TurboQuant fork	2026-09-05 18:09:49 -03:00
test-simulator.sh	hybrid-llama: merge ik_llama IQK CPU GEMM into TurboQuant fork	2026-09-05 18:09:49 -03:00

README.md

llama-eval

Simple evaluation tool for llama.cpp with support for multiple datasets.

For a full description, usage examples, and sample results, see:

PR 21152

Quick start

# Single server
python3 llama-eval.py \
  --server http://localhost:8033 \
  --model my-model \
  --dataset gsm8k --n_cases 100 \
  --grader-type regex --threads 32

# Multiple servers (comma-separated URLs and thread counts)
python3 llama-eval.py \
  --server http://server1:8033,http://server2:8033 \
  --server-name server1,server2 \
  --threads 16,16 \
  --dataset aime2025 --n_cases 240 \
  --grader-type regex