ik_llama_opt/ggml/src/iqk
Kawrakow 86e33fd6f4
Initial Gemma4 support (#1581)
* Gemma4: WIP

* Gemma4: WIP - runs with totally wrong results

* Gemma4: WIP - add CPU 512, 512 FA

* Gemma4: WIP

It gives a meaningful response in llama-cli, but PPL is still much too
high. Is this due to tokenizer issues?

* Gemma4: this works

I had forgotten the softcap on the final output.

* Remove log

* Gemma4: WIP E4B/E2B

* Gemma4: Q4B/E2B appear to work now

* gemma4: tokenizer fixes
2026-04-06 10:01:08 +02:00
..
fa Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
iqk_common.h Even better Q4_0 KV cache (CPU) (#1556) 2026-03-31 08:08:34 +02:00
iqk_config.h Enable AVX-VNNI 256-bit path for Q4_K and Q5_K R4 matmul (#1446) 2026-03-17 10:44:55 +01:00
iqk_cpu_ops.cpp Log HAVE_FANCY_SIMD via LLAMA_LOG_INFO (#1492) 2026-03-23 08:43:29 +01:00
iqk_cpu_ops.h Log HAVE_FANCY_SIMD via LLAMA_LOG_INFO (#1492) 2026-03-23 08:43:29 +01:00
iqk_flash_attn.cpp mtmd: be able to use alternative types for the K*Q multiplication (#1567) 2026-04-02 08:04:05 +02:00
iqk_flash_impl.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
iqk_gemm_1bit.cpp Bonsai support (ARM_NEON) (#1571) 2026-04-03 08:29:15 +02:00
iqk_gemm_1bit.h Faster iq1_s GEMM via repacking to Q8_0_R8 (#517) 2025-06-11 15:01:34 +03:00
iqk_gemm_floats.cpp mtmd: be able to use alternative types for the K*Q multiplication (#1567) 2026-04-02 08:04:05 +02:00
iqk_gemm_floats.h Refactor iqk_mul_mat.cpp (#435) 2025-05-22 10:05:51 +03:00
iqk_gemm_iqk_quants.cpp AVX512+AVXVNNI GEMM implementation for quants using Q8_K for activations (#710) 2025-08-22 06:27:07 +03:00
iqk_gemm_iqk_quants.h Much faster CPU prompt processing (part 2) (#533) 2025-06-18 07:29:33 +03:00
iqk_gemm_iquants.cpp Enable AVX-VNNI 256-bit path for IQ3_XXS and IQ3_S R4 matmul (#1474) 2026-03-20 16:12:07 +01:00
iqk_gemm_iquants.h IQ2_XXS: much faster CPU prompt processing (#515) 2025-06-11 11:12:30 +03:00
iqk_gemm_kquants.cpp Enable AVX-VNNI 256-bit path for Q3_K R4 matmul (#1472) 2026-03-23 09:47:29 +01:00
iqk_gemm_kquants.h Faster CPU prompt processing for Q4_K and Q5_K (#525) 2025-06-13 07:58:15 +03:00
iqk_gemm_ktquants.cpp Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
iqk_gemm_ktquants.h Trellis quants: faster CPU prompt processing (#482) 2025-06-01 15:24:05 +03:00
iqk_gemm_legacy_quants.cpp Optimize mul_mat_q8_1_r8_q8_2 with AVX-512 for faster Q4_K/Q5_K prompt processing (#1578) 2026-04-06 09:07:23 +02:00
iqk_gemm_legacy_quants.h Much faster CPU prompt processing (part 3) (#534) 2025-06-18 15:30:56 +03:00
iqk_mul_mat.cpp Initial Gemma4 support (#1581) 2026-04-06 10:01:08 +02:00
iqk_mul_mat.h Qwen-3.5/Next tweaks (#1447) 2026-03-18 07:32:17 +01:00
iqk_quantize.cpp Bonsai support (AVX2, generic) (#1570) 2026-04-02 16:54:08 +02:00
iqk_quantize.h Bonsai support (AVX2, generic) (#1570) 2026-04-02 16:54:08 +02:00
iqk_utils.h Fix compilation on clang-cl.exe (#1355) 2026-03-04 08:00:28 +01:00