ik_llama_opt/ggml/src/iqk
Kawrakow 2616efa296
Fused delta net 2 (#1320)
* Revive fused delta-net

* Add command line argument for fused delta net

* Simplify/improve CUDA delta-net

* Add -fdn to llama-bench

* More CUDA fused delta net optimizations

* CPU optimizations

* Much faster fused delta-net on the CPU

It seems it is faster than the chunked implementation!

* Change meaning of fdn from bool flag to threshold value

* Use eps = 1e-6

* Give some nodes a name

* Don't re-apply L2 norm - it has already been done

* This seems quite a bit better

* More tweaks

* Restore per context buffer size log

Not everybody uses models split in 2000 parts, and those who do,
actually want to see the biffer sizes.
2026-02-26 06:53:43 +01:00
..
fa Better CPU FA thread strategy 2026-01-31 15:46:16 +00:00
iqk_common.h Unroll for loop for repacked BF16 MATMUL (#1047) 2025-12-08 06:09:45 +01:00
iqk_config.h Fix termux/android build (#336) 2025-04-21 09:13:46 +02:00
iqk_cpu_ops.cpp Faster adaptive_p sampling (#1165) 2026-01-19 16:03:09 +02:00
iqk_cpu_ops.h Faster adaptive_p sampling (#1165) 2026-01-19 16:03:09 +02:00
iqk_flash_attn.cpp Fix CPU FA work buffer size (#1216) 2026-02-02 12:39:41 +02:00
iqk_flash_impl.h Enable CUDA graphs for MoE models + GPT-OSS support (#689) 2025-08-15 09:18:07 +03:00
iqk_gemm_1bit.cpp AVX512+AVXVNNI GEMM implementation for quants using Q8_K for activations (#710) 2025-08-22 06:27:07 +03:00
iqk_gemm_1bit.h Faster iq1_s GEMM via repacking to Q8_0_R8 (#517) 2025-06-11 15:01:34 +03:00
iqk_gemm_floats.cpp Unroll for loop for repacked BF16 MATMUL (#1047) 2025-12-08 06:09:45 +01:00
iqk_gemm_floats.h Refactor iqk_mul_mat.cpp (#435) 2025-05-22 10:05:51 +03:00
iqk_gemm_iqk_quants.cpp AVX512+AVXVNNI GEMM implementation for quants using Q8_K for activations (#710) 2025-08-22 06:27:07 +03:00
iqk_gemm_iqk_quants.h Much faster CPU prompt processing (part 2) (#533) 2025-06-18 07:29:33 +03:00
iqk_gemm_iquants.cpp AVX512+AVXVNNI GEMM implementation for quants using Q8_K for activations (#710) 2025-08-22 06:27:07 +03:00
iqk_gemm_iquants.h IQ2_XXS: much faster CPU prompt processing (#515) 2025-06-11 11:12:30 +03:00
iqk_gemm_kquants.cpp AVX512+AVXVNNI GEMM implementation for quants using Q8_K for activations (#710) 2025-08-22 06:27:07 +03:00
iqk_gemm_kquants.h Faster CPU prompt processing for Q4_K and Q5_K (#525) 2025-06-13 07:58:15 +03:00
iqk_gemm_ktquants.cpp Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
iqk_gemm_ktquants.h Trellis quants: faster CPU prompt processing (#482) 2025-06-01 15:24:05 +03:00
iqk_gemm_legacy_quants.cpp Add missing AVX512 operators for MSVC (#948) 2025-11-14 06:58:51 +02:00
iqk_gemm_legacy_quants.h Much faster CPU prompt processing (part 3) (#534) 2025-06-18 15:30:56 +03:00
iqk_mul_mat.cpp Fused delta net 2 (#1320) 2026-02-26 06:53:43 +01:00
iqk_mul_mat.h Fused delta-net (#1315) 2026-02-25 14:12:48 +01:00
iqk_quantize.cpp Fix KT quantization yet again (#1321) 2026-02-25 18:07:12 +01:00
iqk_quantize.h Check for NaNs while loading the model. (#727) 2025-08-27 19:00:17 +03:00
iqk_utils.h Add missing AVX512 operators for MSVC (#948) 2025-11-14 06:58:51 +02:00