nicolas
/
ik_llama_opt
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ik_llama_opt/ggml/src/ggml-cuda
History
Kawrakow d95ac93027 Fix quantized K cache without FA (#680) 
* Prevent assert with quantized K cache and no FA

* Fix MMQ when running with quantized K cache without FA

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 		2025-08-08 13:51:14 +03:00
..
template-instances IQ4_KSS improvements (#642) 2025-07-23 20:50:57 +02:00
vendors Adding bf16 support to CUDA (#40) 2024-09-14 20:02:32 +03:00
acc.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
acc.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
arange.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
arange.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
argsort.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
argsort.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
binbcast.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
binbcast.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
clamp.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
clamp.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
common.cuh Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
concat.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
concat.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
conv-transpose-1d.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
conv-transpose-1d.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
convert.cu Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
convert.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
cp-async.cuh Faster DeepSeek FA on CUDA (#408) 2025-05-12 07:49:00 +03:00
cpy.cu Remove what appears to be unnecessary asserts in ggml_cuda_cpy (#560) 2025-06-27 17:44:36 +02:00
cpy.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
dequantize.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
diagmask.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
diagmask.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
dmmv.cu Trellis quants with CPU inference (#441) 2025-05-23 09:17:52 +03:00
dmmv.cuh Merge mainline - Aug 12 2024 (#17) 2024-08-12 15:14:32 +02:00
fattn-common.cuh CUDA: faster FA TG for GQA models (#370) 2025-05-04 09:17:44 +03:00
fattn-mma-f16.cuh CUDA: faster FA TG for GQA models (#370) 2025-05-04 09:17:44 +03:00
fattn-new-mma.cu Bug fixes from mainline (#439) 2025-05-20 17:03:14 +03:00
fattn-new-mma.cuh FlashMLA-3 for DeepSeek models on CUDA (#386) 2025-05-07 17:38:22 +03:00
fattn-tile-f16.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
fattn-tile-f16.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
fattn-tile-f32.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
fattn-tile-f32.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
fattn-vec-f16.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
fattn-vec-f32.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
fattn-wmma-f16.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
fattn.cu FlashMLA-3 for DeepSeek models on CUDA (#386) 2025-05-07 17:38:22 +03:00
fattn.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
getrows.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
getrows.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
im2col.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
im2col.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
iqk_cuda_common.h CUDA: Faster prompt processing for several quantization types (#595) 2025-07-10 09:27:28 +02:00
iqk_mmvq.cu Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
iqk_mmvq.cuh Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
mma.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
mma_new.cuh CUDA: faster FA TG for GQA models (#370) 2025-05-04 09:17:44 +03:00
mmq.cu Fix quantized K cache without FA (#680) 2025-08-08 13:51:14 +03:00
mmq.cuh IQ4_KSS improvements (#642) 2025-07-23 20:50:57 +02:00
mmvq.cu Adding IQ1_KT - 1.75 bpw SOTA quants (#616) 2025-07-20 10:05:23 +02:00
mmvq.cuh Trellis quants with CPU inference (#441) 2025-05-23 09:17:52 +03:00
norm.cu cuda: non-contiguous rms norm (#190) 2025-02-07 08:33:42 +02:00
norm.cuh Adding fused rms_norm (#42) 2024-09-08 10:19:21 +03:00
pad.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
pad.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
pool2d.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
pool2d.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
quantize.cu CUDA: small PP performance improvement for MoE models (#589) 2025-07-07 07:23:12 +02:00
quantize.cuh CUDA: small PP performance improvement for MoE models (#589) 2025-07-07 07:23:12 +02:00
rope.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
rope.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
scale.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
scale.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
softcap.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
softcap.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
softmax.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
softmax.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
sumrows.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
sumrows.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
tsembd.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
tsembd.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
unary.cu Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
unary.cuh Add copyright notices (#317) 2025-04-07 10:43:26 +02:00
upscale.cu Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
upscale.cuh Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
vecdotq.cuh CUDA: MMQ for IQ4_KS (#374) 2025-05-04 12:45:00 +03:00