31018dc511
Adds ggml_latent_attn_prefix_ext / ggml_latent_attn_indexed_ext: MLA latent-cache attention with an always-visible learned K/V prefix (openPangu's 128 param_sink rows), joint softmax over [prefix | cache], reading the raw F32/F16/Q8_0 latent cache directly. CUDA implementation plus a scalar CPU reference that pins the op's semantics; the CPU backend reports support truthfully, and openPangu adopts the op only on a non-CPU backend as builder policy. openPangu routes its dense/SWA/MTP full-span attention and the gathered DSA path through the op, capability-gated per layer on the attention output projection's scheduled backend, with the latent cache required resident on that same backend (--no-kv-offload keeps the unfused chain); any layer whose backend cannot run the candidate keeps the exact unfused chain. Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| ggml-alloc.h | ||
| ggml-backend.h | ||
| ggml-cann.h | ||
| ggml-cpp.h | ||
| ggml-cuda.h | ||
| ggml-metal.h | ||
| ggml-rpc.h | ||
| ggml-sycl.h | ||
| ggml-vulkan.h | ||
| ggml.h | ||