ik_llama_opt/ggml
Joel Farthing a0a10da5a9
CUDA: re-capture the graph when a CPY node's read-source address changes (#2136)
ggml_graph_node_has_matching_properties exempted every GGML_OP_CPY node from the source-address check, for both operands. The exemption is needed for KV-cache write copies, whose destination advances each step through the indirect-destination path. It also skips a CPY whose read source (src[0]) moves between graph replays while the surrounding subgraph stays shape-stable, so the captured kernel replays against a stale source pointer and reads the previous step's bytes.

Keep the exemption for the destination operand (src[1]) only, and compare a CPY's read source (src[0]) like any other node. Stable-source copies are unaffected and force no additional re-captures; a CPY whose read source genuinely moves now re-captures instead of reading stale memory.

Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com>
2026-07-17 14:06:33 +03:00
..
cmake Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
include New op: ggml_sum_rows_ext (#2132) 2026-07-15 16:21:17 +03:00
src CUDA: re-capture the graph when a CPY node's read-source address changes (#2136) 2026-07-17 14:06:33 +03:00
.gitignore Merge mainline llama.cpp (#3) 2024-07-27 07:55:01 +02:00
CMakeLists.txt Set GGML_AVXVNNI to OFF by default (#2116) 2026-07-12 08:26:38 +03:00