From a0a10da5a99c7138d866079bdf691ea2c325e563 Mon Sep 17 00:00:00 2001 From: Joel Farthing Date: Fri, 17 Jul 2026 06:06:33 -0500 Subject: [PATCH] CUDA: re-capture the graph when a CPY node's read-source address changes (#2136) ggml_graph_node_has_matching_properties exempted every GGML_OP_CPY node from the source-address check, for both operands. The exemption is needed for KV-cache write copies, whose destination advances each step through the indirect-destination path. It also skips a CPY whose read source (src[0]) moves between graph replays while the surrounding subgraph stays shape-stable, so the captured kernel replays against a stale source pointer and reads the previous step's bytes. Keep the exemption for the destination operand (src[1]) only, and compare a CPY's read source (src[0]) like any other node. Stable-source copies are unaffected and force no additional re-captures; a CPY whose read source genuinely moves now re-captures instead of reading stale memory. Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com> --- ggml/src/ggml-cuda.cu | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ggml/src/ggml-cuda.cu b/ggml/src/ggml-cuda.cu index f11229b4..7e66572e 100644 --- a/ggml/src/ggml-cuda.cu +++ b/ggml/src/ggml-cuda.cu @@ -4462,8 +4462,8 @@ static bool ggml_graph_node_has_matching_properties(ggml_tensor * node, ggml_gra for (int i = 0; i < GGML_MAX_SRC; i++) { if (node->src[i] && node->src[i]->data != graph_node_properties->src_address[i] && - node->op != GGML_OP_CPY && - node->op != GGML_OP_VIEW + node->op != GGML_OP_VIEW && + !(node->op == GGML_OP_CPY && i == 1) ) { return false; }