Adds support for the z-lab gpt-oss DFlash drafts (e.g. z-lab/gpt-oss-20b-DFlash),
which use a Qwen3 backbone trained with attention_bias=true.
- dflash-draft loader: create optional attention bias tensors bq/bk/bv/bo
(TENSOR_NOT_REQUIRED); the Qwen3.5/MiMo drafts have none and are unaffected.
- build_dflash: add those biases at the q/k/v/o projections (cross-context K/V
and the noise block), each guarded so bias-free drafts are unchanged.
- Narrow the DFlash graph contract validator to reject only fused qkv biases
(bqkv/bqk/bkv), which the graph still does not implement, and remove a dead
duplicate of the validator in llama.cpp (the live copy is in llama-dflash.cpp).
The gpt-oss tokenizer recognition needed to convert the draft is a separate,
general gpt-oss conversion fix submitted independently.
Verified on gpt-oss-20b: coherent output and 42% draft acceptance
(~2.95 accepted tokens/cycle) at cross_ctx=128.
Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com>
* DFlash: bound intra-block draft tokens to the SWA window
The SWA mask builder applied the sliding-window distance check only to the
cross-context section; the intra-block draft-token loop masked causal-only,
so a draft token could attend to earlier block tokens beyond n_swa. Apply
the same window bound ((j - block_k) < swa_window) in both the F16 and F32
paths so it matches the cross-context section.
Behavior-neutral for dense models: the SWA mask tensor is only allocated
when the model has SWA layers (build_dflash.cpp needs_swa_mask gate), so
for dense targets the changed block is unreachable.
* DFlash: enable sliding-window attention for draft models
DFlash drafts can be trained with sliding-window attention for long context,
but the runtime ignored it: the draft loader never read the window keys and the
converter never emitted them, so SWA-trained drafts always ran full-attention.
Enable it end to end and fix the dormant SWA graph path it exposes:
- convert_hf_to_gguf.py (DFlashDraftModel): emit attention.sliding_window + an
all-layers sliding_window_pattern when the source config sets use_sliding_window.
- llama-hparams.cpp (LLM_ARCH_DFLASH_DRAFT): read sliding_window + pattern into
n_swa / swa_layers.
- build_dflash.cpp + llama-dflash.cpp: the SWA mask path had never run; an all-SWA
draft turned the full kq_mask into a dead graph node the scheduler never backs
with a buffer, then the input-set wrote it unconditionally (GGML_ASSERT buf!=NULL).
Create + set each mask only when a layer uses it; derive mask dims from whichever
mask is live. Dense/mixed drafts are byte-identical.
Validated on gemma-4-26B-A4B at long context (cross_ctx 8176 > window 2048): no
crash, no short-context regression, SWA-on recovers long-context draft acceptance.
* DFlash: derive draft SWA pattern from layer_types
The converter emitted an all-layers SWA pattern ([True]*n_layers). The z-lab
DFlash drafts are sliding-window on every layer except a final full-attention
(global) layer, so this ran that global layer as sliding-window and clipped its
long-context view. Read layer_types and emit the matching per-layer pattern
(sliding_attention -> True), falling back to all-SWA only when layer_types is
absent.
---------
Co-authored-by: Joel Farthing <262452229+joelfarthing@users.noreply.github.com>
Joel Farthing
Kawrakow
Samuel Oliveira Alves
SamuelOliveirads