* qwen4exp: MTP (NextN) self-speculative decoding support
Wires the checkpoint's 2.6B MTP head into the existing MTP speculative
framework for the qwen4exp architecture, following the deepseek4
conventions: block_count includes the NextN tail, the draft pass walks
[n_layer - nextn, n_layer) through the same layer-loop body, and the
target pass emits the pre-final-mixer wide stream as result_mtp_embd.
- hparams: read nextn_predict_layers; the tail is forced non-recurrent
(the full-attention-interval pattern would classify it as GDN) and
inherits the last main QSA layer's compress ratio
- loader: the tail layers and seven mtp.* entry/mixer tensors load only
when an MTP stage is requested (TENSOR_SKIP otherwise), so nextn-
bearing GGUFs serve normally on plain configs
- graph: MTP entry = per-reference residual_linear_shared fusion (joint
RMS over the flattened hc*n_embd stream, shared fc_hidden per branch,
fc_embedding added as a residual to every branch); exit reuses the
hc_head-shaped mixer with dedicated weights; delta-net construction
skips the draft pass (the draft context has no recurrent slots)
- llama: qwen4exp joins the MTP-capable arch allowlist and
llama_mtp_tail_uses_layer_cache; MTP feature width is the wide stream
(n_embd * hc_mult); the nextn pooling path gets its required
result_embd tensor
Measured on a 125B-A6B GGUF carrying the head (RTX 5090 + 24-core Zen5
hybrid, -ncmoe 38): draft acceptance 0.93-0.99, per-position 95%/pos on
code at n_max=4; draft passes cost ~2ms wall (131-144 nodes).
* cuda: dispatch the ABS and SGN unary ops
The kernels exist in unary.cu but neither the execution switch nor
supports_op routed them, so the qwen4exp PLE gate split to the CPU
scheduler twice per forward pass once its layer is offloaded (the cost
is the round trip, not the arithmetic — noted in the port's PR text).
* refactor(model): add qwen4exp NextN mixer tensors
* fix(qwen4exp): resolve standard NextN head layouts
* qwen4exp: decline per-step spec checkpoints (PLE conv tail not covered)
The per-step checkpoint path sizes its buffers from the ssm_* hparams
alone, but qwen4exp appends the PLE n-gram convolution history to the
tail of the same recurrent state row. A rejected draft token would roll
back the delta-net state and leave the PLE tail advanced. Decline
per-step (like openPangu) so the mode resolves to the whole-slot
gpu-fallback shadow, which covers the full row.
* qwen4exp: keep legacy mtp.* files loadable without MTP
Pre-rework files carried the NextN head under fork-local mtp.* names.
Register those seven names as known-and-skipped so such files still load
for non-MTP use instead of failing tensor accounting; an MTP request
against them reports the standard missing-layout error.
* loader: summarize skipped tensors in one line (per-tensor detail moves to debug)
An unused NextN/MTP head produces 33 ignore warnings per load; collapse
to a single summary at done_getting_tensors.
* spec: fall back to generic copy when checkpoint rows resolve no backend
Partial offload leaves host-resident recurrent state rows (CUDA_Host) matching
no backend's default buffer type between graphs, so checkpoint save/restore
asserted on a null backend. Drain the scheduler once and use the generic
tensor copy for unresolvable rows; the async fast path is unchanged. Also
guards the split-save branch that previously passed a null backend unchecked.
Diagnosed, patched, and validated by @joelfarthing (round-trip exact on
-ngl 6/8, -ncmoe control unchanged).
Co-authored-by: joelfarthing <joelfarthing@users.noreply.github.com>
* qwen4exp: support separate MTP head via -md (companion pair)
Lets a plain community quant load a predictor-only GGUF as the draft, so MTP
works without re-downloading a merged file and the head can sit on GPU at
full precision while the target runs partially offloaded. Relaxes the five
sites that assumed the target carries the NextN block: predict-layer count,
optional trunk tensors for mtp-only files, hc_attn_norm trunk marker,
n_embd_out derivation, and the wide-embd export predicate, plus qwen4exp
companion pairing enforcement.
Authored by @joelfarthing, replicated by @treo (3090, full 262K context).
Known limitation: llama-quantize on a predictor-only file trips the
attention-tensor count assertion; quantize before extraction.
Co-authored-by: joelfarthing <joelfarthing@users.noreply.github.com>
* server: name the stage chain in the single-slot refusal and say the fix
* qwen4exp: condense verbose NextN comments per CONTRIBUTING.md
* qwen4exp: accept the Dzannotti NextN head layout (predictor-only)
A predictor-only (-md) file may omit blk.N.nextn.shared_head_norm and carry the
head mixer norm only in output_hc_norm (the Dzannotti convention, with the +1
baked in as the direct-multiply grouped-RMS kernel expects). Fall back to
model.hc_head_norm as the head norm in that case, gated to mtp_only files where
output_hc_* is the head's own mixer and not the trunk's. Quimmedes-style raw
(non-baked) tensors remain intentionally unsupported.
Reported by @SamuelOliveirads.
* spec: merge the deepseek4/qwen4exp wide-stream feature-width branches
Both arches return n_embd_out under the same condition; one combined branch.
Suggested by @SamuelOliveirads.
---------
Co-authored-by: joelfarthing <joelfarthing@users.noreply.github.com>
563b798ac9