2b8d0d5011
The Metal backend lacked GGML_ROPE_TYPE_MROPE/IMROPE support, so models whose GGUF carries rope_sections (e.g. Qwen 3.5 hybrids) could not run fully offloaded on Apple Silicon. Add rope_multi_f32/f16 kernels with section-based position handling (t/h/w/e blocks, 4 position ids per token), imrope's interleaved section selection, and the corresponding dispatch in ggml-metal.m. Vision-mode mrope is not implemented and is asserted out explicitly. Validated on M2: kernel output matches the CPU backend, and Qwen 3.5-9B (Q4_K_M, -ngl 99) WikiText-2 perplexity over 145 chunks lands within 0.006 of the same model's CPU baseline. |
||
|---|---|---|
| .. | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||