2 Commits

Author	SHA1	Message	Date
Johannes Gäßler	cb7240ad05	CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K) (#7860)	2024-06-11 08:26:07 +02:00
Johannes Gäßler	fc5f0f1647	CUDA: use tensor cores for MMQ (#7676) ... * CUDA: int8 tensor cores for MMQ (legacy quants) * fix out-of-bounds writes * __builtin_assume -> GGML_CUDA_ASSUME * fix writeback returning too early	2024-06-10 11:45:13 +02:00