ik_llama_opt

Commit Graph

Author	SHA1	Message	Date
Stephan Walter	e0a5b08cdc	ggml : generalize `quantize_fns` for simpler FP16 handling (#1237 ) * Generalize quantize_fns for simpler FP16 handling * Remove call to ggml_cuda_mul_mat_get_wsize * ci : disable FMA for mac os actions --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-07-05 19:13:06 +03:00
Borislav Stanimirov	96f634dfed	build : fix and ignore MSVC warnings (#1889 )	2023-06-16 21:23:53 +03:00
Kawrakow	684aeeb3e0	Adding a simple program to measure speed of dot products (#1041 ) On my Mac, the direct Q4_1 product is marginally slower (~69 vs ~55 us for Q4_0). The SIMD-ified ggml version is now almost 2X slower (~121 us). On a Ryzen 7950X CPU, the direct product for Q4_1 quantization is faster than the AVX2 implementation (~60 vs ~62 us). --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>	2023-04-18 19:00:14 +00:00

Author

SHA1

Message

Date

Stephan Walter

e0a5b08cdc

ggml : generalize `quantize_fns` for simpler FP16 handling (#1237 )

* Generalize quantize_fns for simpler FP16 handling

* Remove call to ggml_cuda_mul_mat_get_wsize

* ci : disable FMA for mac os actions

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

2023-07-05 19:13:06 +03:00

Borislav Stanimirov

96f634dfed

build : fix and ignore MSVC warnings (#1889 )

2023-06-16 21:23:53 +03:00

Kawrakow

684aeeb3e0

Adding a simple program to measure speed of dot products (#1041 )

On my Mac, the direct Q4_1 product is marginally slower
(~69 vs ~55 us for Q4_0). The SIMD-ified ggml version
is now almost 2X slower (~121 us).

On a Ryzen 7950X CPU, the direct product for Q4_1 quantization
is faster than the AVX2 implementation (~60 vs ~62 us).

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

2023-04-18 19:00:14 +00:00

3 Commits