Kawrakow 4b35340f45 Bitnet changes (#106) ... * Adapting iq2_bn to work without separate scale tensors Why? It is becoming burdensome to maintain the special Bitnet conversion in convert_hf_to_gguf.py, so I thnk it is better to make iq1_bn and iq2_bn just work with the mainline conversion script (which does not generate scales). * Adapting iq1_bn to work without separate scale tensors * Adapting iq2_bn: CUDA dequantize * Adapting iq2_bn: CUDA works * Adapting iq1_bn: CUDA works * Adapting iq1_bn, iq2_bn: NEON * Adapting iq1_bn, iq2_bn: Metal Dequantize works, but there is still something wrong with the dot products. * WIP Absoolutely don't see what is wrong with the iq1_bn and iq2_bn vector dot product kernels. * Remove iq1_tn and iq2_tn - Part 1 Now that iq1_bn and iq2_bn have per row scales, there is no reason to also have iq1_tn and iq2_tn. * Remove iq1_tn and iq2_tn - Part 2 * Bitnet: use the standard llm_build_kv to build self attention My main motivation was to enable FA. But FA does not work anyway because head size is 100 for the Botnet ternary models (and I had forgotten this little detail). * Revert "Avoid rebuild of GGML graph for each token (#98)" This reverts commit f2d315b46f7aacc7df4b86bd8acba387b30e11ca. As far as I can tell, the commit breaks Metal TG. --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>		2024-10-25 13:08:43 +02:00
..
baby-llama	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
batched	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
batched-bench	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
batched.swift	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
benchmark	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
convert-llama2c-to-ggml	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
cvector-generator	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
deprecation-warning	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
embedding	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
eval-callback	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
export-lora	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
gbnf-validator	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
gguf	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
gguf-hash	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
gguf-split	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
gritlm	llama : allow pooled embeddings on any model (#7477)	2024-06-21 08:38:22 +03:00
imatrix	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
infill	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
jeopardy	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
llama-bench	Adding Q6_0 (#77)	2024-10-02 15:22:13 +03:00
llama.android	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
llama.swiftui	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
llava	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
lookahead	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
lookup	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
main	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
main-cmake-pkg	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
parallel	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
passkey	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
perplexity	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
quantize	Bitnet changes (#106)	2024-10-25 13:08:43 +02:00
quantize-stats	Adding IQ4_KSS: 4.0 bpw quants (#89)	2024-10-16 15:18:26 +03:00
retrieval	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
rpc	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
save-load-state	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
server	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
simple	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
speculative	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
sycl	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
tokenize	Merge mainline - Aug 12 2024 (#17)	2024-08-12 15:14:32 +02:00
CMakeLists.txt	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
Miku.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
base-translate.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
chat-13B.bat	Create chat-13B.bat (#592)	2023-03-29 20:21:09 +03:00
chat-13B.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
chat-persistent.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
chat-vicuna.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
chat.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
convert_legacy_llama.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
json_schema_pydantic_example.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
json_schema_to_grammar.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
llama.vim	llama.vim : added api key support (#5090)	2024-01-23 08:51:27 +02:00
llm.vim	llm.vim : stop generation at multiple linebreaks, bind to <F2> (#2879)	2023-08-30 09:50:55 +03:00
pydantic_models_to_grammar.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
pydantic_models_to_grammar_examples.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
reason-act.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
regex_to_grammar.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
server-llama2-13B.sh	`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)	2024-06-13 00:41:52 +01:00
server_embd.py	Merge mainline llama.cpp (#3)	2024-07-27 07:55:01 +02:00
ts-type-to-grammar.sh	JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555)	2024-04-12 19:43:38 +01:00