Marvin
1dd0700988
Base: AtomicBot-ai/atomic-llama-cpp-turboquant @ cd5609390. IQK source: ikawrakow/ik_llama.cpp @ fe215a8c (ggml/src/iqk only). - GGML_IQK_MUL_MAT / GGML_IQK_FLASH_ATTENTION options (default OFF) - 57 IQK repacked types, blocks, traits; IQK hooks in ggml_compute_forward_mul_mat - ggml-cpu with IQK ON builds and links; IQK OFF build unaffected Assisted-by: opencode (Muse Spark) |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| README.md | ||
| export-lora.cpp | ||
README.md
export-lora
Apply LORA adapters to base model and export the resulting model.
usage: llama-export-lora [options]
options:
-m, --model FNAME model path from which to load base model
--lora FNAME path to LoRA adapter (use comma-separated values to load multiple adapters)
--lora-scaled FNAME:SCALE,... path to LoRA adapter with user defined scaling (format: FNAME:SCALE,...)
-o, --output, --output-file FNAME output file (default: 'ggml-lora-merged-f16.gguf')
For example:
./bin/llama-export-lora \
-m open-llama-3b-v2.gguf \
-o open-llama-3b-v2-english2tokipona-chat.gguf \
--lora lora-open-llama-3b-v2-english2tokipona-chat-LATEST.gguf
Multiple LORA adapters can be applied by passing comma-separated values to --lora FNAME or --lora-scaled FNAME:SCALE,...:
./bin/llama-export-lora \
-m your_base_model.gguf \
-o your_merged_model.gguf \
--lora-scaled lora_task_A.gguf:0.5,lora_task_B.gguf:0.5