Marvin
1dd0700988
Base: AtomicBot-ai/atomic-llama-cpp-turboquant @ cd5609390. IQK source: ikawrakow/ik_llama.cpp @ fe215a8c (ggml/src/iqk only). - GGML_IQK_MUL_MAT / GGML_IQK_FLASH_ATTENTION options (default OFF) - 57 IQK repacked types, blocks, traits; IQK hooks in ggml_compute_forward_mul_mat - ggml-cpu with IQK ON builds and links; IQK OFF build unaffected Assisted-by: opencode (Muse Spark) |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| README.md | ||
| build.sh | ||
| ls-sycl-device.cpp | ||
| run-llama2.sh | ||
| start-svr.sh | ||
| test.sh | ||
| update-ops-doc.sh | ||
| win-build-sycl.bat | ||
| win-run-llama2.bat | ||
| win-start-svr.bat | ||
| win-test.bat | ||
| win-update-ops-doc.bat | ||
README.md
llama.cpp/example/sycl
This example program provides the tools for llama.cpp for SYCL on Intel GPU.
Tool
| Tool Name | Function | Status |
|---|---|---|
| llama-ls-sycl-device | List all SYCL devices with ID, compute capability, max work group size, etc. | Support |
llama-ls-sycl-device
List all SYCL devices with ID, compute capability, max work group size, etc.
-
Build the llama.cpp for SYCL for the specified target (using GGML_SYCL_TARGET).
-
Enable oneAPI running environment (if GGML_SYCL_TARGET is set to INTEL -default-)
source /opt/intel/oneapi/setvars.sh
- Execute
./build/bin/llama-ls-sycl-device
Check the ID in startup log, like:
found 2 SYCL devices:
| | | | |Max | |Max |Global | |
| | | | |compute|Max work|sub |mem | |
|ID| Device Type| Name|Version|units |group |group|size | Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]| Intel Arc A770 Graphics| 1.3| 512| 1024| 32| 16225M| 1.3.29138|
| 1| [level_zero:gpu:1]| Intel UHD Graphics 750| 1.3| 32| 512| 32| 62631M| 1.3.29138|