docs: canonical Ampere+Zen3 build and corrected flags
This commit is contained in:
parent
5d028a7855
commit
bc75f9e602
|
|
@ -92,14 +92,19 @@
|
|||
|
||||
## Validation protocol (every item)
|
||||
|
||||
1. **Build (glove-fit):**
|
||||
1. **Build (glove-fit, canonical):**
|
||||
```
|
||||
cmake -B build -DGGML_NATIVE=ON -DGGML_CUDA=ON \
|
||||
rm -rf build-ampere-zen3
|
||||
cmake -S . -B build-ampere-zen3 \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DGGML_NATIVE=ON \
|
||||
-DGGML_CUDA=ON \
|
||||
-DCMAKE_CUDA_ARCHITECTURES="86-real" \
|
||||
-DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_IQK_MUL_MAT=ON \
|
||||
-DCMAKE_CXX_FLAGS="-march=znver3"
|
||||
cmake --build build --config Release -j$(nproc)
|
||||
-DGGML_CUDA_FA_ALL_QUANTS=ON \
|
||||
-DGGML_IQK_MUL_MAT=ON
|
||||
cmake --build build-ampere-zen3 --config Release -j$(nproc)
|
||||
```
|
||||
Do not add `GGML_CPU_ALL_VARIANTS` / `GGML_CPU_ARM_ARCH` (nonexistent), manual `-march` on top of `GGML_NATIVE=ON`, or raw `-gencode` flags.
|
||||
2. **Harnesses:** `build/bin/test-delta-chunk` → `PASS 42/42`; `build/bin/test-delta-chunk --cuda` → `PASS 42/42`.
|
||||
3. **Micro:** `build/bin/test-moe-perf` before/after (≥3 runs, take median).
|
||||
4. **Server A/B:** `llama-bench -m <model> -p 4096 -n 0 -ts 4/1` for PP, `-p 128 -n 128` for TG; then the real 25k session flags. Accept: PP +2% or TG no-regression; revert anything else.
|
||||
|
|
|
|||
63
README.md
63
README.md
|
|
@ -27,14 +27,67 @@ Target rig: RTX 3090 24GB + RTX 3070 8GB (`sm_86`, no P2P) + Ryzen 5900XT (`znve
|
|||
- Chunked-WY CUDA vs sequential: parity — merged, default-off.
|
||||
- QSA sparse-gather and HC fusion evaluated and deferred (needs a new kernel; <0.5% expected gain at real KV lengths).
|
||||
|
||||
### Recommended build for this rig
|
||||
### Canonical glove-fit build for this exact rig
|
||||
|
||||
Use this command. It is the correct one. The older longer command contained redundant defaults plus two invalid options (`GGML_CPU_ALL_VARIANTS`, `GGML_CPU_ARM_ARCH`) that do not exist anywhere in this tree.
|
||||
|
||||
Prerequisites (Debian/Ubuntu), plus an NVIDIA driver and CUDA toolkit:
|
||||
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y build-essential git cmake libcurl4-openssl-dev curl libgomp1
|
||||
```
|
||||
cmake -B build -DGGML_NATIVE=ON -DGGML_CUDA=ON \
|
||||
|
||||
The CUDA path needs CMake 3.18+. CUDA 12.8+ is needed only for the optional `-compress-mode=size` flag below.
|
||||
|
||||
From a clean tree, always with a fresh build directory:
|
||||
|
||||
```bash
|
||||
rm -rf build-ampere-zen3
|
||||
cmake -S . -B build-ampere-zen3 \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DGGML_NATIVE=ON \
|
||||
-DGGML_CUDA=ON \
|
||||
-DCMAKE_CUDA_ARCHITECTURES="86-real" \
|
||||
-DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_IQK_MUL_MAT=ON \
|
||||
-DGGML_CPU_ALL_VARIANTS=OFF -DGGML_CPU_ARM_ARCH=OFF
|
||||
-DGGML_CUDA_FA_ALL_QUANTS=ON \
|
||||
-DGGML_IQK_MUL_MAT=ON
|
||||
cmake --build build-ampere-zen3 --config Release -j"$(nproc)"
|
||||
```
|
||||
plus `-DCMAKE_CXX_FLAGS="-march=znver3"` / CUDA `-gencode arch=compute_86,code=sm_86` for a glove-fit non-portable binary. See `docs/build.md` for details.
|
||||
|
||||
Why these and only these:
|
||||
|
||||
- `-DCMAKE_BUILD_TYPE=Release`: performance build.
|
||||
- `-DGGML_NATIVE=ON` (`ggml/CMakeLists.txt:57`): GCC gets `-march=native`, which resolves to `znver3` on the 5900XT (AVX2+FMA+F16C, no AVX512). Do not stack a manual `-march=...` on top.
|
||||
- `-DGGML_CUDA=ON` (`ggml/CMakeLists.txt:117`).
|
||||
- `-DCMAKE_CUDA_ARCHITECTURES="86-real"`: device code as SM 8.6 SASS only, no PTX/fat binaries. Explicit because both GPUs are `sm_86`.
|
||||
- `-DGGML_CUDA_FA_ALL_QUANTS=ON` (`ggml/CMakeLists.txt:134`, `docs/build.md:392`): compiles all KV-cache quant combinations for FlashAttention; needed for `-ctk q8_0 -ctv q4_0`. Greatly increases CUDA compile time.
|
||||
- `-DGGML_IQK_MUL_MAT=ON` (`ggml/CMakeLists.txt:115`, default ON): optimized IQK CPU GEMM used by the CPU expert path; explicit so it can never silently turn off.
|
||||
|
||||
This produces `build-ampere-zen3/bin/llama-server`, `llama-bench`, `llama-cli`, `test-delta-chunk`, and `test-moe-perf`.
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
./build-ampere-zen3/bin/test-delta-chunk
|
||||
./build-ampere-zen3/bin/test-delta-chunk --cuda
|
||||
./build-ampere-zen3/bin/llama-server --version
|
||||
```
|
||||
|
||||
Expect `PASS 42/42` from the harness on both CPU and CUDA.
|
||||
|
||||
Optional explicit lock-down (every flag below is real, but redundant on a fresh build dir for this rig — same binary):
|
||||
|
||||
```bash
|
||||
-DGGML_AVX512=OFF -DGGML_AVX512_VBMI=OFF -DGGML_AVX512_VNNI=OFF -DGGML_AVX512_BF16=OFF -DGGML_AVXVNNI=OFF \
|
||||
-DGGML_OPENMP=ON -DGGML_NCCL=ON -DGGML_CUDA_USE_GRAPHS=ON -DGGML_SCHED_MAX_COPIES=1 \
|
||||
-DGGML_CUDA_COMPRESSION_MODE=size # CUDA 12.8+ only; smaller binary, no speed effect
|
||||
```
|
||||
|
||||
Do NOT use:
|
||||
|
||||
- `GGML_CPU_ALL_VARIANTS` / `GGML_CPU_ARM_ARCH`: nonexistent options; CMake reports them as manually-specified variables not used by the project.
|
||||
- Manual `-DCMAKE_CXX_FLAGS="-march=znver3"` together with `GGML_NATIVE=ON`: redundant, and CXX-only so it misses the C kernels.
|
||||
- Raw `-gencode arch=compute_86,code=sm_86`: let `CMAKE_CUDA_ARCHITECTURES` handle device code.
|
||||
|
||||
## TL;DR
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue