# TurboQuant fork — process & infrastructure

This is the `atomic-llama-cpp-turboquant` fork of [llama.cpp](https://github.com/ggml-org/llama.cpp)
used as the primary inference backend of Atomic Chat. This document describes
the branch model, the dev/staging channel, stable releases and the upstream
sync procedure. For what the fork changes technically (TurboQuant KV cache,
custom quant types, Inkling arch, merge history) see `MERGE_NOTES.md`.

## Branch model

| Branch | Role | Rules |
|---|---|---|
| `master` | **Stable.** What Atomic Chat ships. | Changes arrive only via PR from `dev`. Releases are tagged here. |
| `dev` | **Staging.** Feature/fix integration, no strict stability promise. | PRs land here first. Every push builds all platforms and republishes the rolling `dev-latest` prerelease. |
| `upstream` | **Pure mirror** of `ggml-org/llama.cpp` `master`. | Fast-forward only, never contains fork commits. Used as the merge source for upstream syncs. |
| `legacy/master-2025` | Archive of the pre-2026 `master` tip (`24cabf4d0`). | Frozen. |
| `feature/turboquant-kv-cache` | Former trunk, kept as an alias during the transition. | Do not push; will be deleted eventually. |

## Dev channel (staging binaries)

Workflow: `.github/workflows/dev-build.yml`.

Every push to `dev` builds all eleven archives and republishes the
[`dev-latest`](https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant/releases/tag/dev-latest)
rolling prerelease with them. PRs into `dev` — and into `master`, so the
promotion PR reports the same checks — build everything but publish nothing.
If some backend fails, `dev-latest` is still published with the survivors and
the notes list what is missing.

| Platform | Archives |
|---|---|
| Linux x64 | `cpu`, `vulkan`, `cuda-12.4`, `cuda-13.3`, `rocm` |
| Linux arm64 | `cuda-13.3` — NVIDIA DGX Spark / GB10, sm_121 |
| Windows x64 | `cpu`, `vulkan`, `cuda-12.4`, `cuda-13.3` |
| macOS arm64 | `macos-arm64` (Metal) |

The CUDA archives bundle their own `libcudart`/`libcublas` and are linked with
an `$ORIGIN` RPATH, so they do not need a CUDA toolkit on the target machine —
only a recent enough driver. The Linux arm64 archive uses the arm64-SBSA CUDA
build; DGX Spark needs driver r580+ for the bundled CUDA 13.3 runtime.

Grab-and-test on any machine:

```bash
gh release download dev-latest -R AtomicBot-ai/atomic-llama-cpp-turboquant \
  -p 'llama-turboquant-linux-x64-vulkan.tar.gz'   # or your platform
tar -xzf llama-turboquant-linux-x64-vulkan.tar.gz
./build/bin/llama-server --version    # → version: turboquant-vX.Y.Z (<count>, <sha>)
```

macOS dev builds are signed but **not notarized** (release builds are):
`xattr -dr com.apple.quarantine build/` after unpacking.

## Versioning & stable releases

Version format: **`<upstream-base>-<fork-semver>`**, e.g. `b10018-1.2.0`:

- `b10018` — the upstream llama.cpp build the fork is based on
  (`git rev-list --count $(git merge-base master upstream)`, matching
  upstream's `b####` release tags). Changes only on upstream syncs; the sync
  PR updates it in `TURBOQUANT_VERSION` by hand.
- `1.2.0` — the fork's own semver: **major** for breaking changes, **minor**
  for features (e.g. implementing turbo-ops for a new backend), **patch**
  for fixes.

Single source of truth: the `TURBOQUANT_VERSION` file at the repo root.
CMake embeds it via `common/build-info.cpp.in`; `llama-server --version`
prints `version: b10018-1.2.0 (build <count>, commit <sha>)`. Note that
`llama_build_info()` (the OpenAI API `system_fingerprint`) intentionally
keeps the upstream `b<N>-<sha>` format — clients parse it.

Cut a release (from an up-to-date, clean `master` checkout):

```bash
# 1. Write the CHANGELOG section for the version you are about to cut,
#    commit it. `verify-version` refuses the release without it.
# 2. Then:
./scripts/turboquant-release.sh patch|minor|major|X.Y.Z
```

This bumps the fork-semver part, commits `release: b10018-X.Y.Z`, tags
`b10018-X.Y.Z` and pushes. The tag triggers
`.github/workflows/release-turboquant.yml`: all backends are built (macOS
fully notarized) and published as **one** GitHub release with all archives.
`verify-version` fails the release — in seconds, before three hours of
building — if the tag doesn't match `TURBOQUANT_VERSION` or if `CHANGELOG.md`
has no `## <tag>` section.

### Release notes

`CHANGELOG.md` has one section per tag, and the release notes are generated
from it verbatim: that section *is* the release announcement, so write it for
whoever downloads the build — what they get, what changed for them, what to
watch out for. Everything mechanical (the asset table, the full commit list
since the previous tag, versioning boilerplate) the workflow adds by itself,
collapsed below the fold. Do not paste a commit dump into the changelog; the
notes already carry one.

Consumers: `atomic-chat-conf/backends/turboquant-manifest.json` entries all
point at the same `b10018-X.Y.Z` tag; asset names
(`llama-turboquant-<backend>.zip|tar.gz`) are unchanged from the legacy
scheme, so the Atomic-Chat runtime URL builder needs no changes.

Legacy per-platform releases (`turboquant-<platform>-<sha>`) are kept for
old app versions; do not delete them.

## Upstream sync procedure

Small regular syncs instead of 130k-line big bangs:

```bash
# 1. Advance the mirror (fast-forward only — zero conflicts by definition)
git fetch upstream            # remote 'upstream' = https://github.com/ggml-org/llama.cpp.git
git push origin upstream/master:refs/heads/upstream

# 2. Merge into a sync branch off dev
git checkout -b sync/upstream-$(date +%Y-%m-%d) origin/dev
git merge origin/upstream     # resolve conflicts HERE, in the sync branch

# 3. PR the sync branch into dev → CI builds every platform
# 4. Test via dev-latest, then PR dev → master as usual
```

`git merge-base origin/master origin/upstream` always tells you exactly which
upstream commit the fork is based on.

Conflict hot-spots (see `MERGE_NOTES.md` for history): `ggml-cuda.cu`/`fattn.cu`,
`ggml-vulkan.cpp` (SET_ROWS/supports_op), `ggml-metal.metal` kernel naming,
`llama-kv-cache.cpp`, `gguf-py/gguf/constants.py` (quant type ids — the fork
renumbered `Q2_0` to 47; upstream Q2_0 GGUFs are incompatible).

## Known constraints

- Vulkan: turbo3 flash-attn SPIR-V and banded-FA/lightning-indexer kernels are
  not implemented; those ops are rejected via `supports_op` (turbo KV cache
  falls back off on Vulkan). TURBO_WHT / turbo set_rows / GATED_DELTA_NET
  Vulkan kernels DO exist.
- Inkling: no MTP/NextN support yet (heads in GGUF are ignored); the fork's
  MTP subsystem currently serves qwen35/step35/hy-v3. Planned work.
- Upstream removed `-sm row` (CUDA multi-GPU split-buffer) — gone since the
  inkling merge.
- CUDA-11: no TurboQuant build; the app maps such GPUs to the CPU backend.