Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ADR-0007: Pull-through registry cache, bring-your-own by default with an optional managed zot

Status: accepted · Date: 2026-10-13 · Decision makers: James Hebden

Context and Problem Statement

Recreating a local cluster re-pulls every image, which is slow and hits registry rate limits. A host-side pull-through cache that outlives the cluster fixes both. The Go project’s docs/design/registry-cache.md recorded twelve numbered decisions about this feature, and its decision log recorded two more (“BYO-only” and “start colima on containerd”) that were still marked Proposed and in part contradicted the design document. This ADR records the state the code is in, so the contradiction has one resolution.

Decision Drivers

  • The cache has to survive cluster destroy and recreate
  • Caproni shouldn’t own a host service it can’t make reliable on macOS, Linux, and rootless Docker
  • Developers who already run a cache (zot, distribution, Harbor) should be able to point at it
  • k3s only honours mirror configuration through containerd, not through the dockerd shim colima uses by default

Considered Options

  1. Host-side cache; registry_mirror.managed: true runs and supervises one zot, otherwise the user supplies an endpoint, with managed defaulting to false
  2. Bring-your-own only, with no managed zot
  3. In-cluster cache (for example k3s embedded Spegel)
  4. Pre-import a fixed image set into k3s

Decision Outcome

Chosen option: “Host-side cache, bring-your-own by default, optional managed zot”, because it matches what the configuration reference documents (managed unset means bring your own) and what both the Go and Rust code implement. Option 3 resets with the cluster. Option 4 misses incidental pulls. Option 2 was the proposal in the Go project’s decision log (from gitlab-org/caproni!315); the managed-zot code path was kept in both implementations, so option 2 isn’t what shipped.

Managed mode details: one zot is shared across profiles with ref-count marker files, serialised by flock; up starts it, destroy releases the profile’s reference, and only registry stop stops it; it’s started detached with setsid, with a PID file and a log file. Drivers write the mirror config themselves: k3s registries.yaml for k3d, containerd hosts.toml inside the colima VM.

Consequences

Good:

  • Warm caches across destroy and up cycles
  • Developers with an existing cache aren’t forced onto a second one

Bad:

  • zot listens on 0.0.0.0:5555 and the generated config has no authentication section (TM-REG-001); whether anonymous writes are accepted is unproven
  • The PID file is parsed as i32 and signalled without a positivity or ownership check (TM-REG-002)
  • The k3d mirror configuration sets insecureSkipVerify: true (TM-REG-003)
  • The Go project’s decision log rows for this feature were stale; they left with the Go documentation (ADR-0017)

Neutral:

  • driver: external can’t use a mirror, because Caproni can’t rewrite an unmanaged cluster’s container runtime

Implementation Location

  • crates/xianyun-deploy/src/registrycache/mod.rs
  • crates/xianyun-deploy/src/registrycache/process.rs
  • crates/xianyun-deploy/src/registrycache/config.rs
  • crates/xianyun-cluster/src/driver/k3d.rs
  • crates/xianyun-cluster/src/driver/colima.rs

More Information

  • Retroactively documented on 2026-10-13 from the Go project’s registry-cache design (decisions 1 through 12) and configuration reference at the pinned upstream commit, and the merge history (ms-registry-cache-byo merged to main).
  • The original twelve decisions are carried by this ADR and by ADR-0013 through ADR-0016.
  • Related: FR-011, SR-012.