Cache worker
Development preview
This page describes a pinned development revision, not a published release. Its package version may still identify the preceding release while the next version is being prepared.
ngit-ci-cache-worker is the private Nix cache service used to move archive imports, signing, retention, HTTP serving and external publication away from provider signing credentials and decrypted allocation secrets. It never executes repository commands. Its Nix helpers still parse hostile job exports; run it under a dedicated account with a private filesystem view, resource limits and access only to cache-scoped credentials. Account separation alone is not protection against a host kernel exploit.
The provider connects through --cache-worker-socket and fails startup when the worker is unavailable. --managed-cache=false explicitly disables publication and inventory while retaining guest upstream reads. The NixOS module provisions the separate service automatically. Offerings using the same provider credential share one worker, store, namespace keys and retention budget. On NixOS their provider services also share one account, separate from the cache worker. See named NixOS tiers for cache ownership and configuration. An optional external-publication LaunchDaemon recipe is prepared in the Mac guide. The minimal macOS installation keeps managed-cache publication disabled; its VM and installer qualification does not qualify the optional cache service. The external-publication recipe still requires its own native qualification. Managed publication uses repository-scoped trust; external destinations remain operator-managed trust domains.
Configuration
Invoke ngit-ci-cache-worker --config /absolute/path/cache.json, or set NGIT_CI_CACHE_WORKER_CONFIG in its environment/private .env. The JSON file must be a regular file owned by root or the worker and not writable by group or others. Unknown fields are rejected. Never put credential values in JSON, argv, a Nix expression or the provider environment.
| Field | Meaning |
|---|---|
socket | Absolute Unix socket path in a worker-owned directory not writable by peers. Give the provider traversal through a dedicated group; the socket is mode 0660. |
state_dir | Absolute private worker-owned mode-0700 directory. Contains cache, incoming transfers and exclusive state lease. |
provider_user | Shared provider account; must resolve to a non-root UID distinct from the worker. |
nix_binary | Absolute trusted Nix executable path. |
max_export_bytes | Positive aggregate byte limit for one incoming binary-cache export. |
operation_timeout_secs | Positive whole-request deadline, including publication lock waiting. |
staging_min_free_bytes | Filesystem reserve required in addition to incoming bytes. |
managed | Managed-cache settings below, or null for external publication. |
external_destination | Uncredentialed file:// or s3:// destination, or null for managed publication. Exactly one destination mode is required. |
signing_key_file | Optional cache-only signing-key file accessible to the worker. Applies to external publication or the quarantined legacy store. Managed repository namespaces generate independent persistent keys. |
s3_credentials_file | Optional cache-only AWS shared-credentials file for external publication. |
managed contains listen (socket address), max_bytes, max_admission_bytes, min_free_bytes and check_interval_secs. The directory is <state_dir>/cache. Disk budgets remain best-effort accounting, not filesystem quotas. Reserve headroom for both provider staging and worker staging during transfer; the archive is copied across the account boundary. See disk sizing and pressure for the pool reserve, cache floor and simultaneous transfer budget.
Do not reuse a live provider cache directory by weakening its parent directory permissions. A migration must preserve the old cache, signing identity and retention-root paths until separately verified. The worker refuses simultaneous ownership of either its state or socket directory. It removes only its own interrupted incoming/export-* staging directories on startup; unexpected entries require operator review.
Cache helper commands clear their inherited environment and explicitly disable automatic EC2 metadata credentials. Use explicit cache-scoped S3 credentials. This SDK setting is not a network firewall or protection against a compromised helper; do not make unrelated control-plane credentials available through the worker's network or home.
Private protocol and lifecycle
This local protocol is independent of the Nostr provider protocol. Both peers check Unix peer credentials. Only the configured provider UID can issue requests; the client rejects a root or same-account worker. Socket-directory ownership and permissions are part of the trusted installation.
Each connection starts with a four-byte big-endian JSON frame length, bounded to 4 MiB. Version 2 operations are hello, repository-key, inventory and publish. Upgrade both peers together. Repository-key discovery returns only the selected namespace's public key, never a secret. Publication sends only a hashed repository scope, publication authority, store roots, a bounded file manifest and the exact file bytes. Full allocations, secret names and values, clone URLs, shell commands and provider filesystem paths never cross this boundary. The worker cannot authorize jobs or mint allocation grants.
Exports contain at most 16,384 files and 1,024 roots, with bounded path depth, length and aggregate bytes. Traversal paths, symlinks, special files and truncated transfers fail. Incoming files are created exclusively in a private staging directory. Cache metadata is checked before Nix parses archives. Job exports must use uncompressed NARs, with NarSize (and FileSize when present) matching the actual archive length; compressed, duplicate or understated metadata is rejected. This restriction applies to job publication spools, not upstream caches read inside the disposable guest. Nix still checks archive integrity. These checks bound transferred archive bytes, not filesystem inode consumption or the allocated size of unpacked small files. Inventory requests are asynchronous and bounded; failed inventory is a cache miss. Publication errors never cause a local Nix fallback.
At most eight connections and one publication run concurrently. Request timeouts include queue waiting. Client disconnection cancels publication and its helper process group. SIGTERM stops admission, drains bounded requests, then stops cache services; HTTP drain is bounded by the operation timeout plus ten seconds. Successful shutdown removes the socket. The worker must remain available until every attached offering has drained admitted jobs.
On NixOS the worker uses systemd readiness notification after opening the cache, recovering interrupted staging and binding its IPC/HTTP listeners. Provider startup waits for that notification, with a bounded startup deadline. A missing or failed worker still prevents provider startup; there is no direct fallback. The private filesystem view exposes only the systemd notification socket for this handshake, not the manager's control socket.
Qualification
Linux qualification exercises real separate UIDs, rejected unauthorized peers, signed shared publication, HTTP retrieval, and actual Nix reuse into an independent local store. It also checks mutual denial of provider and cache credential files, empty staging after completion, and graceful shutdown. Unit tests cover malformed frames/manifests, traversal and deep trees, truncation cleanup, private file permissions and interrupted staging recovery.
These results do not establish deployment confinement, macOS peer-credential behavior, launchd lifecycle or hostile kernel resistance. Those require the corresponding deployment and hardware acceptance checks before support claims.
Controlled Linux VM preparation uses a separate provider-wide signing namespace in the same managed worker. Its outputs can be reused across repositories; repository and temporary signatures never authorize imports into that namespace. See controlled build preparation for eligibility and the boundary between preparation and workflow execution.
Private controlled-build publication carries a retention class, and a bounded build-use IPC request promotes/refreshes already cached paths after preparation. The worker expands their registered closures; public readers cannot promote retention. See controlled output retention.
Optional public read service
managed.public_listen enables a separate read-only listener (default absent), with no private-session or upload routes. managed.trusted_keys lists approved upstream Nix keys; managed.trust_lock optionally adds authenticated pins from a public data-only lock file. The worker preserves verified upstream signatures instead of signing imported peer objects as locally built. Private runtime handoffs remain signed by the local worker, and cannot be fetched publicly.
NixOS wires these through managedCache.publicListen, cacheTrustedPublicKeys and cacheTrustLockFile; all offerings sharing a provider use the same trust policy. managedCache.publicUrl additionally starts the separate Nostr publisher, whose provider credential never enters the cache worker or a guest. See cache sharing for the complete wire and operator contract.
Delegated trust
Managed settings accept trust_lock (operator pins and optional delegated roots), trusted_keys (direct Nix keys), and trust_relays (one to eight relay URLs when delegation is enabled). NixOS supplies the cache owner's existing relays; these are worker JSON configuration fields under the existing --config interface. The worker persists trust-graph.json, checks expiry for new handoffs and reads, and refreshes in the background. No provider Nostr private key is loaded here. See cache sharing for the wire format, withdrawal semantics and standalone inspection commands.
Historical metadata and the independent replica resolver are described in nix-cache-replicas.md. The worker JSON's optional managed.replica_urls adds up to eight storage-only HTTPS origins (default empty); NixOS exposes managedCache.replicaUrls. Approved provider origins are automatic. The resolver uses admission-scoped trust snapshots and no archive staging.
Managed workers accept managed.upstream_caches for ordinary binary-cache fallback (default https://cache.nixos.org). The local-first preparation gateway contacts these only after a local miss and replica lookup. The NixOS module wires cacheSubstituters automatically; manual worker deployments must configure the same operator-approved URLs here. See cache lookup.