Incus adapter
Development preview
This page describes a pinned development revision, not a published release. Its package version may still identify the preceding release while the next version is being prepared.
ngit-ci-adapter-incus runs each job in a fresh Incus instance. It supports two operator-selected tiers:
- VM mode (
mode = "vm"): an Incus virtual machine. It advertiseshardware-vmisolation and is the default. - Container mode (
mode = "container"): an Incus system container. It shares the host kernel, advertisesweak-container, and requires the explicit--weak-containerizationacknowledgement. Use it only for authorized trusted code.
The adapter never falls back from VM mode to container mode. The tier is fixed at startup and advertised in the environment description used by the native Nix provider.
The adapter is a dumb daemon on a Unix domain socket speaking the Loom execution-adapter contract (execution-adapter-protocol.md): it keeps no queue, rejects immediately at capacity, and enforces no job timeout. The coordinator or native provider owns queueing and timeout enforcement. See execution-isolation.md for the isolation decision and its qualification status.
Job lifecycle
Per job the adapter
- writes a durable cleanup record in its state directory,
- launches an ephemeral Incus instance pinned to the local image's resolved fingerprint, with Incus-enforced
limits.cpu,limits.memoryand (in container mode)limits.processes, plus the deployment's configured profiles, network and storage pool, - runs
truethroughincus execuntil guest command execution is available, - starts the guest supervisor over
incus exec -T -- <agent> --stdioand streams the job request, stdin, environment and output through the existing guest transport, - deletes the instance with
incus delete -fand verifies that it is gone before returning capacity.
Every instance carries user.ngit-ci.adapter=<durable-label> and user.ngit-ci.mode=<vm|container>. The cleanup record is written before launch, and marks it confirmed only after incus launch succeeds. A failed, timed-out, cancelled or interrupted launch leaves an unresolved record: the daemon can continue creating an instance after its CLI exits. Such records quarantine the slot and block startup until operator recovery. For confirmed launches, startup removes recorded instances and other instances matching its own label in its configured Incus project. If deletion cannot be verified, the slot is quarantined rather than reused and the cleanup record is retained for the next preflight. Once polled, normal job teardown continues in an owned task even if the server drops its waiting future at the cleanup deadline. A failed delete also starts a bounded background retry (20 attempts) that reclaims the instance once an unresponsive daemon recovers; that never releases provider capacity, whose slot stays quarantined until the operator confirms cleanup (see docs/nix-provider.md).
No repository path becomes a host path. A job's command, arguments, stdin, environment and artifact stream travel through the guest supervisor; the adapter does not mount repository or coordinator directories into the instance. Network and profile policy remain the operator's responsibility.
The readiness probe does not wait for guest networking, Nix or Docker. Images whose services start asynchronously must arrange bounded readiness checks before jobs depend on them, for example in their guest-agent wrapper. This matters for cache access immediately after a container starts.
Recovery
An unresolved launch is deliberately not cleared by a single incus info "not found" response. Stop the adapter before reconciling it. In the configured Incus project, inspect incus operation list --format json and incus operation show <operation-id> for creation operations involving the recorded instance name. Wait for those operations to finish (or cancel them if Incus supports cancellation), then delete any resulting instance with incus delete -f <instance-name> and verify its absence. If the daemon's operation outcome is still uncertain, retain the record and keep the adapter stopped. Only after independently confirming that creation cannot complete later and that the instance is absent should you remove that instance's record from the state directory and restart the service. Do not delete the deployment lock file or clear the whole state directory.
This also applies to a crash after Incus accepted creation but before the adapter durably recorded a successful launch response. Confirmed launch records are recovered automatically; incomplete or unknown record contents fail closed.
Requirements
- Linux with a running, initialized Incus daemon (
incus admin init) and access to its control socket through theincus-admingroup. - A local Incus image already imported into the image store. The adapter resolves aliases and fingerprints with
incus image list, pins the immutable fingerprint, and refuses an image whose type does not match the requested mode or whose architecture does not match the host. It never pulls an image at launch time. - For VM mode, an Incus VM image with the Incus guest agent installed. The adapter uses
incus execfor readiness and for starting the guest supervisor. - The guest supervisor at the configured path. The defaults are
/run/current-system/sw/bin/ngit-ci-vsock-agentfor VM mode and/bin/ngit-ci-container-agentfor container mode. - Native Nix jobs need the guest supervisor, Nix,
git,bash, and the native provider's guest helpers (ngit-ci-nix-prepareandngit-ci-nix-job). The supervisor is the same one used by the QEMU and container adapters. The guest must provide writable Nix store/state space and a writable scratch directory for local builds. - For the coordinator's
socket-adapteract path, the guest additionally needsactand a Docker-compatible daemon. This is practical in a VM. Container mode does not setsecurity.nestingor privileged mode, so nested Docker and act are not provided by the default container image and are not a qualified use of this adapter.
Image preparation
The adapter does not build or download images. Prepare an image for the tier you will run, import it locally, and pin the alias in the service configuration.
VM image
A NixOS guest is the usual starting point. It must include the Incus agent and the guest supervisor, for example:
nix
{ ... }: {
# Import or copy the existing ngit-ci guest definition, then add the Incus
# agent. The guest supervisor and native job helpers stay in
# environment.systemPackages.
virtualisation.incus.agent.enable = true;
}Build the NixOS metadata and disk image, then import the split image. The NixOS Incus page documents the same metadata/qemuImage import shape:
bash
META=$(nix build .#nixosConfigurations.ngit-ci-incus-vm.config.system.build.metadata --print-out-paths)
DISK=$(nix build .#nixosConfigurations.ngit-ci-incus-vm.config.system.build.qemuImage --print-out-paths)
incus image import --alias ngit-ci-incus-vm \
"$META"/tarball/nixos-system-*.tar.xz \
"$DISK"/nixos.qcow2Adjust the boot loader and image configuration for your Incus VM firmware (Incus commonly boots VMs through UEFI). The existing nix/guest.nix module is a microVM guest definition, not a pre-qualified Incus VM image: reuse its package set and supervisor, but verify the boot path and Incus agent on the target host. An image that lacks the Incus agent fails readiness; an image without the configured supervisor fails the first job execution. Neither case falls back to host execution.
Container image
Container mode needs a root filesystem with the guest supervisor wrapper at /bin/ngit-ci-container-agent. Configure Nix in the image itself, for example with these NixOS settings (or equivalent /etc/nix/nix.conf entries):
nix
nix.settings = {
sandbox = false;
build-users-group = "";
experimental-features = [ "nix-command" "flakes" ];
};An exported NIX_CONFIG in the wrapper does not configure job processes: the supervisor clears its inherited environment before applying the job's explicit environment. The native provider separately supplies its job Nix configuration.
The wrapper must set a working PATH, create the writable directories it needs (including /nix/var/nix, /scratch, /tmp and /root), and then exec the supervisor:
sh
#!/bin/sh
set -eu
export PATH=/run/current-system/sw/bin:/bin
export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt
export SSL_CERT_FILE="$NIX_SSL_CERT_FILE"
mkdir -p /nix/var/nix /scratch /run /tmp /root
exec /run/current-system/sw/bin/ngit-ci-vsock-agent --stdioThe wrapper deliberately does not copy the OCI weak-container entrypoint's /sys/fs/cgroup equality checks. Incus applies the configured CPU, memory and PID limits through its own cgroup hierarchy; the adapter requests the same values in the instance configuration, but the in-container paths and values are not Docker's.
Import a split container image with incus image import:
bash
incus image import --alias ngit-ci-incus-container \
metadata.tar.xz rootfs.tar.xzA NixOS lxc-container image can supply the root filesystem; add the adapter package, Nix provider, nix, git, bash, certificates, and the wrapper above. Container mode is native-only by default. Do not add security.privileged=true or security.nesting=true to make arbitrary act/Docker workflows run; use VM mode for that model.
Configuration
The binary follows the repository precedence: CLI flag, then NGIT_CI_INCUS_* environment variable, then .env, then defaults.
| CLI | Environment | Default | Description |
|---|---|---|---|
--mode | NGIT_CI_INCUS_MODE | vm | vm for hardware isolation, or container for the explicitly weak shared-kernel tier. |
--weak-containerization | NGIT_CI_INCUS_WEAK_CONTAINERIZATION | false | Required with --mode container; rejected with --mode vm. |
--image | NGIT_CI_INCUS_IMAGE | required | Local Incus image alias or fingerprint. Resolved and pinned during preflight. |
--socket | NGIT_CI_INCUS_SOCKET | /run/ngit-ci-adapter-incus/adapter.sock | Adapter Unix socket. |
--state-dir | NGIT_CI_INCUS_STATE_DIR | /var/lib/ngit-ci-adapter-incus/jobs | Cleanup-record and identity directory. |
--incus-bin | NGIT_CI_INCUS_BIN | incus | Incus CLI binary. |
--project | NGIT_CI_INCUS_PROJECT | unset | Incus project for instances. |
--profile | NGIT_CI_INCUS_PROFILES | empty | Repeatable profile flag; environment value is comma-separated. |
--network | NGIT_CI_INCUS_NETWORK | unset | Network for the first NIC. |
--storage-pool | NGIT_CI_INCUS_STORAGE_POOL | unset | Storage pool for the root disk. |
--vcpus | NGIT_CI_INCUS_VCPUS | 2 | vCPUs per instance. |
--memory-mib | NGIT_CI_INCUS_MEMORY_MIB | 4096 | Memory per instance in MiB. Must be at least 256. |
--pids | NGIT_CI_INCUS_PIDS | 1024 | Process ceiling in container mode. |
--disk-gib | NGIT_CI_INCUS_DISK_GIB | unset | Optional root-disk size in GiB. |
--boot-timeout-secs | NGIT_CI_INCUS_BOOT_TIMEOUT_SECS | 120 | Maximum time to answer the incus exec readiness probe. |
--guest-agent | NGIT_CI_INCUS_GUEST_AGENT | mode-specific | Guest supervisor path. |
--max-jobs | NGIT_CI_INCUS_MAX_JOBS | 1 | Concurrent instances; further jobs are rejected immediately. |
--log-level | NGIT_CI_INCUS_LOG_LEVEL | info | Tracing filter. |
The advertised environment contains the pinned image fingerprint, the guest agent path, the mode and isolation tier, and the configured project, network, storage pool and profiles. The advertised resources are vcpus, memory-mib, pids-max in container mode, and disk-mib when a disk size is configured. The native provider's resource-admission: exact-values-v1 contract applies to these values.
NixOS module
The flake exports nixosModules.adapter-incus:
nix
{
services.ngit-ci-adapter-incus = {
enable = true;
mode = "vm";
image = "ngit-ci-incus-vm";
maxJobs = 2;
};
}The module enables local Incus and nftables by default, runs the adapter as a dedicated ngit-ci-adapter-incus service user in the incus-admin group, and creates the socket directory and state directory. Run incus admin init and import the image before starting the service.
For the explicit weak tier:
nix
{
services.ngit-ci-adapter-incus = {
enable = true;
mode = "container";
weakContainerization = true;
image = "ngit-ci-incus-container";
};
}This acknowledgement is separate from the coordinator's native execution authorization and secret consent. It only records that the adapter operator chose a shared-kernel sandbox; the coordinator and native provider still apply their own weak-mode gates.
For native Nix jobs, configure the provider's adapter socket; workflow discovery and trigger matching remain in the coordinator. Existing act deployments using this adapter should follow the migration guide.
Testing
Ordinary tests use a fake Incus CLI and run in test-core:
bash
cargo test -p ngit-ci-adapter-microvm --test incus_lifecycleThe real end-to-end test is ignored by default and selected by test-incus. It requires a running Incus daemon and a prepared local image with the guest supervisor and /bin/sh:
bash
NGIT_CI_TEST_INCUS_IMAGE=ngit-ci-incus-vm nix run .#test-incusFor container mode:
bash
NGIT_CI_TEST_INCUS_IMAGE=ngit-ci-incus-container \
NGIT_CI_TEST_INCUS_MODE=container \
NGIT_CI_TEST_INCUS_WEAK_CONTAINERIZATION=1 \
nix run .#test-incusOptional test variables are NGIT_CI_TEST_INCUS_BIN, NGIT_CI_TEST_INCUS_PROJECT, NGIT_CI_TEST_INCUS_PROFILES, NGIT_CI_TEST_INCUS_NETWORK, NGIT_CI_TEST_INCUS_STORAGE_POOL, NGIT_CI_TEST_INCUS_GUEST_AGENT and NGIT_CI_TEST_INCUS_BOOT_TIMEOUT_SECS. The test fails rather than skipping when a selected prerequisite is missing. It uses a fresh adapter label and cleans only its own labelled instances; do not run it against a production state directory.
Qualification record
On 2026-11-03, a disposable NixOS host running Incus 7.0.2 with nested KVM, dir storage and an IPv4 NAT bridge exercised operator-supplied NixOS images through the packaged adapter and its Unix-socket protocol. VM checks passed for HTTP binary-cache substitution, sandboxed Nix builds, stdout/stderr, artifact return, fresh writable state between jobs, execution cancellation, instance deletion and subsequent capacity reuse. A real act workflow ran in Docker inside the VM and returned its artifact. Cancellation during launch retained an unresolved record, blocked further execution and refused restart until explicit operator reconciliation. Explicit weak-container mode also passed cache substitution, unsandboxed Nix builds, artifact return, fresh writable state, execution cancellation, deletion and capacity reuse; it does not inherit the VM isolation claim.
This is a disposable lab smoke test, not production deployment acceptance. It does not qualify the coordinator/provider pipeline, managed-cache import or publication authority, hostile guest containment, disk exhaustion, arbitrary Incus profiles or production network policy. The cache fixture disabled signature verification for its own disposable test payload; it does not justify doing so in a deployment. Maintainer scripts and logs are retained as local review evidence, while the supported repeatable entry point remains test-incus above.
Limitations
- Beyond the disposable lab checks above, there is no recorded production deployment qualification for this adapter. VM mode makes the hardware-isolation claim only when the host, Incus storage, networking, firmware and guest image are configured and verified for it.
- Container mode is explicitly weak and shares the host kernel. It is not a substitute for VM mode and is never selected automatically.
- Images are operator-supplied. The adapter resolves the local image and checks its type and architecture, but does not build, pull, refresh or inspect guest contents; the configured supervisor path is trusted.
- Startup preflight recovers confirmed launch records and labelled instances in the configured Incus project. Unresolved launches require the operator recovery above; they are never treated as safe merely because an instance is currently absent. Unrelated instances are not touched.
- The durable label lives in the lock file beside the state directory. Changing
--state-dircreates a new identity and leaves old labelled instances behind; clean them up first. Deleting only the lock file changes the label, but cleanup records in the same state directory are still reconciled. - Changing
--projectalso hides old-project instances from preflight; clean those up in the old project before switching. - There is no warm pool. Every job pays the Incus launch and boot/readiness cost. Size
--max-jobsand the coordinator/provider concurrency together. - There is no adapter-managed persistent build cache or host cache mount. Each instance starts from the image's read-only base plus its ephemeral overlay; use an image with a pre-populated store or a binary cache. The provider's managed cache remains available through its configured guest-reachable HTTP URL. Arrange that reachability through the Incus network/profile and host firewall; a provider-host loopback address is not reachable inside an Incus instance.
- The adapter does not implement a separate host-side artifact bridge. Artifact files are streamed by the guest supervisor to the coordinator for both native and act jobs.
- The Incus daemon is part of the trusted computing base. The adapter user can control instances through the daemon's socket; do not expose that socket to job containers or unrelated users.
- Network policy, storage quotas and profile contents are operator-owned. The adapter passes the configured profile, network and storage names but does not create or validate those objects.