Run the microVM adapter

Owned by ngit-ciVerified 0.1.0-rc.2 @ aad99deb773dimported guide

ngit-ci-adapter-microvm (workspace crate crates/adapter-microvm) is the first-party sandbox adapter for the coordinator's socket-adapter runner backend: every job runs in a fresh QEMU/KVM microVM that is destroyed afterwards. Use it when watched repositories are not fully trusted — the VM boundary, not a container, is what contains a hostile job.

It is a dumb execution daemon on a Unix domain socket speaking the Loom execution-adapter contract (execution-adapter-protocol.md): no queue (it rejects immediately when all --max-jobs slots are busy) and no timeouts. Queueing and timeout enforcement stay in the coordinator. Its only optional cross-connection state is opaque, coordinator-partitioned act cache storage; it never derives repository authority itself.

Job lifecycle

Per job the adapter

1. creates an instant qcow2 overlay backed by a read-only template image (qemu-img create -b template.qcow2 — copy-on-write, O(1) on any filesystem),
2. boots a QEMU/KVM guest from the overlay with user-mode (slirp) networking only — outbound-only network, no general host mounts, and hypervisor-enforced vCPU/memory ceilings from adapter config. When the coordinator enables build caching, QEMU exposes exactly one selected repository/trust-scope directory over 9p,
3. runs the job over SSH on a loopback-forwarded port, authenticated by an ephemeral per-job keypair injected via an SMBIOS systemd credential, streaming stdin/stdout/stderr through faithfully,
4. destroys the VM and deletes the overlay on completion — or immediately when the coordinator disconnects (job timeout). No disposable job state survives; on startup the adapter also kills and removes any leftover VM state from a previous crash. Trust-scoped caches beneath the adapter state directory deliberately survive; no overlay or other guest state does.

There is no warm pool in v1, so each job pays a VM boot (roughly 10–20 s with KVM) — negligible against typical CI runtimes.

Requirements

On the adapter host: /dev/kvm (the adapter refuses to start without it, with setup instructions), qemu-system-x86_64 + qemu-img, ssh + ssh-keygen, and a guest template image (below). The coordinator can run on the same host or anywhere it can reach the Unix socket; pair one coordinator per adapter (see the capacity sizing note).

For NixOS the flake ships a service module wiring all of this up — see deploy-nixos.md.

For custom deployments, the adapter daemon can be installed as a Rust binary:

cargo install ngit-ci-adapter-microvm

Tagged releases also provide an ngit-ci-adapter-microvm-<version>-x86_64-unknown-linux-musl.tar.gz static archive as a signed NIP-82 asset, avoiding a Rust build on the adapter host. Verify its SHA-256 against the signed asset event before extracting it. This still installs only the ngit-ci-adapter-microvm executable. It does not install QEMU, configure /dev/kvm, create the Unix socket service, or provide the guest template image. The NixOS module remains the recommended production deployment path for the first-party microVM adapter.

Configuration

All options can be supplied via CLI flag, env var, or .env file (CLI > env > .env > defaults). The env prefix is distinct from the coordinator's so both can share one .env.

Flag	Env var	Required	Description
--socket	NGIT_CI_ADAPTER_MICROVM_SOCKET	yes	Unix socket to listen on; point the coordinator's --adapter-socket here.
--template	NGIT_CI_ADAPTER_MICROVM_TEMPLATE	yes	Read-only guest template qcow2 (see below).
--state-dir	NGIT_CI_ADAPTER_MICROVM_STATE_DIR	no (default .ngit-ci-adapter-microvm)	Per-job state plus persistent trust-scoped act caches. Leftover job-* directories are cleaned unconditionally at startup; <state-dir>/act-job-cache survives.
--cache-max-total-bytes	NGIT_CI_ADAPTER_MICROVM_CACHE_MAX_TOTAL_BYTES	no (default 107374182400, 100 GiB)	Maximum combined persistent act cache bytes across every repository and trust scope. Whole inactive scopes are evicted globally in least-recently-used order after jobs.
--max-jobs	NGIT_CI_ADAPTER_MICROVM_MAX_JOBS	no (default 1)	Concurrent job VMs; further jobs are rejected immediately (never queued). Size the coordinator's --max-concurrent-jobs at or below this. Disk usage can peak at roughly this many qcow2 overlays, each allowed to grow to the guest filesystem size (about 52 GiB with the shipped template).
--vcpus	NGIT_CI_ADAPTER_MICROVM_VCPUS	no (default 2)	vCPUs per job VM (hypervisor-enforced).
--memory-mib	NGIT_CI_ADAPTER_MICROVM_MEMORY_MIB	no (default 4096)	Memory per job VM in MiB (hypervisor-enforced). Host ceiling ≈ --max-jobs × this.
--boot-timeout-secs	NGIT_CI_ADAPTER_MICROVM_BOOT_TIMEOUT_SECS	no (default 120)	Max time for a guest to become SSH-reachable.
--qemu-bin	NGIT_CI_ADAPTER_MICROVM_QEMU_BIN	no (default qemu-system-x86_64)	QEMU system emulator binary.
--log-level	NGIT_CI_ADAPTER_MICROVM_LOG_LEVEL	no (default info)	Tracing filter.

The coordinator's --act-platforms, --act-container-daemon-socket, and --act-container-options settings apply inside the guest for act's job containers; the VM boundary is the actual containment.

The coordinator enables its build cache by default to persist standard actions/cache data across microVMs. The coordinator reuses its existing maintainer graph to choose an opaque repository/trust partition and sends it through reserved entries in the execute environment. Before boot, the adapter validates those fixed-format components, marks the scope active, and attaches only that directory at /run/ngit-ci-act-cache using 9p with mapped ownership. Sibling repositories and trust scopes are not mounted and are therefore unreachable from the guest. --act-cache-max-bytes is enforced per repository after unmount. The adapter then enforces --cache-max-total-bytes across all repositories and scopes. Both limits remove whole scopes in least-recently-used order, and active scopes are never pruned, so concurrent jobs can temporarily put storage above either post-job limit. If another VM is already using the same scope, the concurrent job uses its isolated job-local fallback rather than mounting one mmap-backed Bolt database twice.

The shipped guest runs act through a wrapper that adds --privileged to the effective --container-options while preserving the operator's resource-limit options. This is required for tools such as Nix to create nested user, mount, PID, IPC, UTS, and network namespaces for their own build sandboxes. Nix tests this support by remounting /proc from a nested user namespace; granting only CAP_SYS_ADMIN still leaves Docker restrictions that make the test fail and causes Nix to silently fall back to unsandboxed builds.

The privilege is against the fresh per-workflow guest kernel, not the operator host: the hardware VM remains the security boundary, the coordinator passes --container-daemon-socket -, workflow code cannot control the guest daemon, and embedded-act deployments do not receive this option. Do not reproduce this default on an embedded-act host merely to support Nix; use the microVM adapter when workflows require privileged namespace operations.

Building the guest template

The guest template is a minimal NixOS image (defined in nix/guest.nix) with bash, git, act, ngit, and dockerd, and with the pinned ghcr.io/catthehacker/ubuntu:act-latest platform image pre-seeded into docker storage, so job VMs never re-download it. The build is a pure nix build — no cloud-init, no network at build time (the docker image is a fixed-output dockerTools.pullImage derivation, loaded during a one-shot template-prep boot that powers itself off):

nix build .#microvm-guest-template   # needs Nix + KVM on the build machine
ls result/template.qcow2

Building the template is currently the one step that requires Nix; the resulting template.qcow2 is a plain file that can be copied to any adapter host. (A prebuilt downloadable template is future work.) Any other qcow2 image that provides bash, git, act, a running dockerd, and sshd with SMBIOS-credential key injection would also satisfy the contract.

To bump the pinned platform image, re-run nix-prefetch-docker as noted in nix/guest-template.nix and update the digest/hash there. Rebuilding the template never touches running adapters; restart the adapter with the new --template to roll it out (jobs always boot fresh overlays, so the swap is atomic per job).

Design notes

• qcow2 overlay recycling. Destroy = SIGKILL the QEMU process and delete the overlay. This is deliberately not limactl clone, which performs a full disk copy on filesystems without reflinks (minutes on ext4) and drove loom-adapter-lima to hot pools, a global limactl lock, staggered boots, and ~/.lima cleanup fallbacks. Independent overlay files and per-job QEMU processes share no mutable state, so that entire class of failure mode is absent by construction rather than handled.
• What was carried over from loom-adapter-lima instead: every external process operation has a hard SIGKILL timeout; caller disconnect kills the in-VM job immediately (never leak a busy slot); startup unconditionally cleans leftover overlays and VM processes from a previous crash (matched by pidfile + /proc cmdline before killing); /dev/kvm is checked at startup with actionable failure instructions.
• QEMU with slirp, not Firecracker/cloud-hypervisor. User-mode (slirp) networking gives job VMs outbound-only network with zero host network configuration — no tap devices, no bridges, no NAT rules — which is exactly the job requirement (checkout + dependency downloads). Firecracker and cloud-hypervisor require tap networking plus a kernel/rootfs image pipeline, the cost that kept both prior-art projects from shipping them. The QEMU invocation is contained in one module (crates/adapter-microvm/src/qemu.rs) behind a Hypervisor trait, so a later swap stays realistic.
• SSH over a loopback hostfwd, not vsock. The job command is bash -s with the script on stdin, so the guest transport must stream stdin/stdout/stderr faithfully and return an exit code — precisely what OpenSSH does. Each job VM gets an ephemeral ed25519 keypair, its public half injected via an SMBIOS type-11 systemd credential that the guest installs before sshd starts; the forwarded port binds to 127.0.0.1 on the adapter host only. There is still a local-only port allocation race before QEMU binds hostfwd; because host key checking is disabled for these throwaway guests, a process on the adapter host that wins that race could impersonate the guest and forge job output/exit status. That is outside the current local-attacker threat model but is another reason to prefer a future vsock exec transport. vsock would need vhost-vsock host support plus an in-guest agent speaking a bespoke exec protocol for the same result.
• No warm pool in v1. A cold boot costs ~10–20 s with KVM — small against real CI runtimes. A warm pool would only add held-back resources and lifecycle complexity before profiling shows the boot latency matters.

Testing

The adapter has an opt-in KVM e2e test that boots real microVMs through the full stack; see testing.md.