Skip to content

Execution-adapter protocol ​

Development preview

This page describes a pinned development revision, not a published release. Its package version may still identify the preceding release while the next version is being prepared.

This document vendors the Loom execution-adapter wire contract used by ngit-ci's socket-adapter runner backend. The authoritative source is loom-worker's docs/execution_adapters.md (Arjen / Origami74's Loom project); where that spec is ambiguous, this document records the observed behaviour of the reference implementation loom-adapter-lima and flags the choices ngit-ci resolved unilaterally.

The core execution exchange remains compatible with Loom adapters (shell, multipass, lima). The first-party Nix provider additionally requires the describe extension below; an adapter implementing only the core exchange cannot serve that path.

The Rust message types and line framing live in the workspace crate crates/adapter-protocol, shared by the coordinator's socket-adapter runner backend, the native Nix provider, and the first-party ngit-ci-adapter-microvm adapter binaries. The Firecracker adapter uses the same framing again over AF_VSOCK between its host process and disposable guest agent.

Transport and framing ​

  • Transport: a Unix domain socket the adapter listens on.
  • Framing: line-delimited JSON. Every message is one JSON object serialized onto a single line, terminated by \n. No message may contain a raw newline; newlines inside string values are JSON-escaped (\n). The shared implementation rejects messages larger than 16 MiB, while output and artifact data are chunked well below that ceiling.
  • Aggregate caps: MAX_MESSAGE_BYTES bounds individual frames only, so receivers must budget aggregate output per job. ngit-ci's coordinator caps the reconstructed job log at 4× the configured Blossom log-byte limit and the staged artifact spool at 2× the configured per-artifact byte limit plus 100 000 files; exceeding any of these fails the job closed (the connection is dropped, which obliges the adapter to kill the job) rather than waiting for the terminal message. Artifact member paths are also limited to 1 024 UTF-8 bytes and 16 components before parent directories are created. Counterpart implementations should budget arbitrarily many sub-cap frames the same way.
  • One command per connection. The caller connects, sends exactly one execute message, reads response messages until completed (or error), and the connection is closed. The protocol requires no execution-session state across connections.

Environment discovery extension ​

On a separate connection a caller can send {"type":"describe"}. A supporting adapter replies once with {"type":"description","environment":{...},"resources":{...}} and closes the connection. An unsupported adapter returns error. This read-only exchange does not acquire a job slot, boot a guest, or emit started. The first-party Firecracker adapter generates the description from its installed template, executor and configuration; see environment diagnostics.

The Nix provider hashes the compact serialized environment JSON, publishes those bytes to Blossom and sends the digest on each execute through reserved env metadata NGIT_CI_ADAPTER_ENVIRONMENT_SHA256. The published identity additionally binds provider input policy (configuration.artifacts), which the adapter's raw health baseline excludes; the provider therefore sends that policy alongside as NGIT_CI_ADAPTER_INPUT_LIMITS_JSON (max-input-file-bytes / max-input-total-bytes) so the adapter can reconstruct the published identity before comparing. Callers that omit the limits are compared against the raw adapter digest. A mismatch produces error without execution. The metadata is consumed by the adapter and is not passed into the guest. It requests no download or image installation. The resources object is reported separately and is not part of that identity. For an identity-bound execute, the adapter emits description before started, so the provider logs the resources of that execution even if the adapter restarted with different sizing since initial discovery. This response is not terminal during execution. Older core execution callers may omit the metadata and receive the original execution exchange without a description.

Resource-constrained Nix requests also send reserved env metadata NGIT_CI_ADAPTER_RESOURCES_JSON: a JSON object containing the exact resource values observed when the provider admitted the allocation, restricted to keys required by that allocation. The adapter removes this field and compares every entry against its current description before sending Description or Started, reserving a slot, or launching a VM. Missing, malformed, or changed values return Error; this is a not-started refusal. These values never enter the guest environment. This check closes the restart race where RAM changes without changing the fixed environment hash.

Supporting Firecracker adapters include configuration.resource-admission: "exact-values-v1" in their environment manifest. Providers refuse resource-constrained requests before Execute when this marker is absent. The allocation must bind that environment hash, so a restart onto an older adapter that lacks this mechanism also fails admission. Requests without resource predicates remain compatible with older adapters.

Explicit cancellation extension ​

First-party Nix callers probe a separate connection before executing:

json
{"type":"capabilities"}
{"type":"capabilities","graceful_stop":true}

An adapter without this capability is rejected before admission. The adapter also negotiates it with the guest supervisor before launching a command. Upgrade the adapter and rebuild both QEMU and Firecracker templates together. The same supervisor protocol runs over Firecracker vsock and QEMU SSH stdio.

On the active execution connection, ordinary cancellation supplies the remaining budget in milliseconds; force overrides it without a workload grace:

json
{"type":"stop","grace_ms":600000,"force":false}
{"type":"stop","grace_ms":0,"force":true}
{"type":"cancelled","duration":42,"reason":"forced"}

An ordinary request signals only the command leader with TERM. Duplicate ordinary requests do not extend the deadline. A later force request kills the workload immediately. The Nix guest supervisor translates control into its executor cancellation, preserving the supervisor long enough to stage evidence. At grace expiry the adapter enforces termination independently of the guest. Force and grace expiry permit at most two seconds of best-effort evidence drain before VM destruction; this is not additional application grace. Normal job execution timeouts remain enforced by the Nix executor and caller.

The caller keeps reading logs, step evidence and artifacts throughout shutdown. A completed result can win cancellation, including successful completion during grace. cancelled is sent only after execution stopped and the VM and per-job resources were destroyed. Its optional reason is not_started, exited, grace_expired, or forced; callers must not infer a kill from a request alone. Legacy cancel still requests immediate cancellation.

Provisioning cancellation has a five-second cleanup deadline; VM destruction has a twenty-second deadline. Callers allow the requested grace plus thirty seconds for evidence drain and cleanup acknowledgment. Missing acknowledgment is not cleanup proof. cleanupfailed quarantines adapter and provider capacity. A disconnected client causes sandbox teardown; ordinary writes time out after thirty seconds, and shutdown also bounds a stalled evidence write. Framing failures close the transport rather than continuing a partially written record.

These messages extend Loom's base contract. They affect only the execution on that connection. Generic callers can continue using Execute and disconnect; stop or cancel without an active execution is rejected.

Roles ​

The caller (Loom calls it the worker; in ngit-ci it is either the coordinator's socket-adapter runner backend or the native Nix provider) is smart: it owns queueing, retry, and timeout enforcement.

The adapter is dumb:

  • it keeps no queue — when it has no free capacity it rejects the job immediately with an error message instead of buffering,
  • it enforces no timeouts — the caller enforces its job timeout by closing the connection, at which point the adapter must kill the running job immediately (never leak a busy slot),
  • it holds no execution-session state across connections.

Messages ​

Caller → adapter ​

json
{"type":"execute","identifier":"<job-id>","cmd":"bash","args":["-s"],"stdin":"<script>","env":{"KEY":"value"}}
FieldTypeMeaning
typestringAlways "execute".
identifierstringOpaque job identifier, used by the adapter for logging only.
cmdstringExecutable to run inside the sandbox. Resolved via the sandbox's PATH; each of cmd and args is a separate argv element (no shell splitting by the adapter).
argsstring[]Arguments for cmd.
stdinstringData written in full to the job process's stdin, after which stdin is closed. May be empty.
envobject (string → string)Environment variables set on the job process.

When build caching is enabled, ngit-ci places filesystem-safe, opaque repository/trust partition keys and the byte cap in reserved NGIT_CI_ADAPTER_CACHE_* entries of this existing env object. This is an implementation convention, not a message-schema extension. The first-party microVM adapter consumes the metadata before execution and injects the mounted guest path; adapters that do not implement persistence run against the coordinator script's isolated job-local fallback.

ngit-ci's first-party adapters accept an optional artifact_dir string. When it is present, the adapter streams files staged beneath that guest path after the command exits and before destroying the sandbox. Native Nix jobs always set it for their result manifest, even without user-declared artifacts; it does not identify an act job. Other adapters that do not recognise it may ignore it.

Adapter → caller ​

json
{"type":"started"}
{"type":"stdout","data":"output chunk"}
{"type":"stderr","data":"error chunk"}
{"type":"artifact","path":"1/name/name.zip","data":"<base64 chunk>","eof":false}
{"type":"completed","exitCode":0,"duration":42}
{"type":"error","error":"Pool exhausted - all VMs busy. Retry later."}
MessageMeaning
startedExecution capacity was acquired and the job is about to run. Sent at most once, before any stdout / stderr / completed.
stdout / stderrA chunk of the job's output stream, streamed in real time. data is a UTF-8 string; chunk boundaries are arbitrary (not line-aligned).
artifactOptional ngit-ci extension. A base64 file chunk from the requested artifact_dir; path is relative to it and eof marks that file's final chunk.
completedThe job process exited. exitCode is the process exit code; duration is the adapter-measured runtime in whole seconds (floored). Terminal message; the adapter closes the connection afterwards.
errorSomething other than the job's own exit code went wrong. Terminal message; the adapter closes the connection afterwards.

Message ordering ​

On one connection the caller observes, in order:

  1. either error (rejection — no capacity, malformed request) orstarted,
  2. after started: zero or more stdout / stderr messages, interleaved in the order the adapter read them from the job's pipes,
  3. exactly one terminal message: completed on process exit, or error if execution infrastructure failed mid-job.

Lifecycle obligations ​

Caller:

  • send one execute, then only read,
  • enforce the job timeout by closing the connection; do not expect any further message after doing so,
  • treat connection EOF without a terminal message as job failure.

Adapter:

  • reject immediately (with error) when at capacity — never queue,
  • kill the in-sandbox job as soon as the caller disconnects (loom-adapter-lima learned this the hard way: without the kill, a hung command consumed a pool slot forever),
  • destroy all per-job state after the connection closes; on startup, clean up any leftover job state from a previous crash.

Ambiguities observed and resolved ​

These points are not covered (or not covered crisply) by the upstream spec; ngit-ci resolves them as follows, matching loom-adapter-lima's observed behaviour where one exists:

  1. The error message type. The upstream four-message summary (started, stdout, stderr, completed) omits error, but loom-adapter-lima emits it for pool exhaustion, malformed requests, and internal failures. ngit-ci treats error as part of the contract, in both positions (pre-started rejection and mid-job infrastructure failure).
  2. Rejection is not machine-distinguishable. The error string is free-form (loom-adapter-lima: "Pool exhausted - all VMs busy. Retry later." for capacity, other strings for malformed requests). ngit-ci's caller classifies any error received before started as a retryable rejection. The act socket backend surfaces it as a failed (startup_failure) kind-9842 result without retry. The native Nix provider signs a not-started kind-9841 Job Result, which permits the coordinator's bounded safe-reassignment policy. An error after started never makes an effect repeatable.
  3. duration unit. loom-adapter-lima reports floored whole seconds. ngit-ci treats the field as informational only and measures its own elapsed time.
  4. Non-UTF-8 job output. Adapters decode output bytes to strings; loom-adapter-lima uses lossy decoding (invalid sequences become U+FFFD). Callers must not assume byte-exact output.
  5. completed on execution failure. When in-sandbox execution fails after started, loom-adapter-lima sends completed with exitCode: 1 rather than error. Callers must therefore not treat completed + nonzero exit as distinguishable from "the workflow itself failed"; both are reported as job failure with an exit code.
  6. Multiple execute messages on one connection. Undefined upstream; loom-adapter-lima ignores extra input. ngit-ci's caller never sends more than one, and the ngit-ci adapter is free to reject or ignore extras (resolved unilaterally).
  7. started timing. loom-adapter-lima sends started after acquiring a VM but before the job process is spawned. Callers must not interpret started as "the command is running", only as "capacity acquired, rejection can no longer happen".

How ngit-ci uses this contract ​

The coordinator's deprecated socket-adapter act backend renders one selected workflow plan into a self-contained job script (repo clone with fallback URLs, act event payload, act invocation — the same logic the embedded-act runner owns) and submits it as a single execute message (cmd: "bash", args: ["-s"], stdin: <script>). Streamed stdout / stderr become the job log published in the kind-9841 Job Result. The coordinator's --job-timeout-secs is enforced by dropping the connection.

Workflow discovery, trigger matching, queueing, and CI event publishing stay in the coordinator, per architecture.md. The adapter only ever receives an already-planned job as an opaque command.

The sandbox the script runs in must provide bash, git, act, and a Docker-compatible daemon for act's job containers (the loom-adapter-lima golden template ships all of these; so does the ngit-ci-adapter-microvm guest template, nix/guest.nix).

Adapter service shutdown ​

The bundled adapters stop accepting connections on shutdown, reject unstarted work on existing connections, and await active execution, cleanup and terminal reporting. Active caller connections retain their normal disconnect-cancellation semantics. Supervisors must keep the caller and adapter alive for the job's execution deadline and reporting; see provider shutdown.

Private prepared input stream ​

The native provider may send input messages before execute on the same connection. Each has base64 data and boolean eof. Encoded chunks are bounded to 128 KiB and aggregate decoded private build input to 16 GiB. Exactly one completed stream may precede an execution; truncation, further input after EOF, and other terminal messages are rejected. The input-plus-execute exchange has a five-minute deadline.

Adapters stage bytes in private temporary files, never open a caller-supplied host filename, bound concurrent staging, and remove the file after execution. First-party guest transports forward the stream before starting the command. The guest supervisor supplies the private filename through NGIT_CI_ADAPTER_INPUT, overriding any caller environment value. The Firecracker service and guest place these files on their disk-backed state/scratch volumes. Disconnect/cancellation tears down the guest and drops the private input. This is an internal transport extension, not a Nostr event.

The provider receives the preparation store export with the same 16 GiB bound, separately from its 2 GiB aggregate ordinary-artifact limit. This allowance is restricted to the controlled preparation phase and its designated store-export path; runtime files cannot opt into it by name. Handoff staging checks the configured free-space floor and fails the job explicitly on pressure, while optional cache exports retain their separate limits and discard behavior. Provider transfer logs and guest export/import diagnostics record byte counts and elapsed milliseconds. The allowance does not guarantee that a closure fits in the guest scratch disk alongside its imported store and working files. Update the provider, adapters and guest template together: older peers retain the 2 GiB private-input limit. Message shapes and Nostr events are unchanged.

For pooled Firecracker adapters, the private Description.resources.admission.slots field reports the adapter's local slot ceiling. Automatic provider capacity is bounded by both this ceiling and the shared resource budget. Older adapters without this field retain a conservative one-slot automatic ceiling. This nested object is not copied into public advertisement resource tags.