Idunn/docs/deploy-provenance-map.md

Deployment provenance: map

Status: map, Imagination, 2026-10-23 (campaign idunn-deploy-provenance, session self-2026-10-23-eureka-b). The ends are in deploy-provenance-target.md. Questions, rulings, cut specs and follow-ups live in Eureka's mind; this page holds what has no kind: body facts, rationale and sequencing. Line numbers in T1-T12 are at d74ac8d (origin/main on 2026-10-23); T4, T9 and T13-T16 are at fd040f0, the head of Hands attempt 1, which revision 2 of the spec builds on.

Cut tree-executable-members

Spec idunn-deploy-provenance:cut_spec:cut-tree-executable-members.r2 (revision 2, after Soul pass 1 found the real interpreter tree refused). It serves motion-pipeline:follow_up:idunn-tree-executable-members (Vili's admission waits on it) and will serve gamecult-chat C3 too: the presence launcher runs a child such as upstream/bin/daemon from inside a tree artifact (README, launcher fragment), and Synapse needs a Python interpreter there.

Body facts

• T1. ArtifactOutput (src/deployment.rs:106-117) carries one executable bool. ArtifactReceipt (src/deployment_plan.rs:731-737) copies it and validate (:740-760) requires it to equal the declaration.
• T2. harden_installed_release (src/drivers.rs:6848-6855) builds its executable set from root.join(receipt.destination) only. harden_root_tree (:6858-6892) seals every directory 0555 and every file 0444 unless that exact path is in the set. A directory artifact with executable = true is therefore silently meaningless today: its files are all 0444. Nothing refuses it. No live recipe declares one (CodexConnector, Ghostlight, Heimdall, Odin, StreamPixels service and web surveyed): executable = true is on files only.
• T3. A tree output is copied by copy_artifact/copy_tree (src/drivers.rs:8053-8126), which recreate symlinks verbatim. The digest (digest_tree, :8142-8175) covers names, file bytes and link targets, not mode bits. So runner or git exec bits never reach the release: harden is the only mode authority.
• T4. validate_release_symlink (src/drivers.rs:6897-6916) keeps a symlink only if its target is relative with no ... It runs only inside harden_root_tree, at install, so a tree with a .. or absolute link copies, digests and seals, then fails at install_release (Soul s1: copying and digesting the real tree twice took 160 s before harden refused it). The rule is sound but too narrow: it refuses the real interpreter tree (T13). The frozen-source path has a physical resolver (resolve_frozen_source_symlink, :6617) that walks the filesystem with a 40-link budget and a memo. Its first version was exponential (Soul S4-1, recorded in d562a12), and it needs the tree on disk to answer. Revision 2 does not reuse it (T14).
• T5. The sealed release id hashes (schema, plan_id, artifacts, external_inputs) (src/deployment_plan.rs:1000-1011), and plan_id hashes the whole plan including recipe_blob. A changed member list is therefore already a new plan and a new release. The receipt must still carry the list, because harden_installed_release reads receipts only; its two callers (install_release :2788, validate_installed_release :2822) re-harden every installed release on each observation.
• T6. Encoding is rmp_serde::to_vec, positional. A trailing field with #[serde(default, skip_serializing_if = "BTreeSet::is_empty")] keeps every existing receipt's bytes and release id: the plan's deadlines field (:574-577) is the precedent.
• T7. digest_artifact and digest_tree (:8128-8175) fs::read each file whole. A torch CPU tree's largest file (libtorch_cpu.so, hundreds of MB) is held in Idunn's memory three times per install (seal, install, every validate). hash_frozen_source_tree (:6788-6840) already streams in 64 KiB chunks (S9). The cut collapses both onto one streaming helper.
• T8. require_leaf_path (src/deployment.rs:1951-1958) only keeps the artifact destination a single filename. Members are paths inside that destination, so it does not change. require_relative_path (:1936-1949) with allow_current = false already refuses absolute, .., root and prefix components.
• T9. The host actuator re-spells the seal (src/host.rs:1050-1086): copy, mode check, digest, pinned-digest check and receipt, the same steps as collect_artifact (src/drivers.rs:2305-2326). Soul s1: deleting its mode check survives every test. Revision 2 collapses both onto one seal_artifact in drivers.rs. Each caller keeps only how it finds the source path.
• T10. The transient unit runs with ProtectSystem=strict and the release in ReadOnlyPaths (src/drivers.rs:3675-3698), /tmp mounted noexec, and no MemoryDenyWriteExecute. Executing a 0555 file from the release works. Shared libraries need no exec bit.
• T11. Live body on Yggdrasil: /usr/local/bin/idunn sha256 44cf8a14..., installed 2026-10-20 02:34 CEST, before 6e2a31c (02:40 CEST). src/ is identical between 8760a4f and 6e2a31c, so the live code is the reincarnation c2 merge. origin/main is 26 commits ahead (+4194/-112 in src/: the presence launcher and member route). Installing this cut ships all of them. Idunn's own body changes only through gamecult-ops scripts/install-idunn-yggdrasil-release.sh (staged binaries, unit, bindings). The stage at /home/gamecultadmin/idunn-stage is stale (2026-09-11).
• T13. Link census of the real interpreter tree (Yggdrasil probe of cpython-3.12.15+20261021-x86_64-unknown-linux-gnu-install_only.tar.gz, sha256 5ff9f060...77191124; torch adds no links, as Soul s1 counted 1,049 after pip): 1,049 links. 0 are absolute, 749 have no .., and 300 have all their .. components before the first name (every one is share/terminfo/<c>/<name> -> ../<c2>/<name2>, one .. each). 0 have a .. after a name, 0 climb above the tree, 0 dangle and 0 resolve outside it. Blender 5.2.2 has 79 links, none absolute or with .. (Soul s1).
• T14. Lexically normalising <link's directory>/<target> does not check containment. The kernel resolves a target one component at a time, so a .. after a symlinked name climbs from where that link points, not from where its name sits. Probe on Yggdrasil: root/a/s -> .. and root/a/L -> s/../x normalise to root and root/a/x, both inside, yet L reads the file x beside root. root/a/d -> . with root/a/M -> d/../../x escapes the same way. Idunn has met this before: verify-transaction-cut F5 (D -> ., L -> D/D/../../../etc/passwd passed a lexical check). The sound lexical rule constrains the target's shape: all its .. components come before its first name, and they number no more than the link's depth below the artifact's top. The climb then starts from the link's own directory, which the walk reached through real directories only (copy and digest never descend through a link), so it lands on a real ancestor inside the artifact. After that the target only descends, and any link it passes through obeys the same rule. By induction on the kernel's hops (capped at 40, then ELOOP), every hop stays inside. The check is one pass over the target's components with no filesystem reads, so its cost is linear in the tree and no resolver is needed. The shapes it refuses (s/../x) occur in neither real tree (T13). Prior art, from documentation as known to Imagination (high confidence for Go and securejoin, medium for the tar details):
  • Go path/filepath.IsLocal (1.20) is lexical and says so: it does not account for symlinks. It would admit s/../x. Used alone, it is the T14 bug.
  • Go os.Root (1.24) and cyphar/filepath-securejoin (used by runc, containerd and umoci) resolve each component inside the root when a path is opened, refusing or clamping an escape. They guard one later access, need the tree on disk, and check again on every open. Idunn instead certifies a sealed tree once, so that the kernel's own resolution stays inside on every later access.
  • Zip-slip defences and archive extractors check entry names lexically and protect against link targets by refusing to write through links (libarchive's ARCHIVE_EXTRACT_SECURE_SYMLINKS, and SECURE_NODOTDOT for names; GNU tar creates such links last). They protect the extraction, not what a kept link points at afterwards.
  • OCI layer extraction keeps link targets verbatim, absolute ones included, because the runtime resolves them inside the container's root. Idunn runs a release in the host's mount namespace, so absolute targets stay refused.
  • Nix keeps link targets verbatim and tracks dependencies by scanning for references. The whole store is its root, so it has no per-artifact containment to compare with. None of these is a static per-link rule for a tree that runs unconfined. The T14 shape rule fits that case.
• T15. Host targets: idunn-host observes, starts and stops processes on Windows only (the non-Windows platform module in src/host.rs bails), and host.rs sets no file modes. On Windows a file runs by its type, not by a mode bit, so executable and executable_members have nothing to set there. The file executable flag has always been carried and never applied on host targets. A recipe does not know which workload kind its binding will choose.
• T16. The release root holds only artifact destinations. Runner workspaces, where external inputs land, are removed before SealedRelease::new (src/drivers.rs:2421-2423). install_release (:2781-2788) and validate_installed_release (:2815-2822) digest every receipt's destination before calling harden. So a rule in the digest walk runs at seal, at install and at every validate, over every link harden would see.
• T12. Disk: / on Yggdrasil has 1.3 TB free. A tree release costs about three copies (runner workspace, staging, installed). Nothing collects old releases or staging (ghostlight-verse:follow_up:idunn-generation-retention), so every Vili deploy leaves about 7 GiB behind.

Contract, corrected against the body

• Any directory artifact may declare executable_members (a set of relative paths). The follow-up limited it to runner-output; that limit protects no invariant, and the launcher's worktree or fetched trees need the same feature.
• executable = true names a file artifact. On a directory output the release is refused, and executable_members on a file output is refused (T2).
• A member is refused at recipe parse if it is empty, absolute, or holds ., .., a root or a prefix. At seal it is refused if it is missing, is a symlink, passes through a symlink or non-directory, or is not a regular file. Refusal happens before SealedRelease::new, so no sealed release can name a member it cannot seal.
• Harden seals each member 0555 and fails if any declared executable path was not visited as a regular file. "Every declared executable is sealed 0555, or the install fails" holds structurally, not by a lookup that can miss.
• Symlinks: a link in a tree artifact is kept when its target is relative, its .. components all come before its first name, and they number no more than the link's depth below the artifact's top (T14). One function, check_release_link, decides this. Only the digest walk calls it, so a bad link refuses the release at seal, before SealedRelease::new, and again at install and validate (T16). Harden no longer judges links. The climb is bounded by the artifact's top, not the release root: no live release has a .. link (the old rule refused them all), so nothing installed is newly refused, and each artifact's digest describes a closed tree.
• Host targets carry executable and executable_members in the receipt, shape-checked at seal by the shared seal_artifact, and do not apply them (T15). Refusing members there would make a recipe's validity depend on its binding.
• Vili's trees must not come from python -m venv: its bin/python is an absolute link and its pyvenv.cfg names the build workspace. Install packages into the relocatable interpreter tree instead (motion-pipeline:follow_up:vili-relocatable-python-tree).

Authority

Owner: the seal (seal_artifact, called by collect_artifact for systemd targets and by materialize in host.rs for host targets) decides membership from the recipe and writes it into the receipt. Harden enforces the receipt and decides nothing. Runner exec bits and git 100755 modes are dead inputs for release modes. Link containment belongs to check_release_link, which only digest_tree calls. validate_release_symlink is deleted, and harden's symlink arm leaves links as they are.

Acceptance sizing

The big-tree job (spec verification.operator) needs about 12 GiB of scratch on Yggdrasil: a 3.5 GiB tree, its staging copy, downloads and pip cache. It removes it at the end. Revision 2 runs it on the trees as shipped, with the 300 terminfo links (T13). If a tree holds a link the T14 rule refuses, the job prints it and stops, and the link is reported to Self, not deleted.

Cut isolation-credential-evidence

Spec idunn-deploy-provenance:cut_spec:cut-cut-isolation-credential-evidence.r1, at 112a7a5 (origin/main, live on Yggdrasil since 2026-10-24 11:29 UTC). It serves Vili's admission (motion-pipeline:cut_report:cut-vili-admission.h4, first deviation): every idunn up vili is refused before fencing.

Body facts (probed read-only on Yggdrasil, 2026-10-24)

• I1. idunn up vili runs 6 and 7 failed with systemd workload isolation properties differ from the Idunn contract, the message of observe_unit (src/drivers.rs:2975-3005). Of its conjuncts, only one differs on the failed Vili unit: LoadCredential. Idunn expects "" because Vili has no service credential besides the presence identity (service_credential_names empty); systemd shows [unprintable]. Every other conjunct matches, InaccessiblePaths included (one value across all five idunn-vili/odin units).
• I2. systemd 257.13 systemctl show -p LoadCredential prints LoadCredential=[unprintable] for every unit, whatever the credentials: ssh.service and systemd-journald.service (not DynamicUser, zero LoadCredential), the four failed idunn-vili-* units (zero), and the Odin unit (one). The display form carries no information, so the conjunct was vacuous whenever credentials existed and always false when none did. Odin and StreamPixels passed only because each carries a regular credential.
• I3. The exact value is readable: D-Bus org.freedesktop.systemd1.Service.LoadCredential (busctl get-property) answers a(ss) 0 for the Vili units and ssh, a(ss) 1 "ODIN_TOPOLOGY_IDENTITY" "<path>" for Odin.
• I4. What the workload can actually read is the delivered directory. On the host, /run/credentials/ holds a directory only for units with credentials (getty, journald, Odin, StreamPixels service and web). Inside Odin's MainPID root, /run/credentials/<unit>/ holds exactly ODIN_TOPOLOGY_IDENTITY (-r--r-----+ root root). The presence identity is a parent-only OpenFile descriptor, never a credential.
• I5. The rule arrived whole in 1fc0dad (2026-09-03, "Rebuild Idunn deployment transaction authority"), with its twin clause in validate_launch_observation (:3468-3472, :3513-3516); no later commit touched it. No ruling or finding in any campaign concerns workload isolation verification; gamecult-chat's foreign-workloads cut touched CREDENTIALS_DIRECTORY reservation, not this check.
• I6. SystemdWorkloadObservation is persisted by previous Idunns in both MessagePack encodings, compact (positional) included (persisted_systemd_observations_decode_as_the_systemd_variant, :9915-9930), so deleting the load_credential field shifts every later slot of a live incumbent's record. The field is also in the drift list (:2462) that stop (:4064-4070) uses to refuse stopping a changed workload: changing what the field holds without leaving that list would make the next Idunn refuse to stop the live Odin and StreamPixels incumbents.
• I7. Fixtures: the verbatim 257.13 systemctl show output of base's 27 properties for a Vili unit (presence identity only) and the Odin unit (presence identity plus one service credential), captured 2026-10-24 into this Imagination's scratchpad fixtures/.
• I8. The new rule holds for every live systemd incumbent: inside each MainPID root the delivered entries equal its binding's secret_files keys minus GAMECULT_RUNTIME_PRESENCE_IDENTITY (Odin 1, StreamPixels service 11, StreamPixels web 1; key names compared, no values read). An Idunn restart onto this cut re-observes them (:3955-3970) and passes.

Decision

A property systemd cannot display does not belong in the comparison. The invariant it reached for, that the workload holds exactly the credentials Idunn delivered, is observed where it is decided: the entries of /run/credentials/<unit> in the MainPID's root equal Idunn's service credential names, absent directory equal to none. That covers LoadCredential, SetCredential, ImportCredential and drop-ins alike, and needs no new program. The D-Bus read (I3) is exact but sees only one of those properties and adds a second systemd client and parser; rejected.

Authority

Owner: observe_unit decides the credential rule, from the delivered directory. LoadCredential's display form is no longer read or compared; the load_credential field is a retired wire slot, written empty, out of the drift list, kept only so persisted records decode. Forbidden writers: the observe_unit conjunct and the validate_launch_observation clause. Dropping the slot waits on the next schema revision of the observation.

Deploy

Idunn's body changes only through gamecult-ops scripts/install-idunn-yggdrasil-release.sh (stage idunn only; unit and bindings unchanged). After it, Vili's Hands reruns from step 4 of motion-pipeline:cut_report:cut-vili-admission.h4's undone list.

Cut runner-failure-evidence

Spec idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1, at 804ed75 (origin/main, live on Yggdrasil). It serves the StreamPixels-area Self: idunn up vili (command up-4866467c-ee98-4a2b-b4a9-a008d7c44acd, transaction tx-a04b68c7-c742-4d7d-8220-8aa08b9febae, 2026-10-25 00:50-00:55 UTC) failed in Sealing with Docker runner exited with exit status: 1: and nobody could tell which recipe step failed.

Body facts (source read at 804ed75; host read-only, 2026-10-25)

• R1. The runner is docker run --rm (src/drivers.rs:2088-2090), launched by DockerRunnerDriver::docker (:2173-2195) with stdin null and Command::output(): stdout and stderr are each buffered whole in the daemon's memory, unbounded. On a non-zero exit the error is "Docker runner exited with {status}: {stderr}"; stdout is dropped. The container is gone the moment it exits, so docker ps -a and docker logs hold nothing.
• R2. The empty message: the failing process wrote nothing to stderr, and stdout, the only stream that carried its report, is discarded at :2182-2193. The same interpolation echoes whatever a step does write to stderr into last_error, pre_fencing_abort.error and completion.error (bounded_detail, 2048 bytes), and from there into idunn up's deployment failed: line and idunn status.
• R3. No step identity reaches the error. materialize (:2368-2391) loops over declaration.steps and calls run_in_workspace with argv only; step.id is used only in the environment precondition. External inputs (curl in the runner, :2247-2258) fail through the same helper with the same anonymous message.
• R4. Which step failed on 2026-10-25, inferred from the failed transaction's staging root, which survives (926 MiB, /var/lib/gamecult/idunn/staging/tx-a04b68c7-.../): in .runner-node-build, node_modules/.package-lock.json was written at 00:51:24 (so install-vili-dependencies, npm ci, finished), nothing outside node_modules is newer, and release/vili (the next step's output) is absent; the python workspace holds only fetched inputs. The failing step was test-daemon (node --test ... scripts/test-vili-daemon.mjs), whose test reporter writes to stdout: exactly the stream R1 discards. Inference from residue, not from output.
• R5. The failure path: resume_one_transaction (src/control_plane.rs:6291-6320) sends a pre-fence error to begin_pre_fencing_abort (:9916-9935), which persists bounded_detail(format!("{error:#}")). Nothing reaches the journal on this path: the idunn-yggdrasil journal for 00:45-01:10 holds only provider-exclusion lines.
• R6. Stores. control.cc (root 0640) holds commands, transactions (idunn.deployment_transaction.v4, keys 0-36), generations, supervision and expiry requests; ControlSnapshot::read refuses any other type (:3420). Records must re-encode byte-exactly (decode_record, :3717-3727), so a new transaction field is a schema bump and a v4 lift (:4100-4110). Terminal transactions move verbatim to history.cc (:4366-4396; 39.8 MB on the host), which status reads for --command. incidents.cc lives in the published projection directory and by its own doc (src/incident.rs:1-11) carries no free text and is never control.cc, because a type added there stops continuity for every target on a rollback.
• R7. The daemon runs as root (User=root). Runner secrets reach the container as read-only file mounts named by env (:2041-2047), so any text a step derives from them can land in its output.
• R8. Staging residue: 475 transaction roots, 55 GiB under /var/lib/gamecult/idunn/staging; a failed Sealing leaves its workspaces because materialize returns before remove_tree_inside.

Decision

The failure becomes a typed value at the runner, RunnerStepFailure: the subject (recipe step or external input, by id), the runner id, the exit code or signal, and the last 8 KiB of stdout and of stderr with each stream's total byte count. Its Display carries the subject and status, never output. Capture streams both pipes through bounded tails, so a verbose build no longer buffers whole in the daemon.

The tail lives in a new private store, runner-evidence.cc, beside control.cc (mode 0600), one record per transaction (idunn.runner_failure.v1), newest 32 kept by the writer. Not in the transaction record: that is a v5 bump whose rollback makes every older Idunn refuse its control store (R6), the scar incident.rs already names. Not in incidents.cc: published, and free text is barred there. Not in the journal: journal readers are a wider set than root, so the journal line names the transaction, the subject, the status and the store, and no output.

No redaction. Matching secret bytes would make the daemon read every secret, and still misses any encoding, prefix or transform of one; a claimed redaction that leaks is worse than a stated rule. The rule instead is the reader set: the store's only readers (root) can already read every secret the runner was given.

Keeping the failed container (dropping --rm) is worse: its output then lives in Docker's json-file log, unbounded, untyped, outside Idunn's state and readable by the docker group; containers accumulate with no owner to reap them; docker system prune erases the evidence; and nothing ties a container to the transaction in idunn status.

Authority

Owner: the runner decides what failed and keeps the bounded tail; the engine's pre-fence failure path is the one writer of runner-evidence.cc. last_error, pre_fencing_abort.error and completion.error keep only the failure's Display. Forbidden writers: the stderr interpolation in DockerRunnerDriver::docker. Retention is by count inside the writer, so the store needs no external owner; staging roots (R8) stay with ghostlight-verse:follow_up:idunn-generation-retention.

Deploy

Idunn's body changes only through gamecult-ops scripts/install-idunn-yggdrasil-release.sh (stage idunn only; unit and bindings unchanged). The next failed Sealing then shows its step and tail on sudo idunn status --command <id>.