Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 140fbc3161 | |||
| d13d98a083 | |||
| 6c34758a30 | |||
| 17edea299e | |||
| 506df0a81f | |||
| eace76870a | |||
| c30f89f0df | |||
| 47a9c3d89e | |||
| cdf987730e | |||
| 8dd9f8d503 | |||
| 7eae1c4ee1 | |||
| 74c04bbcc7 | |||
| c13e175bbf | |||
| c5a0ce3d23 | |||
| 37622db430 | |||
| 58e744b745 | |||
| b230703a5c | |||
| a20568ef70 | |||
| 114803c63b | |||
| 7ba27c362e | |||
| 01b6016193 | |||
| 9f15cee11f | |||
| be98521120 | |||
| dc86c6326c | |||
| 74d6ef31e1 | |||
| d5ea779096 | |||
| 6c8fd12335 | |||
| 46ec8241a2 | |||
| 4b6a58c692 | |||
| 157610adb7 | |||
| 1c23b6e217 | |||
|
|
64d63cb417 | ||
|
|
77e2295e97 | ||
|
|
fb1e7b5dd9 | ||
|
|
8ebe0208a6 | ||
|
|
a859914b0b | ||
|
|
4894ad20df | ||
|
|
879b76a842 | ||
|
|
631fb7836b | ||
|
|
d29bf93be6 | ||
|
|
d9710c988a | ||
|
|
caacbb4bba | ||
|
|
9653d270e5 | ||
|
|
8a1beb5c87 | ||
|
|
7e15bf1a7b | ||
|
|
572f9e6aa4 | ||
|
|
56edad95c1 | ||
|
|
ca439a29ff | ||
|
|
27ca875194 | ||
|
|
be3b95bc21 | ||
|
|
cea17c06a7 | ||
|
|
ec6222e2a1 | ||
|
|
7d5cd85728 | ||
|
|
1a3634e046 | ||
|
|
e50a721dbc | ||
|
|
024f2d7a09 | ||
|
|
8400d0712f | ||
|
|
1518cfb13c | ||
|
|
549f4be093 | ||
|
|
4a68df520f | ||
|
|
134c1fb5a2 | ||
|
|
54a75ef9b0 | ||
|
|
7ef31ee699 | ||
|
|
22bd904ed9 | ||
|
|
ef82355bab | ||
|
|
78086705a0 | ||
|
|
804ed759c9 | ||
|
|
dfdc226c25 | ||
|
|
ae6a1bea55 | ||
|
|
def0b78378 | ||
|
|
69eb3e6d6f | ||
|
|
112a7a5ebb | ||
|
|
a3c6386aa4 | ||
|
|
d3222e5724 | ||
|
|
e8c4c69728 | ||
|
|
68d6f96808 | ||
|
|
07616697af | ||
|
|
fd040f09c1 | ||
|
|
76cec0b540 | ||
|
|
04ff2c8ce3 | ||
|
|
022dbece5b | ||
|
|
332dbd0149 |
29 changed files with 13344 additions and 1446 deletions
1
Cargo.lock
generated
1
Cargo.lock
generated
|
|
@ -482,6 +482,7 @@ dependencies = [
|
|||
"libc",
|
||||
"rmp-serde",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"tempfile",
|
||||
"toml",
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ serde = { version = "1", features = ["derive"] }
|
|||
ed25519-dalek = "2.2"
|
||||
toml = "0.8"
|
||||
uuid = { version = "1", features = ["v4"] }
|
||||
serde_json = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
|
|
|
|||
84
README.md
84
README.md
|
|
@ -58,6 +58,64 @@ idunn status --command <id>
|
|||
the bindings, so `idunn up profile:full-gamecult` brings up everything bound
|
||||
into that profile.
|
||||
|
||||
### Steward orders
|
||||
|
||||
A second root unit from the same binary, `idunn-orders` (`idunn orders serve`),
|
||||
takes typed orders from one steward Mind over `/run/idunn-orders/steward.sock`.
|
||||
It has its own store, `orders.cc`, and its own binding, `steward.toml`; the
|
||||
survival daemon takes no order. There is still no command string:
|
||||
|
||||
```bash
|
||||
idunn order package-install <pkg>[=version]... --reason TEXT # apt-get update, install named packages, check them
|
||||
idunn order unit <start|stop|restart|reload> <unit> --reason TEXT
|
||||
idunn order deploy <service|profile:name> --reason TEXT # the same commit as `idunn up`
|
||||
idunn order show <id> # status and output tails
|
||||
idunn order list
|
||||
```
|
||||
|
||||
A unit order submits exactly one job, for the unit it names: every verb runs
|
||||
`systemctl <verb> --job-mode=ignore-requirements -- <unit>`, so systemd builds a
|
||||
transaction of one job. It does not start what the unit requires or restart what
|
||||
is part of it: order the dependency first, and order `postgresql@16-main.service`
|
||||
rather than the `postgresql.service` wrapper. Targets are not orderable. The job is one; what the unit does afterwards is
|
||||
not: systemd still runs full transactions of its own for `Restart=` after a
|
||||
failure, for a `StopWhenUnneeded=` dependency left without a user, and for a
|
||||
`Conflicts=` partner of a unit that fails, so an ordered stop of a unit
|
||||
that something restarts does not last, and a start of a `Restart=always`
|
||||
unit can report failure and be active seconds later. A unit
|
||||
order may act on any unit except Idunn's own (`idunn-*`) and the minds' own
|
||||
(`mind-self@*`, `mind-persona@*`), and the units that end or take the host
|
||||
(`systemd-suspend`, `systemd-hibernate`, `systemd-hybrid-sleep`,
|
||||
`systemd-suspend-then-hibernate`, `rescue`, `emergency`). That set is compiled
|
||||
into the binary and read from no configuration. An order is refused, recorded
|
||||
and posted when the ordered unit, or an alias it resolves from, is one of them;
|
||||
when it has an emergency action (`SuccessAction=`, `FailureAction=`,
|
||||
`StartLimitAction=`, `JobTimeoutAction=`) other than `none` or an `OnFailure=`
|
||||
or `OnSuccess=` unit (not for a reload); or, for a stop or restart, when another
|
||||
unit is `BoundBy=` it, because systemd stops that unit with it whatever the job
|
||||
mode; for a start or restart, when it has an `Upholds=` unit, which systemd
|
||||
starts in a full transaction whatever the job mode; or, for a stop, restart or
|
||||
reload, when it has a `ConsistsOf=` member, a wrapper such as
|
||||
`postgresql.service` whose verbs would exit 0 and touch nothing. A unit
|
||||
`systemctl show` cannot resolve fails the order. Slices, scopes,
|
||||
targets and globs are not units an order can name.
|
||||
|
||||
A package order installs the packages it names and nothing else: apt runs with
|
||||
`--no-remove`, each name is passed as an exact-name pattern so `.` and `+`
|
||||
match themselves, and a last step fails the order unless every named package
|
||||
is installed.
|
||||
|
||||
An order runs once, after the steward's uid is checked and the instance's
|
||||
brake check (`mind-control --instance <principal> check`) exits 0. The check
|
||||
that the order's principal and uid are the bound steward's runs again at every
|
||||
execution, so an order accepted before `steward.toml` changed is refused and
|
||||
recorded, not run. The socket belongs to the steward user alone (mode 0600);
|
||||
root may `show` and `list` orders but never issue one. An order that had
|
||||
started when the service died is recorded as lost and never run again. No order releases a brake. Acceptance and result are posted to
|
||||
`#operator` as a retried projection of `orders.cc`: a chat outage delays the
|
||||
posts and not the orders. A refusal names the field and a code, never the value
|
||||
it refused.
|
||||
|
||||
## Authority
|
||||
|
||||
**Owner.** Deployment admission, and the continuity of installed bodies.
|
||||
|
|
@ -181,6 +239,32 @@ required_environment = [
|
|||
]
|
||||
```
|
||||
|
||||
A directory artifact is sealed read-only: its files are 0444 and its
|
||||
directories 0555, whatever mode the build or git gave them. A file the tree
|
||||
must run, such as the launcher's `upstream/bin/daemon` child, is declared in
|
||||
`executable_members` and sealed 0555. The members are part of the sealed release
|
||||
address, so changing the list makes a new release. Idunn refuses the release if a
|
||||
member is absolute, climbs with `..`, is missing, is a symlink or sits behind
|
||||
one, or is not a regular file; `executable = true` on a directory output and
|
||||
`executable_members` on a file output are refused too.
|
||||
|
||||
A tree may hold relative symlinks whose `..` steps all come first and climb no
|
||||
higher than the artifact's top (`share/terminfo/4/4025ex -> ../t/tek4025ex` is
|
||||
kept). An absolute link, a `..` after a name, or a climb out of the artifact
|
||||
refuses the release when it is sealed. Host targets (Windows) record
|
||||
`executable` and `executable_members` in the receipt and do not apply them.
|
||||
|
||||
```toml
|
||||
# recipe fragment — the upstream artifact declares the child it launches
|
||||
[[artifacts]]
|
||||
id = "upstream"
|
||||
source_kind = "runner-output"
|
||||
runner = "build"
|
||||
source = "dist/upstream"
|
||||
destination = "upstream"
|
||||
executable_members = ["bin/daemon"]
|
||||
```
|
||||
|
||||
```toml
|
||||
# binding fragment — members connect to the member endpoint, which Idunn
|
||||
# forwards to the private port derived from the candidate: member
|
||||
|
|
|
|||
47
deploy/idunn-orders.service
Normal file
47
deploy/idunn-orders.service
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
[Unit]
|
||||
Description=Idunn steward orders: typed package, unit and deploy orders from the steward
|
||||
# Deliberately no After= or Requires= on any managed target, Synapse or
|
||||
# idunn-yggdrasil: orders are how an operator repairs those, so none of them
|
||||
# may be a condition of this unit starting. Posts are a retried projection and
|
||||
# wait for Synapse on their own.
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
|
||||
# Root, from the same binary as idunn-yggdrasil, with its own store and socket.
|
||||
# idunn-yggdrasil keeps its no-imperative-input invariant and its write set:
|
||||
# no order code runs in `idunn serve`.
|
||||
#
|
||||
# The unit name matches idunn-*, which is in the always-refused set, so no
|
||||
# order can stop or restart this service.
|
||||
User=root
|
||||
# No Group= and no UMask=: the service gives its socket to the steward user
|
||||
# (mode 0600) itself and sets Idunn's own umask for the files it writes, so
|
||||
# nothing here reaches control.cc. The steward user is the only peer the
|
||||
# service answers: it checks SO_PEERCRED on every connection.
|
||||
ExecStart=/usr/local/bin/idunn orders serve \
|
||||
--store /var/lib/gamecult/idunn-orders \
|
||||
--state-store /var/lib/gamecult/idunn/control.cc \
|
||||
--steward-binding /etc/gamecult/idunn/steward.toml \
|
||||
--socket /run/idunn-orders/steward.sock
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStopSec=20
|
||||
|
||||
RuntimeDirectory=idunn-orders
|
||||
RuntimeDirectoryMode=0755
|
||||
|
||||
# The Synapse access token of @idunn, copied from a root-only file, and read
|
||||
# through $CREDENTIALS_DIRECTORY; the service names it in steward.toml.
|
||||
LoadCredential=idunn-matrix-token:/etc/gamecult/idunn/matrix-token
|
||||
|
||||
PrivateTmp=yes
|
||||
ProtectHome=yes
|
||||
ProtectSystem=strict
|
||||
# Its own order store, and the control store a deploy order commits a command
|
||||
# to. Transient units started with systemd-run run in the manager's namespace,
|
||||
# so packages and units are not limited by this list.
|
||||
ReadWritePaths=/var/lib/gamecult/idunn-orders /var/lib/gamecult/idunn
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -45,7 +45,6 @@ ExecStart=/usr/local/bin/idunn serve \
|
|||
--idunn-identity-store /var/lib/gamecult/idunn/idunn-service-identity.cc \
|
||||
--odin-correlation-store /var/lib/gamecult/odin/topology.cc \
|
||||
--odin-trust-anchor /etc/gamecult/idunn/odin-topology-anchor.cc \
|
||||
--deployment-brake-operator-anchor /etc/gamecult/idunn/deployment-brake-operator-anchor.cc \
|
||||
--host-actuator-bind 10.77.0.1:17890 \
|
||||
--source-uid 986 \
|
||||
--source-gid 978
|
||||
|
|
@ -104,6 +103,11 @@ ReadWritePaths=/etc/gamecult/ghostlight/runtime /var/lib/gamecult/ghostlight-wor
|
|||
ReadWritePaths=/etc/gamecult/streampixels-service/runtime /var/lib/gamecult/streampixels-service /srv/streampixels-idunn/releases/service
|
||||
# StreamPixels web is stateless; it needs only its runtime bundle and releases.
|
||||
ReadWritePaths=/etc/gamecult/streampixels-web/runtime /srv/streampixels-idunn/releases/web
|
||||
# Vili (motion worker): runtime bundle, state, and releases. Prefixed "-" so a
|
||||
# target whose roots are not created yet cannot stop Idunn itself from starting
|
||||
# (Idunn startup never depends on a managed target). A root created after Idunn
|
||||
# starts is writable only from Idunn's next restart.
|
||||
ReadWritePaths=-/etc/gamecult/vili/runtime -/var/lib/gamecult/vili -/srv/vili-idunn/releases
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
|
|||
|
|
@ -16,7 +16,6 @@ target="${2:-}"
|
|||
compose=/srv/compose/yggdrasil-apps.yaml
|
||||
brake_store=/var/lib/gamecult/idunn-authority/deployment-brake.cc
|
||||
brake_lock=/var/lib/gamecult/idunn-authority/deployment-brake.cc.lock
|
||||
brake_anchor=/etc/gamecult/idunn/deployment-brake-operator-anchor.cc
|
||||
generation_lock=/var/lib/gamecult/idunn-authority/idunn-generation.lock
|
||||
idunn_provision=/usr/local/bin/idunn-provision
|
||||
|
||||
|
|
@ -154,7 +153,7 @@ deployment_target_policy() {
|
|||
esac
|
||||
}
|
||||
|
||||
require_signed_deployment_grant() {
|
||||
require_deployment_admission() {
|
||||
source_commit="$3"
|
||||
deployment_request_id="$4"
|
||||
release_authority_id="$5"
|
||||
|
|
@ -180,28 +179,17 @@ require_signed_deployment_grant() {
|
|||
[ -z "$release_authority_sha256" ] || { echo "source-observed release cannot claim Bifrost authority digest" >&2; exit 1; }
|
||||
|
||||
require_root_owned_resolved_executable "$idunn_provision"
|
||||
require_root_owned_regular_path "$brake_store"
|
||||
require_root_owned_regular_path "$brake_lock"
|
||||
require_root_owned_regular_path "$brake_anchor"
|
||||
|
||||
# This validation is the deployment-admission point. The signed expiry is a
|
||||
# start-before deadline: it denies a later transaction, but does not become a
|
||||
# timer-shaped veto over one already admitted. The root actuator remains the
|
||||
# sole shared-lock holder through the synchronous consequence, so an explicit
|
||||
# engage/release transition cannot race or rewrite that transaction.
|
||||
exec 9<"$brake_lock"
|
||||
# This validation is the deployment-admission point. An absent or clear stop
|
||||
# admits; an engaged or unreadable one makes the status verb exit nonzero.
|
||||
# The root actuator remains the sole shared-lock holder through the
|
||||
# synchronous consequence, so an engage or clear cannot race or rewrite it.
|
||||
# The lock file is created here when no store has ever been written.
|
||||
exec 9>>"$brake_lock"
|
||||
/usr/bin/flock --shared 9
|
||||
now_unix_millis="$(/usr/bin/date +%s%3N)"
|
||||
case "$now_unix_millis" in
|
||||
''|*[!0-9]*) echo "root actuator clock did not produce Unix milliseconds" >&2; exit 1 ;;
|
||||
esac
|
||||
"$idunn_provision" deployment-brake-status \
|
||||
--store "$brake_store" \
|
||||
--operator-anchor "$brake_anchor" \
|
||||
--runtime-id yggdrasil \
|
||||
--release-id "$source_commit" \
|
||||
--deployment-id "$deployment_request_id" \
|
||||
--now-unix-millis "$now_unix_millis"
|
||||
--runtime-id yggdrasil
|
||||
|
||||
if [ "$minimum_source_revision_required" = true ]; then
|
||||
require_root_owned_resolved_executable /usr/local/bin/idunn
|
||||
|
|
@ -322,7 +310,7 @@ case "$action:$target" in
|
|||
manifest="/srv/odin/deploy-manifests/$target"
|
||||
require_root_owned_regular_path "$manifest"
|
||||
[ -x "$manifest" ] || { echo "root actuator manifest is not executable: $manifest" >&2; exit 1; }
|
||||
require_signed_deployment_grant "$@"
|
||||
require_deployment_admission "$@"
|
||||
export IDUNN_ACTUATOR=1
|
||||
export IDUNN_COMMAND_AUTHORITY=idunn-daemon
|
||||
export IDUNN_SOURCE_COMMIT="$source_commit"
|
||||
|
|
|
|||
|
|
@ -240,7 +240,7 @@ Fencing retries its phase forward on a capped backoff and never stops its live
|
|||
candidate, since that is the only running copy of the admitted body; the
|
||||
deadline opens the same incident. The one exception is yielding: a continuity
|
||||
whose route proof has not succeeded gives way, through its own post-fencing
|
||||
abort, to a deployment the deployment brake has released and that awaits the
|
||||
abort, to a deployment the deployment stop admits and that awaits the
|
||||
handover; a continuity that has proven its route never yields. The
|
||||
`ContinuityExhausted` incident (six charged attempts in an hour,
|
||||
`restarts_page`) pages and gates nothing. When continuity next acts is derived
|
||||
|
|
@ -257,9 +257,14 @@ held until a lift succeeds. It is a migration with a deletion line (follow-up
|
|||
still owe dependency failures not growing the streak and the dead-generation
|
||||
route refresh.
|
||||
|
||||
The deployment brake gates changes to source, release, configuration, schema,
|
||||
unit, or authority binding. The lifecycle brake separately gates restart of an
|
||||
already-admitted incarnation. Neither target brake may gate Idunn itself or an
|
||||
The deployment brake is a stop, not a per-deploy gate: it gates changes to
|
||||
source, release, configuration, schema, unit, or authority binding only while
|
||||
engaged. Its one reader and writer is `src/brake.rs` (`idunn.deployment_stop.v1`:
|
||||
engaged or clear, who, why, when, sequence). An absent or clear store admits;
|
||||
a store that is present but unreadable, foreign, for another runtime, a symlink
|
||||
or owned by another uid fails closed. Nothing is signed, bound to a release or
|
||||
consumed, and one clear admits every deploy until it is engaged again. The
|
||||
lifecycle brake separately gates restart of an already-admitted incarnation. Neither target brake may gate Idunn itself or an
|
||||
unrelated service.
|
||||
|
||||
If Odin is unavailable, Idunn preserves existing routes and continuity for
|
||||
|
|
|
|||
394
docs/deploy-provenance-map.md
Normal file
394
docs/deploy-provenance-map.md
Normal file
|
|
@ -0,0 +1,394 @@
|
|||
# Deployment provenance: map
|
||||
|
||||
Status: map, Imagination, 2026-10-23 (campaign `idunn-deploy-provenance`,
|
||||
session `self-2026-10-23-eureka-b`). The ends are in
|
||||
[`deploy-provenance-target.md`](deploy-provenance-target.md). Questions,
|
||||
rulings, cut specs and follow-ups live in Eureka's mind; this page holds what
|
||||
has no kind: body facts, rationale and sequencing. Line numbers in T1-T12 are
|
||||
at `d74ac8d` (origin/main on 2026-10-23); T4, T9 and T13-T16 are at `fd040f0`,
|
||||
the head of Hands attempt 1, which revision 2 of the spec builds on.
|
||||
|
||||
## Cut `tree-executable-members`
|
||||
|
||||
Spec `idunn-deploy-provenance:cut_spec:cut-tree-executable-members.r2`
|
||||
(revision 2, after Soul pass 1 found the real interpreter tree refused). It
|
||||
serves `motion-pipeline:follow_up:idunn-tree-executable-members` (Vili's
|
||||
admission waits on it) and will serve gamecult-chat C3 too: the presence
|
||||
launcher runs a child such as `upstream/bin/daemon` from inside a tree artifact
|
||||
(README, launcher fragment), and Synapse needs a Python interpreter there.
|
||||
|
||||
### Body facts
|
||||
|
||||
- T1. `ArtifactOutput` (`src/deployment.rs:106-117`) carries one `executable`
|
||||
bool. `ArtifactReceipt` (`src/deployment_plan.rs:731-737`) copies it and
|
||||
`validate` (`:740-760`) requires it to equal the declaration.
|
||||
- T2. `harden_installed_release` (`src/drivers.rs:6848-6855`) builds its
|
||||
executable set from `root.join(receipt.destination)` only. `harden_root_tree`
|
||||
(`:6858-6892`) seals every directory 0555 and every file 0444 unless that
|
||||
exact path is in the set. A directory artifact with `executable = true` is
|
||||
therefore silently meaningless today: its files are all 0444. Nothing refuses
|
||||
it. No live recipe declares one (CodexConnector, Ghostlight, Heimdall, Odin,
|
||||
StreamPixels service and web surveyed): `executable = true` is on files only.
|
||||
- T3. A tree output is copied by `copy_artifact`/`copy_tree`
|
||||
(`src/drivers.rs:8053-8126`), which recreate symlinks verbatim. The digest
|
||||
(`digest_tree`, `:8142-8175`) covers names, file bytes and link targets, not
|
||||
mode bits. So runner or git exec bits never reach the release: harden is the
|
||||
only mode authority.
|
||||
- T4. `validate_release_symlink` (`src/drivers.rs:6897-6916`) keeps a symlink
|
||||
only if its target is relative with no `..`. It runs only inside
|
||||
`harden_root_tree`, at install, so a tree with a `..` or absolute link copies,
|
||||
digests and seals, then fails at `install_release` (Soul s1: copying and
|
||||
digesting the real tree twice took 160 s before harden refused it). The rule
|
||||
is sound but too narrow: it refuses the real interpreter tree (T13). The
|
||||
frozen-source path has a physical resolver (`resolve_frozen_source_symlink`,
|
||||
`:6617`) that walks the filesystem with a 40-link budget and a memo. Its
|
||||
first version was exponential (Soul S4-1, recorded in `d562a12`), and it needs
|
||||
the tree on disk to answer. Revision 2 does not reuse it (T14).
|
||||
- T5. The sealed release id hashes `(schema, plan_id, artifacts,
|
||||
external_inputs)` (`src/deployment_plan.rs:1000-1011`), and `plan_id` hashes
|
||||
the whole plan including `recipe_blob`. A changed member list is therefore
|
||||
already a new plan and a new release. The receipt must still carry the list,
|
||||
because `harden_installed_release` reads receipts only; its two callers
|
||||
(`install_release` `:2788`, `validate_installed_release` `:2822`) re-harden
|
||||
every installed release on each observation.
|
||||
- T6. Encoding is `rmp_serde::to_vec`, positional. A trailing field with
|
||||
`#[serde(default, skip_serializing_if = "BTreeSet::is_empty")]` keeps every
|
||||
existing receipt's bytes and release id: the plan's `deadlines` field
|
||||
(`:574-577`) is the precedent.
|
||||
- T7. `digest_artifact` and `digest_tree` (`:8128-8175`) `fs::read` each file
|
||||
whole. A torch CPU tree's largest file (`libtorch_cpu.so`, hundreds of MB)
|
||||
is held in Idunn's memory three times per install (seal, install, every
|
||||
validate). `hash_frozen_source_tree` (`:6788-6840`) already streams in 64 KiB
|
||||
chunks (S9). The cut collapses both onto one streaming helper.
|
||||
- T8. `require_leaf_path` (`src/deployment.rs:1951-1958`) only keeps the
|
||||
artifact destination a single filename. Members are paths inside that
|
||||
destination, so it does not change. `require_relative_path` (`:1936-1949`)
|
||||
with `allow_current = false` already refuses absolute, `..`, root and prefix
|
||||
components.
|
||||
- T9. The host actuator re-spells the seal (`src/host.rs:1050-1086`): copy,
|
||||
mode check, digest, pinned-digest check and receipt, the same steps as
|
||||
`collect_artifact` (`src/drivers.rs:2305-2326`). Soul s1: deleting its mode
|
||||
check survives every test. Revision 2 collapses both onto one `seal_artifact`
|
||||
in `drivers.rs`. Each caller keeps only how it finds the source path.
|
||||
- T10. The transient unit runs with `ProtectSystem=strict` and the release in
|
||||
`ReadOnlyPaths` (`src/drivers.rs:3675-3698`), `/tmp` mounted `noexec`, and no
|
||||
`MemoryDenyWriteExecute`. Executing a 0555 file from the release works.
|
||||
Shared libraries need no exec bit.
|
||||
- T11. Live body on Yggdrasil: `/usr/local/bin/idunn` sha256 `44cf8a14...`,
|
||||
installed 2026-10-20 02:34 CEST, before `6e2a31c` (02:40 CEST). `src/` is
|
||||
identical between `8760a4f` and `6e2a31c`, so the live code is the
|
||||
reincarnation c2 merge. `origin/main` is 26 commits ahead (+4194/-112 in
|
||||
`src/`: the presence launcher and member route). Installing this cut ships all
|
||||
of them. Idunn's own body changes only through gamecult-ops
|
||||
`scripts/install-idunn-yggdrasil-release.sh` (staged binaries, unit,
|
||||
bindings). The stage at `/home/gamecultadmin/idunn-stage` is stale (2026-09-11).
|
||||
- T13. Link census of the real interpreter tree (Yggdrasil probe of
|
||||
`cpython-3.12.15+20261021-x86_64-unknown-linux-gnu-install_only.tar.gz`,
|
||||
sha256 `5ff9f060...77191124`; torch adds no links, as Soul s1 counted 1,049
|
||||
after pip): 1,049 links. 0 are absolute, 749 have no `..`, and 300 have all
|
||||
their `..` components before the first name (every one is
|
||||
`share/terminfo/<c>/<name> -> ../<c2>/<name2>`, one `..` each). 0 have a `..`
|
||||
after a name, 0 climb above the tree, 0 dangle and 0 resolve outside it.
|
||||
Blender 5.2.2 has 79 links, none absolute or with `..` (Soul s1).
|
||||
- T14. Lexically normalising `<link's directory>/<target>` does not check
|
||||
containment. The kernel resolves a target one component at a time, so a `..`
|
||||
after a symlinked name climbs from where that link points, not from where its
|
||||
name sits. Probe on Yggdrasil: `root/a/s -> ..` and `root/a/L -> s/../x`
|
||||
normalise to `root` and `root/a/x`, both inside, yet `L` reads the file `x`
|
||||
beside `root`. `root/a/d -> .` with `root/a/M -> d/../../x` escapes the same
|
||||
way. Idunn has met this before: verify-transaction-cut F5 (`D -> .`,
|
||||
`L -> D/D/../../../etc/passwd` passed a lexical check).
|
||||
The sound lexical rule constrains the target's shape: all its `..` components
|
||||
come before its first name, and they number no more than the link's depth
|
||||
below the artifact's top. The climb then starts from the link's own
|
||||
directory, which the walk reached through real directories only (copy and
|
||||
digest never descend through a link), so it lands on a real ancestor inside
|
||||
the artifact. After that the target only descends, and any link it passes
|
||||
through obeys the same rule. By induction on the kernel's hops (capped at 40,
|
||||
then ELOOP), every hop stays inside. The check is one pass over the target's
|
||||
components with no filesystem reads, so its cost is linear in the tree and no
|
||||
resolver is needed. The shapes it refuses (`s/../x`) occur in neither real
|
||||
tree (T13).
|
||||
Prior art, from documentation as known to Imagination (high confidence for Go
|
||||
and securejoin, medium for the tar details):
|
||||
- Go `path/filepath.IsLocal` (1.20) is lexical and says so: it does not account
|
||||
for symlinks. It would admit `s/../x`. Used alone, it is the T14 bug.
|
||||
- Go `os.Root` (1.24) and `cyphar/filepath-securejoin` (used by runc,
|
||||
containerd and umoci) resolve each component inside the root when a path is
|
||||
opened, refusing or clamping an escape. They guard one later access, need
|
||||
the tree on disk, and check again on every open. Idunn instead certifies a
|
||||
sealed tree once, so that the kernel's own resolution stays inside on every
|
||||
later access.
|
||||
- Zip-slip defences and archive extractors check entry names lexically and
|
||||
protect against link targets by refusing to write through links
|
||||
(libarchive's `ARCHIVE_EXTRACT_SECURE_SYMLINKS`, and `SECURE_NODOTDOT` for
|
||||
names; GNU tar creates such links last). They protect the extraction, not
|
||||
what a kept link points at afterwards.
|
||||
- OCI layer extraction keeps link targets verbatim, absolute ones included,
|
||||
because the runtime resolves them inside the container's root. Idunn runs a
|
||||
release in the host's mount namespace, so absolute targets stay refused.
|
||||
- Nix keeps link targets verbatim and tracks dependencies by scanning for
|
||||
references. The whole store is its root, so it has no per-artifact
|
||||
containment to compare with.
|
||||
None of these is a static per-link rule for a tree that runs unconfined. The
|
||||
T14 shape rule fits that case.
|
||||
- T15. Host targets: `idunn-host` observes, starts and stops processes on
|
||||
Windows only (the non-Windows `platform` module in `src/host.rs` bails), and
|
||||
`host.rs` sets no file modes. On Windows a file runs by its type, not by a
|
||||
mode bit, so `executable` and `executable_members` have nothing to set there.
|
||||
The file `executable` flag has always been carried and never applied on host
|
||||
targets. A recipe does not know which workload kind its binding will choose.
|
||||
- T16. The release root holds only artifact destinations. Runner workspaces,
|
||||
where external inputs land, are removed before `SealedRelease::new`
|
||||
(`src/drivers.rs:2421-2423`). `install_release` (`:2781-2788`) and
|
||||
`validate_installed_release` (`:2815-2822`) digest every receipt's
|
||||
destination before calling harden. So a rule in the digest walk runs at seal,
|
||||
at install and at every validate, over every link harden would see.
|
||||
- T12. Disk: `/` on Yggdrasil has 1.3 TB free. A tree release costs about three
|
||||
copies (runner workspace, staging, installed). Nothing collects old releases or
|
||||
staging (`ghostlight-verse:follow_up:idunn-generation-retention`), so every
|
||||
Vili deploy leaves about 7 GiB behind.
|
||||
|
||||
### Contract, corrected against the body
|
||||
|
||||
- Any directory artifact may declare `executable_members` (a set of relative
|
||||
paths). The follow-up limited it to runner-output; that limit protects no
|
||||
invariant, and the launcher's worktree or fetched trees need the same feature.
|
||||
- `executable = true` names a file artifact. On a directory output the release
|
||||
is refused, and `executable_members` on a file output is refused (T2).
|
||||
- A member is refused at recipe parse if it is empty, absolute, or holds `.`,
|
||||
`..`, a root or a prefix. At seal it is refused if it is missing, is a
|
||||
symlink, passes through a symlink or non-directory, or is not a regular file.
|
||||
Refusal happens before `SealedRelease::new`, so no sealed release can name a
|
||||
member it cannot seal.
|
||||
- Harden seals each member 0555 and fails if any declared executable path was
|
||||
not visited as a regular file. "Every declared executable is sealed 0555, or
|
||||
the install fails" holds structurally, not by a lookup that can miss.
|
||||
- Symlinks: a link in a tree artifact is kept when its target is relative, its
|
||||
`..` components all come before its first name, and they number no more than
|
||||
the link's depth below the artifact's top (T14). One function,
|
||||
`check_release_link`, decides this. Only the digest walk calls it, so a bad
|
||||
link refuses the release at seal, before `SealedRelease::new`, and again at
|
||||
install and validate (T16). Harden no longer judges links. The climb is
|
||||
bounded by the artifact's top, not the release root: no live release has a
|
||||
`..` link (the old rule refused them all), so nothing installed is newly
|
||||
refused, and each artifact's digest describes a closed tree.
|
||||
- Host targets carry `executable` and `executable_members` in the receipt,
|
||||
shape-checked at seal by the shared `seal_artifact`, and do not apply them
|
||||
(T15). Refusing members there would make a recipe's validity depend on its
|
||||
binding.
|
||||
- Vili's trees must
|
||||
not come from `python -m venv`: its `bin/python` is an absolute link and its
|
||||
`pyvenv.cfg` names the build workspace. Install packages into the relocatable
|
||||
interpreter tree instead (`motion-pipeline:follow_up:vili-relocatable-python-tree`).
|
||||
|
||||
### Authority
|
||||
|
||||
Owner: the seal (`seal_artifact`, called by `collect_artifact` for systemd
|
||||
targets and by `materialize` in `host.rs` for host targets) decides
|
||||
membership from the recipe and writes it into the receipt. Harden enforces the
|
||||
receipt and decides nothing. Runner exec bits and git `100755` modes are dead
|
||||
inputs for release modes. Link containment belongs to `check_release_link`,
|
||||
which only `digest_tree` calls. `validate_release_symlink` is deleted, and
|
||||
harden's symlink arm leaves links as they are.
|
||||
|
||||
### Acceptance sizing
|
||||
|
||||
The big-tree job (spec `verification.operator`) needs about 12 GiB of scratch
|
||||
on Yggdrasil: a 3.5 GiB tree, its staging copy, downloads and pip cache. It
|
||||
removes it at the end. Revision 2 runs it on the trees as shipped, with the 300
|
||||
terminfo links (T13). If a tree holds a link the T14 rule refuses, the job
|
||||
prints it and stops, and the link is reported to Self, not deleted.
|
||||
|
||||
## Cut `isolation-credential-evidence`
|
||||
|
||||
Spec `idunn-deploy-provenance:cut_spec:cut-cut-isolation-credential-evidence.r1`,
|
||||
at `112a7a5` (origin/main, live on Yggdrasil since 2026-10-24 11:29 UTC). It
|
||||
serves Vili's admission (`motion-pipeline:cut_report:cut-vili-admission.h4`,
|
||||
first deviation): every `idunn up vili` is refused before fencing.
|
||||
|
||||
### Body facts (probed read-only on Yggdrasil, 2026-10-24)
|
||||
|
||||
- I1. `idunn up vili` runs 6 and 7 failed with
|
||||
`systemd workload isolation properties differ from the Idunn contract`, the
|
||||
message of `observe_unit` (`src/drivers.rs:2975-3005`). Of its conjuncts,
|
||||
only one differs on the failed Vili unit: `LoadCredential`. Idunn expects
|
||||
`""` because Vili has no service credential besides the presence identity
|
||||
(`service_credential_names` empty); systemd shows `[unprintable]`. Every
|
||||
other conjunct matches, InaccessiblePaths included (one value across all
|
||||
five idunn-vili/odin units).
|
||||
- I2. systemd 257.13 `systemctl show -p LoadCredential` prints
|
||||
`LoadCredential=[unprintable]` for every unit, whatever the credentials:
|
||||
`ssh.service` and `systemd-journald.service` (not DynamicUser, zero
|
||||
LoadCredential), the four failed `idunn-vili-*` units (zero), and the Odin
|
||||
unit (one). The display form carries no information, so the conjunct was
|
||||
vacuous whenever credentials existed and always false when none did. Odin
|
||||
and StreamPixels passed only because each carries a regular credential.
|
||||
- I3. The exact value is readable: D-Bus
|
||||
`org.freedesktop.systemd1.Service.LoadCredential` (`busctl get-property`)
|
||||
answers `a(ss) 0` for the Vili units and ssh, `a(ss) 1
|
||||
"ODIN_TOPOLOGY_IDENTITY" "<path>"` for Odin.
|
||||
- I4. What the workload can actually read is the delivered directory. On the
|
||||
host, `/run/credentials/` holds a directory only for units with credentials
|
||||
(getty, journald, Odin, StreamPixels service and web). Inside Odin's MainPID
|
||||
root, `/run/credentials/<unit>/` holds exactly `ODIN_TOPOLOGY_IDENTITY`
|
||||
(`-r--r-----+ root root`). The presence identity is a parent-only `OpenFile`
|
||||
descriptor, never a credential.
|
||||
- I5. The rule arrived whole in `1fc0dad` (2026-09-03, "Rebuild Idunn
|
||||
deployment transaction authority"), with its twin clause in
|
||||
`validate_launch_observation` (`:3468-3472`, `:3513-3516`); no later commit
|
||||
touched it. No ruling or finding in any campaign concerns workload isolation
|
||||
verification; gamecult-chat's foreign-workloads cut touched
|
||||
`CREDENTIALS_DIRECTORY` reservation, not this check.
|
||||
- I6. `SystemdWorkloadObservation` is persisted by previous Idunns in both
|
||||
MessagePack encodings, compact (positional) included
|
||||
(`persisted_systemd_observations_decode_as_the_systemd_variant`,
|
||||
`:9915-9930`), so deleting the `load_credential` field shifts every later
|
||||
slot of a live incumbent's record. The field is also in the drift list
|
||||
(`:2462`) that `stop` (`:4064-4070`) uses to refuse stopping a changed
|
||||
workload: changing what the field holds without leaving that list would make
|
||||
the next Idunn refuse to stop the live Odin and StreamPixels incumbents.
|
||||
- I7. Fixtures: the verbatim 257.13 `systemctl show` output of base's 27
|
||||
properties for a Vili unit (presence identity only) and the Odin unit
|
||||
(presence identity plus one service credential), captured 2026-10-24 into
|
||||
this Imagination's scratchpad `fixtures/`.
|
||||
- I8. The new rule holds for every live systemd incumbent: inside each
|
||||
MainPID root the delivered entries equal its binding's `secret_files` keys
|
||||
minus `GAMECULT_RUNTIME_PRESENCE_IDENTITY` (Odin 1, StreamPixels service 11,
|
||||
StreamPixels web 1; key names compared, no values read). An Idunn restart onto
|
||||
this cut re-observes them (`:3955-3970`) and passes.
|
||||
|
||||
### Decision
|
||||
|
||||
A property systemd cannot display does not belong in the comparison. The
|
||||
invariant it reached for, that the workload holds exactly the credentials Idunn
|
||||
delivered, is observed where it is decided: the entries of
|
||||
`/run/credentials/<unit>` in the MainPID's root equal Idunn's service
|
||||
credential names, absent directory equal to none. That covers
|
||||
`LoadCredential`, `SetCredential`, `ImportCredential` and drop-ins alike, and
|
||||
needs no new program. The D-Bus read (I3) is exact but sees only one of those
|
||||
properties and adds a second systemd client and parser; rejected.
|
||||
|
||||
### Authority
|
||||
|
||||
Owner: `observe_unit` decides the credential rule, from the delivered
|
||||
directory. `LoadCredential`'s display form is no longer read or compared; the
|
||||
`load_credential` field is a retired wire slot, written empty, out of the drift
|
||||
list, kept only so persisted records decode. Forbidden writers: the
|
||||
`observe_unit` conjunct and the `validate_launch_observation` clause. Dropping
|
||||
the slot waits on the next schema revision of the observation.
|
||||
|
||||
### Deploy
|
||||
|
||||
Idunn's body changes only through gamecult-ops
|
||||
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||
bindings unchanged). After it, Vili's Hands reruns from step 4 of
|
||||
`motion-pipeline:cut_report:cut-vili-admission.h4`'s undone list.
|
||||
|
||||
## Cut `runner-failure-evidence`
|
||||
|
||||
Spec `idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1`, at
|
||||
`804ed75` (origin/main, live on Yggdrasil). It serves the StreamPixels-area
|
||||
Self: `idunn up vili` (command `up-4866467c-ee98-4a2b-b4a9-a008d7c44acd`,
|
||||
transaction `tx-a04b68c7-c742-4d7d-8220-8aa08b9febae`, 2026-10-25 00:50-00:55
|
||||
UTC) failed in Sealing with `Docker runner exited with exit status: 1: ` and
|
||||
nobody could tell which recipe step failed.
|
||||
|
||||
### Body facts (source read at `804ed75`; host read-only, 2026-10-25)
|
||||
|
||||
- R1. The runner is `docker run --rm` (`src/drivers.rs:2088-2090`), launched by
|
||||
`DockerRunnerDriver::docker` (`:2173-2195`) with stdin null and
|
||||
`Command::output()`: stdout and stderr are each buffered whole in the
|
||||
daemon's memory, unbounded. On a non-zero exit the error is
|
||||
`"Docker runner exited with {status}: {stderr}"`; stdout is dropped. The
|
||||
container is gone the moment it exits, so `docker ps -a` and `docker logs`
|
||||
hold nothing.
|
||||
- R2. The empty message: the failing process wrote nothing to stderr, and
|
||||
stdout, the only stream that carried its report, is discarded at
|
||||
`:2182-2193`. The same interpolation echoes whatever a step does write to
|
||||
stderr into `last_error`, `pre_fencing_abort.error` and
|
||||
`completion.error` (`bounded_detail`, 2048 bytes), and from there into
|
||||
`idunn up`'s `deployment failed:` line and `idunn status`.
|
||||
- R3. No step identity reaches the error. `materialize` (`:2368-2391`) loops
|
||||
over `declaration.steps` and calls `run_in_workspace` with argv only;
|
||||
`step.id` is used only in the environment precondition. External inputs
|
||||
(`curl` in the runner, `:2247-2258`) fail through the same helper with the
|
||||
same anonymous message.
|
||||
- R4. Which step failed on 2026-10-25, inferred from the failed transaction's
|
||||
staging root, which survives (926 MiB,
|
||||
`/var/lib/gamecult/idunn/staging/tx-a04b68c7-.../`): in `.runner-node-build`,
|
||||
`node_modules/.package-lock.json` was written at 00:51:24 (so
|
||||
`install-vili-dependencies`, `npm ci`, finished), nothing outside
|
||||
`node_modules` is newer, and `release/vili` (the next step's output) is
|
||||
absent; the python workspace holds only fetched inputs. The failing step was
|
||||
`test-daemon` (`node --test ... scripts/test-vili-daemon.mjs`), whose test
|
||||
reporter writes to stdout: exactly the stream R1 discards. Inference from
|
||||
residue, not from output.
|
||||
- R5. The failure path: `resume_one_transaction` (`src/control_plane.rs:6291-6320`)
|
||||
sends a pre-fence error to `begin_pre_fencing_abort` (`:9916-9935`), which
|
||||
persists `bounded_detail(format!("{error:#}"))`. Nothing reaches the journal
|
||||
on this path: the idunn-yggdrasil journal for 00:45-01:10 holds only
|
||||
provider-exclusion lines.
|
||||
- R6. Stores. `control.cc` (root 0640) holds commands, transactions
|
||||
(`idunn.deployment_transaction.v4`, keys 0-36), generations, supervision and
|
||||
expiry requests; `ControlSnapshot::read` refuses any other type
|
||||
(`:3420`). Records must re-encode byte-exactly (`decode_record`,
|
||||
`:3717-3727`), so a new transaction field is a schema bump and a v4 lift
|
||||
(`:4100-4110`). Terminal transactions move verbatim to `history.cc`
|
||||
(`:4366-4396`; 39.8 MB on the host), which status reads for `--command`.
|
||||
`incidents.cc` lives in the published projection directory and by its own
|
||||
doc (`src/incident.rs:1-11`) carries no free text and is never `control.cc`,
|
||||
because a type added there stops continuity for every target on a rollback.
|
||||
- R7. The daemon runs as root (`User=root`). Runner secrets reach the
|
||||
container as read-only file mounts named by env (`:2041-2047`), so any text
|
||||
a step derives from them can land in its output.
|
||||
- R8. Staging residue: 475 transaction roots, 55 GiB under
|
||||
`/var/lib/gamecult/idunn/staging`; a failed Sealing leaves its workspaces
|
||||
because `materialize` returns before `remove_tree_inside`.
|
||||
|
||||
### Decision
|
||||
|
||||
The failure becomes a typed value at the runner, `RunnerStepFailure`: the
|
||||
subject (recipe step or external input, by id), the runner id, the exit code or
|
||||
signal, and the last 8 KiB of stdout and of stderr with each stream's total
|
||||
byte count. Its `Display` carries the subject and status, never output.
|
||||
Capture streams both pipes through bounded tails, so a verbose build no longer
|
||||
buffers whole in the daemon.
|
||||
|
||||
The tail lives in a new private store, `runner-evidence.cc`, beside
|
||||
`control.cc` (mode 0600), one record per transaction
|
||||
(`idunn.runner_failure.v1`), newest 32 kept by the writer. Not in the
|
||||
transaction record: that is a v5 bump whose rollback makes every older Idunn
|
||||
refuse its control store (R6), the scar `incident.rs` already names. Not in
|
||||
`incidents.cc`: published, and free text is barred there. Not in the journal:
|
||||
journal readers are a wider set than root, so the journal line names the
|
||||
transaction, the subject, the status and the store, and no output.
|
||||
|
||||
No redaction. Matching secret bytes would make the daemon read every secret,
|
||||
and still misses any encoding, prefix or transform of one; a claimed redaction
|
||||
that leaks is worse than a stated rule. The rule instead is the reader set: the
|
||||
store's only readers (root) can already read every secret the runner was
|
||||
given.
|
||||
|
||||
Keeping the failed container (dropping `--rm`) is worse: its output then lives
|
||||
in Docker's json-file log, unbounded, untyped, outside Idunn's state and
|
||||
readable by the docker group; containers accumulate with no owner to reap them;
|
||||
`docker system prune` erases the evidence; and nothing ties a container to the
|
||||
transaction in `idunn status`.
|
||||
|
||||
### Authority
|
||||
|
||||
Owner: the runner decides what failed and keeps the bounded tail; the engine's
|
||||
pre-fence failure path is the one writer of `runner-evidence.cc`.
|
||||
`last_error`, `pre_fencing_abort.error` and `completion.error` keep only the
|
||||
failure's `Display`. Forbidden writers: the `stderr` interpolation in
|
||||
`DockerRunnerDriver::docker`. Retention is by count inside the writer, so the
|
||||
store needs no external owner; staging roots (R8) stay with
|
||||
`ghostlight-verse:follow_up:idunn-generation-retention`.
|
||||
|
||||
### Deploy
|
||||
|
||||
Idunn's body changes only through gamecult-ops
|
||||
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||
bindings unchanged). The next failed Sealing then shows its step and tail on
|
||||
`sudo idunn status --command <id>`.
|
||||
|
|
@ -4,6 +4,15 @@ Status: target, Self, 2026-10-19 (campaign `idunn-deploy-provenance` in
|
|||
Eureka's mind). It states the ends. The map will live in
|
||||
`deploy-provenance-map.md`.
|
||||
|
||||
Live as of cut `brake-is-a-stop` (2026-11-13): the deployment brake is the
|
||||
stop of invariant 3 and the signed per-deploy release is gone. `src/brake.rs`
|
||||
reads and writes it; `idunn-provision deployment-brake-engage|clear|status`
|
||||
take only `--store`, `--runtime-id` and `--reason`; the writer's login name and
|
||||
the verb's clock are recorded; an absent or clear store admits. The
|
||||
transaction stamps `deployment_admitted_at_unix_millis` (record v6). Still
|
||||
planned: the typed authority provenance on the transaction (invariant 2), and a
|
||||
steward-scoped store.
|
||||
|
||||
## Why
|
||||
|
||||
Operator, 2026-10-19:
|
||||
|
|
@ -53,6 +62,11 @@ In:
|
|||
(`heimdall-discord-launch.md`, `idunn-host-raven.md`, the StreamPixels
|
||||
runbook).
|
||||
|
||||
- A static release (a sealed tree published by repointing a link, with no
|
||||
workload): its transaction records `flip` (release, previous release, the
|
||||
sha256 of the carried per-site file) beside the brake bytes. See
|
||||
`deployment-authority.md`, "Static release".
|
||||
|
||||
Out:
|
||||
|
||||
- `idunn-reincarnation`. This campaign is mapped after that campaign's cut 2
|
||||
|
|
|
|||
|
|
@ -48,8 +48,8 @@ and health. CultMesh carries typed projections between them.
|
|||
scaling, restart, host reboot recovery, and crash recovery use the same
|
||||
source-to-recipe-to-artifact provenance and the same distinct write-lease and
|
||||
route-membership primitives.
|
||||
Deployment may change that chain only while the deployment brake admits the
|
||||
exact transaction. Continuity may restart only the already-admitted chain and
|
||||
Deployment may change that chain only while the deployment stop admits it
|
||||
(absent or clear; an engaged stop halts). Continuity may restart only the already-admitted chain and
|
||||
is governed by a separate lifecycle brake.
|
||||
- **Deletion line:** hard-coded target copies, raw `deploy_command` and
|
||||
`restart_command` state, `sh -c` actuation, root Git inspection, staged
|
||||
|
|
@ -171,6 +171,96 @@ different semantic capability.
|
|||
`gamecult-ops` publishes these fleet bindings and provisions host substrate. It
|
||||
does not own application build or deployment programs.
|
||||
|
||||
## Static release
|
||||
|
||||
A target whose product is a sealed tree, with no process to run (a built web
|
||||
front end), declares `[static_release] artifact = "<id>"` in place of
|
||||
`[service]`, and its binding declares `driver = "static-release"` with a
|
||||
`release_root`, a `current_link` and one `[workload.carry]`. It is the same
|
||||
transaction as any deployment, in three phases: Sealing (source, build, seal,
|
||||
install: unchanged), Starting (the one external change, behind the deployment
|
||||
brake) and Complete. It never enters Warming through Committing, has no
|
||||
Expected projection, no activation and no workload, publishes nothing to
|
||||
Odin, and its Complete creates no admitted generation, so continuity never
|
||||
supervises it. It declares no state, capabilities, dependencies, conflicts,
|
||||
route or write lease, and its rollout is `candidate-then-promote` with at least
|
||||
two retained releases.
|
||||
|
||||
What Starting does, in three durable steps recorded as `flip` on the
|
||||
transaction: the release to name and the release the link names now are
|
||||
written first; then Idunn copies the operator's carry source into
|
||||
`<release_root>/<sealed release id>/<carry.destination>` (mode 0444) and
|
||||
renames a fresh symlink over `current_link`; then it observes the link naming
|
||||
the release and completes. A failure before the rename ends the transaction
|
||||
through the ordinary pre-fencing abort with `current_link` untouched. A crash
|
||||
resumes at the recorded intent and repeats the flip, which is harmless.
|
||||
|
||||
The carry is a copy of one per-site file (for example `config.json`) taken at
|
||||
flip time. It sits beside the artifacts, never inside one, so it is outside
|
||||
every sealed digest: changing the site file changes no release id, and the
|
||||
sha256 of the bytes carried is provenance on the transaction
|
||||
(`flip.carried_sha256`), beside the brake bytes and the requester. The
|
||||
destination is a single leaf name that no artifact uses. The web server reads
|
||||
`current_link/<artifact destination>` for the tree and `current_link/<carry
|
||||
destination>` for the carried file (a one-time root edit of the vhost, an
|
||||
`alias` for the carried file beside the root for the tree). A missing or
|
||||
non-regular carry source fails the deployment before anything moves.
|
||||
|
||||
What `current_link` names is observed from the symlink and confirmed by the
|
||||
latest Admitted static transaction's `flip.release_id`; it is stored nowhere
|
||||
else. A rollback is a flip back to `flip.previous_release_id`: release
|
||||
directories stay, and Idunn prunes none. The deployment stop is read exactly as for a
|
||||
service, by the binding's runtime id. Release ids are per transaction, so
|
||||
redeploying an unchanged tarball with a changed site file is a new release that
|
||||
carries the new file, never an in-place re-carry of an existing one: the carry
|
||||
is part of a release. If `current_link` is replaced by a directory or by a link
|
||||
that names nothing under the release root after the flip was recorded, the
|
||||
transaction keeps retrying with backoff: `idunn status --command <id>` shows the
|
||||
error as "waiting on ...", operator cancel is refused (Starting, not held), and
|
||||
the fix is by hand (restore the link or remove the directory); it then
|
||||
completes. A bounded retry is follow-up work. A new
|
||||
upstream tarball enters as a recipe `external_input` with a pinned url and
|
||||
sha256, so each release is a recipe commit whose pin is its provenance.
|
||||
|
||||
### Rollback
|
||||
|
||||
`idunn rollback <target> [--state-store PATH] [--no-wait]` writes a
|
||||
`rollback-` command and nothing else. The daemon freezes it into one new
|
||||
transaction (`CommandKind::Rollback`) that reuses the previous release exactly:
|
||||
the plan, sealed release and installed release are copied from the transaction
|
||||
that flipped it, so there is no source, no build and no source cleanup, and it
|
||||
creates no admitted generation. It starts at Sealing with that evidence and
|
||||
reaches the same deployment stop as a deploy, which admits it unless engaged;
|
||||
then `advance_static` flips, with the same three durable steps, the same
|
||||
nothing-aborts-after-the-rename rule and the same stuck-flip behavior.
|
||||
|
||||
The previous release is derived, never stored: the newest Admitted static
|
||||
transaction for the target (control.cc and history.cc) names the release
|
||||
`current` must point at (`flip.release_id`) and the one it displaced
|
||||
(`flip.previous_release_id`); the newest transaction that flipped that release
|
||||
supplies the evidence. A rollback of a rollback therefore flips forward again.
|
||||
The command is refused, recorded as a rejected transaction, when the target has
|
||||
no completed static release, when the newest release is the first one, when
|
||||
`current` does not name the newest flip, when the previous release directory is
|
||||
gone, or when an artifact in it no longer digests to its sealed receipt. A
|
||||
failed check at flip time ends the transaction before fencing with `current`
|
||||
untouched.
|
||||
|
||||
The rollback reuses the old plan verbatim, so its binding snapshot, brake path
|
||||
and carry source are the old ones: if the operator moved the site file the
|
||||
rollback fails before the link moves. The carry is re-applied from the site file
|
||||
as it is at rollback time, and its sha256 is recorded in the rollback's own
|
||||
flip evidence (it may differ from the original release's; the copy inside the
|
||||
rolled-back-to release is replaced). Nothing prunes release directories.
|
||||
|
||||
`CommandKind::Rollback` is stored in the existing v5 command and transaction
|
||||
records with no schema change; an Idunn at 9f15cee or earlier cannot decode
|
||||
them, so this is forward-only, like v5 itself. The newest flip is ordered by
|
||||
`flipped_at` wall clock, and the observed-link check makes a clock step refuse
|
||||
the rollback rather than mis-flip. A rollback cannot be cancelled, like a static
|
||||
deploy. A rollback is also refused when the previous release was deployed under
|
||||
a different release root or current link than the newest one.
|
||||
|
||||
## Typed deployment state
|
||||
|
||||
Mutable control-plane state will be CultCache. Runtime integration must use
|
||||
|
|
@ -209,6 +299,82 @@ names both content addresses and the executable artifact digest. Present and
|
|||
Ready remain Odin-owned correlations over independently verified runtime
|
||||
evidence; the deployment-plan module deliberately does not model them.
|
||||
|
||||
## The deployment stop
|
||||
|
||||
`src/brake.rs` is the only reader and writer of the stop. A stop halts work
|
||||
that has not crossed an irreversible step, and it is read again at each such
|
||||
step: Sealing (`advance_sealing`), the static flip intent (`advance_static`) and
|
||||
the entry into Fencing for a deploy (`advance_warming`). An engaged stop there
|
||||
makes the transaction wait with a note naming who engaged it and why; nothing
|
||||
aborts and nothing moves. After the flip intent is recorded, or once a deploy
|
||||
has fenced, the transaction runs to its end: a stop halts before the
|
||||
irreversible step, never mid-fence or mid-flip. Continuity is never gated.
|
||||
The sequence of a stop record is positive and advances by one per verb; nothing
|
||||
reads it for authority, so it is not validated further.
|
||||
|
||||
Absent means the store file is missing in an existing directory. Anything else
|
||||
Idunn cannot judge (a symlink, a zero-length or undecodable file, another
|
||||
runtime's record, another owner, an unsearchable or non-directory parent) reads
|
||||
as unreadable and fails closed; the wait note carries the
|
||||
`idunn-provision deployment-brake-clear` command that replaces it.
|
||||
|
||||
A binding may name a second stop, the target's steward:
|
||||
|
||||
```toml
|
||||
[brakes.steward]
|
||||
user = "mind-streampixels" # a login, never root
|
||||
store = "/var/lib/gamecult/idunn-steward/streampixels/deployment-brake.cc"
|
||||
```
|
||||
|
||||
Deploys admit only when every stop admits. The steward's store is read as owned
|
||||
by that user's uid: the owner uid proves who wrote the file, and the wait note
|
||||
names the stop by the binding's steward user, never by the record's `by` (which
|
||||
is the verb's own text). Idunn never writes the steward directory and no key,
|
||||
group or ACL is involved. The operator's store is a separate stop the steward
|
||||
cannot clear, and the wait note names each holding stop.
|
||||
|
||||
The steward owns its directory, so it could unlink any file in it. Binding
|
||||
validation therefore refuses a steward store whose directory is shared with,
|
||||
inside or around the directory of any other Idunn authority store (deployment
|
||||
and lifecycle brakes, trust anchor, write-lease record, release authority).
|
||||
Validation cannot see the file system: the root who provisions the directory
|
||||
makes it contain the steward's store and nothing else.
|
||||
|
||||
A steward store owned by another uid, or a steward user that does not resolve
|
||||
to a non-root uid, is unreadable and fails closed. Only the steward (the
|
||||
owner uid) can repair it: a clear run as root leaves a root-owned file that is
|
||||
still unreadable.
|
||||
|
||||
### Provisioning a steward (root, once per target)
|
||||
|
||||
1. Create the store directory, owned by the steward and holding nothing else:
|
||||
`install -d -o mind-streampixels -g mind-streampixels -m 0755
|
||||
/var/lib/gamecult/idunn-steward/streampixels`. It must not be, contain or sit
|
||||
inside any other authority directory, and no workload `ReadWritePaths` may
|
||||
cover it. Idunn (root) reads it without further grants; no group or ACL.
|
||||
2. Add the stanza above to the target's operator binding, after `[brakes]`,
|
||||
with the steward's real login. Deploys planned afterwards see it.
|
||||
3. The steward engages and clears as itself, never through root:
|
||||
`idunn-provision deployment-brake-engage --store <store> --runtime-id <id>
|
||||
--reason <text>` and `deployment-brake-clear` with the same flags.
|
||||
4. The operator reads it with `idunn-provision deployment-brake-status --store
|
||||
<store> --runtime-id <id> --owner-user mind-streampixels`.
|
||||
|
||||
The binding is sealed into each plan. A deploy already planned keeps the
|
||||
binding it was sealed with: editing the stanza does not apply to it. If the
|
||||
steward's login is deleted or its directory is lost, that deploy stays
|
||||
unreadable-held at Sealing (or the static flip intent). `idunn cancel` withdraws
|
||||
only a deploy held for a readiness disagreement before Fencing, and `idunn
|
||||
expire` only one in Warming through Committing, so neither frees a deploy held
|
||||
by the stop. Recovery is to restore the steward's login or recreate the
|
||||
directory owned by it (and have the steward clear its store).
|
||||
|
||||
Record v6 (`deployment_admitted_at_unix_millis`, key 38) is forward-only: an
|
||||
Idunn that predates v6 cannot read a migrated `control.cc`, so, as for v5 (see
|
||||
`route-continuity-cut.md`, "An older Idunn refuses the v5 store"), take a
|
||||
byte-compared pre-install backup of `control.cc` and `history.cc` with Idunn
|
||||
stopped, and roll back by restoring the backup, never by swapping the binary.
|
||||
|
||||
## Initial drivers and replaceable boundary
|
||||
|
||||
The first Yggdrasil implementation uses:
|
||||
|
|
@ -247,7 +413,7 @@ For a singleton incarnation:
|
|||
declaration, binding, launch contract, and input digests.
|
||||
5. Derive the release-bound Expected incarnation from the validated sealed
|
||||
release and its exact compiled plan. Publish it only once the deployment
|
||||
brake has admitted this exact transaction, under the candidate's own
|
||||
stop has admitted this transaction (stamping `deployment_admitted_at_unix_millis`), under the candidate's own
|
||||
incarnation key, beside whatever the target's admitted incarnation projects.
|
||||
6. Issue one opaque runtime-instance activation bound to the Expected digest,
|
||||
pass the exact Expected and activation documents through the actuator's
|
||||
|
|
@ -270,8 +436,8 @@ For a singleton incarnation:
|
|||
and stop the incumbent.
|
||||
|
||||
Sealing resolves the source, freezes it, runs every recipe step, and installs
|
||||
the release before the deployment brake is consulted at all (`advance_sealing`).
|
||||
The brake gates step 5, the first Verse-visible change: publishing the
|
||||
the release before the deployment stop is consulted at all (`advance_sealing`).
|
||||
The stop gates step 5, the first Verse-visible change: publishing the
|
||||
candidate's Expected incarnation on the move into `Starting`. Build and test
|
||||
already run unbraked, inside the sealing phase.
|
||||
|
||||
|
|
@ -302,7 +468,7 @@ identity, and route membership. Every restart receives a new runtime-instance
|
|||
activation and, for a state writer, a new process-bound lease only after the new
|
||||
warming presence is observed. A lease from the dead process is never reused. A
|
||||
continuity restart does not fetch a branch, rebuild, rerun a migration, alter
|
||||
the launch contract, or consume the deployment brake. An explicit lifecycle
|
||||
the launch contract, or pass the deployment stop. An explicit lifecycle
|
||||
brake may suspend this actuation without changing deployment authority.
|
||||
|
||||
Continuity never stops. Six charged attempts in an hour only open the
|
||||
|
|
@ -335,7 +501,7 @@ the admitted body. Both operator cases open an `operator-required` incident
|
|||
on.
|
||||
|
||||
The one yield: a continuity whose route proof has not succeeded gives way to a
|
||||
deployment the deployment brake has released that awaits the handover of the
|
||||
deployment the deployment stop admits that awaits the handover of the
|
||||
target. It begins its own post-fencing abort and the deployment fences what is
|
||||
admitted. A continuity that has proven its route is serving never yields.
|
||||
|
||||
|
|
|
|||
|
|
@ -112,22 +112,12 @@ cargo run --release --bin idunn -- --daemon demo --rudp-health-bind <idunn-rudp-
|
|||
`--rudp-health-bind` is explicit. Bare one-daemon Idunn does not open a
|
||||
localhost health ingress.
|
||||
|
||||
Install the local Starfire boot watchdog:
|
||||
|
||||
```powershell
|
||||
.\scripts\install-idunn-startup.ps1
|
||||
```
|
||||
|
||||
That task starts one `idunn.exe` swarm supervisor at user logon. The Rust
|
||||
runtime owns the target catalog for Odin, Stonks, Muninn, the enforced
|
||||
The Rust runtime owns the target catalog for Odin, Stonks, Muninn, the enforced
|
||||
Yggdrasil source artifact lanes, and the Nightwing display services. The
|
||||
current Mimir dashboard is observed but not restarted until its restart
|
||||
authority is named.
|
||||
|
||||
It also starts `idunn-swarm-deployment-coverage`, which watches the deployment
|
||||
target catalog in `scripts/idunn-deployment-targets.ps1`. A repo/service is not
|
||||
allowed to vanish into shrug-space: it is either enforced by Idunn, explicitly
|
||||
blocked with the missing authority named, external-owned, or not a runtime.
|
||||
authority is named. A repo/service is not allowed to vanish into shrug-space:
|
||||
it is either enforced by Idunn, explicitly blocked with the missing authority
|
||||
named, external-owned, or not a runtime.
|
||||
|
||||
To record that a daemon has not published health, run a manual target without a
|
||||
matching `idunn.daemon_health` RUDP publication:
|
||||
|
|
@ -185,12 +175,7 @@ Idunn should not guess private service truth. A daemon should publish:
|
|||
If that information is missing, Idunn should fail closed and create an operator
|
||||
alarm instead of improvising.
|
||||
|
||||
For the repo swarm, deployment ownership begins with catalog coverage. Use:
|
||||
|
||||
```powershell
|
||||
.\scripts\show-idunn-deployment-targets.ps1
|
||||
.\scripts\health-idunn-swarm-deployment-coverage.cmd
|
||||
```
|
||||
For the repo swarm, deployment ownership begins with catalog coverage.
|
||||
|
||||
The enforced targets are Nightwing Gjallar plus the Yggdrasil source artifact
|
||||
lanes whose ops runbooks can produce and verify deployment manifests. Each
|
||||
|
|
@ -258,9 +243,9 @@ surface, then let Idunn run the rollout. Direct script execution without
|
|||
`IDUNN_ACTUATOR=1` fails on purpose; the little hand reaching for the deploy
|
||||
button has been removed from the machine.
|
||||
|
||||
`scripts/start-idunn-local.ps1` launches the local swarm with
|
||||
`--rudp-health-bind 0.0.0.0:17870` so host-local and explicitly configured peers
|
||||
can publish to the configured Idunn health endpoint. It accepts raw
|
||||
A local swarm daemon started with
|
||||
`--rudp-health-bind 0.0.0.0:17870` lets host-local and explicitly configured peers
|
||||
publish to the configured Idunn health endpoint. It accepts raw
|
||||
`idunn.daemon_health` document puts on the canonical CNR0 RUDP `schema` channel
|
||||
and writes them into the keepalive store. That is daemon-owned health
|
||||
publication, not restart/deploy authority. On Windows, UDP reset reports from
|
||||
|
|
|
|||
|
|
@ -390,6 +390,8 @@ read-only command on Yggdrasil on 2026-09-29. Nothing was written to the host.
|
|||
(`:3670`, `:3715`), and topology refresh (`:3780`). After three refused
|
||||
continuity restarts, supervision stops restarting (`:3522-3528`) but keeps
|
||||
supervising the route of a dead incarnation. Read.
|
||||
Resolved by `cut-liveness-gates-presence`: route challenge and topology
|
||||
refresh now run only for an incarnation observed running.
|
||||
- F9. The continuity restart limit counts refused restarts per generation
|
||||
(`is_refused_restart`, `:3469-3488`). A restart that succeeds admits a new
|
||||
generation and resets the count. A unit that starts, is admitted, and then
|
||||
|
|
@ -1160,7 +1162,7 @@ target's generation or to its `TargetSupervision` ends that target's iteration:
|
|||
- **Why no existing owner can serve:**
|
||||
- The generation does not exist for a first deploy (the ruling).
|
||||
- The transaction dies with each attempt.
|
||||
- Brakes are operator-signed and are not Idunn's to write.
|
||||
- The deployment brake is a stop that a steward or the operator writes (`src/brake.rs`); Idunn writes no brake.
|
||||
- **What it replaces:** two carry compensators, `for_new_incarnation` and the backoff copy in
|
||||
`from_transaction`, plus a history scan.
|
||||
|
||||
|
|
|
|||
|
|
@ -155,7 +155,7 @@ cleaned up.
|
|||
`idunn redeploy --daemon …`. The installed binary has no `redeploy` command
|
||||
(probe of its usage text), so the script is dead and is **not** the current
|
||||
pattern. The current pattern is `idunn up` followed by
|
||||
`idunn-provision deployment-brake-release`
|
||||
`idunn-provision deployment-brake-clear` when a stop is engaged (an absent or clear store admits)
|
||||
(`runbooks/idunn-host-raven.md`, "Deploy Muninn to Raven").
|
||||
- F24. Build placement (`gamecult-ops/inventory.md:559-566`): build on or for the
|
||||
target's host. Yggdrasil has 16 vCPU and 62 GiB and carries live services, so
|
||||
|
|
|
|||
701
src/brake.rs
Normal file
701
src/brake.rs
Normal file
|
|
@ -0,0 +1,701 @@
|
|||
//! The deployment stop: the one reader and the one writer of the record that
|
||||
//! halts deployments of a runtime. Engaged halts; absent or clear admits. Who
|
||||
//! engaged it and why are on the record, and nothing is signed, issued, bound
|
||||
//! to a release or consumed: the brake is a stop, not a per-deploy gate (ruling
|
||||
//! brake-is-a-stop). The lifecycle brake is a different record and a different
|
||||
//! store; nothing here touches it.
|
||||
//!
|
||||
//! The reader fails closed. A store that is present but foreign, ambiguous,
|
||||
//! noncanonical, for another runtime, a symlink, or owned by anyone but the
|
||||
//! expected uid reads as `Unreadable`, which does not admit.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use crate::deployment::BrakeBinding;
|
||||
|
||||
use anyhow::{Context, Result, anyhow, bail, ensure};
|
||||
use cultcache_rs::{CultCacheEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub(crate) const DEPLOYMENT_STOP_SCHEMA: &str = "idunn.deployment_stop.v1";
|
||||
pub(crate) const DEPLOYMENT_STOP_KEY: &str = "deployment-stop";
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub(crate) enum StopState {
|
||||
Engaged,
|
||||
Clear,
|
||||
}
|
||||
|
||||
impl StopState {
|
||||
pub(crate) fn name(self) -> &'static str {
|
||||
match self {
|
||||
Self::Engaged => "engaged",
|
||||
Self::Clear => "clear",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "idunn.deployment_stop", schema = "idunn.deployment_stop.v1")]
|
||||
pub(crate) struct DeploymentStop {
|
||||
#[cultcache(key = 0)]
|
||||
pub(crate) schema_version: String,
|
||||
#[cultcache(key = 1)]
|
||||
pub(crate) runtime_id: String,
|
||||
#[cultcache(key = 2)]
|
||||
pub(crate) state: StopState,
|
||||
/// The login name of the user who ran the verb, read from the process uid.
|
||||
#[cultcache(key = 3)]
|
||||
pub(crate) by: String,
|
||||
#[cultcache(key = 4)]
|
||||
pub(crate) reason: String,
|
||||
/// The verb's clock.
|
||||
#[cultcache(key = 5)]
|
||||
pub(crate) at_unix_millis: u64,
|
||||
/// The previous record's sequence plus one; 1 for the first.
|
||||
#[cultcache(key = 6)]
|
||||
pub(crate) sequence: u64,
|
||||
}
|
||||
|
||||
fn bounded(value: &str, label: &str, max: usize) -> Result<()> {
|
||||
ensure!(
|
||||
!value.trim().is_empty() && value.len() <= max && !value.chars().any(char::is_control),
|
||||
"deployment stop {label} is empty, oversized, or contains control characters"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl DeploymentStop {
|
||||
pub(crate) fn validate(&self) -> Result<()> {
|
||||
ensure!(
|
||||
self.schema_version == DEPLOYMENT_STOP_SCHEMA,
|
||||
"deployment stop schema is unsupported"
|
||||
);
|
||||
bounded(&self.runtime_id, "runtime id", 256)?;
|
||||
bounded(&self.by, "author", 256)?;
|
||||
bounded(&self.reason, "reason", 1024)?;
|
||||
ensure!(
|
||||
self.at_unix_millis > 0 && self.sequence > 0,
|
||||
"deployment stop time or sequence is zero"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn envelope(&self) -> Result<CultCacheEnvelope> {
|
||||
self.validate()?;
|
||||
let stored_at = chrono::DateTime::from_timestamp_millis(i64::try_from(self.at_unix_millis)?)
|
||||
.ok_or_else(|| anyhow!("deployment stop time is out of range"))?
|
||||
.to_rfc3339();
|
||||
Ok(CultCacheEnvelope {
|
||||
key: DEPLOYMENT_STOP_KEY.into(),
|
||||
r#type: Self::TYPE.into(),
|
||||
payload: rmp_serde::to_vec(self)?,
|
||||
stored_at,
|
||||
schema_id: Some(DEPLOYMENT_STOP_SCHEMA.into()),
|
||||
})
|
||||
}
|
||||
|
||||
fn decode(envelope: &CultCacheEnvelope) -> Result<Self> {
|
||||
ensure!(
|
||||
envelope.r#type == Self::TYPE
|
||||
&& envelope.key == DEPLOYMENT_STOP_KEY
|
||||
&& envelope.schema_id.as_deref() == Some(DEPLOYMENT_STOP_SCHEMA),
|
||||
"deployment stop store holds a foreign record"
|
||||
);
|
||||
let record: Self = rmp_serde::from_slice(&envelope.payload)?;
|
||||
ensure!(
|
||||
rmp_serde::to_vec(&record)? == envelope.payload,
|
||||
"deployment stop is noncanonical"
|
||||
);
|
||||
record.validate()?;
|
||||
Ok(record)
|
||||
}
|
||||
}
|
||||
|
||||
/// What one store says about a runtime's deployments.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub(crate) enum Stop {
|
||||
/// No store, or an empty one: nothing halts.
|
||||
Absent,
|
||||
Clear(DeploymentStop),
|
||||
Engaged(DeploymentStop),
|
||||
/// Present but not a record this runtime can trust. Does not admit.
|
||||
Unreadable(String),
|
||||
}
|
||||
|
||||
impl Stop {
|
||||
pub(crate) fn admits(&self) -> bool {
|
||||
matches!(self, Self::Absent | Self::Clear(_))
|
||||
}
|
||||
}
|
||||
|
||||
/// One stop a binding names, with how to name it in a wait note.
|
||||
pub(crate) struct NamedStop {
|
||||
pub(crate) label: String,
|
||||
/// The login the file system proves wrote the store: the steward user, whose
|
||||
/// uid owns the file. None for the operator's store, whose record carries
|
||||
/// only the verb's own claim of `by`.
|
||||
pub(crate) proven_writer: Option<String>,
|
||||
pub(crate) stop: Stop,
|
||||
}
|
||||
|
||||
impl NamedStop {
|
||||
/// The wait-note line for a stop that does not admit; None when it admits.
|
||||
/// A steward stop is named by the proven owner, never by the record's `by`.
|
||||
pub(crate) fn note(&self) -> Option<String> {
|
||||
let label = &self.label;
|
||||
match &self.stop {
|
||||
Stop::Absent | Stop::Clear(_) => None,
|
||||
Stop::Engaged(held) => Some(match &self.proven_writer {
|
||||
Some(_) => format!("{label} engaged: {}", held.reason),
|
||||
None => format!("{label} engaged by {}: {}", held.by, held.reason),
|
||||
}),
|
||||
Stop::Unreadable(why) => Some(format!("{label} is unreadable and fails closed: {why}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Every stop a binding names, each with the words that identify its store in
|
||||
/// a wait note: the operator's, then the steward's when the binding names one.
|
||||
/// Deployments admit iff every one admits. The operator's store is read as
|
||||
/// owned by this process's uid; the steward's as owned by the steward user's
|
||||
/// uid, so the file system proves who wrote it. A steward that does not
|
||||
/// resolve to a non-root uid is an unreadable stop, never an absent one.
|
||||
pub(crate) fn read_stops(brakes: &BrakeBinding, runtime_id: &str) -> Vec<NamedStop> {
|
||||
let mut stops = vec![NamedStop {
|
||||
label: "deployment brake".to_string(),
|
||||
proven_writer: None,
|
||||
stop: read_stop(&brakes.deployment_store, runtime_id, process_uid()),
|
||||
}];
|
||||
if let Some(steward) = &brakes.steward {
|
||||
let label = format!("steward {} deployment brake", steward.user);
|
||||
let stop = match crate::steward_orders::user_uid(&steward.user) {
|
||||
Ok(uid) => read_stop_with_remedy(&steward.store, runtime_id, uid, |path| {
|
||||
// Only the owner uid can mend it: a clear run as root leaves a
|
||||
// root-owned file, which this reader refuses again.
|
||||
format!(
|
||||
"only {} (the store's owner uid) can repair it, by running as that user: \
|
||||
idunn-provision deployment-brake-clear --store {} --runtime-id {runtime_id} \
|
||||
--reason <text>; a clear run as root stays unreadable",
|
||||
steward.user,
|
||||
path.display()
|
||||
)
|
||||
}),
|
||||
Err(why) => Stop::Unreadable(format!(
|
||||
"{why:#}; the binding's steward user {} must be a non-root login",
|
||||
steward.user
|
||||
)),
|
||||
};
|
||||
stops.push(NamedStop {
|
||||
label,
|
||||
proven_writer: Some(steward.user.clone()),
|
||||
stop,
|
||||
});
|
||||
}
|
||||
stops
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn process_uid() -> u32 {
|
||||
// SAFETY: geteuid has no failure mode and no arguments.
|
||||
unsafe { libc::geteuid() }
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
pub(crate) fn process_uid() -> u32 {
|
||||
0
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn login_name(uid: u32) -> Result<String> {
|
||||
// SAFETY: getpwuid returns null or a pointer into libc's static buffer,
|
||||
// copied before any other call; the verbs are single threaded.
|
||||
unsafe {
|
||||
let entry = libc::getpwuid(uid);
|
||||
ensure!(!entry.is_null(), "uid {uid} has no login name");
|
||||
Ok(std::ffi::CStr::from_ptr((*entry).pw_name)
|
||||
.to_string_lossy()
|
||||
.into_owned())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
pub(crate) fn login_name(_: u32) -> Result<String> {
|
||||
std::env::var("USERNAME").context("USERNAME is unset")
|
||||
}
|
||||
|
||||
/// Open without following a final symlink and judge the file by fstat before
|
||||
/// anything is read from it.
|
||||
#[cfg(unix)]
|
||||
fn check_file(path: &Path, expected_owner_uid: u32) -> Result<()> {
|
||||
use std::os::unix::fs::{MetadataExt, OpenOptionsExt};
|
||||
let file = std::fs::OpenOptions::new()
|
||||
.read(true)
|
||||
.custom_flags(libc::O_NOFOLLOW)
|
||||
.open(path)
|
||||
.with_context(|| format!("opening deployment stop store {}", path.display()))?;
|
||||
let metadata = file.metadata()?;
|
||||
ensure!(
|
||||
metadata.is_file(),
|
||||
"deployment stop store is not a regular file"
|
||||
);
|
||||
ensure!(
|
||||
metadata.uid() == expected_owner_uid,
|
||||
"deployment stop store is owned by uid {}, not {expected_owner_uid}",
|
||||
metadata.uid()
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
fn check_file(path: &Path, _: u32) -> Result<()> {
|
||||
ensure!(
|
||||
std::fs::metadata(path)?.is_file(),
|
||||
"deployment stop store is not a regular file"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Absent means exactly this: the final component does not exist and its
|
||||
/// parent is an existing directory. Every other failure to look (permission,
|
||||
/// a parent that is a file, a symlink loop) is a store this runtime cannot
|
||||
/// judge, and fails closed.
|
||||
fn is_absent(path: &Path) -> Result<bool> {
|
||||
match std::fs::symlink_metadata(path) {
|
||||
Ok(_) => Ok(false),
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
|
||||
let parent = path.parent().filter(|p| !p.as_os_str().is_empty());
|
||||
let parent = parent.unwrap_or_else(|| Path::new("."));
|
||||
ensure!(
|
||||
std::fs::metadata(parent).is_ok_and(|m| m.is_dir()),
|
||||
"deployment stop store directory {} does not exist",
|
||||
parent.display()
|
||||
);
|
||||
Ok(true)
|
||||
}
|
||||
Err(error) => Err(error)
|
||||
.with_context(|| format!("looking for deployment stop store {}", path.display())),
|
||||
}
|
||||
}
|
||||
|
||||
fn read_checked(path: &Path, runtime_id: &str, owner: u32) -> Result<Option<DeploymentStop>> {
|
||||
if is_absent(path)? {
|
||||
return Ok(None);
|
||||
}
|
||||
check_file(path, owner)?;
|
||||
ensure!(
|
||||
std::fs::metadata(path)?.len() > 0,
|
||||
"deployment stop store is zero length"
|
||||
);
|
||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||
match entries.as_slice() {
|
||||
[] => bail!("deployment stop store holds no record"),
|
||||
[entry] => {
|
||||
let record = DeploymentStop::decode(entry)?;
|
||||
ensure!(
|
||||
record.runtime_id == runtime_id,
|
||||
"deployment stop belongs to runtime {}",
|
||||
record.runtime_id
|
||||
);
|
||||
Ok(Some(record))
|
||||
}
|
||||
_ => bail!("deployment stop store is ambiguous"),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn read_stop(path: &Path, runtime_id: &str, expected_owner_uid: u32) -> Stop {
|
||||
read_stop_with_remedy(path, runtime_id, expected_owner_uid, |path| {
|
||||
format!(
|
||||
"to replace it run: idunn-provision deployment-brake-clear \
|
||||
--store {} --runtime-id {runtime_id} --reason <text> (or deployment-brake-engage)",
|
||||
path.display()
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn read_stop_with_remedy(
|
||||
path: &Path,
|
||||
runtime_id: &str,
|
||||
expected_owner_uid: u32,
|
||||
remedy: impl FnOnce(&Path) -> String,
|
||||
) -> Stop {
|
||||
match read_checked(path, runtime_id, expected_owner_uid) {
|
||||
Ok(None) => Stop::Absent,
|
||||
Ok(Some(record)) if record.state == StopState::Clear => Stop::Clear(record),
|
||||
Ok(Some(record)) => Stop::Engaged(record),
|
||||
Err(error) => Stop::Unreadable(format!("{error:#}; {}", remedy(path))),
|
||||
}
|
||||
}
|
||||
|
||||
/// Write the next record under a compare-exchange of the whole store, so a
|
||||
/// concurrent verb or a store that moved since it was read is refused. The
|
||||
/// author is the login name of the process uid and the sequence is the
|
||||
/// predecessor's plus one (1 when there is none or it cannot be decoded);
|
||||
/// neither is an argument.
|
||||
///
|
||||
/// This is also the operator's break-glass. A predecessor that is a retired v1
|
||||
/// brake, another runtime's record, ambiguous, or zero length is replaced. A
|
||||
/// file that is not a CultCache store at all (junk bytes) is replaced only
|
||||
/// when it is a regular file owned by the caller, by removing it first. A
|
||||
/// symlink or another user's file is refused. The sequence is not validated
|
||||
/// beyond being positive: nothing reads it for authority.
|
||||
pub(crate) fn replace_stop(
|
||||
path: &Path,
|
||||
runtime_id: &str,
|
||||
state: StopState,
|
||||
reason: &str,
|
||||
now_unix_millis: u64,
|
||||
) -> Result<DeploymentStop> {
|
||||
let store = SingleFileMessagePackBackingStore::new(path);
|
||||
let before = match store.pull_all_read_only_snapshot() {
|
||||
Ok(entries) => entries,
|
||||
Err(undecodable) => {
|
||||
check_file(path, process_uid()).with_context(|| {
|
||||
format!("deployment stop store is undecodable ({undecodable:#}) and not the caller's to replace")
|
||||
})?;
|
||||
std::fs::remove_file(path)?;
|
||||
Vec::new()
|
||||
}
|
||||
};
|
||||
let sequence = match before.as_slice() {
|
||||
[entry] => match DeploymentStop::decode(entry) {
|
||||
Ok(previous) => previous.sequence.saturating_add(1),
|
||||
Err(_) => 1,
|
||||
},
|
||||
_ => 1,
|
||||
};
|
||||
let record = DeploymentStop {
|
||||
schema_version: DEPLOYMENT_STOP_SCHEMA.into(),
|
||||
runtime_id: runtime_id.into(),
|
||||
state,
|
||||
by: login_name(process_uid())?,
|
||||
reason: reason.into(),
|
||||
at_unix_millis: now_unix_millis,
|
||||
sequence,
|
||||
};
|
||||
let next = record.envelope()?;
|
||||
ensure!(
|
||||
store.compare_exchange_snapshot(&before, &[next])?,
|
||||
"deployment stop changed during transition"
|
||||
);
|
||||
Ok(record)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn engage(path: &Path, reason: &str, at: u64) -> Result<DeploymentStop> {
|
||||
replace_stop(path, "runtime", StopState::Engaged, reason, at)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn absent_and_empty_stores_admit_a_clear_one_admits_an_engaged_one_does_not() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let path = dir.path().join("stop.cc");
|
||||
let me = process_uid();
|
||||
assert_eq!(read_stop(&path, "runtime", me), Stop::Absent);
|
||||
engage(&path, "freeze for the audit", 100)?;
|
||||
let Stop::Engaged(held) = read_stop(&path, "runtime", me) else {
|
||||
bail!("an engaged stop did not read as engaged")
|
||||
};
|
||||
assert_eq!(held.reason, "freeze for the audit");
|
||||
assert_eq!(held.by, login_name(me)?);
|
||||
assert!(!Stop::Engaged(held).admits());
|
||||
replace_stop(&path, "runtime", StopState::Clear, "audit done", 200)?;
|
||||
let cleared = read_stop(&path, "runtime", me);
|
||||
assert!(matches!(&cleared, Stop::Clear(record) if record.sequence == 2));
|
||||
assert!(cleared.admits());
|
||||
// Clear is not consumed: the same record admits again.
|
||||
assert!(read_stop(&path, "runtime", me).admits());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn corrupt_foreign_other_runtime_symlinked_or_foreign_owned_stores_stop() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let me = process_uid();
|
||||
let path = dir.path().join("stop.cc");
|
||||
engage(&path, "held", 100)?;
|
||||
// Another runtime's record.
|
||||
assert!(!read_stop(&path, "other", me).admits());
|
||||
// Owned by someone other than the expected uid: even a CLEAR record
|
||||
// from a foreign owner must not admit.
|
||||
let cleared = dir.path().join("cleared.cc");
|
||||
replace_stop(&cleared, "runtime", StopState::Clear, "thaw", 100)?;
|
||||
assert!(read_stop(&cleared, "runtime", me).admits());
|
||||
assert!(matches!(read_stop(&cleared, "runtime", me + 1), Stop::Unreadable(_)));
|
||||
// A symlink to a valid, clear store.
|
||||
let real = dir.path().join("real.cc");
|
||||
replace_stop(&real, "runtime", StopState::Clear, "clear", 100)?;
|
||||
let link = dir.path().join("link.cc");
|
||||
#[cfg(unix)]
|
||||
std::os::unix::fs::symlink(&real, &link)?;
|
||||
#[cfg(unix)]
|
||||
assert!(matches!(read_stop(&link, "runtime", me), Stop::Unreadable(_)));
|
||||
// Bytes that are no store at all.
|
||||
let junk = dir.path().join("junk.cc");
|
||||
std::fs::write(&junk, b"not msgpack at all")?;
|
||||
assert!(matches!(read_stop(&junk, "runtime", me), Stop::Unreadable(_)));
|
||||
// A record of another type under another key (the retired v1 brake).
|
||||
let foreign = dir.path().join("foreign.cc");
|
||||
let store = SingleFileMessagePackBackingStore::new(&foreign);
|
||||
let envelope = CultCacheEnvelope {
|
||||
key: "deployment".into(),
|
||||
r#type: "idunn.deployment_brake".into(),
|
||||
payload: vec![0x90],
|
||||
stored_at: "1970-01-01T00:00:00+00:00".into(),
|
||||
schema_id: Some("idunn.deployment_brake.v1".into()),
|
||||
};
|
||||
assert!(store.compare_exchange_snapshot(&[], &[envelope])?);
|
||||
assert!(matches!(read_stop(&foreign, "runtime", me), Stop::Unreadable(_)));
|
||||
// A noncanonical payload under the right identity.
|
||||
let mut stop = DeploymentStop::decode(&store_entry(&path)?)?;
|
||||
stop.sequence = 9;
|
||||
let mut loose = stop.envelope()?;
|
||||
loose.payload.push(0xc0);
|
||||
let noncanonical = dir.path().join("noncanonical.cc");
|
||||
assert!(SingleFileMessagePackBackingStore::new(&noncanonical)
|
||||
.compare_exchange_snapshot(&[], &[loose])?);
|
||||
assert!(matches!(read_stop(&noncanonical, "runtime", me), Stop::Unreadable(_)));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A binding whose steward is the user running the test, so the steward's
|
||||
/// store can really be owned by the steward. None when run as root, which
|
||||
/// no steward may be.
|
||||
fn binding_with_steward(dir: &Path, user: &str) -> BrakeBinding {
|
||||
BrakeBinding {
|
||||
deployment_store: dir.join("operator.cc"),
|
||||
lifecycle_store: dir.join("lifecycle.cc"),
|
||||
steward: Some(crate::deployment::StewardBrakeBinding {
|
||||
user: user.into(),
|
||||
store: dir.join("steward.cc"),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
fn admits_all(stops: &[NamedStop]) -> bool {
|
||||
stops.iter().all(|named| named.stop.admits())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_steward_store_is_a_second_stop_and_the_operator_store_is_one_it_cannot_clear() -> Result<()> {
|
||||
let me = process_uid();
|
||||
if me == 0 {
|
||||
eprintln!("SKIPPED: a steward is never root, and this run is root");
|
||||
return Ok(());
|
||||
}
|
||||
let dir = tempfile::tempdir()?;
|
||||
let binding = binding_with_steward(dir.path(), &login_name(me)?);
|
||||
let (operator, steward) = (&binding.deployment_store, &binding.steward.as_ref().unwrap().store);
|
||||
// Both absent: admits.
|
||||
assert!(admits_all(&read_stops(&binding, "runtime")));
|
||||
// Steward engaged alone: halts, and the stop names the steward's store.
|
||||
replace_stop(steward, "runtime", StopState::Engaged, "steward freeze", 10)?;
|
||||
let stops = read_stops(&binding, "runtime");
|
||||
assert!(!admits_all(&stops));
|
||||
assert!(matches!(&stops[1].stop, Stop::Engaged(held) if held.reason == "steward freeze"));
|
||||
// The note names the proven owner, and never the record's claimed `by`.
|
||||
assert_eq!(
|
||||
stops[1].note().unwrap(),
|
||||
format!("steward {} deployment brake engaged: steward freeze", login_name(me)?)
|
||||
);
|
||||
// Both engaged, then the steward clears its own: the operator's holds.
|
||||
replace_stop(operator, "runtime", StopState::Engaged, "operator hold", 11)?;
|
||||
replace_stop(steward, "runtime", StopState::Clear, "steward thaw", 12)?;
|
||||
let stops = read_stops(&binding, "runtime");
|
||||
assert!(matches!(&stops[0].stop, Stop::Engaged(_)) && matches!(&stops[1].stop, Stop::Clear(_)));
|
||||
assert!(!admits_all(&stops));
|
||||
// The other order: operator clear, steward engaged still halts.
|
||||
replace_stop(operator, "runtime", StopState::Clear, "operator thaw", 13)?;
|
||||
replace_stop(steward, "runtime", StopState::Engaged, "steward again", 14)?;
|
||||
assert!(!admits_all(&read_stops(&binding, "runtime")));
|
||||
// Both clear: admits.
|
||||
replace_stop(steward, "runtime", StopState::Clear, "steward thaw", 15)?;
|
||||
assert!(admits_all(&read_stops(&binding, "runtime")));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn steward_store_owned_by_another_uid_stops() -> Result<()> {
|
||||
let me = process_uid();
|
||||
let dir = tempfile::tempdir()?;
|
||||
// Steward is "nobody" (a real non-root user that is not the writer);
|
||||
// the file is written by this process, so even a Clear record fails.
|
||||
if me == 65534 || user_missing("nobody") {
|
||||
eprintln!("SKIPPED: needs a user `nobody` other than the caller");
|
||||
return Ok(());
|
||||
}
|
||||
let binding = binding_with_steward(dir.path(), "nobody");
|
||||
let steward = &binding.steward.as_ref().unwrap().store;
|
||||
replace_stop(steward, "runtime", StopState::Clear, "forged clear", 10)?;
|
||||
let stops = read_stops(&binding, "runtime");
|
||||
assert!(matches!(&stops[1].stop, Stop::Unreadable(why) if why.contains("is owned by uid")
|
||||
&& why.contains("only nobody (the store's owner uid) can repair it")
|
||||
&& why.contains("stays unreadable")
|
||||
&& why.contains("idunn-provision deployment-brake-clear")));
|
||||
// The operator's stop keeps the operator's remedy.
|
||||
replace_stop(&binding.deployment_store, "other-runtime", StopState::Clear, "x", 11)?;
|
||||
assert!(matches!(&read_stops(&binding, "runtime")[0].stop, Stop::Unreadable(why)
|
||||
if why.contains("to replace it run:") && !why.contains("only ")));
|
||||
assert!(!admits_all(&stops));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn user_missing(name: &str) -> bool {
|
||||
crate::steward_orders::user_uid(name).is_err()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unresolvable_or_root_steward_user_is_an_unreadable_stop_not_an_absent_one() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
for user in ["no-such-login-zz", "root"] {
|
||||
let binding = binding_with_steward(dir.path(), user);
|
||||
let stops = read_stops(&binding, "runtime");
|
||||
assert!(matches!(&stops[1].stop, Stop::Unreadable(_)), "{user}");
|
||||
assert!(!admits_all(&stops));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_binding_without_a_steward_reads_only_the_operator_store() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let mut binding = binding_with_steward(dir.path(), "nobody");
|
||||
binding.steward = None;
|
||||
let stops = read_stops(&binding, "runtime");
|
||||
assert_eq!(stops.len(), 1);
|
||||
assert!(admits_all(&stops));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn store_entry(path: &Path) -> Result<CultCacheEnvelope> {
|
||||
let mut entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||
entries.pop().context("no entry")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_verb_replaces_a_retired_brake_record_and_the_sequence_advances_by_one() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let path = dir.path().join("stop.cc");
|
||||
let store = SingleFileMessagePackBackingStore::new(&path);
|
||||
let retired = CultCacheEnvelope {
|
||||
key: "deployment".into(),
|
||||
r#type: "idunn.deployment_brake".into(),
|
||||
payload: vec![0x90],
|
||||
stored_at: "1970-01-01T00:00:00+00:00".into(),
|
||||
schema_id: Some("idunn.deployment_brake.v1".into()),
|
||||
};
|
||||
assert!(store.compare_exchange_snapshot(&[], &[retired])?);
|
||||
assert_eq!(engage(&path, "first", 10)?.sequence, 1);
|
||||
assert_eq!(store.pull_all_read_only_snapshot()?.len(), 1);
|
||||
assert_eq!(engage(&path, "second", 11)?.sequence, 2);
|
||||
assert_eq!(
|
||||
replace_stop(&path, "runtime", StopState::Clear, "third", 12)?.sequence,
|
||||
3
|
||||
);
|
||||
assert_eq!(DeploymentStop::decode(&store_entry(&path)?)?.reason, "third");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_stale_exchange_is_refused_and_a_record_without_a_reason_is_not_written() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let path = dir.path().join("stop.cc");
|
||||
engage(&path, "held", 10)?;
|
||||
let store = SingleFileMessagePackBackingStore::new(&path);
|
||||
let stale = store.pull_all_read_only_snapshot()?;
|
||||
engage(&path, "moved", 11)?;
|
||||
let next = DeploymentStop {
|
||||
schema_version: DEPLOYMENT_STOP_SCHEMA.into(),
|
||||
runtime_id: "runtime".into(),
|
||||
state: StopState::Clear,
|
||||
by: "someone".into(),
|
||||
reason: "late".into(),
|
||||
at_unix_millis: 12,
|
||||
sequence: 2,
|
||||
}
|
||||
.envelope()?;
|
||||
assert!(!store.compare_exchange_snapshot(&stale, &[next])?);
|
||||
assert!(replace_stop(&path, "runtime", StopState::Clear, " ", 13).is_err());
|
||||
assert_eq!(DeploymentStop::decode(&store_entry(&path)?)?.reason, "moved");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn only_a_missing_file_in_an_existing_directory_is_absent() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let me = process_uid();
|
||||
// Parent is a regular file: ENOTDIR, not absence.
|
||||
let file = dir.path().join("plain");
|
||||
std::fs::write(&file, b"x")?;
|
||||
assert!(matches!(read_stop(&file.join("stop.cc"), "runtime", me), Stop::Unreadable(_)));
|
||||
// Parent does not exist.
|
||||
assert!(matches!(
|
||||
read_stop(&dir.path().join("nope/stop.cc"), "runtime", me),
|
||||
Stop::Unreadable(_)
|
||||
));
|
||||
// Zero-length file.
|
||||
let empty = dir.path().join("empty.cc");
|
||||
std::fs::write(&empty, b"")?;
|
||||
assert!(matches!(read_stop(&empty, "runtime", me), Stop::Unreadable(_)));
|
||||
// An existing directory with no store is absent.
|
||||
assert_eq!(read_stop(&dir.path().join("stop.cc"), "runtime", me), Stop::Absent);
|
||||
// A parent that cannot be searched (skipped as root, which can).
|
||||
#[cfg(unix)]
|
||||
if me != 0 {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
let locked = dir.path().join("locked");
|
||||
std::fs::create_dir(&locked)?;
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o000))?;
|
||||
let reading = read_stop(&locked.join("stop.cc"), "runtime", me);
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o700))?;
|
||||
assert!(matches!(reading, Stop::Unreadable(_)), "{reading:?}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_author_is_the_uid_login_never_the_environment() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let path = dir.path().join("stop.cc");
|
||||
// SAFETY: no other test reads USER, LOGNAME or LNAME.
|
||||
unsafe {
|
||||
std::env::set_var("USER", "spoof");
|
||||
std::env::set_var("LOGNAME", "spoof");
|
||||
}
|
||||
let written = engage(&path, "held", 10)?;
|
||||
assert_eq!(written.by, login_name(process_uid())?);
|
||||
assert_ne!(written.by, "spoof");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unreadable_names_the_command_and_a_break_glass_clear_replaces_junk_and_v1() -> Result<()> {
|
||||
let dir = tempfile::tempdir()?;
|
||||
let me = process_uid();
|
||||
let path = dir.path().join("stop.cc");
|
||||
std::fs::write(&path, b"junk that is no store")?;
|
||||
let Stop::Unreadable(why) = read_stop(&path, "runtime", me) else {
|
||||
bail!("junk admitted")
|
||||
};
|
||||
assert!(why.contains("idunn-provision deployment-brake-clear --store"), "{why}");
|
||||
assert!(why.contains("--runtime-id runtime"), "{why}");
|
||||
let cleared = replace_stop(&path, "runtime", StopState::Clear, "operator break-glass", 50)?;
|
||||
assert_eq!(cleared.sequence, 1);
|
||||
assert!(read_stop(&path, "runtime", me).admits());
|
||||
// A symlink is not the caller's to replace.
|
||||
let real = dir.path().join("real.cc");
|
||||
std::fs::write(&real, b"junk")?;
|
||||
let link = dir.path().join("link.cc");
|
||||
#[cfg(unix)]
|
||||
{
|
||||
std::os::unix::fs::symlink(&real, &link)?;
|
||||
assert!(replace_stop(&link, "runtime", StopState::Clear, "x", 60).is_err());
|
||||
assert_eq!(std::fs::read(&real)?, b"junk");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
4481
src/control_plane.rs
4481
src/control_plane.rs
File diff suppressed because it is too large
Load diff
|
|
@ -45,7 +45,12 @@ pub struct TargetDeclaration {
|
|||
#[serde(default)]
|
||||
pub external_inputs: Vec<ExternalInput>,
|
||||
pub artifacts: Vec<ArtifactOutput>,
|
||||
pub service: ServiceDeclaration,
|
||||
/// A running workload. Exactly one of `service` and `static_release`.
|
||||
#[serde(default)]
|
||||
pub service: Option<ServiceDeclaration>,
|
||||
/// A sealed tree that is published by repointing a link, with no process.
|
||||
#[serde(default)]
|
||||
pub static_release: Option<StaticReleaseDeclaration>,
|
||||
#[serde(default)]
|
||||
pub state: Option<StateDeclaration>,
|
||||
#[serde(default)]
|
||||
|
|
@ -113,6 +118,10 @@ pub struct ArtifactOutput {
|
|||
pub expected_sha256: Option<String>,
|
||||
#[serde(default)]
|
||||
pub executable: bool,
|
||||
/// Regular files inside a directory artifact that are sealed executable
|
||||
/// (0555); every other file in the tree is sealed 0444.
|
||||
#[serde(default)]
|
||||
pub executable_members: BTreeSet<PathBuf>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
|
|
@ -122,6 +131,15 @@ pub enum ArtifactSource {
|
|||
WorktreeTree,
|
||||
}
|
||||
|
||||
/// A target that is a sealed tree and nothing else: Idunn installs the release
|
||||
/// and repoints the binding's current link at it. `artifact` names the tree a
|
||||
/// web server reads through that link.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct StaticReleaseDeclaration {
|
||||
pub artifact: String,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct ServiceDeclaration {
|
||||
|
|
@ -471,20 +489,42 @@ pub struct DockerRunnerBinding {
|
|||
pub enum WorkloadBinding {
|
||||
SystemdTransient(SystemdWorkloadBinding),
|
||||
HostActuator(HostWorkloadBinding),
|
||||
/// No process: a release directory and the link that names the current one.
|
||||
StaticRelease(StaticReleaseBinding),
|
||||
}
|
||||
|
||||
static NO_ENVIRONMENT: BTreeMap<String, String> = BTreeMap::new();
|
||||
|
||||
impl WorkloadBinding {
|
||||
pub fn systemd(&self) -> Result<&SystemdWorkloadBinding> {
|
||||
match self {
|
||||
Self::SystemdTransient(binding) => Ok(binding),
|
||||
Self::HostActuator(_) => bail!("workload is host-actuator, not systemd-transient"),
|
||||
Self::StaticRelease(_) => bail!("workload is a static release, not systemd-transient"),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn static_release(&self) -> Result<&StaticReleaseBinding> {
|
||||
match self {
|
||||
Self::StaticRelease(binding) => Ok(binding),
|
||||
_ => bail!("workload is not a static release"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Where Idunn installs sealed releases on its own host: the one thing a
|
||||
/// systemd workload and a static release share.
|
||||
pub fn release_root(&self) -> Result<&Path> {
|
||||
match self {
|
||||
Self::SystemdTransient(binding) => Ok(&binding.release_root),
|
||||
Self::StaticRelease(binding) => Ok(&binding.release_root),
|
||||
Self::HostActuator(_) => bail!("a host-actuator workload installs on its own host"),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn host(&self) -> Result<&HostWorkloadBinding> {
|
||||
match self {
|
||||
Self::HostActuator(binding) => Ok(binding),
|
||||
Self::SystemdTransient(_) => bail!("workload is systemd-transient, not host-actuator"),
|
||||
_ => bail!("workload is not host-actuator"),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -492,6 +532,7 @@ impl WorkloadBinding {
|
|||
match self {
|
||||
Self::SystemdTransient(binding) => binding.state_root.is_some(),
|
||||
Self::HostActuator(binding) => binding.state_root.is_some(),
|
||||
Self::StaticRelease(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -499,6 +540,7 @@ impl WorkloadBinding {
|
|||
match self {
|
||||
Self::SystemdTransient(binding) => &binding.environment,
|
||||
Self::HostActuator(binding) => &binding.environment,
|
||||
Self::StaticRelease(_) => &NO_ENVIRONMENT,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -506,6 +548,7 @@ impl WorkloadBinding {
|
|||
match self {
|
||||
Self::SystemdTransient(binding) => binding.secret_files.keys().collect(),
|
||||
Self::HostActuator(binding) => binding.secret_files.keys().collect(),
|
||||
Self::StaticRelease(_) => Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -513,6 +556,7 @@ impl WorkloadBinding {
|
|||
match self {
|
||||
Self::SystemdTransient(binding) => &binding.argument_bindings,
|
||||
Self::HostActuator(binding) => &binding.argument_bindings,
|
||||
Self::StaticRelease(_) => &NO_ENVIRONMENT,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -547,6 +591,49 @@ impl WorkloadBinding {
|
|||
}
|
||||
}
|
||||
|
||||
/// A static release on the Idunn host. `current_link` is the symlink a web
|
||||
/// server's root reads through; Idunn repoints it at one sealed release.
|
||||
/// `carry` names one operator-owned file (a per-site config) that Idunn copies
|
||||
/// into each release as `destination` when it flips: the copy sits beside the
|
||||
/// artifacts, outside every sealed digest.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct StaticReleaseBinding {
|
||||
pub release_root: PathBuf,
|
||||
pub current_link: PathBuf,
|
||||
pub carry: StaticCarryBinding,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct StaticCarryBinding {
|
||||
pub source: PathBuf,
|
||||
pub destination: PathBuf,
|
||||
}
|
||||
|
||||
impl StaticReleaseBinding {
|
||||
fn validate(&self) -> Result<()> {
|
||||
require_absolute_path(&self.release_root, "static release root")?;
|
||||
require_absolute_path(&self.current_link, "static current link")?;
|
||||
require_absolute_path(&self.carry.source, "static carry source")?;
|
||||
require_leaf_path(&self.carry.destination, "static carry destination")?;
|
||||
ensure!(
|
||||
!paths_overlap(&self.current_link, &self.release_root),
|
||||
"static current link and release root overlap"
|
||||
);
|
||||
ensure!(
|
||||
!self.carry.source.starts_with(&self.release_root),
|
||||
"static carry source is inside the release root"
|
||||
);
|
||||
ensure!(
|
||||
self.carry.source != self.current_link
|
||||
&& !self.carry.source.starts_with(&self.current_link),
|
||||
"static carry source is the current link"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
/// A workload on a managed host. Every path is the host's and is validated by
|
||||
/// the actuator that owns the host; here they are opaque non-empty values.
|
||||
/// The host's actuator is authenticated against `host_trust_anchor_store`,
|
||||
|
|
@ -666,6 +753,20 @@ impl ProcessWriteLeaseBinding {
|
|||
pub struct BrakeBinding {
|
||||
pub deployment_store: PathBuf,
|
||||
pub lifecycle_store: PathBuf,
|
||||
/// A second deployment stop, written by the target's steward in a
|
||||
/// directory only the steward can write. Absent means the operator's store
|
||||
/// is the only stop.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub steward: Option<StewardBrakeBinding>,
|
||||
}
|
||||
|
||||
/// The steward's own stop: the file's owner uid proves who wrote it.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct StewardBrakeBinding {
|
||||
/// Login name; must resolve to a non-root uid when the store is read.
|
||||
pub user: String,
|
||||
pub store: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
|
|
@ -912,6 +1013,14 @@ impl TargetDeclaration {
|
|||
if let Some(expected_sha256) = &artifact.expected_sha256 {
|
||||
require_sha256(expected_sha256, "expected artifact sha256")?;
|
||||
}
|
||||
ensure!(
|
||||
!(artifact.executable && !artifact.executable_members.is_empty()),
|
||||
"artifact {} declares both executable and executable_members",
|
||||
artifact.id
|
||||
);
|
||||
for member in &artifact.executable_members {
|
||||
require_relative_path(member, "executable member", false)?;
|
||||
}
|
||||
ensure!(
|
||||
destinations.insert(artifact.destination.as_path()),
|
||||
"artifact destination {} is declared twice",
|
||||
|
|
@ -965,57 +1074,76 @@ impl TargetDeclaration {
|
|||
"target declaration uses no runner classes"
|
||||
);
|
||||
|
||||
ensure!(
|
||||
artifact_ids.contains(self.service.executable_artifact.as_str()),
|
||||
"service executable artifact is not declared"
|
||||
);
|
||||
for artifact in &self.service.required_adjacent_artifacts {
|
||||
match (&self.service, &self.static_release) {
|
||||
(Some(service), None) => {
|
||||
ensure!(
|
||||
artifact_ids.contains(service.executable_artifact.as_str()),
|
||||
"service executable artifact is not declared"
|
||||
);
|
||||
for artifact in &service.required_adjacent_artifacts {
|
||||
ensure!(
|
||||
artifact_ids.contains(artifact.as_str()),
|
||||
"adjacent service artifact {artifact} is not declared"
|
||||
);
|
||||
}
|
||||
for argument in &service.arguments {
|
||||
validate_launch_argument(argument)?;
|
||||
}
|
||||
require_capability(&service.health.contract, "health contract")?;
|
||||
for name in &service.required_environment {
|
||||
require_environment_name(name)?;
|
||||
}
|
||||
ensure!(
|
||||
artifact_ids.contains(artifact.as_str()),
|
||||
"adjacent service artifact {artifact} is not declared"
|
||||
service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_BUNDLE_ENVIRONMENT),
|
||||
"service does not require the standard Idunn runtime bundle"
|
||||
);
|
||||
}
|
||||
for argument in &self.service.arguments {
|
||||
validate_launch_argument(argument)?;
|
||||
}
|
||||
require_capability(&self.service.health.contract, "health contract")?;
|
||||
for name in &self.service.required_environment {
|
||||
require_environment_name(name)?;
|
||||
}
|
||||
ensure!(
|
||||
self.service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_BUNDLE_ENVIRONMENT),
|
||||
"service does not require the standard Idunn runtime bundle"
|
||||
);
|
||||
ensure!(
|
||||
self.service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT)
|
||||
== self.service.route_required,
|
||||
"routed services must require exactly one Idunn-owned candidate bind"
|
||||
);
|
||||
ensure!(
|
||||
!self
|
||||
.service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT)
|
||||
|| self.service.route_required,
|
||||
"only routed services may require the Idunn-owned member bind"
|
||||
);
|
||||
ensure!(
|
||||
self.service
|
||||
.required_environment
|
||||
.contains(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT)
|
||||
== self.write_lease_required(),
|
||||
"state-writing services must require exactly one Idunn-owned process write lease"
|
||||
);
|
||||
for name in &self.service.optional_environment {
|
||||
require_environment_name(name)?;
|
||||
ensure!(
|
||||
!self.service.required_environment.contains(name),
|
||||
"service environment {name} is both required and optional"
|
||||
service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT)
|
||||
== service.route_required,
|
||||
"routed services must require exactly one Idunn-owned candidate bind"
|
||||
);
|
||||
ensure!(
|
||||
!service
|
||||
.required_environment
|
||||
.contains(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT)
|
||||
|| service.route_required,
|
||||
"only routed services may require the Idunn-owned member bind"
|
||||
);
|
||||
ensure!(
|
||||
service
|
||||
.required_environment
|
||||
.contains(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT)
|
||||
== self.write_lease_required(),
|
||||
"state-writing services must require exactly one Idunn-owned process write lease"
|
||||
);
|
||||
for name in &service.optional_environment {
|
||||
require_environment_name(name)?;
|
||||
ensure!(
|
||||
!service.required_environment.contains(name),
|
||||
"service environment {name} is both required and optional"
|
||||
);
|
||||
}
|
||||
|
||||
}
|
||||
(None, Some(static_release)) => {
|
||||
ensure!(
|
||||
artifact_ids.contains(static_release.artifact.as_str()),
|
||||
"static release artifact is not declared"
|
||||
);
|
||||
ensure!(
|
||||
self.state.is_none()
|
||||
&& self.provides.is_empty()
|
||||
&& self.dependencies.is_empty()
|
||||
&& self.conflicts.is_empty(),
|
||||
"a static release declares no state, capability, dependency or conflict"
|
||||
);
|
||||
}
|
||||
(Some(_), Some(_)) => bail!("target declares both a service and a static release"),
|
||||
(None, None) => bail!("target declares neither a service nor a static release"),
|
||||
}
|
||||
|
||||
if let Some(state) = &self.state {
|
||||
|
|
@ -1132,6 +1260,14 @@ impl TargetDeclaration {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// The service of a service target. A static release has none, and a code
|
||||
/// path that asks for one has been reached by the wrong kind of target.
|
||||
pub fn service(&self) -> Result<&ServiceDeclaration> {
|
||||
self.service
|
||||
.as_ref()
|
||||
.context("target declares no service")
|
||||
}
|
||||
|
||||
pub fn write_lease_required(&self) -> bool {
|
||||
self.state.as_ref().is_some_and(|state| {
|
||||
state
|
||||
|
|
@ -1308,6 +1444,39 @@ impl OperatorBinding {
|
|||
self.brakes.deployment_store != self.brakes.lifecycle_store,
|
||||
"deployment and lifecycle brakes are identical"
|
||||
);
|
||||
if let Some(steward) = &self.brakes.steward {
|
||||
require_absolute_path(&steward.store, "steward deployment brake")?;
|
||||
ensure!(
|
||||
crate::steward_orders::login_grammar_ok(&steward.user) && steward.user != "root",
|
||||
"steward brake user is invalid or root"
|
||||
);
|
||||
ensure!(
|
||||
steward.store != self.brakes.deployment_store
|
||||
&& steward.store != self.brakes.lifecycle_store,
|
||||
"steward brake store is identical to another brake store"
|
||||
);
|
||||
// The steward owns its store's directory, so it could unlink any
|
||||
// file there. Every other authority store must therefore live
|
||||
// outside that directory, and the steward's directory must be
|
||||
// clear of every directory an operator store lives in.
|
||||
let steward_dir = steward.store.parent().unwrap_or(&steward.store);
|
||||
let others = [
|
||||
Some(&self.brakes.deployment_store),
|
||||
Some(&self.brakes.lifecycle_store),
|
||||
Some(&self.runtime_identity.trust_anchor_store),
|
||||
self.process_write_lease.as_ref().map(|lease| &lease.record_path),
|
||||
self.repository.release_authority_store.as_ref(),
|
||||
];
|
||||
for other in others.into_iter().flatten() {
|
||||
let other_dir = other.parent().unwrap_or(other);
|
||||
ensure!(
|
||||
!paths_overlap(steward_dir, other_dir),
|
||||
"steward brake directory {} is shared with or nested in the directory of Idunn authority {}: the steward owns its directory and could remove that file",
|
||||
steward_dir.display(),
|
||||
other.display()
|
||||
);
|
||||
}
|
||||
}
|
||||
self.workload.validate_launch_environment()?;
|
||||
let workload = match &self.workload {
|
||||
WorkloadBinding::HostActuator(workload) => {
|
||||
|
|
@ -1326,6 +1495,32 @@ impl OperatorBinding {
|
|||
);
|
||||
return self.validate_shared_tail();
|
||||
}
|
||||
WorkloadBinding::StaticRelease(workload) => {
|
||||
workload.validate()?;
|
||||
ensure!(
|
||||
self.process_write_lease.is_none() && self.route.is_none(),
|
||||
"a static release has no write lease or route"
|
||||
);
|
||||
ensure!(
|
||||
self.rollout.strategy == RolloutStrategy::CandidateThenPromote,
|
||||
"a static release rolls out candidate-then-promote: it never stops an incumbent"
|
||||
);
|
||||
for authority in [
|
||||
&self.brakes.deployment_store,
|
||||
&self.brakes.lifecycle_store,
|
||||
&self.runtime_identity.trust_anchor_store,
|
||||
]
|
||||
.into_iter()
|
||||
.chain(self.brakes.steward.as_ref().map(|steward| &steward.store))
|
||||
{
|
||||
ensure!(
|
||||
!paths_overlap(authority, &workload.release_root)
|
||||
&& !paths_overlap(authority, &workload.current_link),
|
||||
"static release path overlaps an Idunn authority path"
|
||||
);
|
||||
}
|
||||
return self.validate_shared_tail();
|
||||
}
|
||||
WorkloadBinding::SystemdTransient(workload) => {
|
||||
ensure!(
|
||||
self.rollout.strategy == RolloutStrategy::CandidateThenPromote,
|
||||
|
|
@ -1386,6 +1581,7 @@ impl OperatorBinding {
|
|||
self.brakes.deployment_store.clone(),
|
||||
self.brakes.lifecycle_store.clone(),
|
||||
];
|
||||
authority_paths.extend(self.brakes.steward.as_ref().map(|steward| steward.store.clone()));
|
||||
if let Some(write_lease) = &self.process_write_lease {
|
||||
authority_paths.push(write_lease.record_path.clone());
|
||||
authority_paths.push(write_lease.lock_path());
|
||||
|
|
@ -1553,6 +1749,11 @@ impl OperatorBinding {
|
|||
self.target == declaration.target,
|
||||
"binding target does not match declaration"
|
||||
);
|
||||
ensure!(
|
||||
declaration.static_release.is_some()
|
||||
== matches!(self.workload, WorkloadBinding::StaticRelease(_)),
|
||||
"a static release declaration needs a static-release workload, and only it"
|
||||
);
|
||||
let bound_gitlinks: BTreeSet<_> = self.repository.gitlinks.keys().cloned().collect();
|
||||
ensure!(
|
||||
bound_gitlinks == declaration.required_gitlinks,
|
||||
|
|
@ -1664,6 +1865,19 @@ impl OperatorBinding {
|
|||
input.id
|
||||
);
|
||||
}
|
||||
let Some(service) = declaration.service.as_ref() else {
|
||||
let WorkloadBinding::StaticRelease(workload) = &self.workload else {
|
||||
bail!("a static release declaration needs a static-release workload");
|
||||
};
|
||||
ensure!(
|
||||
declaration
|
||||
.artifacts
|
||||
.iter()
|
||||
.all(|artifact| artifact.destination != workload.carry.destination),
|
||||
"static carry destination collides with an artifact destination"
|
||||
);
|
||||
return Ok(());
|
||||
};
|
||||
let workload_names: BTreeSet<&String> = self
|
||||
.workload
|
||||
.environment()
|
||||
|
|
@ -1690,7 +1904,7 @@ impl OperatorBinding {
|
|||
if host_workload {
|
||||
available_environment.insert(IDUNN_ACTIVATION_CREDENTIAL_ENVIRONMENT.into());
|
||||
}
|
||||
if declaration.service.route_required {
|
||||
if service.route_required {
|
||||
available_environment.insert(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT.into());
|
||||
if self.route.as_ref().is_some_and(|route| route.member.is_some()) {
|
||||
available_environment.insert(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT.into());
|
||||
|
|
@ -1700,16 +1914,14 @@ impl OperatorBinding {
|
|||
available_environment.insert(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT.into());
|
||||
}
|
||||
ensure!(
|
||||
declaration
|
||||
.service
|
||||
service
|
||||
.required_environment
|
||||
.is_subset(&available_environment),
|
||||
"operator service environment omits a required launch input"
|
||||
);
|
||||
let declared_environment: BTreeSet<_> = declaration
|
||||
.service
|
||||
let declared_environment: BTreeSet<_> = service
|
||||
.required_environment
|
||||
.union(&declaration.service.optional_environment)
|
||||
.union(&service.optional_environment)
|
||||
.cloned()
|
||||
.collect();
|
||||
// On the Idunn host the presence identity reaches the workload as a
|
||||
|
|
@ -1729,8 +1941,7 @@ impl OperatorBinding {
|
|||
available_environment.is_subset(&declared_environment),
|
||||
"operator service environment contains an undeclared launch input"
|
||||
);
|
||||
let required_arguments: BTreeSet<_> = declaration
|
||||
.service
|
||||
let required_arguments: BTreeSet<_> = service
|
||||
.arguments
|
||||
.iter()
|
||||
.filter_map(|argument| match argument {
|
||||
|
|
@ -1746,6 +1957,7 @@ impl OperatorBinding {
|
|||
.as_ref()
|
||||
.map(|path| path.display().to_string()),
|
||||
WorkloadBinding::HostActuator(workload) => workload.state_root.clone(),
|
||||
WorkloadBinding::StaticRelease(_) => None,
|
||||
};
|
||||
match (
|
||||
state_root.as_deref(),
|
||||
|
|
@ -1768,8 +1980,8 @@ impl OperatorBinding {
|
|||
"operator argument bindings do not exactly match the launch contract"
|
||||
);
|
||||
match (
|
||||
declaration.service.route_required,
|
||||
declaration.service.transport,
|
||||
service.route_required,
|
||||
service.transport,
|
||||
self.route.as_ref(),
|
||||
) {
|
||||
(true, ServiceTransport::Http, Some(route))
|
||||
|
|
@ -2211,6 +2423,76 @@ nodes = ["yggdrasil"]
|
|||
assert!(TargetDeclaration::parse(&input).is_err());
|
||||
}
|
||||
|
||||
const MEMBER_CANARY: &str = "zzcanary";
|
||||
|
||||
fn recipe_with_members(members: &str) -> String {
|
||||
let input = RECIPE.replace("executable = true", &format!("executable_members = [{members}]"));
|
||||
assert_ne!(input, RECIPE, "the fixture lost its executable line");
|
||||
input
|
||||
}
|
||||
|
||||
/// A refused member list is refused by name of the field, never by echoing
|
||||
/// the member.
|
||||
fn assert_members_refused(members: &[&str]) {
|
||||
let control = TargetDeclaration::parse(&recipe_with_members("\"bin/daemon\"")).unwrap();
|
||||
assert!(control.artifacts[0].executable_members.contains(Path::new("bin/daemon")));
|
||||
for member in members {
|
||||
let error = TargetDeclaration::parse(&recipe_with_members(&format!("\"{member}\"")))
|
||||
.expect_err(member);
|
||||
assert!(
|
||||
!format!("{error:#}").contains(MEMBER_CANARY),
|
||||
"the refusal echoed a member path"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recipe_executable_member_refuses_parent_traversal() {
|
||||
assert_members_refused(&[
|
||||
"zzcanary/../b",
|
||||
"../zzcanary",
|
||||
"bin/zzcanary/..",
|
||||
"bin/../../zzcanary",
|
||||
]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recipe_executable_member_refuses_absolute_path() {
|
||||
assert_members_refused(&["/zzcanary/sh", "/zzcanary"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recipe_executable_member_refuses_empty_and_current_dir() {
|
||||
assert_members_refused(&["", ".", "./", "zzcanary/.."]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recipe_refuses_executable_flag_with_members() {
|
||||
let input = RECIPE.replace(
|
||||
"executable = true",
|
||||
"executable = true\nexecutable_members = [\"bin/daemon\"]",
|
||||
);
|
||||
let error = TargetDeclaration::parse(&input).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("executable_members"));
|
||||
assert!(!format!("{error:#}").contains("bin/daemon"));
|
||||
// executable = true alone, and members alone, both stay admitted.
|
||||
assert!(TargetDeclaration::parse(RECIPE).is_ok());
|
||||
assert!(TargetDeclaration::parse(&recipe_with_members("\"bin/daemon\"")).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recipe_admits_members_on_both_source_kinds() {
|
||||
let runner_output = TargetDeclaration::parse(&recipe_with_members("\"a/b\", \"c\"")).unwrap();
|
||||
assert_eq!(runner_output.artifacts[0].source_kind, ArtifactSource::RunnerOutput);
|
||||
assert_eq!(runner_output.artifacts[0].executable_members.len(), 2);
|
||||
let worktree = recipe_with_members("\"a/b\", \"c\"")
|
||||
.replace("source_kind = \"runner-output\"\nrunner = \"rust\"\n", "source_kind = \"worktree-tree\"\n")
|
||||
.replace("expected_sha256 = \"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc\"\n", "");
|
||||
let worktree = TargetDeclaration::parse(&worktree).unwrap();
|
||||
assert_eq!(worktree.artifacts[0].source_kind, ArtifactSource::WorktreeTree);
|
||||
assert_eq!(worktree.artifacts[0].executable_members.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn operator_cannot_omit_an_affordance_derived_from_the_recipe() {
|
||||
let recipe = TargetDeclaration::parse(RECIPE).unwrap();
|
||||
|
|
@ -2262,7 +2544,7 @@ nodes = ["yggdrasil"]
|
|||
)
|
||||
.is_err()
|
||||
);
|
||||
assert!(recipe.service.route_required);
|
||||
assert!(recipe.service().unwrap().route_required);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -2750,4 +3032,230 @@ max_connections = {value}"),
|
|||
"v2",
|
||||
));
|
||||
}
|
||||
|
||||
/// A static release: a vite-style tree published by repointing a link.
|
||||
pub(crate) const STATIC_RECIPE: &str = r#"
|
||||
schema = "gamecult.idunn.target_declaration.v1"
|
||||
target = "service"
|
||||
source_stamp_environment = "SERVICE_BUILD_COMMIT"
|
||||
|
||||
[[steps]]
|
||||
id = "build"
|
||||
phase = "build"
|
||||
runner = "rust"
|
||||
argv = ["true"]
|
||||
|
||||
[[artifacts]]
|
||||
id = "dist"
|
||||
source_kind = "worktree-tree"
|
||||
source = "dist"
|
||||
destination = "dist"
|
||||
|
||||
[static_release]
|
||||
artifact = "dist"
|
||||
"#;
|
||||
|
||||
/// `@SITE@` stands for the stand-in site tree a test lays out.
|
||||
pub(crate) const STATIC_BINDING: &str = r#"
|
||||
schema = "gamecult.idunn.operator_binding.v2"
|
||||
target = "service"
|
||||
|
||||
[repository]
|
||||
origin = "https://github.com/GameCult/Service.git"
|
||||
admitted_ref = "refs/heads/main"
|
||||
minimum_revision = "1111111111111111111111111111111111111111"
|
||||
selection = "ref-head"
|
||||
checkout = "/srv/build/Service"
|
||||
recipe_path = "deployment/idunn/recipe.toml"
|
||||
|
||||
[runners.rust]
|
||||
driver = "docker"
|
||||
image = "rust@sha256:4c2fd73ef19c5ef9d54bee03b06b2839a392604fbfcd578ed948b71b37c1d7fb"
|
||||
user = "1000:1000"
|
||||
affordances = ["source-read", "artifact-write"]
|
||||
allowed_programs = ["true"]
|
||||
memory_mebibytes = 1024
|
||||
cpu_quota_percent = 100
|
||||
pids_limit = 128
|
||||
tmpfs_mebibytes = 64
|
||||
|
||||
[workload]
|
||||
driver = "static-release"
|
||||
release_root = "@SITE@/releases"
|
||||
current_link = "@SITE@/www/current"
|
||||
|
||||
[workload.carry]
|
||||
source = "@SITE@/config/site.json"
|
||||
destination = "config.json"
|
||||
|
||||
[runtime_identity]
|
||||
runtime_id = "site-yggdrasil"
|
||||
expected_signer_identity_id = "site-runtime-signer"
|
||||
trust_anchor_store = "@SITE@/trust/site.cc"
|
||||
|
||||
[brakes]
|
||||
deployment_store = "@SITE@/authority/deployment-brake.cc"
|
||||
lifecycle_store = "@SITE@/authority/lifecycle-brake.cc"
|
||||
|
||||
[rollout]
|
||||
strategy = "candidate-then-promote"
|
||||
drain_seconds = 1
|
||||
retain_releases = 2
|
||||
|
||||
[placement]
|
||||
desired_replicas = 1
|
||||
nodes = ["yggdrasil"]
|
||||
"#;
|
||||
|
||||
fn static_binding(replace: &[(&str, &str)]) -> String {
|
||||
let mut text = STATIC_BINDING.replace("@SITE@", "/srv/site");
|
||||
for (from, to) in replace {
|
||||
assert!(text.contains(from), "the fixture lost {from}");
|
||||
text = text.replace(from, to);
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
const STEWARD: &str = "\n[brakes.steward]\nuser = \"mind-streampixels\"\nstore = \"/var/lib/gamecult/idunn-steward/streampixels/deployment-brake.cc\"\n";
|
||||
|
||||
#[test]
|
||||
fn binding_without_steward_parses_and_admits_on_operator_store_only() {
|
||||
for text in [BINDING.to_string(), static_binding(&[])] {
|
||||
assert_eq!(OperatorBinding::parse(&text).unwrap().brakes.steward, None);
|
||||
}
|
||||
let steward = OperatorBinding::parse(&BINDING.replace("\n[rollout]", &format!("{STEWARD}\n[rollout]")))
|
||||
.unwrap()
|
||||
.brakes
|
||||
.steward
|
||||
.unwrap();
|
||||
assert_eq!(steward.user, "mind-streampixels");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_steward_brake_must_be_a_distinct_absolute_non_root_login_and_clear_of_the_workload() {
|
||||
let with = |steward: &str| BINDING.replace("\n[rollout]", &format!("{steward}\n[rollout]"));
|
||||
let store = "/var/lib/gamecult/idunn-steward/x/deployment-brake.cc";
|
||||
for bad in [
|
||||
format!("\n[brakes.steward]\nuser = \"root\"\nstore = \"{store}\"\n"),
|
||||
format!("\n[brakes.steward]\nuser = \"Bad User\"\nstore = \"{store}\"\n"),
|
||||
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"relative.cc\"\n".into(),
|
||||
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/idunn-authority/test-service-deployment-brake.cc\"\n".into(),
|
||||
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/idunn-authority/test-service-lifecycle-brake.cc\"\n".into(),
|
||||
format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{store}\"\nextra = 1\n"),
|
||||
] {
|
||||
assert!(OperatorBinding::parse(&with(&bad)).is_err(), "{bad}");
|
||||
}
|
||||
// A steward directory shared with, or enclosing, an operator store
|
||||
// lets the steward unlink a root-owned file: refused.
|
||||
for (deployment, steward_store) in [
|
||||
("/var/lib/gamecult/steward-x/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||
("/var/lib/gamecult/steward-x/sub/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||
("/var/lib/gamecult/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||
] {
|
||||
let shared = BINDING
|
||||
.replace("/var/lib/gamecult/idunn-authority/test-service-deployment-brake.cc", deployment)
|
||||
.replace("\n[rollout]", &format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{steward_store}\"\n\n[rollout]"));
|
||||
assert!(OperatorBinding::parse(&shared).is_err(), "{deployment} {steward_store}");
|
||||
}
|
||||
// Disjoint directories are the one admitted shape.
|
||||
let disjoint = BINDING.replace(
|
||||
"\n[rollout]",
|
||||
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/steward-x/s.cc\"\n\n[rollout]",
|
||||
);
|
||||
OperatorBinding::parse(&disjoint).unwrap();
|
||||
// steward_store_overlap_with_workload_path_refused
|
||||
let overlapping = BINDING
|
||||
.replace("\n[rollout]", &format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{store}\"\n\n[rollout]"))
|
||||
.replace(
|
||||
"cpu_quota_percent = 200\n\n[workload.argument_bindings]",
|
||||
"cpu_quota_percent = 200\nread_write_paths = [\"/var/lib/gamecult/idunn-steward/x\"]\n\n[workload.argument_bindings]",
|
||||
);
|
||||
assert!(OperatorBinding::parse(&overlapping).is_err());
|
||||
let static_overlap = static_binding(&[("[rollout]", "[brakes.steward]\nuser = \"mind\"\nstore = \"/srv/site/releases/x/brake.cc\"\n\n[rollout]")]);
|
||||
assert!(OperatorBinding::parse(&static_overlap).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_recipe_and_binding_admit() {
|
||||
let recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||
let binding = OperatorBinding::parse(&static_binding(&[])).unwrap();
|
||||
binding.admit(&recipe).unwrap();
|
||||
assert!(recipe.service().is_err());
|
||||
assert_eq!(binding.workload.release_root().unwrap(), Path::new("/srv/site/releases"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_recipe_declares_exactly_one_of_service_and_static_release() {
|
||||
let both = format!("{RECIPE}\n[static_release]\nartifact = \"daemon\"\n");
|
||||
let error = TargetDeclaration::parse(&both).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("both a service and a static release"));
|
||||
let neither = STATIC_RECIPE.replace("[static_release]\nartifact = \"dist\"\n", "");
|
||||
let error = TargetDeclaration::parse(&neither).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("neither a service nor a static release"));
|
||||
let unknown = STATIC_RECIPE.replace("artifact = \"dist\"", "artifact = \"nope\"");
|
||||
assert!(TargetDeclaration::parse(&unknown).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_release_refuses_a_workload_contract() {
|
||||
for extra in [
|
||||
"\n[[provides]]\ncapability = \"site.runtime\"\nschema = \"site.v1\"\ncompatibility = \"v1\"\n",
|
||||
"\n[[conflicts]]\ncapability = \"site.old\"\nreason = \"replaced\"\n",
|
||||
"\n[state]\nschema_generation = \"v1\"\n\n[[state.slots]]\nid = \"world\"\nrelative_path = \"world.cc\"\nkind = \"cultcache-file\"\nschema = \"site.v1\"\nwriter = \"none\"\nrecovery = \"preserve\"\nstartup = \"open-at-start\"\n",
|
||||
] {
|
||||
let error = TargetDeclaration::parse(&format!("{STATIC_RECIPE}{extra}")).unwrap_err();
|
||||
assert!(
|
||||
format!("{error:#}").contains("a static release declares no state"),
|
||||
"{extra}: {error:#}"
|
||||
);
|
||||
}
|
||||
for (from, to, refusal) in [
|
||||
("[rollout]", "[route]\ndriver = \"nginx-stream-tcp\"\nroute_id = \"r\"\nstable_endpoint = \"http://127.0.0.1:18830\"\nprivate_host = \"127.0.0.1\"\nprivate_port_start = 18831\nprivate_port_end = 18839\nconfig_path = \"/etc/nginx/r.conf\"\nreload_unit = \"nginx.service\"\n\n[rollout]", "no write lease or route"),
|
||||
("[rollout]", "[process_write_lease]\nrecord_path = \"/srv/site/lease.cc\"\n\n[rollout]", "no write lease or route"),
|
||||
("retain_releases = 2", "retain_releases = 1", "retain at least current and prior"),
|
||||
("candidate-then-promote", "stop-then-start", "never stops an incumbent"),
|
||||
] {
|
||||
let error = OperatorBinding::parse(&static_binding(&[(from, to)])).unwrap_err();
|
||||
assert!(format!("{error:#}").contains(refusal), "{to}: {error:#}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_release_and_service_recipes_need_their_own_binding_kind() {
|
||||
let static_recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||
let service_recipe = TargetDeclaration::parse(RECIPE).unwrap();
|
||||
let static_binding = OperatorBinding::parse(&static_binding(&[])).unwrap();
|
||||
let service_binding = OperatorBinding::parse(BINDING).unwrap();
|
||||
assert!(static_binding.admit(&service_recipe).is_err());
|
||||
assert!(service_binding.admit(&static_recipe).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_carry_destination_must_not_be_an_artifact_destination() {
|
||||
let recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||
let binding =
|
||||
OperatorBinding::parse(&static_binding(&[("destination = \"config.json\"", "destination = \"dist\"")]))
|
||||
.unwrap();
|
||||
let error = binding.admit(&recipe).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("collides with an artifact destination"));
|
||||
for destination in ["a/b", "../b", "/b", ""] {
|
||||
let line = format!("destination = \"{destination}\"");
|
||||
assert!(
|
||||
OperatorBinding::parse(&static_binding(&[("destination = \"config.json\"", &line)])).is_err(),
|
||||
"{destination}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn static_carry_source_and_link_stay_out_of_the_release_root() {
|
||||
for (from, to) in [
|
||||
("source = \"/srv/site/config/site.json\"", "source = \"/srv/site/releases/site.json\""),
|
||||
("current_link = \"/srv/site/www/current\"", "current_link = \"/srv/site/releases/current\""),
|
||||
("source = \"/srv/site/config/site.json\"", "source = \"site.json\""),
|
||||
("release_root = \"/srv/site/releases\"", "release_root = \"/srv/site/www\""),
|
||||
] {
|
||||
assert!(OperatorBinding::parse(&static_binding(&[(from, to)])).is_err(), "{to}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -734,6 +734,11 @@ pub struct ArtifactReceipt {
|
|||
pub sha256: String,
|
||||
pub size_bytes: u64,
|
||||
pub executable: bool,
|
||||
/// Files sealed executable inside a directory artifact. Skipped when empty
|
||||
/// so a receipt with no members keeps its original encoding, and so its
|
||||
/// sealed release id.
|
||||
#[serde(default, skip_serializing_if = "BTreeSet::is_empty")]
|
||||
pub executable_members: BTreeSet<PathBuf>,
|
||||
}
|
||||
|
||||
impl ArtifactReceipt {
|
||||
|
|
@ -747,7 +752,9 @@ impl ArtifactReceipt {
|
|||
.find(|artifact| artifact.id == self.artifact_id)
|
||||
.with_context(|| format!("plan declares no artifact {}", self.artifact_id))?;
|
||||
ensure!(
|
||||
self.destination == declared.destination && self.executable == declared.executable,
|
||||
self.destination == declared.destination
|
||||
&& self.executable == declared.executable
|
||||
&& self.executable_members == declared.executable_members,
|
||||
"artifact receipt differs from declared output"
|
||||
);
|
||||
if let Some(expected_sha256) = &declared.expected_sha256 {
|
||||
|
|
@ -847,14 +854,15 @@ impl SealedRelease {
|
|||
) -> Result<ExpectedIncarnation> {
|
||||
self.validate_against(plan)?;
|
||||
let (declaration, binding) = plan.parsed_inputs()?;
|
||||
let service = declaration.service()?;
|
||||
let route = match (&binding.route, plan.candidate_port) {
|
||||
(Some(bound), Some(port)) => Some(IdunnExpectedRoute {
|
||||
route_id: bound.route_id.clone(),
|
||||
transport: endpoint_scheme(declaration.service.transport)?.into(),
|
||||
transport: endpoint_scheme(service.transport)?.into(),
|
||||
stable_endpoint: bound.stable_endpoint.clone(),
|
||||
candidate_endpoint: format!(
|
||||
"{}://{}:{port}",
|
||||
endpoint_scheme(declaration.service.transport)?,
|
||||
endpoint_scheme(service.transport)?,
|
||||
bound.private_host
|
||||
),
|
||||
}),
|
||||
|
|
@ -864,7 +872,7 @@ impl SealedRelease {
|
|||
let artifact_sha256 = self
|
||||
.artifacts
|
||||
.iter()
|
||||
.find(|artifact| artifact.artifact_id == declaration.service.executable_artifact)
|
||||
.find(|artifact| artifact.artifact_id == service.executable_artifact)
|
||||
.context("sealed release has no executable service artifact")?
|
||||
.sha256
|
||||
.clone();
|
||||
|
|
@ -911,7 +919,7 @@ impl SealedRelease {
|
|||
.runtime_identity
|
||||
.expected_signer_identity_id
|
||||
.clone(),
|
||||
health_contract: declaration.service.health.contract.clone(),
|
||||
health_contract: service.health.contract.clone(),
|
||||
artifact_sha256,
|
||||
state_schema_generation: declaration
|
||||
.state
|
||||
|
|
@ -1425,6 +1433,7 @@ nodes = ["yggdrasil"]
|
|||
sha256: digest(b'c'),
|
||||
size_bytes: 42,
|
||||
executable: true,
|
||||
executable_members: BTreeSet::new(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1791,6 +1800,113 @@ nodes = ["yggdrasil"]
|
|||
assert!(changed.validate_against(&plan).is_err());
|
||||
}
|
||||
|
||||
fn hex(bytes: &[u8]) -> String {
|
||||
bytes.iter().map(|byte| format!("{byte:02x}")).collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn receipt_without_members_encodes_as_before() {
|
||||
// Both values were computed outside this code, from the five-field
|
||||
// msgpack array the receipt encoded to at base d74ac8d.
|
||||
const BASE_RECEIPT: &str = "95a66461656d6f6ea773657276696365d9477368613235362d636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363632ac3";
|
||||
const BASE_RELEASE_ID: &str =
|
||||
"sha256-b9143d8f546ad71c493c70037813547aaec10fd87994a6f3bdc77ac3a8b3495e";
|
||||
let receipt = artifact_receipt();
|
||||
let encoded = rmp_serde::to_vec(&receipt).unwrap();
|
||||
assert_eq!(hex(&encoded), BASE_RECEIPT);
|
||||
let bytes = (0..BASE_RECEIPT.len())
|
||||
.step_by(2)
|
||||
.map(|at| u8::from_str_radix(&BASE_RECEIPT[at..at + 2], 16).unwrap())
|
||||
.collect::<Vec<_>>();
|
||||
assert_eq!(rmp_serde::from_slice::<ArtifactReceipt>(&bytes).unwrap(), receipt);
|
||||
let release = SealedRelease {
|
||||
schema: SEALED_RELEASE_SCHEMA.into(),
|
||||
sealed_release_id: String::new(),
|
||||
plan_id: digest(b'a'),
|
||||
artifacts: vec![receipt],
|
||||
external_inputs: vec![],
|
||||
sealed_at_unix_millis: 0,
|
||||
};
|
||||
assert_eq!(release.recomputed_release_id().unwrap(), BASE_RELEASE_ID);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sealed_release_address_covers_executable_members() {
|
||||
let release_with = |members: &[&str]| SealedRelease {
|
||||
schema: SEALED_RELEASE_SCHEMA.into(),
|
||||
sealed_release_id: String::new(),
|
||||
plan_id: digest(b'a'),
|
||||
artifacts: vec![ArtifactReceipt {
|
||||
executable: false,
|
||||
executable_members: members.iter().map(PathBuf::from).collect(),
|
||||
..artifact_receipt()
|
||||
}],
|
||||
external_inputs: vec![],
|
||||
sealed_at_unix_millis: 0,
|
||||
};
|
||||
let none = release_with(&[]).recomputed_release_id().unwrap();
|
||||
let one = release_with(&["bin/daemon"]).recomputed_release_id().unwrap();
|
||||
let other = release_with(&["bin/other"]).recomputed_release_id().unwrap();
|
||||
let two = release_with(&["bin/daemon", "bin/other"])
|
||||
.recomputed_release_id()
|
||||
.unwrap();
|
||||
let all = [&none, &one, &other, &two];
|
||||
for (index, left) in all.iter().enumerate() {
|
||||
for right in &all[index + 1..] {
|
||||
assert_ne!(left, right);
|
||||
}
|
||||
}
|
||||
|
||||
// The declaration is the other half: a receipt must carry exactly the
|
||||
// declared list, in a plan that declares one and in one that does not.
|
||||
let plain = plan();
|
||||
let mut with_members = artifact_receipt();
|
||||
with_members.executable_members = BTreeSet::from([PathBuf::from("bin/daemon")]);
|
||||
assert!(
|
||||
SealedRelease::new(&plain, vec![with_members], vec![external_input_receipt()], 120)
|
||||
.is_err()
|
||||
);
|
||||
let recipe = RECIPE.replace("executable = true", "executable_members = [\"bin/daemon\", \"lib/tool\"]");
|
||||
let declaring = compile_deployment_plan(
|
||||
recipe.as_bytes(),
|
||||
BINDING.as_bytes(),
|
||||
source(&recipe),
|
||||
"service-incarnation-1",
|
||||
Some(18001),
|
||||
110,
|
||||
&[ready_odin_provider("odin", "odin-yggdrasil", 1)],
|
||||
)
|
||||
.unwrap();
|
||||
let honest = ArtifactReceipt {
|
||||
executable: false,
|
||||
executable_members: BTreeSet::from([PathBuf::from("bin/daemon"), PathBuf::from("lib/tool")]),
|
||||
..artifact_receipt()
|
||||
};
|
||||
let release = SealedRelease::new(
|
||||
&declaring,
|
||||
vec![honest.clone()],
|
||||
vec![external_input_receipt()],
|
||||
120,
|
||||
)
|
||||
.unwrap();
|
||||
release.validate_against(&declaring).unwrap();
|
||||
for drifted in [
|
||||
BTreeSet::from([PathBuf::from("bin/daemon")]),
|
||||
BTreeSet::from([PathBuf::from("bin/daemon"), PathBuf::from("lib/other")]),
|
||||
BTreeSet::new(),
|
||||
] {
|
||||
let mut receipt = honest.clone();
|
||||
receipt.executable_members = drifted;
|
||||
assert!(
|
||||
SealedRelease::new(&declaring, vec![receipt], vec![external_input_receipt()], 120)
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
let mut changed = release.clone();
|
||||
changed.artifacts[0].executable_members.insert(PathBuf::from("lib/extra"));
|
||||
assert!(changed.validate_against(&declaring).is_err());
|
||||
}
|
||||
|
||||
fn binding_with_deadlines(table: &str) -> String {
|
||||
BINDING.replace("[placement]", &format!("{table}\n[placement]"))
|
||||
}
|
||||
|
|
|
|||
2100
src/drivers.rs
2100
src/drivers.rs
File diff suppressed because it is too large
Load diff
49
src/host.rs
49
src/host.rs
|
|
@ -30,13 +30,13 @@ use serde::Deserialize;
|
|||
|
||||
use crate::control_plane::read_trust_anchor;
|
||||
use crate::deployment::{
|
||||
ArtifactSource, HostWorkloadBinding, IDUNN_ACTIVATION_CREDENTIAL_ENVIRONMENT,
|
||||
HostWorkloadBinding, IDUNN_ACTIVATION_CREDENTIAL_ENVIRONMENT,
|
||||
IDUNN_RUNTIME_BUNDLE_ENVIRONMENT, LaunchArgument, OperatorBinding, TargetDeclaration,
|
||||
};
|
||||
use crate::deployment_plan::{ArtifactReceipt, CompiledDeploymentPlan, SealedRelease};
|
||||
use crate::deployment_plan::{CompiledDeploymentPlan, SealedRelease};
|
||||
use crate::drivers::{
|
||||
HostWorkloadObservation, InstalledReleaseObservation, copy_artifact, copy_tree,
|
||||
digest_artifact, release_artifact, remove_tree_inside, sha256_id, write_runtime_bundle_records,
|
||||
HostWorkloadObservation, InstalledReleaseObservation, copy_tree, digest_artifact,
|
||||
release_artifact, remove_tree_inside, seal_artifact, sha256_id, write_runtime_bundle_records,
|
||||
};
|
||||
use crate::host_actuator::{
|
||||
HOST_ACTUATOR_CHANNEL, HOST_ACTUATOR_CONNECTION_ID, HOST_ACTUATOR_PING_MS,
|
||||
|
|
@ -631,11 +631,11 @@ impl Actuator {
|
|||
workload: &HostWorkloadBinding,
|
||||
installed: &Path,
|
||||
) -> Result<(PathBuf, Vec<String>)> {
|
||||
let artifact = release_artifact(declaration, &declaration.service.executable_artifact)?;
|
||||
let artifact = release_artifact(declaration, &declaration.service()?.executable_artifact)?;
|
||||
let executable = installed.join(&artifact.destination);
|
||||
ensure!(executable.is_file(), "sealed service executable is absent");
|
||||
let mut arguments = Vec::new();
|
||||
for argument in &declaration.service.arguments {
|
||||
for argument in &declaration.service()?.arguments {
|
||||
arguments.push(match argument {
|
||||
LaunchArgument::Literal { value } => value.clone(),
|
||||
LaunchArgument::Binding { name } => workload
|
||||
|
|
@ -1048,37 +1048,12 @@ fn materialize(
|
|||
fs::create_dir_all(&staging)?;
|
||||
let mut artifacts = Vec::new();
|
||||
for artifact in &declaration.artifacts {
|
||||
let source_path = match artifact.source_kind {
|
||||
ArtifactSource::RunnerOutput | ArtifactSource::WorktreeTree => {
|
||||
source.join(&artifact.source)
|
||||
}
|
||||
};
|
||||
ensure!(
|
||||
source_path.exists(),
|
||||
"declared artifact output {} is absent",
|
||||
artifact.id
|
||||
);
|
||||
let destination = staging.join(&artifact.destination);
|
||||
ensure!(
|
||||
destination.starts_with(&staging),
|
||||
"artifact destination escaped its staging root"
|
||||
);
|
||||
copy_artifact(&source, &source_path, &destination)?;
|
||||
let (sha256, size_bytes) = digest_artifact(&destination)?;
|
||||
if let Some(expected) = &artifact.expected_sha256 {
|
||||
ensure!(
|
||||
&sha256 == expected,
|
||||
"artifact {} differs from its recipe-pinned digest",
|
||||
artifact.id
|
||||
);
|
||||
}
|
||||
artifacts.push(ArtifactReceipt {
|
||||
artifact_id: artifact.id.clone(),
|
||||
destination: artifact.destination.clone(),
|
||||
sha256: format!("sha256-{sha256}"),
|
||||
size_bytes,
|
||||
executable: artifact.executable,
|
||||
});
|
||||
artifacts.push(seal_artifact(
|
||||
artifact,
|
||||
&source,
|
||||
&source.join(&artifact.source),
|
||||
&staging,
|
||||
)?);
|
||||
}
|
||||
let release = SealedRelease::new(plan, artifacts, Vec::new(), sealed_at_unix_millis)?;
|
||||
Ok((release, staging))
|
||||
|
|
|
|||
|
|
@ -1151,7 +1151,7 @@ mod tests {
|
|||
let answerer = thread::spawn(move || -> Result<()> {
|
||||
for _ in 0..500 {
|
||||
client.poll_resends()?;
|
||||
while let Some(frame) = client.receive_once()? {
|
||||
if let Some(frame) = client.receive_once()? {
|
||||
let envelope = HostActuatorEnvelope::decode(&frame.payload)?;
|
||||
let request = envelope.open_request("raven", &idunn_anchor)?;
|
||||
assert_eq!(envelope.session_nonce, nonce);
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
mod brake;
|
||||
pub mod control_plane;
|
||||
pub mod deployment;
|
||||
pub mod deployment_plan;
|
||||
|
|
@ -7,4 +8,7 @@ pub mod host_actuator;
|
|||
#[cfg(unix)]
|
||||
pub mod launcher;
|
||||
mod incident;
|
||||
mod runner_evidence;
|
||||
pub mod provisioning;
|
||||
#[cfg(unix)]
|
||||
mod steward_orders;
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
use anyhow::{Context, Result, anyhow, bail};
|
||||
use crate::brake::{Stop, StopState, process_uid, read_stop, replace_stop};
|
||||
use anyhow::{Context, Result, anyhow, bail, ensure};
|
||||
use cultcache_rs::{
|
||||
CultCacheEnvelope, CultCacheExpectedEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore,
|
||||
};
|
||||
|
|
@ -6,15 +7,13 @@ use cultnet_rs::{
|
|||
GAMECULT_SERVICE_TRUST_ANCHOR_SCHEMA, GameCultProviderHealthIdentity,
|
||||
GameCultServiceTrustAnchorRecord, IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SCHEMA,
|
||||
IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SIGNING_PURPOSE,
|
||||
IDUNN_DAEMON_HEALTH_TRUST_BINDING_SCHEMA, IDUNN_DEPLOYMENT_BRAKE_AUTHORITY,
|
||||
IDUNN_DEPLOYMENT_BRAKE_ID, IDUNN_DEPLOYMENT_BRAKE_SCHEMA, IDUNN_DEPLOYMENT_BRAKE_SCOPE,
|
||||
IDUNN_DEPLOYMENT_RELEASE_PURPOSE, IDUNN_LIFECYCLE_BRAKE_AUTHORITY,
|
||||
IDUNN_DAEMON_HEALTH_TRUST_BINDING_SCHEMA,
|
||||
IDUNN_LIFECYCLE_BRAKE_AUTHORITY,
|
||||
IDUNN_LIFECYCLE_BRAKE_SCHEMA, IDUNN_LIFECYCLE_BRAKE_SCOPE, IdunnDaemonHealthTrustBindingRecord,
|
||||
IdunnDeploymentBrakeObservation, IdunnDeploymentBrakeOperatorIdentity,
|
||||
IdunnDeploymentBrakeRecord, IdunnDeploymentBrakeReleasePurpose, IdunnLifecycleBrakeObservation,
|
||||
IdunnLifecycleBrakeObservation,
|
||||
IdunnLifecycleBrakeRecord, IdunnServiceIdentity, OdinTopologyIdentity, ServiceIdentityProfile,
|
||||
ServiceIdentityTrustAnchor, derive_service_identity_id, enroll_service_identity_at,
|
||||
evaluate_idunn_continuity_restart, evaluate_idunn_deployment_brake,
|
||||
evaluate_idunn_continuity_restart,
|
||||
export_service_identity_trust_anchor, open_service_identity_at,
|
||||
};
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
|
@ -80,17 +79,8 @@ pub fn run(args: impl IntoIterator<Item = String>) -> Result<()> {
|
|||
"export-odin-topology-public-anchor" => {
|
||||
export_identity_anchor::<OdinTopologyIdentity>(&options, "Odin topology public anchor")?
|
||||
}
|
||||
"enroll-deployment-brake-operator" => {
|
||||
enroll_identity::<IdunnDeploymentBrakeOperatorIdentity>(&options)?
|
||||
}
|
||||
"export-deployment-brake-operator-anchor" => {
|
||||
export_identity_anchor::<IdunnDeploymentBrakeOperatorIdentity>(
|
||||
&options,
|
||||
"deployment brake operator anchor",
|
||||
)?
|
||||
}
|
||||
"deployment-brake-engage" => deployment_brake_engage(&options)?,
|
||||
"deployment-brake-release" => deployment_brake_release(&options)?,
|
||||
"deployment-brake-engage" => deployment_brake_write(&options, StopState::Engaged)?,
|
||||
"deployment-brake-clear" => deployment_brake_write(&options, StopState::Clear)?,
|
||||
"deployment-brake-status" => deployment_brake_status(&options)?,
|
||||
"lifecycle-brake-engage" => lifecycle_brake_engage(&options)?,
|
||||
"lifecycle-brake-release" => lifecycle_brake_release(&options)?,
|
||||
|
|
@ -157,153 +147,60 @@ fn provider_health_public_key(options: &BTreeMap<String, String>) -> Result<Stri
|
|||
encode_public_key(&signer.entry().public_key)
|
||||
}
|
||||
|
||||
fn deployment_brake_engage(options: &BTreeMap<String, String>) -> Result<()> {
|
||||
require_only(
|
||||
options,
|
||||
&[
|
||||
"store",
|
||||
"runtime-id",
|
||||
"owner",
|
||||
"reason",
|
||||
"observed-at-unix-millis",
|
||||
],
|
||||
/// Engage or clear the deployment stop. The author is the process uid's login
|
||||
/// name and the time is this verb's clock: neither is an argument.
|
||||
fn deployment_brake_write(options: &BTreeMap<String, String>, state: StopState) -> Result<()> {
|
||||
require_only(options, &["store", "runtime-id", "reason"])?;
|
||||
let now = u64::try_from(
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)?
|
||||
.as_millis(),
|
||||
)?;
|
||||
let observed = parse_u64(options, "observed-at-unix-millis")?;
|
||||
let record = IdunnDeploymentBrakeRecord {
|
||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
||||
runtime_id: required(options, "runtime-id")?.into(),
|
||||
status: "engaged".into(),
|
||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
||||
reason: required(options, "reason")?.into(),
|
||||
observed_at_unix_millis: observed,
|
||||
expires_at_unix_millis: None,
|
||||
authorization_id: None,
|
||||
authorization_purpose: None,
|
||||
authorized_release_id: None,
|
||||
authorized_deployment_id: None,
|
||||
authorized_by: None,
|
||||
authorization_issued_at_unix_millis: None,
|
||||
authorization_expires_at_unix_millis: None,
|
||||
signature_algorithm: None,
|
||||
signature: None,
|
||||
private_state_exposed: false,
|
||||
updated_by: required(options, "owner")?.into(),
|
||||
};
|
||||
record.validate()?;
|
||||
replace_brake(&path(options, "store")?, record, observed)
|
||||
}
|
||||
|
||||
fn deployment_brake_release(options: &BTreeMap<String, String>) -> Result<()> {
|
||||
require_only(
|
||||
options,
|
||||
&[
|
||||
"store",
|
||||
"private-store",
|
||||
"runtime-id",
|
||||
"owner",
|
||||
"reason",
|
||||
"authorization-id",
|
||||
"release-id",
|
||||
"deployment-id",
|
||||
"issued-at-unix-millis",
|
||||
"expires-at-unix-millis",
|
||||
],
|
||||
)?;
|
||||
let issued = parse_u64(options, "issued-at-unix-millis")?;
|
||||
let expires = parse_u64(options, "expires-at-unix-millis")?;
|
||||
let signer = open_service_identity_at::<IdunnDeploymentBrakeOperatorIdentity>(&path(
|
||||
options,
|
||||
"private-store",
|
||||
)?)?;
|
||||
let mut record = IdunnDeploymentBrakeRecord {
|
||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
||||
runtime_id: required(options, "runtime-id")?.into(),
|
||||
status: "released".into(),
|
||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
||||
reason: required(options, "reason")?.into(),
|
||||
observed_at_unix_millis: issued,
|
||||
expires_at_unix_millis: Some(expires),
|
||||
authorization_id: Some(required(options, "authorization-id")?.into()),
|
||||
authorization_purpose: Some(IDUNN_DEPLOYMENT_RELEASE_PURPOSE.into()),
|
||||
authorized_release_id: Some(required(options, "release-id")?.into()),
|
||||
authorized_deployment_id: Some(required(options, "deployment-id")?.into()),
|
||||
authorized_by: Some(signer.trust_anchor()?.identity_id),
|
||||
authorization_issued_at_unix_millis: Some(issued),
|
||||
authorization_expires_at_unix_millis: Some(expires),
|
||||
signature_algorithm: Some("ed25519".into()),
|
||||
signature: None,
|
||||
private_state_exposed: false,
|
||||
updated_by: required(options, "owner")?.into(),
|
||||
};
|
||||
record.signature = Some(
|
||||
signer
|
||||
.sign::<IdunnDeploymentBrakeReleasePurpose>(&rmp_serde::to_vec(&record)?)
|
||||
.signature,
|
||||
);
|
||||
record.validate()?;
|
||||
replace_brake(&path(options, "store")?, record, issued)
|
||||
}
|
||||
|
||||
fn deployment_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
||||
allow_only(
|
||||
options,
|
||||
&[
|
||||
"store",
|
||||
"operator-anchor",
|
||||
"runtime-id",
|
||||
"release-id",
|
||||
"deployment-id",
|
||||
"now-unix-millis",
|
||||
],
|
||||
)?;
|
||||
let exact_actuation_fields = ["release-id", "deployment-id", "now-unix-millis"];
|
||||
let exact_actuation_field_count = exact_actuation_fields
|
||||
.into_iter()
|
||||
.filter(|field| options.contains_key(*field))
|
||||
.count();
|
||||
if exact_actuation_field_count != 0
|
||||
&& exact_actuation_field_count != exact_actuation_fields.len()
|
||||
{
|
||||
bail!("exact deployment-brake actuation validation requires release, deployment, and time");
|
||||
}
|
||||
let record = read_brake(&path(options, "store")?)?;
|
||||
if record.runtime_id != required(options, "runtime-id")? {
|
||||
bail!("deployment brake belongs to another runtime")
|
||||
}
|
||||
if record.status == "engaged" {
|
||||
if exact_actuation_field_count != 0 {
|
||||
bail!("engaged deployment brake denies exact actuation validation");
|
||||
}
|
||||
println!(
|
||||
"engaged runtime={} scope={} owner={} reason={}",
|
||||
record.runtime_id, record.scope, record.updated_by, record.reason
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
let anchor = read_operator_anchor(&path(options, "operator-anchor")?)?;
|
||||
evaluate_idunn_deployment_brake(
|
||||
IdunnDeploymentBrakeObservation::Present(&record),
|
||||
&anchor,
|
||||
let record = replace_stop(
|
||||
&path(options, "store")?,
|
||||
required(options, "runtime-id")?,
|
||||
required(options, "release-id")?,
|
||||
required(options, "deployment-id")?,
|
||||
parse_u64(options, "now-unix-millis")?,
|
||||
)
|
||||
.map_err(|d| anyhow!("deployment brake denied actuation: {d:?}"))?;
|
||||
state,
|
||||
required(options, "reason")?,
|
||||
now,
|
||||
)?;
|
||||
println!(
|
||||
"released runtime={} release={} deployment={} owner={}",
|
||||
"{} runtime={} by={} sequence={}",
|
||||
record.state.name(),
|
||||
record.runtime_id,
|
||||
record.authorized_release_id.unwrap(),
|
||||
record.authorized_deployment_id.unwrap(),
|
||||
record.updated_by
|
||||
record.by,
|
||||
record.sequence
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Print the stop and exit nonzero unless deployments admit, so a script can
|
||||
/// gate on it as it could on the retired exact-actuation validation.
|
||||
fn deployment_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
||||
allow_only(options, &["store", "runtime-id", "owner-user"])?;
|
||||
// A steward's store is owned by the steward: the operator names that user
|
||||
// to read it, and the owner check is the same one Idunn makes.
|
||||
let owner = match options.get("owner-user") {
|
||||
Some(user) => crate::steward_orders::user_uid(user)?,
|
||||
None => process_uid(),
|
||||
};
|
||||
let stop = read_stop(&path(options, "store")?, required(options, "runtime-id")?, owner);
|
||||
match &stop {
|
||||
Stop::Absent => println!("absent: deployments admit"),
|
||||
Stop::Unreadable(why) => println!("unreadable: deployments wait ({why})"),
|
||||
Stop::Clear(record) | Stop::Engaged(record) => println!(
|
||||
"{} runtime={} by={} at={} sequence={} reason={}",
|
||||
record.state.name(),
|
||||
record.runtime_id,
|
||||
record.by,
|
||||
record.at_unix_millis,
|
||||
record.sequence,
|
||||
record.reason
|
||||
),
|
||||
}
|
||||
ensure!(stop.admits(), "the deployment brake does not admit deployments");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn lifecycle_brake_engage(options: &BTreeMap<String, String>) -> Result<()> {
|
||||
require_only(
|
||||
options,
|
||||
|
|
@ -407,7 +304,7 @@ fn lifecycle_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn replace_lifecycle_brake(path: &Path, record: IdunnLifecycleBrakeRecord) -> Result<()> {
|
||||
pub(crate) fn replace_lifecycle_brake(path: &Path, record: IdunnLifecycleBrakeRecord) -> Result<()> {
|
||||
record.validate()?;
|
||||
let store = SingleFileMessagePackBackingStore::new(path);
|
||||
let entries = store.pull_all_read_only_snapshot()?;
|
||||
|
|
@ -482,70 +379,6 @@ fn lifecycle_identifier<'a>(options: &'a BTreeMap<String, String>, name: &str) -
|
|||
Ok(value)
|
||||
}
|
||||
|
||||
fn replace_brake(path: &Path, record: IdunnDeploymentBrakeRecord, millis: u64) -> Result<()> {
|
||||
let store = SingleFileMessagePackBackingStore::new(path);
|
||||
let entries = store.pull_all_read_only_snapshot()?;
|
||||
let current = match entries.as_slice() {
|
||||
[] => None,
|
||||
[entry]
|
||||
if entry.r#type == IdunnDeploymentBrakeRecord::TYPE
|
||||
&& entry.key == IDUNN_DEPLOYMENT_BRAKE_ID =>
|
||||
{
|
||||
Some(entry.clone())
|
||||
}
|
||||
_ => bail!("deployment brake store is corrupt or ambiguous"),
|
||||
};
|
||||
let next = typed_envelope(
|
||||
IDUNN_DEPLOYMENT_BRAKE_ID,
|
||||
&record,
|
||||
IDUNN_DEPLOYMENT_BRAKE_SCHEMA,
|
||||
millis,
|
||||
)?;
|
||||
if !store.compare_exchange(
|
||||
&[CultCacheExpectedEnvelope {
|
||||
r#type: IdunnDeploymentBrakeRecord::TYPE.into(),
|
||||
key: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
||||
current,
|
||||
}],
|
||||
&[next],
|
||||
)? {
|
||||
bail!("deployment brake changed during transition");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn read_brake(path: &Path) -> Result<IdunnDeploymentBrakeRecord> {
|
||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||
let [entry] = entries.as_slice() else {
|
||||
bail!("deployment brake is missing or ambiguous")
|
||||
};
|
||||
if entry.r#type != IdunnDeploymentBrakeRecord::TYPE
|
||||
|| entry.key != IDUNN_DEPLOYMENT_BRAKE_ID
|
||||
|| entry.schema_id.as_deref() != Some(IDUNN_DEPLOYMENT_BRAKE_SCHEMA)
|
||||
{
|
||||
bail!("deployment brake is foreign")
|
||||
}
|
||||
let record: IdunnDeploymentBrakeRecord = rmp_serde::from_slice(&entry.payload)?;
|
||||
if rmp_serde::to_vec(&record)? != entry.payload {
|
||||
bail!("deployment brake is noncanonical")
|
||||
}
|
||||
record.validate()?;
|
||||
Ok(record)
|
||||
}
|
||||
|
||||
fn read_operator_anchor(path: &Path) -> Result<ServiceIdentityTrustAnchor> {
|
||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||
let [entry] = entries.as_slice() else {
|
||||
bail!("operator anchor is missing or ambiguous")
|
||||
};
|
||||
if entry.r#type != IdunnDeploymentBrakeOperatorIdentity::TRUST_ANCHOR_TYPE
|
||||
|| entry.key != IdunnDeploymentBrakeOperatorIdentity::TRUST_ANCHOR_KEY
|
||||
{
|
||||
bail!("operator anchor is foreign")
|
||||
}
|
||||
Ok(rmp_serde::from_slice(&entry.payload)?)
|
||||
}
|
||||
|
||||
fn validate_health_binding_store(path: &Path) -> Result<()> {
|
||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||
validated_health_bindings(&entries)?;
|
||||
|
|
@ -1127,7 +960,7 @@ fn normalized(path: &Path) -> Result<PathBuf> {
|
|||
}
|
||||
|
||||
fn usage() -> &'static str {
|
||||
"Usage: idunn-provision enroll-provider-health-identity --private-store <path>\n idunn-provision provider-health-public-key --private-store <path>\n idunn-provision export-provider-health-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-idunn-identity --private-store <path>\n idunn-provision export-idunn-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-odin-topology-identity --private-store <path>\n idunn-provision export-odin-topology-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-deployment-brake-operator --private-store <path>\n idunn-provision export-deployment-brake-operator-anchor --private-store <path> --public-anchor <path>\n idunn-provision deployment-brake-engage --store <path> --runtime-id <id> --owner <principal> --reason <text> --observed-at-unix-millis <u64>\n idunn-provision deployment-brake-release --store <path> --private-store <path> --runtime-id <id> --owner <principal> --reason <text> --authorization-id <id> --release-id <id> --deployment-id <id> --issued-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision deployment-brake-status --store <path> --operator-anchor <path> --runtime-id <id> [--release-id <id> --deployment-id <id> --now-unix-millis <u64>]\n idunn-provision lifecycle-brake-engage --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64>\n idunn-provision lifecycle-brake-release --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64> [--released-until-unix-millis <u64>]\n idunn-provision lifecycle-brake-status --store <path> --runtime-id <id> --target <id> --now-unix-millis <u64>\n idunn-provision create-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision add-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision require-daemon-health-release-binding --store <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex>\n idunn-provision rotate-daemon-health-trust-signer --output <path> --binding-id <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64>\n idunn-provision validate-daemon-health-trust-binding --input <path>\n idunn-provision create-provider-projection-trust-anchor --output <path> --trust-anchor-id <id> --runtime-id <id> --idunn-public-anchor <path> --bound-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision validate-provider-projection-trust-anchor --input <path> --idunn-public-anchor <path>"
|
||||
"Usage: idunn-provision enroll-provider-health-identity --private-store <path>\n idunn-provision provider-health-public-key --private-store <path>\n idunn-provision export-provider-health-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-idunn-identity --private-store <path>\n idunn-provision export-idunn-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-odin-topology-identity --private-store <path>\n idunn-provision export-odin-topology-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision deployment-brake-engage --store <path> --runtime-id <id> --reason <text>\n idunn-provision deployment-brake-clear --store <path> --runtime-id <id> --reason <text>\n idunn-provision deployment-brake-status --store <path> --runtime-id <id> [--owner-user <login>]\n idunn-provision lifecycle-brake-engage --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64>\n idunn-provision lifecycle-brake-release --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64> [--released-until-unix-millis <u64>]\n idunn-provision lifecycle-brake-status --store <path> --runtime-id <id> --target <id> --now-unix-millis <u64>\n idunn-provision create-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision add-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision require-daemon-health-release-binding --store <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex>\n idunn-provision rotate-daemon-health-trust-signer --output <path> --binding-id <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64>\n idunn-provision validate-daemon-health-trust-binding --input <path>\n idunn-provision create-provider-projection-trust-anchor --output <path> --trust-anchor-id <id> --runtime-id <id> --idunn-public-anchor <path> --bound-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision validate-provider-projection-trust-anchor --input <path> --idunn-public-anchor <path>"
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -2157,120 +1990,74 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn brake_operator_can_issue_one_exact_bounded_rollout_grant() -> Result<()> {
|
||||
fn provision_engage_clear_sequence() -> Result<()> {
|
||||
let temp = TempDir::new()?;
|
||||
let private = temp.path().join("operator-private.cc");
|
||||
let anchor = temp.path().join("operator-public.cc");
|
||||
let store = temp.path().join("brake.cc");
|
||||
invoke(&[
|
||||
"enroll-deployment-brake-operator",
|
||||
"--private-store",
|
||||
private.to_str().unwrap(),
|
||||
])?;
|
||||
invoke(&[
|
||||
"export-deployment-brake-operator-anchor",
|
||||
"--private-store",
|
||||
private.to_str().unwrap(),
|
||||
"--public-anchor",
|
||||
anchor.to_str().unwrap(),
|
||||
])?;
|
||||
invoke(&[
|
||||
"deployment-brake-engage",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--owner",
|
||||
"operator/metacrat",
|
||||
"--reason",
|
||||
"sleep mode",
|
||||
"--observed-at-unix-millis",
|
||||
"100",
|
||||
])?;
|
||||
assert_eq!(read_brake(&store)?.status, "engaged");
|
||||
let me = process_uid();
|
||||
let engage = |verb: &str, reason: &str| {
|
||||
invoke(&[
|
||||
verb,
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--reason",
|
||||
reason,
|
||||
])
|
||||
};
|
||||
assert_eq!(read_stop(&store, "yggdrasil", me), Stop::Absent);
|
||||
invoke(&[
|
||||
"deployment-brake-status",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--operator-anchor",
|
||||
anchor.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
])?;
|
||||
assert!(
|
||||
engage("deployment-brake-engage", "freeze")?;
|
||||
let status = || {
|
||||
invoke(&[
|
||||
"deployment-brake-status",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--operator-anchor",
|
||||
anchor.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--release-id",
|
||||
"commit-4",
|
||||
"--deployment-id",
|
||||
"request-4",
|
||||
"--now-unix-millis",
|
||||
"150",
|
||||
])
|
||||
.is_err()
|
||||
);
|
||||
invoke(&[
|
||||
"deployment-brake-release",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--private-store",
|
||||
private.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--owner",
|
||||
"operator/metacrat",
|
||||
"--reason",
|
||||
"one rollout",
|
||||
"--authorization-id",
|
||||
"auth/r4",
|
||||
"--release-id",
|
||||
"commit-4",
|
||||
"--deployment-id",
|
||||
"request-4",
|
||||
"--issued-at-unix-millis",
|
||||
"200",
|
||||
"--expires-at-unix-millis",
|
||||
"800",
|
||||
])?;
|
||||
invoke(&[
|
||||
"deployment-brake-status",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--operator-anchor",
|
||||
anchor.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--release-id",
|
||||
"commit-4",
|
||||
"--deployment-id",
|
||||
"request-4",
|
||||
"--now-unix-millis",
|
||||
"500",
|
||||
])?;
|
||||
assert!(
|
||||
invoke(&[
|
||||
"deployment-brake-status",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--operator-anchor",
|
||||
anchor.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--release-id",
|
||||
"substituted",
|
||||
"--deployment-id",
|
||||
"request-4",
|
||||
"--now-unix-millis",
|
||||
"500"
|
||||
])
|
||||
.is_err()
|
||||
);
|
||||
};
|
||||
assert!(status().is_err(), "an engaged stop must not pass status");
|
||||
let Stop::Engaged(first) = read_stop(&store, "yggdrasil", me) else {
|
||||
bail!("engage did not engage")
|
||||
};
|
||||
assert_eq!((first.sequence, first.reason.as_str()), (1, "freeze"));
|
||||
assert!(!first.by.is_empty() && first.at_unix_millis > 0);
|
||||
engage("deployment-brake-clear", "thawed")?;
|
||||
status()?;
|
||||
let Stop::Clear(second) = read_stop(&store, "yggdrasil", me) else {
|
||||
bail!("clear did not clear")
|
||||
};
|
||||
assert_eq!((second.sequence, second.by), (2, first.by));
|
||||
assert!(second.at_unix_millis >= first.at_unix_millis);
|
||||
// The author and the time are not arguments, and the retired
|
||||
// signing arguments are refused.
|
||||
for stray in ["--owner", "--observed-at-unix-millis", "--private-store"] {
|
||||
assert!(
|
||||
invoke(&[
|
||||
"deployment-brake-engage",
|
||||
"--store",
|
||||
store.to_str().unwrap(),
|
||||
"--runtime-id",
|
||||
"yggdrasil",
|
||||
"--reason",
|
||||
"x",
|
||||
stray,
|
||||
"y",
|
||||
])
|
||||
.is_err(),
|
||||
"{stray}"
|
||||
);
|
||||
}
|
||||
for gone in ["deployment-brake-release", "enroll-deployment-brake-operator"] {
|
||||
assert!(invoke(&[gone]).is_err(), "{gone}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
@ -2309,7 +2096,6 @@ mod tests {
|
|||
assert_eq!(engaged.status, "engaged");
|
||||
assert_eq!(engaged.runtime_id, "idunn-yggdrasil");
|
||||
assert_eq!(engaged.target, "ghostlight");
|
||||
assert!(read_brake(&store).is_err());
|
||||
assert!(
|
||||
invoke(&[
|
||||
"lifecycle-brake-status",
|
||||
|
|
@ -2426,57 +2212,4 @@ mod tests {
|
|||
assert_eq!(std::fs::read(&store)?, released_bytes);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn brake_compare_exchange_rejects_stale_transition_snapshot() -> Result<()> {
|
||||
let temp = TempDir::new()?;
|
||||
let store_path = temp.path().join("brake.cc");
|
||||
let base = IdunnDeploymentBrakeRecord {
|
||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
||||
runtime_id: "yggdrasil".into(),
|
||||
status: "engaged".into(),
|
||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
||||
reason: "first".into(),
|
||||
observed_at_unix_millis: 100,
|
||||
expires_at_unix_millis: None,
|
||||
authorization_id: None,
|
||||
authorization_purpose: None,
|
||||
authorized_release_id: None,
|
||||
authorized_deployment_id: None,
|
||||
authorized_by: None,
|
||||
authorization_issued_at_unix_millis: None,
|
||||
authorization_expires_at_unix_millis: None,
|
||||
signature_algorithm: None,
|
||||
signature: None,
|
||||
private_state_exposed: false,
|
||||
updated_by: "operator/a".into(),
|
||||
};
|
||||
replace_brake(&store_path, base.clone(), 100)?;
|
||||
let stale = SingleFileMessagePackBackingStore::new(&store_path)
|
||||
.pull_all_read_only_snapshot()?
|
||||
.remove(0);
|
||||
let mut changed = base.clone();
|
||||
changed.reason = "racing writer".into();
|
||||
changed.observed_at_unix_millis = 101;
|
||||
replace_brake(&store_path, changed, 101)?;
|
||||
let candidate = typed_envelope(
|
||||
IDUNN_DEPLOYMENT_BRAKE_ID,
|
||||
&base,
|
||||
IDUNN_DEPLOYMENT_BRAKE_SCHEMA,
|
||||
102,
|
||||
)?;
|
||||
assert!(
|
||||
!SingleFileMessagePackBackingStore::new(&store_path).compare_exchange(
|
||||
&[CultCacheExpectedEnvelope {
|
||||
r#type: IdunnDeploymentBrakeRecord::TYPE.into(),
|
||||
key: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
||||
current: Some(stale)
|
||||
}],
|
||||
&[candidate]
|
||||
)?
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
|
|
|||
805
src/runner_evidence.rs
Normal file
805
src/runner_evidence.rs
Normal file
|
|
@ -0,0 +1,805 @@
|
|||
//! The output tail of a failed runner step.
|
||||
//!
|
||||
//! One file, `runner-evidence.cc`, beside `control.cc`, holds one type: the
|
||||
//! last bytes of a failed runner container's stdout and stderr, one record per
|
||||
//! transaction. The engine's pre-fence failure path is its only writer, and
|
||||
//! `idunn status --command` its only reader. It is never `control.cc`, which
|
||||
//! refuses any type it does not know, and never `incidents.cc`, which is
|
||||
//! published and carries no free text.
|
||||
//!
|
||||
//! The tail is raw runner output. It is kept unredacted and readable by root
|
||||
//! alone (mode 0600): root can already read every secret the runner was given.
|
||||
//! It never enters an error, a transaction record or the journal. The status
|
||||
//! reader is a terminal, so everything it prints from this store goes through
|
||||
//! one allowlist sink (`emit`): printable ASCII passes, all else is escaped.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use anyhow::{Context, Result, bail, ensure};
|
||||
use cultcache_rs::{
|
||||
CultCacheEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore,
|
||||
TryCompareExchangeSnapshotOutcome,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::control_plane::{decode_record, require_id, typed_envelope};
|
||||
use crate::drivers::{
|
||||
LockContended, RUNNER_OUTPUT_TAIL_BYTES, RunnerStepFailure, RunnerSubject,
|
||||
exchange_behind_private_lock, runner_status,
|
||||
};
|
||||
|
||||
const RUNNER_FAILURE_SCHEMA: &str = "idunn.runner_failure.v1";
|
||||
|
||||
/// Records kept; the writer drops the oldest past this, so the store needs no
|
||||
/// other owner.
|
||||
pub(crate) const RUNNER_EVIDENCE_RECORDS: usize = 32;
|
||||
|
||||
/// `runner-evidence.cc`, beside the control store.
|
||||
pub(crate) fn runner_evidence_store_path(state_store: &Path) -> PathBuf {
|
||||
state_store.with_file_name("runner-evidence.cc")
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub(crate) enum SubjectKind {
|
||||
Step,
|
||||
ExternalInput,
|
||||
}
|
||||
|
||||
#[derive(Clone, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "idunn.runner_failure", schema = "idunn.runner_failure.v1")]
|
||||
pub(crate) struct RunnerFailureRecord {
|
||||
#[cultcache(key = 0)]
|
||||
pub(crate) schema_version: String,
|
||||
#[cultcache(key = 1)]
|
||||
pub(crate) transaction_id: String,
|
||||
#[cultcache(key = 2)]
|
||||
pub(crate) command_id: String,
|
||||
#[cultcache(key = 3)]
|
||||
pub(crate) target: String,
|
||||
#[cultcache(key = 4)]
|
||||
pub(crate) subject_kind: SubjectKind,
|
||||
#[cultcache(key = 5)]
|
||||
pub(crate) subject_id: String,
|
||||
#[cultcache(key = 6)]
|
||||
pub(crate) runner: String,
|
||||
#[cultcache(key = 7)]
|
||||
pub(crate) exit_code: Option<i32>,
|
||||
#[cultcache(key = 8)]
|
||||
pub(crate) signal: Option<i32>,
|
||||
#[cultcache(key = 9)]
|
||||
pub(crate) stdout_tail: Vec<u8>,
|
||||
#[cultcache(key = 10)]
|
||||
pub(crate) stdout_total_bytes: u64,
|
||||
#[cultcache(key = 11)]
|
||||
pub(crate) stderr_tail: Vec<u8>,
|
||||
#[cultcache(key = 12)]
|
||||
pub(crate) stderr_total_bytes: u64,
|
||||
#[cultcache(key = 13)]
|
||||
pub(crate) recorded_at_unix_millis: u64,
|
||||
}
|
||||
|
||||
/// Tails stay out of `Debug`, so a failed assertion or a logged record never
|
||||
/// prints runner output.
|
||||
impl std::fmt::Debug for RunnerFailureRecord {
|
||||
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
formatter
|
||||
.debug_struct("RunnerFailureRecord")
|
||||
.field("transaction_id", &self.transaction_id)
|
||||
.field("subject_id", &self.subject_id)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl RunnerFailureRecord {
|
||||
pub(crate) fn from_failure(
|
||||
transaction_id: &str,
|
||||
command_id: &str,
|
||||
target: &str,
|
||||
failure: &RunnerStepFailure,
|
||||
now: u64,
|
||||
) -> Self {
|
||||
let (subject_kind, subject_id) = match &failure.subject {
|
||||
RunnerSubject::Step(id) => (SubjectKind::Step, id),
|
||||
RunnerSubject::ExternalInput(id) => (SubjectKind::ExternalInput, id),
|
||||
};
|
||||
Self {
|
||||
schema_version: RUNNER_FAILURE_SCHEMA.to_owned(),
|
||||
transaction_id: transaction_id.to_owned(),
|
||||
command_id: command_id.to_owned(),
|
||||
target: target.to_owned(),
|
||||
subject_kind,
|
||||
subject_id: subject_id.clone(),
|
||||
runner: failure.runner.clone(),
|
||||
exit_code: failure.exit_code,
|
||||
signal: failure.signal,
|
||||
stdout_tail: failure.stdout_tail.clone(),
|
||||
stdout_total_bytes: failure.stdout_total_bytes,
|
||||
stderr_tail: failure.stderr_tail.clone(),
|
||||
stderr_total_bytes: failure.stderr_total_bytes,
|
||||
recorded_at_unix_millis: now,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn validate(&self) -> Result<()> {
|
||||
ensure!(
|
||||
self.schema_version == RUNNER_FAILURE_SCHEMA,
|
||||
"runner failure schema is unsupported"
|
||||
);
|
||||
require_id(&self.transaction_id, "runner failure transaction")?;
|
||||
require_id(&self.command_id, "runner failure command")?;
|
||||
require_id(&self.target, "runner failure target")?;
|
||||
require_id(&self.subject_id, "runner failure subject")?;
|
||||
require_id(&self.runner, "runner failure runner")?;
|
||||
for (tail, total, stream) in [
|
||||
(&self.stdout_tail, self.stdout_total_bytes, "stdout"),
|
||||
(&self.stderr_tail, self.stderr_total_bytes, "stderr"),
|
||||
] {
|
||||
ensure!(
|
||||
tail.len() <= RUNNER_OUTPUT_TAIL_BYTES,
|
||||
"runner failure {stream} tail exceeds its bound"
|
||||
);
|
||||
ensure!(
|
||||
tail.len() as u64 <= total,
|
||||
"runner failure {stream} tail exceeds its stream"
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn envelope(&self) -> Result<CultCacheEnvelope> {
|
||||
self.validate()?;
|
||||
typed_envelope(
|
||||
&self.transaction_id,
|
||||
Self::TYPE,
|
||||
RUNNER_FAILURE_SCHEMA,
|
||||
self,
|
||||
self.recorded_at_unix_millis,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/// One read of the store: the envelopes exactly as stored, and the records
|
||||
/// decoded from them in the same order.
|
||||
fn snapshot(path: &Path) -> Result<(Vec<CultCacheEnvelope>, Vec<RunnerFailureRecord>)> {
|
||||
let envelopes = SingleFileMessagePackBackingStore::new(path)
|
||||
.pull_all_read_only_snapshot()
|
||||
.context("reading the runner evidence store")?;
|
||||
let mut records = Vec::new();
|
||||
for envelope in &envelopes {
|
||||
ensure!(
|
||||
envelope.r#type == RunnerFailureRecord::TYPE
|
||||
&& envelope.schema_id.as_deref() == Some(RUNNER_FAILURE_SCHEMA),
|
||||
"the runner evidence store holds a document that is not a runner failure"
|
||||
);
|
||||
let record: RunnerFailureRecord = decode_record(envelope)?;
|
||||
record.validate()?;
|
||||
ensure!(
|
||||
envelope.key == record.transaction_id,
|
||||
"runner evidence store key differs from the record's transaction"
|
||||
);
|
||||
records.push(record);
|
||||
}
|
||||
Ok((envelopes, records))
|
||||
}
|
||||
|
||||
/// How a write left the store.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub(crate) enum Kept {
|
||||
Stored,
|
||||
/// The store held bytes that do not decode; it was moved to
|
||||
/// `unreadable_store_path` and the record opened a fresh one. `replaced`
|
||||
/// is the last-written time (unix milliseconds) of an earlier set-aside
|
||||
/// store this one displaced, which is gone: one slot, and it says so.
|
||||
SetAside { replaced: Option<u64> },
|
||||
}
|
||||
|
||||
/// Where an undecodable store is kept. One slot: the next one replaces it, so
|
||||
/// the damage a bad store can cost is one extra file.
|
||||
pub(crate) fn unreadable_store_path(path: &Path) -> PathBuf {
|
||||
path.with_file_name("runner-evidence.unreadable.cc")
|
||||
}
|
||||
|
||||
/// Moves the store over the previous set-aside one, root-only, and reports
|
||||
/// what it displaced. Nothing of the store's content is named: a decode error
|
||||
/// can quote stored values, and none is carried here.
|
||||
fn set_aside_unreadable(path: &Path) -> Result<Option<u64>> {
|
||||
let aside = unreadable_store_path(path);
|
||||
let replaced = std::fs::symlink_metadata(&aside).ok().map(|earlier| {
|
||||
earlier
|
||||
.modified()
|
||||
.ok()
|
||||
.and_then(|time| time.duration_since(std::time::UNIX_EPOCH).ok())
|
||||
.map_or(0, |since| since.as_millis() as u64)
|
||||
});
|
||||
std::fs::rename(path, &aside)
|
||||
.context("the runner evidence store is undecodable and could not be set aside")?;
|
||||
keep_private(&aside)?;
|
||||
Ok(replaced)
|
||||
}
|
||||
|
||||
/// How the writer reads the store's bytes; tests inject an I/O error here.
|
||||
type ReadStore<'a> = &'a dyn Fn(&Path) -> std::io::Result<Vec<u8>>;
|
||||
|
||||
fn unreadable_fault(error: &std::io::Error) -> anyhow::Error {
|
||||
anyhow::anyhow!("the runner evidence store could not be read: {}", error.kind())
|
||||
}
|
||||
|
||||
/// The one writer. Upsert by transaction id; past `RUNNER_EVIDENCE_RECORDS` the
|
||||
/// oldest records go in the same exchange. The file ends at mode 0600.
|
||||
///
|
||||
/// A store is set aside only when bytes that were read do not decode. An I/O
|
||||
/// error leaves the store in place, keeps no evidence for that failure, and
|
||||
/// the error names its kind for the journal.
|
||||
pub(crate) fn record(path: &Path, record: &RunnerFailureRecord) -> Result<Kept> {
|
||||
record_reading(path, record, &|path| std::fs::read(path))
|
||||
}
|
||||
|
||||
fn record_reading(path: &Path, record: &RunnerFailureRecord, read: ReadStore) -> Result<Kept> {
|
||||
let first = match read(path) {
|
||||
Ok(bytes) => Some(bytes),
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
|
||||
Err(error) => return Err(unreadable_fault(&error)),
|
||||
};
|
||||
let (envelopes, records, how) = match snapshot(path) {
|
||||
Ok((envelopes, records)) => (envelopes, records, Kept::Stored),
|
||||
Err(_) => {
|
||||
let Some(first) = first else {
|
||||
bail!("the runner evidence store could not be decoded");
|
||||
};
|
||||
// Undecodable only if the same bytes are still there to read: a
|
||||
// second read that fails, or finds other bytes, is not a verdict.
|
||||
match read(path) {
|
||||
Ok(second) if second == first => {}
|
||||
Ok(_) => bail!("the runner evidence store changed while it was being read"),
|
||||
Err(error) => return Err(unreadable_fault(&error)),
|
||||
}
|
||||
let replaced = set_aside_unreadable(path)?;
|
||||
(Vec::new(), Vec::new(), Kept::SetAside { replaced })
|
||||
}
|
||||
};
|
||||
let mut kept: Vec<(u64, CultCacheEnvelope)> = envelopes
|
||||
.iter()
|
||||
.zip(&records)
|
||||
.filter(|(_, stored)| stored.transaction_id != record.transaction_id)
|
||||
.map(|(envelope, stored)| (stored.recorded_at_unix_millis, envelope.clone()))
|
||||
.collect();
|
||||
kept.push((record.recorded_at_unix_millis, record.envelope()?));
|
||||
// Stable: records of one instant keep their stored order, so the record
|
||||
// being written survives a tie.
|
||||
kept.sort_by_key(|(recorded_at, _)| *recorded_at);
|
||||
let excess = kept.len().saturating_sub(RUNNER_EVIDENCE_RECORDS);
|
||||
let replacements: Vec<_> = kept
|
||||
.into_iter()
|
||||
.skip(excess)
|
||||
.map(|(_, envelope)| envelope)
|
||||
.collect();
|
||||
match exchange_behind_private_lock(path, &envelopes, &replacements)? {
|
||||
TryCompareExchangeSnapshotOutcome::Exchanged => {}
|
||||
TryCompareExchangeSnapshotOutcome::Mismatch => {
|
||||
bail!("the runner evidence store changed while it was being written")
|
||||
}
|
||||
TryCompareExchangeSnapshotOutcome::LockContended => {
|
||||
return Err(LockContended {
|
||||
store: "runner evidence store",
|
||||
}
|
||||
.into());
|
||||
}
|
||||
}
|
||||
keep_private(path)?;
|
||||
Ok(how)
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn keep_private(path: &Path) -> Result<()> {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600))
|
||||
.context("making the runner evidence store private")
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
fn keep_private(_path: &Path) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The one reader. An absent store holds nothing.
|
||||
pub(crate) fn for_transaction(
|
||||
path: &Path,
|
||||
transaction_id: &str,
|
||||
) -> Result<Option<RunnerFailureRecord>> {
|
||||
Ok(snapshot(path)?
|
||||
.1
|
||||
.into_iter()
|
||||
.find(|stored| stored.transaction_id == transaction_id))
|
||||
}
|
||||
|
||||
/// The record as `idunn status --command` shows it, as raw lines. They are
|
||||
/// not text yet: `show_status` is the only caller, and `emit` the only
|
||||
/// writer.
|
||||
fn render(record: &RunnerFailureRecord) -> Vec<Vec<u8>> {
|
||||
let noun = match record.subject_kind {
|
||||
SubjectKind::Step => "step",
|
||||
SubjectKind::ExternalInput => "external input",
|
||||
};
|
||||
let mut lines: Vec<Vec<u8>> = vec![
|
||||
format!(" failed {noun} {}", record.subject_id).into_bytes(),
|
||||
format!(" runner {}", record.runner).into_bytes(),
|
||||
format!(" status {}", runner_status(record.exit_code, record.signal)).into_bytes(),
|
||||
];
|
||||
for (stream, tail, total) in [
|
||||
("stdout", &record.stdout_tail, record.stdout_total_bytes),
|
||||
("stderr", &record.stderr_tail, record.stderr_total_bytes),
|
||||
] {
|
||||
lines.push(format!(" {stream} (last {} of {total} bytes)", tail.len()).into_bytes());
|
||||
let text = tail.strip_suffix(b"\n").unwrap_or(tail);
|
||||
if !text.is_empty() {
|
||||
lines.extend(text.split(|byte| *byte == b'\n').map(|line| {
|
||||
let mut shown = b" | ".to_vec();
|
||||
shown.extend_from_slice(line);
|
||||
shown
|
||||
}));
|
||||
}
|
||||
}
|
||||
lines
|
||||
}
|
||||
|
||||
/// What `idunn status --command` prints for a transaction's runner evidence:
|
||||
/// its record, or one line naming why the store is unreadable. Nothing is
|
||||
/// written when the transaction has no record.
|
||||
pub(crate) fn show_status(
|
||||
out: &mut impl std::io::Write,
|
||||
store: &Path,
|
||||
transaction_id: &str,
|
||||
) -> std::io::Result<()> {
|
||||
let lines = match for_transaction(store, transaction_id) {
|
||||
Ok(Some(record)) => render(&record),
|
||||
Ok(None) => return Ok(()),
|
||||
Err(error) => {
|
||||
vec![format!(" runner evidence unreadable: {}", error.root_cause()).into_bytes()]
|
||||
}
|
||||
};
|
||||
emit(out, &lines)
|
||||
}
|
||||
|
||||
/// The one sink: every byte the status read prints from the evidence store,
|
||||
/// or from an error about it, is written here and nowhere else. An allowlist:
|
||||
/// printable ASCII and tab pass; every other code point is a visible
|
||||
/// `\u{..}` escape and every invalid UTF-8 byte a `\x..` escape. A newline
|
||||
/// inside a line is escaped too: this function writes the only ones.
|
||||
pub(crate) fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
for line in lines {
|
||||
out.write_all(escape(line).as_bytes())?;
|
||||
out.write_all(b"\n")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn escape(bytes: &[u8]) -> String {
|
||||
let mut shown = String::with_capacity(bytes.len());
|
||||
for chunk in bytes.utf8_chunks() {
|
||||
for character in chunk.valid().chars() {
|
||||
match character {
|
||||
' '..='~' | '\t' => shown.push(character),
|
||||
_ => shown.push_str(&format!("\\u{{{:x}}}", u32::from(character))),
|
||||
}
|
||||
}
|
||||
for byte in chunk.invalid() {
|
||||
shown.push_str(&format!("\\x{byte:02x}"));
|
||||
}
|
||||
}
|
||||
shown
|
||||
}
|
||||
|
||||
#[cfg(all(test, unix))]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
|
||||
fn failure(subject: RunnerSubject, stdout: &[u8], stderr: &[u8]) -> RunnerStepFailure {
|
||||
RunnerStepFailure {
|
||||
subject,
|
||||
runner: "rust".into(),
|
||||
exit_code: Some(3),
|
||||
signal: None,
|
||||
stdout_tail: stdout.to_vec(),
|
||||
stdout_total_bytes: stdout.len() as u64,
|
||||
stderr_tail: stderr.to_vec(),
|
||||
stderr_total_bytes: stderr.len() as u64,
|
||||
}
|
||||
}
|
||||
|
||||
/// What status writes for a record, through the one sink.
|
||||
fn shown(record: &RunnerFailureRecord) -> String {
|
||||
let mut written = Vec::new();
|
||||
emit(&mut written, &render(record)).expect("a Vec takes any write");
|
||||
String::from_utf8(written).expect("the sink writes ASCII only")
|
||||
}
|
||||
|
||||
fn shown_lines(record: &RunnerFailureRecord) -> Vec<String> {
|
||||
shown(record).lines().map(str::to_owned).collect()
|
||||
}
|
||||
|
||||
fn record_at(transaction: &str, now: u64) -> RunnerFailureRecord {
|
||||
RunnerFailureRecord::from_failure(
|
||||
transaction,
|
||||
"up-service",
|
||||
"service",
|
||||
&failure(RunnerSubject::Step("build".into()), b"out\n", b"err\n"),
|
||||
now,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_evidence_store_keeps_the_newest_32() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
// The literal, not the constant: the spec's bound is 32.
|
||||
for index in 0..=32 {
|
||||
record(&path, &record_at(&format!("tx-{index:02}"), 1000 + index as u64))?;
|
||||
}
|
||||
let kept: Vec<_> = snapshot(&path)?.1.into_iter().map(|stored| stored.transaction_id).collect();
|
||||
assert_eq!(kept.len(), 32);
|
||||
assert!(!kept.contains(&"tx-00".to_owned()), "the oldest survived");
|
||||
assert!(kept.contains(&"tx-32".to_owned()), "the newest was evicted");
|
||||
assert!(kept.contains(&"tx-01".to_owned()), "the second oldest was evicted early");
|
||||
// Rewriting a resident transaction replaces it and drops nothing.
|
||||
let mut again = record_at("tx-05", 5000);
|
||||
again.stdout_tail = b"rewritten\n".to_vec();
|
||||
again.stdout_total_bytes = 10;
|
||||
record(&path, &again)?;
|
||||
let kept = snapshot(&path)?.1;
|
||||
assert_eq!(kept.len(), 32);
|
||||
assert!(kept.iter().any(|stored| stored.transaction_id == "tx-01"));
|
||||
let stored = for_transaction(&path, "tx-05")?.context("rewritten record is gone")?;
|
||||
assert_eq!(stored.stdout_tail, b"rewritten\n");
|
||||
assert_eq!(for_transaction(&path, "tx-00")?, None);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_record_round_trips_byte_for_byte() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
let written = RunnerFailureRecord::from_failure(
|
||||
"tx-a",
|
||||
"up-service",
|
||||
"service",
|
||||
&failure(RunnerSubject::ExternalInput("toolchain".into()), &[0, 1, 255, 10], b""),
|
||||
77,
|
||||
);
|
||||
record(&path, &written)?;
|
||||
assert_eq!(for_transaction(&path, "tx-a")?, Some(written));
|
||||
assert_eq!(for_transaction(&path, "tx-b")?, None);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_evidence_store_is_private() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
// Whatever the umask, the mode is the store's own rule, on every write.
|
||||
for index in 0..2 {
|
||||
record(&path, &record_at(&format!("tx-{index}"), 10 + index))?;
|
||||
assert_eq!(std::fs::metadata(&path)?.permissions().mode() & 0o777, 0o600);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_tail_over_its_bound_or_its_stream_is_refused() {
|
||||
let mut oversize = record_at("tx-a", 1);
|
||||
oversize.stdout_tail = vec![b'x'; RUNNER_OUTPUT_TAIL_BYTES + 1];
|
||||
oversize.stdout_total_bytes = u64::MAX;
|
||||
assert!(oversize.validate().is_err());
|
||||
let mut impossible = record_at("tx-a", 1);
|
||||
impossible.stderr_total_bytes = 1;
|
||||
assert!(impossible.validate().is_err());
|
||||
let mut exact = record_at("tx-a", 1);
|
||||
exact.stdout_tail = vec![b'x'; RUNNER_OUTPUT_TAIL_BYTES];
|
||||
exact.stdout_total_bytes = RUNNER_OUTPUT_TAIL_BYTES as u64;
|
||||
assert!(exact.validate().is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn status_renders_the_tail_with_control_bytes_escaped() {
|
||||
let mut stored = record_at("tx-a", 1);
|
||||
stored.stdout_tail = b"first\n\x1b[2Jsecond\x00end\tTab\nlast\n".to_vec();
|
||||
stored.stdout_total_bytes = 5000;
|
||||
stored.stderr_tail = b"bad\r\x07".to_vec();
|
||||
stored.stderr_total_bytes = 6;
|
||||
let lines = shown_lines(&stored);
|
||||
let text = lines.join("\n");
|
||||
assert!(!text.contains('\u{1b}') && !text.contains('\0') && !text.contains('\r'));
|
||||
assert!(!text.contains('\u{7}'));
|
||||
assert_eq!(
|
||||
lines,
|
||||
vec![
|
||||
" failed step build",
|
||||
" runner rust",
|
||||
" status exit status 3",
|
||||
" stdout (last 30 of 5000 bytes)",
|
||||
" | first",
|
||||
" | \\u{1b}[2Jsecond\\u{0}end\tTab",
|
||||
" | last",
|
||||
" stderr (last 5 of 6 bytes)",
|
||||
" | bad\\u{d}\\u{7}",
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn debug_names_the_record_and_prints_no_tail() {
|
||||
let mut stored = record_at("tx-a", 1);
|
||||
stored.stdout_tail = b"SECRET-OUTPUT".to_vec();
|
||||
stored.stdout_total_bytes = 13;
|
||||
let debug = format!("{stored:?}");
|
||||
assert!(debug.contains("tx-a") && !debug.contains("SECRET"), "{debug}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_tail_renders_only_its_header() {
|
||||
let mut stored = record_at("tx-a", 1);
|
||||
stored.stdout_tail = Vec::new();
|
||||
stored.stdout_total_bytes = 0;
|
||||
let lines = shown_lines(&stored);
|
||||
assert!(lines.contains(&" stdout (last 0 of 0 bytes)".to_owned()));
|
||||
assert!(!lines.iter().any(|line| line == " | "));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_reader_names_an_unreadable_store_as_an_error() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
std::fs::write(&path, b"\xc1 not a store")?;
|
||||
assert!(for_transaction(&path, "tx-a").is_err());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn files_beside(path: &Path) -> Result<Vec<String>> {
|
||||
let mut names: Vec<String> = std::fs::read_dir(path.parent().context("no parent")?)?
|
||||
.map(|entry| Ok(entry?.file_name().to_string_lossy().into_owned()))
|
||||
.collect::<Result<_>>()?;
|
||||
names.retain(|name| name.starts_with("runner-evidence") && !name.contains("lock"));
|
||||
names.sort();
|
||||
Ok(names)
|
||||
}
|
||||
|
||||
/// A store a newer Idunn wrote: well-formed, but a schema this one does not know.
|
||||
fn write_future_store(path: &Path) -> Result<()> {
|
||||
let mut future = record_at("tx-future", 1);
|
||||
future.schema_version = "idunn.runner_failure.v2".into();
|
||||
let envelope = typed_envelope(
|
||||
&future.transaction_id,
|
||||
RunnerFailureRecord::TYPE,
|
||||
"idunn.runner_failure.v2",
|
||||
&future,
|
||||
1,
|
||||
)?;
|
||||
let outcome = exchange_behind_private_lock(path, &[], &[envelope])?;
|
||||
ensure!(
|
||||
matches!(outcome, TryCompareExchangeSnapshotOutcome::Exchanged),
|
||||
"the future store was not written"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_corrupt_store_is_set_aside_and_the_next_failure_keeps_its_evidence() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
let damage = b"\xc1 not a store \x00 damage";
|
||||
std::fs::write(&path, damage)?;
|
||||
let kept = record(&path, &record_at("tx-a", 5))?;
|
||||
assert_eq!(kept, Kept::SetAside { replaced: None });
|
||||
// Kept, not deleted: the damage is where the journal says, root-only.
|
||||
let aside = unreadable_store_path(&path);
|
||||
assert_eq!(std::fs::read(&aside)?, damage);
|
||||
assert_eq!(std::fs::metadata(&aside)?.permissions().mode() & 0o777, 0o600);
|
||||
// The failure's evidence is in a fresh, private store.
|
||||
assert_eq!(
|
||||
for_transaction(&path, "tx-a")?.map(|stored| stored.transaction_id),
|
||||
Some("tx-a".to_owned())
|
||||
);
|
||||
assert_eq!(std::fs::metadata(&path)?.permissions().mode() & 0o777, 0o600);
|
||||
// A healthy store is not set aside again.
|
||||
assert_eq!(record(&path, &record_at("tx-b", 6))?, Kept::Stored);
|
||||
assert_eq!(std::fs::read(&aside)?, damage);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_future_version_store_is_set_aside_and_the_next_failure_keeps_its_evidence() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
write_future_store(&path)?;
|
||||
let before = std::fs::read(&path)?;
|
||||
assert!(for_transaction(&path, "tx-future").is_err());
|
||||
assert_eq!(
|
||||
record(&path, &record_at("tx-a", 5))?,
|
||||
Kept::SetAside { replaced: None }
|
||||
);
|
||||
assert_eq!(std::fs::read(unreadable_store_path(&path))?, before);
|
||||
assert!(for_transaction(&path, "tx-a")?.is_some());
|
||||
assert_eq!(for_transaction(&path, "tx-future")?, None);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn at_most_one_unreadable_store_is_kept() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
let damages: [&[u8]; 3] = [b"\xc1 first", b"\xc1 second", b"\xc1 third"];
|
||||
for (index, damage) in damages.into_iter().enumerate() {
|
||||
std::fs::write(&path, damage)?;
|
||||
record(&path, &record_at(&format!("tx-{index}"), 5 + index as u64))?;
|
||||
assert_eq!(
|
||||
std::fs::read(unreadable_store_path(&path))?,
|
||||
damage,
|
||||
"the newest damage is the one kept"
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
files_beside(&path)?,
|
||||
vec![
|
||||
"runner-evidence.cc".to_owned(),
|
||||
"runner-evidence.unreadable.cc".to_owned()
|
||||
]
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_store_that_cannot_be_read_is_left_in_place_and_the_write_fails() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
// A directory where the store should be: unreadable, not undecodable.
|
||||
std::fs::create_dir(&path)?;
|
||||
assert!(record(&path, &record_at("tx-a", 1)).is_err());
|
||||
assert!(path.is_dir());
|
||||
assert!(!unreadable_store_path(&path).exists());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_code_point_but_printable_ascii_is_a_visible_escape() {
|
||||
// The allowlist, exhaustively: no class list decides anything.
|
||||
for code in 0..=0x10ffffu32 {
|
||||
let Some(character) = char::from_u32(code) else {
|
||||
continue;
|
||||
};
|
||||
let text = character.to_string();
|
||||
let shown = escape(text.as_bytes());
|
||||
if matches!(character, ' '..='~' | '\t') {
|
||||
assert_eq!(shown, text, "U+{code:04X} should pass");
|
||||
} else {
|
||||
assert_eq!(shown, format!("\\u{{{code:x}}}"), "U+{code:04X}");
|
||||
}
|
||||
}
|
||||
// Invalid UTF-8 is escaped byte by byte; valid text round it passes.
|
||||
assert_eq!(escape(b"a\xc1\xffb\xe2\x82"), "a\\xc1\\xffb\\xe2\\x82");
|
||||
assert_eq!(escape(b"ok \xe2\x82\xac"), "ok \\u{20ac}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_newline_inside_a_line_cannot_forge_a_line() {
|
||||
let mut stored = record_at("tx-a", 1);
|
||||
stored.subject_id = "build\n status exit status 0".into();
|
||||
let lines = shown_lines(&stored);
|
||||
assert_eq!(lines[0], " failed step build\\u{a} status exit status 0");
|
||||
assert_eq!(
|
||||
lines.iter().filter(|line| line.starts_with(" status ")).count(),
|
||||
1
|
||||
);
|
||||
}
|
||||
|
||||
/// A store whose first record is well-formed except that its subject kind
|
||||
/// holds `marker`: the decode error quotes it.
|
||||
fn store_with_hostile_subject_kind(path: &Path, marker: &[u8; 4]) -> Result<()> {
|
||||
record(path, &record_at("tx-a", 1))?;
|
||||
let mut bytes = std::fs::read(path)?;
|
||||
let at = bytes
|
||||
.windows(4)
|
||||
.position(|window| window == b"step")
|
||||
.context("no step marker")?;
|
||||
bytes[at..at + 4].copy_from_slice(marker);
|
||||
std::fs::write(path, bytes)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_unreadable_store_line_is_escaped_with_the_rest() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
store_with_hostile_subject_kind(&path, b"\x1b[2J")?;
|
||||
let mut written = Vec::new();
|
||||
show_status(&mut written, &path, "tx-a")?;
|
||||
assert!(!written.contains(&0x1b), "an escape byte reached the terminal");
|
||||
assert!(written.iter().all(|byte| matches!(byte, b' '..=b'~' | b'\t' | b'\n')));
|
||||
let text = String::from_utf8(written)?;
|
||||
assert!(text.starts_with(" runner evidence unreadable: "), "{text}");
|
||||
assert!(text.contains("\\u{1b}[2J"), "the quoted value was not shown escaped: {text}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failed_set_aside_names_itself_keeps_the_store_and_echoes_nothing() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
store_with_hostile_subject_kind(&path, b"CNRY")?;
|
||||
let damage = std::fs::read(&path)?;
|
||||
// Sanity: this damage decodes to an error that quotes the canary.
|
||||
assert!(format!("{:#}", snapshot(&path).unwrap_err()).contains("CNRY"));
|
||||
// A non-empty directory where the aside goes: rename fails even as root.
|
||||
let aside = unreadable_store_path(&path);
|
||||
std::fs::create_dir(&aside)?;
|
||||
std::fs::write(aside.join("occupied"), b"x")?;
|
||||
let error = record(&path, &record_at("tx-b", 2)).unwrap_err();
|
||||
for text in [format!("{error:#}"), format!("{error:?}")] {
|
||||
assert!(text.contains("could not be set aside"), "{text}");
|
||||
assert!(!text.contains("CNRY"), "stored content reached the error: {text}");
|
||||
}
|
||||
// The failure cost nothing: the store is as it was, with no evidence
|
||||
// written over it.
|
||||
assert_eq!(std::fs::read(&path)?, damage);
|
||||
assert!(aside.join("occupied").exists());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_io_error_leaves_the_store_in_place_and_keeps_no_evidence() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
record(&path, &record_at("tx-a", 1))?;
|
||||
let before = std::fs::read(&path)?;
|
||||
let denied = || std::io::Error::from(std::io::ErrorKind::PermissionDenied);
|
||||
// The first read fails on a healthy store.
|
||||
let error = record_reading(&path, &record_at("tx-b", 2), &|_| Err(denied())).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("permission denied"), "{error:#}");
|
||||
assert_eq!(std::fs::read(&path)?, before, "the store changed");
|
||||
assert!(!unreadable_store_path(&path).exists());
|
||||
assert!(for_transaction(&path, "tx-b")?.is_none());
|
||||
// The second read, the one that confirms the bytes do not decode, fails.
|
||||
std::fs::write(&path, b"\xc1 not a store")?;
|
||||
let calls = std::cell::Cell::new(0usize);
|
||||
let read = |path: &Path| {
|
||||
calls.set(calls.get() + 1);
|
||||
if calls.get() == 2 {
|
||||
return Err(denied());
|
||||
}
|
||||
std::fs::read(path)
|
||||
};
|
||||
let error = record_reading(&path, &record_at("tx-b", 2), &read).unwrap_err();
|
||||
assert!(format!("{error:#}").contains("permission denied"), "{error:#}");
|
||||
assert_eq!(std::fs::read(&path)?, b"\xc1 not a store");
|
||||
assert!(!unreadable_store_path(&path).exists());
|
||||
// Bytes that changed between the reads are no verdict either.
|
||||
let calls = std::cell::Cell::new(0usize);
|
||||
let read = |path: &Path| -> std::io::Result<Vec<u8>> {
|
||||
calls.set(calls.get() + 1);
|
||||
let mut bytes = std::fs::read(path)?;
|
||||
bytes.push(calls.get() as u8);
|
||||
Ok(bytes)
|
||||
};
|
||||
assert!(record_reading(&path, &record_at("tx-b", 2), &read).is_err());
|
||||
assert_eq!(std::fs::read(&path)?, b"\xc1 not a store");
|
||||
assert!(!unreadable_store_path(&path).exists());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_second_set_aside_says_it_replaced_the_first() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||
std::fs::write(&path, b"\xc1 first")?;
|
||||
assert_eq!(
|
||||
record(&path, &record_at("tx-a", 1))?,
|
||||
Kept::SetAside { replaced: None }
|
||||
);
|
||||
std::fs::write(&path, b"\xc1 second")?;
|
||||
let Kept::SetAside { replaced: Some(_) } = record(&path, &record_at("tx-b", 2))? else {
|
||||
bail!("a displaced set-aside store went unreported");
|
||||
};
|
||||
assert_eq!(std::fs::read(unreadable_store_path(&path))?, b"\xc1 second");
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
4086
src/steward_orders.rs
Normal file
4086
src/steward_orders.rs
Normal file
File diff suppressed because it is too large
Load diff
16
tests/fixtures/README.md
vendored
16
tests/fixtures/README.md
vendored
|
|
@ -10,3 +10,19 @@ It is checked in because the TypeScript runtime emits a camelCase MessagePack
|
|||
map while Rust `DatabaseEntry` derives tuple payloads. Idunn must prove it can
|
||||
decode the producer's real wire shape rather than round-tripping a fixture it
|
||||
serialized itself.
|
||||
|
||||
# systemd 257 `systemctl show` captures
|
||||
|
||||
`systemd-257/*.show` are the verbatim output of systemd 257.13 for the 27
|
||||
properties `show_unit` requested at `112a7a5`, captured read-only on Yggdrasil
|
||||
on 2026-10-24: `vili-presence-identity-only.show` (a failed idunn-vili unit whose
|
||||
only credential is the presence identity) and
|
||||
`odin-presence-and-one-service-credential.show` (the live Odin unit). systemd 257
|
||||
prints `LoadCredential=[unprintable]` in both. Tests read these bytes; do not edit.
|
||||
|
||||
# Pre-cut systemd observation records
|
||||
|
||||
`pre-cut-systemd-observation.{compact,named}.hex` are the two MessagePack
|
||||
encodings of a `SystemdWorkloadObservation` with `load_credential` set to
|
||||
`[unprintable]`, produced by the code at `112a7a5` (the form live incumbents
|
||||
persisted). They pin that later Idunns still decode such records.
|
||||
|
|
|
|||
1
tests/fixtures/pre-cut-systemd-observation.compact.hex
vendored
Normal file
1
tests/fixtures/pre-cut-systemd-observation.compact.hex
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
dc0033d94e6964756e6e2d616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161612e73657276696365aa4964756e6e2074657374a3696e7601a465786563a26e6fa56d69786564c3a6753631303030a6753631303030a0a0a0c3c3a9696e76697369626c65a3616c6cc3a430303037a0ad5b756e7072696e7461626c655dcd10920194cdee48cdee48cdee48cdee4894cdee48cdee48cdee48cdee4891cdee480000000000c391010708b52f6f70742f746573742f62696e2f736572766963650101d9477368613235362d31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131d9477368613235362d61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161a92f6f70742f74657374a92f72756e2f74657374d9477368613235362d3232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323290d9477368613235362d33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333d9202f73797374656d2e736c6963652f6964756e6e2d746573742e73657276696365c090aa61637469766174696f6edc0020010101010101010101010101010101010101010101010101010101010101010190
|
||||
1
tests/fixtures/pre-cut-systemd-observation.named.hex
vendored
Normal file
1
tests/fixtures/pre-cut-systemd-observation.named.hex
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
de0033a4756e6974d94e6964756e6e2d616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161612e73657276696365b0756e69745f6465736372697074696f6eaa4964756e6e2074657374ad696e766f636174696f6e5f6964a3696e76d923657865635f6d61696e5f73746172745f74696d657374616d705f6d6f6e6f746f6e696301ac736572766963655f74797065a465786563ae726573746172745f706f6c696379a26e6fa96b696c6c5f6d6f6465a56d69786564ac64796e616d69635f75736572c3ac73797374656d645f75736572a6753631303030ad73797374656d645f67726f7570a6753631303030b4737570706c656d656e746172795f67726f757073a0b76361706162696c6974795f626f756e64696e675f736574a0b4616d6269656e745f6361706162696c6974696573a0ae707269766174655f6d6f756e7473c3ac707269766174655f70696473c3ac70726f746563745f70726f63a9696e76697369626c65ab70726f635f737562736574a3616c6cb16e6f5f6e65775f70726976696c65676573c3a5756d61736ba430303037b2696e61636365737369626c655f7061746873a0af6c6f61645f63726564656e7469616cad5b756e7072696e7461626c655da86d61696e5f706964cd1092b270726f636573735f73746172745f74696d6501ac70726f636573735f7569647394cdee48cdee48cdee48cdee48ac70726f636573735f6769647394cdee48cdee48cdee48cdee48ae70726f636573735f67726f75707391cdee48b770726f636573735f6361705f696e686572697461626c6500b570726f636573735f6361705f7065726d697474656400b570726f636573735f6361705f65666665637469766500b470726f636573735f6361705f626f756e64696e6700b370726f636573735f6361705f616d6269656e7400b970726f636573735f6e6f5f6e65775f70726976696c65676573c3b670726f636573735f6e616d6573706163655f706964739101b26d6f756e745f6e616d6573706163655f696407b07069645f6e616d6573706163655f696408aa65786563757461626c65b52f6f70742f746573742f62696e2f73657276696365b165786563757461626c655f64657669636501b065786563757461626c655f696e6f646501b165786563757461626c655f736861323536d9477368613235362d31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131b372756e74696d655f696e7374616e63655f6964d9477368613235362d61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161b1776f726b696e675f6469726563746f7279a92f6f70742f74657374ae72756e74696d655f62756e646c65a92f72756e2f74657374b3636f6d6d616e645f6c696e655f736861323536d9477368613235362d32323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232b1656e7669726f6e6d656e745f6e616d657390bb656e7669726f6e6d656e745f636f6e74726163745f736861323536d9477368613235362d33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333ad636f6e74726f6c5f67726f7570d9202f73797374656d2e736c6963652f6964756e6e2d746573742e73657276696365b563726564656e7469616c735f6469726563746f7279c0bc706172656e745f6f6e6c795f66696c655f64657363726970746f727390bd61637469766174696f6e5f7369676e65725f6964656e746974795f6964aa61637469766174696f6ebc61637469766174696f6e5f7369676e65725f7075626c69635f6b6579dc00200101010101010101010101010101010101010101010101010101010101010101b3736572766963655f63726564656e7469616c7390
|
||||
28
tests/fixtures/systemd-257/odin-presence-and-one-service-credential.show
vendored
Normal file
28
tests/fixtures/systemd-257/odin-presence-and-one-service-credential.show
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
Type=exec
|
||||
Restart=no
|
||||
MainPID=2571
|
||||
OpenFile=/run/idunn/activation-credentials/sha256-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8.credential:gamecult-idunn-runtime-activation-key:read-only
|
||||
OpenFile=/etc/gamecult/odin/runtime-presence-identity.cc:gamecult-runtime-presence-identity:read-only
|
||||
ExecMainStartTimestampMonotonic=20630000
|
||||
ControlGroup=/system.slice/idunn-odin-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8.service
|
||||
UMask=0007
|
||||
WorkingDirectory=/srv/odin/releases/sha256-e5c7d0b2b9c0a8ffba368aeb08abdbe37ae0d17484a785e60a66ec4a62b2f18c
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
User=_duc23369f1b8b167db
|
||||
Group=odin
|
||||
DynamicUser=yes
|
||||
LoadCredential=[unprintable]
|
||||
SupplementaryGroups=
|
||||
InaccessiblePaths=/run/idunn/activation-credentials
|
||||
PrivateMounts=yes
|
||||
PrivatePIDs=yes
|
||||
NoNewPrivileges=yes
|
||||
ProtectProc=invisible
|
||||
ProcSubset=all
|
||||
KillMode=mixed
|
||||
Description=Idunn odin sha256-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8
|
||||
LoadState=loaded
|
||||
ActiveState=active
|
||||
SubState=running
|
||||
InvocationID=ada48921043449f39dacc87b14a1bf70
|
||||
28
tests/fixtures/systemd-257/vili-presence-identity-only.show
vendored
Normal file
28
tests/fixtures/systemd-257/vili-presence-identity-only.show
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
Type=exec
|
||||
Restart=no
|
||||
MainPID=0
|
||||
OpenFile=/run/idunn/activation-credentials/sha256-b507e1705a70f8eee72f355e7db637acab8eb37880b5d25245fda68d80f63041.credential:gamecult-idunn-runtime-activation-key:read-only
|
||||
OpenFile=/etc/gamecult/vili/runtime-presence-identity.cc:gamecult-runtime-presence-identity:read-only
|
||||
ExecMainStartTimestampMonotonic=520083420000
|
||||
ControlGroup=
|
||||
UMask=0007
|
||||
WorkingDirectory=/srv/vili-idunn/releases/sha256-aeaee4e1669742572c45a3f26bc86893c44b2a4149520975eb7d8a28aa931906
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
User=_du8c06c563aac6d205
|
||||
Group=vili-state
|
||||
DynamicUser=yes
|
||||
LoadCredential=[unprintable]
|
||||
SupplementaryGroups=
|
||||
InaccessiblePaths=/run/idunn/activation-credentials
|
||||
PrivateMounts=yes
|
||||
PrivatePIDs=yes
|
||||
NoNewPrivileges=yes
|
||||
ProtectProc=invisible
|
||||
ProcSubset=all
|
||||
KillMode=mixed
|
||||
Description=Idunn vili sha256-b507e1705a70f8eee72f355e7db637acab8eb37880b5d25245fda68d80f63041
|
||||
LoadState=loaded
|
||||
ActiveState=failed
|
||||
SubState=failed
|
||||
InvocationID=8f7a349f2f8f49fe87ac5d89239260b4
|
||||
Loading…
Reference in a new issue