Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 140fbc3161 | |||
| d13d98a083 | |||
| 6c34758a30 | |||
| 17edea299e | |||
| 506df0a81f | |||
| eace76870a | |||
| c30f89f0df | |||
| 47a9c3d89e | |||
| cdf987730e | |||
| 8dd9f8d503 | |||
| 7eae1c4ee1 | |||
| 74c04bbcc7 | |||
| c13e175bbf | |||
| c5a0ce3d23 | |||
| 37622db430 | |||
| 58e744b745 | |||
| b230703a5c | |||
| a20568ef70 | |||
| 114803c63b | |||
| 7ba27c362e | |||
| 01b6016193 | |||
| 9f15cee11f | |||
| be98521120 | |||
| dc86c6326c | |||
| 74d6ef31e1 | |||
| d5ea779096 | |||
| 6c8fd12335 | |||
| 46ec8241a2 | |||
| 4b6a58c692 | |||
| 157610adb7 | |||
| 1c23b6e217 | |||
|
|
64d63cb417 | ||
|
|
77e2295e97 | ||
|
|
fb1e7b5dd9 | ||
|
|
8ebe0208a6 | ||
|
|
a859914b0b | ||
|
|
4894ad20df | ||
|
|
879b76a842 | ||
|
|
631fb7836b | ||
|
|
d29bf93be6 | ||
|
|
d9710c988a | ||
|
|
caacbb4bba | ||
|
|
9653d270e5 | ||
|
|
8a1beb5c87 | ||
|
|
7e15bf1a7b | ||
|
|
572f9e6aa4 | ||
|
|
56edad95c1 | ||
|
|
ca439a29ff | ||
|
|
27ca875194 | ||
|
|
be3b95bc21 | ||
|
|
cea17c06a7 | ||
|
|
ec6222e2a1 | ||
|
|
7d5cd85728 | ||
|
|
1a3634e046 | ||
|
|
e50a721dbc | ||
|
|
024f2d7a09 | ||
|
|
8400d0712f | ||
|
|
1518cfb13c | ||
|
|
549f4be093 | ||
|
|
4a68df520f | ||
|
|
134c1fb5a2 | ||
|
|
54a75ef9b0 | ||
|
|
7ef31ee699 | ||
|
|
22bd904ed9 | ||
|
|
ef82355bab | ||
|
|
78086705a0 | ||
|
|
804ed759c9 | ||
|
|
dfdc226c25 | ||
|
|
ae6a1bea55 | ||
|
|
def0b78378 | ||
|
|
69eb3e6d6f | ||
|
|
112a7a5ebb | ||
|
|
a3c6386aa4 | ||
|
|
d3222e5724 | ||
|
|
07616697af | ||
|
|
332dbd0149 |
29 changed files with 12376 additions and 1322 deletions
1
Cargo.lock
generated
1
Cargo.lock
generated
|
|
@ -482,6 +482,7 @@ dependencies = [
|
||||||
"libc",
|
"libc",
|
||||||
"rmp-serde",
|
"rmp-serde",
|
||||||
"serde",
|
"serde",
|
||||||
|
"serde_json",
|
||||||
"sha2",
|
"sha2",
|
||||||
"tempfile",
|
"tempfile",
|
||||||
"toml",
|
"toml",
|
||||||
|
|
|
||||||
|
|
@ -38,6 +38,7 @@ serde = { version = "1", features = ["derive"] }
|
||||||
ed25519-dalek = "2.2"
|
ed25519-dalek = "2.2"
|
||||||
toml = "0.8"
|
toml = "0.8"
|
||||||
uuid = { version = "1", features = ["v4"] }
|
uuid = { version = "1", features = ["v4"] }
|
||||||
|
serde_json = "1"
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
tempfile = "3"
|
tempfile = "3"
|
||||||
|
|
|
||||||
58
README.md
58
README.md
|
|
@ -58,6 +58,64 @@ idunn status --command <id>
|
||||||
the bindings, so `idunn up profile:full-gamecult` brings up everything bound
|
the bindings, so `idunn up profile:full-gamecult` brings up everything bound
|
||||||
into that profile.
|
into that profile.
|
||||||
|
|
||||||
|
### Steward orders
|
||||||
|
|
||||||
|
A second root unit from the same binary, `idunn-orders` (`idunn orders serve`),
|
||||||
|
takes typed orders from one steward Mind over `/run/idunn-orders/steward.sock`.
|
||||||
|
It has its own store, `orders.cc`, and its own binding, `steward.toml`; the
|
||||||
|
survival daemon takes no order. There is still no command string:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
idunn order package-install <pkg>[=version]... --reason TEXT # apt-get update, install named packages, check them
|
||||||
|
idunn order unit <start|stop|restart|reload> <unit> --reason TEXT
|
||||||
|
idunn order deploy <service|profile:name> --reason TEXT # the same commit as `idunn up`
|
||||||
|
idunn order show <id> # status and output tails
|
||||||
|
idunn order list
|
||||||
|
```
|
||||||
|
|
||||||
|
A unit order submits exactly one job, for the unit it names: every verb runs
|
||||||
|
`systemctl <verb> --job-mode=ignore-requirements -- <unit>`, so systemd builds a
|
||||||
|
transaction of one job. It does not start what the unit requires or restart what
|
||||||
|
is part of it: order the dependency first, and order `postgresql@16-main.service`
|
||||||
|
rather than the `postgresql.service` wrapper. Targets are not orderable. The job is one; what the unit does afterwards is
|
||||||
|
not: systemd still runs full transactions of its own for `Restart=` after a
|
||||||
|
failure, for a `StopWhenUnneeded=` dependency left without a user, and for a
|
||||||
|
`Conflicts=` partner of a unit that fails, so an ordered stop of a unit
|
||||||
|
that something restarts does not last, and a start of a `Restart=always`
|
||||||
|
unit can report failure and be active seconds later. A unit
|
||||||
|
order may act on any unit except Idunn's own (`idunn-*`) and the minds' own
|
||||||
|
(`mind-self@*`, `mind-persona@*`), and the units that end or take the host
|
||||||
|
(`systemd-suspend`, `systemd-hibernate`, `systemd-hybrid-sleep`,
|
||||||
|
`systemd-suspend-then-hibernate`, `rescue`, `emergency`). That set is compiled
|
||||||
|
into the binary and read from no configuration. An order is refused, recorded
|
||||||
|
and posted when the ordered unit, or an alias it resolves from, is one of them;
|
||||||
|
when it has an emergency action (`SuccessAction=`, `FailureAction=`,
|
||||||
|
`StartLimitAction=`, `JobTimeoutAction=`) other than `none` or an `OnFailure=`
|
||||||
|
or `OnSuccess=` unit (not for a reload); or, for a stop or restart, when another
|
||||||
|
unit is `BoundBy=` it, because systemd stops that unit with it whatever the job
|
||||||
|
mode; for a start or restart, when it has an `Upholds=` unit, which systemd
|
||||||
|
starts in a full transaction whatever the job mode; or, for a stop, restart or
|
||||||
|
reload, when it has a `ConsistsOf=` member, a wrapper such as
|
||||||
|
`postgresql.service` whose verbs would exit 0 and touch nothing. A unit
|
||||||
|
`systemctl show` cannot resolve fails the order. Slices, scopes,
|
||||||
|
targets and globs are not units an order can name.
|
||||||
|
|
||||||
|
A package order installs the packages it names and nothing else: apt runs with
|
||||||
|
`--no-remove`, each name is passed as an exact-name pattern so `.` and `+`
|
||||||
|
match themselves, and a last step fails the order unless every named package
|
||||||
|
is installed.
|
||||||
|
|
||||||
|
An order runs once, after the steward's uid is checked and the instance's
|
||||||
|
brake check (`mind-control --instance <principal> check`) exits 0. The check
|
||||||
|
that the order's principal and uid are the bound steward's runs again at every
|
||||||
|
execution, so an order accepted before `steward.toml` changed is refused and
|
||||||
|
recorded, not run. The socket belongs to the steward user alone (mode 0600);
|
||||||
|
root may `show` and `list` orders but never issue one. An order that had
|
||||||
|
started when the service died is recorded as lost and never run again. No order releases a brake. Acceptance and result are posted to
|
||||||
|
`#operator` as a retried projection of `orders.cc`: a chat outage delays the
|
||||||
|
posts and not the orders. A refusal names the field and a code, never the value
|
||||||
|
it refused.
|
||||||
|
|
||||||
## Authority
|
## Authority
|
||||||
|
|
||||||
**Owner.** Deployment admission, and the continuity of installed bodies.
|
**Owner.** Deployment admission, and the continuity of installed bodies.
|
||||||
|
|
|
||||||
47
deploy/idunn-orders.service
Normal file
47
deploy/idunn-orders.service
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
[Unit]
|
||||||
|
Description=Idunn steward orders: typed package, unit and deploy orders from the steward
|
||||||
|
# Deliberately no After= or Requires= on any managed target, Synapse or
|
||||||
|
# idunn-yggdrasil: orders are how an operator repairs those, so none of them
|
||||||
|
# may be a condition of this unit starting. Posts are a retried projection and
|
||||||
|
# wait for Synapse on their own.
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
|
||||||
|
# Root, from the same binary as idunn-yggdrasil, with its own store and socket.
|
||||||
|
# idunn-yggdrasil keeps its no-imperative-input invariant and its write set:
|
||||||
|
# no order code runs in `idunn serve`.
|
||||||
|
#
|
||||||
|
# The unit name matches idunn-*, which is in the always-refused set, so no
|
||||||
|
# order can stop or restart this service.
|
||||||
|
User=root
|
||||||
|
# No Group= and no UMask=: the service gives its socket to the steward user
|
||||||
|
# (mode 0600) itself and sets Idunn's own umask for the files it writes, so
|
||||||
|
# nothing here reaches control.cc. The steward user is the only peer the
|
||||||
|
# service answers: it checks SO_PEERCRED on every connection.
|
||||||
|
ExecStart=/usr/local/bin/idunn orders serve \
|
||||||
|
--store /var/lib/gamecult/idunn-orders \
|
||||||
|
--state-store /var/lib/gamecult/idunn/control.cc \
|
||||||
|
--steward-binding /etc/gamecult/idunn/steward.toml \
|
||||||
|
--socket /run/idunn-orders/steward.sock
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
TimeoutStopSec=20
|
||||||
|
|
||||||
|
RuntimeDirectory=idunn-orders
|
||||||
|
RuntimeDirectoryMode=0755
|
||||||
|
|
||||||
|
# The Synapse access token of @idunn, copied from a root-only file, and read
|
||||||
|
# through $CREDENTIALS_DIRECTORY; the service names it in steward.toml.
|
||||||
|
LoadCredential=idunn-matrix-token:/etc/gamecult/idunn/matrix-token
|
||||||
|
|
||||||
|
PrivateTmp=yes
|
||||||
|
ProtectHome=yes
|
||||||
|
ProtectSystem=strict
|
||||||
|
# Its own order store, and the control store a deploy order commits a command
|
||||||
|
# to. Transient units started with systemd-run run in the manager's namespace,
|
||||||
|
# so packages and units are not limited by this list.
|
||||||
|
ReadWritePaths=/var/lib/gamecult/idunn-orders /var/lib/gamecult/idunn
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
@ -45,7 +45,6 @@ ExecStart=/usr/local/bin/idunn serve \
|
||||||
--idunn-identity-store /var/lib/gamecult/idunn/idunn-service-identity.cc \
|
--idunn-identity-store /var/lib/gamecult/idunn/idunn-service-identity.cc \
|
||||||
--odin-correlation-store /var/lib/gamecult/odin/topology.cc \
|
--odin-correlation-store /var/lib/gamecult/odin/topology.cc \
|
||||||
--odin-trust-anchor /etc/gamecult/idunn/odin-topology-anchor.cc \
|
--odin-trust-anchor /etc/gamecult/idunn/odin-topology-anchor.cc \
|
||||||
--deployment-brake-operator-anchor /etc/gamecult/idunn/deployment-brake-operator-anchor.cc \
|
|
||||||
--host-actuator-bind 10.77.0.1:17890 \
|
--host-actuator-bind 10.77.0.1:17890 \
|
||||||
--source-uid 986 \
|
--source-uid 986 \
|
||||||
--source-gid 978
|
--source-gid 978
|
||||||
|
|
@ -104,6 +103,11 @@ ReadWritePaths=/etc/gamecult/ghostlight/runtime /var/lib/gamecult/ghostlight-wor
|
||||||
ReadWritePaths=/etc/gamecult/streampixels-service/runtime /var/lib/gamecult/streampixels-service /srv/streampixels-idunn/releases/service
|
ReadWritePaths=/etc/gamecult/streampixels-service/runtime /var/lib/gamecult/streampixels-service /srv/streampixels-idunn/releases/service
|
||||||
# StreamPixels web is stateless; it needs only its runtime bundle and releases.
|
# StreamPixels web is stateless; it needs only its runtime bundle and releases.
|
||||||
ReadWritePaths=/etc/gamecult/streampixels-web/runtime /srv/streampixels-idunn/releases/web
|
ReadWritePaths=/etc/gamecult/streampixels-web/runtime /srv/streampixels-idunn/releases/web
|
||||||
|
# Vili (motion worker): runtime bundle, state, and releases. Prefixed "-" so a
|
||||||
|
# target whose roots are not created yet cannot stop Idunn itself from starting
|
||||||
|
# (Idunn startup never depends on a managed target). A root created after Idunn
|
||||||
|
# starts is writable only from Idunn's next restart.
|
||||||
|
ReadWritePaths=-/etc/gamecult/vili/runtime -/var/lib/gamecult/vili -/srv/vili-idunn/releases
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,6 @@ target="${2:-}"
|
||||||
compose=/srv/compose/yggdrasil-apps.yaml
|
compose=/srv/compose/yggdrasil-apps.yaml
|
||||||
brake_store=/var/lib/gamecult/idunn-authority/deployment-brake.cc
|
brake_store=/var/lib/gamecult/idunn-authority/deployment-brake.cc
|
||||||
brake_lock=/var/lib/gamecult/idunn-authority/deployment-brake.cc.lock
|
brake_lock=/var/lib/gamecult/idunn-authority/deployment-brake.cc.lock
|
||||||
brake_anchor=/etc/gamecult/idunn/deployment-brake-operator-anchor.cc
|
|
||||||
generation_lock=/var/lib/gamecult/idunn-authority/idunn-generation.lock
|
generation_lock=/var/lib/gamecult/idunn-authority/idunn-generation.lock
|
||||||
idunn_provision=/usr/local/bin/idunn-provision
|
idunn_provision=/usr/local/bin/idunn-provision
|
||||||
|
|
||||||
|
|
@ -154,7 +153,7 @@ deployment_target_policy() {
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
require_signed_deployment_grant() {
|
require_deployment_admission() {
|
||||||
source_commit="$3"
|
source_commit="$3"
|
||||||
deployment_request_id="$4"
|
deployment_request_id="$4"
|
||||||
release_authority_id="$5"
|
release_authority_id="$5"
|
||||||
|
|
@ -180,28 +179,17 @@ require_signed_deployment_grant() {
|
||||||
[ -z "$release_authority_sha256" ] || { echo "source-observed release cannot claim Bifrost authority digest" >&2; exit 1; }
|
[ -z "$release_authority_sha256" ] || { echo "source-observed release cannot claim Bifrost authority digest" >&2; exit 1; }
|
||||||
|
|
||||||
require_root_owned_resolved_executable "$idunn_provision"
|
require_root_owned_resolved_executable "$idunn_provision"
|
||||||
require_root_owned_regular_path "$brake_store"
|
|
||||||
require_root_owned_regular_path "$brake_lock"
|
|
||||||
require_root_owned_regular_path "$brake_anchor"
|
|
||||||
|
|
||||||
# This validation is the deployment-admission point. The signed expiry is a
|
# This validation is the deployment-admission point. An absent or clear stop
|
||||||
# start-before deadline: it denies a later transaction, but does not become a
|
# admits; an engaged or unreadable one makes the status verb exit nonzero.
|
||||||
# timer-shaped veto over one already admitted. The root actuator remains the
|
# The root actuator remains the sole shared-lock holder through the
|
||||||
# sole shared-lock holder through the synchronous consequence, so an explicit
|
# synchronous consequence, so an engage or clear cannot race or rewrite it.
|
||||||
# engage/release transition cannot race or rewrite that transaction.
|
# The lock file is created here when no store has ever been written.
|
||||||
exec 9<"$brake_lock"
|
exec 9>>"$brake_lock"
|
||||||
/usr/bin/flock --shared 9
|
/usr/bin/flock --shared 9
|
||||||
now_unix_millis="$(/usr/bin/date +%s%3N)"
|
|
||||||
case "$now_unix_millis" in
|
|
||||||
''|*[!0-9]*) echo "root actuator clock did not produce Unix milliseconds" >&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
"$idunn_provision" deployment-brake-status \
|
"$idunn_provision" deployment-brake-status \
|
||||||
--store "$brake_store" \
|
--store "$brake_store" \
|
||||||
--operator-anchor "$brake_anchor" \
|
--runtime-id yggdrasil
|
||||||
--runtime-id yggdrasil \
|
|
||||||
--release-id "$source_commit" \
|
|
||||||
--deployment-id "$deployment_request_id" \
|
|
||||||
--now-unix-millis "$now_unix_millis"
|
|
||||||
|
|
||||||
if [ "$minimum_source_revision_required" = true ]; then
|
if [ "$minimum_source_revision_required" = true ]; then
|
||||||
require_root_owned_resolved_executable /usr/local/bin/idunn
|
require_root_owned_resolved_executable /usr/local/bin/idunn
|
||||||
|
|
@ -322,7 +310,7 @@ case "$action:$target" in
|
||||||
manifest="/srv/odin/deploy-manifests/$target"
|
manifest="/srv/odin/deploy-manifests/$target"
|
||||||
require_root_owned_regular_path "$manifest"
|
require_root_owned_regular_path "$manifest"
|
||||||
[ -x "$manifest" ] || { echo "root actuator manifest is not executable: $manifest" >&2; exit 1; }
|
[ -x "$manifest" ] || { echo "root actuator manifest is not executable: $manifest" >&2; exit 1; }
|
||||||
require_signed_deployment_grant "$@"
|
require_deployment_admission "$@"
|
||||||
export IDUNN_ACTUATOR=1
|
export IDUNN_ACTUATOR=1
|
||||||
export IDUNN_COMMAND_AUTHORITY=idunn-daemon
|
export IDUNN_COMMAND_AUTHORITY=idunn-daemon
|
||||||
export IDUNN_SOURCE_COMMIT="$source_commit"
|
export IDUNN_SOURCE_COMMIT="$source_commit"
|
||||||
|
|
|
||||||
|
|
@ -240,7 +240,7 @@ Fencing retries its phase forward on a capped backoff and never stops its live
|
||||||
candidate, since that is the only running copy of the admitted body; the
|
candidate, since that is the only running copy of the admitted body; the
|
||||||
deadline opens the same incident. The one exception is yielding: a continuity
|
deadline opens the same incident. The one exception is yielding: a continuity
|
||||||
whose route proof has not succeeded gives way, through its own post-fencing
|
whose route proof has not succeeded gives way, through its own post-fencing
|
||||||
abort, to a deployment the deployment brake has released and that awaits the
|
abort, to a deployment the deployment stop admits and that awaits the
|
||||||
handover; a continuity that has proven its route never yields. The
|
handover; a continuity that has proven its route never yields. The
|
||||||
`ContinuityExhausted` incident (six charged attempts in an hour,
|
`ContinuityExhausted` incident (six charged attempts in an hour,
|
||||||
`restarts_page`) pages and gates nothing. When continuity next acts is derived
|
`restarts_page`) pages and gates nothing. When continuity next acts is derived
|
||||||
|
|
@ -257,9 +257,14 @@ held until a lift succeeds. It is a migration with a deletion line (follow-up
|
||||||
still owe dependency failures not growing the streak and the dead-generation
|
still owe dependency failures not growing the streak and the dead-generation
|
||||||
route refresh.
|
route refresh.
|
||||||
|
|
||||||
The deployment brake gates changes to source, release, configuration, schema,
|
The deployment brake is a stop, not a per-deploy gate: it gates changes to
|
||||||
unit, or authority binding. The lifecycle brake separately gates restart of an
|
source, release, configuration, schema, unit, or authority binding only while
|
||||||
already-admitted incarnation. Neither target brake may gate Idunn itself or an
|
engaged. Its one reader and writer is `src/brake.rs` (`idunn.deployment_stop.v1`:
|
||||||
|
engaged or clear, who, why, when, sequence). An absent or clear store admits;
|
||||||
|
a store that is present but unreadable, foreign, for another runtime, a symlink
|
||||||
|
or owned by another uid fails closed. Nothing is signed, bound to a release or
|
||||||
|
consumed, and one clear admits every deploy until it is engaged again. The
|
||||||
|
lifecycle brake separately gates restart of an already-admitted incarnation. Neither target brake may gate Idunn itself or an
|
||||||
unrelated service.
|
unrelated service.
|
||||||
|
|
||||||
If Odin is unavailable, Idunn preserves existing routes and continuity for
|
If Odin is unavailable, Idunn preserves existing routes and continuity for
|
||||||
|
|
|
||||||
394
docs/deploy-provenance-map.md
Normal file
394
docs/deploy-provenance-map.md
Normal file
|
|
@ -0,0 +1,394 @@
|
||||||
|
# Deployment provenance: map
|
||||||
|
|
||||||
|
Status: map, Imagination, 2026-10-23 (campaign `idunn-deploy-provenance`,
|
||||||
|
session `self-2026-10-23-eureka-b`). The ends are in
|
||||||
|
[`deploy-provenance-target.md`](deploy-provenance-target.md). Questions,
|
||||||
|
rulings, cut specs and follow-ups live in Eureka's mind; this page holds what
|
||||||
|
has no kind: body facts, rationale and sequencing. Line numbers in T1-T12 are
|
||||||
|
at `d74ac8d` (origin/main on 2026-10-23); T4, T9 and T13-T16 are at `fd040f0`,
|
||||||
|
the head of Hands attempt 1, which revision 2 of the spec builds on.
|
||||||
|
|
||||||
|
## Cut `tree-executable-members`
|
||||||
|
|
||||||
|
Spec `idunn-deploy-provenance:cut_spec:cut-tree-executable-members.r2`
|
||||||
|
(revision 2, after Soul pass 1 found the real interpreter tree refused). It
|
||||||
|
serves `motion-pipeline:follow_up:idunn-tree-executable-members` (Vili's
|
||||||
|
admission waits on it) and will serve gamecult-chat C3 too: the presence
|
||||||
|
launcher runs a child such as `upstream/bin/daemon` from inside a tree artifact
|
||||||
|
(README, launcher fragment), and Synapse needs a Python interpreter there.
|
||||||
|
|
||||||
|
### Body facts
|
||||||
|
|
||||||
|
- T1. `ArtifactOutput` (`src/deployment.rs:106-117`) carries one `executable`
|
||||||
|
bool. `ArtifactReceipt` (`src/deployment_plan.rs:731-737`) copies it and
|
||||||
|
`validate` (`:740-760`) requires it to equal the declaration.
|
||||||
|
- T2. `harden_installed_release` (`src/drivers.rs:6848-6855`) builds its
|
||||||
|
executable set from `root.join(receipt.destination)` only. `harden_root_tree`
|
||||||
|
(`:6858-6892`) seals every directory 0555 and every file 0444 unless that
|
||||||
|
exact path is in the set. A directory artifact with `executable = true` is
|
||||||
|
therefore silently meaningless today: its files are all 0444. Nothing refuses
|
||||||
|
it. No live recipe declares one (CodexConnector, Ghostlight, Heimdall, Odin,
|
||||||
|
StreamPixels service and web surveyed): `executable = true` is on files only.
|
||||||
|
- T3. A tree output is copied by `copy_artifact`/`copy_tree`
|
||||||
|
(`src/drivers.rs:8053-8126`), which recreate symlinks verbatim. The digest
|
||||||
|
(`digest_tree`, `:8142-8175`) covers names, file bytes and link targets, not
|
||||||
|
mode bits. So runner or git exec bits never reach the release: harden is the
|
||||||
|
only mode authority.
|
||||||
|
- T4. `validate_release_symlink` (`src/drivers.rs:6897-6916`) keeps a symlink
|
||||||
|
only if its target is relative with no `..`. It runs only inside
|
||||||
|
`harden_root_tree`, at install, so a tree with a `..` or absolute link copies,
|
||||||
|
digests and seals, then fails at `install_release` (Soul s1: copying and
|
||||||
|
digesting the real tree twice took 160 s before harden refused it). The rule
|
||||||
|
is sound but too narrow: it refuses the real interpreter tree (T13). The
|
||||||
|
frozen-source path has a physical resolver (`resolve_frozen_source_symlink`,
|
||||||
|
`:6617`) that walks the filesystem with a 40-link budget and a memo. Its
|
||||||
|
first version was exponential (Soul S4-1, recorded in `d562a12`), and it needs
|
||||||
|
the tree on disk to answer. Revision 2 does not reuse it (T14).
|
||||||
|
- T5. The sealed release id hashes `(schema, plan_id, artifacts,
|
||||||
|
external_inputs)` (`src/deployment_plan.rs:1000-1011`), and `plan_id` hashes
|
||||||
|
the whole plan including `recipe_blob`. A changed member list is therefore
|
||||||
|
already a new plan and a new release. The receipt must still carry the list,
|
||||||
|
because `harden_installed_release` reads receipts only; its two callers
|
||||||
|
(`install_release` `:2788`, `validate_installed_release` `:2822`) re-harden
|
||||||
|
every installed release on each observation.
|
||||||
|
- T6. Encoding is `rmp_serde::to_vec`, positional. A trailing field with
|
||||||
|
`#[serde(default, skip_serializing_if = "BTreeSet::is_empty")]` keeps every
|
||||||
|
existing receipt's bytes and release id: the plan's `deadlines` field
|
||||||
|
(`:574-577`) is the precedent.
|
||||||
|
- T7. `digest_artifact` and `digest_tree` (`:8128-8175`) `fs::read` each file
|
||||||
|
whole. A torch CPU tree's largest file (`libtorch_cpu.so`, hundreds of MB)
|
||||||
|
is held in Idunn's memory three times per install (seal, install, every
|
||||||
|
validate). `hash_frozen_source_tree` (`:6788-6840`) already streams in 64 KiB
|
||||||
|
chunks (S9). The cut collapses both onto one streaming helper.
|
||||||
|
- T8. `require_leaf_path` (`src/deployment.rs:1951-1958`) only keeps the
|
||||||
|
artifact destination a single filename. Members are paths inside that
|
||||||
|
destination, so it does not change. `require_relative_path` (`:1936-1949`)
|
||||||
|
with `allow_current = false` already refuses absolute, `..`, root and prefix
|
||||||
|
components.
|
||||||
|
- T9. The host actuator re-spells the seal (`src/host.rs:1050-1086`): copy,
|
||||||
|
mode check, digest, pinned-digest check and receipt, the same steps as
|
||||||
|
`collect_artifact` (`src/drivers.rs:2305-2326`). Soul s1: deleting its mode
|
||||||
|
check survives every test. Revision 2 collapses both onto one `seal_artifact`
|
||||||
|
in `drivers.rs`. Each caller keeps only how it finds the source path.
|
||||||
|
- T10. The transient unit runs with `ProtectSystem=strict` and the release in
|
||||||
|
`ReadOnlyPaths` (`src/drivers.rs:3675-3698`), `/tmp` mounted `noexec`, and no
|
||||||
|
`MemoryDenyWriteExecute`. Executing a 0555 file from the release works.
|
||||||
|
Shared libraries need no exec bit.
|
||||||
|
- T11. Live body on Yggdrasil: `/usr/local/bin/idunn` sha256 `44cf8a14...`,
|
||||||
|
installed 2026-10-20 02:34 CEST, before `6e2a31c` (02:40 CEST). `src/` is
|
||||||
|
identical between `8760a4f` and `6e2a31c`, so the live code is the
|
||||||
|
reincarnation c2 merge. `origin/main` is 26 commits ahead (+4194/-112 in
|
||||||
|
`src/`: the presence launcher and member route). Installing this cut ships all
|
||||||
|
of them. Idunn's own body changes only through gamecult-ops
|
||||||
|
`scripts/install-idunn-yggdrasil-release.sh` (staged binaries, unit,
|
||||||
|
bindings). The stage at `/home/gamecultadmin/idunn-stage` is stale (2026-09-11).
|
||||||
|
- T13. Link census of the real interpreter tree (Yggdrasil probe of
|
||||||
|
`cpython-3.12.15+20261021-x86_64-unknown-linux-gnu-install_only.tar.gz`,
|
||||||
|
sha256 `5ff9f060...77191124`; torch adds no links, as Soul s1 counted 1,049
|
||||||
|
after pip): 1,049 links. 0 are absolute, 749 have no `..`, and 300 have all
|
||||||
|
their `..` components before the first name (every one is
|
||||||
|
`share/terminfo/<c>/<name> -> ../<c2>/<name2>`, one `..` each). 0 have a `..`
|
||||||
|
after a name, 0 climb above the tree, 0 dangle and 0 resolve outside it.
|
||||||
|
Blender 5.2.2 has 79 links, none absolute or with `..` (Soul s1).
|
||||||
|
- T14. Lexically normalising `<link's directory>/<target>` does not check
|
||||||
|
containment. The kernel resolves a target one component at a time, so a `..`
|
||||||
|
after a symlinked name climbs from where that link points, not from where its
|
||||||
|
name sits. Probe on Yggdrasil: `root/a/s -> ..` and `root/a/L -> s/../x`
|
||||||
|
normalise to `root` and `root/a/x`, both inside, yet `L` reads the file `x`
|
||||||
|
beside `root`. `root/a/d -> .` with `root/a/M -> d/../../x` escapes the same
|
||||||
|
way. Idunn has met this before: verify-transaction-cut F5 (`D -> .`,
|
||||||
|
`L -> D/D/../../../etc/passwd` passed a lexical check).
|
||||||
|
The sound lexical rule constrains the target's shape: all its `..` components
|
||||||
|
come before its first name, and they number no more than the link's depth
|
||||||
|
below the artifact's top. The climb then starts from the link's own
|
||||||
|
directory, which the walk reached through real directories only (copy and
|
||||||
|
digest never descend through a link), so it lands on a real ancestor inside
|
||||||
|
the artifact. After that the target only descends, and any link it passes
|
||||||
|
through obeys the same rule. By induction on the kernel's hops (capped at 40,
|
||||||
|
then ELOOP), every hop stays inside. The check is one pass over the target's
|
||||||
|
components with no filesystem reads, so its cost is linear in the tree and no
|
||||||
|
resolver is needed. The shapes it refuses (`s/../x`) occur in neither real
|
||||||
|
tree (T13).
|
||||||
|
Prior art, from documentation as known to Imagination (high confidence for Go
|
||||||
|
and securejoin, medium for the tar details):
|
||||||
|
- Go `path/filepath.IsLocal` (1.20) is lexical and says so: it does not account
|
||||||
|
for symlinks. It would admit `s/../x`. Used alone, it is the T14 bug.
|
||||||
|
- Go `os.Root` (1.24) and `cyphar/filepath-securejoin` (used by runc,
|
||||||
|
containerd and umoci) resolve each component inside the root when a path is
|
||||||
|
opened, refusing or clamping an escape. They guard one later access, need
|
||||||
|
the tree on disk, and check again on every open. Idunn instead certifies a
|
||||||
|
sealed tree once, so that the kernel's own resolution stays inside on every
|
||||||
|
later access.
|
||||||
|
- Zip-slip defences and archive extractors check entry names lexically and
|
||||||
|
protect against link targets by refusing to write through links
|
||||||
|
(libarchive's `ARCHIVE_EXTRACT_SECURE_SYMLINKS`, and `SECURE_NODOTDOT` for
|
||||||
|
names; GNU tar creates such links last). They protect the extraction, not
|
||||||
|
what a kept link points at afterwards.
|
||||||
|
- OCI layer extraction keeps link targets verbatim, absolute ones included,
|
||||||
|
because the runtime resolves them inside the container's root. Idunn runs a
|
||||||
|
release in the host's mount namespace, so absolute targets stay refused.
|
||||||
|
- Nix keeps link targets verbatim and tracks dependencies by scanning for
|
||||||
|
references. The whole store is its root, so it has no per-artifact
|
||||||
|
containment to compare with.
|
||||||
|
None of these is a static per-link rule for a tree that runs unconfined. The
|
||||||
|
T14 shape rule fits that case.
|
||||||
|
- T15. Host targets: `idunn-host` observes, starts and stops processes on
|
||||||
|
Windows only (the non-Windows `platform` module in `src/host.rs` bails), and
|
||||||
|
`host.rs` sets no file modes. On Windows a file runs by its type, not by a
|
||||||
|
mode bit, so `executable` and `executable_members` have nothing to set there.
|
||||||
|
The file `executable` flag has always been carried and never applied on host
|
||||||
|
targets. A recipe does not know which workload kind its binding will choose.
|
||||||
|
- T16. The release root holds only artifact destinations. Runner workspaces,
|
||||||
|
where external inputs land, are removed before `SealedRelease::new`
|
||||||
|
(`src/drivers.rs:2421-2423`). `install_release` (`:2781-2788`) and
|
||||||
|
`validate_installed_release` (`:2815-2822`) digest every receipt's
|
||||||
|
destination before calling harden. So a rule in the digest walk runs at seal,
|
||||||
|
at install and at every validate, over every link harden would see.
|
||||||
|
- T12. Disk: `/` on Yggdrasil has 1.3 TB free. A tree release costs about three
|
||||||
|
copies (runner workspace, staging, installed). Nothing collects old releases or
|
||||||
|
staging (`ghostlight-verse:follow_up:idunn-generation-retention`), so every
|
||||||
|
Vili deploy leaves about 7 GiB behind.
|
||||||
|
|
||||||
|
### Contract, corrected against the body
|
||||||
|
|
||||||
|
- Any directory artifact may declare `executable_members` (a set of relative
|
||||||
|
paths). The follow-up limited it to runner-output; that limit protects no
|
||||||
|
invariant, and the launcher's worktree or fetched trees need the same feature.
|
||||||
|
- `executable = true` names a file artifact. On a directory output the release
|
||||||
|
is refused, and `executable_members` on a file output is refused (T2).
|
||||||
|
- A member is refused at recipe parse if it is empty, absolute, or holds `.`,
|
||||||
|
`..`, a root or a prefix. At seal it is refused if it is missing, is a
|
||||||
|
symlink, passes through a symlink or non-directory, or is not a regular file.
|
||||||
|
Refusal happens before `SealedRelease::new`, so no sealed release can name a
|
||||||
|
member it cannot seal.
|
||||||
|
- Harden seals each member 0555 and fails if any declared executable path was
|
||||||
|
not visited as a regular file. "Every declared executable is sealed 0555, or
|
||||||
|
the install fails" holds structurally, not by a lookup that can miss.
|
||||||
|
- Symlinks: a link in a tree artifact is kept when its target is relative, its
|
||||||
|
`..` components all come before its first name, and they number no more than
|
||||||
|
the link's depth below the artifact's top (T14). One function,
|
||||||
|
`check_release_link`, decides this. Only the digest walk calls it, so a bad
|
||||||
|
link refuses the release at seal, before `SealedRelease::new`, and again at
|
||||||
|
install and validate (T16). Harden no longer judges links. The climb is
|
||||||
|
bounded by the artifact's top, not the release root: no live release has a
|
||||||
|
`..` link (the old rule refused them all), so nothing installed is newly
|
||||||
|
refused, and each artifact's digest describes a closed tree.
|
||||||
|
- Host targets carry `executable` and `executable_members` in the receipt,
|
||||||
|
shape-checked at seal by the shared `seal_artifact`, and do not apply them
|
||||||
|
(T15). Refusing members there would make a recipe's validity depend on its
|
||||||
|
binding.
|
||||||
|
- Vili's trees must
|
||||||
|
not come from `python -m venv`: its `bin/python` is an absolute link and its
|
||||||
|
`pyvenv.cfg` names the build workspace. Install packages into the relocatable
|
||||||
|
interpreter tree instead (`motion-pipeline:follow_up:vili-relocatable-python-tree`).
|
||||||
|
|
||||||
|
### Authority
|
||||||
|
|
||||||
|
Owner: the seal (`seal_artifact`, called by `collect_artifact` for systemd
|
||||||
|
targets and by `materialize` in `host.rs` for host targets) decides
|
||||||
|
membership from the recipe and writes it into the receipt. Harden enforces the
|
||||||
|
receipt and decides nothing. Runner exec bits and git `100755` modes are dead
|
||||||
|
inputs for release modes. Link containment belongs to `check_release_link`,
|
||||||
|
which only `digest_tree` calls. `validate_release_symlink` is deleted, and
|
||||||
|
harden's symlink arm leaves links as they are.
|
||||||
|
|
||||||
|
### Acceptance sizing
|
||||||
|
|
||||||
|
The big-tree job (spec `verification.operator`) needs about 12 GiB of scratch
|
||||||
|
on Yggdrasil: a 3.5 GiB tree, its staging copy, downloads and pip cache. It
|
||||||
|
removes it at the end. Revision 2 runs it on the trees as shipped, with the 300
|
||||||
|
terminfo links (T13). If a tree holds a link the T14 rule refuses, the job
|
||||||
|
prints it and stops, and the link is reported to Self, not deleted.
|
||||||
|
|
||||||
|
## Cut `isolation-credential-evidence`
|
||||||
|
|
||||||
|
Spec `idunn-deploy-provenance:cut_spec:cut-cut-isolation-credential-evidence.r1`,
|
||||||
|
at `112a7a5` (origin/main, live on Yggdrasil since 2026-10-24 11:29 UTC). It
|
||||||
|
serves Vili's admission (`motion-pipeline:cut_report:cut-vili-admission.h4`,
|
||||||
|
first deviation): every `idunn up vili` is refused before fencing.
|
||||||
|
|
||||||
|
### Body facts (probed read-only on Yggdrasil, 2026-10-24)
|
||||||
|
|
||||||
|
- I1. `idunn up vili` runs 6 and 7 failed with
|
||||||
|
`systemd workload isolation properties differ from the Idunn contract`, the
|
||||||
|
message of `observe_unit` (`src/drivers.rs:2975-3005`). Of its conjuncts,
|
||||||
|
only one differs on the failed Vili unit: `LoadCredential`. Idunn expects
|
||||||
|
`""` because Vili has no service credential besides the presence identity
|
||||||
|
(`service_credential_names` empty); systemd shows `[unprintable]`. Every
|
||||||
|
other conjunct matches, InaccessiblePaths included (one value across all
|
||||||
|
five idunn-vili/odin units).
|
||||||
|
- I2. systemd 257.13 `systemctl show -p LoadCredential` prints
|
||||||
|
`LoadCredential=[unprintable]` for every unit, whatever the credentials:
|
||||||
|
`ssh.service` and `systemd-journald.service` (not DynamicUser, zero
|
||||||
|
LoadCredential), the four failed `idunn-vili-*` units (zero), and the Odin
|
||||||
|
unit (one). The display form carries no information, so the conjunct was
|
||||||
|
vacuous whenever credentials existed and always false when none did. Odin
|
||||||
|
and StreamPixels passed only because each carries a regular credential.
|
||||||
|
- I3. The exact value is readable: D-Bus
|
||||||
|
`org.freedesktop.systemd1.Service.LoadCredential` (`busctl get-property`)
|
||||||
|
answers `a(ss) 0` for the Vili units and ssh, `a(ss) 1
|
||||||
|
"ODIN_TOPOLOGY_IDENTITY" "<path>"` for Odin.
|
||||||
|
- I4. What the workload can actually read is the delivered directory. On the
|
||||||
|
host, `/run/credentials/` holds a directory only for units with credentials
|
||||||
|
(getty, journald, Odin, StreamPixels service and web). Inside Odin's MainPID
|
||||||
|
root, `/run/credentials/<unit>/` holds exactly `ODIN_TOPOLOGY_IDENTITY`
|
||||||
|
(`-r--r-----+ root root`). The presence identity is a parent-only `OpenFile`
|
||||||
|
descriptor, never a credential.
|
||||||
|
- I5. The rule arrived whole in `1fc0dad` (2026-09-03, "Rebuild Idunn
|
||||||
|
deployment transaction authority"), with its twin clause in
|
||||||
|
`validate_launch_observation` (`:3468-3472`, `:3513-3516`); no later commit
|
||||||
|
touched it. No ruling or finding in any campaign concerns workload isolation
|
||||||
|
verification; gamecult-chat's foreign-workloads cut touched
|
||||||
|
`CREDENTIALS_DIRECTORY` reservation, not this check.
|
||||||
|
- I6. `SystemdWorkloadObservation` is persisted by previous Idunns in both
|
||||||
|
MessagePack encodings, compact (positional) included
|
||||||
|
(`persisted_systemd_observations_decode_as_the_systemd_variant`,
|
||||||
|
`:9915-9930`), so deleting the `load_credential` field shifts every later
|
||||||
|
slot of a live incumbent's record. The field is also in the drift list
|
||||||
|
(`:2462`) that `stop` (`:4064-4070`) uses to refuse stopping a changed
|
||||||
|
workload: changing what the field holds without leaving that list would make
|
||||||
|
the next Idunn refuse to stop the live Odin and StreamPixels incumbents.
|
||||||
|
- I7. Fixtures: the verbatim 257.13 `systemctl show` output of base's 27
|
||||||
|
properties for a Vili unit (presence identity only) and the Odin unit
|
||||||
|
(presence identity plus one service credential), captured 2026-10-24 into
|
||||||
|
this Imagination's scratchpad `fixtures/`.
|
||||||
|
- I8. The new rule holds for every live systemd incumbent: inside each
|
||||||
|
MainPID root the delivered entries equal its binding's `secret_files` keys
|
||||||
|
minus `GAMECULT_RUNTIME_PRESENCE_IDENTITY` (Odin 1, StreamPixels service 11,
|
||||||
|
StreamPixels web 1; key names compared, no values read). An Idunn restart onto
|
||||||
|
this cut re-observes them (`:3955-3970`) and passes.
|
||||||
|
|
||||||
|
### Decision
|
||||||
|
|
||||||
|
A property systemd cannot display does not belong in the comparison. The
|
||||||
|
invariant it reached for, that the workload holds exactly the credentials Idunn
|
||||||
|
delivered, is observed where it is decided: the entries of
|
||||||
|
`/run/credentials/<unit>` in the MainPID's root equal Idunn's service
|
||||||
|
credential names, absent directory equal to none. That covers
|
||||||
|
`LoadCredential`, `SetCredential`, `ImportCredential` and drop-ins alike, and
|
||||||
|
needs no new program. The D-Bus read (I3) is exact but sees only one of those
|
||||||
|
properties and adds a second systemd client and parser; rejected.
|
||||||
|
|
||||||
|
### Authority
|
||||||
|
|
||||||
|
Owner: `observe_unit` decides the credential rule, from the delivered
|
||||||
|
directory. `LoadCredential`'s display form is no longer read or compared; the
|
||||||
|
`load_credential` field is a retired wire slot, written empty, out of the drift
|
||||||
|
list, kept only so persisted records decode. Forbidden writers: the
|
||||||
|
`observe_unit` conjunct and the `validate_launch_observation` clause. Dropping
|
||||||
|
the slot waits on the next schema revision of the observation.
|
||||||
|
|
||||||
|
### Deploy
|
||||||
|
|
||||||
|
Idunn's body changes only through gamecult-ops
|
||||||
|
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||||
|
bindings unchanged). After it, Vili's Hands reruns from step 4 of
|
||||||
|
`motion-pipeline:cut_report:cut-vili-admission.h4`'s undone list.
|
||||||
|
|
||||||
|
## Cut `runner-failure-evidence`
|
||||||
|
|
||||||
|
Spec `idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1`, at
|
||||||
|
`804ed75` (origin/main, live on Yggdrasil). It serves the StreamPixels-area
|
||||||
|
Self: `idunn up vili` (command `up-4866467c-ee98-4a2b-b4a9-a008d7c44acd`,
|
||||||
|
transaction `tx-a04b68c7-c742-4d7d-8220-8aa08b9febae`, 2026-10-25 00:50-00:55
|
||||||
|
UTC) failed in Sealing with `Docker runner exited with exit status: 1: ` and
|
||||||
|
nobody could tell which recipe step failed.
|
||||||
|
|
||||||
|
### Body facts (source read at `804ed75`; host read-only, 2026-10-25)
|
||||||
|
|
||||||
|
- R1. The runner is `docker run --rm` (`src/drivers.rs:2088-2090`), launched by
|
||||||
|
`DockerRunnerDriver::docker` (`:2173-2195`) with stdin null and
|
||||||
|
`Command::output()`: stdout and stderr are each buffered whole in the
|
||||||
|
daemon's memory, unbounded. On a non-zero exit the error is
|
||||||
|
`"Docker runner exited with {status}: {stderr}"`; stdout is dropped. The
|
||||||
|
container is gone the moment it exits, so `docker ps -a` and `docker logs`
|
||||||
|
hold nothing.
|
||||||
|
- R2. The empty message: the failing process wrote nothing to stderr, and
|
||||||
|
stdout, the only stream that carried its report, is discarded at
|
||||||
|
`:2182-2193`. The same interpolation echoes whatever a step does write to
|
||||||
|
stderr into `last_error`, `pre_fencing_abort.error` and
|
||||||
|
`completion.error` (`bounded_detail`, 2048 bytes), and from there into
|
||||||
|
`idunn up`'s `deployment failed:` line and `idunn status`.
|
||||||
|
- R3. No step identity reaches the error. `materialize` (`:2368-2391`) loops
|
||||||
|
over `declaration.steps` and calls `run_in_workspace` with argv only;
|
||||||
|
`step.id` is used only in the environment precondition. External inputs
|
||||||
|
(`curl` in the runner, `:2247-2258`) fail through the same helper with the
|
||||||
|
same anonymous message.
|
||||||
|
- R4. Which step failed on 2026-10-25, inferred from the failed transaction's
|
||||||
|
staging root, which survives (926 MiB,
|
||||||
|
`/var/lib/gamecult/idunn/staging/tx-a04b68c7-.../`): in `.runner-node-build`,
|
||||||
|
`node_modules/.package-lock.json` was written at 00:51:24 (so
|
||||||
|
`install-vili-dependencies`, `npm ci`, finished), nothing outside
|
||||||
|
`node_modules` is newer, and `release/vili` (the next step's output) is
|
||||||
|
absent; the python workspace holds only fetched inputs. The failing step was
|
||||||
|
`test-daemon` (`node --test ... scripts/test-vili-daemon.mjs`), whose test
|
||||||
|
reporter writes to stdout: exactly the stream R1 discards. Inference from
|
||||||
|
residue, not from output.
|
||||||
|
- R5. The failure path: `resume_one_transaction` (`src/control_plane.rs:6291-6320`)
|
||||||
|
sends a pre-fence error to `begin_pre_fencing_abort` (`:9916-9935`), which
|
||||||
|
persists `bounded_detail(format!("{error:#}"))`. Nothing reaches the journal
|
||||||
|
on this path: the idunn-yggdrasil journal for 00:45-01:10 holds only
|
||||||
|
provider-exclusion lines.
|
||||||
|
- R6. Stores. `control.cc` (root 0640) holds commands, transactions
|
||||||
|
(`idunn.deployment_transaction.v4`, keys 0-36), generations, supervision and
|
||||||
|
expiry requests; `ControlSnapshot::read` refuses any other type
|
||||||
|
(`:3420`). Records must re-encode byte-exactly (`decode_record`,
|
||||||
|
`:3717-3727`), so a new transaction field is a schema bump and a v4 lift
|
||||||
|
(`:4100-4110`). Terminal transactions move verbatim to `history.cc`
|
||||||
|
(`:4366-4396`; 39.8 MB on the host), which status reads for `--command`.
|
||||||
|
`incidents.cc` lives in the published projection directory and by its own
|
||||||
|
doc (`src/incident.rs:1-11`) carries no free text and is never `control.cc`,
|
||||||
|
because a type added there stops continuity for every target on a rollback.
|
||||||
|
- R7. The daemon runs as root (`User=root`). Runner secrets reach the
|
||||||
|
container as read-only file mounts named by env (`:2041-2047`), so any text
|
||||||
|
a step derives from them can land in its output.
|
||||||
|
- R8. Staging residue: 475 transaction roots, 55 GiB under
|
||||||
|
`/var/lib/gamecult/idunn/staging`; a failed Sealing leaves its workspaces
|
||||||
|
because `materialize` returns before `remove_tree_inside`.
|
||||||
|
|
||||||
|
### Decision
|
||||||
|
|
||||||
|
The failure becomes a typed value at the runner, `RunnerStepFailure`: the
|
||||||
|
subject (recipe step or external input, by id), the runner id, the exit code or
|
||||||
|
signal, and the last 8 KiB of stdout and of stderr with each stream's total
|
||||||
|
byte count. Its `Display` carries the subject and status, never output.
|
||||||
|
Capture streams both pipes through bounded tails, so a verbose build no longer
|
||||||
|
buffers whole in the daemon.
|
||||||
|
|
||||||
|
The tail lives in a new private store, `runner-evidence.cc`, beside
|
||||||
|
`control.cc` (mode 0600), one record per transaction
|
||||||
|
(`idunn.runner_failure.v1`), newest 32 kept by the writer. Not in the
|
||||||
|
transaction record: that is a v5 bump whose rollback makes every older Idunn
|
||||||
|
refuse its control store (R6), the scar `incident.rs` already names. Not in
|
||||||
|
`incidents.cc`: published, and free text is barred there. Not in the journal:
|
||||||
|
journal readers are a wider set than root, so the journal line names the
|
||||||
|
transaction, the subject, the status and the store, and no output.
|
||||||
|
|
||||||
|
No redaction. Matching secret bytes would make the daemon read every secret,
|
||||||
|
and still misses any encoding, prefix or transform of one; a claimed redaction
|
||||||
|
that leaks is worse than a stated rule. The rule instead is the reader set: the
|
||||||
|
store's only readers (root) can already read every secret the runner was
|
||||||
|
given.
|
||||||
|
|
||||||
|
Keeping the failed container (dropping `--rm`) is worse: its output then lives
|
||||||
|
in Docker's json-file log, unbounded, untyped, outside Idunn's state and
|
||||||
|
readable by the docker group; containers accumulate with no owner to reap them;
|
||||||
|
`docker system prune` erases the evidence; and nothing ties a container to the
|
||||||
|
transaction in `idunn status`.
|
||||||
|
|
||||||
|
### Authority
|
||||||
|
|
||||||
|
Owner: the runner decides what failed and keeps the bounded tail; the engine's
|
||||||
|
pre-fence failure path is the one writer of `runner-evidence.cc`.
|
||||||
|
`last_error`, `pre_fencing_abort.error` and `completion.error` keep only the
|
||||||
|
failure's `Display`. Forbidden writers: the `stderr` interpolation in
|
||||||
|
`DockerRunnerDriver::docker`. Retention is by count inside the writer, so the
|
||||||
|
store needs no external owner; staging roots (R8) stay with
|
||||||
|
`ghostlight-verse:follow_up:idunn-generation-retention`.
|
||||||
|
|
||||||
|
### Deploy
|
||||||
|
|
||||||
|
Idunn's body changes only through gamecult-ops
|
||||||
|
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||||
|
bindings unchanged). The next failed Sealing then shows its step and tail on
|
||||||
|
`sudo idunn status --command <id>`.
|
||||||
|
|
@ -4,6 +4,15 @@ Status: target, Self, 2026-10-19 (campaign `idunn-deploy-provenance` in
|
||||||
Eureka's mind). It states the ends. The map will live in
|
Eureka's mind). It states the ends. The map will live in
|
||||||
`deploy-provenance-map.md`.
|
`deploy-provenance-map.md`.
|
||||||
|
|
||||||
|
Live as of cut `brake-is-a-stop` (2026-11-13): the deployment brake is the
|
||||||
|
stop of invariant 3 and the signed per-deploy release is gone. `src/brake.rs`
|
||||||
|
reads and writes it; `idunn-provision deployment-brake-engage|clear|status`
|
||||||
|
take only `--store`, `--runtime-id` and `--reason`; the writer's login name and
|
||||||
|
the verb's clock are recorded; an absent or clear store admits. The
|
||||||
|
transaction stamps `deployment_admitted_at_unix_millis` (record v6). Still
|
||||||
|
planned: the typed authority provenance on the transaction (invariant 2), and a
|
||||||
|
steward-scoped store.
|
||||||
|
|
||||||
## Why
|
## Why
|
||||||
|
|
||||||
Operator, 2026-10-19:
|
Operator, 2026-10-19:
|
||||||
|
|
@ -53,6 +62,11 @@ In:
|
||||||
(`heimdall-discord-launch.md`, `idunn-host-raven.md`, the StreamPixels
|
(`heimdall-discord-launch.md`, `idunn-host-raven.md`, the StreamPixels
|
||||||
runbook).
|
runbook).
|
||||||
|
|
||||||
|
- A static release (a sealed tree published by repointing a link, with no
|
||||||
|
workload): its transaction records `flip` (release, previous release, the
|
||||||
|
sha256 of the carried per-site file) beside the brake bytes. See
|
||||||
|
`deployment-authority.md`, "Static release".
|
||||||
|
|
||||||
Out:
|
Out:
|
||||||
|
|
||||||
- `idunn-reincarnation`. This campaign is mapped after that campaign's cut 2
|
- `idunn-reincarnation`. This campaign is mapped after that campaign's cut 2
|
||||||
|
|
|
||||||
|
|
@ -48,8 +48,8 @@ and health. CultMesh carries typed projections between them.
|
||||||
scaling, restart, host reboot recovery, and crash recovery use the same
|
scaling, restart, host reboot recovery, and crash recovery use the same
|
||||||
source-to-recipe-to-artifact provenance and the same distinct write-lease and
|
source-to-recipe-to-artifact provenance and the same distinct write-lease and
|
||||||
route-membership primitives.
|
route-membership primitives.
|
||||||
Deployment may change that chain only while the deployment brake admits the
|
Deployment may change that chain only while the deployment stop admits it
|
||||||
exact transaction. Continuity may restart only the already-admitted chain and
|
(absent or clear; an engaged stop halts). Continuity may restart only the already-admitted chain and
|
||||||
is governed by a separate lifecycle brake.
|
is governed by a separate lifecycle brake.
|
||||||
- **Deletion line:** hard-coded target copies, raw `deploy_command` and
|
- **Deletion line:** hard-coded target copies, raw `deploy_command` and
|
||||||
`restart_command` state, `sh -c` actuation, root Git inspection, staged
|
`restart_command` state, `sh -c` actuation, root Git inspection, staged
|
||||||
|
|
@ -171,6 +171,96 @@ different semantic capability.
|
||||||
`gamecult-ops` publishes these fleet bindings and provisions host substrate. It
|
`gamecult-ops` publishes these fleet bindings and provisions host substrate. It
|
||||||
does not own application build or deployment programs.
|
does not own application build or deployment programs.
|
||||||
|
|
||||||
|
## Static release
|
||||||
|
|
||||||
|
A target whose product is a sealed tree, with no process to run (a built web
|
||||||
|
front end), declares `[static_release] artifact = "<id>"` in place of
|
||||||
|
`[service]`, and its binding declares `driver = "static-release"` with a
|
||||||
|
`release_root`, a `current_link` and one `[workload.carry]`. It is the same
|
||||||
|
transaction as any deployment, in three phases: Sealing (source, build, seal,
|
||||||
|
install: unchanged), Starting (the one external change, behind the deployment
|
||||||
|
brake) and Complete. It never enters Warming through Committing, has no
|
||||||
|
Expected projection, no activation and no workload, publishes nothing to
|
||||||
|
Odin, and its Complete creates no admitted generation, so continuity never
|
||||||
|
supervises it. It declares no state, capabilities, dependencies, conflicts,
|
||||||
|
route or write lease, and its rollout is `candidate-then-promote` with at least
|
||||||
|
two retained releases.
|
||||||
|
|
||||||
|
What Starting does, in three durable steps recorded as `flip` on the
|
||||||
|
transaction: the release to name and the release the link names now are
|
||||||
|
written first; then Idunn copies the operator's carry source into
|
||||||
|
`<release_root>/<sealed release id>/<carry.destination>` (mode 0444) and
|
||||||
|
renames a fresh symlink over `current_link`; then it observes the link naming
|
||||||
|
the release and completes. A failure before the rename ends the transaction
|
||||||
|
through the ordinary pre-fencing abort with `current_link` untouched. A crash
|
||||||
|
resumes at the recorded intent and repeats the flip, which is harmless.
|
||||||
|
|
||||||
|
The carry is a copy of one per-site file (for example `config.json`) taken at
|
||||||
|
flip time. It sits beside the artifacts, never inside one, so it is outside
|
||||||
|
every sealed digest: changing the site file changes no release id, and the
|
||||||
|
sha256 of the bytes carried is provenance on the transaction
|
||||||
|
(`flip.carried_sha256`), beside the brake bytes and the requester. The
|
||||||
|
destination is a single leaf name that no artifact uses. The web server reads
|
||||||
|
`current_link/<artifact destination>` for the tree and `current_link/<carry
|
||||||
|
destination>` for the carried file (a one-time root edit of the vhost, an
|
||||||
|
`alias` for the carried file beside the root for the tree). A missing or
|
||||||
|
non-regular carry source fails the deployment before anything moves.
|
||||||
|
|
||||||
|
What `current_link` names is observed from the symlink and confirmed by the
|
||||||
|
latest Admitted static transaction's `flip.release_id`; it is stored nowhere
|
||||||
|
else. A rollback is a flip back to `flip.previous_release_id`: release
|
||||||
|
directories stay, and Idunn prunes none. The deployment stop is read exactly as for a
|
||||||
|
service, by the binding's runtime id. Release ids are per transaction, so
|
||||||
|
redeploying an unchanged tarball with a changed site file is a new release that
|
||||||
|
carries the new file, never an in-place re-carry of an existing one: the carry
|
||||||
|
is part of a release. If `current_link` is replaced by a directory or by a link
|
||||||
|
that names nothing under the release root after the flip was recorded, the
|
||||||
|
transaction keeps retrying with backoff: `idunn status --command <id>` shows the
|
||||||
|
error as "waiting on ...", operator cancel is refused (Starting, not held), and
|
||||||
|
the fix is by hand (restore the link or remove the directory); it then
|
||||||
|
completes. A bounded retry is follow-up work. A new
|
||||||
|
upstream tarball enters as a recipe `external_input` with a pinned url and
|
||||||
|
sha256, so each release is a recipe commit whose pin is its provenance.
|
||||||
|
|
||||||
|
### Rollback
|
||||||
|
|
||||||
|
`idunn rollback <target> [--state-store PATH] [--no-wait]` writes a
|
||||||
|
`rollback-` command and nothing else. The daemon freezes it into one new
|
||||||
|
transaction (`CommandKind::Rollback`) that reuses the previous release exactly:
|
||||||
|
the plan, sealed release and installed release are copied from the transaction
|
||||||
|
that flipped it, so there is no source, no build and no source cleanup, and it
|
||||||
|
creates no admitted generation. It starts at Sealing with that evidence and
|
||||||
|
reaches the same deployment stop as a deploy, which admits it unless engaged;
|
||||||
|
then `advance_static` flips, with the same three durable steps, the same
|
||||||
|
nothing-aborts-after-the-rename rule and the same stuck-flip behavior.
|
||||||
|
|
||||||
|
The previous release is derived, never stored: the newest Admitted static
|
||||||
|
transaction for the target (control.cc and history.cc) names the release
|
||||||
|
`current` must point at (`flip.release_id`) and the one it displaced
|
||||||
|
(`flip.previous_release_id`); the newest transaction that flipped that release
|
||||||
|
supplies the evidence. A rollback of a rollback therefore flips forward again.
|
||||||
|
The command is refused, recorded as a rejected transaction, when the target has
|
||||||
|
no completed static release, when the newest release is the first one, when
|
||||||
|
`current` does not name the newest flip, when the previous release directory is
|
||||||
|
gone, or when an artifact in it no longer digests to its sealed receipt. A
|
||||||
|
failed check at flip time ends the transaction before fencing with `current`
|
||||||
|
untouched.
|
||||||
|
|
||||||
|
The rollback reuses the old plan verbatim, so its binding snapshot, brake path
|
||||||
|
and carry source are the old ones: if the operator moved the site file the
|
||||||
|
rollback fails before the link moves. The carry is re-applied from the site file
|
||||||
|
as it is at rollback time, and its sha256 is recorded in the rollback's own
|
||||||
|
flip evidence (it may differ from the original release's; the copy inside the
|
||||||
|
rolled-back-to release is replaced). Nothing prunes release directories.
|
||||||
|
|
||||||
|
`CommandKind::Rollback` is stored in the existing v5 command and transaction
|
||||||
|
records with no schema change; an Idunn at 9f15cee or earlier cannot decode
|
||||||
|
them, so this is forward-only, like v5 itself. The newest flip is ordered by
|
||||||
|
`flipped_at` wall clock, and the observed-link check makes a clock step refuse
|
||||||
|
the rollback rather than mis-flip. A rollback cannot be cancelled, like a static
|
||||||
|
deploy. A rollback is also refused when the previous release was deployed under
|
||||||
|
a different release root or current link than the newest one.
|
||||||
|
|
||||||
## Typed deployment state
|
## Typed deployment state
|
||||||
|
|
||||||
Mutable control-plane state will be CultCache. Runtime integration must use
|
Mutable control-plane state will be CultCache. Runtime integration must use
|
||||||
|
|
@ -209,6 +299,82 @@ names both content addresses and the executable artifact digest. Present and
|
||||||
Ready remain Odin-owned correlations over independently verified runtime
|
Ready remain Odin-owned correlations over independently verified runtime
|
||||||
evidence; the deployment-plan module deliberately does not model them.
|
evidence; the deployment-plan module deliberately does not model them.
|
||||||
|
|
||||||
|
## The deployment stop
|
||||||
|
|
||||||
|
`src/brake.rs` is the only reader and writer of the stop. A stop halts work
|
||||||
|
that has not crossed an irreversible step, and it is read again at each such
|
||||||
|
step: Sealing (`advance_sealing`), the static flip intent (`advance_static`) and
|
||||||
|
the entry into Fencing for a deploy (`advance_warming`). An engaged stop there
|
||||||
|
makes the transaction wait with a note naming who engaged it and why; nothing
|
||||||
|
aborts and nothing moves. After the flip intent is recorded, or once a deploy
|
||||||
|
has fenced, the transaction runs to its end: a stop halts before the
|
||||||
|
irreversible step, never mid-fence or mid-flip. Continuity is never gated.
|
||||||
|
The sequence of a stop record is positive and advances by one per verb; nothing
|
||||||
|
reads it for authority, so it is not validated further.
|
||||||
|
|
||||||
|
Absent means the store file is missing in an existing directory. Anything else
|
||||||
|
Idunn cannot judge (a symlink, a zero-length or undecodable file, another
|
||||||
|
runtime's record, another owner, an unsearchable or non-directory parent) reads
|
||||||
|
as unreadable and fails closed; the wait note carries the
|
||||||
|
`idunn-provision deployment-brake-clear` command that replaces it.
|
||||||
|
|
||||||
|
A binding may name a second stop, the target's steward:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[brakes.steward]
|
||||||
|
user = "mind-streampixels" # a login, never root
|
||||||
|
store = "/var/lib/gamecult/idunn-steward/streampixels/deployment-brake.cc"
|
||||||
|
```
|
||||||
|
|
||||||
|
Deploys admit only when every stop admits. The steward's store is read as owned
|
||||||
|
by that user's uid: the owner uid proves who wrote the file, and the wait note
|
||||||
|
names the stop by the binding's steward user, never by the record's `by` (which
|
||||||
|
is the verb's own text). Idunn never writes the steward directory and no key,
|
||||||
|
group or ACL is involved. The operator's store is a separate stop the steward
|
||||||
|
cannot clear, and the wait note names each holding stop.
|
||||||
|
|
||||||
|
The steward owns its directory, so it could unlink any file in it. Binding
|
||||||
|
validation therefore refuses a steward store whose directory is shared with,
|
||||||
|
inside or around the directory of any other Idunn authority store (deployment
|
||||||
|
and lifecycle brakes, trust anchor, write-lease record, release authority).
|
||||||
|
Validation cannot see the file system: the root who provisions the directory
|
||||||
|
makes it contain the steward's store and nothing else.
|
||||||
|
|
||||||
|
A steward store owned by another uid, or a steward user that does not resolve
|
||||||
|
to a non-root uid, is unreadable and fails closed. Only the steward (the
|
||||||
|
owner uid) can repair it: a clear run as root leaves a root-owned file that is
|
||||||
|
still unreadable.
|
||||||
|
|
||||||
|
### Provisioning a steward (root, once per target)
|
||||||
|
|
||||||
|
1. Create the store directory, owned by the steward and holding nothing else:
|
||||||
|
`install -d -o mind-streampixels -g mind-streampixels -m 0755
|
||||||
|
/var/lib/gamecult/idunn-steward/streampixels`. It must not be, contain or sit
|
||||||
|
inside any other authority directory, and no workload `ReadWritePaths` may
|
||||||
|
cover it. Idunn (root) reads it without further grants; no group or ACL.
|
||||||
|
2. Add the stanza above to the target's operator binding, after `[brakes]`,
|
||||||
|
with the steward's real login. Deploys planned afterwards see it.
|
||||||
|
3. The steward engages and clears as itself, never through root:
|
||||||
|
`idunn-provision deployment-brake-engage --store <store> --runtime-id <id>
|
||||||
|
--reason <text>` and `deployment-brake-clear` with the same flags.
|
||||||
|
4. The operator reads it with `idunn-provision deployment-brake-status --store
|
||||||
|
<store> --runtime-id <id> --owner-user mind-streampixels`.
|
||||||
|
|
||||||
|
The binding is sealed into each plan. A deploy already planned keeps the
|
||||||
|
binding it was sealed with: editing the stanza does not apply to it. If the
|
||||||
|
steward's login is deleted or its directory is lost, that deploy stays
|
||||||
|
unreadable-held at Sealing (or the static flip intent). `idunn cancel` withdraws
|
||||||
|
only a deploy held for a readiness disagreement before Fencing, and `idunn
|
||||||
|
expire` only one in Warming through Committing, so neither frees a deploy held
|
||||||
|
by the stop. Recovery is to restore the steward's login or recreate the
|
||||||
|
directory owned by it (and have the steward clear its store).
|
||||||
|
|
||||||
|
Record v6 (`deployment_admitted_at_unix_millis`, key 38) is forward-only: an
|
||||||
|
Idunn that predates v6 cannot read a migrated `control.cc`, so, as for v5 (see
|
||||||
|
`route-continuity-cut.md`, "An older Idunn refuses the v5 store"), take a
|
||||||
|
byte-compared pre-install backup of `control.cc` and `history.cc` with Idunn
|
||||||
|
stopped, and roll back by restoring the backup, never by swapping the binary.
|
||||||
|
|
||||||
## Initial drivers and replaceable boundary
|
## Initial drivers and replaceable boundary
|
||||||
|
|
||||||
The first Yggdrasil implementation uses:
|
The first Yggdrasil implementation uses:
|
||||||
|
|
@ -247,7 +413,7 @@ For a singleton incarnation:
|
||||||
declaration, binding, launch contract, and input digests.
|
declaration, binding, launch contract, and input digests.
|
||||||
5. Derive the release-bound Expected incarnation from the validated sealed
|
5. Derive the release-bound Expected incarnation from the validated sealed
|
||||||
release and its exact compiled plan. Publish it only once the deployment
|
release and its exact compiled plan. Publish it only once the deployment
|
||||||
brake has admitted this exact transaction, under the candidate's own
|
stop has admitted this transaction (stamping `deployment_admitted_at_unix_millis`), under the candidate's own
|
||||||
incarnation key, beside whatever the target's admitted incarnation projects.
|
incarnation key, beside whatever the target's admitted incarnation projects.
|
||||||
6. Issue one opaque runtime-instance activation bound to the Expected digest,
|
6. Issue one opaque runtime-instance activation bound to the Expected digest,
|
||||||
pass the exact Expected and activation documents through the actuator's
|
pass the exact Expected and activation documents through the actuator's
|
||||||
|
|
@ -270,8 +436,8 @@ For a singleton incarnation:
|
||||||
and stop the incumbent.
|
and stop the incumbent.
|
||||||
|
|
||||||
Sealing resolves the source, freezes it, runs every recipe step, and installs
|
Sealing resolves the source, freezes it, runs every recipe step, and installs
|
||||||
the release before the deployment brake is consulted at all (`advance_sealing`).
|
the release before the deployment stop is consulted at all (`advance_sealing`).
|
||||||
The brake gates step 5, the first Verse-visible change: publishing the
|
The stop gates step 5, the first Verse-visible change: publishing the
|
||||||
candidate's Expected incarnation on the move into `Starting`. Build and test
|
candidate's Expected incarnation on the move into `Starting`. Build and test
|
||||||
already run unbraked, inside the sealing phase.
|
already run unbraked, inside the sealing phase.
|
||||||
|
|
||||||
|
|
@ -302,7 +468,7 @@ identity, and route membership. Every restart receives a new runtime-instance
|
||||||
activation and, for a state writer, a new process-bound lease only after the new
|
activation and, for a state writer, a new process-bound lease only after the new
|
||||||
warming presence is observed. A lease from the dead process is never reused. A
|
warming presence is observed. A lease from the dead process is never reused. A
|
||||||
continuity restart does not fetch a branch, rebuild, rerun a migration, alter
|
continuity restart does not fetch a branch, rebuild, rerun a migration, alter
|
||||||
the launch contract, or consume the deployment brake. An explicit lifecycle
|
the launch contract, or pass the deployment stop. An explicit lifecycle
|
||||||
brake may suspend this actuation without changing deployment authority.
|
brake may suspend this actuation without changing deployment authority.
|
||||||
|
|
||||||
Continuity never stops. Six charged attempts in an hour only open the
|
Continuity never stops. Six charged attempts in an hour only open the
|
||||||
|
|
@ -335,7 +501,7 @@ the admitted body. Both operator cases open an `operator-required` incident
|
||||||
on.
|
on.
|
||||||
|
|
||||||
The one yield: a continuity whose route proof has not succeeded gives way to a
|
The one yield: a continuity whose route proof has not succeeded gives way to a
|
||||||
deployment the deployment brake has released that awaits the handover of the
|
deployment the deployment stop admits that awaits the handover of the
|
||||||
target. It begins its own post-fencing abort and the deployment fences what is
|
target. It begins its own post-fencing abort and the deployment fences what is
|
||||||
admitted. A continuity that has proven its route is serving never yields.
|
admitted. A continuity that has proven its route is serving never yields.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -112,22 +112,12 @@ cargo run --release --bin idunn -- --daemon demo --rudp-health-bind <idunn-rudp-
|
||||||
`--rudp-health-bind` is explicit. Bare one-daemon Idunn does not open a
|
`--rudp-health-bind` is explicit. Bare one-daemon Idunn does not open a
|
||||||
localhost health ingress.
|
localhost health ingress.
|
||||||
|
|
||||||
Install the local Starfire boot watchdog:
|
The Rust runtime owns the target catalog for Odin, Stonks, Muninn, the enforced
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\scripts\install-idunn-startup.ps1
|
|
||||||
```
|
|
||||||
|
|
||||||
That task starts one `idunn.exe` swarm supervisor at user logon. The Rust
|
|
||||||
runtime owns the target catalog for Odin, Stonks, Muninn, the enforced
|
|
||||||
Yggdrasil source artifact lanes, and the Nightwing display services. The
|
Yggdrasil source artifact lanes, and the Nightwing display services. The
|
||||||
current Mimir dashboard is observed but not restarted until its restart
|
current Mimir dashboard is observed but not restarted until its restart
|
||||||
authority is named.
|
authority is named. A repo/service is not allowed to vanish into shrug-space:
|
||||||
|
it is either enforced by Idunn, explicitly blocked with the missing authority
|
||||||
It also starts `idunn-swarm-deployment-coverage`, which watches the deployment
|
named, external-owned, or not a runtime.
|
||||||
target catalog in `scripts/idunn-deployment-targets.ps1`. A repo/service is not
|
|
||||||
allowed to vanish into shrug-space: it is either enforced by Idunn, explicitly
|
|
||||||
blocked with the missing authority named, external-owned, or not a runtime.
|
|
||||||
|
|
||||||
To record that a daemon has not published health, run a manual target without a
|
To record that a daemon has not published health, run a manual target without a
|
||||||
matching `idunn.daemon_health` RUDP publication:
|
matching `idunn.daemon_health` RUDP publication:
|
||||||
|
|
@ -185,12 +175,7 @@ Idunn should not guess private service truth. A daemon should publish:
|
||||||
If that information is missing, Idunn should fail closed and create an operator
|
If that information is missing, Idunn should fail closed and create an operator
|
||||||
alarm instead of improvising.
|
alarm instead of improvising.
|
||||||
|
|
||||||
For the repo swarm, deployment ownership begins with catalog coverage. Use:
|
For the repo swarm, deployment ownership begins with catalog coverage.
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\scripts\show-idunn-deployment-targets.ps1
|
|
||||||
.\scripts\health-idunn-swarm-deployment-coverage.cmd
|
|
||||||
```
|
|
||||||
|
|
||||||
The enforced targets are Nightwing Gjallar plus the Yggdrasil source artifact
|
The enforced targets are Nightwing Gjallar plus the Yggdrasil source artifact
|
||||||
lanes whose ops runbooks can produce and verify deployment manifests. Each
|
lanes whose ops runbooks can produce and verify deployment manifests. Each
|
||||||
|
|
@ -258,9 +243,9 @@ surface, then let Idunn run the rollout. Direct script execution without
|
||||||
`IDUNN_ACTUATOR=1` fails on purpose; the little hand reaching for the deploy
|
`IDUNN_ACTUATOR=1` fails on purpose; the little hand reaching for the deploy
|
||||||
button has been removed from the machine.
|
button has been removed from the machine.
|
||||||
|
|
||||||
`scripts/start-idunn-local.ps1` launches the local swarm with
|
A local swarm daemon started with
|
||||||
`--rudp-health-bind 0.0.0.0:17870` so host-local and explicitly configured peers
|
`--rudp-health-bind 0.0.0.0:17870` lets host-local and explicitly configured peers
|
||||||
can publish to the configured Idunn health endpoint. It accepts raw
|
publish to the configured Idunn health endpoint. It accepts raw
|
||||||
`idunn.daemon_health` document puts on the canonical CNR0 RUDP `schema` channel
|
`idunn.daemon_health` document puts on the canonical CNR0 RUDP `schema` channel
|
||||||
and writes them into the keepalive store. That is daemon-owned health
|
and writes them into the keepalive store. That is daemon-owned health
|
||||||
publication, not restart/deploy authority. On Windows, UDP reset reports from
|
publication, not restart/deploy authority. On Windows, UDP reset reports from
|
||||||
|
|
|
||||||
|
|
@ -390,6 +390,8 @@ read-only command on Yggdrasil on 2026-09-29. Nothing was written to the host.
|
||||||
(`:3670`, `:3715`), and topology refresh (`:3780`). After three refused
|
(`:3670`, `:3715`), and topology refresh (`:3780`). After three refused
|
||||||
continuity restarts, supervision stops restarting (`:3522-3528`) but keeps
|
continuity restarts, supervision stops restarting (`:3522-3528`) but keeps
|
||||||
supervising the route of a dead incarnation. Read.
|
supervising the route of a dead incarnation. Read.
|
||||||
|
Resolved by `cut-liveness-gates-presence`: route challenge and topology
|
||||||
|
refresh now run only for an incarnation observed running.
|
||||||
- F9. The continuity restart limit counts refused restarts per generation
|
- F9. The continuity restart limit counts refused restarts per generation
|
||||||
(`is_refused_restart`, `:3469-3488`). A restart that succeeds admits a new
|
(`is_refused_restart`, `:3469-3488`). A restart that succeeds admits a new
|
||||||
generation and resets the count. A unit that starts, is admitted, and then
|
generation and resets the count. A unit that starts, is admitted, and then
|
||||||
|
|
@ -1160,7 +1162,7 @@ target's generation or to its `TargetSupervision` ends that target's iteration:
|
||||||
- **Why no existing owner can serve:**
|
- **Why no existing owner can serve:**
|
||||||
- The generation does not exist for a first deploy (the ruling).
|
- The generation does not exist for a first deploy (the ruling).
|
||||||
- The transaction dies with each attempt.
|
- The transaction dies with each attempt.
|
||||||
- Brakes are operator-signed and are not Idunn's to write.
|
- The deployment brake is a stop that a steward or the operator writes (`src/brake.rs`); Idunn writes no brake.
|
||||||
- **What it replaces:** two carry compensators, `for_new_incarnation` and the backoff copy in
|
- **What it replaces:** two carry compensators, `for_new_incarnation` and the backoff copy in
|
||||||
`from_transaction`, plus a history scan.
|
`from_transaction`, plus a history scan.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -155,7 +155,7 @@ cleaned up.
|
||||||
`idunn redeploy --daemon …`. The installed binary has no `redeploy` command
|
`idunn redeploy --daemon …`. The installed binary has no `redeploy` command
|
||||||
(probe of its usage text), so the script is dead and is **not** the current
|
(probe of its usage text), so the script is dead and is **not** the current
|
||||||
pattern. The current pattern is `idunn up` followed by
|
pattern. The current pattern is `idunn up` followed by
|
||||||
`idunn-provision deployment-brake-release`
|
`idunn-provision deployment-brake-clear` when a stop is engaged (an absent or clear store admits)
|
||||||
(`runbooks/idunn-host-raven.md`, "Deploy Muninn to Raven").
|
(`runbooks/idunn-host-raven.md`, "Deploy Muninn to Raven").
|
||||||
- F24. Build placement (`gamecult-ops/inventory.md:559-566`): build on or for the
|
- F24. Build placement (`gamecult-ops/inventory.md:559-566`): build on or for the
|
||||||
target's host. Yggdrasil has 16 vCPU and 62 GiB and carries live services, so
|
target's host. Yggdrasil has 16 vCPU and 62 GiB and carries live services, so
|
||||||
|
|
|
||||||
701
src/brake.rs
Normal file
701
src/brake.rs
Normal file
|
|
@ -0,0 +1,701 @@
|
||||||
|
//! The deployment stop: the one reader and the one writer of the record that
|
||||||
|
//! halts deployments of a runtime. Engaged halts; absent or clear admits. Who
|
||||||
|
//! engaged it and why are on the record, and nothing is signed, issued, bound
|
||||||
|
//! to a release or consumed: the brake is a stop, not a per-deploy gate (ruling
|
||||||
|
//! brake-is-a-stop). The lifecycle brake is a different record and a different
|
||||||
|
//! store; nothing here touches it.
|
||||||
|
//!
|
||||||
|
//! The reader fails closed. A store that is present but foreign, ambiguous,
|
||||||
|
//! noncanonical, for another runtime, a symlink, or owned by anyone but the
|
||||||
|
//! expected uid reads as `Unreadable`, which does not admit.
|
||||||
|
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
|
use crate::deployment::BrakeBinding;
|
||||||
|
|
||||||
|
use anyhow::{Context, Result, anyhow, bail, ensure};
|
||||||
|
use cultcache_rs::{CultCacheEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
pub(crate) const DEPLOYMENT_STOP_SCHEMA: &str = "idunn.deployment_stop.v1";
|
||||||
|
pub(crate) const DEPLOYMENT_STOP_KEY: &str = "deployment-stop";
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "kebab-case")]
|
||||||
|
pub(crate) enum StopState {
|
||||||
|
Engaged,
|
||||||
|
Clear,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl StopState {
|
||||||
|
pub(crate) fn name(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Self::Engaged => "engaged",
|
||||||
|
Self::Clear => "clear",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||||
|
#[cultcache(type = "idunn.deployment_stop", schema = "idunn.deployment_stop.v1")]
|
||||||
|
pub(crate) struct DeploymentStop {
|
||||||
|
#[cultcache(key = 0)]
|
||||||
|
pub(crate) schema_version: String,
|
||||||
|
#[cultcache(key = 1)]
|
||||||
|
pub(crate) runtime_id: String,
|
||||||
|
#[cultcache(key = 2)]
|
||||||
|
pub(crate) state: StopState,
|
||||||
|
/// The login name of the user who ran the verb, read from the process uid.
|
||||||
|
#[cultcache(key = 3)]
|
||||||
|
pub(crate) by: String,
|
||||||
|
#[cultcache(key = 4)]
|
||||||
|
pub(crate) reason: String,
|
||||||
|
/// The verb's clock.
|
||||||
|
#[cultcache(key = 5)]
|
||||||
|
pub(crate) at_unix_millis: u64,
|
||||||
|
/// The previous record's sequence plus one; 1 for the first.
|
||||||
|
#[cultcache(key = 6)]
|
||||||
|
pub(crate) sequence: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn bounded(value: &str, label: &str, max: usize) -> Result<()> {
|
||||||
|
ensure!(
|
||||||
|
!value.trim().is_empty() && value.len() <= max && !value.chars().any(char::is_control),
|
||||||
|
"deployment stop {label} is empty, oversized, or contains control characters"
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DeploymentStop {
|
||||||
|
pub(crate) fn validate(&self) -> Result<()> {
|
||||||
|
ensure!(
|
||||||
|
self.schema_version == DEPLOYMENT_STOP_SCHEMA,
|
||||||
|
"deployment stop schema is unsupported"
|
||||||
|
);
|
||||||
|
bounded(&self.runtime_id, "runtime id", 256)?;
|
||||||
|
bounded(&self.by, "author", 256)?;
|
||||||
|
bounded(&self.reason, "reason", 1024)?;
|
||||||
|
ensure!(
|
||||||
|
self.at_unix_millis > 0 && self.sequence > 0,
|
||||||
|
"deployment stop time or sequence is zero"
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn envelope(&self) -> Result<CultCacheEnvelope> {
|
||||||
|
self.validate()?;
|
||||||
|
let stored_at = chrono::DateTime::from_timestamp_millis(i64::try_from(self.at_unix_millis)?)
|
||||||
|
.ok_or_else(|| anyhow!("deployment stop time is out of range"))?
|
||||||
|
.to_rfc3339();
|
||||||
|
Ok(CultCacheEnvelope {
|
||||||
|
key: DEPLOYMENT_STOP_KEY.into(),
|
||||||
|
r#type: Self::TYPE.into(),
|
||||||
|
payload: rmp_serde::to_vec(self)?,
|
||||||
|
stored_at,
|
||||||
|
schema_id: Some(DEPLOYMENT_STOP_SCHEMA.into()),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode(envelope: &CultCacheEnvelope) -> Result<Self> {
|
||||||
|
ensure!(
|
||||||
|
envelope.r#type == Self::TYPE
|
||||||
|
&& envelope.key == DEPLOYMENT_STOP_KEY
|
||||||
|
&& envelope.schema_id.as_deref() == Some(DEPLOYMENT_STOP_SCHEMA),
|
||||||
|
"deployment stop store holds a foreign record"
|
||||||
|
);
|
||||||
|
let record: Self = rmp_serde::from_slice(&envelope.payload)?;
|
||||||
|
ensure!(
|
||||||
|
rmp_serde::to_vec(&record)? == envelope.payload,
|
||||||
|
"deployment stop is noncanonical"
|
||||||
|
);
|
||||||
|
record.validate()?;
|
||||||
|
Ok(record)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What one store says about a runtime's deployments.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub(crate) enum Stop {
|
||||||
|
/// No store, or an empty one: nothing halts.
|
||||||
|
Absent,
|
||||||
|
Clear(DeploymentStop),
|
||||||
|
Engaged(DeploymentStop),
|
||||||
|
/// Present but not a record this runtime can trust. Does not admit.
|
||||||
|
Unreadable(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Stop {
|
||||||
|
pub(crate) fn admits(&self) -> bool {
|
||||||
|
matches!(self, Self::Absent | Self::Clear(_))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One stop a binding names, with how to name it in a wait note.
|
||||||
|
pub(crate) struct NamedStop {
|
||||||
|
pub(crate) label: String,
|
||||||
|
/// The login the file system proves wrote the store: the steward user, whose
|
||||||
|
/// uid owns the file. None for the operator's store, whose record carries
|
||||||
|
/// only the verb's own claim of `by`.
|
||||||
|
pub(crate) proven_writer: Option<String>,
|
||||||
|
pub(crate) stop: Stop,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl NamedStop {
|
||||||
|
/// The wait-note line for a stop that does not admit; None when it admits.
|
||||||
|
/// A steward stop is named by the proven owner, never by the record's `by`.
|
||||||
|
pub(crate) fn note(&self) -> Option<String> {
|
||||||
|
let label = &self.label;
|
||||||
|
match &self.stop {
|
||||||
|
Stop::Absent | Stop::Clear(_) => None,
|
||||||
|
Stop::Engaged(held) => Some(match &self.proven_writer {
|
||||||
|
Some(_) => format!("{label} engaged: {}", held.reason),
|
||||||
|
None => format!("{label} engaged by {}: {}", held.by, held.reason),
|
||||||
|
}),
|
||||||
|
Stop::Unreadable(why) => Some(format!("{label} is unreadable and fails closed: {why}")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every stop a binding names, each with the words that identify its store in
|
||||||
|
/// a wait note: the operator's, then the steward's when the binding names one.
|
||||||
|
/// Deployments admit iff every one admits. The operator's store is read as
|
||||||
|
/// owned by this process's uid; the steward's as owned by the steward user's
|
||||||
|
/// uid, so the file system proves who wrote it. A steward that does not
|
||||||
|
/// resolve to a non-root uid is an unreadable stop, never an absent one.
|
||||||
|
pub(crate) fn read_stops(brakes: &BrakeBinding, runtime_id: &str) -> Vec<NamedStop> {
|
||||||
|
let mut stops = vec![NamedStop {
|
||||||
|
label: "deployment brake".to_string(),
|
||||||
|
proven_writer: None,
|
||||||
|
stop: read_stop(&brakes.deployment_store, runtime_id, process_uid()),
|
||||||
|
}];
|
||||||
|
if let Some(steward) = &brakes.steward {
|
||||||
|
let label = format!("steward {} deployment brake", steward.user);
|
||||||
|
let stop = match crate::steward_orders::user_uid(&steward.user) {
|
||||||
|
Ok(uid) => read_stop_with_remedy(&steward.store, runtime_id, uid, |path| {
|
||||||
|
// Only the owner uid can mend it: a clear run as root leaves a
|
||||||
|
// root-owned file, which this reader refuses again.
|
||||||
|
format!(
|
||||||
|
"only {} (the store's owner uid) can repair it, by running as that user: \
|
||||||
|
idunn-provision deployment-brake-clear --store {} --runtime-id {runtime_id} \
|
||||||
|
--reason <text>; a clear run as root stays unreadable",
|
||||||
|
steward.user,
|
||||||
|
path.display()
|
||||||
|
)
|
||||||
|
}),
|
||||||
|
Err(why) => Stop::Unreadable(format!(
|
||||||
|
"{why:#}; the binding's steward user {} must be a non-root login",
|
||||||
|
steward.user
|
||||||
|
)),
|
||||||
|
};
|
||||||
|
stops.push(NamedStop {
|
||||||
|
label,
|
||||||
|
proven_writer: Some(steward.user.clone()),
|
||||||
|
stop,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
stops
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
pub(crate) fn process_uid() -> u32 {
|
||||||
|
// SAFETY: geteuid has no failure mode and no arguments.
|
||||||
|
unsafe { libc::geteuid() }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
pub(crate) fn process_uid() -> u32 {
|
||||||
|
0
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
pub(crate) fn login_name(uid: u32) -> Result<String> {
|
||||||
|
// SAFETY: getpwuid returns null or a pointer into libc's static buffer,
|
||||||
|
// copied before any other call; the verbs are single threaded.
|
||||||
|
unsafe {
|
||||||
|
let entry = libc::getpwuid(uid);
|
||||||
|
ensure!(!entry.is_null(), "uid {uid} has no login name");
|
||||||
|
Ok(std::ffi::CStr::from_ptr((*entry).pw_name)
|
||||||
|
.to_string_lossy()
|
||||||
|
.into_owned())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
pub(crate) fn login_name(_: u32) -> Result<String> {
|
||||||
|
std::env::var("USERNAME").context("USERNAME is unset")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Open without following a final symlink and judge the file by fstat before
|
||||||
|
/// anything is read from it.
|
||||||
|
#[cfg(unix)]
|
||||||
|
fn check_file(path: &Path, expected_owner_uid: u32) -> Result<()> {
|
||||||
|
use std::os::unix::fs::{MetadataExt, OpenOptionsExt};
|
||||||
|
let file = std::fs::OpenOptions::new()
|
||||||
|
.read(true)
|
||||||
|
.custom_flags(libc::O_NOFOLLOW)
|
||||||
|
.open(path)
|
||||||
|
.with_context(|| format!("opening deployment stop store {}", path.display()))?;
|
||||||
|
let metadata = file.metadata()?;
|
||||||
|
ensure!(
|
||||||
|
metadata.is_file(),
|
||||||
|
"deployment stop store is not a regular file"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
metadata.uid() == expected_owner_uid,
|
||||||
|
"deployment stop store is owned by uid {}, not {expected_owner_uid}",
|
||||||
|
metadata.uid()
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
fn check_file(path: &Path, _: u32) -> Result<()> {
|
||||||
|
ensure!(
|
||||||
|
std::fs::metadata(path)?.is_file(),
|
||||||
|
"deployment stop store is not a regular file"
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Absent means exactly this: the final component does not exist and its
|
||||||
|
/// parent is an existing directory. Every other failure to look (permission,
|
||||||
|
/// a parent that is a file, a symlink loop) is a store this runtime cannot
|
||||||
|
/// judge, and fails closed.
|
||||||
|
fn is_absent(path: &Path) -> Result<bool> {
|
||||||
|
match std::fs::symlink_metadata(path) {
|
||||||
|
Ok(_) => Ok(false),
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
|
||||||
|
let parent = path.parent().filter(|p| !p.as_os_str().is_empty());
|
||||||
|
let parent = parent.unwrap_or_else(|| Path::new("."));
|
||||||
|
ensure!(
|
||||||
|
std::fs::metadata(parent).is_ok_and(|m| m.is_dir()),
|
||||||
|
"deployment stop store directory {} does not exist",
|
||||||
|
parent.display()
|
||||||
|
);
|
||||||
|
Ok(true)
|
||||||
|
}
|
||||||
|
Err(error) => Err(error)
|
||||||
|
.with_context(|| format!("looking for deployment stop store {}", path.display())),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_checked(path: &Path, runtime_id: &str, owner: u32) -> Result<Option<DeploymentStop>> {
|
||||||
|
if is_absent(path)? {
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
check_file(path, owner)?;
|
||||||
|
ensure!(
|
||||||
|
std::fs::metadata(path)?.len() > 0,
|
||||||
|
"deployment stop store is zero length"
|
||||||
|
);
|
||||||
|
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||||
|
match entries.as_slice() {
|
||||||
|
[] => bail!("deployment stop store holds no record"),
|
||||||
|
[entry] => {
|
||||||
|
let record = DeploymentStop::decode(entry)?;
|
||||||
|
ensure!(
|
||||||
|
record.runtime_id == runtime_id,
|
||||||
|
"deployment stop belongs to runtime {}",
|
||||||
|
record.runtime_id
|
||||||
|
);
|
||||||
|
Ok(Some(record))
|
||||||
|
}
|
||||||
|
_ => bail!("deployment stop store is ambiguous"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn read_stop(path: &Path, runtime_id: &str, expected_owner_uid: u32) -> Stop {
|
||||||
|
read_stop_with_remedy(path, runtime_id, expected_owner_uid, |path| {
|
||||||
|
format!(
|
||||||
|
"to replace it run: idunn-provision deployment-brake-clear \
|
||||||
|
--store {} --runtime-id {runtime_id} --reason <text> (or deployment-brake-engage)",
|
||||||
|
path.display()
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_stop_with_remedy(
|
||||||
|
path: &Path,
|
||||||
|
runtime_id: &str,
|
||||||
|
expected_owner_uid: u32,
|
||||||
|
remedy: impl FnOnce(&Path) -> String,
|
||||||
|
) -> Stop {
|
||||||
|
match read_checked(path, runtime_id, expected_owner_uid) {
|
||||||
|
Ok(None) => Stop::Absent,
|
||||||
|
Ok(Some(record)) if record.state == StopState::Clear => Stop::Clear(record),
|
||||||
|
Ok(Some(record)) => Stop::Engaged(record),
|
||||||
|
Err(error) => Stop::Unreadable(format!("{error:#}; {}", remedy(path))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Write the next record under a compare-exchange of the whole store, so a
|
||||||
|
/// concurrent verb or a store that moved since it was read is refused. The
|
||||||
|
/// author is the login name of the process uid and the sequence is the
|
||||||
|
/// predecessor's plus one (1 when there is none or it cannot be decoded);
|
||||||
|
/// neither is an argument.
|
||||||
|
///
|
||||||
|
/// This is also the operator's break-glass. A predecessor that is a retired v1
|
||||||
|
/// brake, another runtime's record, ambiguous, or zero length is replaced. A
|
||||||
|
/// file that is not a CultCache store at all (junk bytes) is replaced only
|
||||||
|
/// when it is a regular file owned by the caller, by removing it first. A
|
||||||
|
/// symlink or another user's file is refused. The sequence is not validated
|
||||||
|
/// beyond being positive: nothing reads it for authority.
|
||||||
|
pub(crate) fn replace_stop(
|
||||||
|
path: &Path,
|
||||||
|
runtime_id: &str,
|
||||||
|
state: StopState,
|
||||||
|
reason: &str,
|
||||||
|
now_unix_millis: u64,
|
||||||
|
) -> Result<DeploymentStop> {
|
||||||
|
let store = SingleFileMessagePackBackingStore::new(path);
|
||||||
|
let before = match store.pull_all_read_only_snapshot() {
|
||||||
|
Ok(entries) => entries,
|
||||||
|
Err(undecodable) => {
|
||||||
|
check_file(path, process_uid()).with_context(|| {
|
||||||
|
format!("deployment stop store is undecodable ({undecodable:#}) and not the caller's to replace")
|
||||||
|
})?;
|
||||||
|
std::fs::remove_file(path)?;
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let sequence = match before.as_slice() {
|
||||||
|
[entry] => match DeploymentStop::decode(entry) {
|
||||||
|
Ok(previous) => previous.sequence.saturating_add(1),
|
||||||
|
Err(_) => 1,
|
||||||
|
},
|
||||||
|
_ => 1,
|
||||||
|
};
|
||||||
|
let record = DeploymentStop {
|
||||||
|
schema_version: DEPLOYMENT_STOP_SCHEMA.into(),
|
||||||
|
runtime_id: runtime_id.into(),
|
||||||
|
state,
|
||||||
|
by: login_name(process_uid())?,
|
||||||
|
reason: reason.into(),
|
||||||
|
at_unix_millis: now_unix_millis,
|
||||||
|
sequence,
|
||||||
|
};
|
||||||
|
let next = record.envelope()?;
|
||||||
|
ensure!(
|
||||||
|
store.compare_exchange_snapshot(&before, &[next])?,
|
||||||
|
"deployment stop changed during transition"
|
||||||
|
);
|
||||||
|
Ok(record)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn engage(path: &Path, reason: &str, at: u64) -> Result<DeploymentStop> {
|
||||||
|
replace_stop(path, "runtime", StopState::Engaged, reason, at)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn absent_and_empty_stores_admit_a_clear_one_admits_an_engaged_one_does_not() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
let me = process_uid();
|
||||||
|
assert_eq!(read_stop(&path, "runtime", me), Stop::Absent);
|
||||||
|
engage(&path, "freeze for the audit", 100)?;
|
||||||
|
let Stop::Engaged(held) = read_stop(&path, "runtime", me) else {
|
||||||
|
bail!("an engaged stop did not read as engaged")
|
||||||
|
};
|
||||||
|
assert_eq!(held.reason, "freeze for the audit");
|
||||||
|
assert_eq!(held.by, login_name(me)?);
|
||||||
|
assert!(!Stop::Engaged(held).admits());
|
||||||
|
replace_stop(&path, "runtime", StopState::Clear, "audit done", 200)?;
|
||||||
|
let cleared = read_stop(&path, "runtime", me);
|
||||||
|
assert!(matches!(&cleared, Stop::Clear(record) if record.sequence == 2));
|
||||||
|
assert!(cleared.admits());
|
||||||
|
// Clear is not consumed: the same record admits again.
|
||||||
|
assert!(read_stop(&path, "runtime", me).admits());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn corrupt_foreign_other_runtime_symlinked_or_foreign_owned_stores_stop() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let me = process_uid();
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
engage(&path, "held", 100)?;
|
||||||
|
// Another runtime's record.
|
||||||
|
assert!(!read_stop(&path, "other", me).admits());
|
||||||
|
// Owned by someone other than the expected uid: even a CLEAR record
|
||||||
|
// from a foreign owner must not admit.
|
||||||
|
let cleared = dir.path().join("cleared.cc");
|
||||||
|
replace_stop(&cleared, "runtime", StopState::Clear, "thaw", 100)?;
|
||||||
|
assert!(read_stop(&cleared, "runtime", me).admits());
|
||||||
|
assert!(matches!(read_stop(&cleared, "runtime", me + 1), Stop::Unreadable(_)));
|
||||||
|
// A symlink to a valid, clear store.
|
||||||
|
let real = dir.path().join("real.cc");
|
||||||
|
replace_stop(&real, "runtime", StopState::Clear, "clear", 100)?;
|
||||||
|
let link = dir.path().join("link.cc");
|
||||||
|
#[cfg(unix)]
|
||||||
|
std::os::unix::fs::symlink(&real, &link)?;
|
||||||
|
#[cfg(unix)]
|
||||||
|
assert!(matches!(read_stop(&link, "runtime", me), Stop::Unreadable(_)));
|
||||||
|
// Bytes that are no store at all.
|
||||||
|
let junk = dir.path().join("junk.cc");
|
||||||
|
std::fs::write(&junk, b"not msgpack at all")?;
|
||||||
|
assert!(matches!(read_stop(&junk, "runtime", me), Stop::Unreadable(_)));
|
||||||
|
// A record of another type under another key (the retired v1 brake).
|
||||||
|
let foreign = dir.path().join("foreign.cc");
|
||||||
|
let store = SingleFileMessagePackBackingStore::new(&foreign);
|
||||||
|
let envelope = CultCacheEnvelope {
|
||||||
|
key: "deployment".into(),
|
||||||
|
r#type: "idunn.deployment_brake".into(),
|
||||||
|
payload: vec![0x90],
|
||||||
|
stored_at: "1970-01-01T00:00:00+00:00".into(),
|
||||||
|
schema_id: Some("idunn.deployment_brake.v1".into()),
|
||||||
|
};
|
||||||
|
assert!(store.compare_exchange_snapshot(&[], &[envelope])?);
|
||||||
|
assert!(matches!(read_stop(&foreign, "runtime", me), Stop::Unreadable(_)));
|
||||||
|
// A noncanonical payload under the right identity.
|
||||||
|
let mut stop = DeploymentStop::decode(&store_entry(&path)?)?;
|
||||||
|
stop.sequence = 9;
|
||||||
|
let mut loose = stop.envelope()?;
|
||||||
|
loose.payload.push(0xc0);
|
||||||
|
let noncanonical = dir.path().join("noncanonical.cc");
|
||||||
|
assert!(SingleFileMessagePackBackingStore::new(&noncanonical)
|
||||||
|
.compare_exchange_snapshot(&[], &[loose])?);
|
||||||
|
assert!(matches!(read_stop(&noncanonical, "runtime", me), Stop::Unreadable(_)));
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A binding whose steward is the user running the test, so the steward's
|
||||||
|
/// store can really be owned by the steward. None when run as root, which
|
||||||
|
/// no steward may be.
|
||||||
|
fn binding_with_steward(dir: &Path, user: &str) -> BrakeBinding {
|
||||||
|
BrakeBinding {
|
||||||
|
deployment_store: dir.join("operator.cc"),
|
||||||
|
lifecycle_store: dir.join("lifecycle.cc"),
|
||||||
|
steward: Some(crate::deployment::StewardBrakeBinding {
|
||||||
|
user: user.into(),
|
||||||
|
store: dir.join("steward.cc"),
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn admits_all(stops: &[NamedStop]) -> bool {
|
||||||
|
stops.iter().all(|named| named.stop.admits())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_steward_store_is_a_second_stop_and_the_operator_store_is_one_it_cannot_clear() -> Result<()> {
|
||||||
|
let me = process_uid();
|
||||||
|
if me == 0 {
|
||||||
|
eprintln!("SKIPPED: a steward is never root, and this run is root");
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let binding = binding_with_steward(dir.path(), &login_name(me)?);
|
||||||
|
let (operator, steward) = (&binding.deployment_store, &binding.steward.as_ref().unwrap().store);
|
||||||
|
// Both absent: admits.
|
||||||
|
assert!(admits_all(&read_stops(&binding, "runtime")));
|
||||||
|
// Steward engaged alone: halts, and the stop names the steward's store.
|
||||||
|
replace_stop(steward, "runtime", StopState::Engaged, "steward freeze", 10)?;
|
||||||
|
let stops = read_stops(&binding, "runtime");
|
||||||
|
assert!(!admits_all(&stops));
|
||||||
|
assert!(matches!(&stops[1].stop, Stop::Engaged(held) if held.reason == "steward freeze"));
|
||||||
|
// The note names the proven owner, and never the record's claimed `by`.
|
||||||
|
assert_eq!(
|
||||||
|
stops[1].note().unwrap(),
|
||||||
|
format!("steward {} deployment brake engaged: steward freeze", login_name(me)?)
|
||||||
|
);
|
||||||
|
// Both engaged, then the steward clears its own: the operator's holds.
|
||||||
|
replace_stop(operator, "runtime", StopState::Engaged, "operator hold", 11)?;
|
||||||
|
replace_stop(steward, "runtime", StopState::Clear, "steward thaw", 12)?;
|
||||||
|
let stops = read_stops(&binding, "runtime");
|
||||||
|
assert!(matches!(&stops[0].stop, Stop::Engaged(_)) && matches!(&stops[1].stop, Stop::Clear(_)));
|
||||||
|
assert!(!admits_all(&stops));
|
||||||
|
// The other order: operator clear, steward engaged still halts.
|
||||||
|
replace_stop(operator, "runtime", StopState::Clear, "operator thaw", 13)?;
|
||||||
|
replace_stop(steward, "runtime", StopState::Engaged, "steward again", 14)?;
|
||||||
|
assert!(!admits_all(&read_stops(&binding, "runtime")));
|
||||||
|
// Both clear: admits.
|
||||||
|
replace_stop(steward, "runtime", StopState::Clear, "steward thaw", 15)?;
|
||||||
|
assert!(admits_all(&read_stops(&binding, "runtime")));
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn steward_store_owned_by_another_uid_stops() -> Result<()> {
|
||||||
|
let me = process_uid();
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
// Steward is "nobody" (a real non-root user that is not the writer);
|
||||||
|
// the file is written by this process, so even a Clear record fails.
|
||||||
|
if me == 65534 || user_missing("nobody") {
|
||||||
|
eprintln!("SKIPPED: needs a user `nobody` other than the caller");
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let binding = binding_with_steward(dir.path(), "nobody");
|
||||||
|
let steward = &binding.steward.as_ref().unwrap().store;
|
||||||
|
replace_stop(steward, "runtime", StopState::Clear, "forged clear", 10)?;
|
||||||
|
let stops = read_stops(&binding, "runtime");
|
||||||
|
assert!(matches!(&stops[1].stop, Stop::Unreadable(why) if why.contains("is owned by uid")
|
||||||
|
&& why.contains("only nobody (the store's owner uid) can repair it")
|
||||||
|
&& why.contains("stays unreadable")
|
||||||
|
&& why.contains("idunn-provision deployment-brake-clear")));
|
||||||
|
// The operator's stop keeps the operator's remedy.
|
||||||
|
replace_stop(&binding.deployment_store, "other-runtime", StopState::Clear, "x", 11)?;
|
||||||
|
assert!(matches!(&read_stops(&binding, "runtime")[0].stop, Stop::Unreadable(why)
|
||||||
|
if why.contains("to replace it run:") && !why.contains("only ")));
|
||||||
|
assert!(!admits_all(&stops));
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn user_missing(name: &str) -> bool {
|
||||||
|
crate::steward_orders::user_uid(name).is_err()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_unresolvable_or_root_steward_user_is_an_unreadable_stop_not_an_absent_one() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
for user in ["no-such-login-zz", "root"] {
|
||||||
|
let binding = binding_with_steward(dir.path(), user);
|
||||||
|
let stops = read_stops(&binding, "runtime");
|
||||||
|
assert!(matches!(&stops[1].stop, Stop::Unreadable(_)), "{user}");
|
||||||
|
assert!(!admits_all(&stops));
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_binding_without_a_steward_reads_only_the_operator_store() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let mut binding = binding_with_steward(dir.path(), "nobody");
|
||||||
|
binding.steward = None;
|
||||||
|
let stops = read_stops(&binding, "runtime");
|
||||||
|
assert_eq!(stops.len(), 1);
|
||||||
|
assert!(admits_all(&stops));
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn store_entry(path: &Path) -> Result<CultCacheEnvelope> {
|
||||||
|
let mut entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||||
|
entries.pop().context("no entry")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_verb_replaces_a_retired_brake_record_and_the_sequence_advances_by_one() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
let store = SingleFileMessagePackBackingStore::new(&path);
|
||||||
|
let retired = CultCacheEnvelope {
|
||||||
|
key: "deployment".into(),
|
||||||
|
r#type: "idunn.deployment_brake".into(),
|
||||||
|
payload: vec![0x90],
|
||||||
|
stored_at: "1970-01-01T00:00:00+00:00".into(),
|
||||||
|
schema_id: Some("idunn.deployment_brake.v1".into()),
|
||||||
|
};
|
||||||
|
assert!(store.compare_exchange_snapshot(&[], &[retired])?);
|
||||||
|
assert_eq!(engage(&path, "first", 10)?.sequence, 1);
|
||||||
|
assert_eq!(store.pull_all_read_only_snapshot()?.len(), 1);
|
||||||
|
assert_eq!(engage(&path, "second", 11)?.sequence, 2);
|
||||||
|
assert_eq!(
|
||||||
|
replace_stop(&path, "runtime", StopState::Clear, "third", 12)?.sequence,
|
||||||
|
3
|
||||||
|
);
|
||||||
|
assert_eq!(DeploymentStop::decode(&store_entry(&path)?)?.reason, "third");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_stale_exchange_is_refused_and_a_record_without_a_reason_is_not_written() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
engage(&path, "held", 10)?;
|
||||||
|
let store = SingleFileMessagePackBackingStore::new(&path);
|
||||||
|
let stale = store.pull_all_read_only_snapshot()?;
|
||||||
|
engage(&path, "moved", 11)?;
|
||||||
|
let next = DeploymentStop {
|
||||||
|
schema_version: DEPLOYMENT_STOP_SCHEMA.into(),
|
||||||
|
runtime_id: "runtime".into(),
|
||||||
|
state: StopState::Clear,
|
||||||
|
by: "someone".into(),
|
||||||
|
reason: "late".into(),
|
||||||
|
at_unix_millis: 12,
|
||||||
|
sequence: 2,
|
||||||
|
}
|
||||||
|
.envelope()?;
|
||||||
|
assert!(!store.compare_exchange_snapshot(&stale, &[next])?);
|
||||||
|
assert!(replace_stop(&path, "runtime", StopState::Clear, " ", 13).is_err());
|
||||||
|
assert_eq!(DeploymentStop::decode(&store_entry(&path)?)?.reason, "moved");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn only_a_missing_file_in_an_existing_directory_is_absent() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let me = process_uid();
|
||||||
|
// Parent is a regular file: ENOTDIR, not absence.
|
||||||
|
let file = dir.path().join("plain");
|
||||||
|
std::fs::write(&file, b"x")?;
|
||||||
|
assert!(matches!(read_stop(&file.join("stop.cc"), "runtime", me), Stop::Unreadable(_)));
|
||||||
|
// Parent does not exist.
|
||||||
|
assert!(matches!(
|
||||||
|
read_stop(&dir.path().join("nope/stop.cc"), "runtime", me),
|
||||||
|
Stop::Unreadable(_)
|
||||||
|
));
|
||||||
|
// Zero-length file.
|
||||||
|
let empty = dir.path().join("empty.cc");
|
||||||
|
std::fs::write(&empty, b"")?;
|
||||||
|
assert!(matches!(read_stop(&empty, "runtime", me), Stop::Unreadable(_)));
|
||||||
|
// An existing directory with no store is absent.
|
||||||
|
assert_eq!(read_stop(&dir.path().join("stop.cc"), "runtime", me), Stop::Absent);
|
||||||
|
// A parent that cannot be searched (skipped as root, which can).
|
||||||
|
#[cfg(unix)]
|
||||||
|
if me != 0 {
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
let locked = dir.path().join("locked");
|
||||||
|
std::fs::create_dir(&locked)?;
|
||||||
|
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o000))?;
|
||||||
|
let reading = read_stop(&locked.join("stop.cc"), "runtime", me);
|
||||||
|
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o700))?;
|
||||||
|
assert!(matches!(reading, Stop::Unreadable(_)), "{reading:?}");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_author_is_the_uid_login_never_the_environment() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
// SAFETY: no other test reads USER, LOGNAME or LNAME.
|
||||||
|
unsafe {
|
||||||
|
std::env::set_var("USER", "spoof");
|
||||||
|
std::env::set_var("LOGNAME", "spoof");
|
||||||
|
}
|
||||||
|
let written = engage(&path, "held", 10)?;
|
||||||
|
assert_eq!(written.by, login_name(process_uid())?);
|
||||||
|
assert_ne!(written.by, "spoof");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unreadable_names_the_command_and_a_break_glass_clear_replaces_junk_and_v1() -> Result<()> {
|
||||||
|
let dir = tempfile::tempdir()?;
|
||||||
|
let me = process_uid();
|
||||||
|
let path = dir.path().join("stop.cc");
|
||||||
|
std::fs::write(&path, b"junk that is no store")?;
|
||||||
|
let Stop::Unreadable(why) = read_stop(&path, "runtime", me) else {
|
||||||
|
bail!("junk admitted")
|
||||||
|
};
|
||||||
|
assert!(why.contains("idunn-provision deployment-brake-clear --store"), "{why}");
|
||||||
|
assert!(why.contains("--runtime-id runtime"), "{why}");
|
||||||
|
let cleared = replace_stop(&path, "runtime", StopState::Clear, "operator break-glass", 50)?;
|
||||||
|
assert_eq!(cleared.sequence, 1);
|
||||||
|
assert!(read_stop(&path, "runtime", me).admits());
|
||||||
|
// A symlink is not the caller's to replace.
|
||||||
|
let real = dir.path().join("real.cc");
|
||||||
|
std::fs::write(&real, b"junk")?;
|
||||||
|
let link = dir.path().join("link.cc");
|
||||||
|
#[cfg(unix)]
|
||||||
|
{
|
||||||
|
std::os::unix::fs::symlink(&real, &link)?;
|
||||||
|
assert!(replace_stop(&link, "runtime", StopState::Clear, "x", 60).is_err());
|
||||||
|
assert_eq!(std::fs::read(&real)?, b"junk");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
4481
src/control_plane.rs
4481
src/control_plane.rs
File diff suppressed because it is too large
Load diff
|
|
@ -45,7 +45,12 @@ pub struct TargetDeclaration {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub external_inputs: Vec<ExternalInput>,
|
pub external_inputs: Vec<ExternalInput>,
|
||||||
pub artifacts: Vec<ArtifactOutput>,
|
pub artifacts: Vec<ArtifactOutput>,
|
||||||
pub service: ServiceDeclaration,
|
/// A running workload. Exactly one of `service` and `static_release`.
|
||||||
|
#[serde(default)]
|
||||||
|
pub service: Option<ServiceDeclaration>,
|
||||||
|
/// A sealed tree that is published by repointing a link, with no process.
|
||||||
|
#[serde(default)]
|
||||||
|
pub static_release: Option<StaticReleaseDeclaration>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub state: Option<StateDeclaration>,
|
pub state: Option<StateDeclaration>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
|
|
@ -126,6 +131,15 @@ pub enum ArtifactSource {
|
||||||
WorktreeTree,
|
WorktreeTree,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A target that is a sealed tree and nothing else: Idunn installs the release
|
||||||
|
/// and repoints the binding's current link at it. `artifact` names the tree a
|
||||||
|
/// web server reads through that link.
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct StaticReleaseDeclaration {
|
||||||
|
pub artifact: String,
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
#[serde(deny_unknown_fields)]
|
#[serde(deny_unknown_fields)]
|
||||||
pub struct ServiceDeclaration {
|
pub struct ServiceDeclaration {
|
||||||
|
|
@ -475,20 +489,42 @@ pub struct DockerRunnerBinding {
|
||||||
pub enum WorkloadBinding {
|
pub enum WorkloadBinding {
|
||||||
SystemdTransient(SystemdWorkloadBinding),
|
SystemdTransient(SystemdWorkloadBinding),
|
||||||
HostActuator(HostWorkloadBinding),
|
HostActuator(HostWorkloadBinding),
|
||||||
|
/// No process: a release directory and the link that names the current one.
|
||||||
|
StaticRelease(StaticReleaseBinding),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static NO_ENVIRONMENT: BTreeMap<String, String> = BTreeMap::new();
|
||||||
|
|
||||||
impl WorkloadBinding {
|
impl WorkloadBinding {
|
||||||
pub fn systemd(&self) -> Result<&SystemdWorkloadBinding> {
|
pub fn systemd(&self) -> Result<&SystemdWorkloadBinding> {
|
||||||
match self {
|
match self {
|
||||||
Self::SystemdTransient(binding) => Ok(binding),
|
Self::SystemdTransient(binding) => Ok(binding),
|
||||||
Self::HostActuator(_) => bail!("workload is host-actuator, not systemd-transient"),
|
Self::HostActuator(_) => bail!("workload is host-actuator, not systemd-transient"),
|
||||||
|
Self::StaticRelease(_) => bail!("workload is a static release, not systemd-transient"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn static_release(&self) -> Result<&StaticReleaseBinding> {
|
||||||
|
match self {
|
||||||
|
Self::StaticRelease(binding) => Ok(binding),
|
||||||
|
_ => bail!("workload is not a static release"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where Idunn installs sealed releases on its own host: the one thing a
|
||||||
|
/// systemd workload and a static release share.
|
||||||
|
pub fn release_root(&self) -> Result<&Path> {
|
||||||
|
match self {
|
||||||
|
Self::SystemdTransient(binding) => Ok(&binding.release_root),
|
||||||
|
Self::StaticRelease(binding) => Ok(&binding.release_root),
|
||||||
|
Self::HostActuator(_) => bail!("a host-actuator workload installs on its own host"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn host(&self) -> Result<&HostWorkloadBinding> {
|
pub fn host(&self) -> Result<&HostWorkloadBinding> {
|
||||||
match self {
|
match self {
|
||||||
Self::HostActuator(binding) => Ok(binding),
|
Self::HostActuator(binding) => Ok(binding),
|
||||||
Self::SystemdTransient(_) => bail!("workload is systemd-transient, not host-actuator"),
|
_ => bail!("workload is not host-actuator"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -496,6 +532,7 @@ impl WorkloadBinding {
|
||||||
match self {
|
match self {
|
||||||
Self::SystemdTransient(binding) => binding.state_root.is_some(),
|
Self::SystemdTransient(binding) => binding.state_root.is_some(),
|
||||||
Self::HostActuator(binding) => binding.state_root.is_some(),
|
Self::HostActuator(binding) => binding.state_root.is_some(),
|
||||||
|
Self::StaticRelease(_) => false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -503,6 +540,7 @@ impl WorkloadBinding {
|
||||||
match self {
|
match self {
|
||||||
Self::SystemdTransient(binding) => &binding.environment,
|
Self::SystemdTransient(binding) => &binding.environment,
|
||||||
Self::HostActuator(binding) => &binding.environment,
|
Self::HostActuator(binding) => &binding.environment,
|
||||||
|
Self::StaticRelease(_) => &NO_ENVIRONMENT,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -510,6 +548,7 @@ impl WorkloadBinding {
|
||||||
match self {
|
match self {
|
||||||
Self::SystemdTransient(binding) => binding.secret_files.keys().collect(),
|
Self::SystemdTransient(binding) => binding.secret_files.keys().collect(),
|
||||||
Self::HostActuator(binding) => binding.secret_files.keys().collect(),
|
Self::HostActuator(binding) => binding.secret_files.keys().collect(),
|
||||||
|
Self::StaticRelease(_) => Vec::new(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -517,6 +556,7 @@ impl WorkloadBinding {
|
||||||
match self {
|
match self {
|
||||||
Self::SystemdTransient(binding) => &binding.argument_bindings,
|
Self::SystemdTransient(binding) => &binding.argument_bindings,
|
||||||
Self::HostActuator(binding) => &binding.argument_bindings,
|
Self::HostActuator(binding) => &binding.argument_bindings,
|
||||||
|
Self::StaticRelease(_) => &NO_ENVIRONMENT,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -551,6 +591,49 @@ impl WorkloadBinding {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A static release on the Idunn host. `current_link` is the symlink a web
|
||||||
|
/// server's root reads through; Idunn repoints it at one sealed release.
|
||||||
|
/// `carry` names one operator-owned file (a per-site config) that Idunn copies
|
||||||
|
/// into each release as `destination` when it flips: the copy sits beside the
|
||||||
|
/// artifacts, outside every sealed digest.
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct StaticReleaseBinding {
|
||||||
|
pub release_root: PathBuf,
|
||||||
|
pub current_link: PathBuf,
|
||||||
|
pub carry: StaticCarryBinding,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct StaticCarryBinding {
|
||||||
|
pub source: PathBuf,
|
||||||
|
pub destination: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl StaticReleaseBinding {
|
||||||
|
fn validate(&self) -> Result<()> {
|
||||||
|
require_absolute_path(&self.release_root, "static release root")?;
|
||||||
|
require_absolute_path(&self.current_link, "static current link")?;
|
||||||
|
require_absolute_path(&self.carry.source, "static carry source")?;
|
||||||
|
require_leaf_path(&self.carry.destination, "static carry destination")?;
|
||||||
|
ensure!(
|
||||||
|
!paths_overlap(&self.current_link, &self.release_root),
|
||||||
|
"static current link and release root overlap"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
!self.carry.source.starts_with(&self.release_root),
|
||||||
|
"static carry source is inside the release root"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
self.carry.source != self.current_link
|
||||||
|
&& !self.carry.source.starts_with(&self.current_link),
|
||||||
|
"static carry source is the current link"
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A workload on a managed host. Every path is the host's and is validated by
|
/// A workload on a managed host. Every path is the host's and is validated by
|
||||||
/// the actuator that owns the host; here they are opaque non-empty values.
|
/// the actuator that owns the host; here they are opaque non-empty values.
|
||||||
/// The host's actuator is authenticated against `host_trust_anchor_store`,
|
/// The host's actuator is authenticated against `host_trust_anchor_store`,
|
||||||
|
|
@ -670,6 +753,20 @@ impl ProcessWriteLeaseBinding {
|
||||||
pub struct BrakeBinding {
|
pub struct BrakeBinding {
|
||||||
pub deployment_store: PathBuf,
|
pub deployment_store: PathBuf,
|
||||||
pub lifecycle_store: PathBuf,
|
pub lifecycle_store: PathBuf,
|
||||||
|
/// A second deployment stop, written by the target's steward in a
|
||||||
|
/// directory only the steward can write. Absent means the operator's store
|
||||||
|
/// is the only stop.
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub steward: Option<StewardBrakeBinding>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The steward's own stop: the file's owner uid proves who wrote it.
|
||||||
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct StewardBrakeBinding {
|
||||||
|
/// Login name; must resolve to a non-root uid when the store is read.
|
||||||
|
pub user: String,
|
||||||
|
pub store: PathBuf,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
|
@ -977,57 +1074,76 @@ impl TargetDeclaration {
|
||||||
"target declaration uses no runner classes"
|
"target declaration uses no runner classes"
|
||||||
);
|
);
|
||||||
|
|
||||||
ensure!(
|
match (&self.service, &self.static_release) {
|
||||||
artifact_ids.contains(self.service.executable_artifact.as_str()),
|
(Some(service), None) => {
|
||||||
"service executable artifact is not declared"
|
ensure!(
|
||||||
);
|
artifact_ids.contains(service.executable_artifact.as_str()),
|
||||||
for artifact in &self.service.required_adjacent_artifacts {
|
"service executable artifact is not declared"
|
||||||
|
);
|
||||||
|
for artifact in &service.required_adjacent_artifacts {
|
||||||
|
ensure!(
|
||||||
|
artifact_ids.contains(artifact.as_str()),
|
||||||
|
"adjacent service artifact {artifact} is not declared"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
for argument in &service.arguments {
|
||||||
|
validate_launch_argument(argument)?;
|
||||||
|
}
|
||||||
|
require_capability(&service.health.contract, "health contract")?;
|
||||||
|
for name in &service.required_environment {
|
||||||
|
require_environment_name(name)?;
|
||||||
|
}
|
||||||
ensure!(
|
ensure!(
|
||||||
artifact_ids.contains(artifact.as_str()),
|
service
|
||||||
"adjacent service artifact {artifact} is not declared"
|
.required_environment
|
||||||
|
.contains(IDUNN_RUNTIME_BUNDLE_ENVIRONMENT),
|
||||||
|
"service does not require the standard Idunn runtime bundle"
|
||||||
);
|
);
|
||||||
}
|
|
||||||
for argument in &self.service.arguments {
|
|
||||||
validate_launch_argument(argument)?;
|
|
||||||
}
|
|
||||||
require_capability(&self.service.health.contract, "health contract")?;
|
|
||||||
for name in &self.service.required_environment {
|
|
||||||
require_environment_name(name)?;
|
|
||||||
}
|
|
||||||
ensure!(
|
|
||||||
self.service
|
|
||||||
.required_environment
|
|
||||||
.contains(IDUNN_RUNTIME_BUNDLE_ENVIRONMENT),
|
|
||||||
"service does not require the standard Idunn runtime bundle"
|
|
||||||
);
|
|
||||||
ensure!(
|
|
||||||
self.service
|
|
||||||
.required_environment
|
|
||||||
.contains(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT)
|
|
||||||
== self.service.route_required,
|
|
||||||
"routed services must require exactly one Idunn-owned candidate bind"
|
|
||||||
);
|
|
||||||
ensure!(
|
|
||||||
!self
|
|
||||||
.service
|
|
||||||
.required_environment
|
|
||||||
.contains(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT)
|
|
||||||
|| self.service.route_required,
|
|
||||||
"only routed services may require the Idunn-owned member bind"
|
|
||||||
);
|
|
||||||
ensure!(
|
|
||||||
self.service
|
|
||||||
.required_environment
|
|
||||||
.contains(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT)
|
|
||||||
== self.write_lease_required(),
|
|
||||||
"state-writing services must require exactly one Idunn-owned process write lease"
|
|
||||||
);
|
|
||||||
for name in &self.service.optional_environment {
|
|
||||||
require_environment_name(name)?;
|
|
||||||
ensure!(
|
ensure!(
|
||||||
!self.service.required_environment.contains(name),
|
service
|
||||||
"service environment {name} is both required and optional"
|
.required_environment
|
||||||
|
.contains(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT)
|
||||||
|
== service.route_required,
|
||||||
|
"routed services must require exactly one Idunn-owned candidate bind"
|
||||||
);
|
);
|
||||||
|
ensure!(
|
||||||
|
!service
|
||||||
|
.required_environment
|
||||||
|
.contains(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT)
|
||||||
|
|| service.route_required,
|
||||||
|
"only routed services may require the Idunn-owned member bind"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
service
|
||||||
|
.required_environment
|
||||||
|
.contains(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT)
|
||||||
|
== self.write_lease_required(),
|
||||||
|
"state-writing services must require exactly one Idunn-owned process write lease"
|
||||||
|
);
|
||||||
|
for name in &service.optional_environment {
|
||||||
|
require_environment_name(name)?;
|
||||||
|
ensure!(
|
||||||
|
!service.required_environment.contains(name),
|
||||||
|
"service environment {name} is both required and optional"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
(None, Some(static_release)) => {
|
||||||
|
ensure!(
|
||||||
|
artifact_ids.contains(static_release.artifact.as_str()),
|
||||||
|
"static release artifact is not declared"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
self.state.is_none()
|
||||||
|
&& self.provides.is_empty()
|
||||||
|
&& self.dependencies.is_empty()
|
||||||
|
&& self.conflicts.is_empty(),
|
||||||
|
"a static release declares no state, capability, dependency or conflict"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
(Some(_), Some(_)) => bail!("target declares both a service and a static release"),
|
||||||
|
(None, None) => bail!("target declares neither a service nor a static release"),
|
||||||
}
|
}
|
||||||
|
|
||||||
if let Some(state) = &self.state {
|
if let Some(state) = &self.state {
|
||||||
|
|
@ -1144,6 +1260,14 @@ impl TargetDeclaration {
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The service of a service target. A static release has none, and a code
|
||||||
|
/// path that asks for one has been reached by the wrong kind of target.
|
||||||
|
pub fn service(&self) -> Result<&ServiceDeclaration> {
|
||||||
|
self.service
|
||||||
|
.as_ref()
|
||||||
|
.context("target declares no service")
|
||||||
|
}
|
||||||
|
|
||||||
pub fn write_lease_required(&self) -> bool {
|
pub fn write_lease_required(&self) -> bool {
|
||||||
self.state.as_ref().is_some_and(|state| {
|
self.state.as_ref().is_some_and(|state| {
|
||||||
state
|
state
|
||||||
|
|
@ -1320,6 +1444,39 @@ impl OperatorBinding {
|
||||||
self.brakes.deployment_store != self.brakes.lifecycle_store,
|
self.brakes.deployment_store != self.brakes.lifecycle_store,
|
||||||
"deployment and lifecycle brakes are identical"
|
"deployment and lifecycle brakes are identical"
|
||||||
);
|
);
|
||||||
|
if let Some(steward) = &self.brakes.steward {
|
||||||
|
require_absolute_path(&steward.store, "steward deployment brake")?;
|
||||||
|
ensure!(
|
||||||
|
crate::steward_orders::login_grammar_ok(&steward.user) && steward.user != "root",
|
||||||
|
"steward brake user is invalid or root"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
steward.store != self.brakes.deployment_store
|
||||||
|
&& steward.store != self.brakes.lifecycle_store,
|
||||||
|
"steward brake store is identical to another brake store"
|
||||||
|
);
|
||||||
|
// The steward owns its store's directory, so it could unlink any
|
||||||
|
// file there. Every other authority store must therefore live
|
||||||
|
// outside that directory, and the steward's directory must be
|
||||||
|
// clear of every directory an operator store lives in.
|
||||||
|
let steward_dir = steward.store.parent().unwrap_or(&steward.store);
|
||||||
|
let others = [
|
||||||
|
Some(&self.brakes.deployment_store),
|
||||||
|
Some(&self.brakes.lifecycle_store),
|
||||||
|
Some(&self.runtime_identity.trust_anchor_store),
|
||||||
|
self.process_write_lease.as_ref().map(|lease| &lease.record_path),
|
||||||
|
self.repository.release_authority_store.as_ref(),
|
||||||
|
];
|
||||||
|
for other in others.into_iter().flatten() {
|
||||||
|
let other_dir = other.parent().unwrap_or(other);
|
||||||
|
ensure!(
|
||||||
|
!paths_overlap(steward_dir, other_dir),
|
||||||
|
"steward brake directory {} is shared with or nested in the directory of Idunn authority {}: the steward owns its directory and could remove that file",
|
||||||
|
steward_dir.display(),
|
||||||
|
other.display()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
self.workload.validate_launch_environment()?;
|
self.workload.validate_launch_environment()?;
|
||||||
let workload = match &self.workload {
|
let workload = match &self.workload {
|
||||||
WorkloadBinding::HostActuator(workload) => {
|
WorkloadBinding::HostActuator(workload) => {
|
||||||
|
|
@ -1338,6 +1495,32 @@ impl OperatorBinding {
|
||||||
);
|
);
|
||||||
return self.validate_shared_tail();
|
return self.validate_shared_tail();
|
||||||
}
|
}
|
||||||
|
WorkloadBinding::StaticRelease(workload) => {
|
||||||
|
workload.validate()?;
|
||||||
|
ensure!(
|
||||||
|
self.process_write_lease.is_none() && self.route.is_none(),
|
||||||
|
"a static release has no write lease or route"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
self.rollout.strategy == RolloutStrategy::CandidateThenPromote,
|
||||||
|
"a static release rolls out candidate-then-promote: it never stops an incumbent"
|
||||||
|
);
|
||||||
|
for authority in [
|
||||||
|
&self.brakes.deployment_store,
|
||||||
|
&self.brakes.lifecycle_store,
|
||||||
|
&self.runtime_identity.trust_anchor_store,
|
||||||
|
]
|
||||||
|
.into_iter()
|
||||||
|
.chain(self.brakes.steward.as_ref().map(|steward| &steward.store))
|
||||||
|
{
|
||||||
|
ensure!(
|
||||||
|
!paths_overlap(authority, &workload.release_root)
|
||||||
|
&& !paths_overlap(authority, &workload.current_link),
|
||||||
|
"static release path overlaps an Idunn authority path"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return self.validate_shared_tail();
|
||||||
|
}
|
||||||
WorkloadBinding::SystemdTransient(workload) => {
|
WorkloadBinding::SystemdTransient(workload) => {
|
||||||
ensure!(
|
ensure!(
|
||||||
self.rollout.strategy == RolloutStrategy::CandidateThenPromote,
|
self.rollout.strategy == RolloutStrategy::CandidateThenPromote,
|
||||||
|
|
@ -1398,6 +1581,7 @@ impl OperatorBinding {
|
||||||
self.brakes.deployment_store.clone(),
|
self.brakes.deployment_store.clone(),
|
||||||
self.brakes.lifecycle_store.clone(),
|
self.brakes.lifecycle_store.clone(),
|
||||||
];
|
];
|
||||||
|
authority_paths.extend(self.brakes.steward.as_ref().map(|steward| steward.store.clone()));
|
||||||
if let Some(write_lease) = &self.process_write_lease {
|
if let Some(write_lease) = &self.process_write_lease {
|
||||||
authority_paths.push(write_lease.record_path.clone());
|
authority_paths.push(write_lease.record_path.clone());
|
||||||
authority_paths.push(write_lease.lock_path());
|
authority_paths.push(write_lease.lock_path());
|
||||||
|
|
@ -1565,6 +1749,11 @@ impl OperatorBinding {
|
||||||
self.target == declaration.target,
|
self.target == declaration.target,
|
||||||
"binding target does not match declaration"
|
"binding target does not match declaration"
|
||||||
);
|
);
|
||||||
|
ensure!(
|
||||||
|
declaration.static_release.is_some()
|
||||||
|
== matches!(self.workload, WorkloadBinding::StaticRelease(_)),
|
||||||
|
"a static release declaration needs a static-release workload, and only it"
|
||||||
|
);
|
||||||
let bound_gitlinks: BTreeSet<_> = self.repository.gitlinks.keys().cloned().collect();
|
let bound_gitlinks: BTreeSet<_> = self.repository.gitlinks.keys().cloned().collect();
|
||||||
ensure!(
|
ensure!(
|
||||||
bound_gitlinks == declaration.required_gitlinks,
|
bound_gitlinks == declaration.required_gitlinks,
|
||||||
|
|
@ -1676,6 +1865,19 @@ impl OperatorBinding {
|
||||||
input.id
|
input.id
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
let Some(service) = declaration.service.as_ref() else {
|
||||||
|
let WorkloadBinding::StaticRelease(workload) = &self.workload else {
|
||||||
|
bail!("a static release declaration needs a static-release workload");
|
||||||
|
};
|
||||||
|
ensure!(
|
||||||
|
declaration
|
||||||
|
.artifacts
|
||||||
|
.iter()
|
||||||
|
.all(|artifact| artifact.destination != workload.carry.destination),
|
||||||
|
"static carry destination collides with an artifact destination"
|
||||||
|
);
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
let workload_names: BTreeSet<&String> = self
|
let workload_names: BTreeSet<&String> = self
|
||||||
.workload
|
.workload
|
||||||
.environment()
|
.environment()
|
||||||
|
|
@ -1702,7 +1904,7 @@ impl OperatorBinding {
|
||||||
if host_workload {
|
if host_workload {
|
||||||
available_environment.insert(IDUNN_ACTIVATION_CREDENTIAL_ENVIRONMENT.into());
|
available_environment.insert(IDUNN_ACTIVATION_CREDENTIAL_ENVIRONMENT.into());
|
||||||
}
|
}
|
||||||
if declaration.service.route_required {
|
if service.route_required {
|
||||||
available_environment.insert(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT.into());
|
available_environment.insert(IDUNN_RUNTIME_CANDIDATE_BIND_ENVIRONMENT.into());
|
||||||
if self.route.as_ref().is_some_and(|route| route.member.is_some()) {
|
if self.route.as_ref().is_some_and(|route| route.member.is_some()) {
|
||||||
available_environment.insert(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT.into());
|
available_environment.insert(IDUNN_RUNTIME_MEMBER_BIND_ENVIRONMENT.into());
|
||||||
|
|
@ -1712,16 +1914,14 @@ impl OperatorBinding {
|
||||||
available_environment.insert(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT.into());
|
available_environment.insert(IDUNN_PROCESS_WRITE_LEASE_ENVIRONMENT.into());
|
||||||
}
|
}
|
||||||
ensure!(
|
ensure!(
|
||||||
declaration
|
service
|
||||||
.service
|
|
||||||
.required_environment
|
.required_environment
|
||||||
.is_subset(&available_environment),
|
.is_subset(&available_environment),
|
||||||
"operator service environment omits a required launch input"
|
"operator service environment omits a required launch input"
|
||||||
);
|
);
|
||||||
let declared_environment: BTreeSet<_> = declaration
|
let declared_environment: BTreeSet<_> = service
|
||||||
.service
|
|
||||||
.required_environment
|
.required_environment
|
||||||
.union(&declaration.service.optional_environment)
|
.union(&service.optional_environment)
|
||||||
.cloned()
|
.cloned()
|
||||||
.collect();
|
.collect();
|
||||||
// On the Idunn host the presence identity reaches the workload as a
|
// On the Idunn host the presence identity reaches the workload as a
|
||||||
|
|
@ -1741,8 +1941,7 @@ impl OperatorBinding {
|
||||||
available_environment.is_subset(&declared_environment),
|
available_environment.is_subset(&declared_environment),
|
||||||
"operator service environment contains an undeclared launch input"
|
"operator service environment contains an undeclared launch input"
|
||||||
);
|
);
|
||||||
let required_arguments: BTreeSet<_> = declaration
|
let required_arguments: BTreeSet<_> = service
|
||||||
.service
|
|
||||||
.arguments
|
.arguments
|
||||||
.iter()
|
.iter()
|
||||||
.filter_map(|argument| match argument {
|
.filter_map(|argument| match argument {
|
||||||
|
|
@ -1758,6 +1957,7 @@ impl OperatorBinding {
|
||||||
.as_ref()
|
.as_ref()
|
||||||
.map(|path| path.display().to_string()),
|
.map(|path| path.display().to_string()),
|
||||||
WorkloadBinding::HostActuator(workload) => workload.state_root.clone(),
|
WorkloadBinding::HostActuator(workload) => workload.state_root.clone(),
|
||||||
|
WorkloadBinding::StaticRelease(_) => None,
|
||||||
};
|
};
|
||||||
match (
|
match (
|
||||||
state_root.as_deref(),
|
state_root.as_deref(),
|
||||||
|
|
@ -1780,8 +1980,8 @@ impl OperatorBinding {
|
||||||
"operator argument bindings do not exactly match the launch contract"
|
"operator argument bindings do not exactly match the launch contract"
|
||||||
);
|
);
|
||||||
match (
|
match (
|
||||||
declaration.service.route_required,
|
service.route_required,
|
||||||
declaration.service.transport,
|
service.transport,
|
||||||
self.route.as_ref(),
|
self.route.as_ref(),
|
||||||
) {
|
) {
|
||||||
(true, ServiceTransport::Http, Some(route))
|
(true, ServiceTransport::Http, Some(route))
|
||||||
|
|
@ -2344,7 +2544,7 @@ nodes = ["yggdrasil"]
|
||||||
)
|
)
|
||||||
.is_err()
|
.is_err()
|
||||||
);
|
);
|
||||||
assert!(recipe.service.route_required);
|
assert!(recipe.service().unwrap().route_required);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -2832,4 +3032,230 @@ max_connections = {value}"),
|
||||||
"v2",
|
"v2",
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A static release: a vite-style tree published by repointing a link.
|
||||||
|
pub(crate) const STATIC_RECIPE: &str = r#"
|
||||||
|
schema = "gamecult.idunn.target_declaration.v1"
|
||||||
|
target = "service"
|
||||||
|
source_stamp_environment = "SERVICE_BUILD_COMMIT"
|
||||||
|
|
||||||
|
[[steps]]
|
||||||
|
id = "build"
|
||||||
|
phase = "build"
|
||||||
|
runner = "rust"
|
||||||
|
argv = ["true"]
|
||||||
|
|
||||||
|
[[artifacts]]
|
||||||
|
id = "dist"
|
||||||
|
source_kind = "worktree-tree"
|
||||||
|
source = "dist"
|
||||||
|
destination = "dist"
|
||||||
|
|
||||||
|
[static_release]
|
||||||
|
artifact = "dist"
|
||||||
|
"#;
|
||||||
|
|
||||||
|
/// `@SITE@` stands for the stand-in site tree a test lays out.
|
||||||
|
pub(crate) const STATIC_BINDING: &str = r#"
|
||||||
|
schema = "gamecult.idunn.operator_binding.v2"
|
||||||
|
target = "service"
|
||||||
|
|
||||||
|
[repository]
|
||||||
|
origin = "https://github.com/GameCult/Service.git"
|
||||||
|
admitted_ref = "refs/heads/main"
|
||||||
|
minimum_revision = "1111111111111111111111111111111111111111"
|
||||||
|
selection = "ref-head"
|
||||||
|
checkout = "/srv/build/Service"
|
||||||
|
recipe_path = "deployment/idunn/recipe.toml"
|
||||||
|
|
||||||
|
[runners.rust]
|
||||||
|
driver = "docker"
|
||||||
|
image = "rust@sha256:4c2fd73ef19c5ef9d54bee03b06b2839a392604fbfcd578ed948b71b37c1d7fb"
|
||||||
|
user = "1000:1000"
|
||||||
|
affordances = ["source-read", "artifact-write"]
|
||||||
|
allowed_programs = ["true"]
|
||||||
|
memory_mebibytes = 1024
|
||||||
|
cpu_quota_percent = 100
|
||||||
|
pids_limit = 128
|
||||||
|
tmpfs_mebibytes = 64
|
||||||
|
|
||||||
|
[workload]
|
||||||
|
driver = "static-release"
|
||||||
|
release_root = "@SITE@/releases"
|
||||||
|
current_link = "@SITE@/www/current"
|
||||||
|
|
||||||
|
[workload.carry]
|
||||||
|
source = "@SITE@/config/site.json"
|
||||||
|
destination = "config.json"
|
||||||
|
|
||||||
|
[runtime_identity]
|
||||||
|
runtime_id = "site-yggdrasil"
|
||||||
|
expected_signer_identity_id = "site-runtime-signer"
|
||||||
|
trust_anchor_store = "@SITE@/trust/site.cc"
|
||||||
|
|
||||||
|
[brakes]
|
||||||
|
deployment_store = "@SITE@/authority/deployment-brake.cc"
|
||||||
|
lifecycle_store = "@SITE@/authority/lifecycle-brake.cc"
|
||||||
|
|
||||||
|
[rollout]
|
||||||
|
strategy = "candidate-then-promote"
|
||||||
|
drain_seconds = 1
|
||||||
|
retain_releases = 2
|
||||||
|
|
||||||
|
[placement]
|
||||||
|
desired_replicas = 1
|
||||||
|
nodes = ["yggdrasil"]
|
||||||
|
"#;
|
||||||
|
|
||||||
|
fn static_binding(replace: &[(&str, &str)]) -> String {
|
||||||
|
let mut text = STATIC_BINDING.replace("@SITE@", "/srv/site");
|
||||||
|
for (from, to) in replace {
|
||||||
|
assert!(text.contains(from), "the fixture lost {from}");
|
||||||
|
text = text.replace(from, to);
|
||||||
|
}
|
||||||
|
text
|
||||||
|
}
|
||||||
|
|
||||||
|
const STEWARD: &str = "\n[brakes.steward]\nuser = \"mind-streampixels\"\nstore = \"/var/lib/gamecult/idunn-steward/streampixels/deployment-brake.cc\"\n";
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn binding_without_steward_parses_and_admits_on_operator_store_only() {
|
||||||
|
for text in [BINDING.to_string(), static_binding(&[])] {
|
||||||
|
assert_eq!(OperatorBinding::parse(&text).unwrap().brakes.steward, None);
|
||||||
|
}
|
||||||
|
let steward = OperatorBinding::parse(&BINDING.replace("\n[rollout]", &format!("{STEWARD}\n[rollout]")))
|
||||||
|
.unwrap()
|
||||||
|
.brakes
|
||||||
|
.steward
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(steward.user, "mind-streampixels");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_steward_brake_must_be_a_distinct_absolute_non_root_login_and_clear_of_the_workload() {
|
||||||
|
let with = |steward: &str| BINDING.replace("\n[rollout]", &format!("{steward}\n[rollout]"));
|
||||||
|
let store = "/var/lib/gamecult/idunn-steward/x/deployment-brake.cc";
|
||||||
|
for bad in [
|
||||||
|
format!("\n[brakes.steward]\nuser = \"root\"\nstore = \"{store}\"\n"),
|
||||||
|
format!("\n[brakes.steward]\nuser = \"Bad User\"\nstore = \"{store}\"\n"),
|
||||||
|
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"relative.cc\"\n".into(),
|
||||||
|
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/idunn-authority/test-service-deployment-brake.cc\"\n".into(),
|
||||||
|
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/idunn-authority/test-service-lifecycle-brake.cc\"\n".into(),
|
||||||
|
format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{store}\"\nextra = 1\n"),
|
||||||
|
] {
|
||||||
|
assert!(OperatorBinding::parse(&with(&bad)).is_err(), "{bad}");
|
||||||
|
}
|
||||||
|
// A steward directory shared with, or enclosing, an operator store
|
||||||
|
// lets the steward unlink a root-owned file: refused.
|
||||||
|
for (deployment, steward_store) in [
|
||||||
|
("/var/lib/gamecult/steward-x/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||||
|
("/var/lib/gamecult/steward-x/sub/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||||
|
("/var/lib/gamecult/op.cc", "/var/lib/gamecult/steward-x/s.cc"),
|
||||||
|
] {
|
||||||
|
let shared = BINDING
|
||||||
|
.replace("/var/lib/gamecult/idunn-authority/test-service-deployment-brake.cc", deployment)
|
||||||
|
.replace("\n[rollout]", &format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{steward_store}\"\n\n[rollout]"));
|
||||||
|
assert!(OperatorBinding::parse(&shared).is_err(), "{deployment} {steward_store}");
|
||||||
|
}
|
||||||
|
// Disjoint directories are the one admitted shape.
|
||||||
|
let disjoint = BINDING.replace(
|
||||||
|
"\n[rollout]",
|
||||||
|
"\n[brakes.steward]\nuser = \"mind\"\nstore = \"/var/lib/gamecult/steward-x/s.cc\"\n\n[rollout]",
|
||||||
|
);
|
||||||
|
OperatorBinding::parse(&disjoint).unwrap();
|
||||||
|
// steward_store_overlap_with_workload_path_refused
|
||||||
|
let overlapping = BINDING
|
||||||
|
.replace("\n[rollout]", &format!("\n[brakes.steward]\nuser = \"mind\"\nstore = \"{store}\"\n\n[rollout]"))
|
||||||
|
.replace(
|
||||||
|
"cpu_quota_percent = 200\n\n[workload.argument_bindings]",
|
||||||
|
"cpu_quota_percent = 200\nread_write_paths = [\"/var/lib/gamecult/idunn-steward/x\"]\n\n[workload.argument_bindings]",
|
||||||
|
);
|
||||||
|
assert!(OperatorBinding::parse(&overlapping).is_err());
|
||||||
|
let static_overlap = static_binding(&[("[rollout]", "[brakes.steward]\nuser = \"mind\"\nstore = \"/srv/site/releases/x/brake.cc\"\n\n[rollout]")]);
|
||||||
|
assert!(OperatorBinding::parse(&static_overlap).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_recipe_and_binding_admit() {
|
||||||
|
let recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||||
|
let binding = OperatorBinding::parse(&static_binding(&[])).unwrap();
|
||||||
|
binding.admit(&recipe).unwrap();
|
||||||
|
assert!(recipe.service().is_err());
|
||||||
|
assert_eq!(binding.workload.release_root().unwrap(), Path::new("/srv/site/releases"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_recipe_declares_exactly_one_of_service_and_static_release() {
|
||||||
|
let both = format!("{RECIPE}\n[static_release]\nartifact = \"daemon\"\n");
|
||||||
|
let error = TargetDeclaration::parse(&both).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains("both a service and a static release"));
|
||||||
|
let neither = STATIC_RECIPE.replace("[static_release]\nartifact = \"dist\"\n", "");
|
||||||
|
let error = TargetDeclaration::parse(&neither).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains("neither a service nor a static release"));
|
||||||
|
let unknown = STATIC_RECIPE.replace("artifact = \"dist\"", "artifact = \"nope\"");
|
||||||
|
assert!(TargetDeclaration::parse(&unknown).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_release_refuses_a_workload_contract() {
|
||||||
|
for extra in [
|
||||||
|
"\n[[provides]]\ncapability = \"site.runtime\"\nschema = \"site.v1\"\ncompatibility = \"v1\"\n",
|
||||||
|
"\n[[conflicts]]\ncapability = \"site.old\"\nreason = \"replaced\"\n",
|
||||||
|
"\n[state]\nschema_generation = \"v1\"\n\n[[state.slots]]\nid = \"world\"\nrelative_path = \"world.cc\"\nkind = \"cultcache-file\"\nschema = \"site.v1\"\nwriter = \"none\"\nrecovery = \"preserve\"\nstartup = \"open-at-start\"\n",
|
||||||
|
] {
|
||||||
|
let error = TargetDeclaration::parse(&format!("{STATIC_RECIPE}{extra}")).unwrap_err();
|
||||||
|
assert!(
|
||||||
|
format!("{error:#}").contains("a static release declares no state"),
|
||||||
|
"{extra}: {error:#}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
for (from, to, refusal) in [
|
||||||
|
("[rollout]", "[route]\ndriver = \"nginx-stream-tcp\"\nroute_id = \"r\"\nstable_endpoint = \"http://127.0.0.1:18830\"\nprivate_host = \"127.0.0.1\"\nprivate_port_start = 18831\nprivate_port_end = 18839\nconfig_path = \"/etc/nginx/r.conf\"\nreload_unit = \"nginx.service\"\n\n[rollout]", "no write lease or route"),
|
||||||
|
("[rollout]", "[process_write_lease]\nrecord_path = \"/srv/site/lease.cc\"\n\n[rollout]", "no write lease or route"),
|
||||||
|
("retain_releases = 2", "retain_releases = 1", "retain at least current and prior"),
|
||||||
|
("candidate-then-promote", "stop-then-start", "never stops an incumbent"),
|
||||||
|
] {
|
||||||
|
let error = OperatorBinding::parse(&static_binding(&[(from, to)])).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains(refusal), "{to}: {error:#}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_release_and_service_recipes_need_their_own_binding_kind() {
|
||||||
|
let static_recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||||
|
let service_recipe = TargetDeclaration::parse(RECIPE).unwrap();
|
||||||
|
let static_binding = OperatorBinding::parse(&static_binding(&[])).unwrap();
|
||||||
|
let service_binding = OperatorBinding::parse(BINDING).unwrap();
|
||||||
|
assert!(static_binding.admit(&service_recipe).is_err());
|
||||||
|
assert!(service_binding.admit(&static_recipe).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_carry_destination_must_not_be_an_artifact_destination() {
|
||||||
|
let recipe = TargetDeclaration::parse(STATIC_RECIPE).unwrap();
|
||||||
|
let binding =
|
||||||
|
OperatorBinding::parse(&static_binding(&[("destination = \"config.json\"", "destination = \"dist\"")]))
|
||||||
|
.unwrap();
|
||||||
|
let error = binding.admit(&recipe).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains("collides with an artifact destination"));
|
||||||
|
for destination in ["a/b", "../b", "/b", ""] {
|
||||||
|
let line = format!("destination = \"{destination}\"");
|
||||||
|
assert!(
|
||||||
|
OperatorBinding::parse(&static_binding(&[("destination = \"config.json\"", &line)])).is_err(),
|
||||||
|
"{destination}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn static_carry_source_and_link_stay_out_of_the_release_root() {
|
||||||
|
for (from, to) in [
|
||||||
|
("source = \"/srv/site/config/site.json\"", "source = \"/srv/site/releases/site.json\""),
|
||||||
|
("current_link = \"/srv/site/www/current\"", "current_link = \"/srv/site/releases/current\""),
|
||||||
|
("source = \"/srv/site/config/site.json\"", "source = \"site.json\""),
|
||||||
|
("release_root = \"/srv/site/releases\"", "release_root = \"/srv/site/www\""),
|
||||||
|
] {
|
||||||
|
assert!(OperatorBinding::parse(&static_binding(&[(from, to)])).is_err(), "{to}");
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -854,14 +854,15 @@ impl SealedRelease {
|
||||||
) -> Result<ExpectedIncarnation> {
|
) -> Result<ExpectedIncarnation> {
|
||||||
self.validate_against(plan)?;
|
self.validate_against(plan)?;
|
||||||
let (declaration, binding) = plan.parsed_inputs()?;
|
let (declaration, binding) = plan.parsed_inputs()?;
|
||||||
|
let service = declaration.service()?;
|
||||||
let route = match (&binding.route, plan.candidate_port) {
|
let route = match (&binding.route, plan.candidate_port) {
|
||||||
(Some(bound), Some(port)) => Some(IdunnExpectedRoute {
|
(Some(bound), Some(port)) => Some(IdunnExpectedRoute {
|
||||||
route_id: bound.route_id.clone(),
|
route_id: bound.route_id.clone(),
|
||||||
transport: endpoint_scheme(declaration.service.transport)?.into(),
|
transport: endpoint_scheme(service.transport)?.into(),
|
||||||
stable_endpoint: bound.stable_endpoint.clone(),
|
stable_endpoint: bound.stable_endpoint.clone(),
|
||||||
candidate_endpoint: format!(
|
candidate_endpoint: format!(
|
||||||
"{}://{}:{port}",
|
"{}://{}:{port}",
|
||||||
endpoint_scheme(declaration.service.transport)?,
|
endpoint_scheme(service.transport)?,
|
||||||
bound.private_host
|
bound.private_host
|
||||||
),
|
),
|
||||||
}),
|
}),
|
||||||
|
|
@ -871,7 +872,7 @@ impl SealedRelease {
|
||||||
let artifact_sha256 = self
|
let artifact_sha256 = self
|
||||||
.artifacts
|
.artifacts
|
||||||
.iter()
|
.iter()
|
||||||
.find(|artifact| artifact.artifact_id == declaration.service.executable_artifact)
|
.find(|artifact| artifact.artifact_id == service.executable_artifact)
|
||||||
.context("sealed release has no executable service artifact")?
|
.context("sealed release has no executable service artifact")?
|
||||||
.sha256
|
.sha256
|
||||||
.clone();
|
.clone();
|
||||||
|
|
@ -918,7 +919,7 @@ impl SealedRelease {
|
||||||
.runtime_identity
|
.runtime_identity
|
||||||
.expected_signer_identity_id
|
.expected_signer_identity_id
|
||||||
.clone(),
|
.clone(),
|
||||||
health_contract: declaration.service.health.contract.clone(),
|
health_contract: service.health.contract.clone(),
|
||||||
artifact_sha256,
|
artifact_sha256,
|
||||||
state_schema_generation: declaration
|
state_schema_generation: declaration
|
||||||
.state
|
.state
|
||||||
|
|
|
||||||
1294
src/drivers.rs
1294
src/drivers.rs
File diff suppressed because it is too large
Load diff
|
|
@ -631,11 +631,11 @@ impl Actuator {
|
||||||
workload: &HostWorkloadBinding,
|
workload: &HostWorkloadBinding,
|
||||||
installed: &Path,
|
installed: &Path,
|
||||||
) -> Result<(PathBuf, Vec<String>)> {
|
) -> Result<(PathBuf, Vec<String>)> {
|
||||||
let artifact = release_artifact(declaration, &declaration.service.executable_artifact)?;
|
let artifact = release_artifact(declaration, &declaration.service()?.executable_artifact)?;
|
||||||
let executable = installed.join(&artifact.destination);
|
let executable = installed.join(&artifact.destination);
|
||||||
ensure!(executable.is_file(), "sealed service executable is absent");
|
ensure!(executable.is_file(), "sealed service executable is absent");
|
||||||
let mut arguments = Vec::new();
|
let mut arguments = Vec::new();
|
||||||
for argument in &declaration.service.arguments {
|
for argument in &declaration.service()?.arguments {
|
||||||
arguments.push(match argument {
|
arguments.push(match argument {
|
||||||
LaunchArgument::Literal { value } => value.clone(),
|
LaunchArgument::Literal { value } => value.clone(),
|
||||||
LaunchArgument::Binding { name } => workload
|
LaunchArgument::Binding { name } => workload
|
||||||
|
|
|
||||||
|
|
@ -1151,7 +1151,7 @@ mod tests {
|
||||||
let answerer = thread::spawn(move || -> Result<()> {
|
let answerer = thread::spawn(move || -> Result<()> {
|
||||||
for _ in 0..500 {
|
for _ in 0..500 {
|
||||||
client.poll_resends()?;
|
client.poll_resends()?;
|
||||||
while let Some(frame) = client.receive_once()? {
|
if let Some(frame) = client.receive_once()? {
|
||||||
let envelope = HostActuatorEnvelope::decode(&frame.payload)?;
|
let envelope = HostActuatorEnvelope::decode(&frame.payload)?;
|
||||||
let request = envelope.open_request("raven", &idunn_anchor)?;
|
let request = envelope.open_request("raven", &idunn_anchor)?;
|
||||||
assert_eq!(envelope.session_nonce, nonce);
|
assert_eq!(envelope.session_nonce, nonce);
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,4 @@
|
||||||
|
mod brake;
|
||||||
pub mod control_plane;
|
pub mod control_plane;
|
||||||
pub mod deployment;
|
pub mod deployment;
|
||||||
pub mod deployment_plan;
|
pub mod deployment_plan;
|
||||||
|
|
@ -7,4 +8,7 @@ pub mod host_actuator;
|
||||||
#[cfg(unix)]
|
#[cfg(unix)]
|
||||||
pub mod launcher;
|
pub mod launcher;
|
||||||
mod incident;
|
mod incident;
|
||||||
|
mod runner_evidence;
|
||||||
pub mod provisioning;
|
pub mod provisioning;
|
||||||
|
#[cfg(unix)]
|
||||||
|
mod steward_orders;
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,5 @@
|
||||||
use anyhow::{Context, Result, anyhow, bail};
|
use crate::brake::{Stop, StopState, process_uid, read_stop, replace_stop};
|
||||||
|
use anyhow::{Context, Result, anyhow, bail, ensure};
|
||||||
use cultcache_rs::{
|
use cultcache_rs::{
|
||||||
CultCacheEnvelope, CultCacheExpectedEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore,
|
CultCacheEnvelope, CultCacheExpectedEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore,
|
||||||
};
|
};
|
||||||
|
|
@ -6,15 +7,13 @@ use cultnet_rs::{
|
||||||
GAMECULT_SERVICE_TRUST_ANCHOR_SCHEMA, GameCultProviderHealthIdentity,
|
GAMECULT_SERVICE_TRUST_ANCHOR_SCHEMA, GameCultProviderHealthIdentity,
|
||||||
GameCultServiceTrustAnchorRecord, IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SCHEMA,
|
GameCultServiceTrustAnchorRecord, IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SCHEMA,
|
||||||
IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SIGNING_PURPOSE,
|
IDUNN_AUTHENTICATED_PROVIDER_HEALTH_PROJECTION_SIGNING_PURPOSE,
|
||||||
IDUNN_DAEMON_HEALTH_TRUST_BINDING_SCHEMA, IDUNN_DEPLOYMENT_BRAKE_AUTHORITY,
|
IDUNN_DAEMON_HEALTH_TRUST_BINDING_SCHEMA,
|
||||||
IDUNN_DEPLOYMENT_BRAKE_ID, IDUNN_DEPLOYMENT_BRAKE_SCHEMA, IDUNN_DEPLOYMENT_BRAKE_SCOPE,
|
IDUNN_LIFECYCLE_BRAKE_AUTHORITY,
|
||||||
IDUNN_DEPLOYMENT_RELEASE_PURPOSE, IDUNN_LIFECYCLE_BRAKE_AUTHORITY,
|
|
||||||
IDUNN_LIFECYCLE_BRAKE_SCHEMA, IDUNN_LIFECYCLE_BRAKE_SCOPE, IdunnDaemonHealthTrustBindingRecord,
|
IDUNN_LIFECYCLE_BRAKE_SCHEMA, IDUNN_LIFECYCLE_BRAKE_SCOPE, IdunnDaemonHealthTrustBindingRecord,
|
||||||
IdunnDeploymentBrakeObservation, IdunnDeploymentBrakeOperatorIdentity,
|
IdunnLifecycleBrakeObservation,
|
||||||
IdunnDeploymentBrakeRecord, IdunnDeploymentBrakeReleasePurpose, IdunnLifecycleBrakeObservation,
|
|
||||||
IdunnLifecycleBrakeRecord, IdunnServiceIdentity, OdinTopologyIdentity, ServiceIdentityProfile,
|
IdunnLifecycleBrakeRecord, IdunnServiceIdentity, OdinTopologyIdentity, ServiceIdentityProfile,
|
||||||
ServiceIdentityTrustAnchor, derive_service_identity_id, enroll_service_identity_at,
|
ServiceIdentityTrustAnchor, derive_service_identity_id, enroll_service_identity_at,
|
||||||
evaluate_idunn_continuity_restart, evaluate_idunn_deployment_brake,
|
evaluate_idunn_continuity_restart,
|
||||||
export_service_identity_trust_anchor, open_service_identity_at,
|
export_service_identity_trust_anchor, open_service_identity_at,
|
||||||
};
|
};
|
||||||
use std::collections::{BTreeMap, BTreeSet};
|
use std::collections::{BTreeMap, BTreeSet};
|
||||||
|
|
@ -80,17 +79,8 @@ pub fn run(args: impl IntoIterator<Item = String>) -> Result<()> {
|
||||||
"export-odin-topology-public-anchor" => {
|
"export-odin-topology-public-anchor" => {
|
||||||
export_identity_anchor::<OdinTopologyIdentity>(&options, "Odin topology public anchor")?
|
export_identity_anchor::<OdinTopologyIdentity>(&options, "Odin topology public anchor")?
|
||||||
}
|
}
|
||||||
"enroll-deployment-brake-operator" => {
|
"deployment-brake-engage" => deployment_brake_write(&options, StopState::Engaged)?,
|
||||||
enroll_identity::<IdunnDeploymentBrakeOperatorIdentity>(&options)?
|
"deployment-brake-clear" => deployment_brake_write(&options, StopState::Clear)?,
|
||||||
}
|
|
||||||
"export-deployment-brake-operator-anchor" => {
|
|
||||||
export_identity_anchor::<IdunnDeploymentBrakeOperatorIdentity>(
|
|
||||||
&options,
|
|
||||||
"deployment brake operator anchor",
|
|
||||||
)?
|
|
||||||
}
|
|
||||||
"deployment-brake-engage" => deployment_brake_engage(&options)?,
|
|
||||||
"deployment-brake-release" => deployment_brake_release(&options)?,
|
|
||||||
"deployment-brake-status" => deployment_brake_status(&options)?,
|
"deployment-brake-status" => deployment_brake_status(&options)?,
|
||||||
"lifecycle-brake-engage" => lifecycle_brake_engage(&options)?,
|
"lifecycle-brake-engage" => lifecycle_brake_engage(&options)?,
|
||||||
"lifecycle-brake-release" => lifecycle_brake_release(&options)?,
|
"lifecycle-brake-release" => lifecycle_brake_release(&options)?,
|
||||||
|
|
@ -157,153 +147,60 @@ fn provider_health_public_key(options: &BTreeMap<String, String>) -> Result<Stri
|
||||||
encode_public_key(&signer.entry().public_key)
|
encode_public_key(&signer.entry().public_key)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn deployment_brake_engage(options: &BTreeMap<String, String>) -> Result<()> {
|
/// Engage or clear the deployment stop. The author is the process uid's login
|
||||||
require_only(
|
/// name and the time is this verb's clock: neither is an argument.
|
||||||
options,
|
fn deployment_brake_write(options: &BTreeMap<String, String>, state: StopState) -> Result<()> {
|
||||||
&[
|
require_only(options, &["store", "runtime-id", "reason"])?;
|
||||||
"store",
|
let now = u64::try_from(
|
||||||
"runtime-id",
|
std::time::SystemTime::now()
|
||||||
"owner",
|
.duration_since(std::time::UNIX_EPOCH)?
|
||||||
"reason",
|
.as_millis(),
|
||||||
"observed-at-unix-millis",
|
|
||||||
],
|
|
||||||
)?;
|
)?;
|
||||||
let observed = parse_u64(options, "observed-at-unix-millis")?;
|
let record = replace_stop(
|
||||||
let record = IdunnDeploymentBrakeRecord {
|
&path(options, "store")?,
|
||||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
|
||||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
|
||||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
|
||||||
runtime_id: required(options, "runtime-id")?.into(),
|
|
||||||
status: "engaged".into(),
|
|
||||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
|
||||||
reason: required(options, "reason")?.into(),
|
|
||||||
observed_at_unix_millis: observed,
|
|
||||||
expires_at_unix_millis: None,
|
|
||||||
authorization_id: None,
|
|
||||||
authorization_purpose: None,
|
|
||||||
authorized_release_id: None,
|
|
||||||
authorized_deployment_id: None,
|
|
||||||
authorized_by: None,
|
|
||||||
authorization_issued_at_unix_millis: None,
|
|
||||||
authorization_expires_at_unix_millis: None,
|
|
||||||
signature_algorithm: None,
|
|
||||||
signature: None,
|
|
||||||
private_state_exposed: false,
|
|
||||||
updated_by: required(options, "owner")?.into(),
|
|
||||||
};
|
|
||||||
record.validate()?;
|
|
||||||
replace_brake(&path(options, "store")?, record, observed)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn deployment_brake_release(options: &BTreeMap<String, String>) -> Result<()> {
|
|
||||||
require_only(
|
|
||||||
options,
|
|
||||||
&[
|
|
||||||
"store",
|
|
||||||
"private-store",
|
|
||||||
"runtime-id",
|
|
||||||
"owner",
|
|
||||||
"reason",
|
|
||||||
"authorization-id",
|
|
||||||
"release-id",
|
|
||||||
"deployment-id",
|
|
||||||
"issued-at-unix-millis",
|
|
||||||
"expires-at-unix-millis",
|
|
||||||
],
|
|
||||||
)?;
|
|
||||||
let issued = parse_u64(options, "issued-at-unix-millis")?;
|
|
||||||
let expires = parse_u64(options, "expires-at-unix-millis")?;
|
|
||||||
let signer = open_service_identity_at::<IdunnDeploymentBrakeOperatorIdentity>(&path(
|
|
||||||
options,
|
|
||||||
"private-store",
|
|
||||||
)?)?;
|
|
||||||
let mut record = IdunnDeploymentBrakeRecord {
|
|
||||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
|
||||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
|
||||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
|
||||||
runtime_id: required(options, "runtime-id")?.into(),
|
|
||||||
status: "released".into(),
|
|
||||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
|
||||||
reason: required(options, "reason")?.into(),
|
|
||||||
observed_at_unix_millis: issued,
|
|
||||||
expires_at_unix_millis: Some(expires),
|
|
||||||
authorization_id: Some(required(options, "authorization-id")?.into()),
|
|
||||||
authorization_purpose: Some(IDUNN_DEPLOYMENT_RELEASE_PURPOSE.into()),
|
|
||||||
authorized_release_id: Some(required(options, "release-id")?.into()),
|
|
||||||
authorized_deployment_id: Some(required(options, "deployment-id")?.into()),
|
|
||||||
authorized_by: Some(signer.trust_anchor()?.identity_id),
|
|
||||||
authorization_issued_at_unix_millis: Some(issued),
|
|
||||||
authorization_expires_at_unix_millis: Some(expires),
|
|
||||||
signature_algorithm: Some("ed25519".into()),
|
|
||||||
signature: None,
|
|
||||||
private_state_exposed: false,
|
|
||||||
updated_by: required(options, "owner")?.into(),
|
|
||||||
};
|
|
||||||
record.signature = Some(
|
|
||||||
signer
|
|
||||||
.sign::<IdunnDeploymentBrakeReleasePurpose>(&rmp_serde::to_vec(&record)?)
|
|
||||||
.signature,
|
|
||||||
);
|
|
||||||
record.validate()?;
|
|
||||||
replace_brake(&path(options, "store")?, record, issued)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn deployment_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
|
||||||
allow_only(
|
|
||||||
options,
|
|
||||||
&[
|
|
||||||
"store",
|
|
||||||
"operator-anchor",
|
|
||||||
"runtime-id",
|
|
||||||
"release-id",
|
|
||||||
"deployment-id",
|
|
||||||
"now-unix-millis",
|
|
||||||
],
|
|
||||||
)?;
|
|
||||||
let exact_actuation_fields = ["release-id", "deployment-id", "now-unix-millis"];
|
|
||||||
let exact_actuation_field_count = exact_actuation_fields
|
|
||||||
.into_iter()
|
|
||||||
.filter(|field| options.contains_key(*field))
|
|
||||||
.count();
|
|
||||||
if exact_actuation_field_count != 0
|
|
||||||
&& exact_actuation_field_count != exact_actuation_fields.len()
|
|
||||||
{
|
|
||||||
bail!("exact deployment-brake actuation validation requires release, deployment, and time");
|
|
||||||
}
|
|
||||||
let record = read_brake(&path(options, "store")?)?;
|
|
||||||
if record.runtime_id != required(options, "runtime-id")? {
|
|
||||||
bail!("deployment brake belongs to another runtime")
|
|
||||||
}
|
|
||||||
if record.status == "engaged" {
|
|
||||||
if exact_actuation_field_count != 0 {
|
|
||||||
bail!("engaged deployment brake denies exact actuation validation");
|
|
||||||
}
|
|
||||||
println!(
|
|
||||||
"engaged runtime={} scope={} owner={} reason={}",
|
|
||||||
record.runtime_id, record.scope, record.updated_by, record.reason
|
|
||||||
);
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
let anchor = read_operator_anchor(&path(options, "operator-anchor")?)?;
|
|
||||||
evaluate_idunn_deployment_brake(
|
|
||||||
IdunnDeploymentBrakeObservation::Present(&record),
|
|
||||||
&anchor,
|
|
||||||
required(options, "runtime-id")?,
|
required(options, "runtime-id")?,
|
||||||
required(options, "release-id")?,
|
state,
|
||||||
required(options, "deployment-id")?,
|
required(options, "reason")?,
|
||||||
parse_u64(options, "now-unix-millis")?,
|
now,
|
||||||
)
|
)?;
|
||||||
.map_err(|d| anyhow!("deployment brake denied actuation: {d:?}"))?;
|
|
||||||
println!(
|
println!(
|
||||||
"released runtime={} release={} deployment={} owner={}",
|
"{} runtime={} by={} sequence={}",
|
||||||
|
record.state.name(),
|
||||||
record.runtime_id,
|
record.runtime_id,
|
||||||
record.authorized_release_id.unwrap(),
|
record.by,
|
||||||
record.authorized_deployment_id.unwrap(),
|
record.sequence
|
||||||
record.updated_by
|
|
||||||
);
|
);
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Print the stop and exit nonzero unless deployments admit, so a script can
|
||||||
|
/// gate on it as it could on the retired exact-actuation validation.
|
||||||
|
fn deployment_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
||||||
|
allow_only(options, &["store", "runtime-id", "owner-user"])?;
|
||||||
|
// A steward's store is owned by the steward: the operator names that user
|
||||||
|
// to read it, and the owner check is the same one Idunn makes.
|
||||||
|
let owner = match options.get("owner-user") {
|
||||||
|
Some(user) => crate::steward_orders::user_uid(user)?,
|
||||||
|
None => process_uid(),
|
||||||
|
};
|
||||||
|
let stop = read_stop(&path(options, "store")?, required(options, "runtime-id")?, owner);
|
||||||
|
match &stop {
|
||||||
|
Stop::Absent => println!("absent: deployments admit"),
|
||||||
|
Stop::Unreadable(why) => println!("unreadable: deployments wait ({why})"),
|
||||||
|
Stop::Clear(record) | Stop::Engaged(record) => println!(
|
||||||
|
"{} runtime={} by={} at={} sequence={} reason={}",
|
||||||
|
record.state.name(),
|
||||||
|
record.runtime_id,
|
||||||
|
record.by,
|
||||||
|
record.at_unix_millis,
|
||||||
|
record.sequence,
|
||||||
|
record.reason
|
||||||
|
),
|
||||||
|
}
|
||||||
|
ensure!(stop.admits(), "the deployment brake does not admit deployments");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
fn lifecycle_brake_engage(options: &BTreeMap<String, String>) -> Result<()> {
|
fn lifecycle_brake_engage(options: &BTreeMap<String, String>) -> Result<()> {
|
||||||
require_only(
|
require_only(
|
||||||
options,
|
options,
|
||||||
|
|
@ -407,7 +304,7 @@ fn lifecycle_brake_status(options: &BTreeMap<String, String>) -> Result<()> {
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn replace_lifecycle_brake(path: &Path, record: IdunnLifecycleBrakeRecord) -> Result<()> {
|
pub(crate) fn replace_lifecycle_brake(path: &Path, record: IdunnLifecycleBrakeRecord) -> Result<()> {
|
||||||
record.validate()?;
|
record.validate()?;
|
||||||
let store = SingleFileMessagePackBackingStore::new(path);
|
let store = SingleFileMessagePackBackingStore::new(path);
|
||||||
let entries = store.pull_all_read_only_snapshot()?;
|
let entries = store.pull_all_read_only_snapshot()?;
|
||||||
|
|
@ -482,70 +379,6 @@ fn lifecycle_identifier<'a>(options: &'a BTreeMap<String, String>, name: &str) -
|
||||||
Ok(value)
|
Ok(value)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn replace_brake(path: &Path, record: IdunnDeploymentBrakeRecord, millis: u64) -> Result<()> {
|
|
||||||
let store = SingleFileMessagePackBackingStore::new(path);
|
|
||||||
let entries = store.pull_all_read_only_snapshot()?;
|
|
||||||
let current = match entries.as_slice() {
|
|
||||||
[] => None,
|
|
||||||
[entry]
|
|
||||||
if entry.r#type == IdunnDeploymentBrakeRecord::TYPE
|
|
||||||
&& entry.key == IDUNN_DEPLOYMENT_BRAKE_ID =>
|
|
||||||
{
|
|
||||||
Some(entry.clone())
|
|
||||||
}
|
|
||||||
_ => bail!("deployment brake store is corrupt or ambiguous"),
|
|
||||||
};
|
|
||||||
let next = typed_envelope(
|
|
||||||
IDUNN_DEPLOYMENT_BRAKE_ID,
|
|
||||||
&record,
|
|
||||||
IDUNN_DEPLOYMENT_BRAKE_SCHEMA,
|
|
||||||
millis,
|
|
||||||
)?;
|
|
||||||
if !store.compare_exchange(
|
|
||||||
&[CultCacheExpectedEnvelope {
|
|
||||||
r#type: IdunnDeploymentBrakeRecord::TYPE.into(),
|
|
||||||
key: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
|
||||||
current,
|
|
||||||
}],
|
|
||||||
&[next],
|
|
||||||
)? {
|
|
||||||
bail!("deployment brake changed during transition");
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_brake(path: &Path) -> Result<IdunnDeploymentBrakeRecord> {
|
|
||||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
|
||||||
let [entry] = entries.as_slice() else {
|
|
||||||
bail!("deployment brake is missing or ambiguous")
|
|
||||||
};
|
|
||||||
if entry.r#type != IdunnDeploymentBrakeRecord::TYPE
|
|
||||||
|| entry.key != IDUNN_DEPLOYMENT_BRAKE_ID
|
|
||||||
|| entry.schema_id.as_deref() != Some(IDUNN_DEPLOYMENT_BRAKE_SCHEMA)
|
|
||||||
{
|
|
||||||
bail!("deployment brake is foreign")
|
|
||||||
}
|
|
||||||
let record: IdunnDeploymentBrakeRecord = rmp_serde::from_slice(&entry.payload)?;
|
|
||||||
if rmp_serde::to_vec(&record)? != entry.payload {
|
|
||||||
bail!("deployment brake is noncanonical")
|
|
||||||
}
|
|
||||||
record.validate()?;
|
|
||||||
Ok(record)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_operator_anchor(path: &Path) -> Result<ServiceIdentityTrustAnchor> {
|
|
||||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
|
||||||
let [entry] = entries.as_slice() else {
|
|
||||||
bail!("operator anchor is missing or ambiguous")
|
|
||||||
};
|
|
||||||
if entry.r#type != IdunnDeploymentBrakeOperatorIdentity::TRUST_ANCHOR_TYPE
|
|
||||||
|| entry.key != IdunnDeploymentBrakeOperatorIdentity::TRUST_ANCHOR_KEY
|
|
||||||
{
|
|
||||||
bail!("operator anchor is foreign")
|
|
||||||
}
|
|
||||||
Ok(rmp_serde::from_slice(&entry.payload)?)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn validate_health_binding_store(path: &Path) -> Result<()> {
|
fn validate_health_binding_store(path: &Path) -> Result<()> {
|
||||||
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
let entries = SingleFileMessagePackBackingStore::new(path).pull_all_read_only_snapshot()?;
|
||||||
validated_health_bindings(&entries)?;
|
validated_health_bindings(&entries)?;
|
||||||
|
|
@ -1127,7 +960,7 @@ fn normalized(path: &Path) -> Result<PathBuf> {
|
||||||
}
|
}
|
||||||
|
|
||||||
fn usage() -> &'static str {
|
fn usage() -> &'static str {
|
||||||
"Usage: idunn-provision enroll-provider-health-identity --private-store <path>\n idunn-provision provider-health-public-key --private-store <path>\n idunn-provision export-provider-health-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-idunn-identity --private-store <path>\n idunn-provision export-idunn-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-odin-topology-identity --private-store <path>\n idunn-provision export-odin-topology-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-deployment-brake-operator --private-store <path>\n idunn-provision export-deployment-brake-operator-anchor --private-store <path> --public-anchor <path>\n idunn-provision deployment-brake-engage --store <path> --runtime-id <id> --owner <principal> --reason <text> --observed-at-unix-millis <u64>\n idunn-provision deployment-brake-release --store <path> --private-store <path> --runtime-id <id> --owner <principal> --reason <text> --authorization-id <id> --release-id <id> --deployment-id <id> --issued-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision deployment-brake-status --store <path> --operator-anchor <path> --runtime-id <id> [--release-id <id> --deployment-id <id> --now-unix-millis <u64>]\n idunn-provision lifecycle-brake-engage --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64>\n idunn-provision lifecycle-brake-release --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64> [--released-until-unix-millis <u64>]\n idunn-provision lifecycle-brake-status --store <path> --runtime-id <id> --target <id> --now-unix-millis <u64>\n idunn-provision create-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision add-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision require-daemon-health-release-binding --store <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex>\n idunn-provision rotate-daemon-health-trust-signer --output <path> --binding-id <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64>\n idunn-provision validate-daemon-health-trust-binding --input <path>\n idunn-provision create-provider-projection-trust-anchor --output <path> --trust-anchor-id <id> --runtime-id <id> --idunn-public-anchor <path> --bound-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision validate-provider-projection-trust-anchor --input <path> --idunn-public-anchor <path>"
|
"Usage: idunn-provision enroll-provider-health-identity --private-store <path>\n idunn-provision provider-health-public-key --private-store <path>\n idunn-provision export-provider-health-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-idunn-identity --private-store <path>\n idunn-provision export-idunn-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision enroll-odin-topology-identity --private-store <path>\n idunn-provision export-odin-topology-public-anchor --private-store <path> --public-anchor <path>\n idunn-provision deployment-brake-engage --store <path> --runtime-id <id> --reason <text>\n idunn-provision deployment-brake-clear --store <path> --runtime-id <id> --reason <text>\n idunn-provision deployment-brake-status --store <path> --runtime-id <id> [--owner-user <login>]\n idunn-provision lifecycle-brake-engage --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64>\n idunn-provision lifecycle-brake-release --store <path> --runtime-id <id> --target <id> --reason <text> --updated-at-unix-millis <u64> [--released-until-unix-millis <u64>]\n idunn-provision lifecycle-brake-status --store <path> --runtime-id <id> --target <id> --now-unix-millis <u64>\n idunn-provision create-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision add-daemon-health-trust-binding --output <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64> --release-binding-required <true|false>\n idunn-provision require-daemon-health-release-binding --store <path> --binding-id <id> --daemon <id> --health-contract <id> --source-runtime <id> --signer-public-key-hex <hex>\n idunn-provision rotate-daemon-health-trust-signer --output <path> --binding-id <id> --signer-public-key-hex <hex> --bound-at-unix-millis <u64>\n idunn-provision validate-daemon-health-trust-binding --input <path>\n idunn-provision create-provider-projection-trust-anchor --output <path> --trust-anchor-id <id> --runtime-id <id> --idunn-public-anchor <path> --bound-at-unix-millis <u64> --expires-at-unix-millis <u64>\n idunn-provision validate-provider-projection-trust-anchor --input <path> --idunn-public-anchor <path>"
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|
@ -2157,120 +1990,74 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn brake_operator_can_issue_one_exact_bounded_rollout_grant() -> Result<()> {
|
fn provision_engage_clear_sequence() -> Result<()> {
|
||||||
let temp = TempDir::new()?;
|
let temp = TempDir::new()?;
|
||||||
let private = temp.path().join("operator-private.cc");
|
|
||||||
let anchor = temp.path().join("operator-public.cc");
|
|
||||||
let store = temp.path().join("brake.cc");
|
let store = temp.path().join("brake.cc");
|
||||||
invoke(&[
|
let me = process_uid();
|
||||||
"enroll-deployment-brake-operator",
|
let engage = |verb: &str, reason: &str| {
|
||||||
"--private-store",
|
invoke(&[
|
||||||
private.to_str().unwrap(),
|
verb,
|
||||||
])?;
|
"--store",
|
||||||
invoke(&[
|
store.to_str().unwrap(),
|
||||||
"export-deployment-brake-operator-anchor",
|
"--runtime-id",
|
||||||
"--private-store",
|
"yggdrasil",
|
||||||
private.to_str().unwrap(),
|
"--reason",
|
||||||
"--public-anchor",
|
reason,
|
||||||
anchor.to_str().unwrap(),
|
])
|
||||||
])?;
|
};
|
||||||
invoke(&[
|
assert_eq!(read_stop(&store, "yggdrasil", me), Stop::Absent);
|
||||||
"deployment-brake-engage",
|
|
||||||
"--store",
|
|
||||||
store.to_str().unwrap(),
|
|
||||||
"--runtime-id",
|
|
||||||
"yggdrasil",
|
|
||||||
"--owner",
|
|
||||||
"operator/metacrat",
|
|
||||||
"--reason",
|
|
||||||
"sleep mode",
|
|
||||||
"--observed-at-unix-millis",
|
|
||||||
"100",
|
|
||||||
])?;
|
|
||||||
assert_eq!(read_brake(&store)?.status, "engaged");
|
|
||||||
invoke(&[
|
invoke(&[
|
||||||
"deployment-brake-status",
|
"deployment-brake-status",
|
||||||
"--store",
|
"--store",
|
||||||
store.to_str().unwrap(),
|
store.to_str().unwrap(),
|
||||||
"--operator-anchor",
|
|
||||||
anchor.to_str().unwrap(),
|
|
||||||
"--runtime-id",
|
"--runtime-id",
|
||||||
"yggdrasil",
|
"yggdrasil",
|
||||||
])?;
|
])?;
|
||||||
assert!(
|
engage("deployment-brake-engage", "freeze")?;
|
||||||
|
let status = || {
|
||||||
invoke(&[
|
invoke(&[
|
||||||
"deployment-brake-status",
|
"deployment-brake-status",
|
||||||
"--store",
|
"--store",
|
||||||
store.to_str().unwrap(),
|
store.to_str().unwrap(),
|
||||||
"--operator-anchor",
|
|
||||||
anchor.to_str().unwrap(),
|
|
||||||
"--runtime-id",
|
"--runtime-id",
|
||||||
"yggdrasil",
|
"yggdrasil",
|
||||||
"--release-id",
|
|
||||||
"commit-4",
|
|
||||||
"--deployment-id",
|
|
||||||
"request-4",
|
|
||||||
"--now-unix-millis",
|
|
||||||
"150",
|
|
||||||
])
|
])
|
||||||
.is_err()
|
};
|
||||||
);
|
assert!(status().is_err(), "an engaged stop must not pass status");
|
||||||
invoke(&[
|
let Stop::Engaged(first) = read_stop(&store, "yggdrasil", me) else {
|
||||||
"deployment-brake-release",
|
bail!("engage did not engage")
|
||||||
"--store",
|
};
|
||||||
store.to_str().unwrap(),
|
assert_eq!((first.sequence, first.reason.as_str()), (1, "freeze"));
|
||||||
"--private-store",
|
assert!(!first.by.is_empty() && first.at_unix_millis > 0);
|
||||||
private.to_str().unwrap(),
|
engage("deployment-brake-clear", "thawed")?;
|
||||||
"--runtime-id",
|
status()?;
|
||||||
"yggdrasil",
|
let Stop::Clear(second) = read_stop(&store, "yggdrasil", me) else {
|
||||||
"--owner",
|
bail!("clear did not clear")
|
||||||
"operator/metacrat",
|
};
|
||||||
"--reason",
|
assert_eq!((second.sequence, second.by), (2, first.by));
|
||||||
"one rollout",
|
assert!(second.at_unix_millis >= first.at_unix_millis);
|
||||||
"--authorization-id",
|
// The author and the time are not arguments, and the retired
|
||||||
"auth/r4",
|
// signing arguments are refused.
|
||||||
"--release-id",
|
for stray in ["--owner", "--observed-at-unix-millis", "--private-store"] {
|
||||||
"commit-4",
|
assert!(
|
||||||
"--deployment-id",
|
invoke(&[
|
||||||
"request-4",
|
"deployment-brake-engage",
|
||||||
"--issued-at-unix-millis",
|
"--store",
|
||||||
"200",
|
store.to_str().unwrap(),
|
||||||
"--expires-at-unix-millis",
|
"--runtime-id",
|
||||||
"800",
|
"yggdrasil",
|
||||||
])?;
|
"--reason",
|
||||||
invoke(&[
|
"x",
|
||||||
"deployment-brake-status",
|
stray,
|
||||||
"--store",
|
"y",
|
||||||
store.to_str().unwrap(),
|
])
|
||||||
"--operator-anchor",
|
.is_err(),
|
||||||
anchor.to_str().unwrap(),
|
"{stray}"
|
||||||
"--runtime-id",
|
);
|
||||||
"yggdrasil",
|
}
|
||||||
"--release-id",
|
for gone in ["deployment-brake-release", "enroll-deployment-brake-operator"] {
|
||||||
"commit-4",
|
assert!(invoke(&[gone]).is_err(), "{gone}");
|
||||||
"--deployment-id",
|
}
|
||||||
"request-4",
|
|
||||||
"--now-unix-millis",
|
|
||||||
"500",
|
|
||||||
])?;
|
|
||||||
assert!(
|
|
||||||
invoke(&[
|
|
||||||
"deployment-brake-status",
|
|
||||||
"--store",
|
|
||||||
store.to_str().unwrap(),
|
|
||||||
"--operator-anchor",
|
|
||||||
anchor.to_str().unwrap(),
|
|
||||||
"--runtime-id",
|
|
||||||
"yggdrasil",
|
|
||||||
"--release-id",
|
|
||||||
"substituted",
|
|
||||||
"--deployment-id",
|
|
||||||
"request-4",
|
|
||||||
"--now-unix-millis",
|
|
||||||
"500"
|
|
||||||
])
|
|
||||||
.is_err()
|
|
||||||
);
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -2309,7 +2096,6 @@ mod tests {
|
||||||
assert_eq!(engaged.status, "engaged");
|
assert_eq!(engaged.status, "engaged");
|
||||||
assert_eq!(engaged.runtime_id, "idunn-yggdrasil");
|
assert_eq!(engaged.runtime_id, "idunn-yggdrasil");
|
||||||
assert_eq!(engaged.target, "ghostlight");
|
assert_eq!(engaged.target, "ghostlight");
|
||||||
assert!(read_brake(&store).is_err());
|
|
||||||
assert!(
|
assert!(
|
||||||
invoke(&[
|
invoke(&[
|
||||||
"lifecycle-brake-status",
|
"lifecycle-brake-status",
|
||||||
|
|
@ -2426,57 +2212,4 @@ mod tests {
|
||||||
assert_eq!(std::fs::read(&store)?, released_bytes);
|
assert_eq!(std::fs::read(&store)?, released_bytes);
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn brake_compare_exchange_rejects_stale_transition_snapshot() -> Result<()> {
|
|
||||||
let temp = TempDir::new()?;
|
|
||||||
let store_path = temp.path().join("brake.cc");
|
|
||||||
let base = IdunnDeploymentBrakeRecord {
|
|
||||||
schema_version: IDUNN_DEPLOYMENT_BRAKE_SCHEMA.into(),
|
|
||||||
brake_id: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
|
||||||
authority: IDUNN_DEPLOYMENT_BRAKE_AUTHORITY.into(),
|
|
||||||
runtime_id: "yggdrasil".into(),
|
|
||||||
status: "engaged".into(),
|
|
||||||
scope: IDUNN_DEPLOYMENT_BRAKE_SCOPE.into(),
|
|
||||||
reason: "first".into(),
|
|
||||||
observed_at_unix_millis: 100,
|
|
||||||
expires_at_unix_millis: None,
|
|
||||||
authorization_id: None,
|
|
||||||
authorization_purpose: None,
|
|
||||||
authorized_release_id: None,
|
|
||||||
authorized_deployment_id: None,
|
|
||||||
authorized_by: None,
|
|
||||||
authorization_issued_at_unix_millis: None,
|
|
||||||
authorization_expires_at_unix_millis: None,
|
|
||||||
signature_algorithm: None,
|
|
||||||
signature: None,
|
|
||||||
private_state_exposed: false,
|
|
||||||
updated_by: "operator/a".into(),
|
|
||||||
};
|
|
||||||
replace_brake(&store_path, base.clone(), 100)?;
|
|
||||||
let stale = SingleFileMessagePackBackingStore::new(&store_path)
|
|
||||||
.pull_all_read_only_snapshot()?
|
|
||||||
.remove(0);
|
|
||||||
let mut changed = base.clone();
|
|
||||||
changed.reason = "racing writer".into();
|
|
||||||
changed.observed_at_unix_millis = 101;
|
|
||||||
replace_brake(&store_path, changed, 101)?;
|
|
||||||
let candidate = typed_envelope(
|
|
||||||
IDUNN_DEPLOYMENT_BRAKE_ID,
|
|
||||||
&base,
|
|
||||||
IDUNN_DEPLOYMENT_BRAKE_SCHEMA,
|
|
||||||
102,
|
|
||||||
)?;
|
|
||||||
assert!(
|
|
||||||
!SingleFileMessagePackBackingStore::new(&store_path).compare_exchange(
|
|
||||||
&[CultCacheExpectedEnvelope {
|
|
||||||
r#type: IdunnDeploymentBrakeRecord::TYPE.into(),
|
|
||||||
key: IDUNN_DEPLOYMENT_BRAKE_ID.into(),
|
|
||||||
current: Some(stale)
|
|
||||||
}],
|
|
||||||
&[candidate]
|
|
||||||
)?
|
|
||||||
);
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
805
src/runner_evidence.rs
Normal file
805
src/runner_evidence.rs
Normal file
|
|
@ -0,0 +1,805 @@
|
||||||
|
//! The output tail of a failed runner step.
|
||||||
|
//!
|
||||||
|
//! One file, `runner-evidence.cc`, beside `control.cc`, holds one type: the
|
||||||
|
//! last bytes of a failed runner container's stdout and stderr, one record per
|
||||||
|
//! transaction. The engine's pre-fence failure path is its only writer, and
|
||||||
|
//! `idunn status --command` its only reader. It is never `control.cc`, which
|
||||||
|
//! refuses any type it does not know, and never `incidents.cc`, which is
|
||||||
|
//! published and carries no free text.
|
||||||
|
//!
|
||||||
|
//! The tail is raw runner output. It is kept unredacted and readable by root
|
||||||
|
//! alone (mode 0600): root can already read every secret the runner was given.
|
||||||
|
//! It never enters an error, a transaction record or the journal. The status
|
||||||
|
//! reader is a terminal, so everything it prints from this store goes through
|
||||||
|
//! one allowlist sink (`emit`): printable ASCII passes, all else is escaped.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use anyhow::{Context, Result, bail, ensure};
|
||||||
|
use cultcache_rs::{
|
||||||
|
CultCacheEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore,
|
||||||
|
TryCompareExchangeSnapshotOutcome,
|
||||||
|
};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
use crate::control_plane::{decode_record, require_id, typed_envelope};
|
||||||
|
use crate::drivers::{
|
||||||
|
LockContended, RUNNER_OUTPUT_TAIL_BYTES, RunnerStepFailure, RunnerSubject,
|
||||||
|
exchange_behind_private_lock, runner_status,
|
||||||
|
};
|
||||||
|
|
||||||
|
const RUNNER_FAILURE_SCHEMA: &str = "idunn.runner_failure.v1";
|
||||||
|
|
||||||
|
/// Records kept; the writer drops the oldest past this, so the store needs no
|
||||||
|
/// other owner.
|
||||||
|
pub(crate) const RUNNER_EVIDENCE_RECORDS: usize = 32;
|
||||||
|
|
||||||
|
/// `runner-evidence.cc`, beside the control store.
|
||||||
|
pub(crate) fn runner_evidence_store_path(state_store: &Path) -> PathBuf {
|
||||||
|
state_store.with_file_name("runner-evidence.cc")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "kebab-case")]
|
||||||
|
pub(crate) enum SubjectKind {
|
||||||
|
Step,
|
||||||
|
ExternalInput,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, PartialEq, Eq, DatabaseEntry)]
|
||||||
|
#[cultcache(type = "idunn.runner_failure", schema = "idunn.runner_failure.v1")]
|
||||||
|
pub(crate) struct RunnerFailureRecord {
|
||||||
|
#[cultcache(key = 0)]
|
||||||
|
pub(crate) schema_version: String,
|
||||||
|
#[cultcache(key = 1)]
|
||||||
|
pub(crate) transaction_id: String,
|
||||||
|
#[cultcache(key = 2)]
|
||||||
|
pub(crate) command_id: String,
|
||||||
|
#[cultcache(key = 3)]
|
||||||
|
pub(crate) target: String,
|
||||||
|
#[cultcache(key = 4)]
|
||||||
|
pub(crate) subject_kind: SubjectKind,
|
||||||
|
#[cultcache(key = 5)]
|
||||||
|
pub(crate) subject_id: String,
|
||||||
|
#[cultcache(key = 6)]
|
||||||
|
pub(crate) runner: String,
|
||||||
|
#[cultcache(key = 7)]
|
||||||
|
pub(crate) exit_code: Option<i32>,
|
||||||
|
#[cultcache(key = 8)]
|
||||||
|
pub(crate) signal: Option<i32>,
|
||||||
|
#[cultcache(key = 9)]
|
||||||
|
pub(crate) stdout_tail: Vec<u8>,
|
||||||
|
#[cultcache(key = 10)]
|
||||||
|
pub(crate) stdout_total_bytes: u64,
|
||||||
|
#[cultcache(key = 11)]
|
||||||
|
pub(crate) stderr_tail: Vec<u8>,
|
||||||
|
#[cultcache(key = 12)]
|
||||||
|
pub(crate) stderr_total_bytes: u64,
|
||||||
|
#[cultcache(key = 13)]
|
||||||
|
pub(crate) recorded_at_unix_millis: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Tails stay out of `Debug`, so a failed assertion or a logged record never
|
||||||
|
/// prints runner output.
|
||||||
|
impl std::fmt::Debug for RunnerFailureRecord {
|
||||||
|
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
formatter
|
||||||
|
.debug_struct("RunnerFailureRecord")
|
||||||
|
.field("transaction_id", &self.transaction_id)
|
||||||
|
.field("subject_id", &self.subject_id)
|
||||||
|
.finish_non_exhaustive()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RunnerFailureRecord {
|
||||||
|
pub(crate) fn from_failure(
|
||||||
|
transaction_id: &str,
|
||||||
|
command_id: &str,
|
||||||
|
target: &str,
|
||||||
|
failure: &RunnerStepFailure,
|
||||||
|
now: u64,
|
||||||
|
) -> Self {
|
||||||
|
let (subject_kind, subject_id) = match &failure.subject {
|
||||||
|
RunnerSubject::Step(id) => (SubjectKind::Step, id),
|
||||||
|
RunnerSubject::ExternalInput(id) => (SubjectKind::ExternalInput, id),
|
||||||
|
};
|
||||||
|
Self {
|
||||||
|
schema_version: RUNNER_FAILURE_SCHEMA.to_owned(),
|
||||||
|
transaction_id: transaction_id.to_owned(),
|
||||||
|
command_id: command_id.to_owned(),
|
||||||
|
target: target.to_owned(),
|
||||||
|
subject_kind,
|
||||||
|
subject_id: subject_id.clone(),
|
||||||
|
runner: failure.runner.clone(),
|
||||||
|
exit_code: failure.exit_code,
|
||||||
|
signal: failure.signal,
|
||||||
|
stdout_tail: failure.stdout_tail.clone(),
|
||||||
|
stdout_total_bytes: failure.stdout_total_bytes,
|
||||||
|
stderr_tail: failure.stderr_tail.clone(),
|
||||||
|
stderr_total_bytes: failure.stderr_total_bytes,
|
||||||
|
recorded_at_unix_millis: now,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn validate(&self) -> Result<()> {
|
||||||
|
ensure!(
|
||||||
|
self.schema_version == RUNNER_FAILURE_SCHEMA,
|
||||||
|
"runner failure schema is unsupported"
|
||||||
|
);
|
||||||
|
require_id(&self.transaction_id, "runner failure transaction")?;
|
||||||
|
require_id(&self.command_id, "runner failure command")?;
|
||||||
|
require_id(&self.target, "runner failure target")?;
|
||||||
|
require_id(&self.subject_id, "runner failure subject")?;
|
||||||
|
require_id(&self.runner, "runner failure runner")?;
|
||||||
|
for (tail, total, stream) in [
|
||||||
|
(&self.stdout_tail, self.stdout_total_bytes, "stdout"),
|
||||||
|
(&self.stderr_tail, self.stderr_total_bytes, "stderr"),
|
||||||
|
] {
|
||||||
|
ensure!(
|
||||||
|
tail.len() <= RUNNER_OUTPUT_TAIL_BYTES,
|
||||||
|
"runner failure {stream} tail exceeds its bound"
|
||||||
|
);
|
||||||
|
ensure!(
|
||||||
|
tail.len() as u64 <= total,
|
||||||
|
"runner failure {stream} tail exceeds its stream"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn envelope(&self) -> Result<CultCacheEnvelope> {
|
||||||
|
self.validate()?;
|
||||||
|
typed_envelope(
|
||||||
|
&self.transaction_id,
|
||||||
|
Self::TYPE,
|
||||||
|
RUNNER_FAILURE_SCHEMA,
|
||||||
|
self,
|
||||||
|
self.recorded_at_unix_millis,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One read of the store: the envelopes exactly as stored, and the records
|
||||||
|
/// decoded from them in the same order.
|
||||||
|
fn snapshot(path: &Path) -> Result<(Vec<CultCacheEnvelope>, Vec<RunnerFailureRecord>)> {
|
||||||
|
let envelopes = SingleFileMessagePackBackingStore::new(path)
|
||||||
|
.pull_all_read_only_snapshot()
|
||||||
|
.context("reading the runner evidence store")?;
|
||||||
|
let mut records = Vec::new();
|
||||||
|
for envelope in &envelopes {
|
||||||
|
ensure!(
|
||||||
|
envelope.r#type == RunnerFailureRecord::TYPE
|
||||||
|
&& envelope.schema_id.as_deref() == Some(RUNNER_FAILURE_SCHEMA),
|
||||||
|
"the runner evidence store holds a document that is not a runner failure"
|
||||||
|
);
|
||||||
|
let record: RunnerFailureRecord = decode_record(envelope)?;
|
||||||
|
record.validate()?;
|
||||||
|
ensure!(
|
||||||
|
envelope.key == record.transaction_id,
|
||||||
|
"runner evidence store key differs from the record's transaction"
|
||||||
|
);
|
||||||
|
records.push(record);
|
||||||
|
}
|
||||||
|
Ok((envelopes, records))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How a write left the store.
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||||
|
pub(crate) enum Kept {
|
||||||
|
Stored,
|
||||||
|
/// The store held bytes that do not decode; it was moved to
|
||||||
|
/// `unreadable_store_path` and the record opened a fresh one. `replaced`
|
||||||
|
/// is the last-written time (unix milliseconds) of an earlier set-aside
|
||||||
|
/// store this one displaced, which is gone: one slot, and it says so.
|
||||||
|
SetAside { replaced: Option<u64> },
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where an undecodable store is kept. One slot: the next one replaces it, so
|
||||||
|
/// the damage a bad store can cost is one extra file.
|
||||||
|
pub(crate) fn unreadable_store_path(path: &Path) -> PathBuf {
|
||||||
|
path.with_file_name("runner-evidence.unreadable.cc")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Moves the store over the previous set-aside one, root-only, and reports
|
||||||
|
/// what it displaced. Nothing of the store's content is named: a decode error
|
||||||
|
/// can quote stored values, and none is carried here.
|
||||||
|
fn set_aside_unreadable(path: &Path) -> Result<Option<u64>> {
|
||||||
|
let aside = unreadable_store_path(path);
|
||||||
|
let replaced = std::fs::symlink_metadata(&aside).ok().map(|earlier| {
|
||||||
|
earlier
|
||||||
|
.modified()
|
||||||
|
.ok()
|
||||||
|
.and_then(|time| time.duration_since(std::time::UNIX_EPOCH).ok())
|
||||||
|
.map_or(0, |since| since.as_millis() as u64)
|
||||||
|
});
|
||||||
|
std::fs::rename(path, &aside)
|
||||||
|
.context("the runner evidence store is undecodable and could not be set aside")?;
|
||||||
|
keep_private(&aside)?;
|
||||||
|
Ok(replaced)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How the writer reads the store's bytes; tests inject an I/O error here.
|
||||||
|
type ReadStore<'a> = &'a dyn Fn(&Path) -> std::io::Result<Vec<u8>>;
|
||||||
|
|
||||||
|
fn unreadable_fault(error: &std::io::Error) -> anyhow::Error {
|
||||||
|
anyhow::anyhow!("the runner evidence store could not be read: {}", error.kind())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The one writer. Upsert by transaction id; past `RUNNER_EVIDENCE_RECORDS` the
|
||||||
|
/// oldest records go in the same exchange. The file ends at mode 0600.
|
||||||
|
///
|
||||||
|
/// A store is set aside only when bytes that were read do not decode. An I/O
|
||||||
|
/// error leaves the store in place, keeps no evidence for that failure, and
|
||||||
|
/// the error names its kind for the journal.
|
||||||
|
pub(crate) fn record(path: &Path, record: &RunnerFailureRecord) -> Result<Kept> {
|
||||||
|
record_reading(path, record, &|path| std::fs::read(path))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn record_reading(path: &Path, record: &RunnerFailureRecord, read: ReadStore) -> Result<Kept> {
|
||||||
|
let first = match read(path) {
|
||||||
|
Ok(bytes) => Some(bytes),
|
||||||
|
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
|
||||||
|
Err(error) => return Err(unreadable_fault(&error)),
|
||||||
|
};
|
||||||
|
let (envelopes, records, how) = match snapshot(path) {
|
||||||
|
Ok((envelopes, records)) => (envelopes, records, Kept::Stored),
|
||||||
|
Err(_) => {
|
||||||
|
let Some(first) = first else {
|
||||||
|
bail!("the runner evidence store could not be decoded");
|
||||||
|
};
|
||||||
|
// Undecodable only if the same bytes are still there to read: a
|
||||||
|
// second read that fails, or finds other bytes, is not a verdict.
|
||||||
|
match read(path) {
|
||||||
|
Ok(second) if second == first => {}
|
||||||
|
Ok(_) => bail!("the runner evidence store changed while it was being read"),
|
||||||
|
Err(error) => return Err(unreadable_fault(&error)),
|
||||||
|
}
|
||||||
|
let replaced = set_aside_unreadable(path)?;
|
||||||
|
(Vec::new(), Vec::new(), Kept::SetAside { replaced })
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let mut kept: Vec<(u64, CultCacheEnvelope)> = envelopes
|
||||||
|
.iter()
|
||||||
|
.zip(&records)
|
||||||
|
.filter(|(_, stored)| stored.transaction_id != record.transaction_id)
|
||||||
|
.map(|(envelope, stored)| (stored.recorded_at_unix_millis, envelope.clone()))
|
||||||
|
.collect();
|
||||||
|
kept.push((record.recorded_at_unix_millis, record.envelope()?));
|
||||||
|
// Stable: records of one instant keep their stored order, so the record
|
||||||
|
// being written survives a tie.
|
||||||
|
kept.sort_by_key(|(recorded_at, _)| *recorded_at);
|
||||||
|
let excess = kept.len().saturating_sub(RUNNER_EVIDENCE_RECORDS);
|
||||||
|
let replacements: Vec<_> = kept
|
||||||
|
.into_iter()
|
||||||
|
.skip(excess)
|
||||||
|
.map(|(_, envelope)| envelope)
|
||||||
|
.collect();
|
||||||
|
match exchange_behind_private_lock(path, &envelopes, &replacements)? {
|
||||||
|
TryCompareExchangeSnapshotOutcome::Exchanged => {}
|
||||||
|
TryCompareExchangeSnapshotOutcome::Mismatch => {
|
||||||
|
bail!("the runner evidence store changed while it was being written")
|
||||||
|
}
|
||||||
|
TryCompareExchangeSnapshotOutcome::LockContended => {
|
||||||
|
return Err(LockContended {
|
||||||
|
store: "runner evidence store",
|
||||||
|
}
|
||||||
|
.into());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
keep_private(path)?;
|
||||||
|
Ok(how)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(unix)]
|
||||||
|
fn keep_private(path: &Path) -> Result<()> {
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600))
|
||||||
|
.context("making the runner evidence store private")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(not(unix))]
|
||||||
|
fn keep_private(_path: &Path) -> Result<()> {
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The one reader. An absent store holds nothing.
|
||||||
|
pub(crate) fn for_transaction(
|
||||||
|
path: &Path,
|
||||||
|
transaction_id: &str,
|
||||||
|
) -> Result<Option<RunnerFailureRecord>> {
|
||||||
|
Ok(snapshot(path)?
|
||||||
|
.1
|
||||||
|
.into_iter()
|
||||||
|
.find(|stored| stored.transaction_id == transaction_id))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The record as `idunn status --command` shows it, as raw lines. They are
|
||||||
|
/// not text yet: `show_status` is the only caller, and `emit` the only
|
||||||
|
/// writer.
|
||||||
|
fn render(record: &RunnerFailureRecord) -> Vec<Vec<u8>> {
|
||||||
|
let noun = match record.subject_kind {
|
||||||
|
SubjectKind::Step => "step",
|
||||||
|
SubjectKind::ExternalInput => "external input",
|
||||||
|
};
|
||||||
|
let mut lines: Vec<Vec<u8>> = vec![
|
||||||
|
format!(" failed {noun} {}", record.subject_id).into_bytes(),
|
||||||
|
format!(" runner {}", record.runner).into_bytes(),
|
||||||
|
format!(" status {}", runner_status(record.exit_code, record.signal)).into_bytes(),
|
||||||
|
];
|
||||||
|
for (stream, tail, total) in [
|
||||||
|
("stdout", &record.stdout_tail, record.stdout_total_bytes),
|
||||||
|
("stderr", &record.stderr_tail, record.stderr_total_bytes),
|
||||||
|
] {
|
||||||
|
lines.push(format!(" {stream} (last {} of {total} bytes)", tail.len()).into_bytes());
|
||||||
|
let text = tail.strip_suffix(b"\n").unwrap_or(tail);
|
||||||
|
if !text.is_empty() {
|
||||||
|
lines.extend(text.split(|byte| *byte == b'\n').map(|line| {
|
||||||
|
let mut shown = b" | ".to_vec();
|
||||||
|
shown.extend_from_slice(line);
|
||||||
|
shown
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
lines
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What `idunn status --command` prints for a transaction's runner evidence:
|
||||||
|
/// its record, or one line naming why the store is unreadable. Nothing is
|
||||||
|
/// written when the transaction has no record.
|
||||||
|
pub(crate) fn show_status(
|
||||||
|
out: &mut impl std::io::Write,
|
||||||
|
store: &Path,
|
||||||
|
transaction_id: &str,
|
||||||
|
) -> std::io::Result<()> {
|
||||||
|
let lines = match for_transaction(store, transaction_id) {
|
||||||
|
Ok(Some(record)) => render(&record),
|
||||||
|
Ok(None) => return Ok(()),
|
||||||
|
Err(error) => {
|
||||||
|
vec![format!(" runner evidence unreadable: {}", error.root_cause()).into_bytes()]
|
||||||
|
}
|
||||||
|
};
|
||||||
|
emit(out, &lines)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The one sink: every byte the status read prints from the evidence store,
|
||||||
|
/// or from an error about it, is written here and nowhere else. An allowlist:
|
||||||
|
/// printable ASCII and tab pass; every other code point is a visible
|
||||||
|
/// `\u{..}` escape and every invalid UTF-8 byte a `\x..` escape. A newline
|
||||||
|
/// inside a line is escaped too: this function writes the only ones.
|
||||||
|
pub(crate) fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
|
||||||
|
for line in lines {
|
||||||
|
out.write_all(escape(line).as_bytes())?;
|
||||||
|
out.write_all(b"\n")?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn escape(bytes: &[u8]) -> String {
|
||||||
|
let mut shown = String::with_capacity(bytes.len());
|
||||||
|
for chunk in bytes.utf8_chunks() {
|
||||||
|
for character in chunk.valid().chars() {
|
||||||
|
match character {
|
||||||
|
' '..='~' | '\t' => shown.push(character),
|
||||||
|
_ => shown.push_str(&format!("\\u{{{:x}}}", u32::from(character))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for byte in chunk.invalid() {
|
||||||
|
shown.push_str(&format!("\\x{byte:02x}"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
shown
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(all(test, unix))]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
|
||||||
|
fn failure(subject: RunnerSubject, stdout: &[u8], stderr: &[u8]) -> RunnerStepFailure {
|
||||||
|
RunnerStepFailure {
|
||||||
|
subject,
|
||||||
|
runner: "rust".into(),
|
||||||
|
exit_code: Some(3),
|
||||||
|
signal: None,
|
||||||
|
stdout_tail: stdout.to_vec(),
|
||||||
|
stdout_total_bytes: stdout.len() as u64,
|
||||||
|
stderr_tail: stderr.to_vec(),
|
||||||
|
stderr_total_bytes: stderr.len() as u64,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What status writes for a record, through the one sink.
|
||||||
|
fn shown(record: &RunnerFailureRecord) -> String {
|
||||||
|
let mut written = Vec::new();
|
||||||
|
emit(&mut written, &render(record)).expect("a Vec takes any write");
|
||||||
|
String::from_utf8(written).expect("the sink writes ASCII only")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn shown_lines(record: &RunnerFailureRecord) -> Vec<String> {
|
||||||
|
shown(record).lines().map(str::to_owned).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn record_at(transaction: &str, now: u64) -> RunnerFailureRecord {
|
||||||
|
RunnerFailureRecord::from_failure(
|
||||||
|
transaction,
|
||||||
|
"up-service",
|
||||||
|
"service",
|
||||||
|
&failure(RunnerSubject::Step("build".into()), b"out\n", b"err\n"),
|
||||||
|
now,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_evidence_store_keeps_the_newest_32() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
// The literal, not the constant: the spec's bound is 32.
|
||||||
|
for index in 0..=32 {
|
||||||
|
record(&path, &record_at(&format!("tx-{index:02}"), 1000 + index as u64))?;
|
||||||
|
}
|
||||||
|
let kept: Vec<_> = snapshot(&path)?.1.into_iter().map(|stored| stored.transaction_id).collect();
|
||||||
|
assert_eq!(kept.len(), 32);
|
||||||
|
assert!(!kept.contains(&"tx-00".to_owned()), "the oldest survived");
|
||||||
|
assert!(kept.contains(&"tx-32".to_owned()), "the newest was evicted");
|
||||||
|
assert!(kept.contains(&"tx-01".to_owned()), "the second oldest was evicted early");
|
||||||
|
// Rewriting a resident transaction replaces it and drops nothing.
|
||||||
|
let mut again = record_at("tx-05", 5000);
|
||||||
|
again.stdout_tail = b"rewritten\n".to_vec();
|
||||||
|
again.stdout_total_bytes = 10;
|
||||||
|
record(&path, &again)?;
|
||||||
|
let kept = snapshot(&path)?.1;
|
||||||
|
assert_eq!(kept.len(), 32);
|
||||||
|
assert!(kept.iter().any(|stored| stored.transaction_id == "tx-01"));
|
||||||
|
let stored = for_transaction(&path, "tx-05")?.context("rewritten record is gone")?;
|
||||||
|
assert_eq!(stored.stdout_tail, b"rewritten\n");
|
||||||
|
assert_eq!(for_transaction(&path, "tx-00")?, None);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_record_round_trips_byte_for_byte() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
let written = RunnerFailureRecord::from_failure(
|
||||||
|
"tx-a",
|
||||||
|
"up-service",
|
||||||
|
"service",
|
||||||
|
&failure(RunnerSubject::ExternalInput("toolchain".into()), &[0, 1, 255, 10], b""),
|
||||||
|
77,
|
||||||
|
);
|
||||||
|
record(&path, &written)?;
|
||||||
|
assert_eq!(for_transaction(&path, "tx-a")?, Some(written));
|
||||||
|
assert_eq!(for_transaction(&path, "tx-b")?, None);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_evidence_store_is_private() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
// Whatever the umask, the mode is the store's own rule, on every write.
|
||||||
|
for index in 0..2 {
|
||||||
|
record(&path, &record_at(&format!("tx-{index}"), 10 + index))?;
|
||||||
|
assert_eq!(std::fs::metadata(&path)?.permissions().mode() & 0o777, 0o600);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_tail_over_its_bound_or_its_stream_is_refused() {
|
||||||
|
let mut oversize = record_at("tx-a", 1);
|
||||||
|
oversize.stdout_tail = vec![b'x'; RUNNER_OUTPUT_TAIL_BYTES + 1];
|
||||||
|
oversize.stdout_total_bytes = u64::MAX;
|
||||||
|
assert!(oversize.validate().is_err());
|
||||||
|
let mut impossible = record_at("tx-a", 1);
|
||||||
|
impossible.stderr_total_bytes = 1;
|
||||||
|
assert!(impossible.validate().is_err());
|
||||||
|
let mut exact = record_at("tx-a", 1);
|
||||||
|
exact.stdout_tail = vec![b'x'; RUNNER_OUTPUT_TAIL_BYTES];
|
||||||
|
exact.stdout_total_bytes = RUNNER_OUTPUT_TAIL_BYTES as u64;
|
||||||
|
assert!(exact.validate().is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn status_renders_the_tail_with_control_bytes_escaped() {
|
||||||
|
let mut stored = record_at("tx-a", 1);
|
||||||
|
stored.stdout_tail = b"first\n\x1b[2Jsecond\x00end\tTab\nlast\n".to_vec();
|
||||||
|
stored.stdout_total_bytes = 5000;
|
||||||
|
stored.stderr_tail = b"bad\r\x07".to_vec();
|
||||||
|
stored.stderr_total_bytes = 6;
|
||||||
|
let lines = shown_lines(&stored);
|
||||||
|
let text = lines.join("\n");
|
||||||
|
assert!(!text.contains('\u{1b}') && !text.contains('\0') && !text.contains('\r'));
|
||||||
|
assert!(!text.contains('\u{7}'));
|
||||||
|
assert_eq!(
|
||||||
|
lines,
|
||||||
|
vec![
|
||||||
|
" failed step build",
|
||||||
|
" runner rust",
|
||||||
|
" status exit status 3",
|
||||||
|
" stdout (last 30 of 5000 bytes)",
|
||||||
|
" | first",
|
||||||
|
" | \\u{1b}[2Jsecond\\u{0}end\tTab",
|
||||||
|
" | last",
|
||||||
|
" stderr (last 5 of 6 bytes)",
|
||||||
|
" | bad\\u{d}\\u{7}",
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn debug_names_the_record_and_prints_no_tail() {
|
||||||
|
let mut stored = record_at("tx-a", 1);
|
||||||
|
stored.stdout_tail = b"SECRET-OUTPUT".to_vec();
|
||||||
|
stored.stdout_total_bytes = 13;
|
||||||
|
let debug = format!("{stored:?}");
|
||||||
|
assert!(debug.contains("tx-a") && !debug.contains("SECRET"), "{debug}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_tail_renders_only_its_header() {
|
||||||
|
let mut stored = record_at("tx-a", 1);
|
||||||
|
stored.stdout_tail = Vec::new();
|
||||||
|
stored.stdout_total_bytes = 0;
|
||||||
|
let lines = shown_lines(&stored);
|
||||||
|
assert!(lines.contains(&" stdout (last 0 of 0 bytes)".to_owned()));
|
||||||
|
assert!(!lines.iter().any(|line| line == " | "));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_reader_names_an_unreadable_store_as_an_error() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
std::fs::write(&path, b"\xc1 not a store")?;
|
||||||
|
assert!(for_transaction(&path, "tx-a").is_err());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn files_beside(path: &Path) -> Result<Vec<String>> {
|
||||||
|
let mut names: Vec<String> = std::fs::read_dir(path.parent().context("no parent")?)?
|
||||||
|
.map(|entry| Ok(entry?.file_name().to_string_lossy().into_owned()))
|
||||||
|
.collect::<Result<_>>()?;
|
||||||
|
names.retain(|name| name.starts_with("runner-evidence") && !name.contains("lock"));
|
||||||
|
names.sort();
|
||||||
|
Ok(names)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A store a newer Idunn wrote: well-formed, but a schema this one does not know.
|
||||||
|
fn write_future_store(path: &Path) -> Result<()> {
|
||||||
|
let mut future = record_at("tx-future", 1);
|
||||||
|
future.schema_version = "idunn.runner_failure.v2".into();
|
||||||
|
let envelope = typed_envelope(
|
||||||
|
&future.transaction_id,
|
||||||
|
RunnerFailureRecord::TYPE,
|
||||||
|
"idunn.runner_failure.v2",
|
||||||
|
&future,
|
||||||
|
1,
|
||||||
|
)?;
|
||||||
|
let outcome = exchange_behind_private_lock(path, &[], &[envelope])?;
|
||||||
|
ensure!(
|
||||||
|
matches!(outcome, TryCompareExchangeSnapshotOutcome::Exchanged),
|
||||||
|
"the future store was not written"
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_corrupt_store_is_set_aside_and_the_next_failure_keeps_its_evidence() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
let damage = b"\xc1 not a store \x00 damage";
|
||||||
|
std::fs::write(&path, damage)?;
|
||||||
|
let kept = record(&path, &record_at("tx-a", 5))?;
|
||||||
|
assert_eq!(kept, Kept::SetAside { replaced: None });
|
||||||
|
// Kept, not deleted: the damage is where the journal says, root-only.
|
||||||
|
let aside = unreadable_store_path(&path);
|
||||||
|
assert_eq!(std::fs::read(&aside)?, damage);
|
||||||
|
assert_eq!(std::fs::metadata(&aside)?.permissions().mode() & 0o777, 0o600);
|
||||||
|
// The failure's evidence is in a fresh, private store.
|
||||||
|
assert_eq!(
|
||||||
|
for_transaction(&path, "tx-a")?.map(|stored| stored.transaction_id),
|
||||||
|
Some("tx-a".to_owned())
|
||||||
|
);
|
||||||
|
assert_eq!(std::fs::metadata(&path)?.permissions().mode() & 0o777, 0o600);
|
||||||
|
// A healthy store is not set aside again.
|
||||||
|
assert_eq!(record(&path, &record_at("tx-b", 6))?, Kept::Stored);
|
||||||
|
assert_eq!(std::fs::read(&aside)?, damage);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_future_version_store_is_set_aside_and_the_next_failure_keeps_its_evidence() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
write_future_store(&path)?;
|
||||||
|
let before = std::fs::read(&path)?;
|
||||||
|
assert!(for_transaction(&path, "tx-future").is_err());
|
||||||
|
assert_eq!(
|
||||||
|
record(&path, &record_at("tx-a", 5))?,
|
||||||
|
Kept::SetAside { replaced: None }
|
||||||
|
);
|
||||||
|
assert_eq!(std::fs::read(unreadable_store_path(&path))?, before);
|
||||||
|
assert!(for_transaction(&path, "tx-a")?.is_some());
|
||||||
|
assert_eq!(for_transaction(&path, "tx-future")?, None);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn at_most_one_unreadable_store_is_kept() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
let damages: [&[u8]; 3] = [b"\xc1 first", b"\xc1 second", b"\xc1 third"];
|
||||||
|
for (index, damage) in damages.into_iter().enumerate() {
|
||||||
|
std::fs::write(&path, damage)?;
|
||||||
|
record(&path, &record_at(&format!("tx-{index}"), 5 + index as u64))?;
|
||||||
|
assert_eq!(
|
||||||
|
std::fs::read(unreadable_store_path(&path))?,
|
||||||
|
damage,
|
||||||
|
"the newest damage is the one kept"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assert_eq!(
|
||||||
|
files_beside(&path)?,
|
||||||
|
vec![
|
||||||
|
"runner-evidence.cc".to_owned(),
|
||||||
|
"runner-evidence.unreadable.cc".to_owned()
|
||||||
|
]
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_store_that_cannot_be_read_is_left_in_place_and_the_write_fails() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
// A directory where the store should be: unreadable, not undecodable.
|
||||||
|
std::fs::create_dir(&path)?;
|
||||||
|
assert!(record(&path, &record_at("tx-a", 1)).is_err());
|
||||||
|
assert!(path.is_dir());
|
||||||
|
assert!(!unreadable_store_path(&path).exists());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_code_point_but_printable_ascii_is_a_visible_escape() {
|
||||||
|
// The allowlist, exhaustively: no class list decides anything.
|
||||||
|
for code in 0..=0x10ffffu32 {
|
||||||
|
let Some(character) = char::from_u32(code) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let text = character.to_string();
|
||||||
|
let shown = escape(text.as_bytes());
|
||||||
|
if matches!(character, ' '..='~' | '\t') {
|
||||||
|
assert_eq!(shown, text, "U+{code:04X} should pass");
|
||||||
|
} else {
|
||||||
|
assert_eq!(shown, format!("\\u{{{code:x}}}"), "U+{code:04X}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Invalid UTF-8 is escaped byte by byte; valid text round it passes.
|
||||||
|
assert_eq!(escape(b"a\xc1\xffb\xe2\x82"), "a\\xc1\\xffb\\xe2\\x82");
|
||||||
|
assert_eq!(escape(b"ok \xe2\x82\xac"), "ok \\u{20ac}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_newline_inside_a_line_cannot_forge_a_line() {
|
||||||
|
let mut stored = record_at("tx-a", 1);
|
||||||
|
stored.subject_id = "build\n status exit status 0".into();
|
||||||
|
let lines = shown_lines(&stored);
|
||||||
|
assert_eq!(lines[0], " failed step build\\u{a} status exit status 0");
|
||||||
|
assert_eq!(
|
||||||
|
lines.iter().filter(|line| line.starts_with(" status ")).count(),
|
||||||
|
1
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A store whose first record is well-formed except that its subject kind
|
||||||
|
/// holds `marker`: the decode error quotes it.
|
||||||
|
fn store_with_hostile_subject_kind(path: &Path, marker: &[u8; 4]) -> Result<()> {
|
||||||
|
record(path, &record_at("tx-a", 1))?;
|
||||||
|
let mut bytes = std::fs::read(path)?;
|
||||||
|
let at = bytes
|
||||||
|
.windows(4)
|
||||||
|
.position(|window| window == b"step")
|
||||||
|
.context("no step marker")?;
|
||||||
|
bytes[at..at + 4].copy_from_slice(marker);
|
||||||
|
std::fs::write(path, bytes)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_unreadable_store_line_is_escaped_with_the_rest() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
store_with_hostile_subject_kind(&path, b"\x1b[2J")?;
|
||||||
|
let mut written = Vec::new();
|
||||||
|
show_status(&mut written, &path, "tx-a")?;
|
||||||
|
assert!(!written.contains(&0x1b), "an escape byte reached the terminal");
|
||||||
|
assert!(written.iter().all(|byte| matches!(byte, b' '..=b'~' | b'\t' | b'\n')));
|
||||||
|
let text = String::from_utf8(written)?;
|
||||||
|
assert!(text.starts_with(" runner evidence unreadable: "), "{text}");
|
||||||
|
assert!(text.contains("\\u{1b}[2J"), "the quoted value was not shown escaped: {text}");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_failed_set_aside_names_itself_keeps_the_store_and_echoes_nothing() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
store_with_hostile_subject_kind(&path, b"CNRY")?;
|
||||||
|
let damage = std::fs::read(&path)?;
|
||||||
|
// Sanity: this damage decodes to an error that quotes the canary.
|
||||||
|
assert!(format!("{:#}", snapshot(&path).unwrap_err()).contains("CNRY"));
|
||||||
|
// A non-empty directory where the aside goes: rename fails even as root.
|
||||||
|
let aside = unreadable_store_path(&path);
|
||||||
|
std::fs::create_dir(&aside)?;
|
||||||
|
std::fs::write(aside.join("occupied"), b"x")?;
|
||||||
|
let error = record(&path, &record_at("tx-b", 2)).unwrap_err();
|
||||||
|
for text in [format!("{error:#}"), format!("{error:?}")] {
|
||||||
|
assert!(text.contains("could not be set aside"), "{text}");
|
||||||
|
assert!(!text.contains("CNRY"), "stored content reached the error: {text}");
|
||||||
|
}
|
||||||
|
// The failure cost nothing: the store is as it was, with no evidence
|
||||||
|
// written over it.
|
||||||
|
assert_eq!(std::fs::read(&path)?, damage);
|
||||||
|
assert!(aside.join("occupied").exists());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_io_error_leaves_the_store_in_place_and_keeps_no_evidence() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
record(&path, &record_at("tx-a", 1))?;
|
||||||
|
let before = std::fs::read(&path)?;
|
||||||
|
let denied = || std::io::Error::from(std::io::ErrorKind::PermissionDenied);
|
||||||
|
// The first read fails on a healthy store.
|
||||||
|
let error = record_reading(&path, &record_at("tx-b", 2), &|_| Err(denied())).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains("permission denied"), "{error:#}");
|
||||||
|
assert_eq!(std::fs::read(&path)?, before, "the store changed");
|
||||||
|
assert!(!unreadable_store_path(&path).exists());
|
||||||
|
assert!(for_transaction(&path, "tx-b")?.is_none());
|
||||||
|
// The second read, the one that confirms the bytes do not decode, fails.
|
||||||
|
std::fs::write(&path, b"\xc1 not a store")?;
|
||||||
|
let calls = std::cell::Cell::new(0usize);
|
||||||
|
let read = |path: &Path| {
|
||||||
|
calls.set(calls.get() + 1);
|
||||||
|
if calls.get() == 2 {
|
||||||
|
return Err(denied());
|
||||||
|
}
|
||||||
|
std::fs::read(path)
|
||||||
|
};
|
||||||
|
let error = record_reading(&path, &record_at("tx-b", 2), &read).unwrap_err();
|
||||||
|
assert!(format!("{error:#}").contains("permission denied"), "{error:#}");
|
||||||
|
assert_eq!(std::fs::read(&path)?, b"\xc1 not a store");
|
||||||
|
assert!(!unreadable_store_path(&path).exists());
|
||||||
|
// Bytes that changed between the reads are no verdict either.
|
||||||
|
let calls = std::cell::Cell::new(0usize);
|
||||||
|
let read = |path: &Path| -> std::io::Result<Vec<u8>> {
|
||||||
|
calls.set(calls.get() + 1);
|
||||||
|
let mut bytes = std::fs::read(path)?;
|
||||||
|
bytes.push(calls.get() as u8);
|
||||||
|
Ok(bytes)
|
||||||
|
};
|
||||||
|
assert!(record_reading(&path, &record_at("tx-b", 2), &read).is_err());
|
||||||
|
assert_eq!(std::fs::read(&path)?, b"\xc1 not a store");
|
||||||
|
assert!(!unreadable_store_path(&path).exists());
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_second_set_aside_says_it_replaced_the_first() -> Result<()> {
|
||||||
|
let temp = tempfile::tempdir()?;
|
||||||
|
let path = runner_evidence_store_path(&temp.path().join("control.cc"));
|
||||||
|
std::fs::write(&path, b"\xc1 first")?;
|
||||||
|
assert_eq!(
|
||||||
|
record(&path, &record_at("tx-a", 1))?,
|
||||||
|
Kept::SetAside { replaced: None }
|
||||||
|
);
|
||||||
|
std::fs::write(&path, b"\xc1 second")?;
|
||||||
|
let Kept::SetAside { replaced: Some(_) } = record(&path, &record_at("tx-b", 2))? else {
|
||||||
|
bail!("a displaced set-aside store went unreported");
|
||||||
|
};
|
||||||
|
assert_eq!(std::fs::read(unreadable_store_path(&path))?, b"\xc1 second");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
4086
src/steward_orders.rs
Normal file
4086
src/steward_orders.rs
Normal file
File diff suppressed because it is too large
Load diff
16
tests/fixtures/README.md
vendored
16
tests/fixtures/README.md
vendored
|
|
@ -10,3 +10,19 @@ It is checked in because the TypeScript runtime emits a camelCase MessagePack
|
||||||
map while Rust `DatabaseEntry` derives tuple payloads. Idunn must prove it can
|
map while Rust `DatabaseEntry` derives tuple payloads. Idunn must prove it can
|
||||||
decode the producer's real wire shape rather than round-tripping a fixture it
|
decode the producer's real wire shape rather than round-tripping a fixture it
|
||||||
serialized itself.
|
serialized itself.
|
||||||
|
|
||||||
|
# systemd 257 `systemctl show` captures
|
||||||
|
|
||||||
|
`systemd-257/*.show` are the verbatim output of systemd 257.13 for the 27
|
||||||
|
properties `show_unit` requested at `112a7a5`, captured read-only on Yggdrasil
|
||||||
|
on 2026-10-24: `vili-presence-identity-only.show` (a failed idunn-vili unit whose
|
||||||
|
only credential is the presence identity) and
|
||||||
|
`odin-presence-and-one-service-credential.show` (the live Odin unit). systemd 257
|
||||||
|
prints `LoadCredential=[unprintable]` in both. Tests read these bytes; do not edit.
|
||||||
|
|
||||||
|
# Pre-cut systemd observation records
|
||||||
|
|
||||||
|
`pre-cut-systemd-observation.{compact,named}.hex` are the two MessagePack
|
||||||
|
encodings of a `SystemdWorkloadObservation` with `load_credential` set to
|
||||||
|
`[unprintable]`, produced by the code at `112a7a5` (the form live incumbents
|
||||||
|
persisted). They pin that later Idunns still decode such records.
|
||||||
|
|
|
||||||
1
tests/fixtures/pre-cut-systemd-observation.compact.hex
vendored
Normal file
1
tests/fixtures/pre-cut-systemd-observation.compact.hex
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
dc0033d94e6964756e6e2d616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161612e73657276696365aa4964756e6e2074657374a3696e7601a465786563a26e6fa56d69786564c3a6753631303030a6753631303030a0a0a0c3c3a9696e76697369626c65a3616c6cc3a430303037a0ad5b756e7072696e7461626c655dcd10920194cdee48cdee48cdee48cdee4894cdee48cdee48cdee48cdee4891cdee480000000000c391010708b52f6f70742f746573742f62696e2f736572766963650101d9477368613235362d31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131d9477368613235362d61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161a92f6f70742f74657374a92f72756e2f74657374d9477368613235362d3232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323290d9477368613235362d33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333d9202f73797374656d2e736c6963652f6964756e6e2d746573742e73657276696365c090aa61637469766174696f6edc0020010101010101010101010101010101010101010101010101010101010101010190
|
||||||
1
tests/fixtures/pre-cut-systemd-observation.named.hex
vendored
Normal file
1
tests/fixtures/pre-cut-systemd-observation.named.hex
vendored
Normal file
|
|
@ -0,0 +1 @@
|
||||||
|
de0033a4756e6974d94e6964756e6e2d616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161612e73657276696365b0756e69745f6465736372697074696f6eaa4964756e6e2074657374ad696e766f636174696f6e5f6964a3696e76d923657865635f6d61696e5f73746172745f74696d657374616d705f6d6f6e6f746f6e696301ac736572766963655f74797065a465786563ae726573746172745f706f6c696379a26e6fa96b696c6c5f6d6f6465a56d69786564ac64796e616d69635f75736572c3ac73797374656d645f75736572a6753631303030ad73797374656d645f67726f7570a6753631303030b4737570706c656d656e746172795f67726f757073a0b76361706162696c6974795f626f756e64696e675f736574a0b4616d6269656e745f6361706162696c6974696573a0ae707269766174655f6d6f756e7473c3ac707269766174655f70696473c3ac70726f746563745f70726f63a9696e76697369626c65ab70726f635f737562736574a3616c6cb16e6f5f6e65775f70726976696c65676573c3a5756d61736ba430303037b2696e61636365737369626c655f7061746873a0af6c6f61645f63726564656e7469616cad5b756e7072696e7461626c655da86d61696e5f706964cd1092b270726f636573735f73746172745f74696d6501ac70726f636573735f7569647394cdee48cdee48cdee48cdee48ac70726f636573735f6769647394cdee48cdee48cdee48cdee48ae70726f636573735f67726f75707391cdee48b770726f636573735f6361705f696e686572697461626c6500b570726f636573735f6361705f7065726d697474656400b570726f636573735f6361705f65666665637469766500b470726f636573735f6361705f626f756e64696e6700b370726f636573735f6361705f616d6269656e7400b970726f636573735f6e6f5f6e65775f70726976696c65676573c3b670726f636573735f6e616d6573706163655f706964739101b26d6f756e745f6e616d6573706163655f696407b07069645f6e616d6573706163655f696408aa65786563757461626c65b52f6f70742f746573742f62696e2f73657276696365b165786563757461626c655f64657669636501b065786563757461626c655f696e6f646501b165786563757461626c655f736861323536d9477368613235362d31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131b372756e74696d655f696e7374616e63655f6964d9477368613235362d61616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616161b1776f726b696e675f6469726563746f7279a92f6f70742f74657374ae72756e74696d655f62756e646c65a92f72756e2f74657374b3636f6d6d616e645f6c696e655f736861323536d9477368613235362d32323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232323232b1656e7669726f6e6d656e745f6e616d657390bb656e7669726f6e6d656e745f636f6e74726163745f736861323536d9477368613235362d33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333ad636f6e74726f6c5f67726f7570d9202f73797374656d2e736c6963652f6964756e6e2d746573742e73657276696365b563726564656e7469616c735f6469726563746f7279c0bc706172656e745f6f6e6c795f66696c655f64657363726970746f727390bd61637469766174696f6e5f7369676e65725f6964656e746974795f6964aa61637469766174696f6ebc61637469766174696f6e5f7369676e65725f7075626c69635f6b6579dc00200101010101010101010101010101010101010101010101010101010101010101b3736572766963655f63726564656e7469616c7390
|
||||||
28
tests/fixtures/systemd-257/odin-presence-and-one-service-credential.show
vendored
Normal file
28
tests/fixtures/systemd-257/odin-presence-and-one-service-credential.show
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
Type=exec
|
||||||
|
Restart=no
|
||||||
|
MainPID=2571
|
||||||
|
OpenFile=/run/idunn/activation-credentials/sha256-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8.credential:gamecult-idunn-runtime-activation-key:read-only
|
||||||
|
OpenFile=/etc/gamecult/odin/runtime-presence-identity.cc:gamecult-runtime-presence-identity:read-only
|
||||||
|
ExecMainStartTimestampMonotonic=20630000
|
||||||
|
ControlGroup=/system.slice/idunn-odin-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8.service
|
||||||
|
UMask=0007
|
||||||
|
WorkingDirectory=/srv/odin/releases/sha256-e5c7d0b2b9c0a8ffba368aeb08abdbe37ae0d17484a785e60a66ec4a62b2f18c
|
||||||
|
CapabilityBoundingSet=
|
||||||
|
AmbientCapabilities=
|
||||||
|
User=_duc23369f1b8b167db
|
||||||
|
Group=odin
|
||||||
|
DynamicUser=yes
|
||||||
|
LoadCredential=[unprintable]
|
||||||
|
SupplementaryGroups=
|
||||||
|
InaccessiblePaths=/run/idunn/activation-credentials
|
||||||
|
PrivateMounts=yes
|
||||||
|
PrivatePIDs=yes
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
ProtectProc=invisible
|
||||||
|
ProcSubset=all
|
||||||
|
KillMode=mixed
|
||||||
|
Description=Idunn odin sha256-7cfe5cd1e87839e029c7425eb1b586b6b0d94dd79d7376eed3b156386f2c7af8
|
||||||
|
LoadState=loaded
|
||||||
|
ActiveState=active
|
||||||
|
SubState=running
|
||||||
|
InvocationID=ada48921043449f39dacc87b14a1bf70
|
||||||
28
tests/fixtures/systemd-257/vili-presence-identity-only.show
vendored
Normal file
28
tests/fixtures/systemd-257/vili-presence-identity-only.show
vendored
Normal file
|
|
@ -0,0 +1,28 @@
|
||||||
|
Type=exec
|
||||||
|
Restart=no
|
||||||
|
MainPID=0
|
||||||
|
OpenFile=/run/idunn/activation-credentials/sha256-b507e1705a70f8eee72f355e7db637acab8eb37880b5d25245fda68d80f63041.credential:gamecult-idunn-runtime-activation-key:read-only
|
||||||
|
OpenFile=/etc/gamecult/vili/runtime-presence-identity.cc:gamecult-runtime-presence-identity:read-only
|
||||||
|
ExecMainStartTimestampMonotonic=520083420000
|
||||||
|
ControlGroup=
|
||||||
|
UMask=0007
|
||||||
|
WorkingDirectory=/srv/vili-idunn/releases/sha256-aeaee4e1669742572c45a3f26bc86893c44b2a4149520975eb7d8a28aa931906
|
||||||
|
CapabilityBoundingSet=
|
||||||
|
AmbientCapabilities=
|
||||||
|
User=_du8c06c563aac6d205
|
||||||
|
Group=vili-state
|
||||||
|
DynamicUser=yes
|
||||||
|
LoadCredential=[unprintable]
|
||||||
|
SupplementaryGroups=
|
||||||
|
InaccessiblePaths=/run/idunn/activation-credentials
|
||||||
|
PrivateMounts=yes
|
||||||
|
PrivatePIDs=yes
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
ProtectProc=invisible
|
||||||
|
ProcSubset=all
|
||||||
|
KillMode=mixed
|
||||||
|
Description=Idunn vili sha256-b507e1705a70f8eee72f355e7db637acab8eb37880b5d25245fda68d80f63041
|
||||||
|
LoadState=loaded
|
||||||
|
ActiveState=failed
|
||||||
|
SubState=failed
|
||||||
|
InvocationID=8f7a349f2f8f49fe87ac5d89239260b4
|
||||||
Loading…
Reference in a new issue