Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
Meta Vens
64d63cb417 Merge steward orders 1 (r3): an order is one systemd job; refusal reads the named unit only
Soul cut-steward-orders-1.s5 holds at 77e2295. Not installed: steward-orders-2 installs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:57:39 +01:00
Meta Vens
77e2295e97 Steward orders: refuse Upholds and ConsistsOf units, pin the rule's gaps, README names the residuals
A unit with Upholds= is refused for start and restart (systemd would start the
upheld unit in a full transaction). A wrapper with ConsistsOf= is refused for
stop, restart and reload with a typed Refused::Wrapper naming the first member,
so it cannot report ran while touching nothing. Tests pin empty BoundBy, a
single show per refusal, no show for a refused name and each posted reason.
README: the one-job rule's residual full transactions. Removes the unused
BTreeMap import and the never_loop in the host actuator test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 23:32:30 +01:00
Meta Vens
fb1e7b5dd9 Steward orders: README states the one-job rule; pin the end-to-end argv
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:45:08 +01:00
Meta Vens
8ebe0208a6 Steward orders: pin the one-job rule on the real unit set and a real user manager
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:41:56 +01:00
Meta Vens
a859914b0b Steward orders: run every unit verb as one job and check only the named unit
Delete the transaction model (Edge, EDGES, action_target, Transaction, the
early-boot rule). Every unit verb runs systemctl <verb>
--job-mode=ignore-requirements -- <unit>; refusal_for reads one systemctl show
of the ordered unit: its names, the host-ending names, what fires when it
ends, and who is bound to it. Targets are no longer orderable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:40:58 +01:00
Meta Vens
4894ad20df README: an order's transaction follows emergency actions and OnFailure/OnSuccess; stopped boot units are refused
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:52:41 +01:00
Meta Vens
879b76a842 Steward orders: pin every emergency action spelling to its target
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:50:23 +01:00
Meta Vens
631fb7836b Steward orders: pin the new show properties; the unknown-action probe counts runs instead of expecting none
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:29:13 +01:00
Meta Vens
d29bf93be6 Steward orders: refuse a start or stop whose unit ends by emergency action or isolates; boot and recovery units are never started from stopped
refusal_for now reads SuccessAction, FailureAction, StartLimitAction and
JobTimeoutAction (each mapped to the target it runs, so reboot, poweroff,
halt, kexec, soft-reboot and exit, with -force and -immediate) and OnFailure
and OnSuccess as units a touched unit fires, followed through what they pull
in and what those fire. An action it does not know fails the order closed.
A stopped DefaultDependencies=no unit in the transaction is refused (Via::
EarlyBoot): rescue.service ends by isolating default.target, which no edge
describes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:26:29 +01:00
Meta Vens
d9710c988a Steward orders: the ledger reads an order only under its own schema too
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 06:35:43 +01:00
Meta Vens
caacbb4bba Steward orders: the end-to-end test waits 10 s for its posts
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 06:35:31 +01:00
Meta Vens
9653d270e5 Steward orders: pin the ledger schema guards, invocation ids and post backoff; tests fail fast instead of waiting
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 06:16:04 +01:00
Meta Vens
8a1beb5c87 README: an order is gated by the steward bound at execution; the socket is the steward's; root reads
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 05:35:30 +01:00
Meta Vens
7e15bf1a7b Steward orders: the Synapse deadline test fails instead of hanging when the deadline is gone
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 05:33:23 +01:00
Meta Vens
572f9e6aa4 Steward orders: the end-to-end test keeps serve's umask until it has read control.cc
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 05:31:34 +01:00
Meta Vens
56edad95c1 Steward orders: one steward gate for fresh and recovered orders; the orders unit leaks no group or umask; root reads; kernel-attested issuer; end-to-end serve test
execute refuses, with a record and a post, an order whose principal or issuing
uid is not the steward bound now, before grammar, scope and brake; the brake
is asked for the bound principal only. idunn-orders.service drops Group= and
UMask=: bind_socket gives the socket to the steward uid at 0600 and serve sets
Idunn's own umask, so a deploy order writes control.cc 0640. Root may show and
list, never order. The issuer unit is the first unit under system.slice, never
a name the peer chose below it. Synapse waits for a reply within one deadline.
due_posts takes the deploy-state reader so a finished deploy's post is tested.
Tests: the gate both ways, root, per-order posts, deploy result, intake and
Synapse timeouts, parse_serve, user_uid, socket and store modes, and the whole
service as root with a fake systemctl, systemd-run and homeserver.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 05:03:17 +01:00
Meta Vens
ca439a29ff Pin the systemctl show property list and unquoted backslashes
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 04:17:12 +01:00
Meta Vens
27ca875194 Steward orders tests: clear four clippy hits in the module
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 03:53:41 +01:00
Meta Vens
be3b95bc21 Import BTreeMap for the transaction facts cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 03:50:20 +01:00
Meta Vens
cea17c06a7 README: a unit order is refused by the systemd transaction; a package order installs only what it names
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 03:49:36 +01:00
Meta Vens
ec6222e2a1 Steward unit orders: refuse by the systemd transaction, install exactly the named packages
An order is refused when the closure systemd would build for it reaches an
always-refused unit: a start pulls Requires/Requisite/BindsTo/Wants/Upholds and
stops Conflicts/ConflictedBy, a stop takes down RequiredBy/BoundBy/RequisiteOf/
ConsistsOf/PropagatesStopTo/StopPropagatedFrom, a restart does both. A refused
unit already running is left alone by a start. Names are unquoted from
systemctl show; a unit systemd cannot show fails the order.

Package installs run apt-get --no-remove with ?exact-name(<name>) per package
and a final dpkg -s step that fails unless every named package is installed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 03:49:26 +01:00
Meta Vens
7d5cd85728 Reword a test comment so the no-brake-release tripwire reads clean
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 02:33:12 +01:00
Meta Vens
1a3634e046 Document idunn order and add the idunn-orders unit
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 02:33:02 +01:00
Meta Vens
e50a721dbc Steward orders: typed package, unit and deploy orders from one steward, in a sibling root unit
idunn orders serve takes orders over a root-only socket from the steward uid, records
order/start/outcome/post in its own orders.cc, and posts acceptance and result to
#operator as a retried projection. Unit orders may act on any unit except the compiled
always-refused set (idunn-*, mind-self@*, mind-persona@*), matched on a unit's Id and every
name, and for stop and restart on the units systemd stops with it. A refusal names a field
and a code, never the value. A started order is never run twice. Deploy orders commit
through the same primitive as idunn up.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-01 02:33:01 +01:00
Meta Vens
024f2d7a09 Workload: incarnation units are collected when they end, so a failed candidate no longer stays loaded and marks the host degraded
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:54:26 +01:00
Meta Vens
8400d0712f Merge idunn/runner-failure-evidence: a failed runner keeps its evidence
Cut idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1,
reports h1-h3, Soul s1-s3. The docker runner kept only stderr, so a step
reporting on stdout failed with an empty message (Vili's node --test). A
failed step's name and bounded stdout/stderr tails go to a root-only typed
store read by idunn status; errors and the journal carry no output; one
allowlist escape at the status sink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 08:32:18 +01:00
Meta Vens
78086705a0 deploy-provenance map: runner failure evidence (body facts R1-R8)
Imagination for cut runner-failure-evidence: the docker runner discards
stdout, so a step that reports on stdout fails with an empty message; the
tail moves to a root-only typed evidence store read by idunn status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 02:04:37 +01:00
11 changed files with 4421 additions and 21 deletions

1
Cargo.lock generated
View file

@ -482,6 +482,7 @@ dependencies = [
"libc",
"rmp-serde",
"serde",
"serde_json",
"sha2",
"tempfile",
"toml",

View file

@ -38,6 +38,7 @@ serde = { version = "1", features = ["derive"] }
ed25519-dalek = "2.2"
toml = "0.8"
uuid = { version = "1", features = ["v4"] }
serde_json = "1"
[dev-dependencies]
tempfile = "3"

View file

@ -58,6 +58,64 @@ idunn status --command <id>
the bindings, so `idunn up profile:full-gamecult` brings up everything bound
into that profile.
### Steward orders
A second root unit from the same binary, `idunn-orders` (`idunn orders serve`),
takes typed orders from one steward Mind over `/run/idunn-orders/steward.sock`.
It has its own store, `orders.cc`, and its own binding, `steward.toml`; the
survival daemon takes no order. There is still no command string:
```bash
idunn order package-install <pkg>[=version]... --reason TEXT # apt-get update, install named packages, check them
idunn order unit <start|stop|restart|reload> <unit> --reason TEXT
idunn order deploy <service|profile:name> --reason TEXT # the same commit as `idunn up`
idunn order show <id> # status and output tails
idunn order list
```
A unit order submits exactly one job, for the unit it names: every verb runs
`systemctl <verb> --job-mode=ignore-requirements -- <unit>`, so systemd builds a
transaction of one job. It does not start what the unit requires or restart what
is part of it: order the dependency first, and order `postgresql@16-main.service`
rather than the `postgresql.service` wrapper. Targets are not orderable. The job is one; what the unit does afterwards is
not: systemd still runs full transactions of its own for `Restart=` after a
failure, for a `StopWhenUnneeded=` dependency left without a user, and for a
`Conflicts=` partner of a unit that fails, so an ordered stop of a unit
that something restarts does not last, and a start of a `Restart=always`
unit can report failure and be active seconds later. A unit
order may act on any unit except Idunn's own (`idunn-*`) and the minds' own
(`mind-self@*`, `mind-persona@*`), and the units that end or take the host
(`systemd-suspend`, `systemd-hibernate`, `systemd-hybrid-sleep`,
`systemd-suspend-then-hibernate`, `rescue`, `emergency`). That set is compiled
into the binary and read from no configuration. An order is refused, recorded
and posted when the ordered unit, or an alias it resolves from, is one of them;
when it has an emergency action (`SuccessAction=`, `FailureAction=`,
`StartLimitAction=`, `JobTimeoutAction=`) other than `none` or an `OnFailure=`
or `OnSuccess=` unit (not for a reload); or, for a stop or restart, when another
unit is `BoundBy=` it, because systemd stops that unit with it whatever the job
mode; for a start or restart, when it has an `Upholds=` unit, which systemd
starts in a full transaction whatever the job mode; or, for a stop, restart or
reload, when it has a `ConsistsOf=` member, a wrapper such as
`postgresql.service` whose verbs would exit 0 and touch nothing. A unit
`systemctl show` cannot resolve fails the order. Slices, scopes,
targets and globs are not units an order can name.
A package order installs the packages it names and nothing else: apt runs with
`--no-remove`, each name is passed as an exact-name pattern so `.` and `+`
match themselves, and a last step fails the order unless every named package
is installed.
An order runs once, after the steward's uid is checked and the instance's
brake check (`mind-control --instance <principal> check`) exits 0. The check
that the order's principal and uid are the bound steward's runs again at every
execution, so an order accepted before `steward.toml` changed is refused and
recorded, not run. The socket belongs to the steward user alone (mode 0600);
root may `show` and `list` orders but never issue one. An order that had
started when the service died is recorded as lost and never run again. No order releases a brake. Acceptance and result are posted to
`#operator` as a retried projection of `orders.cc`: a chat outage delays the
posts and not the orders. A refusal names the field and a code, never the value
it refused.
## Authority
**Owner.** Deployment admission, and the continuity of installed bodies.

View file

@ -0,0 +1,47 @@
[Unit]
Description=Idunn steward orders: typed package, unit and deploy orders from the steward
# Deliberately no After= or Requires= on any managed target, Synapse or
# idunn-yggdrasil: orders are how an operator repairs those, so none of them
# may be a condition of this unit starting. Posts are a retried projection and
# wait for Synapse on their own.
[Service]
Type=simple
# Root, from the same binary as idunn-yggdrasil, with its own store and socket.
# idunn-yggdrasil keeps its no-imperative-input invariant and its write set:
# no order code runs in `idunn serve`.
#
# The unit name matches idunn-*, which is in the always-refused set, so no
# order can stop or restart this service.
User=root
# No Group= and no UMask=: the service gives its socket to the steward user
# (mode 0600) itself and sets Idunn's own umask for the files it writes, so
# nothing here reaches control.cc. The steward user is the only peer the
# service answers: it checks SO_PEERCRED on every connection.
ExecStart=/usr/local/bin/idunn orders serve \
--store /var/lib/gamecult/idunn-orders \
--state-store /var/lib/gamecult/idunn/control.cc \
--steward-binding /etc/gamecult/idunn/steward.toml \
--socket /run/idunn-orders/steward.sock
Restart=always
RestartSec=5
TimeoutStopSec=20
RuntimeDirectory=idunn-orders
RuntimeDirectoryMode=0755
# The Synapse access token of @idunn, copied from a root-only file, and read
# through $CREDENTIALS_DIRECTORY; the service names it in steward.toml.
LoadCredential=idunn-matrix-token:/etc/gamecult/idunn/matrix-token
PrivateTmp=yes
ProtectHome=yes
ProtectSystem=strict
# Its own order store, and the control store a deploy order commits a command
# to. Transient units started with systemd-run run in the manager's namespace,
# so packages and units are not limited by this list.
ReadWritePaths=/var/lib/gamecult/idunn-orders /var/lib/gamecult/idunn
[Install]
WantedBy=multi-user.target

View file

@ -284,3 +284,111 @@ Idunn's body changes only through gamecult-ops
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
bindings unchanged). After it, Vili's Hands reruns from step 4 of
`motion-pipeline:cut_report:cut-vili-admission.h4`'s undone list.
## Cut `runner-failure-evidence`
Spec `idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1`, at
`804ed75` (origin/main, live on Yggdrasil). It serves the StreamPixels-area
Self: `idunn up vili` (command `up-4866467c-ee98-4a2b-b4a9-a008d7c44acd`,
transaction `tx-a04b68c7-c742-4d7d-8220-8aa08b9febae`, 2026-10-25 00:50-00:55
UTC) failed in Sealing with `Docker runner exited with exit status: 1: ` and
nobody could tell which recipe step failed.
### Body facts (source read at `804ed75`; host read-only, 2026-10-25)
- R1. The runner is `docker run --rm` (`src/drivers.rs:2088-2090`), launched by
`DockerRunnerDriver::docker` (`:2173-2195`) with stdin null and
`Command::output()`: stdout and stderr are each buffered whole in the
daemon's memory, unbounded. On a non-zero exit the error is
`"Docker runner exited with {status}: {stderr}"`; stdout is dropped. The
container is gone the moment it exits, so `docker ps -a` and `docker logs`
hold nothing.
- R2. The empty message: the failing process wrote nothing to stderr, and
stdout, the only stream that carried its report, is discarded at
`:2182-2193`. The same interpolation echoes whatever a step does write to
stderr into `last_error`, `pre_fencing_abort.error` and
`completion.error` (`bounded_detail`, 2048 bytes), and from there into
`idunn up`'s `deployment failed:` line and `idunn status`.
- R3. No step identity reaches the error. `materialize` (`:2368-2391`) loops
over `declaration.steps` and calls `run_in_workspace` with argv only;
`step.id` is used only in the environment precondition. External inputs
(`curl` in the runner, `:2247-2258`) fail through the same helper with the
same anonymous message.
- R4. Which step failed on 2026-10-25, inferred from the failed transaction's
staging root, which survives (926 MiB,
`/var/lib/gamecult/idunn/staging/tx-a04b68c7-.../`): in `.runner-node-build`,
`node_modules/.package-lock.json` was written at 00:51:24 (so
`install-vili-dependencies`, `npm ci`, finished), nothing outside
`node_modules` is newer, and `release/vili` (the next step's output) is
absent; the python workspace holds only fetched inputs. The failing step was
`test-daemon` (`node --test ... scripts/test-vili-daemon.mjs`), whose test
reporter writes to stdout: exactly the stream R1 discards. Inference from
residue, not from output.
- R5. The failure path: `resume_one_transaction` (`src/control_plane.rs:6291-6320`)
sends a pre-fence error to `begin_pre_fencing_abort` (`:9916-9935`), which
persists `bounded_detail(format!("{error:#}"))`. Nothing reaches the journal
on this path: the idunn-yggdrasil journal for 00:45-01:10 holds only
provider-exclusion lines.
- R6. Stores. `control.cc` (root 0640) holds commands, transactions
(`idunn.deployment_transaction.v4`, keys 0-36), generations, supervision and
expiry requests; `ControlSnapshot::read` refuses any other type
(`:3420`). Records must re-encode byte-exactly (`decode_record`,
`:3717-3727`), so a new transaction field is a schema bump and a v4 lift
(`:4100-4110`). Terminal transactions move verbatim to `history.cc`
(`:4366-4396`; 39.8 MB on the host), which status reads for `--command`.
`incidents.cc` lives in the published projection directory and by its own
doc (`src/incident.rs:1-11`) carries no free text and is never `control.cc`,
because a type added there stops continuity for every target on a rollback.
- R7. The daemon runs as root (`User=root`). Runner secrets reach the
container as read-only file mounts named by env (`:2041-2047`), so any text
a step derives from them can land in its output.
- R8. Staging residue: 475 transaction roots, 55 GiB under
`/var/lib/gamecult/idunn/staging`; a failed Sealing leaves its workspaces
because `materialize` returns before `remove_tree_inside`.
### Decision
The failure becomes a typed value at the runner, `RunnerStepFailure`: the
subject (recipe step or external input, by id), the runner id, the exit code or
signal, and the last 8 KiB of stdout and of stderr with each stream's total
byte count. Its `Display` carries the subject and status, never output.
Capture streams both pipes through bounded tails, so a verbose build no longer
buffers whole in the daemon.
The tail lives in a new private store, `runner-evidence.cc`, beside
`control.cc` (mode 0600), one record per transaction
(`idunn.runner_failure.v1`), newest 32 kept by the writer. Not in the
transaction record: that is a v5 bump whose rollback makes every older Idunn
refuse its control store (R6), the scar `incident.rs` already names. Not in
`incidents.cc`: published, and free text is barred there. Not in the journal:
journal readers are a wider set than root, so the journal line names the
transaction, the subject, the status and the store, and no output.
No redaction. Matching secret bytes would make the daemon read every secret,
and still misses any encoding, prefix or transform of one; a claimed redaction
that leaks is worse than a stated rule. The rule instead is the reader set: the
store's only readers (root) can already read every secret the runner was
given.
Keeping the failed container (dropping `--rm`) is worse: its output then lives
in Docker's json-file log, unbounded, untyped, outside Idunn's state and
readable by the docker group; containers accumulate with no owner to reap them;
`docker system prune` erases the evidence; and nothing ties a container to the
transaction in `idunn status`.
### Authority
Owner: the runner decides what failed and keeps the bounded tail; the engine's
pre-fence failure path is the one writer of `runner-evidence.cc`.
`last_error`, `pre_fencing_abort.error` and `completion.error` keep only the
failure's `Display`. Forbidden writers: the `stderr` interpolation in
`DockerRunnerDriver::docker`. Retention is by count inside the writer, so the
store needs no external owner; staging roots (R8) stay with
`ghostlight-verse:follow_up:idunn-generation-retention`.
### Deploy
Idunn's body changes only through gamecult-ops
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
bindings unchanged). The next failed Sealing then shows its step and tail on
`sudo idunn status --command <id>`.

View file

@ -2934,6 +2934,10 @@ enum Command {
recipe: PathBuf,
binding: Option<PathBuf>,
},
#[cfg(unix)]
OrdersServe(crate::steward_orders::ServeOptions),
#[cfg(unix)]
Order(crate::steward_orders::OrderCommand),
}
pub fn run(args: impl Iterator<Item = String>) -> Result<()> {
@ -2973,6 +2977,10 @@ pub fn run(args: impl Iterator<Item = String>) -> Result<()> {
requested_by,
} => expire(&state_store, &command_id, &requested_by),
Command::Validate { recipe, binding } => validate(&recipe, binding.as_deref()),
#[cfg(unix)]
Command::OrdersServe(options) => crate::steward_orders::serve(options),
#[cfg(unix)]
Command::Order(order) => crate::steward_orders::run_order(order),
}
}
@ -3025,6 +3033,10 @@ fn parse(args: impl Iterator<Item = String>) -> Result<Command> {
"cancel" => parse_cancel(args),
"expire" => parse_expire(args),
"validate" => parse_validate(args),
#[cfg(unix)]
"orders" => crate::steward_orders::parse_serve(args).map(Command::OrdersServe),
#[cfg(unix)]
"order" => crate::steward_orders::parse_order(args).map(Command::Order),
"--help" | "-h" | "help" => bail!(usage()),
_ => bail!("unknown Idunn command {command:?}\n\n{}", usage()),
}
@ -4730,20 +4742,21 @@ fn resolve_selector(
Ok(targets)
}
fn submit(
/// The one commit primitive for a deployment command: `idunn up` and a steward
/// deploy order both write theirs here. Returns the commit time.
pub(crate) fn commit_deployment_command(
store_path: &Path,
command_id: &str,
selector: &str,
requested_by: &str,
wait: bool,
timeout_seconds: u64,
) -> Result<()> {
) -> Result<u64> {
if let Some(parent) = store_path.parent() {
fs::create_dir_all(parent)?;
}
let now = now_millis()?;
let command = DeploymentCommand {
schema_version: DEPLOYMENT_COMMAND_SCHEMA.into(),
command_id: format!("up-{}", Uuid::new_v4()),
command_id: command_id.into(),
kind: CommandKind::Deploy,
selector: selector.into(),
requested_by: requested_by.into(),
@ -4761,7 +4774,33 @@ fn submit(
)?,
"deployment command id collided"
);
println!("{}", command.command_id);
Ok(now)
}
/// A command's present state as `idunn status --command` derives it, for the
/// steward order poster: `queued` and `running` are not terminal.
#[cfg(unix)]
pub(crate) fn command_state(
state_store: &Path,
command_id: &str,
) -> Result<(&'static str, String)> {
let snapshot = ControlSnapshot::read(state_store)?;
let archived = read_history_transactions(state_store);
let mut transactions = snapshot.transaction_for_command(command_id);
transactions.extend(archived.iter().filter(|value| value.command_id == command_id));
Ok(derived_command_status(&transactions))
}
fn submit(
store_path: &Path,
selector: &str,
requested_by: &str,
wait: bool,
timeout_seconds: u64,
) -> Result<()> {
let command_id = format!("up-{}", Uuid::new_v4());
let now = commit_deployment_command(store_path, &command_id, selector, requested_by)?;
println!("{command_id}");
if !wait {
return Ok(());
}
@ -4772,10 +4811,10 @@ fn submit(
snapshot
.commands
.iter()
.any(|stored| stored.value.command_id == command.command_id),
.any(|stored| stored.value.command_id == command_id),
"submitted deployment command disappeared"
);
let transactions = snapshot.transaction_for_command(&command.command_id);
let transactions = snapshot.transaction_for_command(&command_id);
if !transactions.is_empty() && transactions.iter().all(|value| value.is_terminal()) {
if let Some(error) =
transactions
@ -4800,7 +4839,7 @@ fn submit(
}
}
fn status(
pub(crate) fn status(
out: &mut impl Write,
store_path: &Path,
incident_store: &Path,
@ -10592,7 +10631,13 @@ fn required<'a, T>(value: &'a Option<T>, label: &str) -> Result<&'a T> {
value.as_ref().with_context(|| format!("{label} is absent"))
}
fn now_millis() -> Result<u64> {
/// The control store `idunn up` and `idunn serve` use unless told otherwise.
#[cfg(unix)]
pub(crate) fn default_state_store() -> PathBuf {
RuntimeOptions::default().state_store
}
pub(crate) fn now_millis() -> Result<u64> {
Ok(SystemTime::now()
.duration_since(UNIX_EPOCH)
.context("system clock predates Unix epoch")?
@ -10612,12 +10657,12 @@ fn sibling_path(path: &Path, suffix: &str) -> PathBuf {
PathBuf::from(value)
}
fn string_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<String> {
pub(crate) fn string_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<String> {
args.next()
.ok_or_else(|| anyhow!("{name} requires a value"))
}
fn path_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<PathBuf> {
pub(crate) fn path_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<PathBuf> {
Ok(PathBuf::from(string_value(args, name)?))
}
@ -10633,7 +10678,7 @@ fn u64_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<u64>
.with_context(|| format!("{name} requires a u64"))
}
fn require_selector(value: &str) -> Result<()> {
pub(crate) fn require_selector(value: &str) -> Result<()> {
let value = value.strip_prefix("profile:").unwrap_or(value);
require_id(value, "deployment selector")
}
@ -10757,7 +10802,10 @@ fn usage() -> &'static str {
idunn status [--state-store PATH] [--incident-store PATH] [--command ID]\n\
idunn cancel <command-id> [--state-store PATH]\n\
idunn expire <command-id> [--state-store PATH]\n\
idunn validate --recipe PATH [--binding PATH]\n\n\
idunn validate --recipe PATH [--binding PATH]\n\
idunn orders serve [--store DIR] [--steward-binding PATH] [--socket PATH]\n\
idunn order <package-install PKG... | unit VERB UNIT | deploy SELECTOR> --reason TEXT\n\
idunn order show ID | idunn order list\n\n\
Recipes describe capability and process requirements. Idunn seals exact\n\
source and artifacts, admits one incarnation, and delegates execution to\n\
systemd and routing mechanics to the configured proxy."
@ -11969,9 +12017,45 @@ mod tests {
vec!["serve", "--swarm-profile", "yggdrasil-local"],
vec!["serve", "--restart-command", "bad"],
vec!["validate-runtime-admission"],
vec!["order", "unit", "restart", "nginx.service", "--reason", "r", "--command", "sh -c bad"],
vec!["order", "unit", "restart", "nginx.service", "--reason", "r", "--shell"],
vec!["order", "run", "sh", "--reason", "r"],
vec!["order", "package-install", "git-lfs", "--reason", "r", "--deploy-command", "x"],
vec!["order", "unit", "restart", "nginx.service", "extra", "--reason", "r"],
vec!["order", "deploy", "vili", "--reason", "r", "--requested-by", "someone"],
vec!["orders", "serve", "--allow-units", "nginx.service"],
vec!["orders", "run"],
] {
assert!(parse(arguments.into_iter().map(str::to_owned)).is_err());
}
// The declarative order forms parse, and only as order commands.
#[cfg(unix)]
for arguments in [
vec!["order", "unit", "restart", "nginx.service", "--reason", "r"],
vec!["order", "package-install", "git-lfs", "--reason", "r"],
vec!["order", "deploy", "vili", "--reason", "r"],
vec!["order", "list"],
vec!["orders", "serve"],
] {
let parsed = parse(arguments.into_iter().map(str::to_owned)).unwrap();
assert!(matches!(parsed, Command::Order(_) | Command::OrdersServe(_)));
}
}
#[test]
fn deployment_commit_primitive_is_shared_and_never_overwrites() -> Result<()> {
let temp = TempDir::new()?;
let store = temp.path().join("control.cc");
commit_deployment_command(&store, "up-one", "vili", "steward:eureka order ord-1")?;
let snapshot = ControlSnapshot::read(&store)?;
assert_eq!(snapshot.commands.len(), 1);
let command = &snapshot.commands[0].value;
assert_eq!(command.command_id, "up-one");
assert_eq!(command.selector, "vili");
assert_eq!(command.requested_by, "steward:eureka order ord-1");
assert!(commit_deployment_command(&store, "up-one", "other", "x").is_err());
assert_eq!(ControlSnapshot::read(&store)?.commands.len(), 1);
Ok(())
}
#[test]

View file

@ -605,7 +605,9 @@ pub trait WorkloadPort {
///
/// Idunn launches candidates as transient units with `Restart=no`, so a
/// unit that has failed, or that systemd no longer knows, will not come
/// back on its own. Past the fence every error is otherwise treated as
/// back on its own. Those units are collected once they fail
/// (`INCARNATION_UNIT_LIFECYCLE`), so a failed candidate usually reads as
/// forgotten rather than failed. Past the fence every error is otherwise treated as
/// resumable and retried forever, which is right while a candidate can
/// still recover and a trap once it cannot.
fn is_permanently_stopped(&self, observation: &WorkloadObservation) -> Result<bool>;
@ -2574,6 +2576,13 @@ impl RunnerPort for DockerRunnerDriver {
}
}
/// How every incarnation unit lives and dies. It runs once (`Restart=no`)
/// and systemd forgets it once it is inactive or failed (`--collect`): the
/// outcome is Idunn's to record in its own store, and a failed transient unit
/// left loaded is only clutter that marks the host degraded.
const INCARNATION_UNIT_LIFECYCLE: [&str; 3] =
["--collect", "--property=Type=exec", "--property=Restart=no"];
/// systemd owns process execution. This driver lowers one validated launch
/// contract into a transient unit and then proves the native process and
/// executable that systemd actually started. It does not decide admission,
@ -3764,8 +3773,9 @@ impl SystemdTransientWorkloadDriver {
OsString::from("--no-ask-password"),
OsString::from("--expand-environment=no"),
OsString::from(format!("--unit={unit}")),
OsString::from("--property=Type=exec"),
OsString::from("--property=Restart=no"),
];
args.extend(INCARNATION_UNIT_LIFECYCLE.map(OsString::from));
args.extend([
OsString::from("--property=KillMode=mixed"),
OsString::from("--property=DynamicUser=yes"),
OsString::from("--property=SupplementaryGroups="),
@ -3801,7 +3811,7 @@ impl SystemdTransientWorkloadDriver {
OsString::from(format!("--working-directory={}", installed.display())),
OsString::from(format!("--property=ReadOnlyPaths={}", installed.display())),
OsString::from(format!("--property=ReadOnlyPaths={}", bundle.display())),
];
]);
for open_file in parent_only_open_files {
args.push(OsString::from(format!("--property=OpenFile={open_file}")));
}
@ -9942,6 +9952,16 @@ Content-Le".to_vec()), |_| {}),
Ok(())
}
/// An incarnation unit runs once and is collected when it ends, so a
/// failed candidate does not stay loaded and mark the host degraded.
/// Collection is safe because a forgotten unit already reads as
/// permanently stopped (the test below).
#[test]
fn incarnation_units_run_once_and_are_collected() {
assert!(INCARNATION_UNIT_LIFECYCLE.contains(&"--collect"));
assert!(INCARNATION_UNIT_LIFECYCLE.contains(&"--property=Restart=no"));
}
/// Admission audit, claim 3: `is_permanently_stopped` now lets the
/// Isolation gate skip `prove_isolation` against the incumbent. Enumerate
/// what it answers `true` for, using a fake `systemctl show` that replays

View file

@ -1151,7 +1151,7 @@ mod tests {
let answerer = thread::spawn(move || -> Result<()> {
for _ in 0..500 {
client.poll_resends()?;
while let Some(frame) = client.receive_once()? {
if let Some(frame) = client.receive_once()? {
let envelope = HostActuatorEnvelope::decode(&frame.payload)?;
let request = envelope.open_request("raven", &idunn_anchor)?;
assert_eq!(envelope.session_nonce, nonce);

View file

@ -9,3 +9,5 @@ pub mod launcher;
mod incident;
mod runner_evidence;
pub mod provisioning;
#[cfg(unix)]
mod steward_orders;

View file

@ -366,7 +366,7 @@ pub(crate) fn show_status(
/// printable ASCII and tab pass; every other code point is a visible
/// `\u{..}` escape and every invalid UTF-8 byte a `\x..` escape. A newline
/// inside a line is escaped too: this function writes the only ones.
fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
pub(crate) fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
for line in lines {
out.write_all(escape(line).as_bytes())?;
out.write_all(b"\n")?;
@ -374,7 +374,7 @@ fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()>
Ok(())
}
fn escape(bytes: &[u8]) -> String {
pub(crate) fn escape(bytes: &[u8]) -> String {
let mut shown = String::with_capacity(bytes.len());
for chunk in bytes.utf8_chunks() {
for character in chunk.valid().chars() {

4079
src/steward_orders.rs Normal file

File diff suppressed because it is too large Load diff