Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
64d63cb417 | ||
|
|
77e2295e97 | ||
|
|
fb1e7b5dd9 | ||
|
|
8ebe0208a6 | ||
|
|
a859914b0b | ||
|
|
4894ad20df | ||
|
|
879b76a842 | ||
|
|
631fb7836b | ||
|
|
d29bf93be6 | ||
|
|
d9710c988a | ||
|
|
caacbb4bba | ||
|
|
9653d270e5 | ||
|
|
8a1beb5c87 | ||
|
|
7e15bf1a7b | ||
|
|
572f9e6aa4 | ||
|
|
56edad95c1 | ||
|
|
ca439a29ff | ||
|
|
27ca875194 | ||
|
|
be3b95bc21 | ||
|
|
cea17c06a7 | ||
|
|
ec6222e2a1 | ||
|
|
7d5cd85728 | ||
|
|
1a3634e046 | ||
|
|
e50a721dbc | ||
|
|
024f2d7a09 | ||
|
|
8400d0712f | ||
|
|
78086705a0 |
11 changed files with 4421 additions and 21 deletions
1
Cargo.lock
generated
1
Cargo.lock
generated
|
|
@ -482,6 +482,7 @@ dependencies = [
|
|||
"libc",
|
||||
"rmp-serde",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"tempfile",
|
||||
"toml",
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ serde = { version = "1", features = ["derive"] }
|
|||
ed25519-dalek = "2.2"
|
||||
toml = "0.8"
|
||||
uuid = { version = "1", features = ["v4"] }
|
||||
serde_json = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
|
|
|
|||
58
README.md
58
README.md
|
|
@ -58,6 +58,64 @@ idunn status --command <id>
|
|||
the bindings, so `idunn up profile:full-gamecult` brings up everything bound
|
||||
into that profile.
|
||||
|
||||
### Steward orders
|
||||
|
||||
A second root unit from the same binary, `idunn-orders` (`idunn orders serve`),
|
||||
takes typed orders from one steward Mind over `/run/idunn-orders/steward.sock`.
|
||||
It has its own store, `orders.cc`, and its own binding, `steward.toml`; the
|
||||
survival daemon takes no order. There is still no command string:
|
||||
|
||||
```bash
|
||||
idunn order package-install <pkg>[=version]... --reason TEXT # apt-get update, install named packages, check them
|
||||
idunn order unit <start|stop|restart|reload> <unit> --reason TEXT
|
||||
idunn order deploy <service|profile:name> --reason TEXT # the same commit as `idunn up`
|
||||
idunn order show <id> # status and output tails
|
||||
idunn order list
|
||||
```
|
||||
|
||||
A unit order submits exactly one job, for the unit it names: every verb runs
|
||||
`systemctl <verb> --job-mode=ignore-requirements -- <unit>`, so systemd builds a
|
||||
transaction of one job. It does not start what the unit requires or restart what
|
||||
is part of it: order the dependency first, and order `postgresql@16-main.service`
|
||||
rather than the `postgresql.service` wrapper. Targets are not orderable. The job is one; what the unit does afterwards is
|
||||
not: systemd still runs full transactions of its own for `Restart=` after a
|
||||
failure, for a `StopWhenUnneeded=` dependency left without a user, and for a
|
||||
`Conflicts=` partner of a unit that fails, so an ordered stop of a unit
|
||||
that something restarts does not last, and a start of a `Restart=always`
|
||||
unit can report failure and be active seconds later. A unit
|
||||
order may act on any unit except Idunn's own (`idunn-*`) and the minds' own
|
||||
(`mind-self@*`, `mind-persona@*`), and the units that end or take the host
|
||||
(`systemd-suspend`, `systemd-hibernate`, `systemd-hybrid-sleep`,
|
||||
`systemd-suspend-then-hibernate`, `rescue`, `emergency`). That set is compiled
|
||||
into the binary and read from no configuration. An order is refused, recorded
|
||||
and posted when the ordered unit, or an alias it resolves from, is one of them;
|
||||
when it has an emergency action (`SuccessAction=`, `FailureAction=`,
|
||||
`StartLimitAction=`, `JobTimeoutAction=`) other than `none` or an `OnFailure=`
|
||||
or `OnSuccess=` unit (not for a reload); or, for a stop or restart, when another
|
||||
unit is `BoundBy=` it, because systemd stops that unit with it whatever the job
|
||||
mode; for a start or restart, when it has an `Upholds=` unit, which systemd
|
||||
starts in a full transaction whatever the job mode; or, for a stop, restart or
|
||||
reload, when it has a `ConsistsOf=` member, a wrapper such as
|
||||
`postgresql.service` whose verbs would exit 0 and touch nothing. A unit
|
||||
`systemctl show` cannot resolve fails the order. Slices, scopes,
|
||||
targets and globs are not units an order can name.
|
||||
|
||||
A package order installs the packages it names and nothing else: apt runs with
|
||||
`--no-remove`, each name is passed as an exact-name pattern so `.` and `+`
|
||||
match themselves, and a last step fails the order unless every named package
|
||||
is installed.
|
||||
|
||||
An order runs once, after the steward's uid is checked and the instance's
|
||||
brake check (`mind-control --instance <principal> check`) exits 0. The check
|
||||
that the order's principal and uid are the bound steward's runs again at every
|
||||
execution, so an order accepted before `steward.toml` changed is refused and
|
||||
recorded, not run. The socket belongs to the steward user alone (mode 0600);
|
||||
root may `show` and `list` orders but never issue one. An order that had
|
||||
started when the service died is recorded as lost and never run again. No order releases a brake. Acceptance and result are posted to
|
||||
`#operator` as a retried projection of `orders.cc`: a chat outage delays the
|
||||
posts and not the orders. A refusal names the field and a code, never the value
|
||||
it refused.
|
||||
|
||||
## Authority
|
||||
|
||||
**Owner.** Deployment admission, and the continuity of installed bodies.
|
||||
|
|
|
|||
47
deploy/idunn-orders.service
Normal file
47
deploy/idunn-orders.service
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
[Unit]
|
||||
Description=Idunn steward orders: typed package, unit and deploy orders from the steward
|
||||
# Deliberately no After= or Requires= on any managed target, Synapse or
|
||||
# idunn-yggdrasil: orders are how an operator repairs those, so none of them
|
||||
# may be a condition of this unit starting. Posts are a retried projection and
|
||||
# wait for Synapse on their own.
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
|
||||
# Root, from the same binary as idunn-yggdrasil, with its own store and socket.
|
||||
# idunn-yggdrasil keeps its no-imperative-input invariant and its write set:
|
||||
# no order code runs in `idunn serve`.
|
||||
#
|
||||
# The unit name matches idunn-*, which is in the always-refused set, so no
|
||||
# order can stop or restart this service.
|
||||
User=root
|
||||
# No Group= and no UMask=: the service gives its socket to the steward user
|
||||
# (mode 0600) itself and sets Idunn's own umask for the files it writes, so
|
||||
# nothing here reaches control.cc. The steward user is the only peer the
|
||||
# service answers: it checks SO_PEERCRED on every connection.
|
||||
ExecStart=/usr/local/bin/idunn orders serve \
|
||||
--store /var/lib/gamecult/idunn-orders \
|
||||
--state-store /var/lib/gamecult/idunn/control.cc \
|
||||
--steward-binding /etc/gamecult/idunn/steward.toml \
|
||||
--socket /run/idunn-orders/steward.sock
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStopSec=20
|
||||
|
||||
RuntimeDirectory=idunn-orders
|
||||
RuntimeDirectoryMode=0755
|
||||
|
||||
# The Synapse access token of @idunn, copied from a root-only file, and read
|
||||
# through $CREDENTIALS_DIRECTORY; the service names it in steward.toml.
|
||||
LoadCredential=idunn-matrix-token:/etc/gamecult/idunn/matrix-token
|
||||
|
||||
PrivateTmp=yes
|
||||
ProtectHome=yes
|
||||
ProtectSystem=strict
|
||||
# Its own order store, and the control store a deploy order commits a command
|
||||
# to. Transient units started with systemd-run run in the manager's namespace,
|
||||
# so packages and units are not limited by this list.
|
||||
ReadWritePaths=/var/lib/gamecult/idunn-orders /var/lib/gamecult/idunn
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -284,3 +284,111 @@ Idunn's body changes only through gamecult-ops
|
|||
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||
bindings unchanged). After it, Vili's Hands reruns from step 4 of
|
||||
`motion-pipeline:cut_report:cut-vili-admission.h4`'s undone list.
|
||||
|
||||
## Cut `runner-failure-evidence`
|
||||
|
||||
Spec `idunn-deploy-provenance:cut_spec:cut-runner-failure-evidence.r1`, at
|
||||
`804ed75` (origin/main, live on Yggdrasil). It serves the StreamPixels-area
|
||||
Self: `idunn up vili` (command `up-4866467c-ee98-4a2b-b4a9-a008d7c44acd`,
|
||||
transaction `tx-a04b68c7-c742-4d7d-8220-8aa08b9febae`, 2026-10-25 00:50-00:55
|
||||
UTC) failed in Sealing with `Docker runner exited with exit status: 1: ` and
|
||||
nobody could tell which recipe step failed.
|
||||
|
||||
### Body facts (source read at `804ed75`; host read-only, 2026-10-25)
|
||||
|
||||
- R1. The runner is `docker run --rm` (`src/drivers.rs:2088-2090`), launched by
|
||||
`DockerRunnerDriver::docker` (`:2173-2195`) with stdin null and
|
||||
`Command::output()`: stdout and stderr are each buffered whole in the
|
||||
daemon's memory, unbounded. On a non-zero exit the error is
|
||||
`"Docker runner exited with {status}: {stderr}"`; stdout is dropped. The
|
||||
container is gone the moment it exits, so `docker ps -a` and `docker logs`
|
||||
hold nothing.
|
||||
- R2. The empty message: the failing process wrote nothing to stderr, and
|
||||
stdout, the only stream that carried its report, is discarded at
|
||||
`:2182-2193`. The same interpolation echoes whatever a step does write to
|
||||
stderr into `last_error`, `pre_fencing_abort.error` and
|
||||
`completion.error` (`bounded_detail`, 2048 bytes), and from there into
|
||||
`idunn up`'s `deployment failed:` line and `idunn status`.
|
||||
- R3. No step identity reaches the error. `materialize` (`:2368-2391`) loops
|
||||
over `declaration.steps` and calls `run_in_workspace` with argv only;
|
||||
`step.id` is used only in the environment precondition. External inputs
|
||||
(`curl` in the runner, `:2247-2258`) fail through the same helper with the
|
||||
same anonymous message.
|
||||
- R4. Which step failed on 2026-10-25, inferred from the failed transaction's
|
||||
staging root, which survives (926 MiB,
|
||||
`/var/lib/gamecult/idunn/staging/tx-a04b68c7-.../`): in `.runner-node-build`,
|
||||
`node_modules/.package-lock.json` was written at 00:51:24 (so
|
||||
`install-vili-dependencies`, `npm ci`, finished), nothing outside
|
||||
`node_modules` is newer, and `release/vili` (the next step's output) is
|
||||
absent; the python workspace holds only fetched inputs. The failing step was
|
||||
`test-daemon` (`node --test ... scripts/test-vili-daemon.mjs`), whose test
|
||||
reporter writes to stdout: exactly the stream R1 discards. Inference from
|
||||
residue, not from output.
|
||||
- R5. The failure path: `resume_one_transaction` (`src/control_plane.rs:6291-6320`)
|
||||
sends a pre-fence error to `begin_pre_fencing_abort` (`:9916-9935`), which
|
||||
persists `bounded_detail(format!("{error:#}"))`. Nothing reaches the journal
|
||||
on this path: the idunn-yggdrasil journal for 00:45-01:10 holds only
|
||||
provider-exclusion lines.
|
||||
- R6. Stores. `control.cc` (root 0640) holds commands, transactions
|
||||
(`idunn.deployment_transaction.v4`, keys 0-36), generations, supervision and
|
||||
expiry requests; `ControlSnapshot::read` refuses any other type
|
||||
(`:3420`). Records must re-encode byte-exactly (`decode_record`,
|
||||
`:3717-3727`), so a new transaction field is a schema bump and a v4 lift
|
||||
(`:4100-4110`). Terminal transactions move verbatim to `history.cc`
|
||||
(`:4366-4396`; 39.8 MB on the host), which status reads for `--command`.
|
||||
`incidents.cc` lives in the published projection directory and by its own
|
||||
doc (`src/incident.rs:1-11`) carries no free text and is never `control.cc`,
|
||||
because a type added there stops continuity for every target on a rollback.
|
||||
- R7. The daemon runs as root (`User=root`). Runner secrets reach the
|
||||
container as read-only file mounts named by env (`:2041-2047`), so any text
|
||||
a step derives from them can land in its output.
|
||||
- R8. Staging residue: 475 transaction roots, 55 GiB under
|
||||
`/var/lib/gamecult/idunn/staging`; a failed Sealing leaves its workspaces
|
||||
because `materialize` returns before `remove_tree_inside`.
|
||||
|
||||
### Decision
|
||||
|
||||
The failure becomes a typed value at the runner, `RunnerStepFailure`: the
|
||||
subject (recipe step or external input, by id), the runner id, the exit code or
|
||||
signal, and the last 8 KiB of stdout and of stderr with each stream's total
|
||||
byte count. Its `Display` carries the subject and status, never output.
|
||||
Capture streams both pipes through bounded tails, so a verbose build no longer
|
||||
buffers whole in the daemon.
|
||||
|
||||
The tail lives in a new private store, `runner-evidence.cc`, beside
|
||||
`control.cc` (mode 0600), one record per transaction
|
||||
(`idunn.runner_failure.v1`), newest 32 kept by the writer. Not in the
|
||||
transaction record: that is a v5 bump whose rollback makes every older Idunn
|
||||
refuse its control store (R6), the scar `incident.rs` already names. Not in
|
||||
`incidents.cc`: published, and free text is barred there. Not in the journal:
|
||||
journal readers are a wider set than root, so the journal line names the
|
||||
transaction, the subject, the status and the store, and no output.
|
||||
|
||||
No redaction. Matching secret bytes would make the daemon read every secret,
|
||||
and still misses any encoding, prefix or transform of one; a claimed redaction
|
||||
that leaks is worse than a stated rule. The rule instead is the reader set: the
|
||||
store's only readers (root) can already read every secret the runner was
|
||||
given.
|
||||
|
||||
Keeping the failed container (dropping `--rm`) is worse: its output then lives
|
||||
in Docker's json-file log, unbounded, untyped, outside Idunn's state and
|
||||
readable by the docker group; containers accumulate with no owner to reap them;
|
||||
`docker system prune` erases the evidence; and nothing ties a container to the
|
||||
transaction in `idunn status`.
|
||||
|
||||
### Authority
|
||||
|
||||
Owner: the runner decides what failed and keeps the bounded tail; the engine's
|
||||
pre-fence failure path is the one writer of `runner-evidence.cc`.
|
||||
`last_error`, `pre_fencing_abort.error` and `completion.error` keep only the
|
||||
failure's `Display`. Forbidden writers: the `stderr` interpolation in
|
||||
`DockerRunnerDriver::docker`. Retention is by count inside the writer, so the
|
||||
store needs no external owner; staging roots (R8) stay with
|
||||
`ghostlight-verse:follow_up:idunn-generation-retention`.
|
||||
|
||||
### Deploy
|
||||
|
||||
Idunn's body changes only through gamecult-ops
|
||||
`scripts/install-idunn-yggdrasil-release.sh` (stage `idunn` only; unit and
|
||||
bindings unchanged). The next failed Sealing then shows its step and tail on
|
||||
`sudo idunn status --command <id>`.
|
||||
|
|
|
|||
|
|
@ -2934,6 +2934,10 @@ enum Command {
|
|||
recipe: PathBuf,
|
||||
binding: Option<PathBuf>,
|
||||
},
|
||||
#[cfg(unix)]
|
||||
OrdersServe(crate::steward_orders::ServeOptions),
|
||||
#[cfg(unix)]
|
||||
Order(crate::steward_orders::OrderCommand),
|
||||
}
|
||||
|
||||
pub fn run(args: impl Iterator<Item = String>) -> Result<()> {
|
||||
|
|
@ -2973,6 +2977,10 @@ pub fn run(args: impl Iterator<Item = String>) -> Result<()> {
|
|||
requested_by,
|
||||
} => expire(&state_store, &command_id, &requested_by),
|
||||
Command::Validate { recipe, binding } => validate(&recipe, binding.as_deref()),
|
||||
#[cfg(unix)]
|
||||
Command::OrdersServe(options) => crate::steward_orders::serve(options),
|
||||
#[cfg(unix)]
|
||||
Command::Order(order) => crate::steward_orders::run_order(order),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -3025,6 +3033,10 @@ fn parse(args: impl Iterator<Item = String>) -> Result<Command> {
|
|||
"cancel" => parse_cancel(args),
|
||||
"expire" => parse_expire(args),
|
||||
"validate" => parse_validate(args),
|
||||
#[cfg(unix)]
|
||||
"orders" => crate::steward_orders::parse_serve(args).map(Command::OrdersServe),
|
||||
#[cfg(unix)]
|
||||
"order" => crate::steward_orders::parse_order(args).map(Command::Order),
|
||||
"--help" | "-h" | "help" => bail!(usage()),
|
||||
_ => bail!("unknown Idunn command {command:?}\n\n{}", usage()),
|
||||
}
|
||||
|
|
@ -4730,20 +4742,21 @@ fn resolve_selector(
|
|||
Ok(targets)
|
||||
}
|
||||
|
||||
fn submit(
|
||||
/// The one commit primitive for a deployment command: `idunn up` and a steward
|
||||
/// deploy order both write theirs here. Returns the commit time.
|
||||
pub(crate) fn commit_deployment_command(
|
||||
store_path: &Path,
|
||||
command_id: &str,
|
||||
selector: &str,
|
||||
requested_by: &str,
|
||||
wait: bool,
|
||||
timeout_seconds: u64,
|
||||
) -> Result<()> {
|
||||
) -> Result<u64> {
|
||||
if let Some(parent) = store_path.parent() {
|
||||
fs::create_dir_all(parent)?;
|
||||
}
|
||||
let now = now_millis()?;
|
||||
let command = DeploymentCommand {
|
||||
schema_version: DEPLOYMENT_COMMAND_SCHEMA.into(),
|
||||
command_id: format!("up-{}", Uuid::new_v4()),
|
||||
command_id: command_id.into(),
|
||||
kind: CommandKind::Deploy,
|
||||
selector: selector.into(),
|
||||
requested_by: requested_by.into(),
|
||||
|
|
@ -4761,7 +4774,33 @@ fn submit(
|
|||
)?,
|
||||
"deployment command id collided"
|
||||
);
|
||||
println!("{}", command.command_id);
|
||||
Ok(now)
|
||||
}
|
||||
|
||||
/// A command's present state as `idunn status --command` derives it, for the
|
||||
/// steward order poster: `queued` and `running` are not terminal.
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn command_state(
|
||||
state_store: &Path,
|
||||
command_id: &str,
|
||||
) -> Result<(&'static str, String)> {
|
||||
let snapshot = ControlSnapshot::read(state_store)?;
|
||||
let archived = read_history_transactions(state_store);
|
||||
let mut transactions = snapshot.transaction_for_command(command_id);
|
||||
transactions.extend(archived.iter().filter(|value| value.command_id == command_id));
|
||||
Ok(derived_command_status(&transactions))
|
||||
}
|
||||
|
||||
fn submit(
|
||||
store_path: &Path,
|
||||
selector: &str,
|
||||
requested_by: &str,
|
||||
wait: bool,
|
||||
timeout_seconds: u64,
|
||||
) -> Result<()> {
|
||||
let command_id = format!("up-{}", Uuid::new_v4());
|
||||
let now = commit_deployment_command(store_path, &command_id, selector, requested_by)?;
|
||||
println!("{command_id}");
|
||||
if !wait {
|
||||
return Ok(());
|
||||
}
|
||||
|
|
@ -4772,10 +4811,10 @@ fn submit(
|
|||
snapshot
|
||||
.commands
|
||||
.iter()
|
||||
.any(|stored| stored.value.command_id == command.command_id),
|
||||
.any(|stored| stored.value.command_id == command_id),
|
||||
"submitted deployment command disappeared"
|
||||
);
|
||||
let transactions = snapshot.transaction_for_command(&command.command_id);
|
||||
let transactions = snapshot.transaction_for_command(&command_id);
|
||||
if !transactions.is_empty() && transactions.iter().all(|value| value.is_terminal()) {
|
||||
if let Some(error) =
|
||||
transactions
|
||||
|
|
@ -4800,7 +4839,7 @@ fn submit(
|
|||
}
|
||||
}
|
||||
|
||||
fn status(
|
||||
pub(crate) fn status(
|
||||
out: &mut impl Write,
|
||||
store_path: &Path,
|
||||
incident_store: &Path,
|
||||
|
|
@ -10592,7 +10631,13 @@ fn required<'a, T>(value: &'a Option<T>, label: &str) -> Result<&'a T> {
|
|||
value.as_ref().with_context(|| format!("{label} is absent"))
|
||||
}
|
||||
|
||||
fn now_millis() -> Result<u64> {
|
||||
/// The control store `idunn up` and `idunn serve` use unless told otherwise.
|
||||
#[cfg(unix)]
|
||||
pub(crate) fn default_state_store() -> PathBuf {
|
||||
RuntimeOptions::default().state_store
|
||||
}
|
||||
|
||||
pub(crate) fn now_millis() -> Result<u64> {
|
||||
Ok(SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.context("system clock predates Unix epoch")?
|
||||
|
|
@ -10612,12 +10657,12 @@ fn sibling_path(path: &Path, suffix: &str) -> PathBuf {
|
|||
PathBuf::from(value)
|
||||
}
|
||||
|
||||
fn string_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<String> {
|
||||
pub(crate) fn string_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<String> {
|
||||
args.next()
|
||||
.ok_or_else(|| anyhow!("{name} requires a value"))
|
||||
}
|
||||
|
||||
fn path_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<PathBuf> {
|
||||
pub(crate) fn path_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<PathBuf> {
|
||||
Ok(PathBuf::from(string_value(args, name)?))
|
||||
}
|
||||
|
||||
|
|
@ -10633,7 +10678,7 @@ fn u64_value(args: &mut impl Iterator<Item = String>, name: &str) -> Result<u64>
|
|||
.with_context(|| format!("{name} requires a u64"))
|
||||
}
|
||||
|
||||
fn require_selector(value: &str) -> Result<()> {
|
||||
pub(crate) fn require_selector(value: &str) -> Result<()> {
|
||||
let value = value.strip_prefix("profile:").unwrap_or(value);
|
||||
require_id(value, "deployment selector")
|
||||
}
|
||||
|
|
@ -10757,7 +10802,10 @@ fn usage() -> &'static str {
|
|||
idunn status [--state-store PATH] [--incident-store PATH] [--command ID]\n\
|
||||
idunn cancel <command-id> [--state-store PATH]\n\
|
||||
idunn expire <command-id> [--state-store PATH]\n\
|
||||
idunn validate --recipe PATH [--binding PATH]\n\n\
|
||||
idunn validate --recipe PATH [--binding PATH]\n\
|
||||
idunn orders serve [--store DIR] [--steward-binding PATH] [--socket PATH]\n\
|
||||
idunn order <package-install PKG... | unit VERB UNIT | deploy SELECTOR> --reason TEXT\n\
|
||||
idunn order show ID | idunn order list\n\n\
|
||||
Recipes describe capability and process requirements. Idunn seals exact\n\
|
||||
source and artifacts, admits one incarnation, and delegates execution to\n\
|
||||
systemd and routing mechanics to the configured proxy."
|
||||
|
|
@ -11969,9 +12017,45 @@ mod tests {
|
|||
vec!["serve", "--swarm-profile", "yggdrasil-local"],
|
||||
vec!["serve", "--restart-command", "bad"],
|
||||
vec!["validate-runtime-admission"],
|
||||
vec!["order", "unit", "restart", "nginx.service", "--reason", "r", "--command", "sh -c bad"],
|
||||
vec!["order", "unit", "restart", "nginx.service", "--reason", "r", "--shell"],
|
||||
vec!["order", "run", "sh", "--reason", "r"],
|
||||
vec!["order", "package-install", "git-lfs", "--reason", "r", "--deploy-command", "x"],
|
||||
vec!["order", "unit", "restart", "nginx.service", "extra", "--reason", "r"],
|
||||
vec!["order", "deploy", "vili", "--reason", "r", "--requested-by", "someone"],
|
||||
vec!["orders", "serve", "--allow-units", "nginx.service"],
|
||||
vec!["orders", "run"],
|
||||
] {
|
||||
assert!(parse(arguments.into_iter().map(str::to_owned)).is_err());
|
||||
}
|
||||
// The declarative order forms parse, and only as order commands.
|
||||
#[cfg(unix)]
|
||||
for arguments in [
|
||||
vec!["order", "unit", "restart", "nginx.service", "--reason", "r"],
|
||||
vec!["order", "package-install", "git-lfs", "--reason", "r"],
|
||||
vec!["order", "deploy", "vili", "--reason", "r"],
|
||||
vec!["order", "list"],
|
||||
vec!["orders", "serve"],
|
||||
] {
|
||||
let parsed = parse(arguments.into_iter().map(str::to_owned)).unwrap();
|
||||
assert!(matches!(parsed, Command::Order(_) | Command::OrdersServe(_)));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deployment_commit_primitive_is_shared_and_never_overwrites() -> Result<()> {
|
||||
let temp = TempDir::new()?;
|
||||
let store = temp.path().join("control.cc");
|
||||
commit_deployment_command(&store, "up-one", "vili", "steward:eureka order ord-1")?;
|
||||
let snapshot = ControlSnapshot::read(&store)?;
|
||||
assert_eq!(snapshot.commands.len(), 1);
|
||||
let command = &snapshot.commands[0].value;
|
||||
assert_eq!(command.command_id, "up-one");
|
||||
assert_eq!(command.selector, "vili");
|
||||
assert_eq!(command.requested_by, "steward:eureka order ord-1");
|
||||
assert!(commit_deployment_command(&store, "up-one", "other", "x").is_err());
|
||||
assert_eq!(ControlSnapshot::read(&store)?.commands.len(), 1);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -605,7 +605,9 @@ pub trait WorkloadPort {
|
|||
///
|
||||
/// Idunn launches candidates as transient units with `Restart=no`, so a
|
||||
/// unit that has failed, or that systemd no longer knows, will not come
|
||||
/// back on its own. Past the fence every error is otherwise treated as
|
||||
/// back on its own. Those units are collected once they fail
|
||||
/// (`INCARNATION_UNIT_LIFECYCLE`), so a failed candidate usually reads as
|
||||
/// forgotten rather than failed. Past the fence every error is otherwise treated as
|
||||
/// resumable and retried forever, which is right while a candidate can
|
||||
/// still recover and a trap once it cannot.
|
||||
fn is_permanently_stopped(&self, observation: &WorkloadObservation) -> Result<bool>;
|
||||
|
|
@ -2574,6 +2576,13 @@ impl RunnerPort for DockerRunnerDriver {
|
|||
}
|
||||
}
|
||||
|
||||
/// How every incarnation unit lives and dies. It runs once (`Restart=no`)
|
||||
/// and systemd forgets it once it is inactive or failed (`--collect`): the
|
||||
/// outcome is Idunn's to record in its own store, and a failed transient unit
|
||||
/// left loaded is only clutter that marks the host degraded.
|
||||
const INCARNATION_UNIT_LIFECYCLE: [&str; 3] =
|
||||
["--collect", "--property=Type=exec", "--property=Restart=no"];
|
||||
|
||||
/// systemd owns process execution. This driver lowers one validated launch
|
||||
/// contract into a transient unit and then proves the native process and
|
||||
/// executable that systemd actually started. It does not decide admission,
|
||||
|
|
@ -3764,8 +3773,9 @@ impl SystemdTransientWorkloadDriver {
|
|||
OsString::from("--no-ask-password"),
|
||||
OsString::from("--expand-environment=no"),
|
||||
OsString::from(format!("--unit={unit}")),
|
||||
OsString::from("--property=Type=exec"),
|
||||
OsString::from("--property=Restart=no"),
|
||||
];
|
||||
args.extend(INCARNATION_UNIT_LIFECYCLE.map(OsString::from));
|
||||
args.extend([
|
||||
OsString::from("--property=KillMode=mixed"),
|
||||
OsString::from("--property=DynamicUser=yes"),
|
||||
OsString::from("--property=SupplementaryGroups="),
|
||||
|
|
@ -3801,7 +3811,7 @@ impl SystemdTransientWorkloadDriver {
|
|||
OsString::from(format!("--working-directory={}", installed.display())),
|
||||
OsString::from(format!("--property=ReadOnlyPaths={}", installed.display())),
|
||||
OsString::from(format!("--property=ReadOnlyPaths={}", bundle.display())),
|
||||
];
|
||||
]);
|
||||
for open_file in parent_only_open_files {
|
||||
args.push(OsString::from(format!("--property=OpenFile={open_file}")));
|
||||
}
|
||||
|
|
@ -9942,6 +9952,16 @@ Content-Le".to_vec()), |_| {}),
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// An incarnation unit runs once and is collected when it ends, so a
|
||||
/// failed candidate does not stay loaded and mark the host degraded.
|
||||
/// Collection is safe because a forgotten unit already reads as
|
||||
/// permanently stopped (the test below).
|
||||
#[test]
|
||||
fn incarnation_units_run_once_and_are_collected() {
|
||||
assert!(INCARNATION_UNIT_LIFECYCLE.contains(&"--collect"));
|
||||
assert!(INCARNATION_UNIT_LIFECYCLE.contains(&"--property=Restart=no"));
|
||||
}
|
||||
|
||||
/// Admission audit, claim 3: `is_permanently_stopped` now lets the
|
||||
/// Isolation gate skip `prove_isolation` against the incumbent. Enumerate
|
||||
/// what it answers `true` for, using a fake `systemctl show` that replays
|
||||
|
|
|
|||
|
|
@ -1151,7 +1151,7 @@ mod tests {
|
|||
let answerer = thread::spawn(move || -> Result<()> {
|
||||
for _ in 0..500 {
|
||||
client.poll_resends()?;
|
||||
while let Some(frame) = client.receive_once()? {
|
||||
if let Some(frame) = client.receive_once()? {
|
||||
let envelope = HostActuatorEnvelope::decode(&frame.payload)?;
|
||||
let request = envelope.open_request("raven", &idunn_anchor)?;
|
||||
assert_eq!(envelope.session_nonce, nonce);
|
||||
|
|
|
|||
|
|
@ -9,3 +9,5 @@ pub mod launcher;
|
|||
mod incident;
|
||||
mod runner_evidence;
|
||||
pub mod provisioning;
|
||||
#[cfg(unix)]
|
||||
mod steward_orders;
|
||||
|
|
|
|||
|
|
@ -366,7 +366,7 @@ pub(crate) fn show_status(
|
|||
/// printable ASCII and tab pass; every other code point is a visible
|
||||
/// `\u{..}` escape and every invalid UTF-8 byte a `\x..` escape. A newline
|
||||
/// inside a line is escaped too: this function writes the only ones.
|
||||
fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
pub(crate) fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
for line in lines {
|
||||
out.write_all(escape(line).as_bytes())?;
|
||||
out.write_all(b"\n")?;
|
||||
|
|
@ -374,7 +374,7 @@ fn emit(out: &mut impl std::io::Write, lines: &[Vec<u8>]) -> std::io::Result<()>
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn escape(bytes: &[u8]) -> String {
|
||||
pub(crate) fn escape(bytes: &[u8]) -> String {
|
||||
let mut shown = String::with_capacity(bytes.len());
|
||||
for chunk in bytes.utf8_chunks() {
|
||||
for character in chunk.valid().chars() {
|
||||
|
|
|
|||
4079
src/steward_orders.rs
Normal file
4079
src/steward_orders.rs
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Reference in a new issue