Commit graph Idunn/src/drivers.rs
Author SHA1 Message Date
Meta Vens
4ac96e959d Route connect test: the listener never accepts, so the test cannot hang
route_connect_retries_connection_refused_until_listener_is_ready blocked in
accept() with no deadline. If the client connected to another listener on the
reused port, accept() waited forever and the whole verify job held its slot
for 39 minutes. The kernel completes a handshake from a listener's backlog,
so the connect succeeds without an accept: the listener now only outlives the
connect, waiting on a channel with a 30 s ceiling. A lost port race fails
the test instead of hanging it.

Builds.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 23:13:52 +02:00
Meta Vens
32cf1f7bf4 Incident writes never wait; exhaustion closes on health and the window; one retention predicate
Every incident write is one nonblocking try_compare_exchange_snapshot of the
whole file against the snapshot it was decided on (as drivers.rs does for the
write lease). A held lock or a lost race is an error and a no-op for the tick,
retried on a later pass; it is also what makes one open incident per
(condition, subject) real. The incident .lock is no longer published: readers
of incidents.cc take no lock, so only Idunn can hold it. History is written the
same way.

An exhaustion incident closes as recovered only when the workload is observed
running and restarts_exhausted is false (ruling exhaustion-close-rule), so a
release that survives one pass after each restart keeps one incident.

Reconcile closes only incidents whose condition ends with admission and whose
subject is not admitted; its dead accumulator is gone. IncidentRecord owns the
retention predicate that retirement and status both use. status writes to a
writer and serve's loop is serve_while, so tests reach both.

Not verified yet: this commit has not been built.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 18:56:33 +02:00
Meta Vens
202e5a2e10 Idunn records continuity exhaustion as a durable incident in incidents.cc
Idunn's daemon, at the continuity decision site, writes one
idunn.operator_incident.v1 record per exhaustion into its own published
incidents.cc (--incident-store), never control.cc. The record opens once per
(target, condition), closes on an observed running workload (recovered) or on
the target leaving the admitted set, and retires to incident-history.cc seven
days after closing. A store failure is traced once and never returned into the
tick. idunn status prints each target's incidents. ReportOnce is demoted to a
stderr trace.

Does not build the fixture test until tests/fixtures/idunn.operator_incident.v1.cc
lands in the next commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 16:21:02 +02:00
Meta Vens
e9c718e073 B2-3d: the route actuation ceiling counts only Forward
Ruling 3d A (2026-09-30): the 12/h per-target route actuation ceiling
counts only Forward actuations (deploys). Survival actuations (continuity
restarts, rollbacks, withdrawals, admitted-route repairs) are recorded for
visibility in a separate TargetSupervision.route_survivals log (key 6,
newest 12 kept) and are never counted against the ceiling or refused by it.
A target whose continuity spent its six restarts can therefore always be
rescued by a deploy.

Each Survival path keeps its own bound: a continuity's actuations the
restart log and its doubling wait, an admitted-route repair the challenge
backoff, a deployment's rollback or withdrawal the Forward change it
undoes. A candidate whose route proof keeps failing now reloads at most
twice the ceiling per window (12 installs, each with its rollback) instead
of 12.

status prints the survival record beside the ceiling line.

TargetSupervision stays at v1 with the added slot: no installed Idunn has
written the record (B2 is merged, not installed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:21:58 +02:00
Meta Vens
cccfe8e174 x
# Conflicts:
#	src/control_plane.rs
2026-09-30 06:23:11 +02:00
Meta Vens
8447c22201 B2 fix batch: a continuity is survival, and survival runs uncounted
The route gate derives the charge from the transaction's command kind: a
continuity's preflight and install are Survival (never refused, counted), only
a deployment's are Forward. A Survival that cannot be recorded is logged and
proceeds; a Forward that cannot be recorded is refused. A proved route
challenge whose write is lost ends the pass like a failed one instead of
failing it. status() draws its target union from one function.

Tests: continuity through a full ledger, uncounted survival vs refused forward,
the second-failure wait, the lost proved-challenge write, each v3 route
supervision field, and the admitted-or-metered union. The routed world takes a
command kind, so the refusal tests run as deployments.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 02:44:43 +02:00
Meta Vens
61842a3952 Pin the chunk-size line against vertical tab and carriage return
Rows for a vertical tab before an extension, a carriage return before an
extension, and a size line ending in one or two carriage returns, so a
wider trim than SP and HTAB, or a stripped trailing carriage return,
cannot be taken.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 00:17:27 +02:00
Meta Vens
76c7b13b33 B2 rework: per-target supervision, typed route gate, generation v4 (production)
A target's route actuations and continuity restarts are sliding logs in one
TargetSupervision record (idunn.target_supervision.v1), created by the first
charge so first deployments are metered. The route driver asks a
RouteActuationGate before every group of host mutations; Forward changes can
be refused with a typed RouteActuationRefused, Survival ones (restoring the
admitted or incumbent route) are always admitted and always counted.

Generation moves to v4 with a typed lift from v3 that retires keys 18 and 20,
refuses a record showing the retired meters were written, and adds last_error
at key 21. Continuity restarts are counted in the mint CAS against the
target's own log; the history scan and both carry compensators are gone. Any
write to a generation or to its target record ends that target's supervision
pass. A due time further ahead than its owner's longest wait reads as due.

Tests do not build in this commit; the next one rebuilds them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-30 00:02:24 +02:00
Meta Vens
bf5347c370 The chunk-size line admits whitespace only before an extension
RFC 9112 7.1 allows whitespace before a chunk extension and nowhere
else. The reader trimmed both ends of the size, so " 3" and "3 " were
taken. It now splits at the first ';', trims trailing SP and HTAB only
when an extension follows, and leaves parse_digits as the digit
authority. Rows use an honest body so only the size line differs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 23:40:11 +02:00
Meta Vens
08d94ee61b Fetch blobs in bulk the way Git's lazy fetch does for a promisor remote
In a blob:none partial clone, fetching bare blob ids with --stdin made Git's
post-fetch connectivity check treat the id as a revision and fail with
"bad revision" / "did not send all necessary objects" against GitHub, which
broke a live deploy fetching a vendor gitlink's blob. Add
fetch.negotiationAlgorithm=noop, --recurse-submodules=no and
--filter=blob:none, the arguments Git's own lazy fetch uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:13:22 +02:00
Meta Vens
f9cebb33d0 Pin the partial-clone bulk fetch arguments in the spy test
A file:// origin delivers an explicitly wanted blob with or without the
promisor-style flags, so the real-shape fixture cannot fail on the bug that
GitHub triggers. The spy test already records the bulk fetch line; assert
it carries fetch.negotiationAlgorithm=noop, --recurse-submodules=no and
--filter=blob:none. Fails on the current arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:12:06 +02:00
Meta Vens
7f776ec3a6 Pin bulk_fetch_objects against a real blob:none partial clone
Every existing source-driver fixture uses an origin without
uploadpack.allowFilter, so --filter=blob:none is ignored, the clone is
full, and bulk_fetch_objects never has a missing blob to fetch. This
fixture advertises the filter, so the checkout really lacks the blob, and
checks delivery with GIT_NO_LAZY_FETCH=1 so only the bulk fetch can supply
it. It fails on the current fetch arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 21:06:03 +02:00
Meta Vens
d3db58227a Define the frozen-source symlink limit on Windows so idunn-host builds
The constant was unix-only since 9475dfa while its uses are not, so the
Windows idunn-host has not compiled since S1. Found installing B3: Raven's
actuator (built 2026-09-11) cannot decode the new plan types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 20:57:45 +02:00
Meta Vens
8432d45bbe B3 follow-up: bound the chunked trailer loop, refuse signed numbers and spaced header names
Trailers draw on the header budget instead of an unbounded loop. Content-Length
and chunk sizes are digits only (RFC 9112), and a header name with whitespace
is refused. Pins found by cargo-mutants: partial trailer, exact bounds on all
three framings, one byte then stop, unterminated chunk from a peer that keeps
going. ChallengeFailure::Silent documents what it actually is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:36:54 +02:00
Meta Vens
e8b44878b7 B3 fix: the over-bound header test must be over the bound
The previous commit's over-bound case used floor(bound / line) lines, which
can still fit under the bound once the status and Content-Length lines are
counted; the answer was accepted and the test failed. It now sends one whole
line more.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 18:23:11 +02:00
Meta Vens
4b3b86d863 B3 fix: pin the header bound at its edge and the chunked trailer loop
Two more mutants survived: the header-size bound was only tested far from its
edge, so doubling each line's cost went unseen; and the chunked trailer loop
was only exercised with a trailer present, so inverting it went unseen. The
bound is now tested one line inside and at its edge, and a chunked answer with
no trailer is read from a peer that keeps the socket open.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 18:19:07 +02:00
Meta Vens
d5669af352 B3 fix: pin what cargo-mutants found unpinned in the framed route reader
The header-size bound was tested only against answers that also stopped
short, so it never had to hold on its own; the interrupted-read retry was
exercised by no reader that could be interrupted. The header test now sends an
honest, whole answer whose headers alone exceed the bound, and a scripted
reader pins the retry and the silent/refused split at the Answer boundary.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 17:49:56 +02:00
Meta Vens
2f22dcac96 B3 fix: read a route answer by its framing, and judge a stopped peer one way
The HTTP challenge read to end of stream, so a peer that answered in full and
kept the socket open (or answered 404 and kept it open) read as silence. It
now reads by Content-Length or chunked framing, refuses any non-2xx status,
and refuses bytes that are not a CultNet frame.

One rule for both transports: a peer that sent no byte is silent; a peer that
sent some bytes and stopped is refused. A truncated TCP frame used to read as
silence; that pinned test now expects a refusal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 17:13:37 +02:00
Meta Vens
86ee21dac5 B3 fix: pin what cargo-mutants found unpinned, delete what it found redundant
First cargo-mutants --in-diff pass over a61540b..aa1889b left 16 misses.

Deleted:
- the candidate challenge's own route-id check: rendering the Expected next
  holds the same binding, so it could never decide anything.
- odin_can_observe in advance_warming: is_odin && !(!is_odin || observed) is
  is_odin && !observed; two of the misses were operators in that detour.

Moved: which transactions are held for their evidence is now
DeploymentTransaction::held_disagreement, so "finished or aborting work is
never held" is a value with a test, not a local in resume_candidate.

Pinned: the admitted-generation topology refresh (taken once, then not
again), WarmingEvidence shape checks per source, the PresenceDisagrees text,
the plan's readiness declaration needing the shared-infrastructure kind, TCP
framed challenges (exact answer, bound, empty, non-CultNet, silence), RUDP
silence, and ChallengeFailure's text and source.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 15:24:45 +02:00
Meta Vens
aa1889b288 B3 fix: kill the three surviving mutants
Boot re-proof by tag: the held-generation test used receipts that still
re-proved cleanly (boot reads the current Odin authority, not the stored
one), so re-proving them passed. The receipts are now authentic but not
Ready, so only skipping them lets the store boot.

Config error as silence: the candidate path renders Expected first, which
parses the same endpoints, so the challenge's own parse is reached only from
the stable path. A new test drives that parse directly.

First-Odin gate by name: the stub can now answer a first-Odin Warming
challenge, and a stateful provider not named odin is accepted while a
stateful target named odin that provides nothing is refused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 14:41:25 +02:00
Meta Vens
e11b457458 B3 fix: pin the route-proof rules, and delete guards another owner already holds
Tests for each rule the Soul pass found unpinned, each written to fail
under its own mutation:
- a stateful route-proof target through the Engine, Fencing to commit: the
  lease is prepared and granted, Ready is refused without the exact lease
  and recorded with it, boot re-authenticates the store at every stage and
  Odin is never read;
- Warming challenges once and then advances on every tick;
- a stateful candidate may not answer its Warming challenge active;
- a presence holds the write lease exactly when it is not warming, and is
  minted after its challenge;
- direct Warming evidence is bound to the bytes and digest it authenticated;
- candidate-vouched evidence is refused on a target Odin reports on;
- a pre-B3 transaction and a pre-B3 generation over a route-proof target are
  held and reported once, not stepped every tick, and not re-derived;
- a stored record with no declared readiness is held;
- Odin is the class OdinSelf, not the name "odin";
- admission refuses an undeclared recipe with a typed error, before the
  source is frozen; a recipe that declares its proof is planned;
- the driver's challenge failures: silence waits, every bad answer and every
  local configuration error is an error;
- the capacity shortfall is read as a typed PresenceDisagrees.

Deleted because another owner already holds them, so no test can reach them:
the Routing and commit "route-proof Ready evidence" ensures (validation
refuses a missing receipt, the hold refuses the other party's), the write
lease comparison's warming arm (the authenticator refuses a warming presence
that claims a lease), and the direct-Warming refresh check (rehydration
authenticates the recorded answer as Odin's first).

Engine.source is a SourcePort, the seam the admission test needs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 13:32:42 +02:00
Meta Vens
6ea824b5a9 B3 fix: readiness class comes from the recipe alone
ReadinessClass::of is the only class authority. A target whose recipe
declares neither a route nor an Odin dependency, and does not provide the
rendezvous, has declared no way to prove readiness: of() returns a typed
UndeclaredReadiness naming the target, and admission refuses it after plan
compile, before anything is frozen or installed. The OdinCorrelated fallback
for an unrouted target is gone.

The evidence tag no longer decides the path. Boot re-authentication and
admitted-topology refresh choose by class; a stored transaction or admitted
generation whose evidence was vouched for by the other party than its
Expected requires is a typed ReadinessDisagreement, reported once and held,
not a step that errors every tick. Nothing re-derives around it.

"Is this Odin" is the class OdinSelf, not the name "odin": validation,
warming, the Odin authority lookup and the direct-warming checks read the
class. canonical_sha256 on WarmingEvidence and the token's unread evidence
field are deleted.

A bad answer to a direct challenge is an error. The driver splits its
failures into ChallengeFailure::Silent (refused, dropped, hung up, timed
out) and ::Refused (a malformed, foreign, oversized or non-raw answer, or a
local configuration error); only Silent waits. The capacity/disagreement
shortfall is returned as a typed PresenceDisagrees, not flattened to prose.

Odin-driven fixture worlds declare the Odin dependency in Expected and the
correlation they sign names its provider.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 13:12:46 +02:00
Meta Vens
cfe71f680c WIP B3: route-proof harness and tests
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 10:54:22 +02:00
Meta Vens
9e5b7f84fd Close the B1 transition: lift, precondition, boot reconciliation, closed history
Soul found B1's rule sound and its transition unsafe. This makes the rule's
old inputs impossible to reintroduce a stuck state from.

- The control store's legacy lift owns the transition. A pre-B1 continuity
  abort that issued an activation lifts with topology_reconciliation Pending,
  so the one resolution demotes its own activation. A record already terminal
  under the old rule is reopened at Starting, since a terminal record cannot
  owe work. History lifts records as written. Before this, one resident
  record of that shape failed validation and ControlSnapshot::read refused the
  whole store, so Idunn could not boot.
- Supervision owns the Expected-only precondition. When its pre-restart
  demotion fails it mints no continuity and writes next_restart_at_unix_millis
  on the generation's ContinuityBackoff (attempts and window untouched), so it
  neither restarts over an undemotable projection nor retries every tick.
- Boot reconciles once: a projected activation that exactly matches the
  activation a failed continuity in history (or a resident terminal record)
  issued is demoted. Nothing is adopted; an activation no admitted generation,
  live transaction or failed issuer accounts for is reported and left alone.
  New read-only CultCacheTopologyDriver::projected_activation.
- Unreadable history.cc fails closed. The continuity restart ceiling and the
  consumed-command check no longer read an empty history; they stop, and the
  fault is reported once while it lasts (ReportOnce). The undecodable-record
  report is a pure function, pinned.
- docs/deployment-authority.md no longer says the projection never drifts.

Tests: legacy fixtures written by the 9001b58 encoder for the three abort
shapes (a second throwaway generator, see the fixtures README), driven through
the Engine; the seeded-transaction helper now takes a command kind and an
incumbent, and a Deploy carries a frozen-source receipt and a signed brake
release, so Deploy commit, Deploy pre- and post-fence aborts and a continuity
abort over an incumbent run through the real phase machine.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 06:11:29 +02:00
Meta Vens
4b5e60fb12 Merge origin/main into idunn/route-b1
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 05:13:07 +02:00
Meta Vens
87fa57dd8c Revert "Recover fenced rollback from current activation"
This reverts commit a148802cb6.

Rollback after fencing adopted whatever activation was projected under the
shared Expected key, so a continuity abort's leftover activation could be
demoted by a later deploy. The drift is closed at its source by the next
commit: a continuity abort demotes its own exact activation. Rollback demotes
the admitted generation's exact activation again.

The doc paragraph is dropped; the 9f00e7a paragraph above it stays until B4.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 01:30:09 +02:00
Meta Vens
b619f9875c idunn: route supervision observes; it actuates only on drift (S1)
Delete the stale-observation -> repair-intent -> restore path from
supervise_admitted_route. An exact fragment with a stale observation now
yields one route challenge and nothing else. restore_admitted_membership
returns before nginx -t, the private-mount unit, ufw and reload when the
fragment already equals the admitted membership. route_repair_started_at
stays readable but no longer decides anything.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 00:56:30 +02:00
Meta Vens
29946bbe66 Wait for Idunn stable route listener 2026-09-28 17:29:25 +02:00
Meta Vens
ae8ac45df4 Treat process title as launch evidence 2026-09-28 17:11:37 +02:00
Meta Vens
5e94eb82f0 Name native identity fields that drift 2026-09-28 17:02:46 +02:00
Meta Vens
e976d32243 Name fields when workload identity drifts 2026-09-28 16:41:57 +02:00
Meta Vens
865190611f Accept dynamic primary group in workload audit 2026-09-28 15:19:52 +02:00
Meta Vens
a148802cb6 Recover fenced rollback from current activation 2026-09-28 13:56:05 +02:00
Meta Vens
9475dfa742 Land the fourth Cut 1 fix batch (S4-1 through S4-6)
Soul's fourth pass found the symlink resolver newly exponential
(resolve_frozen_source_symlink recursed per path component and
re-resolved every shared prefix from scratch, so an n-link
self-doubling chain cost 2^n real filesystem resolutions), the
streamed hash unpinned, three of the five S5 guard tests not killing
their own guard, all three S1 fsck guards unpinned against
already-local objects, no test proving a legitimate in-root ".."
freezes, and the symlink writer still buffering a whole blob for a
120000 entry.

S4-1: resolve_frozen_source_symlink is now an explicit iterative walk
(a work-stack of frames standing in for the old call stack) with
per-symlink memoisation and one shared traversal budget charged per
distinct symlink read, not per nesting depth. Soul's n=20 doubling
chain (a0 -> ., ak -> a{k-1}/a{k-1}) now resolves in milliseconds
instead of 43.7 s, and freezes rather than refusing: nothing in the
chain dangles or escapes the root, matching S6.

S4-2: two trees differing only past the first 64 KiB chunk now assert
different digests, killing the zero-length-buffer and first-chunk-only
mutants cargo-mutants found.

S4-3: the duplicate-path, ensure_frozen_directory containment, and
create_dir guards each get a fixture that only that guard refuses --
calling git_tree_entries directly (before any fsck runs) for the
duplicate path, and pre-seeding created_dirs with state a second,
independent bookkeeping pass could produce for the other two.

S4-4: Soul's eleven already-local hostile fixtures (dup-trees, six
.git look-alikes, an unsorted tree, mode 120000 on a tree, mode 040000
on a blob, a zero-padded mode) land as tests against freeze_exact's
own explicit fsck. A second test proves freeze_exact's own fetch fsck
flag is what refuses a hostile revision that was never local, by
asserting which context wraps the error.

resolve()'s own fetch fsck flag is still unpinned: exercising it needs
a real HTTPS origin (OperatorBinding::validate requires one), and
url.insteadOf does not work around that -- `git remote get-url` expands
the rewrite, so it can never satisfy ensure_checkout's own
origin-equality check. Confirmed by hand against a local git; a real
smart-HTTPS harness like Soul's S1 rig is what closing this needs.
Recorded as a gap, not faked with a broken test.

S4-5: a legitimate in-root ".." link (two hops up from a nested
directory, back to a file at the root) now has a test proving it
resolves and freezes, not just that an escaping "..' refuses.

S4-6: materialize_tree_raw refuses a 120000 entry whose blob exceeds
the platform's PATH_MAX before reading it, for both the streamed-blob
and the shares-content-with-a-regular-file cases.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 20:30:12 +02:00
Meta Vens
f540d50835 Fix S1 test: use a safe.directory-scoped git for post-chown commit-tree/update-ref
git_at runs plain `git` with no `-c safe.directory=*`. By the time the
hostile commit is built, the origin repo is already chowned to the
unprivileged fixture identity, so root's plain git refuses it as dubious
ownership. Use the same safe.directory-scoped helper the duplicate-tree
fixture above already uses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:28:22 +02:00
Meta Vens
ad31f51273 Fix S1's fetch-guard test order: establish the checkout on the good commit first
The test was moving `main` to the hostile commit before the first checkout
existed, so the initial (also fsck-guarded) clone refused it, never
exercising the fetch line S1 actually changed. Establish the checkout on the
good root commit first, then advance `main`, matching the real scenario: an
already-fetched deploy target whose admitted ref later moves to a hostile
commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:27:04 +02:00
Meta Vens
748e9f3dfd Fix the fix batch: S6 test against the new symlink split, S1 test around the HTTPS-only binding
frozen_source_rejects_a_symlink_that_escapes_the_transaction_tree asserted
against harden_frozen_source, which no longer resolves symlink chains (S6
moved that to validate_frozen_source_symlinks, against the tree's final
published path). Point it at the right function.

resolve_then_freeze_refuses_hostile_trees_through_the_production_path could
not go through the literal SourcePort::resolve()/freeze() trait methods:
OperatorBinding::validate() requires an https:// origin unconditionally, and
this suite has no HTTPS Git fixture. Replaced with a fetch that reproduces
resolve()'s exact invocation (this fix's -c transfer.fsckObjects=true
included) against the checkout, the same approximation Soul's own probe used,
followed by the real freeze_exact as defense in depth.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:25:32 +02:00
Meta Vens
57ecd43d6b Land the third Cut 1 fix batch: fsck on every fetch, published-location symlink checks, streamed hashing
S1: guard resolve()'s admitted-ref fetch with transfer.fsckObjects=true (it
had none), and have freeze_exact additionally run an explicit `git fsck`
against the selected revision so objects that were already local -- not only
freshly transferred ones -- get checked too. Refusal now happens through the
production path (resolve() then freeze()), not only a hand-built fetch.

S5: behavioural tests for six guards Soul's third pass found unpinned: root
ownership in harden_frozen_source, an exact duplicate leaf path, the
ensure_frozen_directory root-escape check on its own, the writer's pass-1
create_dir semantics, and copy_artifact's containment check. (The sixth,
case-sensitive ancestor alias, goes away under S7.)

S6: symlinks are now judged against the tree's real, final published path,
not `.partial`, the temporary name the tree is built under before the atomic
rename that publishes it. A dangling link whose target stays inside the root,
resolving every component that exists, is accepted; one that escapes is
refused, dangling or not. harden_frozen_source_tree defers symlink chain
resolution; the new validate_frozen_source_symlinks walk runs once, right
after the rename, and validate_frozen_source_symlink (used by observe_frozen
too) now shares the same dangling-tolerant resolver, so freeze-time and
observe-time agree.

S7: delete the case-folding collision/alias refusal in
refuse_conflicting_tree_entries. The host is case-sensitive ext4, the check
was inconsistent (leaf paths only, plain to_lowercase), and .git look-alikes
are now refused by git fsck itself (S1), independent of case. The exact-name
alias case (a leaf sharing a root with a tree recursing into it) stays
refused at write time by ensure_frozen_directory, which never writes through
an existing entry.

S9: hash_frozen_source_tree streams each file through the hasher in 64 KiB
chunks instead of fs::read-ing it whole, so peak RSS no longer scales with
the largest file in the tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:21:37 +02:00
Meta Vens
82e0edfaf7 Land the second Cut 1 fix batch: F2/F3 write-outside-root regression, F4 hardening, F5 chain-aware symlink check
Soul's second pass on the Cut 1 fix batch found a new regression (a
duplicate/aliased tree name lets the raw freeze write outside its root) and
three pre-existing gaps: two redundant hardening passes with no pin, a
lexical (not chain-resolving) symlink escape check, and unpinned Docker argv
combinations. F1's perf finding (40-80x slower than the old git archive path)
is addressed here too.

F2/F3 (write-outside-root, two required layers):
- git_tree_entries now calls refuse_conflicting_tree_entries, which refuses
  a tree where two leaf paths collide case-insensitively, or where one leaf
  path is a case-folded ancestor of another (the exact shape
  `git hash-object --literally` lets through: a symlink and a subtree
  sharing one name). The exact-revision fetch and both clone paths
  (ensure_checkout, materialize_gitlink_raw) enable transfer.fsckObjects.
- The writer itself never trusts that check alone. ensure_frozen_directory
  is the only way materialize_tree_raw obtains a directory to write into: a
  path must already be a directory this freeze created (tracked in
  created_dirs, shared across the superproject and every Gitlink), or must
  not exist at all; anything else is refused rather than traversed.
  Symlinks are written in a dedicated last pass, after every directory and
  regular file the tree needs already exists.

F4: freeze_exact now calls harden_frozen_source once, last, instead of
harden then a separate validate_frozen_source pass over the same tree.
validate_frozen_source stays as the read-only re-check observe_frozen uses
on an already-hardened tree it did not just write.

F5: validate_frozen_source_symlink resolves the whole symlink chain with
fs::canonicalize instead of lexically counting ".." components, which
undercounts a chain that loops back through another symlink (D -> .,
L -> D/D/.../../../../etc/passwd). copy_artifact (the one place besides
freeze that reads through a possibly-attacker-controlled tree) gets the same
whole-chain resolution: it now takes the artifact's containment root and
refuses a source that resolves outside it, closing the gap where an
intermediate symlink in a runner's own build output could be followed.

F1: materialize_tree_raw bulk-fetches every blob a tree needs in one
`git fetch --stdin` round trip (Git 2.36+, no argv-length limit) instead of
letting a blob:none partial clone lazily fetch each one as `cat-file --batch`
reads it. Blobs stream straight to their destination file instead of being
held in a BTreeMap for the whole tree. read_blobs is replaced by
stream_blobs, which drains stderr on a dedicated thread and waits on the
child on every return path, including an error from the per-object
callback (F9).

F6: pins four previously-unpinned Docker argv/environment combinations
(N1: --read-only survives a Named network plus a cache mount; N2: a secret
mount stays read-only alongside a plain environment entry; N3: the source
stamp is never dropped alongside a secret and a plain environment entry;
N5: an LFS pointer file sets contains_lfs_pointers).

Tests: 142 to 152. cut1.psd1 grows by 12 entries (5 for F1-F5, 4 for F6's
N1/N2/N3/N5, and one recording the F2 fsck flag as not yet reached -- see
below); all 28 non-comment entries in the suite now kill.

Discrepancy: cut1-fix-f2-no-fsck-on-fetch could not be pinned. Every tree
this crate's own tools can build that fsck's duplicateEntries check rejects
also has a literal duplicate path or a case-folded ancestor collision that
refuse_conflicting_tree_entries rejects first, unconditionally, before the
fetch's fsck result can matter. The flag stays as defense in depth (fsck's
other structural checks are broader than this crate's own parser), recorded
as not yet reached in cut1.psd1 rather than removed or falsely claimed
unreachable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 18:29:36 +02:00
Meta Vens
43c7a6e46d wip5 fix spy log perms 2026-09-22 17:47:29 +02:00
Meta Vens
f8baa93dac wip3 tests+psd1 2026-09-22 17:46:02 +02:00
Meta Vens
2ee7c225e2 wip2 2026-09-22 17:41:06 +02:00
Meta Vens
95016e817d Land F2: freeze writes blobs raw, per Self's ruling on the Eve survey finding
Self reviewed the F2 survey finding (Eve's gitlink under Ghostlight carries
`packages/*/dist/** text eol=lf`) and ruled it is not a real dependency on an
archive transform: Git stores text blobs as LF, `eol=lf` checks them out as
LF, so no conversion happens. Self verified this directly at Ghostlight's
pinned Eve gitlink revision `672c0c1e9fc828b63752345385727b06c50491d0` — all
26 files under `packages/*/dist/` hash identically between `git archive` and
`git hash-object --no-filters`, and none contains a CR. Stopping for the
survey was correct; the ruling is to proceed.

Survey table (repo, attributes found, verdict), recorded here as ruled:

| Repo (deploy target) | Binding source | .gitattributes findings | Verdict |
|---|---|---|---|
| Ghostlight | idunn/yggdrasil/bindings/ghostlight.toml.in | none | clean |
| Ghostlight's gitlink vendor/eve -> Eve | same binding | packages/*/dist/** text eol=lf | transform present, but a no-op on stored LF content (Self's evidence above) |
| Odin | idunn/yggdrasil/bindings/odin.toml.in | none | clean |
| CodexConnector | idunn/yggdrasil/bindings/codex-connector.toml.in | none | clean |
| Muninn (raven-muninn) | idunn/yggdrasil/bindings/raven-muninn.toml.in | none | clean |
| Heimdall | deployment/idunn-bindings/heimdall.toml.template (a second, previously unlisted binding directory, installed live 2026-09-06) | none | clean |
| Heimdall's gitlink -> CultLib | same binding | * text=auto only (F12) | clean |

`freeze_exact` now writes every tree entry raw from the object store:
`git ls-tree -r -z` for the manifest, `git cat-file --batch` for content, with
no `git archive` anywhere in the path and so no attribute-driven transform of
any kind. Modes and symlinks are preserved exactly as the tree records them.
Gitlinks are frozen the same raw way, through the same primitive, at their
own path. LFS content is never fetched; `freeze_exact` now also returns
whether any materialized file is an LFS pointer (recorded in the frozen
result's own return value — not yet plumbed into `FrozenSourceReceipt`, which
has no field for it; that is follow-up scope, not this fix).

Deleted: `git_archive_into`, `materialize_gitlink_archive`, and the
now-unused `tar_program` field — the archive-and-extract path they built has
no remaining caller. `GitSourceDriver::new` no longer takes a tar program.

Tests:
- `freeze_exact_is_byte_exact_across_every_attribute_transform`: a fixture
  with `eol=crlf`, `export-ignore`, `export-subst`, `ident`, a symlink, the
  executable bit, and a Gitlink. Every frozen file's bytes are asserted equal
  to `git cat-file blob` at the selected revision.
- `freeze_exact_matches_archive_when_untransformed_and_diverges_when_transformed`:
  the deploy-equivalence test Self's ruling asked for. An `eol=lf` file over
  already-LF content matches today's independently-reconstructed `git archive`
  tree exactly; a second file with `eol=crlf` over the same LF content
  diverges from it, so the equivalence check has teeth rather than trivially
  passing for every fixture.
- `exact_git_blobs_become_root_owned_immutable_source_without_git_metadata`
  (renamed from the git-archive-era test) now exercises `materialize_tree_raw`.
- `tools/verify-mutations/cut1.psd1`: `cut1-fix-freeze-skips-gitlinks`
  retargeted at the new call, and a new `cut1-fix-f2-revert-to-git-archive`
  entry that reverts `materialize_tree_raw` to a `git archive | tar` pipeline;
  it must die against the byte-exact test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 16:48:17 +02:00
Meta Vens
7a5d32f02d Land the Cut 1 fix batch: pin every argv branch, one fetch, dead StepPort gone
Soul's pass on Cuts 0-1 found the argv-lowering test pinned only the
default runner path (13 of 14 mutants survived), freeze fetching the
same revision twice, an untested Gitlink loop, dead StepPort/StepOutcome
code, and a stale guide.md line. This lands Self's rulings for all of
it except F2 (the byte-exact freeze): the F2 survey found that Eve, a
Gitlink of the Ghostlight deploy target, depends on an archive
transform (`.gitattributes` eol=lf under packages/*/dist/**), so per
ruling 1 the byte-exact rewrite is withheld and reported instead.

- F1: `docker_run_args`/`ContainerSpec::for_step` are now pinned across
  every network branch (none/bridge/named), the full CPU quota range
  (100/150/250/333/800), cache root present/absent, and secret mounts,
  with dedicated fixtures for the cache-root and secret-file validation
  calls so a mutant that skips either is caught.
- F3: `freeze` fetches the exact revision once, through `freeze_exact`,
  instead of fetching before it and again inside it.
- F5: `freeze_exact_materializes_gitlinks` pins that the Gitlink loop
  actually runs.
- F7: deletes the dead `StepPort`/`StepOutcome` types; `run_step` never
  had a caller, and `docker()` is the one Docker spawn path.
- Stamp override: `ContainerSpec::for_step` now refuses a step
  environment name that collides with the Idunn source stamp, so the
  invariant holds at the one place every runner's environment is built,
  not only at deploy's separate `admit` check.
- docs/guide.md:11 no longer claims the retired generation still runs
  on Yggdrasil.

F8 (the `--cap-drop` negative check) is a reporting fix, not a code
change: `rg -n '"--cap-drop"' src/` now returns 4 hits (1 real site
plus 3 test fixtures, one of which this batch adds), never the "1 hit"
the map's verification text describes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 16:15:38 +02:00
Meta Vens
95e15efca3 Cut 1: one step primitive and one freeze primitive (behaviour-preserving)
Deletes the driver-private Docker argv assembly in run_in_workspace and
base_run_args and the OperatorBinding/CompiledDeploymentPlan coupling inside
GitSourceDriver::freeze's fetch-and-archive core. Both become one shared
primitive apiece, the shape Cut 4's verify worker will reuse instead of
growing a second copy:

- ContainerSpec + docker_run_args(spec, workspace, working_directory, argv):
  a plain, pure lowering to the exact `docker run` argv. ContainerSpec::for_step
  is the one constructor (a DockerRunnerBinding, a step's required
  environment, and the source stamp); it does the cache-root and secret
  validation against the filesystem before docker_run_args ever runs.
  run_in_workspace and materialize_external_input both build a ContainerSpec
  and call docker_run_args now; base_run_args is gone (rg '"--cap-drop"'
  src/ has exactly one hit).
- ExactSource {origin, checkout, gitlinks, recipe_path} plus
  GitSourceDriver::freeze_exact(&ExactSource, revision, transaction_id, root),
  which fetches, archives and hardens an exact revision with no
  OperatorBinding or compiled plan. freeze() now derives an ExactSource from
  the binding, keeps its own verify_exact_source pre-check against the
  durable resolution, and delegates the archive to freeze_exact.
- A StepPort trait (DockerRunnerDriver::run_step) that runs a lowered step
  and returns exit status plus a bounded stderr tail instead of bailing.
  Not yet reached from materialize (Cut 4 wires it into the verify worker);
  deploy keeps DockerRunnerDriver::docker's existing bail-on-failure error
  text unchanged.

materialize_external_input's curl step now goes through the same lowering,
which adds a harmless --workdir /workspace and one inert IDUNN_EXTERNAL_INPUT_ID
env var to its argv; curl's behavior (an absolute --output path) is
unaffected. materialize_gitlink_archive's cache-directory name changed from
the operator binding's target label to a hash of the checkout path, since
ExactSource carries no target field; this is bookkeeping only; a UUID
already made every gitlink checkout unique.

Adds three unit tests: container_spec_lowers_to_exact_docker_argv pins the
full docker run argv; required_environment_is_the_only_environment pins F6
(a binding environment entry the step does not require never leaks in);
freeze_exact_is_byte_exact_and_recipe_checked and its negative twin
freeze_exact_rejects_a_recipe_the_archive_transforms (an export-subst
recipe, which git archive substitutes and git cat-file does not) exercise
freeze_exact directly, with no OperatorBinding in sight.

tools/verify-mutations/cut1.psd1 covers all five rules with a revert or
loosening mutant apiece, run through the Eureka harness in the stopgap
container. Verified on Yggdrasil at this commit: cargo test --lib is 130
passed, 0 failed, 2 ignored (the P1 baseline of 126/2 plus these four new
tests), and every mutation entry was killed by its own test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 15:46:57 +02:00
Meta Vens
144581f9c7 Host actuator: Idunn manages a workload on a Windows host it cannot touch
A managed host runs idunn-host in the user session. It dials Idunn's hub
over CultNet RUDP on the mesh, proves itself with a host actuator service
identity, and executes the runner and workload ports for its host: fetch
the exact frozen revision, build with the host's cargo under the binding's
allowed programs, install the sealed release, start the workload as a
child, report pid, creation time and executable digest, stop on request.
It keeps no transaction state and decides nothing.

On the Idunn side the engine chooses the runner and workload driver per
plan from the binding's driver tag; the yggdrasil daemon is still the
single deployment and continuity authority, and a binding of the old kind
takes the same path it always did. Every request is signed by Idunn's
identity and every report by the host's; both ends refuse the other's
message unless signature, session nonce and sequence hold. The hub admits
its own firewall allow the way the route driver does.

idunn-provision learns to enroll and export the host actuator identity.
The unit gains --host-actuator-bind on the mesh address.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 13:49:25 +02:00
Meta Vens
c848459471 Bindings are driver-tagged and the observation is a variant per host kind
RunnerBinding and WorkloadBinding become enums tagged by driver, so a
host-native runner and a host-actuator workload can be declared without
counterfeiting docker images or systemd units. The single-variant
RunnerDriver and WorkloadDriver enums are gone; the driver is the tag.

WorkloadObservation is now Systemd or Host, untagged on the wire so every
transaction the previous Idunn persisted decodes unchanged (proved under
both MessagePack encodings CultCache uses). IsolationEvidence follows:
Linux proves uid, pid and mount namespaces; a host proves two live
processes distinct by pid and creation time. Mixed kinds are refused.

The build machine-id seal is unix-only so the library compiles on the
Windows host that will run the actuator.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 13:25:50 +02:00
Meta Vens
f16fb46c95 Route driver owns the host firewall allow for its stable endpoint
A route that is admitted must be reachable without an operator opening a
port by hand. NginxRouteDriver admits one ufw allow for the binding's stable
address, port and transport, tagged with the route id, before it reloads a
fragment, and withdraws that one rule when the last membership is removed.
A rule already gone reads as the state asked for. Odin on yggdrasil is
admitted on 10.77.0.1:17971 with ufw still admitting only the previous
generation's 17871; this closes that class.

Verified with cargo test --lib in rust:1.95-bookworm on yggdrasil: 118 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 00:23:28 +02:00
Meta Vens
1899df2244 Withdraw stale incarnations from the projection in supervise
A replaced incumbent, an aborted candidate whose reconciliation was skipped,
and a record left by the projection migration all leave an Expected-only
incarnation in the projection that nothing owns. Supervise now withdraws one
per tick for each admitted target: any projected incarnation that is neither
the admitted generation nor a live transaction's. One owner for all three
sources, instead of a per-commit field.

Verified with cargo test --lib in rust:1.95-bookworm on yggdrasil: 117 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 23:58:05 +02:00
Meta Vens
903e1c8e63 Key the projection by incarnation; publish Expected after the brake; consume commands once; add cancel
The projection kept one Expected/activation/lease slot per target. A
deployment's Sealing overwrote that slot with the candidate's Expected, the
incumbent failed activation-expected-projection on its next heartbeat and
exited, the deployment was aborted for having killed its incumbent, and the
command -- left resident after its transaction was retired to history --
read as queued and was frozen again. Odin cycled on that every few minutes.

- Topology driver records are keyed {target}@{expected sha}; the incumbent and
  the sealed candidate are projected side by side. One CAS mutation replaces
  the six bespoke loops. withdraw_incarnation removes only one incarnation;
  demote_to_expected_only replaces restore_admitted_expected_only and is used
  only for an incarnation whose process is gone. Legacy target-keyed records
  are retired on the first publish for that target.
- receive() takes the incarnation; the read-as-absence compensator for a
  correlation about another incarnation is gone.
- Candidate Expected is published as Starting's first step, after the brake
  admitted the transaction, not in Sealing before it.
- A pre-fencing abort withdraws only the candidate; a post-fencing abort also
  demotes the stopped incumbent.
- A consumed command is archived with its last transaction; status still
  answers for it from history.
- An engaged lifecycle brake stops continuity being minted instead of parking
  a transaction on the target.
- idunn cancel <command-id> retires a queued command as an operator refusal.

Verified with cargo test --lib in rust:1.95-bookworm on yggdrasil: 114 passed,
2 ignored.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 22:22:46 +02:00