Commit graph Idunn/src
Author SHA1 Message Date
Meta Vens
5ca7760fdd WIP B3: pin route admission past the gate 2026-09-29 11:06:25 +02:00
Meta Vens
4dcdd3608c WIP B3: route admission test 2026-09-29 11:03:47 +02:00
Meta Vens
54507060a5 WIP B3: pin the state rules, the lease refresh and boot re-authentication 2026-09-29 11:02:06 +02:00
Meta Vens
b9864b5041 WIP B3: Odin-providing target is driven through the engine 2026-09-29 11:00:47 +02:00
Meta Vens
66fda86d8f WIP B3: two private ports in the routed world 2026-09-29 10:56:26 +02:00
Meta Vens
cfe71f680c WIP B3: route-proof harness and tests
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 10:54:22 +02:00
Meta Vens
a7b602ae65 WIP B3: route-proof readiness in the phase machine
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 10:48:33 +02:00
Meta Vens
5743a25389 B1 fix batch 4: faults re-arm only on recovery; notes reach marked records
- Fault reports clear only when a step ran and succeeded, not on a backoff
  skip or a handled failure.
- Fault notes are written through cleanup_evidence::last_error_envelope, so
  an unretirable pre-B1 terminal record shows its fault; a failed note is
  printed with the fault instead of swallowed.
- backoff_wait and is_waiting are pure and unit-tested; tick and resume
  return values are pinned.
- migrate_transaction_record is the migration's only entry; the encoder that
  can write a marker is private to cleanup_evidence.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 10:28:34 +02:00
Meta Vens
287aa92382 B1 fix batch 3: progress means state advanced; one wedge starves no one
- A tick's progress is a state change, never a last_error rewrite; freezing a
  queued command no longer waits on other targets' progress (still at most
  one freeze per tick, busy targets refused as before).
- Failing transaction steps back off (process-local, doubling from the poll
  interval to 60s) instead of retrying every tick.
- Boot reconciliation runs the resident-issuer pass with history unreadable.
- Resume and retire faults are reported once per transaction and left in
  last_error, which status renders.
- The legacy marker is a private-constructor type inside cleanup_evidence;
  transaction_envelope refuses a record carrying it, the schema migration
  writes through its own path.
- Tests: changing-error wedge, backoff, dead-candidate post-fence branch,
  fault note, marker refusal, torn-history boot residue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 08:41:25 +02:00
Meta Vens
8998cfcf02 Seed the wedged post-fence abort by hand
A transaction seeded at AwaitingReady fails validation (no fencing evidence),
and nothing at Fencing makes a workload error, so the post-fence abort is
begun directly and the test pins that its failing step only waits.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 07:56:38 +02:00
Meta Vens
81478e6ddd Pin the scheduler's progress report and the post-fence abort path
Cargo-mutants on the previous commit missed the per-candidate progress flag
and the post-fence branch selection in resume_candidate. A repeated error
reports no progress; a dead candidate past the fence aborts; a wedged
post-fence abort only waits.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 07:36:43 +02:00
Meta Vens
81ec1c7f40 Give the wedged-abort test the operator binding a freeze needs
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 07:21:21 +02:00
Meta Vens
b75fbdf96c Boot reconciliation owns legacy residue; the scheduler isolates per-record faults
Soul's pass on the B1 fix batch found the lift's reopen of a terminal pre-B1
continuity abort could stop Idunn booting (the reopened record claims target
authority beside a live transaction, and the store was already migrated to v4),
and that one wedged abort stopped the whole scheduler.

- The lift no longer reopens anything. A terminal pre-B1 continuity abort that
  issued an activation keeps its phase and completion and carries the typed
  marker CleanupEvidence::LegacySkippedBeforeB1 on its topology_reconciliation.
  Validation accepts that shape only for a terminal continuity abort. Boot
  reconciliation, already the exact-activation owner of that residue, is the
  only cleanup path. A pre-B1 abort still in flight lifts to Pending: it
  already holds its target's authority, so the lift adds no claimant.
- A failed step of a pre-fence abort is recorded as a resumable error, never
  handed to the post-fence abort, which refuses a pre-fence phase and failed
  every tick.
- One transaction's error no longer fails run_scheduler_tick: each candidate
  is resumed on its own and its fault logged, and retiring a terminal record
  tries the next one when an archive fails. An unreadable control store still
  fails the tick.
- serve is boot plus the loop, so a test can run the boot sequence.

Tests pin, through the Engine: the marker and no-reopen for each pre-B1
fixture; a resident terminal abort plus a live transaction plus a second
resident abort for one target; a withdrawn incarnation never re-published and
history keeping one copy after a crash between the archive steps; a wedged
abort leaving freezing alive; an unarchivable terminal leaving resume alive;
and boot demoting the residue.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 07:17:11 +02:00
Meta Vens
ff3cd51910 Pin the B1 fix batch's survivors
cargo-mutants over the batch's diff missed the lift's gate, the unexplained
projection accounting, the refusal count and the deferral clock. Tests now
cover each: the lift owes only a continuity that issued an activation; a live
transaction accounts for its own activation and nobody else's; three refusals
in history hold the restart ceiling; the deferral lands between now and now
plus its length.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 06:45:37 +02:00
Meta Vens
9e5b7f84fd Close the B1 transition: lift, precondition, boot reconciliation, closed history
Soul found B1's rule sound and its transition unsafe. This makes the rule's
old inputs impossible to reintroduce a stuck state from.

- The control store's legacy lift owns the transition. A pre-B1 continuity
  abort that issued an activation lifts with topology_reconciliation Pending,
  so the one resolution demotes its own activation. A record already terminal
  under the old rule is reopened at Starting, since a terminal record cannot
  owe work. History lifts records as written. Before this, one resident
  record of that shape failed validation and ControlSnapshot::read refused the
  whole store, so Idunn could not boot.
- Supervision owns the Expected-only precondition. When its pre-restart
  demotion fails it mints no continuity and writes next_restart_at_unix_millis
  on the generation's ContinuityBackoff (attempts and window untouched), so it
  neither restarts over an undemotable projection nor retries every tick.
- Boot reconciles once: a projected activation that exactly matches the
  activation a failed continuity in history (or a resident terminal record)
  issued is demoted. Nothing is adopted; an activation no admitted generation,
  live transaction or failed issuer accounts for is reported and left alone.
  New read-only CultCacheTopologyDriver::projected_activation.
- Unreadable history.cc fails closed. The continuity restart ceiling and the
  consumed-command check no longer read an empty history; they stop, and the
  fault is reported once while it lasts (ReportOnce). The undecodable-record
  report is a pure function, pinned.
- docs/deployment-authority.md no longer says the projection never drifts.

Tests: legacy fixtures written by the 9001b58 encoder for the three abort
shapes (a second throwaway generator, see the fixtures README), driven through
the Engine; the seeded-transaction helper now takes a command kind and an
incumbent, and a Deploy carries a frozen-source receipt and a signed brake
release, so Deploy commit, Deploy pre- and post-fence aborts and a continuity
abort over an incumbent run through the real phase machine.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 06:11:29 +02:00
Meta Vens
4b5e60fb12 Merge origin/main into idunn/route-b1
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 05:13:07 +02:00
Meta Vens
6e5f87676b Merge route continuity F0: typed readiness, deadlines and actuation state
The generation moves to v3 and the transaction to v4. They carry
ReadinessEvidence (the class is derived from Expected by ReadinessClass::of),
phase deadlines owned by one writer (enter_phase), RouteSupervisionState,
ContinuityBackoff, LeaseAdoptionEvidence and TerminalRecovery. The plan
moves to v3 with frozen deadline durations. A boot migration lifts every
legacy record with stored_at preserved. No decision reads the new fields
yet, so behaviour does not change at this cut.

Soul closed it after one fix batch. The first pass found every commit and
post-fence abort wedged by a deadline nothing cleared; Engine tests now
drive admissions and aborts to Complete. 204 tests. Filed forward: B3 must
give Odin its own readiness class before any route-proof admission ships,
or boot wedges; an unreadable history.cc must fail closed (B1 fix batch).
Back up control.cc before deploying: an older binary refuses a v4 store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 05:08:50 +02:00
Meta Vens
4d8ac7a7e8 Drive the pre-fence abort through the Engine (F0 fix)
advance_pre_fencing_abort survived cargo-mutants as Ok(()): the earlier
pre-fence abort test ran with candidate cleanup already complete. The seeded
transaction now takes its phase, so a Warming transaction is aborted and
driven to Complete through the stub workload port, and the abort's terminal
write is checked to carry no deadline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:50:29 +02:00
Meta Vens
945cb0b322 Count undecodable history, refuse a route-proof Odin authority fallback, test the F0 rules (F0 fix)
History: read_history_transactions dropped undecodable records with .ok(),
which undercounts continuity restarts for backoff. decode_history_transactions
now returns each failure with its key and reason, and the reader prints every
one plus a count. An unreadable history.cc is reported instead of read as
empty. The pre-c848459 IsolationEvidence shape needs no Legacy arm: that
commit made the enum untagged with a Linux variant of the same six fields, and
a test now decodes the old shape under both MessagePack encodings to prove it.
The Legacy structs' stated death condition now says history.cc is never
migrated or pruned, so they die only when history is.

Odin authority: current_odin_authority fell back to the bootstrap key when the
admitted Odin generation carried none (route-proof readiness). It now refuses.
OdinReceipts.authority, read only by tests, is deleted.

Tests through the production path for the rules cargo-mutants showed
unexercised: ControlSnapshot::read of a lifted legacy generation; the
route-proof arm of AdmittedGeneration::validate and route-proof shape checks;
Odin readiness, lease adoption, route supervision and continuity backoff
validation; the admitted half of max_odin_sequence; and the receipt-equality
comparisons in AwaitingReady and Routing, driven through the Engine.

The awaiting-ready default gets a comment that it is unsafe for stateful
targets whose migrations run in AwaitingReady; B5 owns the fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:25:20 +02:00
Meta Vens
fd75014b6e Route every phase write through one owner that sets the deadline with it (F0 fix)
Every admission commit and every post-fencing abort failed validation:
transition() stamped a phase deadline on entry to Fencing..Committing and
nothing cleared it, so the record cloned into Complete still carried a
Committing deadline and validate() refused it. The first deploy or continuity
past Fencing wedged in Committing; a post-fence abort never completed.

DeploymentTransaction::enter_phase is now the only writer of phase. It sets
the phase, the update time and the deadline together, so a record outside
Fencing..=Committing carries no deadline by construction. transition(), both
abort completions, the commit, rejected() and Starting/Warming all use it.

Wall-clock ordering rules are deleted: entered_at <= updated_at, created_at <=
updated_at and promoted_at <= updated_at each wedged a transaction after an
NTP step backwards, since same-phase writes stamp now_millis(). The deadline
is entered_at + duration, fixed at entry.

The Engine takes its systemd workload port as a trait object
(Engine::open_with_systemd_workload), which lets three Engine-level tests drive
a continuity transaction from Fencing through Leasing, AwaitingReady, Routing
and Committing to Complete, and a post-fence abort to Complete, with real
Odin-signed evidence and no systemd, nginx or network. deployment_plan's test
fixtures become pub(crate) for them.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 03:11:21 +02:00
Meta Vens
5d8eb533ab Pin stored_at preservation in the legacy migration test
The fixture envelope's stored_at now differs from the record's own update
time, so a migration that re-stamps the envelope fails the test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:28:14 +02:00
Meta Vens
09222ad8eb Bump admitted generation to v3 and transaction to v4 with typed readiness (F0)
One migration for the route-continuity foundation. No decision reads the new
state.

- ReadinessEvidence (OdinCorrelated | RouteProof) replaces the bare Odin Ready
  receipt on both records. The class a target must collect is derived from its
  Expected (ReadinessClass::of): a shared-infrastructure odin.verse-rendezvous
  dependency means Odin-correlated, none means route-proof. It is not stored.
  RouteProof evidence is refused for a target that declares Odin.
- Generation carries route supervision state (challenge times, failures,
  degraded marker, actuation window) and continuity backoff. A new generation
  inherits the actuation window and backoff and starts unobserved and
  undegraded. route_repair_started_at stays readable.
- Transaction carries the current phase deadline (stamped by transition from
  the plan's frozen durations) and lease adoption evidence.
- FailedAfterFencing names its recovery: RestartAdmitted, RestoreIncumbent
  (the default for older records) or OperatorRequired{reason}.
- Every v2 generation lifts as Odin-correlated with its receipts unchanged;
  v2 and v3 transactions lift through LegacyDeploymentTransaction. The boot
  migration rewrites both, preserving stored_at, and history reads lift the
  same way. The Legacy structs die when no older record can remain.

Fixtures under tests/fixtures/idunn-control-legacy are the bytes the 9001b58
encoder wrote for records built through its own constructors.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:14:47 +02:00
Meta Vens
6fcc1ba6f3 Freeze per-phase deadline durations into the compiled plan (F0)
An operator binding may now declare a [deadlines] table naming any of the
five post-fencing phases (fencing, leasing, awaiting_ready, routing,
committing) in seconds, 1..=86400. compile_deployment_plan resolves the
binding over Idunn's defaults and freezes the result in the plan, so a later
change to a default cannot move an in-flight plan.

The plan becomes schema v3, with the frozen durations as a trailing field.
A v2 plan keeps its nine-field encoding (the field is skipped when absent),
so every existing plan id still verifies; it runs on Idunn's defaults at read
time. Nothing reads the durations yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 02:14:47 +02:00
Meta Vens
4a9fe87787 Import incarnation_key into the continuity projection tests
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 01:32:59 +02:00
Meta Vens
44277698c3 A continuity abort resolves its own projection (route continuity B1)
A continuity candidate shares the admitted Expected key. Both abort intents
marked its topology reconciliation Skipped, so a failed continuity left its
candidate activation projected under the admitted key while the admitted
receipt named the older one, and a later deploy's post-fence demotion refused
as substituted.

DeploymentTransaction::abort_topology_reconciliation is now the one rule for
both intents and pre-fence validation: a deployment owes the projection when it
published its Expected, a continuity owes it when it issued an activation.
reconcile_failed_candidate_projection is the one resolution for both abort
paths: a deployment withdraws its own key, a continuity demotes the shared key
to Expected-only with its own exact activation and never withdraws the
Expected. The post-fence incumbent demotion stays deployment-only and exact.

Tests read the persisted topology store after the production
publish/demote sequence.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 01:30:19 +02:00
Meta Vens
87fa57dd8c Revert "Recover fenced rollback from current activation"
This reverts commit a148802cb6.

Rollback after fencing adopted whatever activation was projected under the
shared Expected key, so a continuity abort's leftover activation could be
demoted by a later deploy. The drift is closed at its source by the next
commit: a continuity abort demotes its own exact activation. Rollback demotes
the admitted generation's exact activation again.

The doc paragraph is dropped; the 9f00e7a paragraph above it stays until B4.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 01:30:09 +02:00
Meta Vens
b619f9875c idunn: route supervision observes; it actuates only on drift (S1)
Delete the stale-observation -> repair-intent -> restore path from
supervise_admitted_route. An exact fragment with a stale observation now
yields one route challenge and nothing else. restore_admitted_membership
returns before nginx -t, the private-mount unit, ufw and reload when the
fragment already equals the admitted membership. route_repair_started_at
stays readable but no longer decides anything.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 00:56:30 +02:00
Meta Vens
8ae00a1244 Allow Idunn to cancel stateless candidates 2026-09-28 17:59:02 +02:00
Meta Vens
29946bbe66 Wait for Idunn stable route listener 2026-09-28 17:29:25 +02:00
Meta Vens
ae8ac45df4 Treat process title as launch evidence 2026-09-28 17:11:37 +02:00
Meta Vens
5e94eb82f0 Name native identity fields that drift 2026-09-28 17:02:46 +02:00
Meta Vens
e0f514745a Revert "Expose current pre-fence cleanup blocker"
This reverts commit 4200330fcc.
2026-09-28 17:00:38 +02:00
Meta Vens
4200330fcc Expose current pre-fence cleanup blocker 2026-09-28 16:48:39 +02:00
Meta Vens
e976d32243 Name fields when workload identity drifts 2026-09-28 16:41:57 +02:00
Meta Vens
865190611f Accept dynamic primary group in workload audit 2026-09-28 15:19:52 +02:00
Meta Vens
9f00e7ab84 Replay admitted provider receipts at admission time 2026-09-28 14:47:55 +02:00
Meta Vens
7a61c77420 Revert "Continue commit from refreshed topology receipt"
This reverts commit 55e5d559d6.
2026-09-28 14:20:26 +02:00
Meta Vens
55e5d559d6 Continue commit from refreshed topology receipt 2026-09-28 14:18:22 +02:00
Meta Vens
a148802cb6 Recover fenced rollback from current activation 2026-09-28 13:56:05 +02:00
Meta Vens
9475dfa742 Land the fourth Cut 1 fix batch (S4-1 through S4-6)
Soul's fourth pass found the symlink resolver newly exponential
(resolve_frozen_source_symlink recursed per path component and
re-resolved every shared prefix from scratch, so an n-link
self-doubling chain cost 2^n real filesystem resolutions), the
streamed hash unpinned, three of the five S5 guard tests not killing
their own guard, all three S1 fsck guards unpinned against
already-local objects, no test proving a legitimate in-root ".."
freezes, and the symlink writer still buffering a whole blob for a
120000 entry.

S4-1: resolve_frozen_source_symlink is now an explicit iterative walk
(a work-stack of frames standing in for the old call stack) with
per-symlink memoisation and one shared traversal budget charged per
distinct symlink read, not per nesting depth. Soul's n=20 doubling
chain (a0 -> ., ak -> a{k-1}/a{k-1}) now resolves in milliseconds
instead of 43.7 s, and freezes rather than refusing: nothing in the
chain dangles or escapes the root, matching S6.

S4-2: two trees differing only past the first 64 KiB chunk now assert
different digests, killing the zero-length-buffer and first-chunk-only
mutants cargo-mutants found.

S4-3: the duplicate-path, ensure_frozen_directory containment, and
create_dir guards each get a fixture that only that guard refuses --
calling git_tree_entries directly (before any fsck runs) for the
duplicate path, and pre-seeding created_dirs with state a second,
independent bookkeeping pass could produce for the other two.

S4-4: Soul's eleven already-local hostile fixtures (dup-trees, six
.git look-alikes, an unsorted tree, mode 120000 on a tree, mode 040000
on a blob, a zero-padded mode) land as tests against freeze_exact's
own explicit fsck. A second test proves freeze_exact's own fetch fsck
flag is what refuses a hostile revision that was never local, by
asserting which context wraps the error.

resolve()'s own fetch fsck flag is still unpinned: exercising it needs
a real HTTPS origin (OperatorBinding::validate requires one), and
url.insteadOf does not work around that -- `git remote get-url` expands
the rewrite, so it can never satisfy ensure_checkout's own
origin-equality check. Confirmed by hand against a local git; a real
smart-HTTPS harness like Soul's S1 rig is what closing this needs.
Recorded as a gap, not faked with a broken test.

S4-5: a legitimate in-root ".." link (two hops up from a nested
directory, back to a file at the root) now has a test proving it
resolves and freezes, not just that an escaping "..' refuses.

S4-6: materialize_tree_raw refuses a 120000 entry whose blob exceeds
the platform's PATH_MAX before reading it, for both the streamed-blob
and the shares-content-with-a-regular-file cases.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 20:30:12 +02:00
Meta Vens
f540d50835 Fix S1 test: use a safe.directory-scoped git for post-chown commit-tree/update-ref
git_at runs plain `git` with no `-c safe.directory=*`. By the time the
hostile commit is built, the origin repo is already chowned to the
unprivileged fixture identity, so root's plain git refuses it as dubious
ownership. Use the same safe.directory-scoped helper the duplicate-tree
fixture above already uses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:28:22 +02:00
Meta Vens
ad31f51273 Fix S1's fetch-guard test order: establish the checkout on the good commit first
The test was moving `main` to the hostile commit before the first checkout
existed, so the initial (also fsck-guarded) clone refused it, never
exercising the fetch line S1 actually changed. Establish the checkout on the
good root commit first, then advance `main`, matching the real scenario: an
already-fetched deploy target whose admitted ref later moves to a hostile
commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:27:04 +02:00
Meta Vens
748e9f3dfd Fix the fix batch: S6 test against the new symlink split, S1 test around the HTTPS-only binding
frozen_source_rejects_a_symlink_that_escapes_the_transaction_tree asserted
against harden_frozen_source, which no longer resolves symlink chains (S6
moved that to validate_frozen_source_symlinks, against the tree's final
published path). Point it at the right function.

resolve_then_freeze_refuses_hostile_trees_through_the_production_path could
not go through the literal SourcePort::resolve()/freeze() trait methods:
OperatorBinding::validate() requires an https:// origin unconditionally, and
this suite has no HTTPS Git fixture. Replaced with a fetch that reproduces
resolve()'s exact invocation (this fix's -c transfer.fsckObjects=true
included) against the checkout, the same approximation Soul's own probe used,
followed by the real freeze_exact as defense in depth.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:25:32 +02:00
Meta Vens
57ecd43d6b Land the third Cut 1 fix batch: fsck on every fetch, published-location symlink checks, streamed hashing
S1: guard resolve()'s admitted-ref fetch with transfer.fsckObjects=true (it
had none), and have freeze_exact additionally run an explicit `git fsck`
against the selected revision so objects that were already local -- not only
freshly transferred ones -- get checked too. Refusal now happens through the
production path (resolve() then freeze()), not only a hand-built fetch.

S5: behavioural tests for six guards Soul's third pass found unpinned: root
ownership in harden_frozen_source, an exact duplicate leaf path, the
ensure_frozen_directory root-escape check on its own, the writer's pass-1
create_dir semantics, and copy_artifact's containment check. (The sixth,
case-sensitive ancestor alias, goes away under S7.)

S6: symlinks are now judged against the tree's real, final published path,
not `.partial`, the temporary name the tree is built under before the atomic
rename that publishes it. A dangling link whose target stays inside the root,
resolving every component that exists, is accepted; one that escapes is
refused, dangling or not. harden_frozen_source_tree defers symlink chain
resolution; the new validate_frozen_source_symlinks walk runs once, right
after the rename, and validate_frozen_source_symlink (used by observe_frozen
too) now shares the same dangling-tolerant resolver, so freeze-time and
observe-time agree.

S7: delete the case-folding collision/alias refusal in
refuse_conflicting_tree_entries. The host is case-sensitive ext4, the check
was inconsistent (leaf paths only, plain to_lowercase), and .git look-alikes
are now refused by git fsck itself (S1), independent of case. The exact-name
alias case (a leaf sharing a root with a tree recursing into it) stays
refused at write time by ensure_frozen_directory, which never writes through
an existing entry.

S9: hash_frozen_source_tree streams each file through the hasher in 64 KiB
chunks instead of fs::read-ing it whole, so peak RSS no longer scales with
the largest file in the tree.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 19:21:37 +02:00
Meta Vens
82e0edfaf7 Land the second Cut 1 fix batch: F2/F3 write-outside-root regression, F4 hardening, F5 chain-aware symlink check
Soul's second pass on the Cut 1 fix batch found a new regression (a
duplicate/aliased tree name lets the raw freeze write outside its root) and
three pre-existing gaps: two redundant hardening passes with no pin, a
lexical (not chain-resolving) symlink escape check, and unpinned Docker argv
combinations. F1's perf finding (40-80x slower than the old git archive path)
is addressed here too.

F2/F3 (write-outside-root, two required layers):
- git_tree_entries now calls refuse_conflicting_tree_entries, which refuses
  a tree where two leaf paths collide case-insensitively, or where one leaf
  path is a case-folded ancestor of another (the exact shape
  `git hash-object --literally` lets through: a symlink and a subtree
  sharing one name). The exact-revision fetch and both clone paths
  (ensure_checkout, materialize_gitlink_raw) enable transfer.fsckObjects.
- The writer itself never trusts that check alone. ensure_frozen_directory
  is the only way materialize_tree_raw obtains a directory to write into: a
  path must already be a directory this freeze created (tracked in
  created_dirs, shared across the superproject and every Gitlink), or must
  not exist at all; anything else is refused rather than traversed.
  Symlinks are written in a dedicated last pass, after every directory and
  regular file the tree needs already exists.

F4: freeze_exact now calls harden_frozen_source once, last, instead of
harden then a separate validate_frozen_source pass over the same tree.
validate_frozen_source stays as the read-only re-check observe_frozen uses
on an already-hardened tree it did not just write.

F5: validate_frozen_source_symlink resolves the whole symlink chain with
fs::canonicalize instead of lexically counting ".." components, which
undercounts a chain that loops back through another symlink (D -> .,
L -> D/D/.../../../../etc/passwd). copy_artifact (the one place besides
freeze that reads through a possibly-attacker-controlled tree) gets the same
whole-chain resolution: it now takes the artifact's containment root and
refuses a source that resolves outside it, closing the gap where an
intermediate symlink in a runner's own build output could be followed.

F1: materialize_tree_raw bulk-fetches every blob a tree needs in one
`git fetch --stdin` round trip (Git 2.36+, no argv-length limit) instead of
letting a blob:none partial clone lazily fetch each one as `cat-file --batch`
reads it. Blobs stream straight to their destination file instead of being
held in a BTreeMap for the whole tree. read_blobs is replaced by
stream_blobs, which drains stderr on a dedicated thread and waits on the
child on every return path, including an error from the per-object
callback (F9).

F6: pins four previously-unpinned Docker argv/environment combinations
(N1: --read-only survives a Named network plus a cache mount; N2: a secret
mount stays read-only alongside a plain environment entry; N3: the source
stamp is never dropped alongside a secret and a plain environment entry;
N5: an LFS pointer file sets contains_lfs_pointers).

Tests: 142 to 152. cut1.psd1 grows by 12 entries (5 for F1-F5, 4 for F6's
N1/N2/N3/N5, and one recording the F2 fsck flag as not yet reached -- see
below); all 28 non-comment entries in the suite now kill.

Discrepancy: cut1-fix-f2-no-fsck-on-fetch could not be pinned. Every tree
this crate's own tools can build that fsck's duplicateEntries check rejects
also has a literal duplicate path or a case-folded ancestor collision that
refuse_conflicting_tree_entries rejects first, unconditionally, before the
fetch's fsck result can matter. The flag stays as defense in depth (fsck's
other structural checks are broader than this crate's own parser), recorded
as not yet reached in cut1.psd1 rather than removed or falsely claimed
unreachable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 18:29:36 +02:00
Meta Vens
43c7a6e46d wip5 fix spy log perms 2026-09-22 17:47:29 +02:00
Meta Vens
f8baa93dac wip3 tests+psd1 2026-09-22 17:46:02 +02:00
Meta Vens
2ee7c225e2 wip2 2026-09-22 17:41:06 +02:00
Meta Vens
95016e817d Land F2: freeze writes blobs raw, per Self's ruling on the Eve survey finding
Self reviewed the F2 survey finding (Eve's gitlink under Ghostlight carries
`packages/*/dist/** text eol=lf`) and ruled it is not a real dependency on an
archive transform: Git stores text blobs as LF, `eol=lf` checks them out as
LF, so no conversion happens. Self verified this directly at Ghostlight's
pinned Eve gitlink revision `672c0c1e9fc828b63752345385727b06c50491d0` — all
26 files under `packages/*/dist/` hash identically between `git archive` and
`git hash-object --no-filters`, and none contains a CR. Stopping for the
survey was correct; the ruling is to proceed.

Survey table (repo, attributes found, verdict), recorded here as ruled:

| Repo (deploy target) | Binding source | .gitattributes findings | Verdict |
|---|---|---|---|
| Ghostlight | idunn/yggdrasil/bindings/ghostlight.toml.in | none | clean |
| Ghostlight's gitlink vendor/eve -> Eve | same binding | packages/*/dist/** text eol=lf | transform present, but a no-op on stored LF content (Self's evidence above) |
| Odin | idunn/yggdrasil/bindings/odin.toml.in | none | clean |
| CodexConnector | idunn/yggdrasil/bindings/codex-connector.toml.in | none | clean |
| Muninn (raven-muninn) | idunn/yggdrasil/bindings/raven-muninn.toml.in | none | clean |
| Heimdall | deployment/idunn-bindings/heimdall.toml.template (a second, previously unlisted binding directory, installed live 2026-09-06) | none | clean |
| Heimdall's gitlink -> CultLib | same binding | * text=auto only (F12) | clean |

`freeze_exact` now writes every tree entry raw from the object store:
`git ls-tree -r -z` for the manifest, `git cat-file --batch` for content, with
no `git archive` anywhere in the path and so no attribute-driven transform of
any kind. Modes and symlinks are preserved exactly as the tree records them.
Gitlinks are frozen the same raw way, through the same primitive, at their
own path. LFS content is never fetched; `freeze_exact` now also returns
whether any materialized file is an LFS pointer (recorded in the frozen
result's own return value — not yet plumbed into `FrozenSourceReceipt`, which
has no field for it; that is follow-up scope, not this fix).

Deleted: `git_archive_into`, `materialize_gitlink_archive`, and the
now-unused `tar_program` field — the archive-and-extract path they built has
no remaining caller. `GitSourceDriver::new` no longer takes a tar program.

Tests:
- `freeze_exact_is_byte_exact_across_every_attribute_transform`: a fixture
  with `eol=crlf`, `export-ignore`, `export-subst`, `ident`, a symlink, the
  executable bit, and a Gitlink. Every frozen file's bytes are asserted equal
  to `git cat-file blob` at the selected revision.
- `freeze_exact_matches_archive_when_untransformed_and_diverges_when_transformed`:
  the deploy-equivalence test Self's ruling asked for. An `eol=lf` file over
  already-LF content matches today's independently-reconstructed `git archive`
  tree exactly; a second file with `eol=crlf` over the same LF content
  diverges from it, so the equivalence check has teeth rather than trivially
  passing for every fixture.
- `exact_git_blobs_become_root_owned_immutable_source_without_git_metadata`
  (renamed from the git-archive-era test) now exercises `materialize_tree_raw`.
- `tools/verify-mutations/cut1.psd1`: `cut1-fix-freeze-skips-gitlinks`
  retargeted at the new call, and a new `cut1-fix-f2-revert-to-git-archive`
  entry that reverts `materialize_tree_raw` to a `git archive | tar` pipeline;
  it must die against the byte-exact test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 16:48:17 +02:00
Meta Vens
7a5d32f02d Land the Cut 1 fix batch: pin every argv branch, one fetch, dead StepPort gone
Soul's pass on Cuts 0-1 found the argv-lowering test pinned only the
default runner path (13 of 14 mutants survived), freeze fetching the
same revision twice, an untested Gitlink loop, dead StepPort/StepOutcome
code, and a stale guide.md line. This lands Self's rulings for all of
it except F2 (the byte-exact freeze): the F2 survey found that Eve, a
Gitlink of the Ghostlight deploy target, depends on an archive
transform (`.gitattributes` eol=lf under packages/*/dist/**), so per
ruling 1 the byte-exact rewrite is withheld and reported instead.

- F1: `docker_run_args`/`ContainerSpec::for_step` are now pinned across
  every network branch (none/bridge/named), the full CPU quota range
  (100/150/250/333/800), cache root present/absent, and secret mounts,
  with dedicated fixtures for the cache-root and secret-file validation
  calls so a mutant that skips either is caught.
- F3: `freeze` fetches the exact revision once, through `freeze_exact`,
  instead of fetching before it and again inside it.
- F5: `freeze_exact_materializes_gitlinks` pins that the Gitlink loop
  actually runs.
- F7: deletes the dead `StepPort`/`StepOutcome` types; `run_step` never
  had a caller, and `docker()` is the one Docker spawn path.
- Stamp override: `ContainerSpec::for_step` now refuses a step
  environment name that collides with the Idunn source stamp, so the
  invariant holds at the one place every runner's environment is built,
  not only at deploy's separate `admit` check.
- docs/guide.md:11 no longer claims the retired generation still runs
  on Yggdrasil.

F8 (the `--cap-drop` negative check) is a reporting fix, not a code
change: `rg -n '"--cap-drop"' src/` now returns 4 hits (1 real site
plus 3 test fixtures, one of which this batch adds), never the "1 hit"
the map's verification text describes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 16:15:38 +02:00