Idunn/docs/reincarnation-map.md
Meta Vens 2f8671ac35 Reincarnation map: cut 2 installed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 02:36:25 +02:00

41 KiB

Reincarnation after reboot: map

Status: map, Imagination, 2026-10-18 (campaign idunn-reincarnation, session idunn-reincarnation-2026-10-18). The ends are in reincarnation-target.md. Questions, rulings, the cut spec and follow-ups live in Eureka's mind; this page holds what has no kind: body facts, the model page, the authority map's rationale and the cut sequence. Line numbers are at 74a7008; src/ is byte-identical to c2477d4, so the earlier anchors still hold.

Cut 2 pass, Imagination, 2026-10-19 (same campaign and session): body facts B13-B18, the cut-2 rows of the model page and the cut-2 parts of sections 3-5 are anchored at 9a0a6a2, whose src/ is byte-identical to ffb4293 (cut 1's head). The B1-B12 anchors above stay at 74a7008.

Read the in-force set with the rulings-in-force and open-follow-ups recipes for root idunn-reincarnation; the cut in flight is the specs-with-no-report recipe. The first spec (cut-c1-durable-stop.r1) was withdrawn when the operator ruled that continuity never stops; its replacement is cut c1-never-stop. Cited from idunn-watchdog: exhaustion-close-rule-op and paging-conditions.

1. Body facts

Evidence: the idunn-yggdrasil journal from 11:40 UTC (94,670 lines), the per-unit journals of the streampixels-service candidates and of Odin (idunn-odin-7cfe5cd1…), idunn status and idunn status --command <id> on Yggdrasil, journalctl -u nginx, and the source at c2477d4. All times UTC, 2026-10-18.

B1. The reboot did not stop continuity from re-materializing anything. Every streampixels-service continuity attempt minted a fresh transient unit from the installed release sha256-df663f0a…: 54 continuity commands from continuity-85887ef2 (11:46) to continuity-52bbbd66 (19:57), most listed by idunn status as continuity-… failed systemd unit idunn-streampixels-service-<hash>.service is not running. A continuity transaction is a full DeploymentTransaction built by from_continuity (control_plane.rs:1678) from the admitted plan, sealed release, installed release and Expected. It runs every phase: Starting, Warming, Fencing, Leasing, AwaitingReady, Routing, Committing. The absent admitted unit 461c6443… was never the problem. Each new candidate was.

B2. Every candidate died of a dependency, not of itself. The streampixels-service process exits about 30 s after start when Odin does not admit its initial presence. Unit 2bc02905… at 11:47:21 logged "StreamPixels service failed to enter its Idunn incarnation: Error: Odin did not admit the initial Idunn runtime presence ... Timed out waiting for CultNet RUDP response after 5000 ms." The same is logged by 42ecf363… at 13:47:13, 03acf44d… at 19:47:14, 45276008… at 19:49:54 and 599b705b… at 19:51:18. Odin's RUDP endpoint is rudp://10.77.0.1:17871, which nginx proxies. nginx failed at 11:46:26 ("bind() to 10.77.0.1:17871 failed (99: Cannot assign requested address)") and started at 19:46:24. Between 11:46 and 19:44 Odin's journal shows no presence message from streampixels-service at all: none arrived. After nginx returned, the first one arrived at 19:49:25 and was rejected: "runtime presence has no exact current activation and provider anchor". The attempts minted at about 19:53 and 19:55 got past Fencing and then failed "the Leasing phase passed its deadline" (FailedAfterFencing, RestoreIncumbent). Other early failures: "observed write lease is not the current Idunn grant" (11:46:47, the first attempt after boot), and three "reading process stat /proc/N/stat: No such file or directory" races.

B3. Every attempt was charged at mint. record_restart(now) runs inside the mint CAS (control_plane.rs:6156-6157). Nothing reads a continuity's outcome back into the log. The only exemption anywhere is HostUnobservable (:5944), and it applies to an observation, not to a candidate. So the 6-per- hour window (CONTINUITY_RESTART_ATTEMPTS = 6, window 3,600,000 ms, :71-72) filled at 11:50:52 and refilled every hour as entries slid out. The bursts are in the journal at :47, :48, :49, :51, :52 and :53 of every hour, 39 times "Idunn stopped restarting admitted streampixels-service". This answers Q2: the nginx outage counted against the body's budget, and the place that decides it is the unconditional charge at :6156.

B4. The deployment was killed by a window slide, not by a live continuity. up-3ad6b05b (release sha256-67e227b2…, a new release) was recorded at 20:24:51. It was still before Fencing at 20:46:41, when it was completed FailedBeforeFencing with "admitted incumbent failed before candidate fencing; deployment yielded to continuity: systemd unit idunn-streampixels-service-461c6443….service is absent". Two seconds later continuity minted tx-0804185a with the OLD release df663f0a…, and it was admitted. The restart that left the window was minted at about 19:46:40 (its unit 03acf44d… started at 19:46:42), exactly one window before the abort. The mechanism, from the source:

  • :5805-5821: a live transaction for the target is the blocker. From Fencing on (or Starting for stop-then-start) supervision skips the target entirely.
  • :6033-6061: with a blocker present, continuity never restarts and never reports. It may abort the blocker, and the predicate for that is Deploy && !supervision.restarts_exhausted(now) && phase < Fencing && no abort && not held (:6045-6050).
  • :6062-6084: only with no blocker does an exhausted continuity open the incident and say "The target is free for a deployment to replace it".

So from 20:24:51 the deployment owned the target (owns_target_authority, phase != Complete, :1772). Continuity deferred to it through the blocker continue at :6060. The deployment deferred to continuity whenever the window had a free slot, through :6046. "Stopped" existed only as a durable incident (incidents.cc, a projection) and a report. The ownership decision read the window. When the window gained a slot, continuity reclaimed the target with no record that it had resumed. This answers Q3: the two owners are the live DeploymentTransaction (it owns while not Complete) and continuity-as-window (it owns whenever the window is not full). Continuity defers to the deployment while any transaction is live. The deployment defers to continuity while the window has room. The "free for a deployment" message names the first ownership, and the code acts on the second.

B5. Fencing an absent incumbent already works. advance_fencing (:6869-6952) stops the exact incumbent and revokes its lease. The systemd driver's stop returns Ok(()) for an absent unit (drivers.rs:4003-4007). A deployment that is not aborted therefore replaces an absent incumbent with nothing to fence, which is target invariant 3's last clause, already true. exact_incumbent (:7556-7569) fails "target incumbent changed after transaction creation" if a continuity commit replaced the generation during the deployment. That is why a phase-split handover needs a rebase (handover-pre-fencing).

B6. The two rejection streams are consequences of a dead admitted generation, not separate faults. "preserved admitted streampixels-service after rejecting topology observation: topology correlation does not bind the current activation" was logged 46,271 times, from 11:46:49 to exactly 20:46:48, about 5,300 an hour. Ghostlight's 42,482 identical lines ran to 19:56:29. Since 20:47, raven-muninn, newly exhausted, shows the same line 621 times. refresh_admitted_topology (:6395-6457) authenticates Odin's correlation against the admitted generation's activation. When that incarnation is dead and Odin's latest record names another one (a continuity candidate's, or a stale one), authentication fails with an Err rather than Ok(false), and :5834-5840 prints it on every pass. The pass order is what does it: route supervision (:5822) and topology refresh (:5834) run before the workload is observed (:5891). So a dead generation is "supervised" every tick. "rejected admitted streampixels-service route continuity: admitted process write lease is no longer exact" (3,615, from 20:02:23 to 20:46:48) followed the two post-fencing aborts at 19:54 and 19:57. Fencing revoked the dead incumbent's lease, "restore incumbent" cannot re-grant a lease to a process that does not exist, and supervise_admitted_route (:6250-6255) then refuses the admitted generation on every challenge. Both streams stopped the moment tx-0804185a was admitted. This answers Q4: both streams follow from one model gap, that a dead admitted generation is treated as a live incumbent. That gap is cut 4. They are not the deadlock.

B7. Ghostlight is a separate fault. Its continuity tx-039b91e9 is wedged in a post-fencing abort. It was "lease-not-adopted" at its AwaitingReady deadline, and the abort withdraws the candidate's lease (:8895-8928) before it stops the candidate (:8929-8946). The candidate unit 6c51e35e… has been running since 19:55:15 and holds the lease's shared lock, so every attempt fails "process write lease is held by a shared-lock consumer" (drivers.rs:4179/4224). The transaction owns the target, so continuity is locked out and the route degraded (53 consecutive failures). This is follow-up ghostlight-abort-lease-before-stop (route-continuity invariant 5), not the stopped-owner deadlock.

B8. Unexplained, recorded. Idunn's record of Odin's stable route reads "degraded-since 11:46:31, consecutive-failures 1, next-challenge-at 11:47:01" about nine hours later. It has not been re-challenged since. This is follow-up odin-route-record-stale. It blocks cut 3. Under budget-never-stop, a failure attributed to a declared dependency must not grow the backoff, and the only dependency-health record Idunn holds for Odin's stable route is this one.

B9. Luna's weekend commits on this path (Q5). None of them caused the deadlock. The yield predicate dates from the route-continuity and watchdog campaigns, not from the weekend. Where they touch:

Commit On the path? Status
9f00e7a replay provider receipts at admission time Yes: the deployment's Sealing compile (advance_sealing → current_ready_provider_tokens) logged "excluded non-current provider ghostlight / raven-muninn: admitted provider's latest receipt is not Ready at its admission time" at 20:24:53 (:8588, :8609) Still live: route-continuity B4 has not landed. It did not block this deployment
29946bb wait for the stable route listener Yes: the route proofs during the outage were "Connection refused" on 127.0.0.1:8831-8833 Kept by route-continuity (honest scope)
8ae00a1 cancel live stateless candidates No: no cancel ran Still present (:2888); route-continuity S2 owns its deletion
ae8ac45, 8651906, e976d32, 5e94eb8 identity and launch evidence Only in observe and stop; nothing here drifted Kept
a148802 No Reverted (87fa57d)

The target forbids building on them. Cut 1 touches none of their lines.

B10. The stop was the only door a deployment had into a crash loop. Source read at 74a7008. A queued idunn up becomes a transaction only in freeze_command, which refuses while any transaction for any of its targets blocks_new_target_mutation (:5741-5747); the command just stays queued. The scheduler tick runs retire_one_terminal_transaction (returns early when it retires), then resume_one_transaction, then supervise_one_admitted_generation, then freeze_one_queued_command (:5414-5423). Today's doubling wait is anchored at the previous attempt's mint (next_restart_at, :1163-1170). So once an attempt lasts longer than its wait, the tick after its transaction is retired mints the next attempt before the freeze runs, and the queued deployment never finds the target free. At c2477d4 that ended when the window filled and continuity stopped. Delete the stop without opening another door and a crash-looping release can never be replaced. The door the cut uses is kubelet's own anchor (docs/research/restart-budget-prior-art.md §1.1, the exited container's FinishedAt): the wait runs from the first supervision pass that finds the target dead and free. That leaves at least one wait (5 s or more) with no live transaction, and the freeze runs every tick. This is reasoning over the source, not a probe. The cut pins it with a tick-level test.

B11. The yield cannot outlive the stop. With the window term gone from :6046, the yield aborts every pre-fencing Deploy whenever the incumbent is dead (:6045-6059). For a crash-looping release the incumbent is always dead, so the deployment that would fix it is aborted at the first supervision pass after it is frozen. That is the 2026-10-18 failure: on that day the operator's idunn up existed to replace a release that would not start. Ruling handover-phase-split already says nothing yields before Fencing. So cut 1 deletes the whole yield. Continuity keeps deferring to any live transaction for its target (:6033, the continue). Cut 2 keeps the incumbent alive alongside the build. Between them, a dead incumbent stays down while a pre-fencing deployment over it builds. That is bounded by the deployment's own deadlines, and the deployment then fences it as nothing (B5).

B12. The doubling function already exists once. backoff_wait (:4000-4007) doubles from a base up to RESUME_BACKOFF_CEILING_MILLIS, and has one caller (back_off, :5621). Continuity's wait is a second spelling of the same rule (5_000 << (live.len() - 1), :1169). The cut gives backoff_wait a ceiling parameter and deletes the second spelling. #[cultcache(key = N, default)] is how an absent key decodes as its default (precedent DeploymentTransaction key 34, :1552-1557). New TargetSupervision keys use it, because every record on Yggdrasil predates them.

B13. Two live transactions for one target make the control store unreadable. Source read at 9a0a6a2. ControlSnapshot::validate refuses "multiple transactions claim current-incarnation authority for one target" over owns_target_authority (:3245-3254), and that predicate is just phase != Complete (:1896-1898). Every reader of the store goes through it: the scheduler tick, idunn status, cancel, expire. So if supervision minted a continuity beside a pre-fencing deployment today, the next read would fail the whole engine. Coexistence needs the identity rule changed first, and that is the cut's first commit.

B14. One predicate answers four questions, and the phase rule lives in one caller. blocks_new_target_mutation (:1903-1909, live or post-commit cleanup pending) is read by the freeze busy check (:5934-5942), the supervision blocker (:5998-6001, a single .find), candidate-port reservation (:7997) and status (:4613). owns_target_authority is read by validation (:3249) and by sibling ordering inside one command (:3384). Neither looks at the phase. The only phase-aware ownership rule is inline in supervision (:6002-6012, owns from Fencing, or from Starting for stop-then-start). The freeze refusal is why a profile: command needs every target free in one tick (follow-up profile-deploy-joint-gap).

B15. What a pre-fencing deployment reads about its incumbent. Warming's semantic warming check uses the incumbent's lease digest (:6904-6905, is_semantic_warming :9530-9560: Odin reports the candidate's one disagreement as the lease the incumbent holds). Route preflight uses the incumbent's route observation (:7026-7047). Isolation is proven against the incumbent's workload (:7050-7076). Each goes through exact_incumbent (:7768-7781), which bails "target incumbent changed after transaction creation" once a continuity commit replaced the generation. Sealing, and Starting for candidate-then-promote, read no incumbent. Stop-then-start's Starting stops it (:6857-6863), which is already the owner's act. The Sealing-to-Starting write is a replace_transaction at :6806-6808, not transition (:8914-8927), so the two edges at which a transaction can take a target are written by two different functions today.

B16. The commit already guards the incumbent; nothing guards the handover. advance_committing expects the admitted envelope it read (:7701-7702, CAS :7743-7764), so a deployment can never commit over a generation that moved under it. Entering Fencing is an unguarded replace_transaction of the transaction alone. Continuity's mint CAS (:6371-6400) guards its command, transaction and supervision record, and no other transaction. The store accepts read-only expectations: an expected entry with no replacement only guards (CultLib packages/cultcache-rs/src/lib.rs:1327-1368). One engine holds the store (ProcessLock::acquire, :5514), but the CLI writes transactions too: cancel sets a pre-fencing abort on a live deployment (:3014-3083). So a handover decided by one CAS over the target's records is exact, and a tick order is not.

B17. What coexisting candidates share, and what they do not. Continuity reuses the admitted plan, so its candidate port is the admitted generation's, already reserved (:7988-7992); a deployment picks another. Topology is keyed per incarnation (:8495-8498): a continuity publishes under the admitted key, a deployment under its own. No candidate holds a write lease before its own Leasing, which follows Fencing, so two pre-fencing candidates cannot both write. Route actuation by a continuity is always Survival, never refused (:4870-4876); a deployment's Forward cannot starve it. The Odin replay floor max_odin_sequence (:3399-3435) spans every transaction of the target and the admitted cursor; incumbent refresh and candidate warming already share it during every deployment today, so the replay check (:9498-9501) gains no new exposure. Stop-then-start is declared only by route-less host-actuator bindings (deployment.rs:756-764, :1245-1249), and no control-plane test builds one (grep StopThenStart in src/control_plane.rs: the definition site only). It is live on Yggdrasil: raven-muninn declares it (git grep stop-then-start in gamecult-ops: idunn/yggdrasil/bindings/raven-muninn.toml.in). So the Starting edge of the handover carries production traffic, and its engine-level proof is owed before the closing install (follow-up stop-first-handover-fixture).

B18. A pre-fencing error is an abort. resume_candidate turns any Err from a pre-fencing step into begin_pre_fencing_abort unless it is LockContended (:5768-5783). So anything the cut makes a deployment wait on must return Ok (a gate wait, record_gate_wait :8170-8182), or the wait becomes the yield that ruling handover-phase-split forbids. The same path is how a superseded continuity ends itself, which is wanted.

2. Model page

One row per persistent kind the campaign touches.

Kind What names it Lifecycle Who decides
Paging log: TargetSupervision.continuity_restarts (control.cc, idunn.target_supervision.v1, key 3) key = target The newest 6 charged attempt times, ascending, read through a sliding 1 h window. Outlives every generation. After cut 1 it is the paging threshold only (restarts_page): it opens and holds the ContinuityExhausted incident and is shown in status. It gates no mint, wait or yield Continuity supervision, pushed in the mint CAS
Attempt streak (new, cut 1: key 7, default) target Charged attempts since the admitted body last ran a sustained run. +1 in each charged mint CAS. Reset to 0 when the dead generation was admitted after the last charged attempt and ran ≥ 600 s before it was found dead. Saturating, and the wait is capped, so it can be unbounded Continuity supervision only
Backoff anchor (new, cut 1: key 8, default) target Written on the first pass that finds the admitted generation dead and the target free after an attempt. The next attempt is due at anchor + backoff_wait(5 s, streak - 1, 300 s). Cleared by every mint. Stale, and ignored, once older than the admitted generation's admitted_at Continuity supervision only
Re-materialization bound (new, cut 1: key 9, default) target Set by the one attempt that is not a backoff step, when the driver reports a reboot loss. Spent while it is not older than the admitted generation's admitted_at Continuity supervision, from the workload driver's evidence
Projection deferral (keys 4-5) target Unchanged: a failed demotion defers the next mint 30 s. It spaces a re-materialization too Continuity supervision
Admitted generation (idunn.admitted_generation) key = target, generation_id = generation-<tx> Replaced atomically by every committed deployment or continuity. A dead incarnation keeps its record until replaced The committing transaction
Incident ContinuityExhausted (incidents.cc) condition + subject (target) Opened at 6 charged attempts in the hour while the target is dead and free. Closed when the target runs and the window no longer holds 6 (exhaustion-close-rule-op). Retired after retention. The name is the wire name and stays, though nothing is exhausted any more Idunn's incident writer, a notification projection: never read by ownership
Transaction and pre-fencing abort (idunn.deployment_transaction.v4, history.cc once terminal) tx-<uuid> and its command_id (up-… or continuity-…). Keyed by transaction, so a continuity and a deployment for one target are two records with no shared key Sealing … Complete. A pre-fencing abort completes FailedBeforeFencing, a post-fencing abort FailedAfterFencing with a named recovery. Retired to history. After cut 2, per target at any instant: at most one live Deploy, at most one live Continuity, at most one owner (validated by ControlSnapshot::validate, B13) The scheduler advances it. After cut 1, continuity supervision never aborts a transaction; the transaction's own steps do. After cut 2, a continuity whose target was taken, or whose source generation was replaced, aborts itself (superseded)
Target ownership (cut 2: derived, no record) target A transaction owns its target from takes_target_at() (Fencing, or Starting for stop-then-start) until Complete; a Complete transaction with pending post-commit cleanup drains it. Entered only through the handover CAS (Engine::take_target) Derived from the phase. The handover CAS is the one writer of a phase entry into takes_target_at()
incumbent_generation_id on a transaction (key 9, unchanged format) the generation's id Deploy: set at freeze; before the handover it names the generation its incumbent-relative evidence was proven against and is rebased when the admitted generation moves (clearing that evidence); pinned by the handover CAS, exact from then on. Continuity: its source generation, never rebased Deploy: its own pre-handover rebase step and the handover CAS. Continuity: the mint
Unit materialization (systemd transient unit idunn-<target>-<hash>) the activation's hash Created by Starting. Lost on a host reboot (/run/systemd/transient). A failed unit stays loaded; a cleanly stopped one is collected The workload driver actuates. Whether a loss was a reboot is the driver's evidence (WorkloadPort::lost_to_host_reboot, cut 1)

No cell is open. Both forks are ruled (budget-never-stop, handover-phase-split). The numbers that budget-never-stop left open are the defaulted ruling continuity-backoff-defaults. Cut 2 opens one question, fencing-over-owning-continuity (does a deployment ready to fence wait for a continuity that already owns the target, or preempt it); the spec is written for the recommended answer, wait.

3. Authority map: continuity and deployment

The authority map for cut 1 is in its spec. This section is the campaign-level map, and cut 1 is one step of it.

  • Owner. Exactly one authority actuates a target at each instant. A live DeploymentTransaction owns the target from Fencing on, and a stop-first rollout owns it from Starting. Before Fencing (handover-phase-split), a deployment owns only its private candidate, and continuity owns the incumbent slot. Continuity owns the admitted generation's survival and never stops. When it next acts is a function of its own record: the streak, the backoff anchor and the re-materialization bound.
  • Inputs. For continuity: the admitted generation (id, admitted_at, workload, plan), the workload observation and the driver's reboot evidence, its own record, and whether a live transaction blocks the target. After cut 3 it also reads Idunn's dependency-health observation. For a deployment: its own plan, its brake, and at Fencing whatever generation is admitted at that moment (cut 2).
  • Outputs. Continuity: mints, either charged or the one re-materialization; backoff anchors; deferrals; and, as notifications only, the ContinuityExhausted incident and its once-report. Deployment: a new admitted generation, or an abort that names its recovery.
  • Derived state. restarts_page(now) (6 charged attempts in the hour) is derived from the paging log and is notification-only: incident open and close, and status. The next attempt's due time is derived from the anchor and the streak. The incident and idunn status are projections.
  • Forbidden writers. The paging window gates no mint, no wait and no yield. Continuity supervision never aborts a deployment: begin_pre_fencing_abort has no caller in it. No deployment path writes continuity's fields. The incident store decides nothing. No stopped state exists: no field, and no branch that ends a pass because a count is reached.
  • Shared paths. A crash restart, a restart after a sustained run, and a reboot re-materialization all mint through the one CAS (:6158-6183). The dead-and-free pass is the only writer of the backoff anchor. Every deployment, operator or scheduled, enters through freeze_command and advances through advance_transaction.
  • Deletion line. Cut 1 deletes the yield (:6034-6059), the exhaustion continue and its "free for a deployment" text (:6062-6085), and the mint-anchored wait (:1160-1184). Cut 2 (anchors at 9a0a6a2) deletes owns_target_authority and blocks_new_target_mutation (:1894-1909), the inline phase rule and the single-blocker .find in supervision (:5998-6012), the blocker continue (:6229-6233), the freeze busy check (:5934-5942), the one-live-transaction-per-target validation (:3245-3254), and the "target incumbent changed after transaction creation" text (:7779).

Cut 2: the two slots and the handover

Each target has one incumbent slot: its AdmittedGeneration, the body running it, its write lease and its stable-route membership. Each live transaction has its own candidate slot: its incarnation key, activation, candidate port and unit.

  • Owner. Before any transaction takes the target, continuity supervision owns the incumbent slot: it alone decides to mint a restart. From the handover CAS until Complete, the one transaction that took the target owns it, fencing, leasing, routing and committing. Its commit hands the slot back to supervision as a new generation. A transaction owns its own candidate slot from mint or freeze until it is terminal. A pre-handover deployment owns nothing else.
  • The handover. Engine::take_target is the only writer of a phase entry into takes_target_at() (Fencing, or Starting for stop-then-start), for both kinds. It is one CAS. It expects the transaction's own envelope, the target's admitted envelope (the one incumbent_generation_id names, or absent), and every other non-terminal transaction of the target, each unchanged, and it writes only the transaction. Of two transactions entering from one view, exactly one CAS succeeds. The loser's next step re-reads: a deployment gate-waits for the owner, and a continuity aborts itself as superseded.
  • Inputs. Continuity: as cut 1, plus whether any transaction owns or drains the target, or a continuity is live for it (the mint gate). A pre-handover deployment: its plan, its brake, and the admitted generation as it is now, which it rebases onto. The handover: the target's records above.
  • Outputs. The handover writes the new phase. A rebase rewrites incumbent_generation_id and clears the evidence proven against the old incumbent: warming, route_preflight, isolation. A superseded continuity writes its own pre-fencing abort.
  • Derived state. owns_target() is derived from the phase and takes_target_at(). drains_incumbent() is derived from the post-commit cleanup. The freeze refusal is derived from "a live Deploy for this target". incumbent_generation_id on a pre-handover deployment is the generation its evidence was proven against, not a pin.
  • Forbidden writers. No path enters takes_target_at() except take_target. Continuity supervision aborts nothing, and mints nothing while another transaction owns or drains the target or a continuity is live. A deployment never aborts because another transaction owns the target: it gate-waits (B18). A continuity is never rebased. Freeze reads no continuity state. No new persistent record decides ownership: it is the phase.
  • Shared paths. Candidate-then-promote's Warming-to-Fencing and stop-then-start's Sealing-to-Starting go through one phase-entry seam, which calls take_target exactly when the entered phase is takes_target_at(). Deployments and continuities use the same CAS. A single-target and a profile: command use the same freeze rule.

Demotions, named: incumbent_generation_id is no longer a pin before the handover; it is derived from the admitted generation by the rebase. blocks_new_target_mutation is no longer an owner; it splits into is_live, owns_target and drains_incumbent, each read by the consumer that asks that question. The freeze busy check is no longer a target-ownership gate; it guards only one live deployment per target.

Demotions, named: restarts_exhausted is no longer an owner of anything. It is renamed restarts_page, and it opens and holds the incident. The ContinuityExhausted incident was never read as "stopped", and there is now no stopped state for it to stand for. The "free for a deployment" report is deleted: every target is free for a deployment between attempts (B10).

4. Rationale

The fault is a split brain, not a missing rule. Continuity's "I have stopped" was written in two places, a durable incident and a log message, and neither was what the ownership decision read: the yield re-derived "stopped" from a clock. The withdrawn first spec moved "stopped" into control state. The operator's ruling is better, and it is also smaller: there is no stopped state. The window it was derived from becomes a pager. Every case the stop existed for is then handled by an owner that already exists. A broken release is replaced by a deployment, which gets in through the gap after each attempt (B10) and is never yielded (B11). A host-dependency outage costs one attempt every five minutes or so, and pages.

Why the wait is anchored at "found dead and free" rather than at the mint: the anchor is the door. A mint-anchored wait can be swallowed by a long attempt, and then no tick ever sees the target free (B10). Kubelet anchors CrashLoopBackOff on FinishedAt for the same reason, so this is the established spelling, not a new one. A streak is stored, not recomputed from the paging log. The log holds only six entries and slides by the hour, while the streak must outlive both. Reading the backoff from the window count was today's spelling, and it is exactly the mutant the cut's tests must kill.

The reset, "admitted after the last charged attempt and ran at least 600 s before it was found dead", needs no new writer. Supervision observes the admitted generation on every pass, so the first pass that finds it dead is within one poll of the death. A death hidden behind a live transaction or an unobservable host overstates the run, and so resets early. That is the safe side, because it shortens one wait.

The reboot invariant needs evidence of what happened, not a guess. "Unit absent" alone is not a reboot, because a cleanly stopped transient unit is collected too. "Host booted after admission" alone is not a loss, because the unit might have been started this boot. The systemd driver owns both facts (a systemctl show that returns nothing, and the /proc/stat btime). So it answers one question, lost_to_host_reboot, and supervision keeps the budget. The once-per-admission bound is still needed under never-stop. A failing re-materialization leaves the same admitted generation, unit absent, booted after admission. Without the bound it would repeat with no step, no wait and no page, every tick. The clock comparison is safe: admission happens at least one full transaction, tens of seconds, after boot.

What cut 1 deliberately does not fix. A dependency failure still grows the streak (cut 3). A dead incumbent stays down while a pre-fencing deployment builds over it (cut 2). A dead generation is still route-supervised and topology-refreshed every tick (cut 4). Paging residual: the cap gives 3600 / (300 + D) attempts an hour, where D is the attempt's duration, so a crash loop whose attempts last longer than 300 s settles under 6 an hour. Its incident opens in the first hour's burst and stays open while the release never runs, but a release that runs for one pass after each restart can see it close and not reopen. That is the operator's threshold, applied as ruled. It is recorded here, not changed.

Replayed against 2026-10-18 with only cut 1. The reboot loss at 11:46 is re-materialized at once, with no step. It fails on the nginx outage. The next attempt resets the streak, because the generation had run for hours, and then waits 5, 10, 20, 40, 80, 160 s, and 300 s after that. The incident opens within the first hour and stays open (the release never runs), so the operator is paged during the outage, not at 20:24. The idunn up at 20:24 freezes in the next gap. Continuity defers to it, nothing aborts it, and it fences the absent incumbent as nothing and admits the new release.

Why cut 2 is shaped this way. The handover is the phase, guarded by one CAS over the target's records, rather than a new per-target "owner" record. A record would be a second spelling of what the phase already says, and it would need its own lifecycle, crash repair and migration. A CAS that reads every other live transaction of the target is enough because a newly minted or frozen transaction starts at Sealing and cannot own anything (B16). The rebase clears exactly the evidence that names an incumbent (B15): warming names its lease, preflight its route and isolation its workload. Keeping any of them would let the deployment fence one generation with proofs about another. A deployment waits in Warming while another transaction owns the target, because Warming's steps read the incumbent that owner is fencing, and a failed pre-fencing step is an abort (B18). Sealing and Starting keep running, so the long part, the build, still overlaps. A continuity that loses the target is aborted, not parked: it restarts a generation that is about to be replaced, and while it lives it blocks continuity of the generation that replaces it.

Prior art on question fencing-over-owning-continuity (wait for a continuity that already owns the target, or preempt it), from docs/research/restart-budget-prior-art.md at 74a7008. It leans the other way from the recommendation, and the recommendation says why. Kubernetes' rolling update scales old ReplicaSets down unhealthy pods first, so that "unhealthy replicas will block deployment" does not happen (§1.4, rolling.go). That is preemption of a failing old replica. Nomad separates the two by layer: the client restarts old allocations under their own policy, and the server reconciler places the new ones (§2.3). Neither waits on an old replica's restart. But deleting a pod is the kubelet's ordinary teardown. In Idunn, preempting a continuity past its own Fencing means a deployment driving another transaction's post-fencing abort. That abort revokes a lease and restores a projection, and it is the path that wedged Ghostlight (B7). Waiting is bounded by the owner's own phase deadlines (PhaseDeadline, :938-960) and adds no new abort path. Recommendation: wait, at medium confidence. Preempt is the better answer once post-fencing aborts are known not to wedge, which is follow-up ghostlight-abort-lease-before-stop.

Residuals, recorded and not fixed. A wedged post-fencing abort of a continuity (B7, follow-up ghostlight-abort-lease-before-stop) holds a waiting deployment as long as it holds the target. A continuity superseded after its mint stays charged; that lengthens one wait for a generation that is being replaced. Stop-then-start has no control-plane fixture (B17), so in cut 2 its edge is pinned structurally (one seam, one call) and by a unit test of the phase mapping, not by an engine run. raven-muninn uses it, so the engine test is owed before cut 5.

5. Cut sequence

  1. c1-never-stop: spec in the mind (specs-with-no-report recipe). Depends on nothing.
  2. c2-phase-split (handover-phase-split): spec cut-c2-phase-split.r1 in the mind, branch idunn/reincarnation-c2 from 9a0a6a2. Continuity restarts the incumbent while a pre-fencing deployment builds. The ownership predicates split (B13, B14). The handover is one CAS at takes_target_at(). A pre-handover deployment rebases onto whatever is admitted, and a superseded continuity aborts itself. Freeze refuses only a second live deployment for a target, which dissolves follow-up profile-deploy-joint-gap. It folds in the test-only follow-up c1-sustained-reset-test-reach. Open question: fencing-over-owning-continuity. Depends on cut 1 (merged at 1da150a).
  3. Dependency failures do not grow the backoff (budget-never-stop): an attempt that failed because of a declared dependency (Odin's stable route, the target's own stable route listener), as Idunn itself observed it, leaves the streak unchanged. Waits on follow-up odin-route-record-stale, because the record it would read is not trustworthy, and on cut 1.
  4. A dead generation is not supervised as alive (target invariant 5): observe the workload before route supervision and topology refresh, so that a dead admitted generation gets no topology authentication and no lease-exactness challenge. Its non-binding correlation is reported once, not every tick. Waits on cut 1, because it rewrites the same function.
  5. Closing proof on Yggdrasil: an operator-approved install of the merged cuts. streampixels-service survives the install, and idunn status is checked before and after. Waits on 1-4 and on follow-up stop-first-handover-fixture (raven-muninn is stop-then-start, B17).

Ops, out of scope: nginx boot order (gamecult-ops), and retention of the roughly 130 dead transient units.

Installed

Cut 1 (1da150a, merge of idunn/reincarnation at ffb4293) was installed on Yggdrasil on 2026-10-19 at 08:34:45 UTC with the operator's OK. The binary is /usr/local/bin/idunn sha256 f0e58308…, built in /srv/build/idunn-1da150a (cargo test --lib: 414 passed). The pre-install store backup is /root/idunn-store-backup-*-pre-reincarnation-c1. Restoring it means restoring its stores and idunn.running together, because the cb60965 binary cannot read idunn.target_supervision.v2. At first open Idunn logged "migrated 5 control record(s) … idunn.target_supervision.v2". streampixels-service kept running through the restart (unit 73ca40ba, PID 968752, NRestarts 0). The catalog returned 200, and 4443/udp was listening.

Installed: cut 2

Cuts c2-phase-split, c2b-owner-deadlines, c2c-post-fence-forward and c2d-continuity-yields (branch tip 8760a4f) were installed on Yggdrasil on 2026-10-20 at 00:34:06 UTC, with the operator's OK.

  • The hotfix before it. ec45906 (bounded_detail) was installed on its own on 2026-10-19 at 17:32 UTC. It unwedged tx-f978ab86.
  • Build. /srv/build/idunn-8760a4f; cargo test --lib passed 471.
  • Backup. /root/idunn-store-backup-*-pre-reincarnation-c2 holds the stores, incidents.cc and the binaries. A rollback must restore them together: older binaries refuse two live transactions on one target, and they refuse the operator-required incident condition.
  • First boot. The boot lift returned exactly tx-039b91e9 (ghostlight) and tx-4eaed669 (odin) to forward retry, as the Soul install gate predicted. Odin's continuity, wedged in Routing since the 2026-10-18 reboot, committed 4 s later. Generation tx-4eaed669 is admitted, its route is healthy, and the same PID 2571 still runs. Ghostlight's continuity is retrying forward in AwaitingReady. Its daemon's readiness is a ghostlight-side fault (Permission denied reading its session-wrapping key).
  • Live processes. Every live MainPID was the same before and after the install. The streampixels catalog returned 200, and 4443/udp was listening.