Body map after reincarnation cut 2: two slots, take_target, forward retry, operator-required
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
2f8671ac35
commit
6e2a31cf31
5 changed files with 80 additions and 33 deletions
|
|
@ -173,24 +173,40 @@ health plus Ready correlation. It changes and observes route membership only
|
|||
afterward. Only after the proxy has adopted the candidate may Idunn clear the
|
||||
promotion fence and drain the incumbent.
|
||||
|
||||
Ownership of a target is exclusive at each instant. A live
|
||||
`DeploymentTransaction` owns it from Fencing on (a stop-first rollout, from
|
||||
Starting); before Fencing a deployment owns only its private candidate and
|
||||
continuity owns the incumbent slot. Continuity owns the admitted generation's
|
||||
survival and never stops. There is no stopped state: six charged attempts in
|
||||
an hour (`restarts_page`) open the ContinuityExhausted incident and gate
|
||||
nothing, and no deployment path yields to continuity. When continuity next acts
|
||||
is derived only from its own record in `TargetSupervision` (schema v2, a v1
|
||||
record lifts at store open): the attempt streak, the backoff anchor (written
|
||||
by the dead-and-free pass, the only writer), and the once-per-admission
|
||||
re-materialization bound. The workload driver alone answers whether a lost unit
|
||||
was a host reboot (`WorkloadPort::lost_to_host_reboot`). Crash restart, restart
|
||||
after a sustained run and reboot re-materialization all mint through one CAS;
|
||||
continuity supervision never aborts a transaction. Operator-facing incidents
|
||||
and `idunn status` are projections that decide nothing. Cut line: cuts 2 to 4
|
||||
of `reincarnation-map.md` still owe continuity restarting alongside a
|
||||
pre-fencing build, dependency failures not growing the streak, and the
|
||||
dead-generation route refresh.
|
||||
A target has two slots: the incumbent (the admitted generation) and a
|
||||
transaction's candidate. Continuity owns the incumbent slot and keeps the
|
||||
admitted generation alive; it never stops. A live `DeploymentTransaction` owns
|
||||
its private candidate before Fencing and takes the target itself at Fencing (a
|
||||
stop-then-start rollout, at Starting) through one `take_target` CAS, so of two
|
||||
transactions entering from one view exactly one succeeds. A continuity
|
||||
restart is itself a transaction and takes the target the same way.
|
||||
|
||||
Phases from Warming through Committing carry deadlines, as do aborts and
|
||||
drains; an expired deadline never waits silently. A stateless deployment
|
||||
aborts to its incumbent. A deployment whose candidate was issued a write lease
|
||||
is left to the operator (`operator-required` incident, the second incident
|
||||
condition beside `continuity-exhausted`; its subject is the transaction id and
|
||||
it closes when the record ends or its step goes on). A continuity after
|
||||
Fencing retries its phase forward on a capped backoff and never stops its live
|
||||
candidate, since that is the only running copy of the admitted body; the
|
||||
deadline opens the same incident. The one exception is yielding: a continuity
|
||||
whose route proof has not succeeded gives way, through its own post-fencing
|
||||
abort, to a deployment the deployment brake has released and that awaits the
|
||||
handover; a continuity that has proven its route never yields. The
|
||||
`ContinuityExhausted` incident (six charged attempts in an hour,
|
||||
`restarts_page`) pages and gates nothing. When continuity next acts is derived
|
||||
only from its own record in `TargetSupervision` (schema v2): the attempt
|
||||
streak, the backoff anchor (written by the dead-and-free pass, the only
|
||||
writer), and the once-per-admission re-materialization bound. The workload
|
||||
driver alone answers whether a lost unit was a host reboot
|
||||
(`WorkloadPort::lost_to_host_reboot`). Operator-facing incidents and
|
||||
`idunn status` are projections that decide nothing. A boot lift returns a
|
||||
continuity abort left by an older Idunn, one that had not stopped its running
|
||||
candidate, to forward retry; if it cannot decide, every continuity abort is
|
||||
held until a lift succeeds. It is a migration with a deletion line (follow-up
|
||||
`c2c-delete-boot-lift`). Cut line: cuts 3 and 4 of `reincarnation-map.md`
|
||||
still owe dependency failures not growing the streak and the dead-generation
|
||||
route refresh.
|
||||
|
||||
The deployment brake gates changes to source, release, configuration, schema,
|
||||
unit, or authority binding. The lifecycle brake separately gates restart of an
|
||||
|
|
|
|||
|
|
@ -313,10 +313,36 @@ written when the pass first finds the admitted generation dead and the target
|
|||
free, and a capped doubling wait from that anchor (5 s base, 300 s cap). A run
|
||||
of 600 s or more by the admitted body resets the streak. A generation lost to a
|
||||
host reboot (the workload driver's evidence, `lost_to_host_reboot`) is
|
||||
re-materialized once per admission without charge or wait. The target is free
|
||||
for a deployment's candidate between attempts, and a deployment never yields to
|
||||
continuity before Fencing; from Fencing on (or from Starting for a stop-first
|
||||
rollout) the live transaction alone owns the target.
|
||||
re-materialized once per admission without charge or wait.
|
||||
|
||||
Each target has two slots, the incumbent and a transaction's candidate. The
|
||||
incumbent is continuity's until a transaction takes the target, at Fencing (at
|
||||
Starting for a stop-then-start rollout), through one `take_target` CAS; of two
|
||||
transactions entering from one view exactly one succeeds. Before that, a
|
||||
deployment owns only its private candidate and the target is free for it
|
||||
between continuity attempts.
|
||||
|
||||
Phases Warming through Committing, post-fencing aborts and drains all carry
|
||||
deadlines (the plan's frozen `PhaseDeadlines`; `WARMING_DEADLINE_SECONDS`, and
|
||||
`PRE_FENCING_ABORT_DEADLINE_SECONDS` for a pre-fencing abort). What an expired
|
||||
post-fencing deadline does depends on the owner. A deployment whose candidate
|
||||
was never issued a write lease aborts to the incumbent. One whose candidate was
|
||||
issued a lease may have written under a new state contract and is left to the
|
||||
operator. A continuity retries its phase forward on a capped backoff and never
|
||||
stops its live candidate, because that candidate is the only running copy of
|
||||
the admitted body. Both operator cases open an `operator-required` incident
|
||||
(subject: the transaction id) that closes when the record ends or its step goes
|
||||
on.
|
||||
|
||||
The one yield: a continuity whose route proof has not succeeded gives way to a
|
||||
deployment the deployment brake has released that awaits the handover of the
|
||||
target. It begins its own post-fencing abort and the deployment fences what is
|
||||
admitted. A continuity that has proven its route is serving never yields.
|
||||
|
||||
At boot, a lift returns to forward retry any continuity abort an older Idunn
|
||||
left over a running candidate. If the lift errs or cannot observe the
|
||||
candidate, the abort is held, not run. The lift is a migration with a deletion
|
||||
line (follow-up `c2c-delete-boot-lift`).
|
||||
|
||||
An admitted generation keeps its Ready and latest Odin receipts as durable
|
||||
proof of the exact activation at their recorded admission times. Dependency
|
||||
|
|
|
|||
|
|
@ -97,7 +97,8 @@ Each of these was found by running the pipe against Raven, not by reading.
|
|||
bind time. `rollout.strategy = "stop-then-start"` exists for exactly a
|
||||
route-less host-actuator workload: the admitted incumbent is stopped at
|
||||
Starting, before the candidate starts, and supervision leaves the target
|
||||
alone from Starting on. A systemd-transient workload may not declare it.
|
||||
alone from Starting on: the transaction takes the target there (one
|
||||
`take_target` CAS) rather than at Fencing. A systemd-transient workload may not declare it.
|
||||
- **An unobservable host is silence.** The hub returns a typed
|
||||
`HostUnobservable` for not-attached, detached, reconnected and unanswered.
|
||||
Supervision skips that tick and logs it. Before this, an Idunn restart
|
||||
|
|
|
|||
|
|
@ -21,8 +21,8 @@ probes ran on Yggdrasil over `ssh ygg`, read-only, 2026-09-30 13:51-13:55 UTC
|
|||
| I2 | Continuity exhaustion (since `idunn-reincarnation` cut 1: `restarts_page`, a paging threshold only; nothing stops, and the line numbers below are pre-cut) | `:68-71` (6 restarts / 3,600,000 ms, 5 s doubling), `:1066-1072` (`restarts_used`, `restarts_exhausted`), decided `:5709-5722` under key `continuity:<target>` (`:5678`) | Exhaustion is **derived** from `TargetSupervision.continuity_restarts` (newest 6 times, sliding window). No write marks it. The fault clears (`:5723`) as soon as one entry leaves the window, so a permanently dead target leaves and re-enters exhaustion several times each hour. |
|
||||
| I3 | Route degradation | `RouteSupervisionState` `:890-942`; `record_failed_challenge` sets `degraded_since_unix_millis` on the **first** failed challenge, `record_proved_challenge` clears it; reported at `:5975` under `route:<target>` | Durable per incarnation (inside `admitted_generation.v4`); a new generation starts from `default()`. Backoff doubles from the topology max age to a cap. |
|
||||
| I4 | Other operator-facing faults | `:5645-5651` (`generation-down:<target>`, "holds a dead admitted generation down"), `:6016` (`route-record:<target>`), `note_fault` per transaction | Also stderr plus `last_error`. Not named by the target; listed so the paging-set question sees them. |
|
||||
| I5 | `OperatorRequired` on main | `TerminalRecovery::OperatorRequired` `:878-885`, validated `:1954`; constructed only in tests (`:12525`, `:12535`) | No production path produces it on main. |
|
||||
| I6 | `OperatorRequired` on B5 | `hands/b5` `resolve_phase_deadline` (diff hunk at `control_plane.rs` ~8560-8625 on the branch), `PhaseEnd::OperatorRequired` | Deadline passed on a stateful deploy with an issued lease: `report_once("deadline:<transaction_id>")`, returns `Ok(None)`, the transaction stays live. No terminal transition and no operator action that ends it were found on the branch (`idunn expire` writes `idunn.expiry_request.v1`, which re-enters the same arm). |
|
||||
| I5 | `OperatorRequired` on main (pre-cut-2 reading; since `idunn-reincarnation` cut 2 `resolve` reaches it in production through `PhaseEnd::OperatorRequired` and `PhaseEnd::RetryForward`, both calling `leave_to_operator`, which opens the `operator-required` incident `IncidentCondition::OperatorRequired`, subject the transaction id; `close_spent_operator_incidents` closes it when the record no longer `leaves_to_operator`) | `TerminalRecovery::OperatorRequired` `:878-885`, validated `:1954`; constructed only in tests (`:12525`, `:12535`) | No production path produces it on main. |
|
||||
| I6 | `OperatorRequired` on B5 (superseded by cut 2: B5's deadlines merged as `PhaseDeadline`, and a continuity past its deadline now also lands here) | `hands/b5` `resolve_phase_deadline` (diff hunk at `control_plane.rs` ~8560-8625 on the branch), `PhaseEnd::OperatorRequired` | Deadline passed on a stateful deploy with an issued lease: `report_once("deadline:<transaction_id>")`, returns `Ok(None)`, the transaction stays live. No terminal transition and no operator action that ends it were found on the branch (`idunn expire` writes `idunn.expiry_request.v1`, which re-enters the same arm). |
|
||||
| I7 | Store kinds | `:55-61`, `:936-966`; B5 adds `idunn.expiry_request.v1` | `control.cc`: `idunn.deployment_command`, `idunn.deployment_transaction`, `idunn.admitted_generation.v4`, `idunn.target_supervision.v1`. `history.cc` beside it holds terminal transactions (`:3653`), retired one per tick (`:5068-5091`). |
|
||||
| I8 | Single-writer precedent inside `control.cc` | B5 `ExpiryRequest` doc comment | "The CLI only writes this record. The daemon's deadline resolver reads it ... No CLI writes a transaction field." One writer per kind. |
|
||||
| I9 | Serve loop | `:5041-5058`, `run_scheduler_tick` `:5098-5110` | 500 ms poll. A tick error is logged, never propagated. |
|
||||
|
|
@ -166,9 +166,10 @@ cut spec fixes them.
|
|||
|
||||
Empty cells, stated plainly:
|
||||
|
||||
- `operator-required` closure: no path on `hands/b5` ends a transaction left to
|
||||
the operator (I6). The incident can open and never close until B5 names that
|
||||
action.
|
||||
- `operator-required` closure: the incident closes when the transaction ends or
|
||||
its step goes on (`close_spent_operator_incidents`). A stateful deployment
|
||||
left to the operator still has no Idunn-owned terminal action; the operator's
|
||||
`idunn expire` re-enters the same arm.
|
||||
- Delivery-failure visibility: no owner.
|
||||
- Discord `enforce_nonce` window: not verified against Discord's current docs
|
||||
in this pass.
|
||||
|
|
|
|||
|
|
@ -30,9 +30,12 @@ journald (`report_once`, `control_plane.rs:4599`):
|
|||
- An admitted route is degraded. It fails its stable-route challenge
|
||||
(`supervise_admitted_route`, ~5973). The degraded mark is durable, but no
|
||||
one is told.
|
||||
- A post-fencing phase deadline expires into `OperatorRequired`. This lives on
|
||||
B5, which is unmerged on `hands/b5`. The failure record is durable and the
|
||||
notice is a log line.
|
||||
- A post-fencing phase deadline leaves a transaction to the operator: a
|
||||
stateful deployment whose candidate was issued a lease, or a continuity
|
||||
retrying forward past its deadline. Since `idunn-reincarnation` cut 2 it is
|
||||
the second incident condition, `operator-required` (subject: the transaction
|
||||
id), beside `continuity-exhausted`; it closes when the record ends or its
|
||||
step goes on.
|
||||
|
||||
Odin's own store condition (`odin.store_condition.v1`, `contested`) is mapped
|
||||
but not built. Idunn reads nothing about Odin's store health today.
|
||||
|
|
@ -85,8 +88,8 @@ the machine has a detector with no record, and a deliverer that nothing feeds.
|
|||
- Odin's store-recovery cuts, O2 included. They produce `odin.store_condition`
|
||||
and are Odin's campaign, still in prose. This campaign consumes the record
|
||||
once it exists.
|
||||
- Idunn's B5 phase deadlines. B5 is in prose on `hands/b5`. This campaign
|
||||
reports its `OperatorRequired` outcome once B5 merges.
|
||||
- Idunn's phase deadlines. They landed in `idunn-reincarnation` cut 2, which
|
||||
also owns the `operator-required` incident condition.
|
||||
- VoidBot's `notify_owner` MCP tool and its separate DM code.
|
||||
- Bifrost's Persona delivery crossing.
|
||||
- Eve/TUI lowering of incidents.
|
||||
|
|
|
|||
Loading…
Reference in a new issue