Handle Heartbeat shutdown signals cleanly

This commit is contained in:
Meta Vens 2026-08-10 12:23:42 +01:00
commit bfb92cbb96
4 changed files with 112 additions and 8 deletions

View file

@ -39,6 +39,10 @@ use std::env;
use std::fs;
use std::path::Path;
use std::path::PathBuf;
use std::sync::{
Arc,
atomic::{AtomicBool, Ordering},
};
use std::thread;
use std::time::Duration;
use uuid::Uuid;
@ -549,8 +553,13 @@ fn main() -> Result<()> {
let mut completed_iterations = highest_pulse_artifact_sequence(&artifact_dir)?;
let mut completed_routines = 0_u64;
let mut refused_pulses = 0_u64;
let shutdown_requested = install_shutdown_signal_owner()?;
loop {
if max_iterations > 0 && completed_iterations >= max_iterations {
if heartbeat_serve_should_stop(
&shutdown_requested,
max_iterations,
completed_iterations,
) {
break;
}
let iteration = completed_iterations + 1;
@ -617,7 +626,10 @@ fn main() -> Result<()> {
if max_iterations > 0 && completed_iterations >= max_iterations {
break;
}
thread::sleep(Duration::from_secs(interval_seconds));
wait_for_shutdown(
&shutdown_requested,
Duration::from_secs(interval_seconds),
);
continue;
}
}
@ -650,7 +662,7 @@ fn main() -> Result<()> {
if max_iterations > 0 && completed_iterations >= max_iterations {
break;
}
thread::sleep(Duration::from_secs(interval_seconds));
wait_for_shutdown(&shutdown_requested, Duration::from_secs(interval_seconds));
continue;
}
let persona_pulse = pulse_persona_heartbeat(
@ -681,7 +693,7 @@ fn main() -> Result<()> {
if max_iterations > 0 && completed_iterations >= max_iterations {
break;
}
thread::sleep(Duration::from_secs(interval_seconds));
wait_for_shutdown(&shutdown_requested, Duration::from_secs(interval_seconds));
continue;
}
let result = run_void_routine_store(
@ -720,7 +732,7 @@ fn main() -> Result<()> {
if max_iterations > 0 && completed_iterations >= max_iterations {
break;
}
thread::sleep(Duration::from_secs(interval_seconds));
wait_for_shutdown(&shutdown_requested, Duration::from_secs(interval_seconds));
}
println!(
"{}",
@ -736,6 +748,7 @@ fn main() -> Result<()> {
"completedRoutines": completed_routines,
"refusedPulses": refused_pulses,
"bounded": max_iterations > 0,
"shutdownRequested": shutdown_requested.load(Ordering::SeqCst),
"privateStateExposed": false
})
);
@ -755,6 +768,36 @@ fn main() -> Result<()> {
Ok(())
}
fn install_shutdown_signal_owner() -> Result<Arc<AtomicBool>> {
let requested = Arc::new(AtomicBool::new(false));
let signal = Arc::clone(&requested);
ctrlc::try_set_handler(move || {
signal.store(true, Ordering::SeqCst);
})
.context("failed to install Heartbeat shutdown signal owner")?;
Ok(requested)
}
fn heartbeat_serve_should_stop(
shutdown_requested: &AtomicBool,
max_iterations: u64,
completed_iterations: u64,
) -> bool {
shutdown_requested.load(Ordering::SeqCst)
|| (max_iterations > 0 && completed_iterations >= max_iterations)
}
fn wait_for_shutdown(requested: &AtomicBool, duration: Duration) {
let deadline = std::time::Instant::now() + duration;
while !requested.load(Ordering::SeqCst) {
let remaining = deadline.saturating_duration_since(std::time::Instant::now());
if remaining.is_zero() {
break;
}
thread::sleep(remaining.min(Duration::from_millis(50)));
}
}
fn next_path(args: &mut impl Iterator<Item = String>, name: &str) -> Result<PathBuf> {
Ok(PathBuf::from(next_value(args, name)?))
}
@ -879,6 +922,24 @@ fn active_swarm_brake(
mod brake_tests {
use super::*;
#[test]
fn heartbeat_serve_stops_for_signal_or_bounded_completion() {
let requested = AtomicBool::new(false);
assert!(!heartbeat_serve_should_stop(&requested, 0, 42));
assert!(!heartbeat_serve_should_stop(&requested, 43, 42));
assert!(heartbeat_serve_should_stop(&requested, 42, 42));
requested.store(true, Ordering::SeqCst);
assert!(heartbeat_serve_should_stop(&requested, 0, 42));
}
#[test]
fn heartbeat_shutdown_interrupts_interval_wait() {
let requested = AtomicBool::new(true);
let started = std::time::Instant::now();
wait_for_shutdown(&requested, Duration::from_secs(1));
assert!(started.elapsed() < Duration::from_millis(100));
}
#[test]
fn heartbeat_brake_lookup_uses_the_exact_requested_namespace() -> Result<()> {
let temp = tempfile::tempdir()?;

View file

@ -3135,3 +3135,28 @@ three. Exact live replay remains the package acceptance boundary.
package inspection; copied active-attempt settlement; actual corrupted-state
intervention; two bounded actual Heartbeat pulses with brake present and grant
absent.
# Heartbeat graceful shutdown authority map — 2026-08-10
- Owner: the Heartbeat serve process owns its termination signal and the point
at which its scheduling loop stops.
- Inputs: SIGINT/SIGTERM, the current atomic pulse transaction, the configured
interval, and an optional bounded iteration ceiling.
- Outputs: no new pulse after shutdown is observed, interruptible idle wait, one
final `epiphany.heartbeat.serve_receipt.v0` with `shutdownRequested=true`, and
process exit zero.
- Derived state: Docker stop timeout is supervision pressure only; it is no
longer the termination mechanism. A signal cannot roll back an in-flight
atomic pulse and therefore takes effect at the next transaction boundary.
- Forbidden writers: Docker's eventual SIGKILL, artifact sequence, provider
readiness, and the resident Self process do not decide Heartbeat closure.
- Shared paths: bounded completion and signal completion converge on the same
final typed serve receipt. Every interval wait observes the same atomic signal
owner.
- Cut line: install one process-local signal owner before entering `serve`; stop
before the next pulse after a signal and replace every blocking interval sleep
with a signal-aware wait. Do not add persisted shutdown authority or a repair
loop.
- Verification layer: pure stop/wait tests, focused Heartbeat binary tests, then
exact packaged Linux serve under the copied c006 cognitive brake with SIGTERM,
exit zero, final shutdown receipt, and no new grant.

View file

@ -1,5 +1,23 @@
# Fresh workspace handoff
## Heartbeat graceful SIGTERM source complete — 2026-08-10
Heartbeat `serve` now owns SIGINT/SIGTERM with one process-local atomic signal
owner. Shutdown is observed before admitting the next pulse, all interval waits
are interruptible, and signal or bounded completion converge on the existing
typed serve receipt. The receipt now states `shutdownRequested`; an in-flight
atomic pulse is allowed to finish before closure. No persisted shutdown command,
retry writer, or Docker-timeout compensator was added.
Six focused Heartbeat binary tests pass, including signal/boundary stopping and
immediate interruption of the interval wait. The Heartbeat binary checks without
warnings and `git diff --check` passes. This cut is not yet committed, packaged,
or accepted on Linux. Next: commit/push, package exactly once on Starfire, and
run packaged serve on copied c006 state under its cognitive brake. Send SIGTERM
during an interval and require exit zero, a final typed receipt with
`shutdownRequested=true`, no new grant, and byte-identical resident/runtime
authority. Do not release actual c006. Yggdrasil never builds.
## Typed-attempt non-resurrection accepted and actual c006 settled — 2026-08-10
Exact pushed source `c693aaeadbb17204724c9986f16952e2f93e8fc9` packages

View file

@ -36,9 +36,9 @@ invariants:
current_status:
summary: Epiphany is a supervised engineering alpha on Starfire, not deployment-ready. Exact b8138a6b active-child SIGTERM and restart, exact 9f369528 deterministic warm packaging, exact c930f505 cross-container ownership, exact ed15f83a resident lifecycle retention, exact 16c4b19f coordinator receipt and Heartbeat artifact retention, exact cce2b8ac narrow model-generation retention, exact eea15e82 model-turn JSON ingress, exact b1e8c89a semantic unattempted-generation retention, exact 6bc3ec3f semantic physical-epoch retirement, and exact 44230a05 Persona conversation retention are proven. Gamecult-ops 059821f owns Yggdrasil stream retention and exact 6ee08a4 owns stable Starfire Persona-permit firewall ingress. Exact 16c4b19f genuine idle and exact a987d2f7 active cognition completed 900-pulse bounded plateaus. Runtime-owned typed fulfillment is accepted live under exact 6dd9b132. Yggdrasil is the small public crossing body, never the release forge.
latest_phase: Exact pushed c693aaea separates structural coordinator-receipt binding from terminal success and makes immutable typed attempts non-requeueable. Its warning-free 24-binary release sha256-fd46323a with witness sha256-8ad95c0a settled a copy of the exact c006 attempt-two failure: resident exited zero at revision 22, the exact grant became unfulfilled, its pressure stayed consumed, and two braked Heartbeat pulses issued no attempt three. The same authenticated release then settled actual c006 with exit zero and no OOM; actual bounded Heartbeat replay remained braked with no grant. Exact pre-intervention evidence and post-settlement receipts are preserved. The cognitive brake remains engaged; old 8d92c826 processes remain stopped.
latest_phase: Exact pushed c693aaea and release sha256-fd46323a settled copied and actual c006 attempt-two authority without resurrection; the cognitive brake remains engaged and old processes remain stopped. Heartbeat graceful shutdown is now source-complete: serve owns SIGINT/SIGTERM, observes it before each pulse, interrupts every interval wait, and converges bounded or signalled closure on one typed receipt with shutdownRequested. Six focused binary tests and warning-free check pass. Exact packaging and copied Linux SIGTERM replay remain.
operational_state:
local_worktree: clean_at_pushed_c693aaea_after_actual_c006_settlement
local_worktree: heartbeat_graceful_sigterm_source_verified_pending_commit
operator_wireguard: restored
starfire_persona_permit_firewall: persistent_exact_gamecult_ops_6ee08a4
yggdrasil: canonical_small_public_crossing_body
@ -70,7 +70,7 @@ current_status:
yggdrasil_epiphany_service_contract: repaired_in_gamecult_ops_812b427_not_installed
resident_physiology: c006_exact_c693aaea_settled_braked_no_active_lease_no_attempt_three_old_processes_stopped
unresolved_authority: Coordinator receipt, Heartbeat pulse-artifact, deployed supervisor/projector log retention, genuine-idle and active resource plateaus, exact-grant terminality, restart-persistent cooldown refusal, schedule/action issuance cardinality, runtime-owned typed fulfillment, narrow model-generation retention, published model-turn JSON ingress, semantic logical/physical retention, Persona conversation retention, stable Starfire firewall ownership, completed model-adapter session archival, and normal coordinator lifecycle opening/error closure/legacy repair/archival are accepted live. Receipt-free hard-death recovery is source- and package-accepted but its live kill matrix remains unproven because the first copied run exposed non-atomic storage publication. A typed worker launch without valid terminal fulfillment deliberately remains active awaiting an owner for exact orphan-worker death and supersession; it is not retryable yet. Historical migrated semantic attempts without recorded physical namespace remain durable and non-retirable. Repository Body observation still performs two authenticated full-tree Git index/manifest passes. Execute-mode workers require a persistent process body. Mutable Starfire bind storage is not accepted as resident authority until the packaged CultCache repair and recovery replay pass. One completed Discord consequence remains blocked by the absent Yggdrasil credential.
next_action: Map and cut Heartbeat graceful SIGTERM ownership so its resident scheduling body exits zero with a final typed receipt instead of Docker timeout exit 137. Verify focused source, package exact source once on Starfire, and replay against copied c006 state under the cognitive brake. Then resume the copied receipt-free death and orphan-worker Continuity matrix. Do not release the c006 brake. Yggdrasil never builds.
next_action: Commit and push the Heartbeat graceful SIGTERM cut, package exact source once with the persistent Starfire cache, then run exact Linux Heartbeat serve against copied c006 state under the cognitive brake. Require SIGTERM exit zero, final typed shutdownRequested receipt, no new grant, and byte-identical resident/runtime authority. Then resume the copied receipt-free death and orphan-worker Continuity matrix. Do not release the c006 brake. Yggdrasil never builds.
active_subgoals:
- Prove Persona, Self, Eyes, Modeling, Imagination, Hands, Soul, Mind, Continuity, and nervous-system behavior individually.