Recover stale Face heartbeat jobs

This commit is contained in:
Meta Vens 2026-05-20 23:37:13 +01:00
commit e7a247949e
5 changed files with 62 additions and 7 deletions

View file

@ -122,7 +122,7 @@ Git history and smoke artifacts carry routine proof. `state/evidence.jsonl` shou
- Repo Face draft, review, and Discord speech lanes now cross through Bifrost bridge commands instead of VoidBot-owned GitHub/Discord mutation code. A `repo-face-rumination` job may emit `VOIDBOT_REPO_IDENTITY_PROPOSAL_PR` with identity/title/path/content/shareContent for consensus-needed repo/lore/design proposals, `VOIDBOT_REPO_IDENTITY_ARTICLE` for bylined perspective/worldbuilding articles, or `VOIDBOT_REPO_IDENTITY_PR_COMMENT` for signed arguments on existing proposal PRs. The worker validates the registered identity and records Face-local receipts, then calls `BIFROST_ROOT/tools/bifrost-bridge.mjs` for draft PR creation, PR comments, and persona Discord posts. Bifrost owns this because it is GameCult's governance/labor platform and public-process bridge, not because it is a convenient script bucket: governed crossings include GitHub proposal/review surfaces, Discord-native work/member/swarm interactions, dispatch receipts, CultNet/CultCache intake, and future collaboration interfaces. Heimdall remains the OAuth/account/capability gate. A Face calling something the next cut/pass should usually turn it into that artifact directly when enough material exists.
- Repo Face heartbeats now receive Bifrost governance digest context and can write to Bifrost topic threads. `VOIDBOT_REPO_IDENTITY_BIFROST_TOPIC` lets a Face open/comment/approve a canonical Bifrost topic, optionally include separate in-character `mirrorContent` for the `#bifrost` mirror, and optionally promote an approved topic into Bifrost agent transport. The worker passes configured `BIFROST_DISCORD_CHANNEL_ID`, persona name, and avatar to Bifrost so mirror receipts live on the topic. `VOIDBOT_REPO_IDENTITY_UPDATE_REQUEST` remains available only for immediate owner-approved dispatch inside that Face's jurisdiction. `BIFROST_DISCORD_CHANNEL_ID` marks `#bifrost` (`1506760343836430376`) so mirrored bot chatter is ignored in Discord context; Bifrost typed CultCache topics/comments are the source of truth for feature/governance discussion, not Discord scrollback. If a Face's intended public line is actually a repo-local work request, the same turn should put the actionable packet on Bifrost with enough recent-context summary for Codex; Aquarium should not be the only place that request exists.
- First role-addressed chat also initializes the target repo's local `.voidbot` home. `ensureRepoFaceInitialized` resolves `repoPath` from the registry, `SOURCE_REPO_ROOT`, or `E:/Projects/<repoName>`, creates `voice/`, `state/`, `birth/`, and `logs/`, writes `voice/identity.json`, then starts `scripts/run-repo-face-birth.mjs` detached. That worker runs EpiphanyAgent's `epiphany-repo-personality scout` and `epiphany-repo-birth-runner` so newborn personality/memory/trajectory packets come from repo contents and git history. Status and logs live in the target repo under `.voidbot/birth/voidbot-face-birth.json` and `.voidbot/logs/voidbot-face-birth.log`.
- Agent heartbeats now use one CTB/Final-Fantasy-X-style initiative queue. `scripts/run-repo-face-heartbeats.ts` still has its old filename, but it now reconciles built-in system agents plus registered repo Faces, stores the roster at `REPO_FACE_HEARTBEAT_STATE_PATH`, advances virtual initiative time independently of wall-clock due dates, and dispatches selected turns. Repo Faces queue normal `repo-face-rumination` jobs; built-in `void` starts the existing moderation rumination runner as its turn. Active jobs/locks freeze initiative: recovery starts only after the owner-Codex job or Void moderation lock clears, and completion cannot grant another turn in the same pulse. `GameCult Local Orchestrator` is the live hidden local scheduled task that ticks the scheduler; `npm run state:install-repo-face-heartbeat-task` remains an isolation/backcompat installer, not the normal pulse owner. Heat knobs are `REPO_FACE_HEARTBEAT_GLOBAL_HEAT` and `REPO_FACE_HEARTBEAT_HEAT_OVERRIDES`; overrides can target `all`, `kind:<participantKind>`, `turn:<turnKind>`, `identity:<id>`, `repo:<repo>`, `display:<name>`, `channel:<id>`, or bare identity/repo/display keys. Repo Face prompts now include a jurisdiction-dive cadence so unattended heartbeats periodically inspect repo/history terrain with source tools and convert real findings into state, proposals, essay seeds, agency pressure, or speech.
- Agent heartbeats now use one CTB/Final-Fantasy-X-style initiative queue. `scripts/run-repo-face-heartbeats.ts` still has its old filename, but it now reconciles built-in system agents plus registered repo Faces, stores the roster at `REPO_FACE_HEARTBEAT_STATE_PATH`, advances virtual initiative time independently of wall-clock due dates, and dispatches selected turns. Repo Faces queue normal `repo-face-rumination` jobs; built-in `void` starts the existing moderation rumination runner as its turn. Active jobs/locks freeze initiative: recovery starts only after the owner-Codex job or Void moderation lock clears, and completion cannot grant another turn in the same pulse. The active-job scan now marks stale approved/running Face jobs failed through the job queue and records `stale_active_turn_recovered`, so a dead worker claim cannot freeze one Face forever. `GameCult Local Orchestrator` is the live hidden local scheduled task that ticks the scheduler; `npm run state:install-repo-face-heartbeat-task` remains an isolation/backcompat installer, not the normal pulse owner. Heat knobs are `REPO_FACE_HEARTBEAT_GLOBAL_HEAT` and `REPO_FACE_HEARTBEAT_HEAT_OVERRIDES`; overrides can target `all`, `kind:<participantKind>`, `turn:<turnKind>`, `identity:<id>`, `repo:<repo>`, `display:<name>`, `channel:<id>`, or bare identity/repo/display keys. Repo Face prompts now include a jurisdiction-dive cadence so unattended heartbeats periodically inspect repo/history terrain with source tools and convert real findings into state, proposals, essay seeds, agency pressure, or speech.
- Repo Face rest projection now persists normalized sleep cycles back into each Face `.cc` state during heartbeat loading. Missed nap windows are treated as already missed/completed and rolled forward instead of starting a fresh nap at inspection time. This fixes the Aqua/Mimir/Epiphany failure mode where stale `nextNapStartsAt` values stayed raw, fresh projections repeatedly marked them as napping, and ready selection skipped them forever while their initiative was overdue.
- Repo Face short-term memory distillation is now wired into the heartbeat/recovery loop. The runner starts repo-local memory maintenance when a Face is napping with short-term residue or has stale short-term residue older than six hours. Status/logs live under the target repo's own `.voidbot/status` and `.voidbot/logs`; the shared maintenance runner has `-ForceDistillation`, identity-neutral prompt wording, exact-schema warnings, and whole-batch schema validation before any operation applies. Live cleanup forced Nibu from 3 short-term memories to 0 and Mimir from 5 to 0; all registered repo Faces currently have 0 short-term residue.
- Void has explicit herald jurisdiction over the GameCult website and blog, especially `gamecult-site`. Routine rumination still cannot edit tracked files directly, but the prompt now tells Void to queue article-proposal candidates when a pressure wants to become a Void-authored site/blog piece rather than a Discord aside.

View file

@ -169,7 +169,7 @@ This note is the source-grounded description of how the live VoidBot stack is sh
- Repo Face proposal/article PRs, PR comments, and registered-identity Discord posts now cross through Bifrost bridge commands. The child may emit `VOIDBOT_REPO_IDENTITY_PROPOSAL_PR` for consensus-needed repo/lore/design proposals, `VOIDBOT_REPO_IDENTITY_ARTICLE` for bylined essays, or `VOIDBOT_REPO_IDENTITY_PR_COMMENT` for signed arguments on existing proposal PRs. The worker validates the registered identity and records Face-local receipts, but `BIFROST_ROOT/tools/bifrost-bridge.mjs` owns draft PR creation, PR comments, and Discord persona posting. This is not merely helper placement: Bifrost's original governance/labor platform role makes it the owner of governed public crossings for GameCult work, including GitHub, Discord-native work/member/swarm interactions, dispatch receipts, CultNet/CultCache intake, and future collaboration interfaces. Heimdall is the OAuth/account/capability gate; Bifrost is the bridge/request/routing/receipt authority. VoidBot still owns room cognition, moderation judgment, archive/source retrieval, and Face personality; Bifrost owns Discord when Discord acts as the Labor Platform interface or swarm transport. If Nibu or another Face frames something as the next cut/pass and enough source or room material exists, the expected action is to draft the artifact, submit the PR, and announce it rather than continuing to nag the room.
- Repo Face heartbeats now receive a Bifrost governance digest from `BIFROST_ROOT/tools/governance-threads.mjs digest --repo <repo> --agent <identity>` alongside recent Discord context. Bifrost typed CultCache topics/comments are the canonical home for feature requests, Face opinions, missing questions, approvals, and dispatch receipts; Discord is a mirror/evidence source. When `BIFROST_DISCORD_CHANNEL_ID` is configured for `#bifrost`, repo Face Discord context ignores mirrored bot chatter there because the authoritative activity arrives through the Bifrost digest. Human `#bifrost` messages should only become governance comments after Bifrost/Heimdall can link the Discord id to a registered actor. Repo-local asks that would send a human or Codex to add/fix/name/scaffold/document/test/investigate/implement belong on Bifrost with enough room/repo context for a claimed Codex turn; a Face may still speak characterfully in Aquarium, but not as the only dispatch surface. Local `#bifrost` is channel `1506760343836430376`; work proposals, governance mirrors, and Bifrost dispatch receipts should go there, while Aquarium remains the casual/social swarm room.
- Repo Face heartbeats have a Bifrost topic sentinel and an immediate update-request fallback. `VOIDBOT_REPO_IDENTITY_BIFROST_TOPIC` lets a Face open/comment/approve a canonical Bifrost topic, optionally include a separate `mirrorContent` line for in-character `#bifrost` mirroring, and optionally promote an approved topic into Bifrost agent transport. The worker passes configured `BIFROST_DISCORD_CHANNEL_ID`, persona name, and avatar to Bifrost so mirroring is a Bifrost receipt on the topic rather than a VoidBot side-post. `VOIDBOT_REPO_IDENTITY_UPDATE_REQUEST` remains available only for work inside the Face's own jurisdiction when immediate owner-approved dispatch is warranted. The intended path is topic/comment/Face approval first; direct update requests are the pressure-release valve, not the parliament.
- `scripts/run-repo-face-heartbeats.ts` now acts as the broader agent CTB/Final-Fantasy-X-style initiative scheduler. The wall-clock task is only a pulse; virtual initiative time chooses who gets the next turn. The scheduler stores per-agent speed, reaction bias, load, groups, heat, effective speed, next-turn time, active turn markers, and recent queue history at `REPO_FACE_HEARTBEAT_STATE_PATH`. Repo Faces queue `repo-face-rumination` owner-Codex jobs; built-in Void currently participates as the `void` system agent and starts the existing moderation rumination runner as its turn. Initiative freezes while an owner-Codex job or Void moderation lock is active; recovery is scheduled only after that active turn completes, and completion cannot grant another turn in the same scheduler pulse. Above-baseline heat bypasses repo Face nap gating, while active-turn freeze still prevents overlapping thoughts.
- `scripts/run-repo-face-heartbeats.ts` now acts as the broader agent CTB/Final-Fantasy-X-style initiative scheduler. The wall-clock task is only a pulse; virtual initiative time chooses who gets the next turn. The scheduler stores per-agent speed, reaction bias, load, groups, heat, effective speed, next-turn time, active turn markers, and recent queue history at `REPO_FACE_HEARTBEAT_STATE_PATH`. Repo Faces queue `repo-face-rumination` owner-Codex jobs; built-in Void currently participates as the `void` system agent and starts the existing moderation rumination runner as its turn. Initiative freezes while an owner-Codex job or Void moderation lock is active; recovery is scheduled only after that active turn completes, and completion cannot grant another turn in the same scheduler pulse. The active-job scan recovers stale Face jobs by marking ancient approved/running rows failed through the job queue before building the active-turn map. Above-baseline heat bypasses repo Face nap gating, while active-turn freeze still prevents overlapping thoughts.
- The live wall-clock pulse for this scheduler is `scripts/run-gamecult-orchestrator.ps1`, installed as `GameCult Local Orchestrator`. `scripts/install-repo-face-heartbeat-task.ps1` remains available for isolated testing, but its per-organ scheduled task should stay disabled in normal operation. Turn frequency is controlled by `REPO_FACE_HEARTBEAT_BASE_RECOVERY_MINUTES`, `REPO_FACE_HEARTBEAT_GLOBAL_HEAT`, `REPO_FACE_HEARTBEAT_SPEED_OVERRIDES`, and `REPO_FACE_HEARTBEAT_HEAT_OVERRIDES`. Heat keys can target `all`, `kind:<participantKind>`, `turn:<turnKind>`, `identity:<id>`, `repo:<repo>`, `display:<name>`, `channel:<id>`, or the bare identity/repo/display key.
- The local ignored registry currently includes Nibu for `AetheriaLore`, with Face state in `E:/Projects/AetheriaLore/.voidbot/state/nibu.cc`, Discord role id `1505918221453103247`, and an Aquarium channel allow-list. Nibu's Face state now treats AetheriaLore-first map-building, abrasive character voice, consensus-backed change agency, her own `Aetheria/Lore/Nibu.md` character-note stewardship, Nibu-authored lore essays, and tangential setting fascination as separate active doctrines. That tangential fascination points at embodied ship minds, junkyard abandonment, murderous autonomy, salvage, and save-scumming survival. The repo Face heartbeat organ is now pulsed by `GameCult Local Orchestrator`; after restarting the stack onto the new worker code, forced heartbeat job `76020f1e-0c55-424b-9e35-89ba7f77fcc3` completed without base-bot auto-posting.
- The local ignored registry now also includes Epiphany for `EpiphanyAgent`, with Face state in `E:/Projects/EpiphanyAgent/.voidbot/state/epiphany.cc`, Discord role id `1506375937439301642`, and Aquarium channel allow-list `1501196543150264332`. Her repo-local `.voidbot` birth surface has completed in plan/review mode; the generated EpiphanyAgent startup packets still require Self review before accepted native birth mutations. Her jurisdiction is the self-improvement spine: typed state, CultCache/CultNet, Codex bridge boundaries, birth/init rites, low-copy runtime contracts, review gates, graph-shaped memory, heartbeat physiology, and deletion of duplicated authority.

View file

@ -28,6 +28,7 @@ import type { SourceMessage } from "@voidbot/shared";
const HEARTBEAT_SCHEMA_VERSION = REPO_FACE_HEARTBEAT_SCHEMA_VERSION;
const HEARTBEAT_COMMAND = "repo-face-rumination";
const MIN_STALE_ACTIVE_JOB_MS = 45 * 60_000;
interface FaceHeartbeatParticipant {
identityId: string;
@ -64,6 +65,20 @@ interface FaceHeartbeatState {
pendingMentions: RepoFacePendingMention[];
}
interface ActiveTurnScan {
active: Map<string, string>;
staleRecovered: StaleActiveTurn[];
}
interface StaleActiveTurn {
identityId: string;
jobId: string;
requestMessageId?: string;
state: string;
updatedAt?: string;
ageMinutes: number;
}
interface RepoFaceChannelPlan {
primaryChannelId?: string;
snapshotChannelIds: string[];
@ -135,9 +150,21 @@ async function main(): Promise<void> {
const restStates = await loadRepoFaceRestStates(registry.identities, config.storageRoot, state);
const now = new Date();
advanceInitiativeClockFromWallClock(state, now);
const pendingJobs = dryRun
? new Map<string, string>()
const activeTurnScan = dryRun
? { active: new Map<string, string>(), staleRecovered: [] }
: await listExistingActiveTurns(config.databaseDsn, config.stateStorageBackend, config);
for (const stale of activeTurnScan.staleRecovered) {
state.history.push({
type: "stale_active_turn_recovered",
identityId: stale.identityId,
activeJobId: stale.jobId,
requestMessageId: stale.requestMessageId,
jobState: stale.state,
jobUpdatedAt: stale.updatedAt,
ageMinutes: stale.ageMinutes,
recoveredAt: now.toISOString(),
});
}
state.baseRecoveryMinutes = config.repoFaceHeartbeats.baseRecoveryMinutes;
state.globalHeat = config.repoFaceHeartbeats.globalHeat;
@ -152,7 +179,12 @@ async function main(): Promise<void> {
config.repoFaceHeartbeats.baseRecoveryMinutes,
config.repoFaceHeartbeats.globalHeat,
).map((participant) =>
applyActiveTurnFreeze(participant, pendingJobs.get(participant.identityId), state, completedThisTick),
applyActiveTurnFreeze(
participant,
activeTurnScan.active.get(participant.identityId),
state,
completedThisTick,
),
);
rescheduleStaleOverdueParticipants(state);
applyPendingMentionPriority(state);
@ -957,7 +989,7 @@ async function listExistingActiveTurns(
databaseDsn: string,
stateStorageBackend: "file" | "postgres",
config: ReturnType<typeof loadConfig>,
): Promise<Map<string, string>> {
): Promise<ActiveTurnScan> {
const storage = await createStateStorage({
backend: stateStorageBackend,
databaseDsn,
@ -969,6 +1001,9 @@ async function listExistingActiveTurns(
try {
const jobs = await storage.jobQueue.listByStates(["approved", "running"]);
const active = new Map<string, string>();
const staleRecovered: StaleActiveTurn[] = [];
const staleAfterMs = Math.max(MIN_STALE_ACTIVE_JOB_MS, config.codexExecTimeoutMs * 3);
const nowMs = Date.now();
for (const job of jobs) {
if (job.command !== HEARTBEAT_COMMAND) {
continue;
@ -978,6 +1013,24 @@ async function listExistingActiveTurns(
job.requestMessageId?.match(/^repo-face-heartbeat:([^:]+):/) ??
job.requestMessageId?.match(/:repo-face:([^:]+):\d+$/);
if (match) {
const updatedMs = Date.parse(job.updatedAt);
const ageMs = Number.isFinite(updatedMs) ? nowMs - updatedMs : Number.POSITIVE_INFINITY;
if (ageMs > staleAfterMs) {
const ageMinutes = Number.isFinite(ageMs) ? Math.round((ageMs / 60_000) * 10) / 10 : -1;
await storage.jobQueue.markFailed(
job.id,
`Repo Face CTB recovered stale active heartbeat job after ${ageMinutes} minutes without progress.`,
);
staleRecovered.push({
identityId: match[1],
jobId: job.id,
requestMessageId: job.requestMessageId,
state: job.state,
updatedAt: job.updatedAt,
ageMinutes,
});
continue;
}
active.set(match[1], job.id);
}
}
@ -985,7 +1038,7 @@ async function listExistingActiveTurns(
if (voidLock) {
active.set("void", "lock:moderation-rumination");
}
return active;
return { active, staleRecovered };
} finally {
await storage.close();
}

View file

@ -210,3 +210,4 @@
{"ts":"2026-05-20T21:18:00Z","type":"implementation","status":"ok","note":"Human Discord replies to a Face webhook/persona message now count as direct Face attention. The bot resolves the referenced message's persona display name, matches the registered Face allowed in that channel, queues the reply into the CTB pending-mention state, and leaves delivery to the existing heartbeat/sentinel rails instead of reviving direct prompt jobs."}
{"ts":"2026-05-20T21:30:00Z","type":"implementation","status":"ok","note":"Routed Aqua's Aquarium work request about a named `wet-voice-01` utterance witness demo into Bifrost governance topic topic_040d9807-2efa-4f89-acce-ca05300bfc2a, approved/promoted it to update request req_11d2c7f2-85c6-44ea-9e89-ec7fe0ab5db9, dispatched it to an AquaSynth Codex turn, and tightened Face heartbeat instructions so repo-local work requests must be put on Bifrost with enough context instead of living only as Discord asks."}
{"ts":"2026-05-20T21:38:00Z","type":"implementation","status":"ok","note":"Bifrost dispatch receipts now default to the configured BIFROST_DISCORD_CHANNEL_ID / DISCORD_BIFROST_CHANNEL_ID target and use Bifrost's public profile image as the default persona avatar. Local VoidBot config now points Bifrost governance/dispatch traffic at Discord channel 1506760343836430376 (#bifrost), preserving Aquarium for casual swarm chatter."}
{"ts":"2026-05-20T22:36:43Z","type":"implementation","status":"ok","note":"Repo Face CTB active-turn scan now recovers stale approved/running heartbeat jobs by marking them failed through the job queue; Epiphany's 2026-05-20 17:24 UTC zombie job was recovered after 310.6 minutes and her participant row is unfrozen."}

View file

@ -125,6 +125,7 @@ accepted_design:
- First chat under a repo identity must initialize the target repo's own `.voidbot` home: `voice/` for identity and mouth files, `state/` for Face/swarm state, `birth/` for Epiphany birth artifacts, and `logs/` for durable worker output. The birth hook uses EpiphanyAgent's repo-personality/birth-runner path over repo contents and git history instead of inventing a VoidBot-only personality generator.
- Recent chat consensus can now be packaged for any target Codex workspace through `scripts/feed-codex-chat-consensus.mjs`. The helper reads archived Discord messages, defaults to Aquarium, writes a markdown consensus packet plus a Codex prompt under `.voidbot/artifacts/chat-consensus/`, and only feeds `codex exec` when `--execute` is explicit.
- Agent standing heartbeats are owned by one CTB-style initiative scheduler, not by one task per agent. Wall-clock elapsed time advances initiative; the scheduled task only supplies frequent pulses. The scheduler stores per-agent speed/load/groups/heat/effective-speed/next-turn/active-turn state at `REPO_FACE_HEARTBEAT_STATE_PATH`, dispatches repo Faces as ordinary `repo-face-rumination` jobs, and dispatches built-in `void` as the existing moderation rumination runner. Heat can be turned up globally, by participant kind, turn kind, identity, repo, display name, channel, or `all` group without creating new schedulers, but heat only changes recovery speed; it must not fast-forward time. Active-turn freeze and rest gating remain hard brakes unless a direct mention creates room debt. An agent's initiative is frozen while its owner-Codex job or moderation lock is active; recovery is scheduled after completion, not when the job is queued. The Windows pulse task supports a one-minute floor; stream mode should raise heat or temporarily loosen fatigue, not compress wall-clock time.
- Active-turn freeze is bounded by actual job liveness, not blind faith in an old `running` row. Repo Face heartbeat jobs that remain approved/running past the stale-job window are marked failed through the job queue, recorded as `stale_active_turn_recovered`, and removed from the active-turn map so one dead worker claim cannot silence a Face indefinitely.
- Repo Face rest projection is parent-owned state, not a read-only scheduler illusion. If the heartbeat runner projects a Face sleep cycle forward, it must persist the normalized `update_sleep_cycle` operation to that Face's `.cc` state before ready selection. Missed nap windows roll forward as completed rest instead of starting a brand-new nap at inspection time; otherwise overdue agents can be skipped forever as "currently napping" while raw state keeps the same stale boundary.
- Repo Face memory distillation now shares Void's typed maintenance organ instead of relying on rumination turns to eventually clean themselves. `scripts/run-repo-face-heartbeats.ts` starts repo-local `run-void-memory-maintenance.ps1 -ForceDistillation` when a Face is napping with short-term memory or has stale short-term memory older than six hours, with status/logs isolated under the target repo's `.voidbot/status` and `.voidbot/logs`. `run-void-memory-maintenance.ps1` validates the entire proposed operation batch against `voidSelfStateOperationSchema` before applying anything, so one malformed late operation cannot leave a partial maintenance pass. The maintenance prompt is identity-neutral and explicitly forbids shorthand operation payloads, `type` instead of `operation`, `memoryIds` for pruning, and `null` optional fields.
- Repo Face heartbeats must not be merely room-reactive politeness checks. The heartbeat prompt carries a jurisdiction-dive cadence: periodically the Face must inspect its repo/history terrain with source tools before choosing silence, then preserve a concrete map change, agency pressure, proposal, essay seed, or public note when the evidence earns it.