GameCult/Epiphany
Archived
Watch
9
0
Fork
You've already forked Epiphany
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Read Epiphany heartbeat results from runtime spine

Browse source
This commit is contained in:
Meta Vens 2026-05-06 11:38:10 +01:00
commit 7e962ed796
10 changed files with 563 additions and 208 deletions
Download patch file Download diff file Expand all files Collapse all files

8
notes/epiphany-current-algorithmic-map.md
View file

@ -23,7 +23,7 @@ The spine exists in more than twenty-five live paths:
- explicit state updates: app-server routes `thread/epiphany/update` through a loaded `CodexThread` update method that rejects malformed appended observation/evidence graph records and structurally invalid replacement fields, including investigation checkpoints that cite missing evidence and planning records with broken ids/source refs/backlog links/draft acceptance criteria, mutates live `SessionState` only after validation, bumps the revision, and persists an immediate `RolloutItem::EpiphanyState` snapshot in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4620), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:376), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:552), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:1294), and [promotion.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/promotion.rs:44).
- live state notifications and write responses: app-server declares experimental `thread/epiphany/stateUpdated` and emits it with the updated typed state, typed `source`, event-level `revision`, and typed `changedFields` after successful direct updates and accepted promotions, but not after rejected promotions; direct `thread/epiphany/update` and accepted `thread/epiphany/promote` responses expose the same revision/change metadata, accepted promotions always report `Evidence` because verifier evidence is appended even when the patch evidence list is empty, and successful write response/notification states now use the same client-visible live projection as `thread/read`, including retrieval-summary backfill when durable state has no persisted retrieval metadata, in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:1113), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4354), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4419), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4428), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4555), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4608), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4652), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4693), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11327), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11370).
- scene projection: app-server declares experimental read-only `thread/epiphany/scene`, reads the same live/stored Epiphany state view used by `thread/read`, and derives a compact client scene with objective, active subgoal, invariant status counts, graph focus/counts, retrieval status, investigation-checkpoint summary counts, observation/evidence summaries, churn, live/stored source, and available control-plane actions without mutating or persisting state in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:340), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:3924), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4115), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10757).
- job/progress reflection: app-server declares experimental read-only `thread/epiphany/jobs`, reads the same thread view as scene, optionally fills retrieval state for loaded threads with no Epiphany state, and derives four non-authoritative progress slots for retrieval indexing, graph remap, invariant verification, and specialist work. Durable `job_bindings` now act as a thin Epiphany-owned launcher seam with launcher id, authority scope, backend kind, and backend job id. New bindings use the `heartbeat` backend and project as pending heartbeat turns until typed heartbeat/runtime-spine result reporting lands; old `agent_jobs` bindings remain a legacy read overlay only.
- job/progress reflection: app-server declares experimental read-only `thread/epiphany/jobs`, reads the same thread view as scene, optionally fills retrieval state for loaded threads with no Epiphany state, and derives four non-authoritative progress slots for retrieval indexing, graph remap, invariant verification, and specialist work. Durable `job_bindings` now act as a thin Epiphany-owned launcher seam with launcher id, authority scope, backend kind, and backend job id. New bindings use the `heartbeat` backend and app-server opens matching typed runtime-spine job receipts under `state/runtime-spine.msgpack`; old `agent_jobs` bindings remain a legacy read overlay only.
- bound-job progress notifications: app-server still declares experimental `thread/epiphany/jobsUpdated` for legacy Codex evacuation telemetry. It can translate old `agent_job_progress:{json}` background events for old bindings, but heartbeat-owned activation is now the live Epiphany path.
- explicit job control: app-server declares experimental `thread/epiphany/jobLaunch` and `thread/epiphany/jobInterrupt`, routes them through core launch/interrupt handlers that persist heartbeat-backed durable `job_bindings` revisions through the authoritative Epiphany state path, and emit `thread/epiphany/stateUpdated` with source `jobLaunch` or `jobInterrupt` instead of creating SQLite `agent_jobs` rows or pretending jobs reflection is the scheduler.
- freshness reflection: app-server declares experimental read-only `thread/epiphany/freshness`, reads the same live/stored thread view used by scene/jobs/context, fills live retrieval state for loaded threads, and derives one bounded retrieval/graph freshness lens plus, for loaded threads, watcher-backed invalidation inputs with watched root, changed paths, mapped graph-node hits, active-frontier hits, and revision/source identity without mutating, scheduling, notifying, or performing automatic semantic invalidation in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:350), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4167), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4259), [epiphany_invalidation.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/epiphany_invalidation.rs:19), [epiphany_invalidation.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/epiphany_invalidation.rs:67), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4219), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11073).
@ -35,15 +35,15 @@ The spine exists in more than twenty-five live paths:
- reorientation policy: app-server declares experimental read-only `thread/epiphany/reorient`, reuses the same live/stored thread view plus mapped pressure/freshness/watcher signals, and derives a bounded `resume`/`regather` verdict with checkpoint-path/frontier reasons without mutating, compacting, scheduling, notifying, or continuing work in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:365), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4514), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4394), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11424).
- CRRC coordinator recommendation: app-server declares experimental read-only `thread/epiphany/crrc`, reuses the same pressure/freshness/watcher/reorientation mapping plus the fixed `reorient-worker` binding and result read-back helper, recognizes already accepted reorientation findings so an accepted result is not recommended for acceptance again, and returns one bounded recommendation such as continue, prepare checkpoint, launch worker, wait, review, accept, or regather manually without mutating, launching, accepting, compacting, notifying, scheduling, or silently continuing work in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:370), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4661), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4546).
- role ownership projection: app-server declares experimental read-only `thread/epiphany/roles`, reuses the same state/jobs/pressure/reorient/result/CRRC signals as the coordinator loop, and projects implementation, Imagination/planning, modeling/checkpoint, verification/review, and reorientation lanes with owner roles, status, linked jobs, authority scopes, and recommended scene actions without launching, scheduling, accepting, compacting, notifying, or mutating state in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:350), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4161), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4310).
- role specialist launch/read-back/acceptance: app-server declares experimental `thread/epiphany/roleLaunch`, read-only `thread/epiphany/roleResult`, and write-gated `thread/epiphany/roleAccept`. `roleLaunch` accepts fixed Imagination/planning, modeling/checkpoint, and verification/review role ids, builds bounded specialist payloads from authoritative Epiphany state, routes them through the existing `epiphany_launch_job` job-control seam, and emits the normal `jobLaunch` state update for `jobBindings` only. `roleResult` resolves the fixed or requested binding through the current `agent_jobs` backend and projects structured output as a reviewable finding with verdict, summary, next safe move, inspected files, frontier/evidence ids, optional `statePatch`, optional `selfPatch`, coordinator `selfPersistence` review, raw result, and errors without mutation, promotion, scheduling, or hidden continuation. `selfPatch` is a Ghostlight-shaped lane-memory request, not project truth: the coordinator accepts only role-matched, bounded, evidence-grounded memory/goal/value/private-note mutations and refuses objectives, graphs, checkpoints, scratch, planning, jobs, authority, code edits, or raw transcripts with reasons. `roleAccept` accepts completed Imagination findings only when they contain a valid planning-only patch with draft Objective Draft review gates, accepts completed modeling/checkpoint findings only with a modeling-safe `statePatch`, applies accepted project patches through the existing Epiphany state validator, appends audit observation/evidence, and carries the same `selfPersistence` review. GUI/coordinator accept paths then apply accepted `selfPatch` requests to typed CultCache role dossiers in `state/agents.msgpack` through the native `epiphany-agent-memory-store` binary. Implementation stays owned by the main coding agent, verification output remains review-only, and reorientation stays on the CRRC-specific `reorientLaunch` / `reorientResult` / `reorientAccept` path.
- role specialist launch/read-back/acceptance: app-server declares experimental `thread/epiphany/roleLaunch`, read-only `thread/epiphany/roleResult`, and write-gated `thread/epiphany/roleAccept`. `roleLaunch` accepts fixed Imagination/planning, modeling/checkpoint, and verification/review role ids, builds bounded specialist payloads from authoritative Epiphany state, routes them through the heartbeat-backed `epiphany_launch_job` seam, opens a typed runtime-spine job receipt, and emits the normal `jobLaunch` state update for `jobBindings` only. `roleResult` resolves heartbeat bindings through typed runtime-spine job results first, falling back to legacy `agent_jobs` only for old bindings, and projects structured output as a reviewable finding with verdict, summary, next safe move, inspected files, frontier/evidence ids, optional `statePatch`, optional `selfPatch`, coordinator `selfPersistence` review, raw result, and errors without mutation, promotion, scheduling, or hidden continuation. `selfPatch` is a Ghostlight-shaped lane-memory request, not project truth: the coordinator accepts only role-matched, bounded, evidence-grounded memory/goal/value/private-note mutations and refuses objectives, graphs, checkpoints, scratch, planning, jobs, authority, code edits, or raw transcripts with reasons. `roleAccept` accepts completed heartbeat findings from typed runtime-spine results, accepts completed Imagination findings only when they contain a valid planning-only patch with draft Objective Draft review gates, accepts completed modeling/checkpoint findings only with a modeling-safe `statePatch`, applies accepted project patches through the existing Epiphany state validator, appends audit observation/evidence, and carries the same `selfPersistence` review. GUI/coordinator accept paths then apply accepted `selfPatch` requests to typed CultCache role dossiers in `state/agents.msgpack` through the native `epiphany-agent-memory-store` binary. Implementation stays owned by the main coding agent, verification output remains review-only, and reorientation stays on the CRRC-specific `reorientLaunch` / `reorientResult` / `reorientAccept` path.
- shared prompt memory projection and role dossiers: the prompt config now carries `[shared].persistent_memory` in [epiphany_specialists.toml](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/prompts/epiphany_specialists.toml:2). App-server parses that block, prepends it through `epiphany_agent_prompt_with_memory`, and therefore projects the same Ghostlight-derived memory into fixed role specialist instructions, reorientation-worker instructions, coordinator notes, and CRRC checkpoint steering in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:13260), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:13349), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:13594), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:13603), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:16119). The GUI-launched Hands implementation lane prepends the same memory in [epiphany_gui_action.py](E:/Projects/EpiphanyAgent/tools/epiphany_gui_action.py:52), while the base `<epiphany_state>` doctrine reinforces the Perfect Machine rule through [prompt.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/prompt.rs:31) and [epiphany_doctrine.md](E:/Projects/EpiphanyAgent/epiphany-core/src/prompts/epiphany_doctrine.md:1). Individual Ghostlight-shaped role dossiers now live in typed CultCache `state/agents.msgpack`, and the native `epiphany-agent-memory-store` binary validates, reviews, and applies bounded self-memory patches through the Rust store.
- fixed-lane coordinator: app-server declares experimental read-only `thread/epiphany/coordinator`, composes role, pressure, reorient, CRRC, role result, and reorient result signals, and returns one current action without mutation. Review acceptance and implementation clearance are intentionally different: an accepted verification finding clears implementation only when the verifier verdict is `pass`; accepted `needs-evidence`, `needs-review`, or `fail` findings route back toward modeling/checkpoint strengthening before code work resumes. `regatherManually` is now fallback, not a supervisor-puppeteering excuse: completed unaccepted modeling findings stop at `reviewModelingResult`, modeling acceptance precedes verification, and fixed lane actions are considered before manual regather.
- MVP operator view: native `epiphany-mvp-status` launches app-server over stdio, starts or reads one thread, gathers scene, planning, pressure, reorient, jobs, roles, Imagination/modeling/verification role results, reorient result, CRRC responses, heartbeat, and Face bubble artifacts, and renders them as text or JSON so the loop can be dogfooded without Python owning the product surface.
- MVP dogfood runner: `tools/epiphany_mvp_dogfood.py` launches app-server over stdio, drives a bounded state/role/CRRC/reorientation loop, and writes operator-safe JSON/text artifacts plus sealed transcript diagnostics under `.epiphany-dogfood/mvp-loop` so a human can audit the run through projections and compare it against a vanilla Codex reference.
- Agent function/API telemetry: native `epiphany-agent-telemetry` mechanically parses sealed AppServerClient JSONL transcripts into operator-safe telemetry with method counts, request shape, job/status/path fields, and visible function/tool names while sealing text, turn items, raw results, and direct agent messages.
- reorientation launch: app-server now declares experimental `thread/epiphany/reorientLaunch`, requires a loaded thread plus durable investigation checkpoint, consumes the live reorientation verdict, and launches one fixed `reorient-worker` binding through the existing job-control seam with explicit resume-versus-regather scope, checkpoint-derived payload, and structured-output expectations instead of pretending CRRC suddenly grew a scheduler in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:370), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4575), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4519).
- reorientation result read-back: app-server now declares experimental read-only `thread/epiphany/reorientResult`, defaults to the fixed `reorient-worker` binding, resolves that binding through the current `agent_jobs` backend when available, reads the single backend item result, and projects a reviewable finding with mode, summary, next safe move, checkpoint-validity flag, inspected files, frontier ids, evidence ids, raw result, and any job/item errors without mutating state, promoting evidence, or scheduling follow-up work in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:374), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4610), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4684).
- reorientation finding acceptance: app-server now declares experimental `thread/epiphany/reorientAccept`, requires a loaded thread and a completed `reorient-worker` backend result, appends accepted observation/evidence records, optionally banks the finding into scratch and the durable investigation checkpoint, persists through the normal typed update path, and emits `thread/epiphany/stateUpdated` with source `reorientAccept` without accepting pending work, auto-promoting arbitrary worker output, launching follow-up work, or silently continuing implementation in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:378), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4635), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4885).
- reorientation result read-back: app-server now declares experimental read-only `thread/epiphany/reorientResult`, defaults to the fixed `reorient-worker` binding, resolves heartbeat bindings through typed runtime-spine job results first, falls back to legacy `agent_jobs` only for old bindings, and projects a reviewable finding with mode, summary, next safe move, checkpoint-validity flag, inspected files, frontier ids, evidence ids, raw result, and any errors without mutating state, promoting evidence, or scheduling follow-up work.
- reorientation finding acceptance: app-server now declares experimental `thread/epiphany/reorientAccept`, requires a loaded thread and a completed `reorient-worker` result, accepts heartbeat-backed results from typed runtime-spine receipts before falling back to old bindings, appends accepted observation/evidence records, optionally banks the finding into scratch and the durable investigation checkpoint, persists through the normal typed update path, and emits `thread/epiphany/stateUpdated` with source `reorientAccept` without accepting pending work, auto-promoting arbitrary worker output, launching follow-up work, or silently continuing implementation in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:378), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4635), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4885).
- retrieval/indexing: app-server routes `thread/epiphany/retrieve` and `thread/epiphany/index` through loaded `CodexThread` host methods into `epiphany-core` in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4356), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4432), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:467), and [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:449).
The thick Epiphany-owned implementation is mostly outside vendored Codex now: