Add Epiphany jobsUpdated notification

This commit is contained in:
Meta Vens 2026-04-26 15:02:45 +01:00
commit 48d6820d7d
22 changed files with 923 additions and 23 deletions

View file

@ -27,6 +27,7 @@ The current spine is real, not aspirational:
- read-only map/churn proposal through `thread/epiphany/propose`
- verifier-backed promotion through `thread/epiphany/promote`
- successful-write notifications through `thread/epiphany/stateUpdated`
- live bound-job progress notifications through `thread/epiphany/jobsUpdated`
- read-only reflection surfaces through `thread/epiphany/scene`, `jobs`, `freshness`, `context`, and `pressure`
- durable investigation checkpointing for compaction-safe planning
- repo-owned heavy Epiphany organs in `epiphany-core/`, with vendored Codex kept as the host seam where practical

View file

@ -46,6 +46,7 @@ The rule is:
| `thread/epiphany/propose` | read-only proposal | landed | Drafts graph/frontier/churn candidates from verified evidence-backed observations. |
| `thread/epiphany/promote` | verifier gate | landed | Rejects or applies candidates through the durable update path. |
| `thread/epiphany/stateUpdated` | notification | landed | Emits updated projected state, source, revision, and changed fields after successful update/promote writes. |
| `thread/epiphany/jobsUpdated` | notification | landed | Emits changed bound-job snapshots for real runtime progress events when the mapped job payload actually changes. |
| `thread/epiphany/scene` | read-only reflection | landed, live-smoked | Compact client scene derived from authoritative Epiphany state, including checkpoint summary reflection. |
| `thread/epiphany/jobs` | read-only reflection | landed, live-smoked | Derived indexing, remap, verification, and specialist-progress slots from typed state and retrieval summaries, with durable `jobBindings` plus live runtime `agent_jobs` overlay when a real owner exists. |
| `thread/epiphany/freshness` | read-only reflection | landed, live-smoked | Retrieval and graph freshness lens derived from retrieval summaries plus graph frontier/churn state, with watcher-backed invalidation inputs for loaded threads. |
@ -130,7 +131,6 @@ Metaphor is compression after source context. It is not decoration for guesses.
These are not landed yet:
- `thread/epiphany/jobsUpdated` or equivalent progress notifications
- an Epiphany-owned long-running job launcher beyond binding to existing runtime `agent_jobs`
- richer evidence-range and graph-shard inspection beyond the landed context shard
- automatic watcher-driven graph/retrieval/invariant invalidation policy on top of the landed freshness reflection
@ -151,7 +151,13 @@ verification, and specialist work, and can overlay durable `jobBindings` onto
live runtime `agent_jobs` snapshots. It does not start, schedule, create, or
notify jobs.
The next missing organ is live progress notification for bound runtime work.
The first live bound-runtime progress notification is also landed as
`thread/epiphany/jobsUpdated`. It rides existing `agent_job_progress:{json}`
background events from the runtime job runner, resolves matching durable
bindings against live `agent_jobs` snapshots, and only emits when the mapped
bound-job payload actually changes. It does not poll state runtime in a loop,
start work, schedule follow-up work, or turn the jobs read surface into a
writer.
Future live job state should describe work like:

View file

@ -24,6 +24,7 @@ The spine exists in seventeen live paths:
- live state notifications and write responses: app-server declares experimental `thread/epiphany/stateUpdated` and emits it with the updated typed state, typed `source`, event-level `revision`, and typed `changedFields` after successful direct updates and accepted promotions, but not after rejected promotions; direct `thread/epiphany/update` and accepted `thread/epiphany/promote` responses expose the same revision/change metadata, accepted promotions always report `Evidence` because verifier evidence is appended even when the patch evidence list is empty, and successful write response/notification states now use the same client-visible live projection as `thread/read`, including retrieval-summary backfill when durable state has no persisted retrieval metadata, in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:1113), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4354), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4419), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4428), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4555), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4608), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4652), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4693), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11327), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11370).
- scene projection: app-server declares experimental read-only `thread/epiphany/scene`, reads the same live/stored Epiphany state view used by `thread/read`, and derives a compact client scene with objective, active subgoal, invariant status counts, graph focus/counts, retrieval status, investigation-checkpoint summary counts, observation/evidence summaries, churn, live/stored source, and available control-plane actions without mutating or persisting state in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:340), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:3924), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4115), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10757).
- job/progress reflection: app-server declares experimental read-only `thread/epiphany/jobs`, reads the same thread view as scene, optionally fills retrieval state for loaded threads with no Epiphany state, and derives four non-authoritative progress slots for retrieval indexing, graph remap, invariant verification, and specialist work. Durable `job_bindings` can now overlay those slots onto live runtime `agent_jobs` snapshots so the surface can show real owner/progress/thread information without scheduling, notifying, mutating, or persisting runtime jobs in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:347), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4142), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4176), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11998).
- bound-job progress notifications: app-server now declares experimental `thread/epiphany/jobsUpdated`, listens for `agent_job_progress:{json}` background events from the runtime job runner, resolves matching durable `job_bindings` against live `agent_jobs` snapshots, and emits only the changed bound-job payloads when the mapped job state actually differs from the last notification for that thread. It does not poll in a loop, mutate Epiphany state, schedule work, or make `thread/epiphany/jobs` a writer.
- freshness reflection: app-server declares experimental read-only `thread/epiphany/freshness`, reads the same live/stored thread view used by scene/jobs/context, fills live retrieval state for loaded threads, and derives one bounded retrieval/graph freshness lens plus, for loaded threads, watcher-backed invalidation inputs with watched root, changed paths, mapped graph-node hits, active-frontier hits, and revision/source identity without mutating, scheduling, notifying, or performing automatic semantic invalidation in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:350), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4167), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4259), [epiphany_invalidation.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/epiphany_invalidation.rs:19), [epiphany_invalidation.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/epiphany_invalidation.rs:67), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4219), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11073).
- targeted context reflection: app-server declares experimental read-only `thread/epiphany/context`, reads the same live/stored Epiphany state view used by scene/jobs, and derives a bounded state shard containing selected graph nodes/edges, links, active frontier, checkpoint, the full durable investigation-checkpoint packet, observations, direct evidence, linked evidence, and missing requested ids without retrieval, proposal, promotion, notification, mutation, or persistence in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:355), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4253), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4261), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11152).
- context-pressure reflection: core token telemetry carries `model_auto_compact_token_limit` beside total/last usage and `model_context_window`; app-server declares experimental read-only `thread/epiphany/pressure`, reads live token usage or the latest stored rollout `TokenCount`, and derives `unknown`/`low`/`elevated`/`high`/`critical` pressure plus a non-acting compaction-prep recommendation without mutating Epiphany state or scheduling compaction in [protocol.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/protocol/src/protocol.rs:2237), [turn_context.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/turn_context.rs:103), [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:2716), [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:360), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4361), [token_usage_replay.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor/token_usage_replay.rs:75), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4306), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11028).
@ -1061,7 +1062,7 @@ The app-server handler:
### Output
The response is a typed progress board derived from the current Epiphany machine. The Phase 6 jobs smoke proved that missing-state jobs can still reflect retrieval while graph/remap and verification stay blocked, ready-state jobs expose revision identity and needed remap/verification pressure, a bound specialist runtime job surfaces `running` plus its runtime job id and active worker thread id, no `thread/epiphany/stateUpdated` notification is emitted by the read, and final thread state revision does not change.
The response is a typed progress board derived from the current Epiphany machine. The Phase 6 jobs smoke proved that missing-state jobs can still reflect retrieval while graph/remap and verification stay blocked, ready-state jobs expose revision identity and needed remap/verification pressure, a bound specialist runtime job surfaces `running` plus its runtime job id and active worker thread id, no `thread/epiphany/stateUpdated` or `thread/epiphany/jobsUpdated` notification is emitted by the read, and final thread state revision does not change.
### Invariant
@ -1278,7 +1279,7 @@ These are deliberately not shipped yet:
- no automatic watcher-driven graph or semantic invalidation.
- no code-intelligence graph.
- no specialist-agent scheduler.
- no `thread/epiphany/jobsUpdated` progress notification or Epiphany-owned long-running job executor.
- no Epiphany-owned long-running job executor beyond the current runtime `agent_jobs` seam.
- no GUI implementation beyond the read-only scene, jobs, freshness, context, and pressure projection surfaces.
- no durable retrieval-summary write from `thread/epiphany/retrieve`.
@ -1303,7 +1304,7 @@ stored Epiphany state
-> successful writes emit thread/epiphany/stateUpdated
```
The current remaining missing organ is not the red pen, not the first chewing motion, not the first state-change bell, not the first client-shaped scene, not the first job/progress reflection, not the first freshness lens, not the first watcher-backed invalidation read, not the first targeted graph/evidence context read, not the first context-pressure gauge, not the first durable investigation checkpoint packet, and no longer the first bounded reorientation verdict. Those exist. Proposal now has the first bits of map memory, focus, and selection hygiene: it can auto-select a bounded evidence-backed observation cluster when ids are omitted, requires accepting recent evidence behind selected observations, prioritizes stronger selected observations for proposal wording, reuses existing architecture nodes by concrete code-ref/path/id checks before creating new path nodes, can rescue unanchored graph nodes through strict unique semantic overlap, follows graph links and incident edges into the frontier, and reports match-kind-aware churn pressure from the actual proposal shape. Distillation now also has the first source-output-aware teeth for tool/model summaries. Promotion now notices risky deltas instead of just checking the shape of the envelope, treats graph expansion as risky even if the patch tries to mumble "low pressure" while dragging a new wall into the blueprint, token-matches strong verifier kinds so `contest` cannot sneak past as `test`, and validates investigation checkpoints against known evidence before the packet can be inked. Successful update/promote writes now emit `thread/epiphany/stateUpdated` with `source: "update"` or `source: "promote"`, event-level revision, and typed changed fields so clients no longer need to poll blindly, infer the cause from timing, or diff the whole state to know which top-level sections moved. The scene, jobs, freshness, context, pressure, and reorient surfaces give clients compact or bounded read-only views without making the GUI/client authoritative, scene/context surface the checkpoint packet back out without becoming writers, freshness surfaces watcher telemetry without becoming a hidden worker, and reorient now turns those signals into an explicit wakeup verdict. The remaining missing organs are a runtime consumer that can act on that verdict without becoming a hidden tyrant, live job progress notifications once real long-running job owners exist, and eventually role-scoped specialist ownership without silently auto-promoting anything.
The current remaining missing organ is not the red pen, not the first chewing motion, not the first state-change bell, not the first client-shaped scene, not the first job/progress reflection, not the first live bound-job notification, not the first freshness lens, not the first watcher-backed invalidation read, not the first targeted graph/evidence context read, not the first context-pressure gauge, not the first durable investigation checkpoint packet, and no longer the first bounded reorientation verdict. Those exist. Proposal now has the first bits of map memory, focus, and selection hygiene: it can auto-select a bounded evidence-backed observation cluster when ids are omitted, requires accepting recent evidence behind selected observations, prioritizes stronger selected observations for proposal wording, reuses existing architecture nodes by concrete code-ref/path/id checks before creating new path nodes, can rescue unanchored graph nodes through strict unique semantic overlap, follows graph links and incident edges into the frontier, and reports match-kind-aware churn pressure from the actual proposal shape. Distillation now also has the first source-output-aware teeth for tool/model summaries. Promotion now notices risky deltas instead of just checking the shape of the envelope, treats graph expansion as risky even if the patch tries to mumble "low pressure" while dragging a new wall into the blueprint, token-matches strong verifier kinds so `contest` cannot sneak past as `test`, and validates investigation checkpoints against known evidence before the packet can be inked. Successful update/promote writes now emit `thread/epiphany/stateUpdated` with `source: "update"` or `source: "promote"`, event-level revision, and typed changed fields so clients no longer need to poll blindly, infer the cause from timing, or diff the whole state to know which top-level sections moved. Runtime `agent_job_progress` background events now also emit `thread/epiphany/jobsUpdated` with changed bound-job payloads so clients no longer need to poll `thread/epiphany/jobs` just to watch one real owner move. The scene, jobs, freshness, context, pressure, and reorient surfaces give clients compact or bounded read-only views without making the GUI/client authoritative, scene/context surface the checkpoint packet back out without becoming writers, freshness surfaces watcher telemetry without becoming a hidden worker, and reorient now turns those signals into an explicit wakeup verdict. The remaining missing organs are a stricter launcher seam if runtime `agent_jobs` stops being enough, a runtime consumer that can act on the reorientation verdict without becoming a hidden tyrant, and eventually role-scoped specialist ownership without silently auto-promoting anything.
```text
model/tool observations
@ -1317,7 +1318,7 @@ model/tool observations
The typed write path exists, the first deterministic observation/evidence distillation path exists, source-output-aware distillation now summarizes noisy tool/model output into typed evidence with final result/error/finished lines ahead of generic warnings, the first deterministic map/churn proposal path exists, proposal-quality hardening now auto-selects bounded observation clusters when ids are omitted, requires accepting recent evidence, reuses existing graph nodes by observed code refs, prioritizes stronger selected observations, rescues unanchored graph nodes by strict semantic overlap, focuses linked frontier context, reports match-kind-aware map-delta pressure, and the first verifier-backed promotion gate exists. The first map/churn safety layer also exists: promotion can reject state replacement patches that are not tied to explicit observations/evidence, that contain structurally broken subgoal, invariant, graph, frontier, checkpoint, investigation-checkpoint, or churn fields, that try to promote risky churn without a warning and stronger verifier evidence, that claim a graph expansion is low-pressure to dodge the warning gate, or that rely on substring verifier-kind accidents. The first live state notification layer also exists: accepted writes publish updated typed state plus event-level revision and typed changed fields; rejected promotions stay quiet.
In natural language: current Epiphany can preserve a map, show a map, project a compact scene from the map, project derived job/progress slots from the map and retrieval summary, project a read-only freshness lens over retrieval and graph staleness, layer watcher-backed invalidation telemetry onto that lens for loaded threads, project a targeted graph/evidence context shard from the map, project context pressure from token telemetry, bank an in-flight investigation packet in durable typed state, reflect that packet back into prompt/scene/context, turn those signals into a read-only resume-versus-regather verdict, retrieve evidence for the map, draft one explicit observation/evidence patch, derive a bounded graph/frontier/churn candidate from verified evidence-backed observations with code refs, choose a small coherent observation set when the caller does not provide one, favor the strongest selected observation when drafting that candidate, reject or accept a verified candidate, apply explicit map/evidence/churn edits, notify clients after successful writes, and sanity-check those edits before they hit the ledger. It still does not invalidate stale graph regions automatically, execute long-running Epiphany jobs as tracked background work, emit job progress notifications, or coordinate specialist ownership. Teeth are installed; chewing is supervised; the first bell rings; the first wall board exists; the first shift board exists; the first freshness clipboard exists; the first door sensor now exists too; the first binder lens exists; the first pressure gauge exists; the first ember packet exists; the first verdict slip now exists too; the next slices are about safe waking and real ownership, not jawbones.
In natural language: current Epiphany can preserve a map, show a map, project a compact scene from the map, project derived job/progress slots from the map and retrieval summary, emit live bound-job progress notifications from runtime `agent_job_progress` events, project a read-only freshness lens over retrieval and graph staleness, layer watcher-backed invalidation telemetry onto that lens for loaded threads, project a targeted graph/evidence context shard from the map, project context pressure from token telemetry, bank an in-flight investigation packet in durable typed state, reflect that packet back into prompt/scene/context, turn those signals into a read-only resume-versus-regather verdict, retrieve evidence for the map, draft one explicit observation/evidence patch, derive a bounded graph/frontier/churn candidate from verified evidence-backed observations with code refs, choose a small coherent observation set when the caller does not provide one, favor the strongest selected observation when drafting that candidate, reject or accept a verified candidate, apply explicit map/evidence/churn edits, notify clients after successful writes, and sanity-check those edits before they hit the ledger. It still does not invalidate stale graph regions automatically, execute Epiphany-owned long-running jobs beyond the borrowed runtime `agent_jobs` seam, or coordinate specialist ownership. Teeth are installed; chewing is supervised; the first bell rings; the first wall board exists; the first shift board exists; the first freshness clipboard exists; the first door sensor now exists too; the first binder lens exists; the first pressure gauge exists; the first ember packet exists; the first verdict slip now exists too; the next slices are about safe waking and real ownership, not jawbones.
## Verification Hooks
@ -1332,7 +1333,8 @@ Current tests cover the landed flows at useful seams:
- map/churn promotion validation for evidence-backed state replacements in `epiphany-core`.
- typed state-update patch application in `codex-core`.
- retrieval ranking, fallback, stale manifest detection, and mocked Qdrant/Ollama indexing in `epiphany-core`.
- app-server protocol serde for `thread/epiphany/retrieve`, `thread/epiphany/index`, `thread/epiphany/distill`, `thread/epiphany/propose`, `thread/epiphany/promote`, `thread/epiphany/update`, and `thread/epiphany/stateUpdated`.
- app-server protocol serde for `thread/epiphany/retrieve`, `thread/epiphany/index`, `thread/epiphany/distill`, `thread/epiphany/propose`, `thread/epiphany/promote`, `thread/epiphany/update`, `thread/epiphany/stateUpdated`, and `thread/epiphany/jobsUpdated`.
- focused app-server notification coverage for `thread/epiphany/jobsUpdated`, including duplicate-suppression when repeated runtime progress would otherwise emit the same bound-job payload twice.
- app-server mapping of core retrieval/index summaries into protocol responses.
- reusable app-server smoke for the richer Phase 5 chain in `tools/epiphany_phase5_smoke.py`, including update/promote response metadata, `thread/epiphany/stateUpdated` notifications with source, revision, changed-field checks, retrieval-summary presence in successful write response/notification states, the verifier-only promotion edge where `Evidence` changes because verifier evidence is appended, invalid direct observation/evidence and replacement update rejection without mutation or notification, and explicit zero-notification assertions for rejected promotions.
- live app-server smoke for the first Phase 6 scene boundary in `tools/epiphany_phase6_scene_smoke.py`.

View file

@ -51,6 +51,7 @@ The landed machine now has:
- read-only map/churn proposal through `thread/epiphany/propose`
- verifier-backed promotion through `thread/epiphany/promote`
- successful-write notification through `thread/epiphany/stateUpdated`
- live bound-runtime progress notification through `thread/epiphany/jobsUpdated`
- response-level and notification-level revision and changed-field metadata
- direct-update validation for malformed appended records and structural replacements
- proposal and promotion rules that reduce map/churn Jenga pressure
@ -83,6 +84,7 @@ These boundaries are more important than the individual method names:
- `thread/epiphany/index` may update the semantic retrieval catalog, but it is not a hidden Epiphany-state writer.
- `thread/epiphany/scene` is a client reflection, not a second source of truth.
- `thread/epiphany/jobs` is a derived reflection over retrieval summaries, typed `jobBindings`, and optional runtime `agent_jobs` snapshots, not a scheduler or durable runtime job store.
- `thread/epiphany/jobsUpdated` is a live notification derived from runtime progress events and bound-job snapshots, not a scheduler, polling daemon, or durable runtime job store.
- `thread/epiphany/freshness` is a derived reflection, not automatic watcher-driven invalidation, a mutation gate, or a hidden refresh scheduler.
- `thread/epiphany/context` is a targeted reflection, not a state writer or hidden proposal engine.
- `thread/epiphany/pressure` is a context-pressure reflection, not an automatic compactor, scheduler, or CRRC coordinator.
@ -123,7 +125,7 @@ and notify typed state. It can.
The next unknowns are:
- how live job progress notifications should work now that `thread/epiphany/jobs` can bind to real runtime owners without making the GUI authoritative
- whether future Epiphany-owned long-running work should keep parasitizing runtime `agent_jobs` now that `thread/epiphany/jobsUpdated` exists, or whether the fork needs a stricter launcher seam
- how the landed watcher-backed invalidation telemetry should be consumed without turning freshness into a secret worker
- how the landed reorientation verdict should be consumed by future automatic CRRC without becoming ceremony machinery
- how much automatic CRRC coordination belongs in runtime once pressure, freshness, watcher, checkpoint, and reorientation signals all exist
@ -135,8 +137,8 @@ Phase 6 should grow observable harness state outward from the typed spine.
Useful candidates:
1. Add live job progress notifications now that durable `jobBindings` can point at real runtime owners.
2. Decide whether future Epiphany-owned long-running work should keep parasitizing runtime `agent_jobs` or needs a stricter fork-owned launcher seam.
1. Decide whether future Epiphany-owned long-running work should keep parasitizing runtime `agent_jobs` or needs a stricter fork-owned launcher seam.
2. Decide how the landed reorientation verdict should be consumed by future automatic CRRC without becoming ceremony machinery.
3. Add targeted scene/jobs/context/pressure/reorient fields only when a client or smoke exposes a real gap.
Do not spend Phase 6 polishing Phase 5 out of anxiety. The Phase 5 smoke harness

View file

@ -28,7 +28,7 @@ Do not trust this file for the exact live HEAD. Always check git.
- Do not copy exact branch or HEAD from this note. Run `git status --short --branch` and `git log --oneline -5`.
- Phase 1 through Phase 5 are complete enough.
- Phase 6 has read-only `thread/epiphany/scene`, `thread/epiphany/jobs`, `thread/epiphany/freshness`, `thread/epiphany/context`, `thread/epiphany/pressure`, and `thread/epiphany/reorient`; jobs can now overlay durable `jobBindings` onto live runtime `agent_jobs` progress for loaded threads, freshness carries watcher-backed invalidation inputs, and reorient turns checkpoint plus freshness/pressure/watcher signals into a read-only resume-versus-regather verdict.
- Phase 6 has read-only `thread/epiphany/scene`, `thread/epiphany/jobs`, `thread/epiphany/freshness`, `thread/epiphany/context`, `thread/epiphany/pressure`, and `thread/epiphany/reorient`; jobs can now overlay durable `jobBindings` onto live runtime `agent_jobs` progress for loaded threads, and `thread/epiphany/jobsUpdated` now translates live `agent_job_progress:{json}` background events into changed bound-job notifications without polling or scheduling. Freshness carries watcher-backed invalidation inputs, and reorient turns checkpoint plus freshness/pressure/watcher signals into a read-only resume-versus-regather verdict.
- Durable in-flight investigation checkpointing is now landed in authoritative typed state, writable through `thread/epiphany/update` or accepted `thread/epiphany/promote`, rendered into the prompt, and reflected through scene/context.
- The repo is an Epiphany fork of Codex, not a Codex preset.
- `vendor/codex` is tracked directly, not a submodule.
@ -62,6 +62,7 @@ The current spine:
- read-only map/churn proposal through `thread/epiphany/propose`
- verifier-backed promotion through `thread/epiphany/promote`
- successful-write notification through `thread/epiphany/stateUpdated`
- live bound-job progress notification through `thread/epiphany/jobsUpdated`
- read-only compact reflection through `thread/epiphany/scene`
- read-only job/progress reflection through `thread/epiphany/jobs`, with durable `jobBindings` plus live runtime `agent_jobs` overlay
- read-only retrieval/graph freshness reflection plus watcher-backed invalidation inputs through `thread/epiphany/freshness`
@ -212,11 +213,10 @@ automatic CRRC. Runtime coordination and next-action execution are still future
work even though read-only job ownership/progress reflection now has a real
runtime seam.
When the user asks to continue, the next likely organ is live
`thread/epiphany/jobsUpdated` or equivalent progress notification now that
durable `jobBindings` can point at real runtime owners through
`thread/epiphany/jobs`, with automatic runtime CRRC still waiting beyond that
boundary.
When the user asks to continue, the next likely organ is deciding whether
future Epiphany-owned long-running work should keep parasitizing runtime
`agent_jobs` now that `thread/epiphany/jobsUpdated` exists, or whether the
fork needs a stricter launcher seam before specialist ownership grows teeth.
Live `thread/epiphany/scene`, `thread/epiphany/jobs`,
`thread/epiphany/freshness`, `thread/epiphany/context`,
@ -230,7 +230,7 @@ guardrails, not the next organs.
- specialist-agent scheduling
- GUI-as-source-of-truth
- automatic runtime CRRC coordinator acting on the landed reorientation verdict
- Epiphany-owned long-running job execution or `thread/epiphany/jobsUpdated`
- Epiphany-owned long-running job execution beyond the current runtime `agent_jobs` seam
- broad event stream beyond the landed state update notification
The machine is good enough to move outward. Do not sand the same edge until the

View file

@ -43,3 +43,4 @@
{"ts": "2026-04-26T06:51:04+00:00", "type": "verification", "status": "ok", "note": "Phase 6 watcher-backed invalidation inputs landed inside read-only thread/epiphany/freshness. Loaded threads now expose runtime watcher telemetry with watched root, changed paths, mapped graph-node/frontier hits, and no state mutation, scheduling, notification, or automatic semantic invalidation. Focused protocol/app-server tests, schema fixture regeneration/checks, app-server build, and tools/epiphany_phase6_invalidation_smoke.py plus tools/epiphany_phase6_freshness_smoke.py passed."}
{"ts": "2026-04-26T10:45:36+00:00", "type": "verification", "status": "ok", "note": "Phase 6 bounded CRRC reorientation policy landed as read-only thread/epiphany/reorient. The surface consumes checkpoint, freshness, watcher, and pressure signals to return resume-versus-regather with explicit checkpoint-path/frontier reasons, without mutation, compaction, scheduling, notification, or silent continuation. Focused protocol/schema checks, app-server mapper tests/build, and tools/epiphany_phase6_reorient_smoke.py plus tools/epiphany_phase6_scene_smoke.py passed."}
{"ts": "2026-04-26T11:30:59+00:00", "type": "verification", "status": "ok", "note": "Phase 6 jobs reflection now carries durable jobBindings in EpiphanyThreadState and overlays real runtime agent_jobs progress in read-only thread/epiphany/jobs, including runtime job ids, worker thread ids, and progress counts. The live seam required direct state-runtime lookup rather than rollout get_state_db backfill. Focused protocol/core/app-server tests, schema fixture checks, app-server build, and tools/epiphany_phase6_jobs_smoke.py passed."}
{"ts": "2026-04-26T14:01:57+00:00", "type": "verification", "status": "ok", "note": "Phase 6 live bound-job progress notification landed as experimental thread/epiphany/jobsUpdated. App-server now translates runtime agent_job_progress:{json} background events into changed bound-job payloads by resolving durable jobBindings against live agent_jobs snapshots, suppressing duplicate unchanged payloads without polling, state mutation, or scheduling. Focused app-server notification coverage, schema fixture checks, and tools/epiphany_phase6_jobs_smoke.py passed."}

View file

@ -61,7 +61,7 @@ accepted_design:
- Epiphany is a fork-level modeling architecture, not merely a Codex collaboration mode.
- Authoritative Epiphany state lives in Codex protocol/core session and rollout state.
- The prompt reads typed state through a bounded `<epiphany_state>` developer fragment.
- Clients read typed state through `Thread.epiphanyState`, compact reflections such as `thread/epiphany/scene`, `thread/epiphany/jobs`, and `thread/epiphany/freshness`, targeted state shards through `thread/epiphany/context`, runtime pressure through `thread/epiphany/pressure`, and bounded CRRC reorientation verdicts through `thread/epiphany/reorient`.
- Clients read typed state through `Thread.epiphanyState`, compact reflections such as `thread/epiphany/scene`, `thread/epiphany/jobs`, and `thread/epiphany/freshness`, targeted state shards through `thread/epiphany/context`, runtime pressure through `thread/epiphany/pressure`, bounded CRRC reorientation verdicts through `thread/epiphany/reorient`, and live bound-job progress through `thread/epiphany/jobsUpdated`.
- Durable typed state writes route through `thread/epiphany/update` or accepted `thread/epiphany/promote`.
- `thread/epiphany/retrieve`, `thread/epiphany/distill`, `thread/epiphany/propose`, `thread/epiphany/scene`, `thread/epiphany/jobs`, `thread/epiphany/freshness`, `thread/epiphany/context`, `thread/epiphany/pressure`, and `thread/epiphany/reorient` stay read-only.
- `thread/epiphany/index` writes the retrieval catalog, not durable Epiphany understanding.
@ -74,13 +74,14 @@ accepted_design:
- The harness now has an in-flight investigation checkpoint packet in durable typed state; automatic CRRC still needs runtime logic that reads the packet and decides whether to resume or force reorientation and re-gathering after compaction.
- GUI/client work must reflect or steer typed state rather than manufacture a second source of truth.
- `thread/epiphany/jobs` is a read-only job/progress reflection: it derives indexing, remap, verification, and specialist slots from authoritative state and retrieval summaries, and can overlay durable `job_bindings` onto live runtime `agent_jobs` snapshots when a real owner exists; it is not a scheduler, notification stream, or durable runtime job table.
- `thread/epiphany/jobsUpdated` is the first live bound-job progress notification: it translates runtime `agent_job_progress:{json}` background events into changed bound-job payloads by resolving durable `job_bindings` against live `agent_jobs` snapshots, without polling in a loop, mutating state, or scheduling work.
- `thread/epiphany/freshness` is the first read-only freshness surface: it derives retrieval and graph staleness from live retrieval summaries plus frontier/churn state, and for loaded threads reflects watcher-backed invalidation inputs with changed paths plus mapped graph/frontier hits, without mutating, scheduling, or performing automatic semantic invalidation.
- `thread/epiphany/context` is the first targeted read-only state-shard surface: it selects graph nodes/edges, active frontier, checkpoint, observations, and evidence from authoritative typed state without mutating or notifying.
- `thread/epiphany/pressure` is the first read-only context-pressure surface: it derives pressure from token telemetry and auto-compact/context limits without compacting, scheduling, mutating, or notifying.
- `thread/epiphany/reorient` is the first bounded CRRC policy surface: it derives a read-only `resume` versus `regather` verdict from checkpoint, freshness, watcher, and pressure signals without mutating state, compacting, scheduling, or silently continuing work.
- Epiphany safety doctrine is to separate cognition from actuation, keep authority explicit, make interruption sacred, stage capability release, and treat operator misuse as a normal design input rather than a rude hypothetical.
- Epiphany is built in the open, but publication mode is not one-size-fits-all: use FOSS where viable, source-available licensing where economics or capability risk demand it, and commercial terms where organizations extract real enterprise value.
- Epiphany-owned long-running job execution, live job progress notifications, and specialist scheduling are still future work.
- Epiphany-owned long-running job execution beyond the current runtime `agent_jobs` seam and specialist scheduling are still future work.
- Handoff, plan, harness-surface, and scratch files must have distinct jobs and must not become duplicate brains.
- A machine is perfect when every remaining part visibly deserves to exist; this applies to persistent memory too.
- Evidence is not an activity feed; repeated "I just did this" entries should be cut or distilled when git already proves the work.
@ -95,7 +96,7 @@ landed_baseline:
- Phase 3: typed client read surface through `Thread.epiphanyState`.
- Phase 4: hybrid retrieval plus explicit Qdrant indexing with BM25 fallback.
- Phase 5: typed update/distill/propose/promote/read/notification control plane.
- Phase 6: read-only scene, job/progress, freshness with watcher-backed invalidation inputs, targeted context, context-pressure, and reorientation policy surfaces, plus durable investigation checkpointing for compaction-safe planning.
- Phase 6: read-only scene, job/progress, freshness with watcher-backed invalidation inputs, targeted context, context-pressure, and reorientation policy surfaces, plus durable investigation checkpointing for compaction-safe planning and live bound-job progress notifications.
guardrails:
- Use `tools/epiphany_phase5_smoke.py` before modifying update, distill, propose, promote, read-projection, or notification behavior.
- Use `tools/epiphany_phase6_scene_smoke.py` before modifying scene projection behavior.
@ -123,5 +124,5 @@ rejected_paths:
reason: persistent state that accumulates every local truth becomes memory cruft and weakens rehydration
current_status:
summary: Phase 1 durable state, Phase 2 prompt integration, Phase 3 typed read, Phase 4 retrieval/indexing, Phase 5 update/distill/propose/promote/read/notification, and Phase 6 read-only scene, jobs, freshness, targeted context, context-pressure reflection, watcher-backed invalidation inputs, durable investigation checkpointing, and bounded CRRC reorientation policy are landed. `thread/epiphany/jobs` now carries durable `job_bindings` in authoritative typed state and overlays real runtime `agent_jobs` progress for bound work, including runtime job id, worker thread ids, and progress counts, without becoming a scheduler or notification stream. `thread/epiphany/reorient` still consumes checkpoint, freshness, watcher, and pressure signals to return a read-only `resume` versus `regather` verdict with explicit checkpoint-path/frontier reasons, but it does not mutate state, compact, schedule, notify, or silently continue work. Focused protocol/core/app-server checks plus the live jobs smoke cover the seam. Investigation checkpoints still live in authoritative typed state, flow through the prompt, and reflect through scene/context without adding a second writer or fake scheduler. Persistent state was cut down so map, handoff, plan, harness surfaces, scratch, and evidence each carry one job instead of becoming a Jenga tower; evidence is now a distilled ledger rather than an activity feed. The global AGENTS Epiphany core command says persistent state is the agent's mind and explicitly permits the agent to ask for self-modification when memory or workflow makes it worse. The continuity goal is to bank the fire before the dark so compaction becomes less like death and more like sleep.
next_action: After compaction, rehydrate and reorient. If the user asks to continue, the next likely Phase 6 organ is live `thread/epiphany/jobsUpdated` or equivalent progress notification now that durable `job_bindings` can point at real runtime owners, while automatic runtime CRRC remains future work that may eventually consume the landed read-only reorientation verdict. Do not restart the completed pressure, freshness, watcher-invalidation, investigation-checkpoint, reorientation, or landed jobs-binding slice unless a concrete regression appears.
summary: Phase 1 durable state, Phase 2 prompt integration, Phase 3 typed read, Phase 4 retrieval/indexing, Phase 5 update/distill/propose/promote/read/notification, and Phase 6 read-only scene, jobs, freshness, targeted context, context-pressure reflection, watcher-backed invalidation inputs, durable investigation checkpointing, bounded CRRC reorientation policy, and live bound-job progress notifications are landed. `thread/epiphany/jobs` now carries durable `job_bindings` in authoritative typed state and overlays real runtime `agent_jobs` progress for bound work, including runtime job id, worker thread ids, and progress counts, without becoming a scheduler. `thread/epiphany/jobsUpdated` now translates runtime `agent_job_progress:{json}` background events into changed bound-job payloads only when the mapped job state actually changes, so clients can watch real owners move without polling the jobs read surface. `thread/epiphany/reorient` still consumes checkpoint, freshness, watcher, and pressure signals to return a read-only `resume` versus `regather` verdict with explicit checkpoint-path/frontier reasons, but it does not mutate state, compact, schedule, notify, or silently continue work. Focused protocol/app-server tests, schema fixture regeneration/checks, app-server build, and the live jobs smoke cover the seam. Investigation checkpoints still live in authoritative typed state, flow through the prompt, and reflect through scene/context without adding a second writer or fake scheduler. Persistent state was cut down so map, handoff, plan, harness surfaces, scratch, and evidence each carry one job instead of becoming a Jenga tower; evidence is now a distilled ledger rather than an activity feed. The global AGENTS Epiphany core command says persistent state is the agent's mind and explicitly permits the agent to ask for self-modification when memory or workflow makes it worse. The continuity goal is to bank the fire before the dark so compaction becomes less like death and more like sleep.
next_action: After compaction, rehydrate and reorient. If the user asks to continue, the next likely Phase 6 organ is deciding whether future Epiphany-owned long-running work should keep parasitizing runtime `agent_jobs` now that `thread/epiphany/jobsUpdated` exists, or whether the fork needs a stricter launcher seam before specialist ownership grows teeth. Automatic runtime CRRC remains future work that may eventually consume the landed read-only reorientation verdict. Do not restart the completed pressure, freshness, watcher-invalidation, investigation-checkpoint, reorientation, jobs-binding, or jobsUpdated slice unless a concrete regression appears.

View file

@ -302,6 +302,10 @@ def run_smoke(args: argparse.Namespace) -> dict[str, Any]:
"thread/epiphany/stateUpdated",
start_index=missing_notification_start,
)
client.require_no_notification(
"thread/epiphany/jobsUpdated",
start_index=missing_notification_start,
)
update_notification_start = len(client.notifications)
update = client.send(
@ -327,6 +331,10 @@ def run_smoke(args: argparse.Namespace) -> dict[str, Any]:
"thread/epiphany/stateUpdated",
start_index=jobs_notification_start,
)
client.require_no_notification(
"thread/epiphany/jobsUpdated",
start_index=jobs_notification_start,
)
final_read = client.send("thread/read", {"threadId": thread_id, "includeTurns": False})
assert final_read is not None
@ -360,6 +368,10 @@ def run_smoke(args: argparse.Namespace) -> dict[str, Any]:
"thread/epiphany/stateUpdated",
start_index=jobs_notification_start,
),
"jobsUpdatedNotificationCount": client.count_notifications(
"thread/epiphany/jobsUpdated",
start_index=jobs_notification_start,
),
"finalReadRevision": final_read["thread"]["epiphanyState"]["revision"],
}

View file

@ -3856,6 +3856,150 @@
],
"type": "object"
},
"ThreadEpiphanyJob": {
"properties": {
"activeThreadIds": {
"items": {
"type": "string"
},
"type": "array"
},
"blockingReason": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"itemsProcessed": {
"format": "uint32",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"itemsTotal": {
"format": "uint32",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"kind": {
"$ref": "#/definitions/ThreadEpiphanyJobKind"
},
"lastCheckpointAtUnixSeconds": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"linkedGraphNodeIds": {
"items": {
"type": "string"
},
"type": "array"
},
"linkedSubgoalIds": {
"items": {
"type": "string"
},
"type": "array"
},
"ownerRole": {
"type": "string"
},
"progressNote": {
"type": [
"string",
"null"
]
},
"runtimeAgentJobId": {
"type": [
"string",
"null"
]
},
"scope": {
"type": "string"
},
"status": {
"$ref": "#/definitions/ThreadEpiphanyJobStatus"
}
},
"required": [
"id",
"kind",
"ownerRole",
"scope",
"status"
],
"type": "object"
},
"ThreadEpiphanyJobKind": {
"enum": [
"indexing",
"remap",
"verification",
"specialist"
],
"type": "string"
},
"ThreadEpiphanyJobStatus": {
"enum": [
"idle",
"needed",
"pending",
"running",
"completed",
"failed",
"cancelled",
"blocked",
"unavailable"
],
"type": "string"
},
"ThreadEpiphanyJobsUpdatedNotification": {
"properties": {
"jobs": {
"items": {
"$ref": "#/definitions/ThreadEpiphanyJob"
},
"type": "array"
},
"source": {
"$ref": "#/definitions/ThreadEpiphanyJobsUpdatedSource"
},
"stateRevision": {
"format": "uint64",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"threadId": {
"type": "string"
}
},
"required": [
"source",
"threadId"
],
"type": "object"
},
"ThreadEpiphanyJobsUpdatedSource": {
"enum": [
"runtimeProgress"
],
"type": "string"
},
"ThreadEpiphanyStateUpdatedField": {
"enum": [
"objective",
@ -6363,6 +6507,26 @@
"title": "Thread/epiphany/stateUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {
"enum": [
"thread/epiphany/jobsUpdated"
],
"title": "Thread/epiphany/jobsUpdatedNotificationMethod",
"type": "string"
},
"params": {
"$ref": "#/definitions/ThreadEpiphanyJobsUpdatedNotification"
}
},
"required": [
"method",
"params"
],
"title": "Thread/epiphany/jobsUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {

View file

@ -4524,6 +4524,26 @@
"title": "Thread/epiphany/stateUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {
"enum": [
"thread/epiphany/jobsUpdated"
],
"title": "Thread/epiphany/jobsUpdatedNotificationMethod",
"type": "string"
},
"params": {
"$ref": "#/definitions/v2/ThreadEpiphanyJobsUpdatedNotification"
}
},
"required": [
"method",
"params"
],
"title": "Thread/epiphany/jobsUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {
@ -15328,6 +15348,43 @@
],
"type": "string"
},
"ThreadEpiphanyJobsUpdatedNotification": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"jobs": {
"items": {
"$ref": "#/definitions/v2/ThreadEpiphanyJob"
},
"type": "array"
},
"source": {
"$ref": "#/definitions/v2/ThreadEpiphanyJobsUpdatedSource"
},
"stateRevision": {
"format": "uint64",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"threadId": {
"type": "string"
}
},
"required": [
"source",
"threadId"
],
"title": "ThreadEpiphanyJobsUpdatedNotification",
"type": "object"
},
"ThreadEpiphanyJobsUpdatedSource": {
"enum": [
"runtimeProgress"
],
"type": "string"
},
"ThreadEpiphanyPressure": {
"properties": {
"basis": {

View file

@ -11726,6 +11726,26 @@
"title": "Thread/epiphany/stateUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {
"enum": [
"thread/epiphany/jobsUpdated"
],
"title": "Thread/epiphany/jobsUpdatedNotificationMethod",
"type": "string"
},
"params": {
"$ref": "#/definitions/ThreadEpiphanyJobsUpdatedNotification"
}
},
"required": [
"method",
"params"
],
"title": "Thread/epiphany/jobsUpdatedNotification",
"type": "object"
},
{
"properties": {
"method": {
@ -13215,6 +13235,43 @@
],
"type": "string"
},
"ThreadEpiphanyJobsUpdatedNotification": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"jobs": {
"items": {
"$ref": "#/definitions/ThreadEpiphanyJob"
},
"type": "array"
},
"source": {
"$ref": "#/definitions/ThreadEpiphanyJobsUpdatedSource"
},
"stateRevision": {
"format": "uint64",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"threadId": {
"type": "string"
}
},
"required": [
"source",
"threadId"
],
"title": "ThreadEpiphanyJobsUpdatedNotification",
"type": "object"
},
"ThreadEpiphanyJobsUpdatedSource": {
"enum": [
"runtimeProgress"
],
"type": "string"
},
"ThreadEpiphanyPressure": {
"properties": {
"basis": {

View file

@ -0,0 +1,148 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ThreadEpiphanyJob": {
"properties": {
"activeThreadIds": {
"items": {
"type": "string"
},
"type": "array"
},
"blockingReason": {
"type": [
"string",
"null"
]
},
"id": {
"type": "string"
},
"itemsProcessed": {
"format": "uint32",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"itemsTotal": {
"format": "uint32",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"kind": {
"$ref": "#/definitions/ThreadEpiphanyJobKind"
},
"lastCheckpointAtUnixSeconds": {
"format": "int64",
"type": [
"integer",
"null"
]
},
"linkedGraphNodeIds": {
"items": {
"type": "string"
},
"type": "array"
},
"linkedSubgoalIds": {
"items": {
"type": "string"
},
"type": "array"
},
"ownerRole": {
"type": "string"
},
"progressNote": {
"type": [
"string",
"null"
]
},
"runtimeAgentJobId": {
"type": [
"string",
"null"
]
},
"scope": {
"type": "string"
},
"status": {
"$ref": "#/definitions/ThreadEpiphanyJobStatus"
}
},
"required": [
"id",
"kind",
"ownerRole",
"scope",
"status"
],
"type": "object"
},
"ThreadEpiphanyJobKind": {
"enum": [
"indexing",
"remap",
"verification",
"specialist"
],
"type": "string"
},
"ThreadEpiphanyJobStatus": {
"enum": [
"idle",
"needed",
"pending",
"running",
"completed",
"failed",
"cancelled",
"blocked",
"unavailable"
],
"type": "string"
},
"ThreadEpiphanyJobsUpdatedSource": {
"enum": [
"runtimeProgress"
],
"type": "string"
}
},
"properties": {
"jobs": {
"items": {
"$ref": "#/definitions/ThreadEpiphanyJob"
},
"type": "array"
},
"source": {
"$ref": "#/definitions/ThreadEpiphanyJobsUpdatedSource"
},
"stateRevision": {
"format": "uint64",
"minimum": 0.0,
"type": [
"integer",
"null"
]
},
"threadId": {
"type": "string"
}
},
"required": [
"source",
"threadId"
],
"title": "ThreadEpiphanyJobsUpdatedNotification",
"type": "object"
}

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,7 @@
// GENERATED CODE! DO NOT MODIFY BY HAND!
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { ThreadEpiphanyJob } from "./ThreadEpiphanyJob";
import type { ThreadEpiphanyJobsUpdatedSource } from "./ThreadEpiphanyJobsUpdatedSource";
export type ThreadEpiphanyJobsUpdatedNotification = { threadId: string, source: ThreadEpiphanyJobsUpdatedSource, stateRevision?: bigint | null, jobs?: Array<ThreadEpiphanyJob>, };

View file

@ -0,0 +1,5 @@
// GENERATED CODE! DO NOT MODIFY BY HAND!
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
export type ThreadEpiphanyJobsUpdatedSource = "runtimeProgress";

View file

@ -331,6 +331,8 @@ export type { ThreadEpiphanyJob } from "./ThreadEpiphanyJob";
export type { ThreadEpiphanyJobKind } from "./ThreadEpiphanyJobKind";
export type { ThreadEpiphanyJobStatus } from "./ThreadEpiphanyJobStatus";
export type { ThreadEpiphanyJobsSource } from "./ThreadEpiphanyJobsSource";
export type { ThreadEpiphanyJobsUpdatedNotification } from "./ThreadEpiphanyJobsUpdatedNotification";
export type { ThreadEpiphanyJobsUpdatedSource } from "./ThreadEpiphanyJobsUpdatedSource";
export type { ThreadEpiphanyPressure } from "./ThreadEpiphanyPressure";
export type { ThreadEpiphanyPressureBasis } from "./ThreadEpiphanyPressureBasis";
export type { ThreadEpiphanyPressureLevel } from "./ThreadEpiphanyPressureLevel";

View file

@ -1126,6 +1126,8 @@ server_notification_definitions! {
ConfigWarning => "configWarning" (v2::ConfigWarningNotification),
#[experimental("thread/epiphany/stateUpdated")]
ThreadEpiphanyStateUpdated => "thread/epiphany/stateUpdated" (v2::ThreadEpiphanyStateUpdatedNotification),
#[experimental("thread/epiphany/jobsUpdated")]
ThreadEpiphanyJobsUpdated => "thread/epiphany/jobsUpdated" (v2::ThreadEpiphanyJobsUpdatedNotification),
FuzzyFileSearchSessionUpdated => "fuzzyFileSearch/sessionUpdated" (FuzzyFileSearchSessionUpdatedNotification),
FuzzyFileSearchSessionCompleted => "fuzzyFileSearch/sessionCompleted" (FuzzyFileSearchSessionCompletedNotification),
#[experimental("thread/realtime/started")]
@ -2552,6 +2554,60 @@ mod tests {
Ok(())
}
#[test]
fn serialize_thread_epiphany_jobs_updated_notification() -> Result<()> {
let notification = ServerNotification::ThreadEpiphanyJobsUpdated(
v2::ThreadEpiphanyJobsUpdatedNotification {
thread_id: "thr_123".to_string(),
source: v2::ThreadEpiphanyJobsUpdatedSource::RuntimeProgress,
state_revision: Some(7),
jobs: vec![v2::ThreadEpiphanyJob {
id: "specialist-work".to_string(),
kind: v2::ThreadEpiphanyJobKind::Specialist,
scope: "runtime-bound specialist work".to_string(),
owner_role: "epiphany-harness".to_string(),
status: v2::ThreadEpiphanyJobStatus::Running,
runtime_agent_job_id: Some("job-123".to_string()),
items_processed: Some(1),
items_total: Some(3),
progress_note: Some("Runtime agent job is running.".to_string()),
last_checkpoint_at_unix_seconds: Some(1_744_500_123),
blocking_reason: None,
active_thread_ids: vec!["worker-thread".to_string()],
linked_subgoal_ids: vec!["phase6".to_string()],
linked_graph_node_ids: vec!["job-surface".to_string()],
}],
},
);
assert_eq!(
json!({
"method": "thread/epiphany/jobsUpdated",
"params": {
"threadId": "thr_123",
"source": "runtimeProgress",
"stateRevision": 7,
"jobs": [{
"id": "specialist-work",
"kind": "specialist",
"scope": "runtime-bound specialist work",
"ownerRole": "epiphany-harness",
"status": "running",
"runtimeAgentJobId": "job-123",
"itemsProcessed": 1,
"itemsTotal": 3,
"progressNote": "Runtime agent job is running.",
"lastCheckpointAtUnixSeconds": 1744500123,
"activeThreadIds": ["worker-thread"],
"linkedSubgoalIds": ["phase6"],
"linkedGraphNodeIds": ["job-surface"]
}]
}
}),
serde_json::to_value(&notification)?,
);
Ok(())
}
#[test]
fn serialize_config_requirements_read() -> Result<()> {
let request = ClientRequest::ConfigRequirementsRead {
@ -3241,6 +3297,35 @@ mod tests {
assert_eq!(reason, Some("thread/epiphany/stateUpdated"));
}
#[test]
fn thread_epiphany_jobs_updated_notification_is_marked_experimental() {
let notification = ServerNotification::ThreadEpiphanyJobsUpdated(
v2::ThreadEpiphanyJobsUpdatedNotification {
thread_id: "thr_123".to_string(),
source: v2::ThreadEpiphanyJobsUpdatedSource::RuntimeProgress,
state_revision: Some(1),
jobs: vec![v2::ThreadEpiphanyJob {
id: "specialist-work".to_string(),
kind: v2::ThreadEpiphanyJobKind::Specialist,
scope: "runtime-bound specialist work".to_string(),
owner_role: "epiphany-harness".to_string(),
status: v2::ThreadEpiphanyJobStatus::Running,
runtime_agent_job_id: Some("job-123".to_string()),
items_processed: Some(1),
items_total: Some(3),
progress_note: Some("Runtime agent job is running.".to_string()),
last_checkpoint_at_unix_seconds: Some(1_744_500_123),
blocking_reason: None,
active_thread_ids: vec!["worker-thread".to_string()],
linked_subgoal_ids: vec!["phase6".to_string()],
linked_graph_node_ids: vec!["job-surface".to_string()],
}],
},
);
let reason = crate::experimental_api::ExperimentalApi::experimental_reason(&notification);
assert_eq!(reason, Some("thread/epiphany/jobsUpdated"));
}
#[test]
fn thread_realtime_start_is_marked_experimental() {
let request = ClientRequest::ThreadRealtimeStart {

View file

@ -4800,6 +4800,26 @@ pub enum ThreadEpiphanyStateUpdatedField {
Mode,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]
pub struct ThreadEpiphanyJobsUpdatedNotification {
pub thread_id: String,
pub source: ThreadEpiphanyJobsUpdatedSource,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[ts(optional = nullable)]
pub state_revision: Option<u64>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub jobs: Vec<ThreadEpiphanyJob>,
}
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]
pub enum ThreadEpiphanyJobsUpdatedSource {
RuntimeProgress,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]

View file

@ -146,6 +146,7 @@ Experimental Epiphany retrieval:
- `thread/epiphany/promote` is an explicit loaded-thread-only promotion gate. It evaluates a patch plus verifier evidence, rejects with reasons without mutating state when policy fails, and applies the patch through the durable update path when accepted. Accepted responses include the landed `revision` and `changedFields`.
- `thread/epiphany/update` is the explicit loaded-thread-only control-plane write path for Epiphany state. It can append observations/evidence and replace bounded typed state fields such as objective, subgoals, graph frontier/checkpoint, scratch, churn, and mode; it bumps the Epiphany revision, returns `revision` plus `changedFields`, and persists a rollout snapshot immediately.
- `thread/epiphany/stateUpdated` is an experimental server notification emitted after successful `thread/epiphany/update` and accepted `thread/epiphany/promote`. It carries the updated typed `epiphanyState`, a `source` of `update` or `promote`, event-level `revision`, and typed `changedFields`; rejected promotions do not emit it.
- `thread/epiphany/jobsUpdated` is an experimental server notification emitted from live `agent_job_progress:{json}` background events when a runtime `agent_jobs` owner is bound through durable `jobBindings` and the mapped bound-job payload actually changes. It carries the current `stateRevision` plus the changed bound job snapshots; it does not turn `thread/epiphany/jobs` into a writer or scheduler.
- `thread/epiphany/freshness` is a read-only reflection surface for retrieval and graph staleness. It reports revision/source identity, retrieval freshness, graph freshness, and exact dirty-path pressure, and for loaded threads now also reports watcher-backed invalidation inputs with watched root, changed paths, mapped graph-node hits, and active-frontier hits. It still does not mutate state, schedule refresh work, or perform automatic watcher-driven semantic invalidation.
- `thread/epiphany/reorient` is a read-only CRRC policy surface. It consumes the durable investigation checkpoint plus freshness, watcher, and pressure signals to return a bounded `resume` versus `regather` verdict with explicit checkpoint-path/frontier reasons. It does not mutate state, compact, notify, schedule, or silently continue work on the caller's behalf.
The request envelope is camelCase, but nested reused Epiphany core DTOs currently keep their core snake_case fields, such as `source_kind`, `understanding_status`, and `diff_pressure`.

View file

@ -2,6 +2,7 @@ use crate::codex_message_processor::ApiVersion;
use crate::codex_message_processor::read_rollout_items_from_rollout;
use crate::codex_message_processor::read_summary_from_rollout;
use crate::codex_message_processor::summary_to_thread;
use crate::codex_message_processor::thread_epiphany_jobs_updated_notification_for_runtime_job;
use crate::error_code::INTERNAL_ERROR_CODE;
use crate::error_code::INVALID_REQUEST_ERROR_CODE;
use crate::outgoing_message::ClientRequestResult;
@ -77,6 +78,7 @@ use codex_app_server_protocol::ServerNotification;
use codex_app_server_protocol::ServerRequestPayload;
use codex_app_server_protocol::SkillsChangedNotification;
use codex_app_server_protocol::TerminalInteractionNotification;
use codex_app_server_protocol::ThreadEpiphanyJobsUpdatedNotification;
use codex_app_server_protocol::ThreadItem;
use codex_app_server_protocol::ThreadNameUpdatedNotification;
use codex_app_server_protocol::ThreadRealtimeClosedNotification;
@ -145,6 +147,7 @@ use codex_protocol::request_user_input::RequestUserInputResponse as CoreRequestU
use codex_sandboxing::policy_transforms::intersect_permission_profiles;
use codex_shell_command::parse_command::shlex_join;
use codex_utils_absolute_path::AbsolutePathBuf;
use serde::Deserialize;
use std::collections::HashMap;
use std::path::Path;
use std::sync::Arc;
@ -167,6 +170,13 @@ struct CommandExecutionCompletionItem {
command_actions: Vec<V2ParsedCommand>,
}
const AGENT_JOB_PROGRESS_PREFIX: &str = "agent_job_progress:";
#[derive(Debug, Deserialize)]
struct AgentJobProgressBackgroundEvent {
job_id: String,
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn apply_bespoke_event_handling(
event: Event,
@ -1974,6 +1984,18 @@ pub(crate) async fn apply_bespoke_event_handling(
)
.await;
}
EventMsg::BackgroundEvent(background_event) => {
if let ApiVersion::V2 = api_version {
maybe_emit_thread_epiphany_jobs_updated(
conversation_id,
&conversation,
background_event.message.as_str(),
&outgoing,
&thread_state,
)
.await;
}
}
EventMsg::ShutdownComplete => {
thread_watch_manager
.note_thread_shutdown(&conversation_id.to_string())
@ -1984,6 +2006,55 @@ pub(crate) async fn apply_bespoke_event_handling(
}
}
async fn maybe_emit_thread_epiphany_jobs_updated(
conversation_id: ThreadId,
conversation: &Arc<CodexThread>,
message: &str,
outgoing: &ThreadScopedOutgoingMessageSender,
thread_state: &Arc<Mutex<ThreadState>>,
) {
let Some(payload_json) = message.strip_prefix(AGENT_JOB_PROGRESS_PREFIX) else {
return;
};
let progress = match serde_json::from_str::<AgentJobProgressBackgroundEvent>(payload_json) {
Ok(progress) => progress,
Err(err) => {
warn!(
thread_id = %conversation_id,
"failed to parse agent job progress background event: {err}"
);
return;
}
};
let Some(notification) = thread_epiphany_jobs_updated_notification_for_runtime_job(
conversation_id,
conversation.as_ref(),
progress.job_id.as_str(),
)
.await
else {
return;
};
if !should_emit_thread_epiphany_jobs_updated(thread_state, &notification).await {
return;
}
outgoing
.send_server_notification(ServerNotification::ThreadEpiphanyJobsUpdated(notification))
.await;
}
async fn should_emit_thread_epiphany_jobs_updated(
thread_state: &Arc<Mutex<ThreadState>>,
notification: &ThreadEpiphanyJobsUpdatedNotification,
) -> bool {
let mut thread_state = thread_state.lock().await;
thread_state.record_epiphany_jobs_update(notification.jobs.as_slice())
}
async fn handle_turn_diff(
conversation_id: ThreadId,
event_turn_id: &str,
@ -4493,6 +4564,206 @@ mod tests {
Ok(())
}
#[tokio::test]
async fn background_agent_job_progress_emits_epiphany_jobs_updated_once() -> Result<()> {
let codex_home = TempDir::new()?;
let config = load_default_config_for_test(&codex_home).await;
let thread_manager = Arc::new(
codex_core::test_support::thread_manager_with_models_provider_and_home(
CodexAuth::create_dummy_chatgpt_auth_for_testing(),
config.model_provider.clone(),
config.codex_home.to_path_buf(),
Arc::new(codex_exec_server::EnvironmentManager::default_for_tests()),
),
);
let codex_core::NewThread {
thread_id: conversation_id,
thread: conversation,
..
} = thread_manager.start_thread(config).await?;
let state_db = conversation.state_db().expect("state db should exist");
state_db
.create_agent_job(
&codex_state::AgentJobCreateParams {
id: "job-specialist".to_string(),
name: "epiphany-specialist".to_string(),
instruction: "Run the runtime-bound specialist smoke.".to_string(),
auto_export: false,
max_runtime_seconds: Some(60),
output_schema_json: None,
input_headers: vec!["row".to_string()],
input_csv_path: "input.csv".to_string(),
output_csv_path: "output.csv".to_string(),
},
&[
codex_state::AgentJobItemCreateParams {
item_id: "item-completed".to_string(),
row_index: 0,
source_id: Some("row-0".to_string()),
row_json: json!({"row": 0}),
},
codex_state::AgentJobItemCreateParams {
item_id: "item-running".to_string(),
row_index: 1,
source_id: Some("row-1".to_string()),
row_json: json!({"row": 1}),
},
codex_state::AgentJobItemCreateParams {
item_id: "item-pending".to_string(),
row_index: 2,
source_id: Some("row-2".to_string()),
row_json: json!({"row": 2}),
},
],
)
.await?;
state_db.mark_agent_job_running("job-specialist").await?;
assert!(
state_db
.mark_agent_job_item_running_with_thread(
"job-specialist",
"item-completed",
"worker-thread-completed",
)
.await?
);
assert!(
state_db
.report_agent_job_item_result(
"job-specialist",
"item-completed",
"worker-thread-completed",
&json!({"accepted": true}),
)
.await?
);
assert!(
state_db
.mark_agent_job_item_running_with_thread(
"job-specialist",
"item-running",
"worker-thread-running",
)
.await?
);
conversation
.epiphany_update_state(codex_core::EpiphanyStateUpdate {
expected_revision: Some(0),
job_bindings: Some(vec![codex_protocol::protocol::EpiphanyJobBinding {
id: "specialist-work".to_string(),
kind: codex_protocol::protocol::EpiphanyJobKind::Specialist,
scope: "runtime-bound specialist work".to_string(),
owner_role: "epiphany-harness".to_string(),
runtime_agent_job_id: Some("job-specialist".to_string()),
linked_subgoal_ids: vec!["phase6-jobs".to_string()],
linked_graph_node_ids: vec!["job-surface".to_string()],
progress_note: Some("Bound to a real runtime specialist job.".to_string()),
blocking_reason: None,
}]),
..Default::default()
})
.await?;
let thread_state = new_thread_state();
let thread_watch_manager = ThreadWatchManager::new();
let (tx, mut rx) = mpsc::channel(CHANNEL_CAPACITY);
let outgoing = Arc::new(OutgoingMessageSender::new(tx));
let outgoing = ThreadScopedOutgoingMessageSender::new(
outgoing,
vec![ConnectionId(1)],
conversation_id,
);
let message = format!(
"{AGENT_JOB_PROGRESS_PREFIX}{}",
json!({
"job_id": "job-specialist",
"total_items": 3,
"pending_items": 1,
"running_items": 1,
"completed_items": 1,
"failed_items": 0,
"eta_seconds": 42,
})
);
apply_bespoke_event_handling(
Event {
id: "turn-epiphany-jobs".to_string(),
msg: EventMsg::BackgroundEvent(codex_protocol::protocol::BackgroundEventEvent {
message: message.clone(),
}),
},
conversation_id,
conversation.clone(),
thread_manager.clone(),
None,
outgoing.clone(),
thread_state.clone(),
thread_watch_manager.clone(),
ApiVersion::V2,
"test-provider".to_string(),
codex_home.path(),
)
.await;
let first = recv_broadcast_message(&mut rx).await?;
match first {
OutgoingMessage::AppServerNotification(
ServerNotification::ThreadEpiphanyJobsUpdated(payload),
) => {
assert_eq!(payload.thread_id, conversation_id.to_string());
assert_eq!(
payload.source,
codex_app_server_protocol::ThreadEpiphanyJobsUpdatedSource::RuntimeProgress
);
assert_eq!(payload.state_revision, Some(1));
assert_eq!(payload.jobs.len(), 1);
let job = &payload.jobs[0];
assert_eq!(job.id, "specialist-work");
assert_eq!(
job.status,
codex_app_server_protocol::ThreadEpiphanyJobStatus::Running
);
assert_eq!(job.runtime_agent_job_id.as_deref(), Some("job-specialist"));
assert_eq!(job.items_processed, Some(1));
assert_eq!(job.items_total, Some(3));
assert_eq!(
job.active_thread_ids,
vec!["worker-thread-running".to_string()]
);
}
other => bail!("unexpected notification: {other:?}"),
}
apply_bespoke_event_handling(
Event {
id: "turn-epiphany-jobs".to_string(),
msg: EventMsg::BackgroundEvent(codex_protocol::protocol::BackgroundEventEvent {
message,
}),
},
conversation_id,
conversation,
thread_manager,
None,
outgoing,
thread_state,
thread_watch_manager,
ApiVersion::V2,
"test-provider".to_string(),
codex_home.path(),
)
.await;
assert!(
rx.try_recv().is_err(),
"duplicate background progress should not emit unchanged jobsUpdated payloads"
);
Ok(())
}
#[tokio::test]
async fn test_construct_mcp_tool_call_begin_notification_with_args() {
let begin_event = McpToolCallBeginEvent {

View file

@ -172,6 +172,8 @@ use codex_app_server_protocol::ThreadEpiphanyJobStatus;
use codex_app_server_protocol::ThreadEpiphanyJobsParams;
use codex_app_server_protocol::ThreadEpiphanyJobsResponse;
use codex_app_server_protocol::ThreadEpiphanyJobsSource;
use codex_app_server_protocol::ThreadEpiphanyJobsUpdatedNotification;
use codex_app_server_protocol::ThreadEpiphanyJobsUpdatedSource;
use codex_app_server_protocol::ThreadEpiphanyPressure;
use codex_app_server_protocol::ThreadEpiphanyPressureBasis;
use codex_app_server_protocol::ThreadEpiphanyPressureLevel;
@ -11934,6 +11936,46 @@ struct EpiphanyAgentJobSnapshot {
active_thread_ids: Vec<String>,
}
pub(crate) async fn thread_epiphany_jobs_updated_notification_for_runtime_job(
thread_id: ThreadId,
thread: &CodexThread,
runtime_agent_job_id: &str,
) -> Option<ThreadEpiphanyJobsUpdatedNotification> {
let state = thread.epiphany_state().await?;
let binding_ids = state
.job_bindings
.iter()
.filter(|binding| binding.runtime_agent_job_id.as_deref() == Some(runtime_agent_job_id))
.map(|binding| binding.id.clone())
.collect::<HashSet<_>>();
if binding_ids.is_empty() {
return None;
}
let state_db_ctx = thread.state_db()?;
let retrieval_override = if state.retrieval.is_none() && binding_ids.contains("retrieval-index")
{
Some(thread.epiphany_retrieval_state().await)
} else {
None
};
let job_resolution = load_epiphany_agent_job_snapshots(Some(&state_db_ctx), Some(&state)).await;
let jobs = map_epiphany_jobs(Some(&state), retrieval_override.as_ref(), &job_resolution)
.into_iter()
.filter(|job| binding_ids.contains(job.id.as_str()))
.collect::<Vec<_>>();
if jobs.is_empty() {
return None;
}
Some(ThreadEpiphanyJobsUpdatedNotification {
thread_id: thread_id.to_string(),
source: ThreadEpiphanyJobsUpdatedSource::RuntimeProgress,
state_revision: Some(state.revision),
jobs,
})
}
async fn load_epiphany_agent_job_snapshots(
state_db_ctx: Option<&StateDbHandle>,
state: Option<&EpiphanyThreadState>,

View file

@ -1,6 +1,7 @@
use crate::outgoing_message::ConnectionId;
use crate::outgoing_message::ConnectionRequestId;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::ThreadEpiphanyJob;
use codex_app_server_protocol::ThreadHistoryBuilder;
use codex_app_server_protocol::Turn;
use codex_app_server_protocol::TurnError;
@ -64,6 +65,7 @@ pub(crate) struct ThreadState {
pub(crate) listener_generation: u64,
listener_command_tx: Option<mpsc::UnboundedSender<ThreadListenerCommand>>,
current_turn_history: ThreadHistoryBuilder,
last_epiphany_jobs_by_job_id: HashMap<String, ThreadEpiphanyJob>,
listener_thread: Option<Weak<CodexThread>>,
}
@ -96,6 +98,7 @@ impl ThreadState {
}
self.listener_command_tx = None;
self.current_turn_history.reset();
self.last_epiphany_jobs_by_job_id.clear();
self.listener_thread = None;
}
@ -124,6 +127,18 @@ impl ThreadState {
self.current_turn_history.reset();
}
}
pub(crate) fn record_epiphany_jobs_update(&mut self, jobs: &[ThreadEpiphanyJob]) -> bool {
let mut changed = false;
for job in jobs {
if self.last_epiphany_jobs_by_job_id.get(job.id.as_str()) != Some(job) {
changed = true;
self.last_epiphany_jobs_by_job_id
.insert(job.id.clone(), job.clone());
}
}
changed
}
}
pub(crate) async fn resolve_server_request_on_thread_listener(