Add Epiphany investigation checkpoints
This commit is contained in:
Meta Vens
parent
e1298e58a9
commit
71ea1ed2ea
17 changed files with 578 additions and 40 deletions
|
|
@ -35,9 +35,9 @@ The rule is:
|
|||
|
||||
| Surface | Kind | Status | Role |
|
||||
| --- | --- | --- | --- |
|
||||
| `EpiphanyThreadState` | protocol/core state | landed | Authoritative typed model of objective, subgoals, invariants, graphs, retrieval, evidence, scratch, churn, and mode. |
|
||||
| `EpiphanyThreadState` | protocol/core state | landed | Authoritative typed model of objective, subgoals, invariants, graphs, retrieval, investigation checkpoint, evidence, scratch, churn, and mode. |
|
||||
| `RolloutItem::EpiphanyState` | persistence | landed | Durable snapshot for resume, fork, rollback, and compaction-safe reconstruction. |
|
||||
| `<epiphany_state>` prompt fragment | prompt input | landed | Bounded developer-context summary rendered from typed state. |
|
||||
| `<epiphany_state>` prompt fragment | prompt input | landed | Bounded developer-context summary rendered from typed state, including the durable investigation packet when present. |
|
||||
| `Thread.epiphanyState` | client read | landed | Typed state on hydrated thread payloads. |
|
||||
| `thread/epiphany/retrieve` | read-only query | landed | Hybrid repo retrieval; exact/path plus semantic BM25/Qdrant when available. |
|
||||
| `thread/epiphany/index` | retrieval catalog write | landed | Explicit semantic indexing path; updates retrieval catalog, not durable Epiphany understanding. |
|
||||
|
|
@ -46,9 +46,10 @@ The rule is:
|
|||
| `thread/epiphany/propose` | read-only proposal | landed | Drafts graph/frontier/churn candidates from verified evidence-backed observations. |
|
||||
| `thread/epiphany/promote` | verifier gate | landed | Rejects or applies candidates through the durable update path. |
|
||||
| `thread/epiphany/stateUpdated` | notification | landed | Emits updated projected state, source, revision, and changed fields after successful update/promote writes. |
|
||||
| `thread/epiphany/scene` | read-only reflection | landed, live-smoked | Compact client scene derived from authoritative Epiphany state. |
|
||||
| `thread/epiphany/scene` | read-only reflection | landed, live-smoked | Compact client scene derived from authoritative Epiphany state, including checkpoint summary reflection. |
|
||||
| `thread/epiphany/jobs` | read-only reflection | landed, live-smoked | Derived indexing, remap, verification, and specialist-progress slots from typed state and retrieval summaries. |
|
||||
| `thread/epiphany/context` | read-only reflection | landed, live-smoked | Targeted state shard for graph nodes/edges, active frontier, checkpoint, observations, and evidence. |
|
||||
| `thread/epiphany/context` | read-only reflection | landed, live-smoked | Targeted state shard for graph nodes/edges, active frontier, graph checkpoint, investigation checkpoint, observations, and evidence. |
|
||||
| `thread/epiphany/pressure` | read-only reflection | landed, live-smoked | Context-pressure gauge derived from token telemetry and recorded auto-compact/context limits. |
|
||||
|
||||
## Write Authority
|
||||
|
||||
|
|
@ -68,6 +69,7 @@ The following must stay read-only:
|
|||
- `thread/epiphany/scene`
|
||||
- `thread/epiphany/jobs`
|
||||
- `thread/epiphany/context`
|
||||
- `thread/epiphany/pressure`
|
||||
|
||||
`thread/epiphany/index` is a write to the retrieval catalog. It is not a
|
||||
license to mutate map/evidence/churn state as a side effect.
|
||||
|
|
@ -100,6 +102,7 @@ The important categories are:
|
|||
- architecture/dataflow graphs
|
||||
- graph frontier and checkpoint
|
||||
- retrieval summary
|
||||
- investigation checkpoint
|
||||
- scratch
|
||||
- observations
|
||||
- recent evidence
|
||||
|
|
@ -179,6 +182,7 @@ It selects from existing typed state only:
|
|||
- graph edges by id plus incident edges for selected nodes
|
||||
- graph links touching selected nodes
|
||||
- active frontier and current graph checkpoint
|
||||
- full durable investigation checkpoint packet when present
|
||||
- observations by id
|
||||
- evidence by id plus evidence linked from selected observations by default
|
||||
- missing requested or active-frontier ids, so clients do not have to guess whether a shard was empty or stale
|
||||
|
|
@ -207,10 +211,12 @@ Future runtime behavior should:
|
|||
- distinguish what survived from what was discarded
|
||||
- make compaction visible instead of pretending continuity is magic
|
||||
|
||||
Until that signal exists, agents can only infer pressure from indirect cues:
|
||||
long transcripts, compaction warnings, quota pressure, and a shrinking safe
|
||||
landing zone. That is a scar, not a sensor. Automatic CRRC needs typed
|
||||
pressure/freshness telemetry instead of vibes with a clipboard.
|
||||
The pressure signal now exists as `thread/epiphany/pressure`, and durable
|
||||
investigation packets now exist in typed state plus prompt/scene/context
|
||||
reflection. What does not exist yet is the runtime coordinator that consumes
|
||||
those signals. Automatic CRRC still needs typed freshness/invalidity telemetry
|
||||
and an explicit policy for when a checkpoint means "resume" versus "re-gather"
|
||||
instead of vibes with a clipboard.
|
||||
|
||||
Compaction should squeeze scratch, not the map.
|
||||
|
||||
|
|
|
|||
|
|
@ -14,17 +14,17 @@ The spine exists in fifteen live paths:
|
|||
|
||||
- protocol state shape: `EpiphanyThreadState`, `RolloutItem::EpiphanyState`, retrieval summaries, and prompt tags live in [protocol.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/protocol/src/protocol.rs:103).
|
||||
- in-memory session state: `SessionState` stores an optional `EpiphanyThreadState`, and `Session` exposes thin async accessors over it in [state/session.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/state/session.rs:36) and [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:1240).
|
||||
- prompt injection: `build_initial_context` reads the optional state and adds a bounded `<epiphany_state>` developer fragment in [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:2434).
|
||||
- prompt injection: `build_initial_context` reads the optional state and adds a bounded `<epiphany_state>` developer fragment; the renderer now includes a durable investigation-checkpoint section when authoritative state carries one, so a resumed agent can see whether planning/source gathering was banked or must be re-gathered in [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:2434) and [prompt.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/prompt.rs:33).
|
||||
- per-turn persistence: normal turn setup persists one `RolloutItem::EpiphanyState` after the `TurnContext` when state exists in [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:2677).
|
||||
- client read hydration: app-server thread views attach live or reconstructed `thread.epiphanyState` in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4706) and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4822).
|
||||
- explicit distillation proposals: app-server routes read-only `thread/epiphany/distill` through a loaded-thread handler into `epiphany-core` so one explicit observation can become a patch candidate without mutating state; tool/command/shell/model sources now get typed evidence kinds and bounded salient-output summaries instead of dumping raw output into durable state in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4358) and [distillation.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/distillation.rs:28).
|
||||
- explicit map/churn proposals: app-server routes read-only `thread/epiphany/propose` through a loaded-thread handler into `epiphany-core` so verified observations with code refs and accepting recent evidence can be selected explicitly or auto-selected as a bounded path cluster, prioritize the strongest selected observation, focus or extend architecture graph nodes, rescue unanchored graph nodes through strict semantic overlap, carry linked dataflow nodes and incident graph edges into the frontier, update churn candidates with match-kind-aware map-delta judgment and pressure, and still avoid mutation until promotion in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4429) and [proposal.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/proposal.rs:115).
|
||||
- explicit promotion gates: app-server routes `thread/epiphany/promote` through a loaded-thread handler into `epiphany-core` policy evaluation, rejects failed verifier evidence without mutation, applies accepted candidates through the same durable update path, and now treats medium/high/expanded/broadening/semantic churn deltas as needing explicit rationale plus stronger verifier evidence whose kind is token-matched rather than substring-matched in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4501) and [promotion.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/promotion.rs:33).
|
||||
- explicit state updates: app-server routes `thread/epiphany/update` through a loaded `CodexThread` update method that rejects malformed appended observation/evidence graph records and structurally invalid replacement fields, mutates live `SessionState` only after validation, bumps the revision, and persists an immediate `RolloutItem::EpiphanyState` snapshot in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4620), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:376), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:552), and [promotion.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/promotion.rs:44).
|
||||
- explicit promotion gates: app-server routes `thread/epiphany/promote` through a loaded-thread handler into `epiphany-core` policy evaluation, rejects failed verifier evidence without mutation, applies accepted candidates through the same durable update path, and now treats medium/high/expanded/broadening/semantic churn deltas as needing explicit rationale plus stronger verifier evidence whose kind is token-matched rather than substring-matched; the same gate also validates durable investigation-checkpoint packets, including linked evidence ids, before they can be inked into state in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4501) and [promotion.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/promotion.rs:33).
|
||||
- explicit state updates: app-server routes `thread/epiphany/update` through a loaded `CodexThread` update method that rejects malformed appended observation/evidence graph records and structurally invalid replacement fields, including investigation checkpoints that cite missing evidence, mutates live `SessionState` only after validation, bumps the revision, and persists an immediate `RolloutItem::EpiphanyState` snapshot in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4620), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:376), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:552), and [promotion.rs](E:/Projects/EpiphanyAgent/epiphany-core/src/promotion.rs:44).
|
||||
- live state notifications and write responses: app-server declares experimental `thread/epiphany/stateUpdated` and emits it with the updated typed state, typed `source`, event-level `revision`, and typed `changedFields` after successful direct updates and accepted promotions, but not after rejected promotions; direct `thread/epiphany/update` and accepted `thread/epiphany/promote` responses expose the same revision/change metadata, accepted promotions always report `Evidence` because verifier evidence is appended even when the patch evidence list is empty, and successful write response/notification states now use the same client-visible live projection as `thread/read`, including retrieval-summary backfill when durable state has no persisted retrieval metadata, in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:1113), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4354), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4419), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4428), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4555), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4608), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4652), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4693), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11327), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:11370).
|
||||
- scene projection: app-server declares experimental read-only `thread/epiphany/scene`, reads the same live/stored Epiphany state view used by `thread/read`, and derives a compact client scene with objective, active subgoal, invariant status counts, graph focus/counts, retrieval status, observation/evidence summaries, churn, live/stored source, and available control-plane actions without mutating or persisting state in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:340), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:3924), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4090), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10629).
|
||||
- scene projection: app-server declares experimental read-only `thread/epiphany/scene`, reads the same live/stored Epiphany state view used by `thread/read`, and derives a compact client scene with objective, active subgoal, invariant status counts, graph focus/counts, retrieval status, investigation-checkpoint summary counts, observation/evidence summaries, churn, live/stored source, and available control-plane actions without mutating or persisting state in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:340), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:3924), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4090), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10629).
|
||||
- job/progress reflection: app-server declares experimental read-only `thread/epiphany/jobs`, reads the same thread view as scene, optionally fills retrieval state for loaded threads with no Epiphany state, and derives four non-authoritative progress slots for retrieval indexing, graph remap, invariant verification, and future specialist work without scheduling, notifying, mutating, or persisting jobs in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:345), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4104), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4125), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10992).
|
||||
- targeted context reflection: app-server declares experimental read-only `thread/epiphany/context`, reads the same live/stored Epiphany state view used by scene/jobs, and derives a bounded state shard containing selected graph nodes/edges, links, active frontier, checkpoint, observations, direct evidence, linked evidence, and missing requested ids without retrieval, proposal, promotion, notification, mutation, or persistence in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:350), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4182), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4186), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10791).
|
||||
- targeted context reflection: app-server declares experimental read-only `thread/epiphany/context`, reads the same live/stored Epiphany state view used by scene/jobs, and derives a bounded state shard containing selected graph nodes/edges, links, active frontier, checkpoint, the full durable investigation-checkpoint packet, observations, direct evidence, linked evidence, and missing requested ids without retrieval, proposal, promotion, notification, mutation, or persistence in [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:350), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4182), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4186), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10791).
|
||||
- context-pressure reflection: core token telemetry carries `model_auto_compact_token_limit` beside total/last usage and `model_context_window`; app-server declares experimental read-only `thread/epiphany/pressure`, reads live token usage or the latest stored rollout `TokenCount`, and derives `unknown`/`low`/`elevated`/`high`/`critical` pressure plus a non-acting compaction-prep recommendation without mutating Epiphany state or scheduling compaction in [protocol.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/protocol/src/protocol.rs:2237), [turn_context.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/turn_context.rs:103), [session/mod.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/session/mod.rs:2716), [common.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs:355), [v2.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs:4284), [token_usage_replay.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor/token_usage_replay.rs:75), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4244), and [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:10811).
|
||||
- retrieval/indexing: app-server routes `thread/epiphany/retrieve` and `thread/epiphany/index` through loaded `CodexThread` host methods into `epiphany-core` in [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4231), [codex_message_processor.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs:4149), [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:467), and [codex_thread.rs](E:/Projects/EpiphanyAgent/vendor/codex/codex-rs/core/src/codex_thread.rs:449).
|
||||
|
||||
|
|
@ -129,7 +129,7 @@ The current landed machine is still coherent. The core shape is not ornamental:
|
|||
|
||||
Nothing in the current typed spine obviously deserves deletion right now. The parts that would turn this into Jenga are still correctly listed as non-flows: automatic tool-output ingestion, watcher invalidation, specialist scheduling, broad event streaming, and an automatic Compact-Rehydrate-Reorient-Continue coordinator. Thin GUI/client reflection is no longer forbidden as a category, provided it reflects and steers typed state rather than inventing a second source of truth.
|
||||
|
||||
The next Perfect Machine move is not adding another writer, not re-hardening Phase 5, and no longer the first scene, job, targeted context, or pressure reflection. Those surfaces now let clients inspect the machine Epiphany already built at summary, progress, bounded-detail, and context-pressure levels. The next outward work should add a missing observation source only when there is a concrete consumer: in-flight investigation checkpoints, watcher/freshness inputs, or live progress notifications once real long-running job owners exist.
|
||||
The next Perfect Machine move is not adding another writer, not re-hardening Phase 5, and no longer the first scene, job, targeted context, pressure, or investigation-checkpoint reflection. Those surfaces now let clients inspect the machine Epiphany already built at summary, progress, bounded-detail, context-pressure, and durable-planning-packet levels. The next outward work should add a missing observation source only when there is a concrete consumer: watcher/freshness inputs, live progress notifications once real long-running job owners exist, or future CRRC coordination that consumes the landed checkpoint instead of inventing transcript telepathy.
|
||||
|
||||
## Natural Language Spine
|
||||
|
||||
|
|
@ -144,9 +144,9 @@ This is the same flow in the language Epiphany is supposed to make the model car
|
|||
| Turn execution | Codex still runs the ordinary model/tool loop; Epiphany is context and discipline around that loop, not a replacement engine yet. | The same workshop machines run, but now there is a work order on the wall. |
|
||||
| Persistence | After a real user turn, Epiphany snapshots the current model beside the normal turn context. | The field notebook gets dated and shelved after each real work session. |
|
||||
| Thread read | Clients can ask for the current Epiphany model as typed data instead of scraping prompt text or transcript debris. | The dashboard reads the blueprint file directly, not a photo of the workbench. |
|
||||
| Scene projection | Clients can ask for a compact reflection of the typed model: active goal, graph focus, retrieval status, evidence counts, churn, and available control actions. It is derived from state, not a second state store. | A foreman-facing wall board copied from the blueprint ledger; useful at a glance, legally not the blueprint. |
|
||||
| Scene projection | Clients can ask for a compact reflection of the typed model: active goal, graph focus, retrieval status, compact investigation-checkpoint summary, evidence counts, churn, and available control actions. It is derived from state, not a second state store. | A foreman-facing wall board copied from the blueprint ledger; useful at a glance, legally not the blueprint. |
|
||||
| Job projection | Clients can ask which Epiphany work slots are idle, needed, blocked, running, or unavailable for indexing, remap, verification, and future specialists. The slots are derived from state and retrieval summaries; they do not start work. | A shift board listing which crews would be needed if work begins, not a foreman secretly hiring people in the back room. |
|
||||
| Context projection | Clients can ask for one bounded slice of the typed model: selected graph nodes/edges, active frontier, checkpoint, observations, direct evidence, linked evidence, and missing ids. It is derived from state and does not propose or promote anything. | A librarian opens the exact ledger pages you named, adds the active bookmark by default, and says which requested tabs were not in the book. |
|
||||
| Context projection | Clients can ask for one bounded slice of the typed model: selected graph nodes/edges, active frontier, graph checkpoint, full investigation checkpoint packet, observations, direct evidence, linked evidence, and missing ids. It is derived from state and does not propose or promote anything. | A librarian opens the exact ledger pages you named, adds the active bookmark by default, and says which requested tabs were not in the book. |
|
||||
| Pressure projection | Clients can ask how close the live or stored thread is to compaction pressure using token usage, context window, and the recorded auto-compact threshold. The answer may recommend preparation, but it does not act. | A pressure gauge on the boiler; it does not grab the wrench or pretend steam is a moral failing. |
|
||||
| Retrieval | A loaded thread can ask a structured question of the repo through one typed retrieval surface. | A librarian brings back marked pages instead of making the agent rummage through every shelf. |
|
||||
| Indexing | Persistent semantic memory is built only through an explicit indexing path. | The librarian updates the card catalog only when asked, not while pretending to answer a question. |
|
||||
|
|
@ -259,7 +259,7 @@ It then builds the normal developer bundle. After collaboration-mode instruction
|
|||
|
||||
- overview
|
||||
- operating discipline
|
||||
- bounded optional sections for subgoals, invariants, graphs, scratch, observations, evidence, churn, and mode
|
||||
- bounded optional sections for subgoals, invariants, graphs, investigation checkpoint, scratch, observations, evidence, churn, and mode
|
||||
|
||||
Code refs:
|
||||
|
||||
|
|
@ -288,6 +288,7 @@ The renderer selects the useful current state instead of dumping raw JSON:
|
|||
- operating discipline
|
||||
- invariants
|
||||
- graph frontier/checkpoint
|
||||
- investigation checkpoint
|
||||
- focused graph nodes/edges/links
|
||||
- scratch summary
|
||||
- observations
|
||||
|
|
@ -963,9 +964,10 @@ The app-server handler:
|
|||
4. summarizes invariant status counts with deterministic status ordering.
|
||||
5. summarizes architecture/dataflow graph counts, active frontier ids, dirty paths, open question/gap counts, and checkpoint identity.
|
||||
6. summarizes retrieval status, semantic availability, index revision, shard count, dirty-path count, and indexed file/chunk counts.
|
||||
7. includes bounded latest observation/evidence summaries in newest-first order, matching the durable state's recent-record ordering.
|
||||
8. reflects churn pressure/warning fields.
|
||||
9. reports available actions only when the thread is loaded, including the read-only `context` and `jobs` doors and adding `propose`/`promote` only when state exists.
|
||||
7. includes a compact investigation-checkpoint summary when one exists, including disposition plus open-question/code-ref/evidence counts.
|
||||
8. includes bounded latest observation/evidence summaries in newest-first order, matching the durable state's recent-record ordering.
|
||||
9. reflects churn pressure/warning fields.
|
||||
10. reports available actions only when the thread is loaded, including the read-only `context` and `jobs` doors and adding `propose`/`promote` only when state exists.
|
||||
|
||||
### Output
|
||||
|
||||
|
|
@ -1098,6 +1100,7 @@ Response shape:
|
|||
- `links`
|
||||
- optional `frontier`
|
||||
- optional `checkpoint`
|
||||
- optional `investigationCheckpoint`
|
||||
- `observations`
|
||||
- `evidence`
|
||||
- `missing`
|
||||
|
|
@ -1132,7 +1135,7 @@ The app-server handler:
|
|||
5. selects architecture and dataflow nodes by id.
|
||||
6. selects graph edges by explicit edge id and incident edges touching selected nodes.
|
||||
7. includes graph links touching selected architecture or dataflow nodes.
|
||||
8. includes active frontier and graph checkpoint when available.
|
||||
8. includes active frontier, graph checkpoint, and the full durable investigation checkpoint when available.
|
||||
9. selects observations by id.
|
||||
10. selects directly requested evidence plus evidence ids linked from selected observations.
|
||||
11. reports any requested or active-frontier graph ids, observation ids, or evidence ids not found in the authoritative typed state.
|
||||
|
|
@ -1223,7 +1226,7 @@ These are deliberately not shipped yet:
|
|||
|
||||
- no broad live Epiphany event stream beyond `thread/epiphany/stateUpdated` for successful update/promote writes.
|
||||
- no automatic evidence promotion from tool output; the current distill/propose/promote path still requires explicit verifier-backed calls.
|
||||
- no automatic Compact-Rehydrate-Reorient-Continue coordinator; current code preserves and rehydrates valid Epiphany snapshots across resume/rollback/compaction and exposes context pressure, but reorientation, in-flight planning checkpoints, and next-action selection are still enforced by repo-local state, handoff notes, and operating discipline rather than a runtime scheduler.
|
||||
- no automatic Compact-Rehydrate-Reorient-Continue coordinator; current code preserves and rehydrates valid Epiphany snapshots across resume/rollback/compaction, exposes context pressure, and now carries a durable investigation checkpoint packet, but runtime reorientation and next-action selection are still enforced by repo-local state, handoff notes, and operating discipline rather than a scheduler.
|
||||
- no watcher-driven graph or semantic invalidation.
|
||||
- no code-intelligence graph.
|
||||
- no specialist-agent scheduler.
|
||||
|
|
@ -1250,7 +1253,7 @@ stored Epiphany state
|
|||
-> successful writes emit thread/epiphany/stateUpdated
|
||||
```
|
||||
|
||||
The current remaining missing organ is not the red pen, not the first chewing motion, not the first state-change bell, not the first client-shaped scene, not the first job/progress reflection, not the first targeted graph/evidence context read, and not the first context-pressure gauge. Those exist. Proposal now has the first bits of map memory, focus, and selection hygiene: it can auto-select a bounded evidence-backed observation cluster when ids are omitted, requires accepting recent evidence behind selected observations, prioritizes stronger selected observations for proposal wording, reuses existing architecture nodes by concrete code-ref/path/id checks before creating new path nodes, can rescue unanchored graph nodes through strict unique semantic overlap, follows graph links and incident edges into the frontier, and reports match-kind-aware churn pressure from the actual proposal shape. Distillation now also has the first source-output-aware teeth for tool/model summaries. Promotion now notices risky deltas instead of just checking the shape of the envelope, treats graph expansion as risky even if the patch tries to mumble "low pressure" while dragging a new wall into the blueprint, and token-matches strong verifier kinds so `contest` cannot sneak past as `test`. Successful update/promote writes now emit `thread/epiphany/stateUpdated` with `source: "update"` or `source: "promote"`, event-level revision, and typed changed fields so clients no longer need to poll blindly, infer the cause from timing, or diff the whole state to know which top-level sections moved. The scene, jobs, context, and pressure surfaces give clients compact or bounded read-only views without making the GUI/client authoritative, and live app-server smoke now guards those boundaries. The remaining missing organs are in-flight investigation checkpointing, watcher/freshness inputs, live job progress notifications once real long-running job owners exist, and eventually role-scoped specialist ownership without silently auto-promoting anything.
|
||||
The current remaining missing organ is not the red pen, not the first chewing motion, not the first state-change bell, not the first client-shaped scene, not the first job/progress reflection, not the first targeted graph/evidence context read, not the first context-pressure gauge, and no longer the first durable investigation checkpoint packet. Those exist. Proposal now has the first bits of map memory, focus, and selection hygiene: it can auto-select a bounded evidence-backed observation cluster when ids are omitted, requires accepting recent evidence behind selected observations, prioritizes stronger selected observations for proposal wording, reuses existing architecture nodes by concrete code-ref/path/id checks before creating new path nodes, can rescue unanchored graph nodes through strict unique semantic overlap, follows graph links and incident edges into the frontier, and reports match-kind-aware churn pressure from the actual proposal shape. Distillation now also has the first source-output-aware teeth for tool/model summaries. Promotion now notices risky deltas instead of just checking the shape of the envelope, treats graph expansion as risky even if the patch tries to mumble "low pressure" while dragging a new wall into the blueprint, token-matches strong verifier kinds so `contest` cannot sneak past as `test`, and validates investigation checkpoints against known evidence before the packet can be inked. Successful update/promote writes now emit `thread/epiphany/stateUpdated` with `source: "update"` or `source: "promote"`, event-level revision, and typed changed fields so clients no longer need to poll blindly, infer the cause from timing, or diff the whole state to know which top-level sections moved. The scene, jobs, context, and pressure surfaces give clients compact or bounded read-only views without making the GUI/client authoritative, and scene/context now surface the checkpoint packet back out without becoming writers. The remaining missing organs are watcher/freshness inputs, live job progress notifications once real long-running job owners exist, and eventually role-scoped specialist ownership without silently auto-promoting anything.
|
||||
|
||||
```text
|
||||
model/tool observations
|
||||
|
|
@ -1262,9 +1265,9 @@ model/tool observations
|
|||
-> live stateUpdated notification
|
||||
```
|
||||
|
||||
The typed write path exists, the first deterministic observation/evidence distillation path exists, source-output-aware distillation now summarizes noisy tool/model output into typed evidence with final result/error/finished lines ahead of generic warnings, the first deterministic map/churn proposal path exists, proposal-quality hardening now auto-selects bounded observation clusters when ids are omitted, requires accepting recent evidence, reuses existing graph nodes by observed code refs, prioritizes stronger selected observations, rescues unanchored graph nodes by strict semantic overlap, focuses linked frontier context, reports match-kind-aware map-delta pressure, and the first verifier-backed promotion gate exists. The first map/churn safety layer also exists: promotion can reject state replacement patches that are not tied to explicit observations/evidence, that contain structurally broken subgoal, invariant, graph, frontier, checkpoint, or churn fields, that try to promote risky churn without a warning and stronger verifier evidence, that claim a graph expansion is low-pressure to dodge the warning gate, or that rely on substring verifier-kind accidents. The first live state notification layer also exists: accepted writes publish updated typed state plus event-level revision and typed changed fields; rejected promotions stay quiet.
|
||||
The typed write path exists, the first deterministic observation/evidence distillation path exists, source-output-aware distillation now summarizes noisy tool/model output into typed evidence with final result/error/finished lines ahead of generic warnings, the first deterministic map/churn proposal path exists, proposal-quality hardening now auto-selects bounded observation clusters when ids are omitted, requires accepting recent evidence, reuses existing graph nodes by observed code refs, prioritizes stronger selected observations, rescues unanchored graph nodes by strict semantic overlap, focuses linked frontier context, reports match-kind-aware map-delta pressure, and the first verifier-backed promotion gate exists. The first map/churn safety layer also exists: promotion can reject state replacement patches that are not tied to explicit observations/evidence, that contain structurally broken subgoal, invariant, graph, frontier, checkpoint, investigation-checkpoint, or churn fields, that try to promote risky churn without a warning and stronger verifier evidence, that claim a graph expansion is low-pressure to dodge the warning gate, or that rely on substring verifier-kind accidents. The first live state notification layer also exists: accepted writes publish updated typed state plus event-level revision and typed changed fields; rejected promotions stay quiet.
|
||||
|
||||
In natural language: current Epiphany can preserve a map, show a map, project a compact scene from the map, project derived job/progress slots from the map and retrieval summary, project a targeted graph/evidence context shard from the map, project context pressure from token telemetry, retrieve evidence for the map, draft one explicit observation/evidence patch, derive a bounded graph/frontier/churn candidate from verified evidence-backed observations with code refs, choose a small coherent observation set when the caller does not provide one, favor the strongest selected observation when drafting that candidate, reject or accept a verified candidate, apply explicit map/evidence/churn edits, notify clients after successful writes, and sanity-check those edits before they hit the ledger. It still does not checkpoint in-flight investigation, ingest fresh tool/model output into richer map deltas automatically, invalidate stale graph regions, execute long-running Epiphany jobs as tracked background work, emit job progress notifications, or coordinate specialist ownership. Teeth are installed; chewing is supervised; the first bell rings; the first wall board exists; the first shift board exists; the first binder lens exists; the first pressure gauge exists; the next slices are about safe waking and senses, not jawbones.
|
||||
In natural language: current Epiphany can preserve a map, show a map, project a compact scene from the map, project derived job/progress slots from the map and retrieval summary, project a targeted graph/evidence context shard from the map, project context pressure from token telemetry, bank an in-flight investigation packet in durable typed state, reflect that packet back into prompt/scene/context, retrieve evidence for the map, draft one explicit observation/evidence patch, derive a bounded graph/frontier/churn candidate from verified evidence-backed observations with code refs, choose a small coherent observation set when the caller does not provide one, favor the strongest selected observation when drafting that candidate, reject or accept a verified candidate, apply explicit map/evidence/churn edits, notify clients after successful writes, and sanity-check those edits before they hit the ledger. It still does not invalidate stale graph regions, execute long-running Epiphany jobs as tracked background work, emit job progress notifications, or coordinate specialist ownership. Teeth are installed; chewing is supervised; the first bell rings; the first wall board exists; the first shift board exists; the first binder lens exists; the first pressure gauge exists; the first ember packet exists; the next slices are about safe waking and senses, not jawbones.
|
||||
|
||||
## Verification Hooks
|
||||
|
||||
|
|
|
|||
|
|
@ -63,6 +63,7 @@ The landed machine now has:
|
|||
- live Phase 6 context app-server smoke coverage in `tools/epiphany_phase6_context_smoke.py`
|
||||
- read-only Phase 6 context-pressure reflection through `thread/epiphany/pressure`
|
||||
- live Phase 6 pressure app-server smoke coverage in `tools/epiphany_phase6_pressure_smoke.py`
|
||||
- durable Phase 6 investigation checkpointing in authoritative typed state, prompt rendering, and scene/context reflection
|
||||
|
||||
The current phase is Phase 6: reflection boundary and observable harness state.
|
||||
|
||||
|
|
@ -117,8 +118,8 @@ The next unknowns are:
|
|||
|
||||
- how to expose live long-running job progress without making the GUI authoritative
|
||||
- when watcher-driven invalidation becomes necessary instead of merely tempting
|
||||
- how to checkpoint in-flight source gathering and slice planning before automatic CRRC so post-compaction agents do not continue from false continuity
|
||||
- how much automatic CRRC coordination belongs in runtime before it becomes ceremony machinery
|
||||
- how the landed investigation checkpoint should be consumed by future automatic CRRC without becoming ceremony machinery
|
||||
- how much automatic CRRC coordination belongs in runtime once pressure and checkpoint signals both exist
|
||||
- what Phase 6 should prove before specialist scheduling begins
|
||||
|
||||
## Phase 6 Direction
|
||||
|
|
@ -129,7 +130,7 @@ Useful candidates:
|
|||
|
||||
1. Add watcher/freshness inputs when stale graph or retrieval state needs earlier warning.
|
||||
2. Add live job progress notifications only after there is a real long-running owner to report.
|
||||
3. Add targeted scene/jobs/context/pressure fields only when a client or smoke exposes a real gap.
|
||||
3. Add targeted scene/jobs/context/pressure/checkpoint fields only when a client or smoke exposes a real gap.
|
||||
|
||||
Do not spend Phase 6 polishing Phase 5 out of anxiety. The Phase 5 smoke harness
|
||||
is a regression guardrail, not a ritual drum circle for summoning more tiny
|
||||
|
|
@ -145,7 +146,6 @@ These remain later work:
|
|||
- role-scoped specialist-agent registry and scheduling
|
||||
- mutation gates that warn or block broad writes when map freshness is stale
|
||||
- automatic CRRC runtime coordination on top of the landed typed context-pressure telemetry
|
||||
- in-flight investigation checkpointing for compaction-safe planning
|
||||
- GUI workflows for graph, evidence, job, invariant, and frontier steering
|
||||
|
||||
Do not start these from vibes. Each one needs a source-grounded slice plan and a
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ Do not trust this file for the exact live HEAD. Always check git.
|
|||
- Do not copy exact branch or HEAD from this note. Run `git status --short --branch` and `git log --oneline -5`.
|
||||
- Phase 1 through Phase 5 are complete enough.
|
||||
- Phase 6 has read-only `thread/epiphany/scene`, `thread/epiphany/jobs`, `thread/epiphany/context`, and `thread/epiphany/pressure`; all four have live app-server smoke coverage.
|
||||
- Durable in-flight investigation checkpointing is now landed in authoritative typed state, writable through `thread/epiphany/update` or accepted `thread/epiphany/promote`, rendered into the prompt, and reflected through scene/context.
|
||||
- The repo is an Epiphany fork of Codex, not a Codex preset.
|
||||
- `vendor/codex` is tracked directly, not a submodule.
|
||||
- `epiphany-core` owns the heavy Epiphany organs where practical.
|
||||
|
|
@ -64,6 +65,7 @@ The current spine:
|
|||
- read-only job/progress reflection through `thread/epiphany/jobs`
|
||||
- read-only targeted state-shard reflection through `thread/epiphany/context`
|
||||
- read-only context-pressure reflection through `thread/epiphany/pressure`
|
||||
- durable investigation checkpoint packet through typed state, prompt, scene, and context
|
||||
- live scene app-server smoke through `tools/epiphany_phase6_scene_smoke.py`
|
||||
- live jobs app-server smoke through `tools/epiphany_phase6_jobs_smoke.py`
|
||||
- live context app-server smoke through `tools/epiphany_phase6_context_smoke.py`
|
||||
|
|
@ -161,14 +163,18 @@ The Phase 6 context-pressure slice is landed. It exposes read-only
|
|||
auto-compact/context limits. It does not build automatic CRRC, a scheduler, a
|
||||
hidden compaction trigger, or a vibes-based gauge.
|
||||
|
||||
When the user asks to continue, choose the next Phase 6 slice from the current
|
||||
map: likely in-flight investigation checkpointing for compaction-safe planning,
|
||||
watcher/freshness inputs, or live progress notifications once real job owners
|
||||
exist.
|
||||
The Phase 6 investigation-checkpoint slice is also landed. It banks an
|
||||
authoritative planning/source-gathering packet in typed state, validates linked
|
||||
evidence, and reflects the packet into prompt/scene/context so post-compaction
|
||||
wakeups can tell whether they have a real ember or only ash.
|
||||
|
||||
Also keep the newly surfaced guardrail in mind: a pressure signal alone is not
|
||||
enough. Automatic CRRC needs an in-flight investigation checkpoint so compaction
|
||||
during source-gathering/planning does not wake into false continuity.
|
||||
When the user asks to continue, choose the next Phase 6 slice from the current
|
||||
map: watcher/freshness inputs are the most likely next organ, with live
|
||||
progress notifications waiting until real job owners exist.
|
||||
|
||||
Also keep the guardrail in mind: pressure plus a checkpoint packet is still not
|
||||
automatic CRRC. Runtime coordination, reorientation policy, and next-action
|
||||
selection are still future work.
|
||||
|
||||
Live `thread/epiphany/scene`, `thread/epiphany/jobs`,
|
||||
`thread/epiphany/context`, and `thread/epiphany/pressure` smokes are now
|
||||
|
|
@ -180,7 +186,7 @@ guardrails, not the next organs.
|
|||
- automatic observation promotion
|
||||
- specialist-agent scheduling
|
||||
- GUI-as-source-of-truth
|
||||
- automatic runtime CRRC coordinator using the landed context-pressure telemetry
|
||||
- automatic runtime CRRC coordinator using the landed context-pressure telemetry and investigation checkpoints
|
||||
- live long-running job execution or `thread/epiphany/jobsUpdated`
|
||||
- broad event stream beyond the landed state update notification
|
||||
|
||||
|
|
|
|||
Reference in a new issue