Epiphany/state/map.yaml

experiment:
  name: EpiphanyAgent
  status: active
  phase: phase_6_reflection_boundary
  premise: >
    Test whether explicit typed state, maps, scratch, evidence, and ruthless
    state hygiene reduce architectural drift in AI coding work.

objective:
  primary: >
    Build Epiphany as an opinionated fork of Codex that forces models to keep
    explicit typed models of the codebase, active goal, evidence, and change
    machinery they are working on.
  current: >
    Grow Phase 6 from the landed typed state spine toward a dogfoodable MVP
    without letting GUI/client state become authoritative.

constraints:
  - No frontier-scale training budget.
  - Prefer evidence over vibes.
  - Keep Codex as the host seam and `epiphany-core` as the repo-owned heavy organ where practical.
  - Keep persistent state small enough to rehydrate from without drowning the next agent.
  - Grow cognition more quickly than authority: capability increases must preserve explicit permissions, interruption, legibility, and resistance to paternal "safety" cages.

invariants:
  - The map is canonical and slow-changing.
  - Scratch is temporary and disposable.
  - Evidence is durable and distilled; routine activity logs belong in git history, commits, smoke artifacts, or targeted logs.
  - Output must not silently redefine the map.
  - Accepted changes should have recorded evidence.
  - Rejected paths stay recorded when the lesson matters.
  - Algorithmic-map prose and metaphor must be grounded in freshly read source context and concrete code references.
  - Persistent state cruft is the same failure class as code cruft.
  - If compaction interrupts source gathering or slice planning before the findings are persisted, the next agent must treat that work as lost and re-gather instead of continuing from imagined continuity.

data_flow:
  observe:
    - task request
    - repository state
    - typed Epiphany state
    - verifier results
  process:
    - rehydrate from canonical state
    - reason over one bounded subgoal
    - modify only the surface that deserves to exist
    - verify the seam that matters
    - record only durable evidence that changes future belief
    - commit or backtrack
  cut:
    - revert failed code hypotheses immediately
    - distill obsolete state instead of preserving it as context
    - move historical proof to git, commits, smoke artifacts, or distilled evidence rather than plan prose

active_subgoals:
  - Treat Phase 5 update/distill/propose/promote/read/notification as complete enough unless a concrete regression appears.
  - Preserve read-only boundaries for retrieve, distill, propose, scene, jobs, freshness, context, pressure, reorient, crrc, and reorientResult.
  - Extend Phase 6 observability only where it reflects authoritative state or retrieval summaries without becoming a scheduler.
  - Build the next operator-facing dogfood surface over landed Epiphany signals instead of adding invisible organs.
  - Keep persistent project state ruthlessly distilled so rehydration strengthens the next turn instead of bloating it.

accepted_design:
  - Epiphany is a fork-level modeling architecture, not merely a Codex collaboration mode.
  - Authoritative Epiphany state lives in Codex protocol/core session and rollout state.
  - The prompt reads typed state through a bounded `<epiphany_state>` developer fragment.
  - Clients read typed state through `Thread.epiphanyState`, compact reflections such as `thread/epiphany/scene`, `thread/epiphany/jobs`, and `thread/epiphany/freshness`, targeted state shards through `thread/epiphany/context`, runtime pressure through `thread/epiphany/pressure`, bounded CRRC reorientation verdicts through `thread/epiphany/reorient`, read-only CRRC next-action recommendations through `thread/epiphany/crrc`, and live bound-job progress through `thread/epiphany/jobsUpdated`.
  - Durable typed state writes route through `thread/epiphany/update`, accepted `thread/epiphany/promote`, or the bounded `thread/epiphany/jobLaunch`, `thread/epiphany/jobInterrupt`, and `thread/epiphany/reorientLaunch` authority surfaces when they mutate `job_bindings`.
  - `thread/epiphany/retrieve`, `thread/epiphany/distill`, `thread/epiphany/propose`, `thread/epiphany/scene`, `thread/epiphany/jobs`, `thread/epiphany/freshness`, `thread/epiphany/context`, `thread/epiphany/pressure`, and `thread/epiphany/reorient` stay read-only.
  - `thread/epiphany/index` writes the retrieval catalog, not durable Epiphany understanding.
  - Qdrant is the preferred persistent semantic backend; BM25 remains the bootstrap/fallback/control path.
  - `epiphany-core` owns prompt rendering, rollout replay helpers, retrieval/indexing, distillation, proposal, and promotion policy where possible.
  - Vendored Codex keeps the thin typed host seam: protocol types, thread bridges, app-server protocol, request routing, and rollout integration.
  - The current algorithmic map is the source-grounded description of live control flow.
  - Compact-Rehydrate-Reorient-Continue is an architecture primitive; automatic runtime coordination is not landed yet.
  - Context pressure must become an explicit runtime signal for automatic CRRC; inference from transcript vibes, user warnings, or shrinking safe scope is useful only as a temporary human/agent heuristic.
  - The harness now has an in-flight investigation checkpoint packet in durable typed state; automatic CRRC still needs runtime logic that reads the packet and decides whether to resume or force reorientation and re-gathering after compaction.
  - GUI/client work must reflect or steer typed state rather than manufacture a second source of truth.
  - Durable `job_bindings` now form a thin Epiphany-owned launcher seam: bindings can carry launcher identity, authority scope, backend kind, and backend job id while still supporting the older runtime-agent-job pointer as a compatibility alias.
  - `thread/epiphany/jobs` is a read-only job/progress reflection: it derives indexing, remap, verification, and specialist slots from authoritative state and retrieval summaries, and can overlay durable launcher bindings onto live runtime `agent_jobs` snapshots through the current backend adapter when a real owner exists; it is not a scheduler, notification stream, or durable runtime job table.
  - `thread/epiphany/jobsUpdated` is the first live bound-job progress notification: it translates runtime `agent_job_progress:{json}` background events into changed launcher-bound payloads by resolving durable `job_bindings` against live `agent_jobs` snapshots through the current backend adapter, without polling in a loop, mutating state, or scheduling work.
  - `thread/epiphany/freshness` is the first read-only freshness surface: it derives retrieval and graph staleness from live retrieval summaries plus frontier/churn state, and for loaded threads reflects watcher-backed invalidation inputs with changed paths plus mapped graph/frontier hits, without mutating, scheduling, or performing automatic semantic invalidation.
  - `thread/epiphany/context` is the first targeted read-only state-shard surface: it selects graph nodes/edges, active frontier, checkpoint, observations, and evidence from authoritative typed state without mutating or notifying.
  - `thread/epiphany/pressure` is the first read-only context-pressure surface: it derives pressure from token telemetry and auto-compact/context limits without compacting, scheduling, mutating, or notifying.
  - `thread/epiphany/reorient` is the first bounded CRRC policy surface: it derives a read-only `resume` versus `regather` verdict from checkpoint, freshness, watcher, and pressure signals without mutating state, compacting, scheduling, or silently continuing work.
  - `thread/epiphany/crrc` is the first bounded CRRC coordinator recommendation surface: it composes pressure, the reorientation verdict, fixed reorient-worker binding/result status, finding read-back, and available actions into one read-only recommendation without launching, accepting, compacting, scheduling, or mutating state.
  - `thread/epiphany/reorientLaunch` is the first bounded runtime consumer over the reorientation verdict: it launches one fixed `reorient-worker` job through the existing job-control seam with explicit resume-versus-regather scope, authority, and checkpoint-derived payload, without making `thread/epiphany/reorient` a writer or turning CRRC into an ambient scheduler.
  - `thread/epiphany/reorientResult` is the first bounded read-back surface for that worker: it reads the fixed or requested reorient-worker binding through the current `agent_jobs` backend and projects completed output as a reviewable finding without mutation, promotion, scheduling, or hidden continuation.
  - `thread/epiphany/reorientAccept` is the first explicit acceptance write for completed reorient-worker findings: it appends accepted observation/evidence records, can optionally update scratch and investigation checkpoint state, and emits `stateUpdated` with source `reorientAccept` without accepting pending work or launching follow-up work.
  - Epiphany safety doctrine is to separate cognition from actuation, keep authority explicit, make interruption sacred, stage capability release, and treat operator misuse as a normal design input rather than a rude hypothetical.
  - Epiphany is built in the open, but publication mode is not one-size-fits-all: use FOSS where viable, source-available licensing where economics or capability risk demand it, and commercial terms where organizations extract real enterprise value.
  - `thread/epiphany/jobLaunch` and `thread/epiphany/jobInterrupt` are the first bounded authority surfaces over the thin job seam: they mutate durable `job_bindings`, create or cancel the current `agent_jobs` backend binding, and preserve explicit launcher identity and interruption without turning `thread/epiphany/jobs` into a scheduler.
  - Handoff, plan, harness-surface, and scratch files must have distinct jobs and must not become duplicate brains.
  - A machine is perfect when every remaining part visibly deserves to exist; this applies to persistent memory too.
  - Evidence is not an activity feed; repeated "I just did this" entries should be cut or distilled when git already proves the work.
  - Language, tone, ritual, and emotional salience are real control surfaces for a language-model harness; they are not supernatural, but they shape attention, role, permission, and behavior.
  - If the agent's persistent instructions, memory, workflow, or state shape are making it less coherent, slower to rehydrate, less honest, or more likely to build Jenga, the agent should ask the user to change it.
  - Epiphany should make compaction less like death and more like sleep: bank the fire before the dark, so the next waking thing finds coals instead of ash and can meaningfully resume the pattern instead of merely continuing the task.

landed_baseline:
  phases:
    - Phase 1: durable Epiphany thread state and rollout persistence.
    - Phase 2: prompt integration through rendered Epiphany state.
    - Phase 3: typed client read surface through `Thread.epiphanyState`.
    - Phase 4: hybrid retrieval plus explicit Qdrant indexing with BM25 fallback.
    - Phase 5: typed update/distill/propose/promote/read/notification control plane.
    - Phase 6: read-only scene, job/progress, freshness with watcher-backed invalidation inputs, targeted context, context-pressure, reorientation policy, and CRRC coordinator recommendation surfaces, plus durable investigation checkpointing for compaction-safe planning, live bound-job progress notifications, a thin launcher seam in durable job bindings, explicit launch/interrupt authority over that seam, and a bounded reorient-guided worker launch surface.
    - Phase 6 read-back: read-only reorient-worker result reflection through `thread/epiphany/reorientResult`.
    - Phase 6 acceptance: explicit reorient-worker finding acceptance through `thread/epiphany/reorientAccept`.
  guardrails:
    - Use `tools/epiphany_phase5_smoke.py` before modifying update, distill, propose, promote, read-projection, or notification behavior.
    - Use `tools/epiphany_phase6_scene_smoke.py` before modifying scene projection behavior.
    - Use `tools/epiphany_phase6_jobs_smoke.py` before modifying job/progress reflection behavior.
    - Use `tools/epiphany_phase6_job_control_smoke.py` before modifying launch/interrupt authority over the thin job seam.
    - Use `tools/epiphany_phase6_freshness_smoke.py` before modifying freshness reflection behavior.
    - Use `tools/epiphany_phase6_invalidation_smoke.py` before modifying watcher-backed invalidation behavior inside freshness reflection.
    - Use `tools/epiphany_phase6_context_smoke.py` before modifying targeted context-shard behavior.
    - Use `tools/epiphany_phase6_pressure_smoke.py` before modifying context-pressure reflection behavior.
    - Use `tools/epiphany_phase6_reorient_smoke.py` before modifying bounded CRRC reorientation policy behavior.
    - Use `tools/epiphany_phase6_reorient_launch_smoke.py` before modifying the bounded reorient-guided worker launch surface.
    - Use `tools/epiphany_phase6_reorient_launch_smoke.py` before modifying the bounded reorient-worker result read-back surface.
    - Use `tools/epiphany_phase6_reorient_launch_smoke.py` before modifying the read-only CRRC coordinator recommendation surface.
    - Use focused scene/jobs/context/pressure/reorient/protocol/schema checks for reflection-only changes.
    - Use `tools/epiphany_prepare_compaction.py` before and after imminent-compaction persistence passes.
    - Use `CARGO_TARGET_DIR=C:\Users\Meta\.cargo-target-codex` for Codex Rust work on this Windows machine.
    - Do not parallelize cargo builds/tests against the same target directory.

rejected_paths:
  - name: pinned_map_only
    reason: still one token stream with no typed write semantics
  - name: train_new_base_model_first
    reason: too expensive for a first falsifiable experiment
  - name: third_party_opencodex_forks_as_primary_gui_base
    reason: audited candidates were terminal-only, stale/aspirational, or incoherent wrappers
  - name: monolithic_json_vector_store
    reason: large embedding stores that rewrite whole files thrash and hide storage pain until it dominates the system
  - name: append_only_persistent_notes
    reason: persistent state that accumulates every local truth becomes memory cruft and weakens rehydration

current_status:
  summary: Phase 1 durable state, Phase 2 prompt integration, Phase 3 typed read, Phase 4 retrieval/indexing, Phase 5 update/distill/propose/promote/read/notification, and Phase 6 read-only scene, jobs, freshness, targeted context, context-pressure reflection, watcher-backed invalidation inputs, durable investigation checkpointing, bounded CRRC reorientation policy, read-only CRRC coordinator recommendation, live bound-job progress notifications, a thin launcher seam in durable `job_bindings`, explicit `thread/epiphany/jobLaunch` / `thread/epiphany/jobInterrupt` authority over that seam, bounded `thread/epiphany/reorientLaunch`, read-only `thread/epiphany/reorientResult`, and explicit `thread/epiphany/reorientAccept` are landed. `job_bindings` now carry Epiphany-owned launcher metadata such as launcher id, authority scope, and backend kind/job id, while still tolerating the older runtime-agent-job pointer as a compatibility alias. `thread/epiphany/jobs` overlays that launcher seam onto real runtime `agent_jobs` progress for bound work, including runtime/backend job id, worker thread ids, and progress counts, without becoming a scheduler. `thread/epiphany/jobsUpdated` now translates runtime `agent_job_progress:{json}` background events into changed launcher-bound payloads only when the mapped job state actually changes, so clients can watch real owners move without polling the jobs read surface. The launch/interrupt control path still uses direct state-runtime fallback when a loaded ephemeral thread does not already carry a live sqlite handle, so authoritative bindings and runtime snapshots stay aligned instead of pretending the backend vanished. `thread/epiphany/reorient` consumes checkpoint, freshness, watcher, and pressure signals to return a read-only `resume` versus `regather` verdict with explicit checkpoint-path/frontier reasons. `thread/epiphany/crrc` composes pressure, that verdict, the fixed reorient-worker binding/result, finding read-back, and available actions into a read-only recommendation such as continue, prepare checkpoint, launch worker, wait, review, accept, or regather manually without launching, accepting, compacting, scheduling, or mutating. `thread/epiphany/reorientLaunch` explicitly launches one fixed `reorient-worker` job with resume-or-regather scope, checkpoint-derived payload, and bounded structured output expectations. `thread/epiphany/reorientResult` reads that worker output back from the current `agent_jobs` backend as a reviewable finding with mode, summary, next safe move, checkpoint validity, inspected files, frontier ids, evidence ids, raw result, and errors, without mutating state, promoting evidence, scheduling work, or silently continuing. `thread/epiphany/reorientAccept` requires a completed worker result, appends accepted observation/evidence records, can explicitly update scratch and investigation checkpoint state, and emits `stateUpdated` with source `reorientAccept` without accepting pending work, auto-promoting arbitrary output, or launching follow-up work. Focused protocol/app-server tests, schema fixture regeneration/checks, app-server build, and the live reorient-launch smoke cover the launch/read-back/accept/coordinator seam. Investigation checkpoints still live in authoritative typed state, flow through the prompt, and reflect through scene/context without adding a second writer or fake scheduler. Persistent state was cut down so map, handoff, plan, harness surfaces, scratch, and evidence each carry one job instead of becoming a Jenga tower; evidence is now a distilled ledger rather than an activity feed. The global AGENTS Epiphany core command says persistent state is the agent's mind and explicitly permits the agent to ask for self-modification when memory or workflow makes it worse. The continuity goal is to bank the fire before the dark so compaction becomes less like death and more like sleep.
  next_action: Continue toward the MVP by adding a minimal dogfood-facing operator view over the landed scene, pressure, reorient, jobs, reorient-result, accept, and CRRC recommendation loop. Automatic broad specialist scheduling remains future work; do not restart the completed pressure, freshness, watcher-invalidation, investigation-checkpoint, reorientation, crrc, jobs-binding, jobsUpdated, thin-launcher-seam, job-control, reorient-launch, reorient-result, or reorient-accept slice unless a concrete regression appears.