Epiphany/state/map.yaml

experiment:
  name: EpiphanyAgent
  status: scaffolded
  phase: manual_external_typed_state
  premise: >
    Test whether separating map, scratch, and output reduces architectural drift
    for coding tasks when using a strong general model.

objective:
  primary: >
    Build a model-agnostic protocol that Codex can use in this workspace and that
    can later be evaluated on open models without requiring frontier-scale
    training.
  immediate:
    - Finish the initial workspace scaffold.
    - Keep controller actions and write rules explicit.
    - Ground the Codex epiphany-mode idea against the real open-source codebase.
    - Use the first minimal runner or manual operating loop on a small real task.
    - Try the protocol on a small real task.

constraints:
  - No frontier-scale training budget.
  - Start with external state and prompting.
  - Keep the protocol light enough to use during normal work.
  - Prefer evidence over vibes.

invariants:
  - The map is canonical and slow-changing.
  - Scratch is temporary and allowed to be wrong.
  - Output must not silently redefine the map.
  - Accepted changes should have recorded evidence.
  - Rejected paths stay recorded.

data_flow:
  observe:
    - task request
    - repository state
    - verifier results
  process:
    - update map only with stable understanding
    - think about one subgoal at a time
    - propose one bounded change
    - verify against task goals and invariants
    - record evidence
    - commit or backtrack

active_subgoals:
  - Clean up and ship the verified Phase 4 repo-local hybrid retrieval slice without widening scope.
  - Keep the canonical map, handoff note, evidence log, and checked-in stable schema state aligned with the verified landing zone.
  - Preserve BM25 as the no-hassle fallback while treating Qdrant as the preferred later persistent semantic backend.

accepted_design:
  - external typed state files
  - bounded controller actions
  - verifier-guided acceptance
  - manual first, automation later
  - patch Codex via collaboration-mode presets before touching protocol enums
  - make Phase 1 preset selection, labeling, and instruction lookup preset-name-aware where needed because Epiphany initially rides on ModeKind::Default
  - for a GUI-centered experiment, build a dedicated wrapper around customized vendored Codex instead of adopting third-party OpenCodex forks
  - treat Epiphany as core-owned structured thread state in Codex, with the GUI as a reflection and steering layer rather than the source of truth
  - represent the Epiphany machine map as two linked graphs (architecture and dataflow) with rich natural-language node fields such as purpose, mechanism, and metaphor, plus line-anchored code references
  - make the graph model compaction-aware: preserve a durable spine, checkpoint and reselect a frontier, and allow scratch to be rewritten or discarded
  - for mapping work, distinguish a completed pass from actual understanding: require representative concrete transformation questions, record unresolved mechanism gaps durably, and carry those gates across compaction
  - treat Epiphany as a supervising harness over specialist agents sharing a typed state plane, with private role-scoped scratch by default and explicit ownership for map, evidence, and code-change domains
  - add a repo-local hybrid retrieval subsystem to Epiphany: combine exact path/symbol/lexical search with semantic chunk retrieval and graph-linked lookup, shard it by repo/subsystem, and track freshness from diffs
  - treat Codex's existing file watcher, subagent lifecycle, memory pipeline, and diff stream as reusable substrate, but add the missing semantic layer above them: typed repo understanding, freshness invalidation, structured tool-output distillation, and shared state across specialist agents
  - insert a structured observation layer between raw tool output and durable Epiphany state so evidence, invalidation, and graph updates are promoted from typed distillation rather than transcript osmosis
  - design Epiphany as GUI-first rather than transcript-first: the primary artifact is typed thread state, chat is one client of that state, user steering should go through typed updates, and long-running retrieval/remap/verification work should surface as explicit observable jobs
  - treat compaction as a role-specific state transition with safe points, explicit checkpoints, and resume packets; workers may compact aggressively between implementation passes, while mapper, verifier, and coordinator preserve different state at different boundaries
  - preserve handoff context in repo-local AGENTS and notes files
  - keep a durable repository-level algorithmic map of vendored Codex so GUI and Epiphany changes hook into the existing app-server/core boundary instead of reimplementing the turn loop
  - keep a dedicated Epiphany delta algorithmic map that documents the currently landed divergence from plain Codex separately from the full Codex machine map and the future-facing harness spec
  - keep the Codex algorithmic map honest about auxiliary side channels and limitations: file watching, unified diffs, compaction, subagent messaging, and memory summarization are real substrate, but they do not by themselves amount to semantic repo understanding
  - vendor Codex directly into the parent repo instead of keeping it as a gitlink/submodule so Epiphany harness changes can be reviewed and committed atomically with the surrounding project state
  - for Phase 2, treat the presence of SessionState.epiphany_state as the internal activation signal and inject a bounded Epiphany summary into the developer-context path during build_initial_context rather than inventing a public operator surface early
  - Phase 2 is landed: a dedicated EpiphanyStateInstructions fragment now renders bounded Epiphany thread state into the developer prompt path during build_initial_context, with omission/inclusion/resume coverage and a stable snapshot
  - Phase 3 read-surface slice is landed: app-server protocol Thread payloads now carry optional typed epiphany_state hydrated from live CodexThread state when available and otherwise reconstructed from rollout with rollback/compaction semantics; dedicated Epiphany update RPCs and live notifications remain deferred
  - Epiphany should be an opinionated software development agent: pre-compaction and pre-handoff persistence of map, evidence, and next action are part of the workflow, not optional operator folklore
  - compaction handling should be pressure-aware rather than reactive: when context pressure rises, Epiphany should narrow the active move and persist checkpoints before forced compaction actually hits
  - repo-local AGENTS instructions should carry the session bootstrap and re-entry protocol explicitly, including running epiphany_state.py status on fresh load and after compaction
  - implement the first Phase 4 retrieval slice as a query-time hybrid retriever: reuse existing codex_file_search substrate for exact/path lookup and use workspace-local BM25 chunk search as the first semantic backend so the retrieval seam is proven before adding persistence
  - for the first persistent semantic retrieval store after the verified BM25 baseline, prefer a real vector database such as Qdrant over JSON/blob/postgres-thrash designs; treat Qdrant as the preferred persistent semantic backend rather than a hard requirement, keep exact/path lookup as a first-class organ, and keep BM25 available as a bootstrap/fallback/control path when Qdrant is unavailable or not configured instead of turning retrieval into vector-only soup

rejected_paths:
  - name: pinned_map_only
    reason: still one token stream with no typed write semantics
  - name: train_new_base_model_first
    reason: too expensive for a first falsifiable experiment
  - name: exact_name_open_codex_cli_as_gui_surface
    reason: ymichael/open-codex is terminal-focused and has no Web UI or desktop surface
  - name: read_opencodex_web_readme_claude_models_as_current_gui_support
    reason: the Web UI has stale/aspirational Claude/Gemini model definitions, but the new-session selector only exposes Codex and the serve backend rejects non-Codex agents
  - name: imdigitalashish_opencodex_as_primary_gui_surface
    reason: the repo is a Tauri wrapper around an archived/vendored opencode CLI, has mismatched GUI/API model IDs versus the CLI model registry, runs the non-interactive tool path with broad auto-approval, and commits generated/binary/local-state artifacts
  - name: third_party_opencodex_forks_as_gui_base
    reason: audited candidates were either terminal-only, stale/aspirational, or incoherent wrappers; the comparison clones were removed from vendor to avoid building on untrusted drift

current_status:
  summary: Workspace initialized with protocol files, typed state, a tiny CLI, a git repo, fresh-workspace handoff notes, architecture rationale, a Codex algorithmic map note, an Epiphany core-harness surface spec, and a rolling implementation plan against vendored Codex. Phase 1 durable Epiphany state, Phase 2 prompt integration, and a minimal Phase 3 typed client read surface are all implemented and verified with a real Rust/MSVC toolchain; vendored Codex is tracked directly in the parent repo; app-server Thread payloads can now carry typed Epiphany state without scraping prompt text; pre-compaction persistence is now an explicit part of the intended Epiphany workflow; and the first bounded Phase 4 repo-local hybrid retrieval slice is now wired and verified in the working tree across protocol, core, app-server protocol, and app-server, including stable schema fixture regeneration and the Windows `CARGO_TARGET_DIR` verification path.
  next_action: Clean up and push the current verified query-time hybrid retriever as the Phase 4 slice 1 landing zone. Do not bolt durable EpiphanyState writes onto `thread/epiphany/retrieve` without a clean out-of-band rollout semantic; after the push and reboot, the next persistent semantic retrieval slice should target a Qdrant-backed store with explicit freshness/indexing semantics as the preferred semantic backend while BM25 remains the fallback when Qdrant is unavailable.