This repository has been archived on 2026-10-29. You can view files and clone it, but you cannot make any changes to its state, such as pushing and creating new issues, pull requests or comments.
Updated on 2026-04-24 after landing the Phase 4 retrieval/indexing/core-extraction baseline on `main`.
This note tracks Epiphany as a fork of Codex with an opinionated modeling architecture. The point is not to offer Codex another collaboration preset. The point is to make the harness force the model to carry explicit structure about the codebase, the active subgoal, the evidence trail, and the machine it is modifying.
- a verified Phase 4 slice 2 follow-up that adds explicit persistent semantic indexing through `thread/epiphany/index` while keeping `thread/epiphany/retrieve` read-only
- the heavy Epiphany-owned prompt/replay/retrieval implementation has now been extracted into the repo-owned `epiphany-core` crate, leaving vendored Codex with thin host adapters plus the typed integration seam
- a first typed state-update slice that adds explicit loaded-thread-only `thread/epiphany/update`, appends observations/evidence, replaces bounded map/scratch/churn fields, bumps the state revision, and persists an immediate rollout `EpiphanyState` snapshot
- a first typed distillation/proposal slice that adds read-only loaded-thread-only `thread/epiphany/distill`, producing deterministic observation/evidence patches for explicit promotion through `thread/epiphany/update`
- a first richer promotion-policy slice that treats map/churn/frontier/checkpoint replacements as higher-risk state edits: accepted promotions must be tied to explicit observations and patch evidence, and structural graph/churn mistakes are rejected before the durable update path
The next job is no longer to prove the retriever exists, sketch the persistent follow-up in prose, invent the first red-pen path, build the first observation proposal surface, or make promotion notice broken map/churn replacements. Those parts are landed. The next job is to keep the landing zone honest: do not casually turn `thread/epiphany/retrieve` or `thread/epiphany/distill` into durable Epiphany-state writers, and do not widen the explicit indexing/update paths into watcher-driven or GUI-shaped machinery before they earn it.
Build a **hybrid repo retrieval subsystem** with one typed query shape and one typed result shape. Exact lookup and semantic lookup should be different modes of the same machine, not rival religions.
- reuse existing `codex_file_search` substrate for exact/path-ish lookup
- use workspace-local BM25 chunk search as the initial semantic backend
- defer embeddings, vector stores, persistent indexing, and watcher-driven invalidation
Updated follow-up direction after landing the verified slice:
- keep the BM25 implementation as the proven baseline/fallback
- for the first persistent semantic backend, prefer Qdrant over JSON/blob/postgres-thrash designs, but do not make it a hard requirement for basic Epiphany use
- do not turn retrieval into vector-only soup; exact/path lookup remains first-class and BM25 remains useful as bootstrap/control/fallback when Qdrant is unavailable
- live app-server smoke verified the explicit Qdrant path against real local services using an ephemeral loaded thread rooted at `epiphany-core`; the smoke indexed 5 files into 198 chunks, created Qdrant collection `epiphany_workspace_fa24bab116f8d229`, and retrieved persistent semantic chunks from `src/prompt.rs`
- env/default backend configuration is good enough for the current slice; defer a first-class backend config surface until a real operator pain proves it deserves to exist
- targeted no-run compile for `codex-core`, `codex-app-server-protocol`, and `codex-app-server` passed
- targeted Phase 4 tests passed in core, app-server protocol, and app-server
- full `cargo test -p codex-app-server-protocol` passed after regenerating stable schema fixtures
- stable schema fixtures are the safe checked-in state for this slice; `write_schema_fixtures --experimental` rewrites the same tree and cannot be left behind as the repo state
If the first semantic backend needs to be intentionally narrow, that is fine. A small honest subsystem beats a magical blob with terrible write behavior.
Follow-up addition now landed in the working tree:
-`thread/epiphany/update`
Current bounded shape of that follow-up:
- it is the explicit write path for durable Epiphany state, not a retrieval side effect
- it requires a loaded thread
- it optionally checks `expectedRevision`
- it can append observations and evidence
- it can replace bounded typed fields: objective, active subgoal, subgoals, invariants, graphs, graph frontier/checkpoint, scratch, churn, and mode
- it increments the Epiphany revision, updates `lastUpdatedTurnId` when a reference turn exists, writes live `SessionState`, persists `RolloutItem::EpiphanyState`, and flushes the rollout
- rollout replay now accepts an out-of-band Epiphany snapshot before the first real user turn so pre-turn seed updates survive resume
Follow-up addition now landed after the update surface:
-`thread/epiphany/distill`
Current bounded shape of that follow-up:
- it is a read-only proposal path, not a second state writer
- it requires a loaded thread so the response can include the current `expectedRevision`
- it normalizes one explicit source/status/text observation plus optional subject, evidence kind, and code refs
- it returns deterministic observation/evidence records inside a `ThreadEpiphanyUpdatePatch`
- callers must pass the patch to `thread/epiphany/update` to make it durable
Follow-up addition now landed after the promotion gate:
- richer `thread/epiphany/promote` policy for state replacement patches
Current bounded shape of that follow-up:
- it keeps the wire protocol stable
- it still rejects failed verifier evidence without mutation
- it now passes replacement fields into `epiphany-core` policy evaluation instead of reducing them to a generic boolean
- map/churn/frontier/checkpoint replacements must include explicit observations and patch evidence, so state edits are tied to a verified observation trail
- subgoals, invariants, graphs, graph links, frontier ids, checkpoint ids, and churn fields get lightweight structural validation before accepted patches are sent to `thread/epiphany/update`
- it still does not infer graph edits automatically; callers must submit explicit typed replacements
Treat the current retrieval baseline, explicit indexing follow-up, explicit update path, and first distillation proposal path as real and live-smoked. The next machine gap is no longer "can Epiphany retrieve code?" or "can it propose one durable observation patch?" It can. The next gap is "can Epiphany decide which observations deserve promotion, and what map/churn edits they imply?"
