# Scratch This file is intentionally disposable. ## Current Subgoal - Clean up, persist, and ship the verified first bounded Phase 4 repo-local hybrid retrieval slice in vendored Codex. ## Working Notes - The old lone-diff situation is gone. The current in-flight Phase 4 wiring now spans: - `vendor/codex/codex-rs/protocol/src/protocol.rs` - `vendor/codex/codex-rs/core/src/epiphany_retrieval.rs` - `vendor/codex/codex-rs/core/src/codex_thread.rs` - `vendor/codex/codex-rs/core/src/lib.rs` - `vendor/codex/codex-rs/core/Cargo.toml` - `vendor/codex/codex-rs/core/src/session/tests.rs` - `vendor/codex/codex-rs/app-server-protocol/src/protocol/common.rs` - `vendor/codex/codex-rs/app-server-protocol/src/protocol/v2.rs` - `vendor/codex/codex-rs/app-server-protocol/src/export.rs` - `vendor/codex/codex-rs/app-server/src/codex_message_processor.rs` - `vendor/codex/codex-rs/app-server/README.md` - regenerated stable fixtures under `vendor/codex/codex-rs/app-server-protocol/schema/` - The current implementation shape is the bounded one we wanted: - exact/path-ish lookup via `codex_file_search` - semantic lookup via query-time BM25 chunk search in the new core retrieval module - additive experimental app-server method `thread/epiphany/retrieve` - lightweight retrieval-summary backfill for live `thread.epiphanyState` when the thread already has Epiphany state but no persisted retrieval summary yet - `codex-core` now depends on both `bm25` and `codex-file-search`; `ignore` is used to walk a gitignore-respecting local corpus for semantic chunking. - The semantic side is intentionally small: - text-only files - size cap per file - fixed line-window chunking with overlap - no embeddings, no persistent blob, no watcher invalidation - Corpus preflight for the tracked `vendor/codex/codex-rs` source at `vendor/codex` HEAD `d45ab10`: `3642` tracked files, `31.09 MB`. - Do not use raw working-tree size as the retrieval denominator here; build artifacts and other debris inflate it badly and will lie to the design. - Verification is no longer theoretical: - `cargo fmt --all` passed - `cargo test -p codex-core -p codex-app-server-protocol -p codex-app-server --lib --no-run` passed with `CARGO_TARGET_DIR=C:\Users\Meta\.cargo-target-codex` - targeted Phase 4 tests passed in core, app-server protocol, and app-server - full `cargo test -p codex-app-server-protocol` passed after regenerating stable schema fixtures - Small fallout that had to be patched: - `codex_message_processor.rs` had an unnecessary `live_thread.as_ref()` - `core/src/session/tests.rs` needed `retrieval: None` in the prompt fixture Epiphany state - the exact-hit retrieval test fixture needed an actually matchable path (`session_checkpoint.rs`) - `EpiphanyThreadState` needed an explicit exemption in the protocol export test because it is a sparse durable state object, not a params bag - Important schema footgun: - `cargo run -p codex-app-server-protocol --bin write_schema_fixtures -- --experimental` rewrites the same checked-in stable schema tree - leaving the repo in that state makes stable protocol tests fail - the checked-in repo state must stay on the stable fixture generation path - despite the giant `git status` scream wall, the real schema content diff is small and expected: - `git diff --numstat` shows actual content changes in `15` generated schema files - plus the new untracked Epiphany TypeScript schema files - the broader wall is mostly line-ending/worktree noise, not extra logical surface area - After inspecting the live code path, do not widen `thread/epiphany/retrieve` into a durable Epiphany-state write right now: - durable `EpiphanyState` snapshots are currently persisted on real user-turn boundaries - making retrieval calls append durable snapshots would require a new out-of-band rollout semantic or a fake turn boundary - that is a bigger machine than this slice deserves - New infrastructure direction: - for the first persistent semantic backend after this verified BM25 slice, use Qdrant instead of inventing another monolithic JSON/blob/postgres-embedding store - Qdrant should be the preferred persistent semantic backend later, not a hard runtime requirement for basic Epiphany use - exact/path lookup stays first-class - BM25 should stay available as a bootstrap/fallback/control path for users who do not want Qdrant or when Qdrant is unavailable ## Open Questions - When retrieval freshness/persistence semantics come later, should they ride a dedicated Epiphany update path instead of piggybacking on a read/query method? Do not promote anything from here into the map unless it survives verification or repeated reuse without contradiction.