diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index fa829ec8..198193c4 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -60,6 +60,7 @@ remembers doctrine; the branch remembers the blade. - Turn control routes now live in `codex_message_processor/turn_routes.rs`: turn start, thread item injection, app-server client info setting, turn steer, turn interrupt, mode normalization, and input-limit validation with module-local unit tests. - Thread projection now lives in `codex_message_processor/thread_projection.rs`: state-db/thread-store/rollout summary projection, `ConversationSummary` to API `Thread` mapping, thread-title attachment helpers, turn pagination, and rollout turn reconstruction. `codex_message_processor.rs` is about 1,009 lines after this extraction and mostly owns JSON-RPC dispatch, processor construction, shared thread lookup/submit/error helpers, and lifecycle shutdown. - Turn paging/reconstruction now lives in `codex_message_processor/thread_turn_projection.rs`; `thread_projection.rs` keeps state-db/thread-store/rollout summary projection and `ConversationSummary` to API `Thread` mapping. Focused `thread_turns_list` cursor tests and stale in-progress turn interruption coverage pass. +- The `thread/turns/list` route handler now lives in `codex_message_processor/thread_turn_routes.rs`, beside the turn projection helpers it consumes. `thread_read_routes.rs` is back to persisted/live thread listing, `thread/read`, loaded-thread listing, and summary reads. - Listener/subscription lifecycle now lives in `codex_message_processor/listener_lifecycle.rs`: connection attach/detach, listener startup, idle unload, unsubscribe, teardown, archive preparation, listener task context, idle-unload state, shutdown result handling, and listener command routing. - The May 2026 foundation migration in `notes/epiphany-ideal-architecture-rebuild-plan.md` is complete enough to defend. Read `notes/epiphany-architectural-teardown.md` before touching Epiphany control-plane code, but do not treat it as an open-ended excuse to keep shaving the altar. The rebuild moved scene projection, freshness derivation, pressure policy, targeted context shards, bounded graph traversal, job/progress view derivation, planning view derivation, reorientation resume/regather verdict policy, pure CRRC recommendation policy, fixed-lane coordinator decision policy, role-board projection policy, role/reorient result interpretation, role self-persistence review policy, and role/reorient acceptance bundle policy into `epiphany-core`; typed acceptance receipts replaced summary-string identity for live accept paths, runtime links are dual-written on launch, and result read-back now prefers runtime links. - Read `notes/epiphany-ideal-architecture-rebuild-plan.md` immediately after the teardown. It defines the smallest coherent replacement architecture: durable `EpiphanyState`, runtime-spine-owned `RuntimeState`, separate role memory/heartbeat state, `epiphany-core` as policy owner, app-server as adapter, Aquarium as reflector, typed acceptance receipts, and derived view lenses instead of a protocol verb zoo. diff --git a/state/ledgers.msgpack b/state/ledgers.msgpack index df8b0de0..5c933e5b 100644 Binary files a/state/ledgers.msgpack and b/state/ledgers.msgpack differ diff --git a/state/map.yaml b/state/map.yaml index 63b14c25..a6f81b4b 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -286,10 +286,8 @@ rejected_paths: current_status: summary: The Codex starvation pass has five typing receipts, fifty-six processor/foundation cuts, a real physical extraction in `epiphany-codex-bridge`, a root `epiphany-state-model` crate for durable Epiphany state, a source-grounded Codex auth spine inventory, the native `epiphany-openai-adapter` document boundary, a buildable Codex auth/model wrapper in `epiphany-openai-codex-spine`, the first native runtime caller in `epiphany-openai-runtime`, native heartbeat launch planning/job opening in `epiphany-core`, native state-update contract/validation/application in `epiphany-core`, native exact retrieval path matching in `epiphany-core`, and the route-facing mutation/view/coordinator projection services in `epiphany-codex-bridge`. `codex-protocol` no longer defines the durable `Epiphany*` state structs; it re-exports the native model for legacy rollout/API compatibility while `epiphany-core` imports the native model directly. `epiphany-core` now has no direct vendored Codex dependency: pressure derivation accepts native `EpiphanyTokenUsageSnapshot`, Codex rollout replay moved to `codex-core::epiphany_rollout`, and exact retrieval search uses Epiphany-owned workspace walking/path scoring instead of `codex-file-search`. The bridge owns Codex-facing Epiphany adapters for scene, jobs, context/planning/graph query, retrieval/indexing projection, result projection, runtime-spine result adaptation, launch/prompt doctrine, mutation document mechanics, mutation service update/promote/role-accept/reorient-accept/role-launch/reorient-launch/job-launch/job-interrupt application, live-state projection, pressure/pre-compaction rendering, freshness/reorientation projection, CRRC/coordinator/role-board projection, watcher invalidation adaptation, full `thread/epiphany/view` response construction, freshness/context/graph-query response construction, role/reorient result response construction, distill/propose patch-response construction, coordinator status/view construction, and acceptance/evidence signal helpers. Vendored app-server no longer retains root Epiphany modules; `codex_message_processor.rs` still contains route dispatch, JSON-RPC response shaping, watcher registration/snapshotting calls, and notification emission, but its root no longer imports the evacuated Epiphany policy/protocol registry or owns the watcher invalidation manager. Codex product surfaces for apps, skills, plugins, marketplace, and external agent config import are cut from the live app-server/protocol path: list routes return empty lists only where compatibility shells still exist, mutation/detail routes report disabled where they survive, `externalAgentConfig/*` routes and import-completed notification are gone, helper/import modules and stale implementation-shape tests were deleted, runtime feature enablement no longer supports `apps` or `plugins`, config writes no longer emit plugin-toggle analytics, plugin/skill cache invalidation on config writes is gone, plugin startup warmups and app-list refreshes were removed, and app-server no longer directly depends on `codex-plugin`, `codex-chatgpt`, or `codex-core-plugins`. `Config::to_mcp_config` now keeps user-declared MCP servers but no longer accepts `PluginsManager`, folds plugin MCP servers into the runtime, enables Codex apps, enables skill MCP dependency install, or advertises plugin capability summaries; `TurnContext::apps_enabled()` now returns false so model turns cannot awaken Codex Apps through ChatGPT auth or feature flags; session startup and per-turn context no longer load skills, turns receive an empty `SkillLoadOutcome`, internal `Op::ListSkills` is gone from `codex-protocol` / `codex-core` / TUI command routing, the Codex skill watcher has been collapsed to a no-op marker with no file watcher, skill-root registration, cache-clear task, or `SkillsUpdateAvailable` session listener, model turns no longer run skill mention collection, skill env-var prompts, skill MCP dependency installation, skill instruction injection, or implicit skill invocation analytics from shell tools, and the turn prompt/router path no longer loads plugin/app capability summaries, injects available-skill/available-plugin/plugin instruction fragments, parses plugin/app mentions, records app/plugin invocation telemetry, or filters MCP tools through Codex app connectors. Skill mutation/list/notification is gone from the app-server edge too: `skills/list`, `skills/config/write`, `skills/changed`, `Op::ListSkills`, `EventMsg::ListSkillsResponse`, `EventMsg::SkillsUpdateAvailable`, the app-server handlers, TUI slash/startup/list/manage-skill surfaces, generated schema entries, core skill-list suite, and fake live-reload/developer-message tests were deleted. Skill mention plumbing is also gone from the live TUI/protocol edge: no `UserInput::Skill` variant, app-server schema branch, TUI skill popup, `skills_all` cache, composer `set_skill_mentions`, skill mention insertion, `skill://` / `SKILL.md` history decoding, or stale skill popup snapshot remains. The `codex-core-skills` crate, core `skills` re-export shim, no-op `skills_watcher`, session `SkillsManager` service, and per-turn `TurnSkillsContext` are deleted; core/protocol no longer keep legacy `` or `` prompt compatibility, memory usage classification, snapshot normalization, or cleanup paths. Remaining product-shaped skill residue is also cut from memory templates, app-server docs/tests/warnings, TUI tooltips/placeholders/warnings/comments, app-server v1 docs, agent-tool schema text, stale `UserInput::Skill` test arms, and obsolete feature flags/schema entries. App-instruction prompt compatibility is cut too: no `APPS_INSTRUCTIONS` protocol constants, no `` snapshot canonicalization, and MCP fixture tests now use neutral `demo_mcp` / `calendar_mcp` names instead of `codex_apps` / `/api/codex/apps` branding. The TUI app connector product path is also deleted: no `/apps` slash command, app-link view, connector AppEvent variants, app connector cache/popup/refresh loop, app mention insertion, `app://` mention codec support, stale app-popup tests, or TUI browser-opening shim remains. The `app/list` route and `app/list/updated` notification are gone from app-server protocol, generated schemas, app-server dispatch, test helpers, and docs; `AppInfo`/app-list metadata schema structs are deleted; the orphaned `codex-connectors` crate and `core::connectors` shim are deleted from the workspace; Codex app analytics no longer tracks `codex_app_mentioned` / `codex_app_used` facts, events, reducer arms, client APIs, or dedupe cache state; `[apps.*]`, `Feature::Apps`, `include_apps_instructions`, app config/read schemas, app managed requirements, app prompt tests, and app file-MCP tests are gone; `Feature::Plugins`, `Feature::RemotePlugin`, `plugin://` structured mentions, plugin instruction tags, and stale plugin popup snapshots are gone too; `[plugins]` / `[marketplaces]`, `PluginConfig`, `MarketplaceConfig`, marketplace edit helpers, stale plugin/external-agent README route claims, and generated schema entries are removed from config; and the bundled `codex-skills` sample-skill crate plus embedded imagegen/openai-docs/plugin-creator/skill-creator/skill-installer payloads are deleted from the workspace. The vendored `vendor/codex/sdk` subtree is deleted as non-spine app-server client/release scaffolding, including stale generated Python SDK route types for removed product surfaces; the root pnpm workspace no longer lists `sdk/typescript`. Live MCP tool approval is named `McpToolApproval`, not `AppToolApproval`, and remains under MCP server config; live structured user-input mentions are MCP-shaped only. `codex_message_processor.rs` is about 1,009 lines after route/lifecycle/projection/resume-helper/config-helper extraction; root `message_processor.rs` is about 1,292 lines after the external-agent/config-product cuts; `epiphany_read_routes.rs` is about 579 lines after the full view and read-response extractions; `epiphany_mutation_routes.rs` is about 694 lines after route-load/notification helper cuts and product-surface starvation; the old vendored `epiphany_state_helpers.rs` rollout-state loader is deleted and its remaining function lives in `epiphany-codex-bridge::state`; and `codex_thread.rs` is about 1,425 lines after state-update law moved native. `epiphany-openai-adapter` defines CultCache `DatabaseEntry` request/event/receipt/auth-status documents with no Codex app-server or generic JSON dependency. `epiphany-openai-codex-spine` owns the native OpenAI model transport path: it exposes `default_codex_home` and `auth_manager`, projects `epiphany-openai-auth-spine` `AuthManager` / `CodexAuth` into typed `EpiphanyOpenAiAdapterStatus`, builds a local serializable OpenAI Responses request body instead of Codex protocol request/item/config types, opens HTTP Responses streams through `codex-client`, parses Responses SSE frames into typed stream events and receipts, and no longer directly or transitively depends on Codex's Responses client, model-provider stack, `codex-protocol`, `codex-api`, `codex-model-provider`, or `codex-model-provider-info`; it intentionally reaches retained vendored `codex-login` only through `epiphany-openai-auth-spine`. Auth identity, credential storage, and ChatGPT token refresh remain owned by the Codex-compatible auth organ; external bearer command auth and agent identity remain retained Codex auth capabilities to expose only if a typed Epiphany surface needs them. `epiphany-openai-runtime` no longer depends directly on `codex-login`; it writes OpenAI status/request/stream-event/receipt records plus runtime sessions/jobs/results/events into `epiphany-core::runtime_spine_cache`, can emit the assistant text artifact for legacy result consumers, is now the default executor for startup-only repo birth specialists, and now has `run-worker` to load a durable `EpiphanyRuntimeWorkerLaunchRequest`, call the typed OpenAI runtime through the spine, persist typed `EpiphanyRuntimeRoleWorkerResult` / `EpiphanyRuntimeReorientWorkerResult` documents, and complete the original heartbeat/specialist runtime job without Codex worker execution; `codex exec` survives only as explicit fallback in repo-birth startup. `epiphany-mvp-coordinator` now invokes `epiphany-openai-runtime run-worker` for launched runtime job ids and no longer creates a second coordinator-owned shadow runtime job/result as a compensator. `epiphany-core::plan_runtime_spine_heartbeat_launch` validates launch contracts/conflicts and projects durable job bindings/runtime links; `epiphany-core::open_runtime_spine_heartbeat_job` owns the initialize-runtime/ensure-session/create-job sequence and now persists `EpiphanyRuntimeWorkerLaunchRequest` CultCache records keyed by runtime job id, with indexed kind/contract fields plus MessagePack-encoded typed launch documents instead of discarding work intent after queue receipt creation; `roleResult` and `reorientResult` now require typed worker result documents; generic job summaries are lifecycle receipts, not reviewable findings; `epiphany-core::EpiphanyStateUpdate`, `epiphany_state_update_validation_errors`, and `apply_epiphany_state_update` own state-update document shape and mutation law while `codex-core` calls them for compatibility. Vendored `CodexThread::epiphany_launch_job` is now mostly revision check, persistence validation, launch request handoff, and rollout/session writeback for the compatibility route. Vendored `codex-core` no longer has `epiphany_distillation`, `epiphany_promotion`, `epiphany_proposal`, or `epiphany_retrieval` husk modules; it re-exports native surfaces from `epiphany-core` directly, and now owns only the real Codex rollout adapter because it is a host seam. `epiphany-runtime-spine` advertises OpenAI adapter status/model-request/stream-event/receipt CultNet schemas and mutation contracts, so the native caller now has a real storage route instead of only JSON-RPC. MCP itself is correctly JSON at its protocol edge; the target is typed Epiphany intent/result/receipt wrappers around MCP, not replacing MCP JSON. The current OpenAI request path no longer pulls `codex-api`; `codex-login` remains deliberately sealed behind `epiphany-openai-auth-spine`, and the next spine decision is another Epiphany-in-vendor evacuation surface unless a concrete typed auth-status/runtime need requires exposing external bearer or agent-identity auth. latest_cut: > - Turn projection was split out of thread summary projection. - `thread_turn_projection.rs` owns turn-list cursors, page slicing, rollout - turn reconstruction, active-turn merge, and stale in-progress turn - interruption. `thread_projection.rs` now keeps state-db/thread-store/rollout - summary projection and API thread mapping. Focused turn-list cursor and - stale-turn interruption tests pass. + The `thread/turns/list` route handler was split out of generic thread read + routing. `thread_turn_routes.rs` owns rollout lookup plus turn-list response + shaping, and `thread_turn_projection.rs` owns cursor/page/reconstruction + mechanics. Focused turn-list cursor tests pass. next_action: Continue the Codex starvation campaign at the next Epiphany-in-vendor evacuation surface; keep vendored Codex code as host routing/lifecycle glue only, preserve retained `codex-login` auth ownership, and prefer cuts that expose testable ownership boundaries over line-count theater. diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs index 99a25f74..16f8e1e3 100644 --- a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs @@ -19,6 +19,7 @@ mod thread_read_routes; mod thread_resume_routes; mod thread_start_routes; mod thread_turn_projection; +mod thread_turn_routes; mod turn_routes; use self::auth_routes::ActiveLogin; diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs index bb0ff2cf..f4a6bd5a 100644 --- a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs @@ -408,156 +408,6 @@ impl CodexMessageProcessor { Ok(()) } - pub(super) async fn thread_turns_list( - &self, - request_id: ConnectionRequestId, - params: ThreadTurnsListParams, - ) { - let ThreadTurnsListParams { - thread_id, - cursor, - limit, - sort_direction, - } = params; - - let thread_uuid = match ThreadId::from_string(&thread_id) { - Ok(id) => id, - Err(err) => { - self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) - .await; - return; - } - }; - - let state_db_ctx = get_state_db(&self.config).await; - let mut rollout_path = self - .resolve_rollout_path(thread_uuid, state_db_ctx.as_ref()) - .await; - if rollout_path.is_none() { - rollout_path = - match find_thread_path_by_id_str(&self.config.codex_home, &thread_uuid.to_string()) - .await - { - Ok(Some(path)) => Some(path), - Ok(None) => match find_archived_thread_path_by_id_str( - &self.config.codex_home, - &thread_uuid.to_string(), - ) - .await - { - Ok(path) => path, - Err(err) => { - self.send_invalid_request_error( - request_id, - format!("failed to locate archived thread id {thread_uuid}: {err}"), - ) - .await; - return; - } - }, - Err(err) => { - self.send_invalid_request_error( - request_id, - format!("failed to locate thread id {thread_uuid}: {err}"), - ) - .await; - return; - } - }; - } - - if rollout_path.is_none() { - match self.thread_manager.get_thread(thread_uuid).await { - Ok(thread) => { - rollout_path = thread.rollout_path(); - if rollout_path.is_none() { - self.send_invalid_request_error( - request_id, - "ephemeral threads do not support thread/turns/list".to_string(), - ) - .await; - return; - } - } - Err(_) => { - self.send_invalid_request_error( - request_id, - format!("thread not loaded: {thread_uuid}"), - ) - .await; - return; - } - } - } - - let Some(rollout_path) = rollout_path.as_ref() else { - self.send_internal_error( - request_id, - format!("failed to locate rollout for thread {thread_uuid}"), - ) - .await; - return; - }; - - match read_rollout_items_from_rollout(rollout_path).await { - Ok(items) => { - // This API optimizes network transfer by letting clients page through a - // thread's turns incrementally, but it still replays the entire rollout on - // every request. Rollback and compaction events can change earlier turns, so - // the server has to rebuild the full turn list until turn metadata is indexed - // separately. - let has_live_in_progress_turn = - match self.thread_manager.get_thread(thread_uuid).await { - Ok(thread) => matches!(thread.agent_status().await, AgentStatus::Running), - Err(_) => false, - }; - let turns = reconstruct_thread_turns_from_rollout_items( - &items, - self.thread_watch_manager - .loaded_status_for_thread(&thread_uuid.to_string()) - .await, - has_live_in_progress_turn, - ); - let page = match paginate_thread_turns( - turns, - cursor.as_deref(), - limit, - sort_direction.unwrap_or(SortDirection::Desc), - ) { - Ok(page) => page, - Err(error) => { - self.outgoing.send_error(request_id, error).await; - return; - } - }; - let response = ThreadTurnsListResponse { - data: page.turns, - next_cursor: page.next_cursor, - backwards_cursor: page.backwards_cursor, - }; - self.outgoing.send_response(request_id, response).await; - } - Err(err) if err.kind() == std::io::ErrorKind::NotFound => { - self.send_invalid_request_error( - request_id, - format!( - "thread {thread_uuid} is not materialized yet; thread/turns/list is unavailable before first user message" - ), - ) - .await; - } - Err(err) => { - self.send_internal_error( - request_id, - format!( - "failed to load rollout `{}` for thread {thread_uuid}: {err}", - rollout_path.display() - ), - ) - .await; - } - } - } pub(super) async fn get_thread_summary( &self, request_id: ConnectionRequestId, diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_turn_routes.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_turn_routes.rs new file mode 100644 index 00000000..2ef1e2e4 --- /dev/null +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_turn_routes.rs @@ -0,0 +1,154 @@ +use super::*; + +impl CodexMessageProcessor { + pub(super) async fn thread_turns_list( + &self, + request_id: ConnectionRequestId, + params: ThreadTurnsListParams, + ) { + let ThreadTurnsListParams { + thread_id, + cursor, + limit, + sort_direction, + } = params; + + let thread_uuid = match ThreadId::from_string(&thread_id) { + Ok(id) => id, + Err(err) => { + self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) + .await; + return; + } + }; + + let state_db_ctx = get_state_db(&self.config).await; + let mut rollout_path = self + .resolve_rollout_path(thread_uuid, state_db_ctx.as_ref()) + .await; + if rollout_path.is_none() { + rollout_path = + match find_thread_path_by_id_str(&self.config.codex_home, &thread_uuid.to_string()) + .await + { + Ok(Some(path)) => Some(path), + Ok(None) => match find_archived_thread_path_by_id_str( + &self.config.codex_home, + &thread_uuid.to_string(), + ) + .await + { + Ok(path) => path, + Err(err) => { + self.send_invalid_request_error( + request_id, + format!("failed to locate archived thread id {thread_uuid}: {err}"), + ) + .await; + return; + } + }, + Err(err) => { + self.send_invalid_request_error( + request_id, + format!("failed to locate thread id {thread_uuid}: {err}"), + ) + .await; + return; + } + }; + } + + if rollout_path.is_none() { + match self.thread_manager.get_thread(thread_uuid).await { + Ok(thread) => { + rollout_path = thread.rollout_path(); + if rollout_path.is_none() { + self.send_invalid_request_error( + request_id, + "ephemeral threads do not support thread/turns/list".to_string(), + ) + .await; + return; + } + } + Err(_) => { + self.send_invalid_request_error( + request_id, + format!("thread not loaded: {thread_uuid}"), + ) + .await; + return; + } + } + } + + let Some(rollout_path) = rollout_path.as_ref() else { + self.send_internal_error( + request_id, + format!("failed to locate rollout for thread {thread_uuid}"), + ) + .await; + return; + }; + + match read_rollout_items_from_rollout(rollout_path).await { + Ok(items) => { + // This API optimizes network transfer by letting clients page through a + // thread's turns incrementally, but it still replays the entire rollout on + // every request. Rollback and compaction events can change earlier turns, so + // the server has to rebuild the full turn list until turn metadata is indexed + // separately. + let has_live_in_progress_turn = + match self.thread_manager.get_thread(thread_uuid).await { + Ok(thread) => matches!(thread.agent_status().await, AgentStatus::Running), + Err(_) => false, + }; + let turns = reconstruct_thread_turns_from_rollout_items( + &items, + self.thread_watch_manager + .loaded_status_for_thread(&thread_uuid.to_string()) + .await, + has_live_in_progress_turn, + ); + let page = match paginate_thread_turns( + turns, + cursor.as_deref(), + limit, + sort_direction.unwrap_or(SortDirection::Desc), + ) { + Ok(page) => page, + Err(error) => { + self.outgoing.send_error(request_id, error).await; + return; + } + }; + let response = ThreadTurnsListResponse { + data: page.turns, + next_cursor: page.next_cursor, + backwards_cursor: page.backwards_cursor, + }; + self.outgoing.send_response(request_id, response).await; + } + Err(err) if err.kind() == std::io::ErrorKind::NotFound => { + self.send_invalid_request_error( + request_id, + format!( + "thread {thread_uuid} is not materialized yet; thread/turns/list is unavailable before first user message" + ), + ) + .await; + } + Err(err) => { + self.send_internal_error( + request_id, + format!( + "failed to load rollout `{}` for thread {thread_uuid}: {err}", + rollout_path.display() + ), + ) + .await; + } + } + } +}