diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index b2d73cf1..9449b492 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -48,7 +48,7 @@ remembers doctrine; the branch remembers the blade. - Realtime conversation routes now live in `codex_message_processor/realtime_routes.rs`, and review-start orchestration now lives in `codex_message_processor/review_routes.rs`; the root processor is about 6,938 lines. - Model/catalog routes now live in `codex_message_processor/catalog_routes.rs`: model list, collaboration-mode list, experimental feature list, and mock experimental method. The root processor is about 6,738 lines. - Thread administrative mutation routes now live in `codex_message_processor/thread_admin_routes.rs`: archive/unarchive, elicitation counters, set-name, memory mode, memory reset, metadata update, rollback, compact, background terminal cleanup, shell command submission, and guardian approval. The root processor is about 5,750 lines. -- Thread read/list projection routes now live in `codex_message_processor/thread_read_routes.rs`: persisted/live thread listing, loaded-thread listing, `thread/read`, Epiphany state attachment for reads, rollout turn paging, and read-side rollout field hydration. The root processor is about 5,241 lines. +- Thread read/list projection routes now live in `codex_message_processor/thread_read_routes.rs`: persisted/live thread listing, loaded-thread listing, `thread/read`, conversation summary reads, shared list pagination, Epiphany state attachment for reads, rollout turn paging, and read-side rollout field hydration. The root processor is about 3,105 lines after later route extractions. - Thread resume/fork orchestration now lives in `codex_message_processor/thread_resume_routes.rs`; it owns resume from history/rollout, running-thread resume response replay, persisted resume metadata hydration, fork response construction, and the shared thread-title attachment helper. The root processor is about 4,336 lines. - Thread start route handling now lives in `codex_message_processor/thread_start_routes.rs`; the route shell delegates to the root lifecycle `thread_start_task`. The root processor is about 4,250 lines. - Turn control routes now live in `codex_message_processor/turn_routes.rs`: turn start, thread item injection, app-server client info setting, turn steer, and turn interrupt. The root processor is about 3,803 lines. diff --git a/state/ledgers.msgpack b/state/ledgers.msgpack index 0d53fc7c..b1957625 100644 Binary files a/state/ledgers.msgpack and b/state/ledgers.msgpack differ diff --git a/state/map.yaml b/state/map.yaml index 0d28fedd..9f2d99b0 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -286,10 +286,8 @@ rejected_paths: current_status: summary: The Codex starvation pass has five typing receipts, fifty-two processor/foundation cuts, a real physical extraction in `epiphany-codex-bridge`, a root `epiphany-state-model` crate for durable Epiphany state, a source-grounded Codex auth spine inventory, the native `epiphany-openai-adapter` document boundary, a buildable Codex auth/model wrapper in `epiphany-openai-codex-spine`, the first native runtime caller in `epiphany-openai-runtime`, native heartbeat launch planning/job opening in `epiphany-core`, native state-update contract/validation/application in `epiphany-core`, native exact retrieval path matching in `epiphany-core`, and the route-facing mutation/view/coordinator projection services in `epiphany-codex-bridge`. `codex-protocol` no longer defines the durable `Epiphany*` state structs; it re-exports the native model for legacy rollout/API compatibility while `epiphany-core` imports the native model directly. `epiphany-core` now has no direct vendored Codex dependency: pressure derivation accepts native `EpiphanyTokenUsageSnapshot`, Codex rollout replay moved to `codex-core::epiphany_rollout`, and exact retrieval search uses Epiphany-owned workspace walking/path scoring instead of `codex-file-search`. The bridge owns Codex-facing Epiphany adapters for scene, jobs, context/planning/graph query, retrieval/indexing projection, result projection, runtime-spine result adaptation, launch/prompt doctrine, mutation document mechanics, mutation service update/promote/role-accept/reorient-accept/role-launch/reorient-launch/job-launch/job-interrupt application, live-state projection, pressure/pre-compaction rendering, freshness/reorientation projection, CRRC/coordinator/role-board projection, watcher invalidation adaptation, full `thread/epiphany/view` response construction, freshness/context/graph-query response construction, role/reorient result response construction, distill/propose patch-response construction, coordinator status/view construction, and acceptance/evidence signal helpers. Vendored app-server no longer retains root Epiphany modules; `codex_message_processor.rs` still contains route dispatch, JSON-RPC response shaping, watcher registration/snapshotting calls, and notification emission, but its root no longer imports the evacuated Epiphany policy/protocol registry or owns the watcher invalidation manager. Codex product surfaces for apps, skills, plugins, marketplace, and external agent config import are cut from the live app-server/protocol path: list routes return empty lists only where compatibility shells still exist, mutation/detail routes report disabled where they survive, `externalAgentConfig/*` routes and import-completed notification are gone, helper/import modules and stale implementation-shape tests were deleted, runtime feature enablement no longer supports `apps` or `plugins`, config writes no longer emit plugin-toggle analytics, plugin/skill cache invalidation on config writes is gone, plugin startup warmups and app-list refreshes were removed, and app-server no longer directly depends on `codex-plugin`, `codex-chatgpt`, or `codex-core-plugins`. `Config::to_mcp_config` now keeps user-declared MCP servers but no longer accepts `PluginsManager`, folds plugin MCP servers into the runtime, enables Codex apps, enables skill MCP dependency install, or advertises plugin capability summaries; `TurnContext::apps_enabled()` now returns false so model turns cannot awaken Codex Apps through ChatGPT auth or feature flags; session startup and per-turn context no longer load skills, turns receive an empty `SkillLoadOutcome`, internal `Op::ListSkills` is gone from `codex-protocol` / `codex-core` / TUI command routing, the Codex skill watcher has been collapsed to a no-op marker with no file watcher, skill-root registration, cache-clear task, or `SkillsUpdateAvailable` session listener, model turns no longer run skill mention collection, skill env-var prompts, skill MCP dependency installation, skill instruction injection, or implicit skill invocation analytics from shell tools, and the turn prompt/router path no longer loads plugin/app capability summaries, injects available-skill/available-plugin/plugin instruction fragments, parses plugin/app mentions, records app/plugin invocation telemetry, or filters MCP tools through Codex app connectors. Skill mutation/list/notification is gone from the app-server edge too: `skills/list`, `skills/config/write`, `skills/changed`, `Op::ListSkills`, `EventMsg::ListSkillsResponse`, `EventMsg::SkillsUpdateAvailable`, the app-server handlers, TUI slash/startup/list/manage-skill surfaces, generated schema entries, core skill-list suite, and fake live-reload/developer-message tests were deleted. Skill mention plumbing is also gone from the live TUI/protocol edge: no `UserInput::Skill` variant, app-server schema branch, TUI skill popup, `skills_all` cache, composer `set_skill_mentions`, skill mention insertion, `skill://` / `SKILL.md` history decoding, or stale skill popup snapshot remains. The `codex-core-skills` crate, core `skills` re-export shim, no-op `skills_watcher`, session `SkillsManager` service, and per-turn `TurnSkillsContext` are deleted; core/protocol no longer keep legacy `` or `` prompt compatibility, memory usage classification, snapshot normalization, or cleanup paths. Remaining product-shaped skill residue is also cut from memory templates, app-server docs/tests/warnings, TUI tooltips/placeholders/warnings/comments, app-server v1 docs, agent-tool schema text, stale `UserInput::Skill` test arms, and obsolete feature flags/schema entries. App-instruction prompt compatibility is cut too: no `APPS_INSTRUCTIONS` protocol constants, no `` snapshot canonicalization, and MCP fixture tests now use neutral `demo_mcp` / `calendar_mcp` names instead of `codex_apps` / `/api/codex/apps` branding. The TUI app connector product path is also deleted: no `/apps` slash command, app-link view, connector AppEvent variants, app connector cache/popup/refresh loop, app mention insertion, `app://` mention codec support, stale app-popup tests, or TUI browser-opening shim remains. The `app/list` route and `app/list/updated` notification are gone from app-server protocol, generated schemas, app-server dispatch, test helpers, and docs; `AppInfo`/app-list metadata schema structs are deleted; the orphaned `codex-connectors` crate and `core::connectors` shim are deleted from the workspace; Codex app analytics no longer tracks `codex_app_mentioned` / `codex_app_used` facts, events, reducer arms, client APIs, or dedupe cache state; `[apps.*]`, `Feature::Apps`, `include_apps_instructions`, app config/read schemas, app managed requirements, app prompt tests, and app file-MCP tests are gone; `Feature::Plugins`, `Feature::RemotePlugin`, `plugin://` structured mentions, plugin instruction tags, and stale plugin popup snapshots are gone too; `[plugins]` / `[marketplaces]`, `PluginConfig`, `MarketplaceConfig`, marketplace edit helpers, stale plugin/external-agent README route claims, and generated schema entries are removed from config; and the bundled `codex-skills` sample-skill crate plus embedded imagegen/openai-docs/plugin-creator/skill-creator/skill-installer payloads are deleted from the workspace. The vendored `vendor/codex/sdk` subtree is deleted as non-spine app-server client/release scaffolding, including stale generated Python SDK route types for removed product surfaces; the root pnpm workspace no longer lists `sdk/typescript`. Live MCP tool approval is named `McpToolApproval`, not `AppToolApproval`, and remains under MCP server config; live structured user-input mentions are MCP-shaped only. `codex_message_processor.rs` is about 9,482 lines; root `message_processor.rs` is about 1,292 lines after the external-agent/config-product cuts; `epiphany_read_routes.rs` is about 579 lines after the full view and read-response extractions; `epiphany_mutation_routes.rs` is about 694 lines after route-load/notification helper cuts and product-surface starvation; the old vendored `epiphany_state_helpers.rs` rollout-state loader is deleted and its remaining function lives in `epiphany-codex-bridge::state`; and `codex_thread.rs` is about 1,425 lines after state-update law moved native. `epiphany-openai-adapter` defines CultCache `DatabaseEntry` request/event/receipt/auth-status documents with no Codex app-server or generic JSON dependency. `epiphany-openai-codex-spine` owns the native OpenAI model transport path: it exposes `default_codex_home` and `auth_manager`, projects `epiphany-openai-auth-spine` `AuthManager` / `CodexAuth` into typed `EpiphanyOpenAiAdapterStatus`, builds a local serializable OpenAI Responses request body instead of Codex protocol request/item/config types, opens HTTP Responses streams through `codex-client`, parses Responses SSE frames into typed stream events and receipts, and no longer directly or transitively depends on Codex's Responses client, model-provider stack, `codex-protocol`, `codex-api`, `codex-model-provider`, or `codex-model-provider-info`; it intentionally reaches retained vendored `codex-login` only through `epiphany-openai-auth-spine`. Auth identity, credential storage, and ChatGPT token refresh remain owned by the Codex-compatible auth organ; external bearer command auth and agent identity remain retained Codex auth capabilities to expose only if a typed Epiphany surface needs them. `epiphany-openai-runtime` no longer depends directly on `codex-login`; it writes OpenAI status/request/stream-event/receipt records plus runtime sessions/jobs/results/events into `epiphany-core::runtime_spine_cache`, can emit the assistant text artifact for legacy result consumers, is now the default executor for startup-only repo birth specialists, and now has `run-worker` to load a durable `EpiphanyRuntimeWorkerLaunchRequest`, call the typed OpenAI runtime through the spine, persist typed `EpiphanyRuntimeRoleWorkerResult` / `EpiphanyRuntimeReorientWorkerResult` documents, and complete the original heartbeat/specialist runtime job without Codex worker execution; `codex exec` survives only as explicit fallback in repo-birth startup. `epiphany-mvp-coordinator` now invokes `epiphany-openai-runtime run-worker` for launched runtime job ids and no longer creates a second coordinator-owned shadow runtime job/result as a compensator. `epiphany-core::plan_runtime_spine_heartbeat_launch` validates launch contracts/conflicts and projects durable job bindings/runtime links; `epiphany-core::open_runtime_spine_heartbeat_job` owns the initialize-runtime/ensure-session/create-job sequence and now persists `EpiphanyRuntimeWorkerLaunchRequest` CultCache records keyed by runtime job id, with indexed kind/contract fields plus MessagePack-encoded typed launch documents instead of discarding work intent after queue receipt creation; `roleResult` and `reorientResult` now require typed worker result documents; generic job summaries are lifecycle receipts, not reviewable findings; `epiphany-core::EpiphanyStateUpdate`, `epiphany_state_update_validation_errors`, and `apply_epiphany_state_update` own state-update document shape and mutation law while `codex-core` calls them for compatibility. Vendored `CodexThread::epiphany_launch_job` is now mostly revision check, persistence validation, launch request handoff, and rollout/session writeback for the compatibility route. Vendored `codex-core` no longer has `epiphany_distillation`, `epiphany_promotion`, `epiphany_proposal`, or `epiphany_retrieval` husk modules; it re-exports native surfaces from `epiphany-core` directly, and now owns only the real Codex rollout adapter because it is a host seam. `epiphany-runtime-spine` advertises OpenAI adapter status/model-request/stream-event/receipt CultNet schemas and mutation contracts, so the native caller now has a real storage route instead of only JSON-RPC. MCP itself is correctly JSON at its protocol edge; the target is typed Epiphany intent/result/receipt wrappers around MCP, not replacing MCP JSON. The current OpenAI request path no longer pulls `codex-api`; `codex-login` remains deliberately sealed behind `epiphany-openai-auth-spine`, and the next spine decision is another Epiphany-in-vendor evacuation surface unless a concrete typed auth-status/runtime need requires exposing external bearer or agent-identity auth. latest_cut: > - Listener/subscription lifecycle moved out of root - `codex_message_processor.rs` into - `codex_message_processor/listener_lifecycle.rs`. Connection attach/detach, - listener startup, idle unload, unsubscribe, teardown, and archive - preparation are split from dispatch/load/rollout helpers; the root - processor is about 3,275 lines after the cut. + Read-side leftovers moved from root `codex_message_processor.rs` into + `codex_message_processor/thread_read_routes.rs`: conversation summary reads + and shared thread-list pagination now live with the rest of read/list + projection. The root processor is about 3,105 lines after the cut. next_action: Continue the Codex starvation campaign at the next Epiphany-in-vendor evacuation surface; keep vendored Codex code as host routing/lifecycle glue only, and do not cut the retained `codex-login` auth organ unless replacing it with an explicitly permitted OpenAI path rather than a lookalike. diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs index a4a64e5f..8899fa5e 100644 --- a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs @@ -1534,176 +1534,6 @@ impl CodexMessageProcessor { } } - async fn get_thread_summary( - &self, - request_id: ConnectionRequestId, - params: GetConversationSummaryParams, - ) { - let fallback_provider = self.config.model_provider_id.as_str(); - let read_result = match params { - GetConversationSummaryParams::ThreadId { conversation_id } => self - .thread_store - .read_thread(StoreReadThreadParams { - thread_id: conversation_id, - include_archived: true, - include_history: false, - }) - .await - .map_err(|err| conversation_summary_thread_id_read_error(conversation_id, err)), - GetConversationSummaryParams::RolloutPath { rollout_path } => { - let Some(local_thread_store) = self - .thread_store - .as_any() - .downcast_ref::() - else { - let error = JSONRPCErrorError { - code: INVALID_REQUEST_ERROR_CODE, - message: - "rollout path queries are only supported with the local thread store" - .to_string(), - data: None, - }; - return self.outgoing.send_error(request_id, error).await; - }; - - local_thread_store - .read_thread_by_rollout_path( - rollout_path.clone(), - /*include_archived*/ true, - /*include_history*/ false, - ) - .await - .map_err(|err| conversation_summary_rollout_path_read_error(&rollout_path, err)) - } - }; - - match read_result { - Ok(stored_thread) => { - let Some(summary) = summary_from_stored_thread(stored_thread, fallback_provider) - else { - let error = JSONRPCErrorError { - code: INTERNAL_ERROR_CODE, - message: - "failed to load conversation summary: thread is missing rollout path" - .to_string(), - data: None, - }; - self.outgoing.send_error(request_id, error).await; - return; - }; - let response = GetConversationSummaryResponse { summary }; - self.outgoing.send_response(request_id, response).await; - } - Err(error) => { - self.outgoing.send_error(request_id, error).await; - } - } - } - - async fn list_threads_common( - &self, - requested_page_size: usize, - cursor: Option, - sort_key: StoreThreadSortKey, - sort_direction: SortDirection, - filters: ThreadListFilters, - ) -> Result<(Vec, Option), JSONRPCErrorError> { - let ThreadListFilters { - model_providers, - source_kinds, - archived, - cwd_filters, - search_term, - use_state_db_only, - } = filters; - let mut cursor_obj = cursor; - let mut last_cursor = cursor_obj.clone(); - let mut remaining = requested_page_size; - let mut items = Vec::with_capacity(requested_page_size); - let mut next_cursor: Option = None; - - let model_provider_filter = match model_providers { - Some(providers) => { - if providers.is_empty() { - None - } else { - Some(providers) - } - } - None => Some(vec![self.config.model_provider_id.clone()]), - }; - let fallback_provider = self.config.model_provider_id.clone(); - let (allowed_sources_vec, source_kind_filter) = compute_source_filters(source_kinds); - let allowed_sources = allowed_sources_vec.as_slice(); - let store_sort_direction = match sort_direction { - SortDirection::Asc => StoreSortDirection::Asc, - SortDirection::Desc => StoreSortDirection::Desc, - }; - - while remaining > 0 { - let page_size = remaining.min(THREAD_LIST_MAX_LIMIT); - let page = self - .thread_store - .list_threads(StoreListThreadsParams { - page_size, - cursor: cursor_obj.clone(), - sort_key, - sort_direction: store_sort_direction, - allowed_sources: allowed_sources.to_vec(), - model_providers: model_provider_filter.clone(), - cwd_filters: cwd_filters.clone(), - archived, - search_term: search_term.clone(), - use_state_db_only, - }) - .await - .map_err(thread_store_list_error)?; - - let mut filtered = Vec::with_capacity(page.items.len()); - for it in page.items { - let Some(summary) = summary_from_stored_thread(it, fallback_provider.as_str()) - else { - continue; - }; - if source_kind_filter - .as_ref() - .is_none_or(|filter| source_kind_matches(&summary.source, filter)) - && cwd_filters.as_ref().is_none_or(|expected_cwds| { - expected_cwds.iter().any(|expected_cwd| { - path_utils::paths_match_after_normalization(&summary.cwd, expected_cwd) - }) - }) - { - filtered.push(summary); - if filtered.len() >= remaining { - break; - } - } - } - items.extend(filtered); - remaining = requested_page_size.saturating_sub(items.len()); - - next_cursor = page.next_cursor; - if remaining == 0 { - break; - } - - let Some(cursor_val) = next_cursor.clone() else { - break; - }; - // Break if our pagination would reuse the same cursor again; this avoids - // an infinite loop when filtering drops everything on the page. - if last_cursor.as_ref() == Some(&cursor_val) { - next_cursor = None; - break; - } - last_cursor = Some(cursor_val.clone()); - cursor_obj = Some(cursor_val); - } - - Ok((items, next_cursor)) - } - async fn send_invalid_request_error(&self, request_id: ConnectionRequestId, message: String) { let error = JSONRPCErrorError { code: INVALID_REQUEST_ERROR_CODE, diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs index 98f5793f..ac1a246d 100644 --- a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs @@ -518,4 +518,173 @@ impl CodexMessageProcessor { } } } + pub(super) async fn get_thread_summary( + &self, + request_id: ConnectionRequestId, + params: GetConversationSummaryParams, + ) { + let fallback_provider = self.config.model_provider_id.as_str(); + let read_result = match params { + GetConversationSummaryParams::ThreadId { conversation_id } => self + .thread_store + .read_thread(StoreReadThreadParams { + thread_id: conversation_id, + include_archived: true, + include_history: false, + }) + .await + .map_err(|err| conversation_summary_thread_id_read_error(conversation_id, err)), + GetConversationSummaryParams::RolloutPath { rollout_path } => { + let Some(local_thread_store) = self + .thread_store + .as_any() + .downcast_ref::() + else { + let error = JSONRPCErrorError { + code: INVALID_REQUEST_ERROR_CODE, + message: + "rollout path queries are only supported with the local thread store" + .to_string(), + data: None, + }; + return self.outgoing.send_error(request_id, error).await; + }; + + local_thread_store + .read_thread_by_rollout_path( + rollout_path.clone(), + /*include_archived*/ true, + /*include_history*/ false, + ) + .await + .map_err(|err| conversation_summary_rollout_path_read_error(&rollout_path, err)) + } + }; + + match read_result { + Ok(stored_thread) => { + let Some(summary) = summary_from_stored_thread(stored_thread, fallback_provider) + else { + let error = JSONRPCErrorError { + code: INTERNAL_ERROR_CODE, + message: + "failed to load conversation summary: thread is missing rollout path" + .to_string(), + data: None, + }; + self.outgoing.send_error(request_id, error).await; + return; + }; + let response = GetConversationSummaryResponse { summary }; + self.outgoing.send_response(request_id, response).await; + } + Err(error) => { + self.outgoing.send_error(request_id, error).await; + } + } + } + + async fn list_threads_common( + &self, + requested_page_size: usize, + cursor: Option, + sort_key: StoreThreadSortKey, + sort_direction: SortDirection, + filters: ThreadListFilters, + ) -> Result<(Vec, Option), JSONRPCErrorError> { + let ThreadListFilters { + model_providers, + source_kinds, + archived, + cwd_filters, + search_term, + use_state_db_only, + } = filters; + let mut cursor_obj = cursor; + let mut last_cursor = cursor_obj.clone(); + let mut remaining = requested_page_size; + let mut items = Vec::with_capacity(requested_page_size); + let mut next_cursor: Option = None; + + let model_provider_filter = match model_providers { + Some(providers) => { + if providers.is_empty() { + None + } else { + Some(providers) + } + } + None => Some(vec![self.config.model_provider_id.clone()]), + }; + let fallback_provider = self.config.model_provider_id.clone(); + let (allowed_sources_vec, source_kind_filter) = compute_source_filters(source_kinds); + let allowed_sources = allowed_sources_vec.as_slice(); + let store_sort_direction = match sort_direction { + SortDirection::Asc => StoreSortDirection::Asc, + SortDirection::Desc => StoreSortDirection::Desc, + }; + + while remaining > 0 { + let page_size = remaining.min(THREAD_LIST_MAX_LIMIT); + let page = self + .thread_store + .list_threads(StoreListThreadsParams { + page_size, + cursor: cursor_obj.clone(), + sort_key, + sort_direction: store_sort_direction, + allowed_sources: allowed_sources.to_vec(), + model_providers: model_provider_filter.clone(), + cwd_filters: cwd_filters.clone(), + archived, + search_term: search_term.clone(), + use_state_db_only, + }) + .await + .map_err(thread_store_list_error)?; + + let mut filtered = Vec::with_capacity(page.items.len()); + for it in page.items { + let Some(summary) = summary_from_stored_thread(it, fallback_provider.as_str()) + else { + continue; + }; + if source_kind_filter + .as_ref() + .is_none_or(|filter| source_kind_matches(&summary.source, filter)) + && cwd_filters.as_ref().is_none_or(|expected_cwds| { + expected_cwds.iter().any(|expected_cwd| { + path_utils::paths_match_after_normalization(&summary.cwd, expected_cwd) + }) + }) + { + filtered.push(summary); + if filtered.len() >= remaining { + break; + } + } + } + items.extend(filtered); + remaining = requested_page_size.saturating_sub(items.len()); + + next_cursor = page.next_cursor; + if remaining == 0 { + break; + } + + let Some(cursor_val) = next_cursor.clone() else { + break; + }; + // Break if our pagination would reuse the same cursor again; this avoids + // an infinite loop when filtering drops everything on the page. + if last_cursor.as_ref() == Some(&cursor_val) { + next_cursor = None; + break; + } + last_cursor = Some(cursor_val.clone()); + cursor_obj = Some(cursor_val); + } + + Ok((items, next_cursor)) + } }