diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index ae4f222a..ff1f9d5a 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -48,6 +48,7 @@ remembers doctrine; the branch remembers the blade. - Realtime conversation routes now live in `codex_message_processor/realtime_routes.rs`, and review-start orchestration now lives in `codex_message_processor/review_routes.rs`; the root processor is about 6,938 lines. - Model/catalog routes now live in `codex_message_processor/catalog_routes.rs`: model list, collaboration-mode list, experimental feature list, and mock experimental method. The root processor is about 6,738 lines. - Thread administrative mutation routes now live in `codex_message_processor/thread_admin_routes.rs`: archive/unarchive, elicitation counters, set-name, memory mode, memory reset, metadata update, rollback, compact, background terminal cleanup, shell command submission, and guardian approval. The root processor is about 5,750 lines. +- Thread read/list projection routes now live in `codex_message_processor/thread_read_routes.rs`: persisted/live thread listing, loaded-thread listing, `thread/read`, Epiphany state attachment for reads, rollout turn paging, and read-side rollout field hydration. The root processor is about 5,241 lines. - The May 2026 foundation migration in `notes/epiphany-ideal-architecture-rebuild-plan.md` is complete enough to defend. Read `notes/epiphany-architectural-teardown.md` before touching Epiphany control-plane code, but do not treat it as an open-ended excuse to keep shaving the altar. The rebuild moved scene projection, freshness derivation, pressure policy, targeted context shards, bounded graph traversal, job/progress view derivation, planning view derivation, reorientation resume/regather verdict policy, pure CRRC recommendation policy, fixed-lane coordinator decision policy, role-board projection policy, role/reorient result interpretation, role self-persistence review policy, and role/reorient acceptance bundle policy into `epiphany-core`; typed acceptance receipts replaced summary-string identity for live accept paths, runtime links are dual-written on launch, and result read-back now prefers runtime links. - Read `notes/epiphany-ideal-architecture-rebuild-plan.md` immediately after the teardown. It defines the smallest coherent replacement architecture: durable `EpiphanyState`, runtime-spine-owned `RuntimeState`, separate role memory/heartbeat state, `epiphany-core` as policy owner, app-server as adapter, Aquarium as reflector, typed acceptance receipts, and derived view lenses instead of a protocol verb zoo. - Do not continue Aquarium UI, bridge, Face, or dogfood expansion until the teardown has a source-grounded cleanup slice plan. Epiphany is the foundation; patches on patches are not a purification rite, they are how the altar becomes load-bearing garbage. diff --git a/state/ledgers.msgpack b/state/ledgers.msgpack index 536bb1cb..67fa5e25 100644 Binary files a/state/ledgers.msgpack and b/state/ledgers.msgpack differ diff --git a/state/map.yaml b/state/map.yaml index ffc49aae..16b8f9a4 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -286,11 +286,10 @@ rejected_paths: current_status: summary: The Codex starvation pass has five typing receipts, fifty-two processor/foundation cuts, a real physical extraction in `epiphany-codex-bridge`, a root `epiphany-state-model` crate for durable Epiphany state, a source-grounded Codex auth spine inventory, the native `epiphany-openai-adapter` document boundary, a buildable Codex auth/model wrapper in `epiphany-openai-codex-spine`, the first native runtime caller in `epiphany-openai-runtime`, native heartbeat launch planning/job opening in `epiphany-core`, native state-update contract/validation/application in `epiphany-core`, native exact retrieval path matching in `epiphany-core`, and the route-facing mutation/view/coordinator projection services in `epiphany-codex-bridge`. `codex-protocol` no longer defines the durable `Epiphany*` state structs; it re-exports the native model for legacy rollout/API compatibility while `epiphany-core` imports the native model directly. `epiphany-core` now has no direct vendored Codex dependency: pressure derivation accepts native `EpiphanyTokenUsageSnapshot`, Codex rollout replay moved to `codex-core::epiphany_rollout`, and exact retrieval search uses Epiphany-owned workspace walking/path scoring instead of `codex-file-search`. The bridge owns Codex-facing Epiphany adapters for scene, jobs, context/planning/graph query, retrieval/indexing projection, result projection, runtime-spine result adaptation, launch/prompt doctrine, mutation document mechanics, mutation service update/promote/role-accept/reorient-accept/role-launch/reorient-launch/job-launch/job-interrupt application, live-state projection, pressure/pre-compaction rendering, freshness/reorientation projection, CRRC/coordinator/role-board projection, watcher invalidation adaptation, full `thread/epiphany/view` response construction, freshness/context/graph-query response construction, role/reorient result response construction, distill/propose patch-response construction, coordinator status/view construction, and acceptance/evidence signal helpers. Vendored app-server no longer retains root Epiphany modules; `codex_message_processor.rs` still contains route dispatch, JSON-RPC response shaping, watcher registration/snapshotting calls, and notification emission, but its root no longer imports the evacuated Epiphany policy/protocol registry or owns the watcher invalidation manager. Codex product surfaces for apps, skills, plugins, marketplace, and external agent config import are cut from the live app-server/protocol path: list routes return empty lists only where compatibility shells still exist, mutation/detail routes report disabled where they survive, `externalAgentConfig/*` routes and import-completed notification are gone, helper/import modules and stale implementation-shape tests were deleted, runtime feature enablement no longer supports `apps` or `plugins`, config writes no longer emit plugin-toggle analytics, plugin/skill cache invalidation on config writes is gone, plugin startup warmups and app-list refreshes were removed, and app-server no longer directly depends on `codex-plugin`, `codex-chatgpt`, or `codex-core-plugins`. `Config::to_mcp_config` now keeps user-declared MCP servers but no longer accepts `PluginsManager`, folds plugin MCP servers into the runtime, enables Codex apps, enables skill MCP dependency install, or advertises plugin capability summaries; `TurnContext::apps_enabled()` now returns false so model turns cannot awaken Codex Apps through ChatGPT auth or feature flags; session startup and per-turn context no longer load skills, turns receive an empty `SkillLoadOutcome`, internal `Op::ListSkills` is gone from `codex-protocol` / `codex-core` / TUI command routing, the Codex skill watcher has been collapsed to a no-op marker with no file watcher, skill-root registration, cache-clear task, or `SkillsUpdateAvailable` session listener, model turns no longer run skill mention collection, skill env-var prompts, skill MCP dependency installation, skill instruction injection, or implicit skill invocation analytics from shell tools, and the turn prompt/router path no longer loads plugin/app capability summaries, injects available-skill/available-plugin/plugin instruction fragments, parses plugin/app mentions, records app/plugin invocation telemetry, or filters MCP tools through Codex app connectors. Skill mutation/list/notification is gone from the app-server edge too: `skills/list`, `skills/config/write`, `skills/changed`, `Op::ListSkills`, `EventMsg::ListSkillsResponse`, `EventMsg::SkillsUpdateAvailable`, the app-server handlers, TUI slash/startup/list/manage-skill surfaces, generated schema entries, core skill-list suite, and fake live-reload/developer-message tests were deleted. Skill mention plumbing is also gone from the live TUI/protocol edge: no `UserInput::Skill` variant, app-server schema branch, TUI skill popup, `skills_all` cache, composer `set_skill_mentions`, skill mention insertion, `skill://` / `SKILL.md` history decoding, or stale skill popup snapshot remains. The `codex-core-skills` crate, core `skills` re-export shim, no-op `skills_watcher`, session `SkillsManager` service, and per-turn `TurnSkillsContext` are deleted; core/protocol no longer keep legacy `` or `` prompt compatibility, memory usage classification, snapshot normalization, or cleanup paths. Remaining product-shaped skill residue is also cut from memory templates, app-server docs/tests/warnings, TUI tooltips/placeholders/warnings/comments, app-server v1 docs, agent-tool schema text, stale `UserInput::Skill` test arms, and obsolete feature flags/schema entries. App-instruction prompt compatibility is cut too: no `APPS_INSTRUCTIONS` protocol constants, no `` snapshot canonicalization, and MCP fixture tests now use neutral `demo_mcp` / `calendar_mcp` names instead of `codex_apps` / `/api/codex/apps` branding. The TUI app connector product path is also deleted: no `/apps` slash command, app-link view, connector AppEvent variants, app connector cache/popup/refresh loop, app mention insertion, `app://` mention codec support, stale app-popup tests, or TUI browser-opening shim remains. The `app/list` route and `app/list/updated` notification are gone from app-server protocol, generated schemas, app-server dispatch, test helpers, and docs; `AppInfo`/app-list metadata schema structs are deleted; the orphaned `codex-connectors` crate and `core::connectors` shim are deleted from the workspace; Codex app analytics no longer tracks `codex_app_mentioned` / `codex_app_used` facts, events, reducer arms, client APIs, or dedupe cache state; `[apps.*]`, `Feature::Apps`, `include_apps_instructions`, app config/read schemas, app managed requirements, app prompt tests, and app file-MCP tests are gone; `Feature::Plugins`, `Feature::RemotePlugin`, `plugin://` structured mentions, plugin instruction tags, and stale plugin popup snapshots are gone too; `[plugins]` / `[marketplaces]`, `PluginConfig`, `MarketplaceConfig`, marketplace edit helpers, stale plugin/external-agent README route claims, and generated schema entries are removed from config; and the bundled `codex-skills` sample-skill crate plus embedded imagegen/openai-docs/plugin-creator/skill-creator/skill-installer payloads are deleted from the workspace. The vendored `vendor/codex/sdk` subtree is deleted as non-spine app-server client/release scaffolding, including stale generated Python SDK route types for removed product surfaces; the root pnpm workspace no longer lists `sdk/typescript`. Live MCP tool approval is named `McpToolApproval`, not `AppToolApproval`, and remains under MCP server config; live structured user-input mentions are MCP-shaped only. `codex_message_processor.rs` is about 9,482 lines; root `message_processor.rs` is about 1,292 lines after the external-agent/config-product cuts; `epiphany_read_routes.rs` is about 579 lines after the full view and read-response extractions; `epiphany_mutation_routes.rs` is about 694 lines after route-load/notification helper cuts and product-surface starvation; the old vendored `epiphany_state_helpers.rs` rollout-state loader is deleted and its remaining function lives in `epiphany-codex-bridge::state`; and `codex_thread.rs` is about 1,425 lines after state-update law moved native. `epiphany-openai-adapter` defines CultCache `DatabaseEntry` request/event/receipt/auth-status documents with no Codex app-server or generic JSON dependency. `epiphany-openai-codex-spine` owns the native OpenAI model transport path: it exposes `default_codex_home` and `auth_manager`, projects `epiphany-openai-auth-spine` `AuthManager` / `CodexAuth` into typed `EpiphanyOpenAiAdapterStatus`, builds a local serializable OpenAI Responses request body instead of Codex protocol request/item/config types, opens HTTP Responses streams through `codex-client`, parses Responses SSE frames into typed stream events and receipts, and no longer directly or transitively depends on Codex's Responses client, model-provider stack, `codex-protocol`, `codex-api`, `codex-model-provider`, or `codex-model-provider-info`; it intentionally reaches retained vendored `codex-login` only through `epiphany-openai-auth-spine`. Auth identity, credential storage, and ChatGPT token refresh remain owned by the Codex-compatible auth organ; external bearer command auth and agent identity remain retained Codex auth capabilities to expose only if a typed Epiphany surface needs them. `epiphany-openai-runtime` no longer depends directly on `codex-login`; it writes OpenAI status/request/stream-event/receipt records plus runtime sessions/jobs/results/events into `epiphany-core::runtime_spine_cache`, can emit the assistant text artifact for legacy result consumers, is now the default executor for startup-only repo birth specialists, and now has `run-worker` to load a durable `EpiphanyRuntimeWorkerLaunchRequest`, call the typed OpenAI runtime through the spine, persist typed `EpiphanyRuntimeRoleWorkerResult` / `EpiphanyRuntimeReorientWorkerResult` documents, and complete the original heartbeat/specialist runtime job without Codex worker execution; `codex exec` survives only as explicit fallback in repo-birth startup. `epiphany-mvp-coordinator` now invokes `epiphany-openai-runtime run-worker` for launched runtime job ids and no longer creates a second coordinator-owned shadow runtime job/result as a compensator. `epiphany-core::plan_runtime_spine_heartbeat_launch` validates launch contracts/conflicts and projects durable job bindings/runtime links; `epiphany-core::open_runtime_spine_heartbeat_job` owns the initialize-runtime/ensure-session/create-job sequence and now persists `EpiphanyRuntimeWorkerLaunchRequest` CultCache records keyed by runtime job id, with indexed kind/contract fields plus MessagePack-encoded typed launch documents instead of discarding work intent after queue receipt creation; `roleResult` and `reorientResult` now require typed worker result documents; generic job summaries are lifecycle receipts, not reviewable findings; `epiphany-core::EpiphanyStateUpdate`, `epiphany_state_update_validation_errors`, and `apply_epiphany_state_update` own state-update document shape and mutation law while `codex-core` calls them for compatibility. Vendored `CodexThread::epiphany_launch_job` is now mostly revision check, persistence validation, launch request handoff, and rollout/session writeback for the compatibility route. Vendored `codex-core` no longer has `epiphany_distillation`, `epiphany_promotion`, `epiphany_proposal`, or `epiphany_retrieval` husk modules; it re-exports native surfaces from `epiphany-core` directly, and now owns only the real Codex rollout adapter because it is a host seam. `epiphany-runtime-spine` advertises OpenAI adapter status/model-request/stream-event/receipt CultNet schemas and mutation contracts, so the native caller now has a real storage route instead of only JSON-RPC. MCP itself is correctly JSON at its protocol edge; the target is typed Epiphany intent/result/receipt wrappers around MCP, not replacing MCP JSON. The current OpenAI request path no longer pulls `codex-api`; `codex-login` remains deliberately sealed behind `epiphany-openai-auth-spine`, and the next spine decision is another Epiphany-in-vendor evacuation surface unless a concrete typed auth-status/runtime need requires exposing external bearer or agent-identity auth. latest_cut: > - Thread administrative mutation routes moved out of root + Thread read/list projection routes moved out of root `codex_message_processor.rs` into - `codex_message_processor/thread_admin_routes.rs`. Archive/unarchive, - elicitation counters, set-name, memory mode/reset, metadata update, - rollback, compact, background terminal cleanup, shell command submission, - and guardian approval are split from loaded-thread orchestration; the root - processor is about 5,750 lines after the cut. + `codex_message_processor/thread_read_routes.rs`. Persisted/live thread + listing, loaded-thread listing, `thread/read`, Epiphany state attachment, + read-side rollout hydration, and turn paging are split from lifecycle + orchestration; the root processor is about 5,241 lines after the cut. next_action: Continue the Codex starvation campaign at the next Epiphany-in-vendor evacuation surface; keep vendored Codex code as host routing/lifecycle glue only, and do not cut the retained `codex-login` auth organ unless replacing it with an explicitly permitted OpenAI path rather than a lookalike. diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs index 0a56e14c..b3766c15 100644 --- a/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor.rs @@ -8,6 +8,7 @@ mod mcp_routes; mod realtime_routes; mod review_routes; mod thread_admin_routes; +mod thread_read_routes; use self::auth_routes::ActiveLogin; pub(crate) use self::epiphany_automation::maybe_run_epiphany_coordinator_automation_for_turn_boundary; @@ -1616,516 +1617,6 @@ impl CodexMessageProcessor { } } - async fn thread_list(&self, request_id: ConnectionRequestId, params: ThreadListParams) { - let ThreadListParams { - cursor, - limit, - sort_key, - sort_direction, - model_providers, - source_kinds, - archived, - cwd, - use_state_db_only, - search_term, - } = params; - let cwd_filters = match normalize_thread_list_cwd_filters(cwd) { - Ok(cwd_filters) => cwd_filters, - Err(error) => { - self.outgoing.send_error(request_id, error).await; - return; - } - }; - - let requested_page_size = limit - .map(|value| value as usize) - .unwrap_or(THREAD_LIST_DEFAULT_LIMIT) - .clamp(1, THREAD_LIST_MAX_LIMIT); - let store_sort_key = match sort_key.unwrap_or(ThreadSortKey::CreatedAt) { - ThreadSortKey::CreatedAt => StoreThreadSortKey::CreatedAt, - ThreadSortKey::UpdatedAt => StoreThreadSortKey::UpdatedAt, - }; - let sort_direction = sort_direction.unwrap_or(SortDirection::Desc); - let list_result = self - .list_threads_common( - requested_page_size, - cursor, - store_sort_key, - sort_direction, - ThreadListFilters { - model_providers, - source_kinds, - archived: archived.unwrap_or(false), - cwd_filters, - search_term, - use_state_db_only, - }, - ) - .await; - let (summaries, next_cursor) = match list_result { - Ok(r) => r, - Err(error) => { - self.outgoing.send_error(request_id, error).await; - return; - } - }; - let backwards_cursor = summaries.first().and_then(|summary| { - thread_backwards_cursor_for_sort_key(summary, store_sort_key, sort_direction) - }); - let mut threads = Vec::with_capacity(summaries.len()); - let mut thread_ids = HashSet::with_capacity(summaries.len()); - let mut status_ids = Vec::with_capacity(summaries.len()); - - for summary in summaries { - let conversation_id = summary.conversation_id; - thread_ids.insert(conversation_id); - - let thread = summary_to_thread(summary, &self.config.cwd); - status_ids.push(thread.id.clone()); - threads.push((conversation_id, thread)); - } - - let names = thread_titles_by_ids(&self.config, &thread_ids).await; - - let statuses = self - .thread_watch_manager - .loaded_statuses_for_threads(status_ids) - .await; - - let data: Vec<_> = threads - .into_iter() - .map(|(conversation_id, mut thread)| { - if let Some(title) = names.get(&conversation_id).cloned() { - set_thread_name_from_title(&mut thread, title); - } - if let Some(status) = statuses.get(&thread.id) { - thread.status = status.clone(); - } - thread - }) - .collect(); - let response = ThreadListResponse { - data, - next_cursor, - backwards_cursor, - }; - self.outgoing.send_response(request_id, response).await; - } - - async fn thread_loaded_list( - &self, - request_id: ConnectionRequestId, - params: ThreadLoadedListParams, - ) { - let ThreadLoadedListParams { cursor, limit } = params; - let mut data = self - .thread_manager - .list_thread_ids() - .await - .into_iter() - .map(|thread_id| thread_id.to_string()) - .collect::>(); - - if data.is_empty() { - let response = ThreadLoadedListResponse { - data, - next_cursor: None, - }; - self.outgoing.send_response(request_id, response).await; - return; - } - - data.sort(); - let total = data.len(); - let start = match cursor { - Some(cursor) => { - let cursor = match ThreadId::from_string(&cursor) { - Ok(id) => id.to_string(), - Err(_) => { - let error = JSONRPCErrorError { - code: INVALID_REQUEST_ERROR_CODE, - message: format!("invalid cursor: {cursor}"), - data: None, - }; - self.outgoing.send_error(request_id, error).await; - return; - } - }; - match data.binary_search(&cursor) { - Ok(idx) => idx + 1, - Err(idx) => idx, - } - } - None => 0, - }; - - let effective_limit = limit.unwrap_or(total as u32).max(1) as usize; - let end = start.saturating_add(effective_limit).min(total); - let page = data[start..end].to_vec(); - let next_cursor = page.last().filter(|_| end < total).cloned(); - - let response = ThreadLoadedListResponse { - data: page, - next_cursor, - }; - self.outgoing.send_response(request_id, response).await; - } - - async fn thread_read(&self, request_id: ConnectionRequestId, params: ThreadReadParams) { - let ThreadReadParams { - thread_id, - include_turns, - } = params; - - let thread_uuid = match ThreadId::from_string(&thread_id) { - Ok(id) => id, - Err(err) => { - self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) - .await; - return; - } - }; - - let thread = match self.read_thread_view(thread_uuid, include_turns).await { - Ok(thread) => thread, - Err(ThreadReadViewError::InvalidRequest(message)) => { - self.send_invalid_request_error(request_id, message).await; - return; - } - Err(ThreadReadViewError::Internal(message)) => { - self.send_internal_error(request_id, message).await; - return; - } - }; - let response = ThreadReadResponse { thread }; - self.outgoing.send_response(request_id, response).await; - } - - /// Builds the API view for `thread/read` from persisted metadata plus optional live state. - async fn read_thread_view( - &self, - thread_id: ThreadId, - include_turns: bool, - ) -> Result { - let loaded_thread = self.load_live_thread_for_read(thread_id).await; - let mut thread = if let Some(thread) = self - .load_persisted_thread_for_read(thread_id, include_turns) - .await? - { - thread - } else if let Some(thread) = self - .load_live_thread_view(thread_id, include_turns, loaded_thread.as_ref()) - .await? - { - thread - } else { - return Err(ThreadReadViewError::InvalidRequest(format!( - "thread not loaded: {thread_id}" - ))); - }; - - self.apply_thread_read_epiphany_state(&mut thread, loaded_thread.as_ref()) - .await?; - - let has_live_in_progress_turn = if let Some(loaded_thread) = loaded_thread.as_ref() { - matches!(loaded_thread.agent_status().await, AgentStatus::Running) - } else { - false - }; - - let thread_status = self - .thread_watch_manager - .loaded_status_for_thread(&thread.id) - .await; - - set_thread_status_and_interrupt_stale_turns( - &mut thread, - thread_status, - has_live_in_progress_turn, - ); - Ok(thread) - } - - async fn load_live_thread_for_read(&self, thread_id: ThreadId) -> Option> { - self.thread_manager.get_thread(thread_id).await.ok() - } - - async fn load_persisted_thread_for_read( - &self, - thread_id: ThreadId, - include_turns: bool, - ) -> Result, ThreadReadViewError> { - let fallback_provider = self.config.model_provider_id.as_str(); - match self - .thread_store - .read_thread(StoreReadThreadParams { - thread_id, - include_archived: true, - include_history: include_turns, - }) - .await - { - Ok(stored_thread) => { - let (mut thread, history) = - thread_from_stored_thread(stored_thread, fallback_provider, &self.config.cwd); - if include_turns && let Some(history) = history { - thread.turns = build_turns_from_rollout_items(&history.items); - } - Ok(Some(thread)) - } - Err(ThreadStoreError::InvalidRequest { message }) - if message == format!("no rollout found for thread id {thread_id}") => - { - Ok(None) - } - Err(ThreadStoreError::ThreadNotFound { - thread_id: missing_thread_id, - }) if missing_thread_id == thread_id => Ok(None), - Err(ThreadStoreError::InvalidRequest { message }) => { - Err(ThreadReadViewError::InvalidRequest(message)) - } - Err(err) => Err(ThreadReadViewError::Internal(format!( - "failed to read thread: {err}" - ))), - } - } - - async fn load_live_thread_view( - &self, - thread_id: ThreadId, - include_turns: bool, - loaded_thread: Option<&Arc>, - ) -> Result, ThreadReadViewError> { - let Some(thread) = loaded_thread else { - return Ok(None); - }; - let config_snapshot = thread.config_snapshot().await; - let loaded_rollout_path = thread.rollout_path(); - if include_turns && loaded_rollout_path.is_none() { - return Err(ThreadReadViewError::InvalidRequest( - "ephemeral threads do not support includeTurns".to_string(), - )); - } - let mut thread = - build_thread_from_snapshot(thread_id, &config_snapshot, loaded_rollout_path.clone()); - self.apply_thread_read_rollout_fields( - thread_id, - &mut thread, - loaded_rollout_path.as_deref(), - include_turns, - ) - .await?; - Ok(Some(thread)) - } - - async fn apply_thread_read_epiphany_state( - &self, - thread: &mut Thread, - loaded_thread: Option<&Arc>, - ) -> Result<(), ThreadReadViewError> { - if let Some(loaded_thread) = loaded_thread { - thread.epiphany_state = live_thread_epiphany_state(loaded_thread).await; - return Ok(()); - } - - let Some(rollout_path) = thread.path.as_deref() else { - thread.epiphany_state = None; - return Ok(()); - }; - - thread.epiphany_state = load_epiphany_state_from_rollout_path(rollout_path) - .await - .map_err(ThreadReadViewError::Internal)?; - Ok(()) - } - - async fn apply_thread_read_rollout_fields( - &self, - thread_id: ThreadId, - thread: &mut Thread, - rollout_path: Option<&Path>, - include_turns: bool, - ) -> Result<(), ThreadReadViewError> { - if thread.forked_from_id.is_none() - && let Some(rollout_path) = rollout_path - { - thread.forked_from_id = forked_from_id_from_rollout(rollout_path).await; - } - self.attach_thread_name(thread_id, thread).await; - - if include_turns && let Some(rollout_path) = rollout_path { - match read_rollout_items_from_rollout(rollout_path).await { - Ok(items) => { - thread.turns = build_turns_from_rollout_items(&items); - } - Err(err) if err.kind() == std::io::ErrorKind::NotFound => { - return Err(ThreadReadViewError::InvalidRequest(format!( - "thread {thread_id} is not materialized yet; includeTurns is unavailable before first user message" - ))); - } - Err(err) => { - return Err(ThreadReadViewError::Internal(format!( - "failed to load rollout `{}` for thread {thread_id}: {err}", - rollout_path.display() - ))); - } - } - } - - Ok(()) - } - - async fn thread_turns_list( - &self, - request_id: ConnectionRequestId, - params: ThreadTurnsListParams, - ) { - let ThreadTurnsListParams { - thread_id, - cursor, - limit, - sort_direction, - } = params; - - let thread_uuid = match ThreadId::from_string(&thread_id) { - Ok(id) => id, - Err(err) => { - self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) - .await; - return; - } - }; - - let state_db_ctx = get_state_db(&self.config).await; - let mut rollout_path = self - .resolve_rollout_path(thread_uuid, state_db_ctx.as_ref()) - .await; - if rollout_path.is_none() { - rollout_path = - match find_thread_path_by_id_str(&self.config.codex_home, &thread_uuid.to_string()) - .await - { - Ok(Some(path)) => Some(path), - Ok(None) => match find_archived_thread_path_by_id_str( - &self.config.codex_home, - &thread_uuid.to_string(), - ) - .await - { - Ok(path) => path, - Err(err) => { - self.send_invalid_request_error( - request_id, - format!("failed to locate archived thread id {thread_uuid}: {err}"), - ) - .await; - return; - } - }, - Err(err) => { - self.send_invalid_request_error( - request_id, - format!("failed to locate thread id {thread_uuid}: {err}"), - ) - .await; - return; - } - }; - } - - if rollout_path.is_none() { - match self.thread_manager.get_thread(thread_uuid).await { - Ok(thread) => { - rollout_path = thread.rollout_path(); - if rollout_path.is_none() { - self.send_invalid_request_error( - request_id, - "ephemeral threads do not support thread/turns/list".to_string(), - ) - .await; - return; - } - } - Err(_) => { - self.send_invalid_request_error( - request_id, - format!("thread not loaded: {thread_uuid}"), - ) - .await; - return; - } - } - } - - let Some(rollout_path) = rollout_path.as_ref() else { - self.send_internal_error( - request_id, - format!("failed to locate rollout for thread {thread_uuid}"), - ) - .await; - return; - }; - - match read_rollout_items_from_rollout(rollout_path).await { - Ok(items) => { - // This API optimizes network transfer by letting clients page through a - // thread's turns incrementally, but it still replays the entire rollout on - // every request. Rollback and compaction events can change earlier turns, so - // the server has to rebuild the full turn list until turn metadata is indexed - // separately. - let has_live_in_progress_turn = - match self.thread_manager.get_thread(thread_uuid).await { - Ok(thread) => matches!(thread.agent_status().await, AgentStatus::Running), - Err(_) => false, - }; - let turns = reconstruct_thread_turns_from_rollout_items( - &items, - self.thread_watch_manager - .loaded_status_for_thread(&thread_uuid.to_string()) - .await, - has_live_in_progress_turn, - ); - let page = match paginate_thread_turns( - turns, - cursor.as_deref(), - limit, - sort_direction.unwrap_or(SortDirection::Desc), - ) { - Ok(page) => page, - Err(error) => { - self.outgoing.send_error(request_id, error).await; - return; - } - }; - let response = ThreadTurnsListResponse { - data: page.turns, - next_cursor: page.next_cursor, - backwards_cursor: page.backwards_cursor, - }; - self.outgoing.send_response(request_id, response).await; - } - Err(err) if err.kind() == std::io::ErrorKind::NotFound => { - self.send_invalid_request_error( - request_id, - format!( - "thread {thread_uuid} is not materialized yet; thread/turns/list is unavailable before first user message" - ), - ) - .await; - } - Err(err) => { - self.send_internal_error( - request_id, - format!( - "failed to load rollout `{}` for thread {thread_uuid}: {err}", - rollout_path.display() - ), - ) - .await; - } - } - } - pub(crate) fn thread_created_receiver(&self) -> broadcast::Receiver { self.thread_manager.subscribe_thread_created() } diff --git a/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs new file mode 100644 index 00000000..98f5793f --- /dev/null +++ b/vendor/codex/codex-rs/app-server/src/codex_message_processor/thread_read_routes.rs @@ -0,0 +1,521 @@ +use super::*; + +impl CodexMessageProcessor { + pub(super) async fn thread_list( + &self, + request_id: ConnectionRequestId, + params: ThreadListParams, + ) { + let ThreadListParams { + cursor, + limit, + sort_key, + sort_direction, + model_providers, + source_kinds, + archived, + cwd, + use_state_db_only, + search_term, + } = params; + let cwd_filters = match normalize_thread_list_cwd_filters(cwd) { + Ok(cwd_filters) => cwd_filters, + Err(error) => { + self.outgoing.send_error(request_id, error).await; + return; + } + }; + + let requested_page_size = limit + .map(|value| value as usize) + .unwrap_or(THREAD_LIST_DEFAULT_LIMIT) + .clamp(1, THREAD_LIST_MAX_LIMIT); + let store_sort_key = match sort_key.unwrap_or(ThreadSortKey::CreatedAt) { + ThreadSortKey::CreatedAt => StoreThreadSortKey::CreatedAt, + ThreadSortKey::UpdatedAt => StoreThreadSortKey::UpdatedAt, + }; + let sort_direction = sort_direction.unwrap_or(SortDirection::Desc); + let list_result = self + .list_threads_common( + requested_page_size, + cursor, + store_sort_key, + sort_direction, + ThreadListFilters { + model_providers, + source_kinds, + archived: archived.unwrap_or(false), + cwd_filters, + search_term, + use_state_db_only, + }, + ) + .await; + let (summaries, next_cursor) = match list_result { + Ok(r) => r, + Err(error) => { + self.outgoing.send_error(request_id, error).await; + return; + } + }; + let backwards_cursor = summaries.first().and_then(|summary| { + thread_backwards_cursor_for_sort_key(summary, store_sort_key, sort_direction) + }); + let mut threads = Vec::with_capacity(summaries.len()); + let mut thread_ids = HashSet::with_capacity(summaries.len()); + let mut status_ids = Vec::with_capacity(summaries.len()); + + for summary in summaries { + let conversation_id = summary.conversation_id; + thread_ids.insert(conversation_id); + + let thread = summary_to_thread(summary, &self.config.cwd); + status_ids.push(thread.id.clone()); + threads.push((conversation_id, thread)); + } + + let names = thread_titles_by_ids(&self.config, &thread_ids).await; + + let statuses = self + .thread_watch_manager + .loaded_statuses_for_threads(status_ids) + .await; + + let data: Vec<_> = threads + .into_iter() + .map(|(conversation_id, mut thread)| { + if let Some(title) = names.get(&conversation_id).cloned() { + set_thread_name_from_title(&mut thread, title); + } + if let Some(status) = statuses.get(&thread.id) { + thread.status = status.clone(); + } + thread + }) + .collect(); + let response = ThreadListResponse { + data, + next_cursor, + backwards_cursor, + }; + self.outgoing.send_response(request_id, response).await; + } + + pub(super) async fn thread_loaded_list( + &self, + request_id: ConnectionRequestId, + params: ThreadLoadedListParams, + ) { + let ThreadLoadedListParams { cursor, limit } = params; + let mut data = self + .thread_manager + .list_thread_ids() + .await + .into_iter() + .map(|thread_id| thread_id.to_string()) + .collect::>(); + + if data.is_empty() { + let response = ThreadLoadedListResponse { + data, + next_cursor: None, + }; + self.outgoing.send_response(request_id, response).await; + return; + } + + data.sort(); + let total = data.len(); + let start = match cursor { + Some(cursor) => { + let cursor = match ThreadId::from_string(&cursor) { + Ok(id) => id.to_string(), + Err(_) => { + let error = JSONRPCErrorError { + code: INVALID_REQUEST_ERROR_CODE, + message: format!("invalid cursor: {cursor}"), + data: None, + }; + self.outgoing.send_error(request_id, error).await; + return; + } + }; + match data.binary_search(&cursor) { + Ok(idx) => idx + 1, + Err(idx) => idx, + } + } + None => 0, + }; + + let effective_limit = limit.unwrap_or(total as u32).max(1) as usize; + let end = start.saturating_add(effective_limit).min(total); + let page = data[start..end].to_vec(); + let next_cursor = page.last().filter(|_| end < total).cloned(); + + let response = ThreadLoadedListResponse { + data: page, + next_cursor, + }; + self.outgoing.send_response(request_id, response).await; + } + + pub(super) async fn thread_read( + &self, + request_id: ConnectionRequestId, + params: ThreadReadParams, + ) { + let ThreadReadParams { + thread_id, + include_turns, + } = params; + + let thread_uuid = match ThreadId::from_string(&thread_id) { + Ok(id) => id, + Err(err) => { + self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) + .await; + return; + } + }; + + let thread = match self.read_thread_view(thread_uuid, include_turns).await { + Ok(thread) => thread, + Err(ThreadReadViewError::InvalidRequest(message)) => { + self.send_invalid_request_error(request_id, message).await; + return; + } + Err(ThreadReadViewError::Internal(message)) => { + self.send_internal_error(request_id, message).await; + return; + } + }; + let response = ThreadReadResponse { thread }; + self.outgoing.send_response(request_id, response).await; + } + + /// Builds the API view for `thread/read` from persisted metadata plus optional live state. + pub(super) async fn read_thread_view( + &self, + thread_id: ThreadId, + include_turns: bool, + ) -> Result { + let loaded_thread = self.load_live_thread_for_read(thread_id).await; + let mut thread = if let Some(thread) = self + .load_persisted_thread_for_read(thread_id, include_turns) + .await? + { + thread + } else if let Some(thread) = self + .load_live_thread_view(thread_id, include_turns, loaded_thread.as_ref()) + .await? + { + thread + } else { + return Err(ThreadReadViewError::InvalidRequest(format!( + "thread not loaded: {thread_id}" + ))); + }; + + self.apply_thread_read_epiphany_state(&mut thread, loaded_thread.as_ref()) + .await?; + + let has_live_in_progress_turn = if let Some(loaded_thread) = loaded_thread.as_ref() { + matches!(loaded_thread.agent_status().await, AgentStatus::Running) + } else { + false + }; + + let thread_status = self + .thread_watch_manager + .loaded_status_for_thread(&thread.id) + .await; + + set_thread_status_and_interrupt_stale_turns( + &mut thread, + thread_status, + has_live_in_progress_turn, + ); + Ok(thread) + } + + async fn load_live_thread_for_read(&self, thread_id: ThreadId) -> Option> { + self.thread_manager.get_thread(thread_id).await.ok() + } + + async fn load_persisted_thread_for_read( + &self, + thread_id: ThreadId, + include_turns: bool, + ) -> Result, ThreadReadViewError> { + let fallback_provider = self.config.model_provider_id.as_str(); + match self + .thread_store + .read_thread(StoreReadThreadParams { + thread_id, + include_archived: true, + include_history: include_turns, + }) + .await + { + Ok(stored_thread) => { + let (mut thread, history) = + thread_from_stored_thread(stored_thread, fallback_provider, &self.config.cwd); + if include_turns && let Some(history) = history { + thread.turns = build_turns_from_rollout_items(&history.items); + } + Ok(Some(thread)) + } + Err(ThreadStoreError::InvalidRequest { message }) + if message == format!("no rollout found for thread id {thread_id}") => + { + Ok(None) + } + Err(ThreadStoreError::ThreadNotFound { + thread_id: missing_thread_id, + }) if missing_thread_id == thread_id => Ok(None), + Err(ThreadStoreError::InvalidRequest { message }) => { + Err(ThreadReadViewError::InvalidRequest(message)) + } + Err(err) => Err(ThreadReadViewError::Internal(format!( + "failed to read thread: {err}" + ))), + } + } + + async fn load_live_thread_view( + &self, + thread_id: ThreadId, + include_turns: bool, + loaded_thread: Option<&Arc>, + ) -> Result, ThreadReadViewError> { + let Some(thread) = loaded_thread else { + return Ok(None); + }; + let config_snapshot = thread.config_snapshot().await; + let loaded_rollout_path = thread.rollout_path(); + if include_turns && loaded_rollout_path.is_none() { + return Err(ThreadReadViewError::InvalidRequest( + "ephemeral threads do not support includeTurns".to_string(), + )); + } + let mut thread = + build_thread_from_snapshot(thread_id, &config_snapshot, loaded_rollout_path.clone()); + self.apply_thread_read_rollout_fields( + thread_id, + &mut thread, + loaded_rollout_path.as_deref(), + include_turns, + ) + .await?; + Ok(Some(thread)) + } + + async fn apply_thread_read_epiphany_state( + &self, + thread: &mut Thread, + loaded_thread: Option<&Arc>, + ) -> Result<(), ThreadReadViewError> { + if let Some(loaded_thread) = loaded_thread { + thread.epiphany_state = live_thread_epiphany_state(loaded_thread).await; + return Ok(()); + } + + let Some(rollout_path) = thread.path.as_deref() else { + thread.epiphany_state = None; + return Ok(()); + }; + + thread.epiphany_state = load_epiphany_state_from_rollout_path(rollout_path) + .await + .map_err(ThreadReadViewError::Internal)?; + Ok(()) + } + + async fn apply_thread_read_rollout_fields( + &self, + thread_id: ThreadId, + thread: &mut Thread, + rollout_path: Option<&Path>, + include_turns: bool, + ) -> Result<(), ThreadReadViewError> { + if thread.forked_from_id.is_none() + && let Some(rollout_path) = rollout_path + { + thread.forked_from_id = forked_from_id_from_rollout(rollout_path).await; + } + self.attach_thread_name(thread_id, thread).await; + + if include_turns && let Some(rollout_path) = rollout_path { + match read_rollout_items_from_rollout(rollout_path).await { + Ok(items) => { + thread.turns = build_turns_from_rollout_items(&items); + } + Err(err) if err.kind() == std::io::ErrorKind::NotFound => { + return Err(ThreadReadViewError::InvalidRequest(format!( + "thread {thread_id} is not materialized yet; includeTurns is unavailable before first user message" + ))); + } + Err(err) => { + return Err(ThreadReadViewError::Internal(format!( + "failed to load rollout `{}` for thread {thread_id}: {err}", + rollout_path.display() + ))); + } + } + } + + Ok(()) + } + + pub(super) async fn thread_turns_list( + &self, + request_id: ConnectionRequestId, + params: ThreadTurnsListParams, + ) { + let ThreadTurnsListParams { + thread_id, + cursor, + limit, + sort_direction, + } = params; + + let thread_uuid = match ThreadId::from_string(&thread_id) { + Ok(id) => id, + Err(err) => { + self.send_invalid_request_error(request_id, format!("invalid thread id: {err}")) + .await; + return; + } + }; + + let state_db_ctx = get_state_db(&self.config).await; + let mut rollout_path = self + .resolve_rollout_path(thread_uuid, state_db_ctx.as_ref()) + .await; + if rollout_path.is_none() { + rollout_path = + match find_thread_path_by_id_str(&self.config.codex_home, &thread_uuid.to_string()) + .await + { + Ok(Some(path)) => Some(path), + Ok(None) => match find_archived_thread_path_by_id_str( + &self.config.codex_home, + &thread_uuid.to_string(), + ) + .await + { + Ok(path) => path, + Err(err) => { + self.send_invalid_request_error( + request_id, + format!("failed to locate archived thread id {thread_uuid}: {err}"), + ) + .await; + return; + } + }, + Err(err) => { + self.send_invalid_request_error( + request_id, + format!("failed to locate thread id {thread_uuid}: {err}"), + ) + .await; + return; + } + }; + } + + if rollout_path.is_none() { + match self.thread_manager.get_thread(thread_uuid).await { + Ok(thread) => { + rollout_path = thread.rollout_path(); + if rollout_path.is_none() { + self.send_invalid_request_error( + request_id, + "ephemeral threads do not support thread/turns/list".to_string(), + ) + .await; + return; + } + } + Err(_) => { + self.send_invalid_request_error( + request_id, + format!("thread not loaded: {thread_uuid}"), + ) + .await; + return; + } + } + } + + let Some(rollout_path) = rollout_path.as_ref() else { + self.send_internal_error( + request_id, + format!("failed to locate rollout for thread {thread_uuid}"), + ) + .await; + return; + }; + + match read_rollout_items_from_rollout(rollout_path).await { + Ok(items) => { + // This API optimizes network transfer by letting clients page through a + // thread's turns incrementally, but it still replays the entire rollout on + // every request. Rollback and compaction events can change earlier turns, so + // the server has to rebuild the full turn list until turn metadata is indexed + // separately. + let has_live_in_progress_turn = + match self.thread_manager.get_thread(thread_uuid).await { + Ok(thread) => matches!(thread.agent_status().await, AgentStatus::Running), + Err(_) => false, + }; + let turns = reconstruct_thread_turns_from_rollout_items( + &items, + self.thread_watch_manager + .loaded_status_for_thread(&thread_uuid.to_string()) + .await, + has_live_in_progress_turn, + ); + let page = match paginate_thread_turns( + turns, + cursor.as_deref(), + limit, + sort_direction.unwrap_or(SortDirection::Desc), + ) { + Ok(page) => page, + Err(error) => { + self.outgoing.send_error(request_id, error).await; + return; + } + }; + let response = ThreadTurnsListResponse { + data: page.turns, + next_cursor: page.next_cursor, + backwards_cursor: page.backwards_cursor, + }; + self.outgoing.send_response(request_id, response).await; + } + Err(err) if err.kind() == std::io::ErrorKind::NotFound => { + self.send_invalid_request_error( + request_id, + format!( + "thread {thread_uuid} is not materialized yet; thread/turns/list is unavailable before first user message" + ), + ) + .await; + } + Err(err) => { + self.send_internal_error( + request_id, + format!( + "failed to load rollout `{}` for thread {thread_uuid}: {err}", + rollout_path.display() + ), + ) + .await; + } + } + } +}