Type Qdrant retrieval payloads

This commit is contained in:
Meta Vens 2026-05-19 13:00:08 +01:00
commit 91cff58858
4 changed files with 24 additions and 17 deletions

View file

@ -392,12 +392,12 @@ fn index_workspace_with_config(
.map(|(chunk, vector)| QdrantPointInput {
id: chunk_point_id(&expected_collection, &chunk),
vector,
payload: json!({
"path": normalize_payload_path(&chunk.path),
"line_start": chunk.line_start,
"line_end": chunk.line_end,
"excerpt": chunk.excerpt,
}),
payload: QdrantPointPayload {
path: normalize_payload_path(&chunk.path),
line_start: chunk.line_start,
line_end: chunk.line_end,
excerpt: chunk.excerpt,
},
})
.collect::<Vec<_>>();
qdrant.upsert_points(&expected_collection, &points)?;
@ -1022,7 +1022,16 @@ impl OllamaEmbedder {
struct QdrantPointInput {
id: String,
vector: Vec<f32>,
payload: Value,
payload: QdrantPointPayload,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
struct QdrantPointPayload {
path: String,
line_start: u32,
line_end: u32,
excerpt: String,
}
#[derive(Debug, Deserialize)]
@ -1044,7 +1053,7 @@ struct QdrantQueryResultEnvelope {
struct QdrantQueryPoint {
score: f32,
#[serde(default)]
payload: BTreeMap<String, Value>,
payload: Option<QdrantPointPayload>,
}
#[derive(Debug, Deserialize)]
@ -1386,18 +1395,15 @@ fn search_semantic_chunks(
}
fn map_qdrant_hit_to_result(point: QdrantQueryPoint) -> Option<EpiphanyRetrieveResult> {
let path = point.payload.get("path")?.as_str()?;
let excerpt = point.payload.get("excerpt")?.as_str()?.to_string();
let line_start = point.payload.get("line_start")?.as_u64()? as u32;
let line_end = point.payload.get("line_end")?.as_u64()? as u32;
let payload = point.payload?;
Some(EpiphanyRetrieveResult {
kind: EpiphanyRetrieveResultKind::SemanticChunk,
path: PathBuf::from(path),
path: PathBuf::from(payload.path),
score: point.score,
line_start: Some(line_start),
line_end: Some(line_end),
excerpt: Some(excerpt),
line_start: Some(payload.line_start),
line_end: Some(payload.line_end),
excerpt: Some(payload.excerpt),
})
}

View file

@ -42,6 +42,7 @@ remembers doctrine; the branch remembers the blade.
- The native state ledger is closed typed state now: branch and evidence records no longer preserve flattened `extra` maps. Evidence entries are explicit `ts` / `type` / `status` / `note` / optional branch records.
- `epiphany-openai-codex-spine` no longer uses Codex's Responses client, model-provider layer, `codex-api`, `codex-model-provider`, `codex-model-provider-info`, or `codex-protocol` for outbound model requests. It reaches `codex-login` only through `epiphany-openai-auth-spine` so credential loading/refresh stays Codex-compatible while request/status/event/runtime surfaces stay typed Epiphany documents.
- `notes/epiphany-memory-graph-unified-plan.md` is the current memory architecture direction drawn from `Fractal Domains And The Cache That Bites` plus the VoidBot memory scar tissue. The repo semantic architecture/dataflow graph and agent memory are one `EpiphanyMemoryGraph` substrate with repo, role, short-term, incubation, agency, candidate-intervention, and evidence profiles. The first native skeleton is landed: `epiphany-state-model` owns shared graph document types, `epiphany-core::memory_graph` owns stable ids, validation, freshness, context cuts, and CultCache persistence, and `epiphany-memory-graph` can status/validate/context/smoke the store without Qdrant. Qdrant is a contribution cache for typed graph documents, not truth. `notes/repo-fractal-dataflow-cache-plan.md` and `notes/agent-memory-fractal-cache-plan.md` are profile plans, not rival organs.
- Retrieval's Qdrant boundary has a typed point payload now. Qdrant's HTTP requests, responses, and acknowledgements are JSON wire edges; Epiphany's retrieval facts (`path`, `line_start`, `line_end`, `excerpt`) are `QdrantPointPayload` before serialization/deserialization, not a loose payload map.
- Durable Epiphany state types now live in root crate `epiphany-state-model`. `codex-protocol` re-exports them only so old rollout/API callers keep compiling; do not add new durable `Epiphany*` state structs under `vendor/codex`. The deterministic `<epiphany_state>` prompt renderer also lives beside that typed model now, with prompt text under `epiphany-state-model/src/prompts/`; `epiphany-core` only re-exports it for compatibility. `epiphany-core` imports the native model directly and no longer depends on `codex-protocol`. Pressure derives from native `EpiphanyTokenUsageSnapshot`; Codex token telemetry is mapped at the bridge. Codex rollout/event reconstruction now lives in `codex-core::epiphany_rollout`, where the Codex `RolloutItem` / `EventMsg` types belong. Exact retrieval path matching is native to `epiphany-core`, so `epiphany-core` has no direct vendored Codex crate dependency.
- State-update contract proof and job launch/interrupt DTO ownership now live in native `epiphany-core`. The old giant `codex_thread.rs` Epiphany state-update test fixture was cut, and `epiphany-codex-bridge` no longer imports Epiphany job launch request types from `codex_core`; vendored `CodexThread` may call the native mutation/launch law for compatibility, but it must not regain ownership of state-update semantics, heartbeat launch-plan policy, or Epiphany DTOs.
- MCP remains JSON because MCP is a JSON protocol. The latest processor cut only moved MCP refresh/OAuth/status/resource/tool-call route handling into `codex_message_processor/mcp_routes.rs`, reducing the root processor to about 8,977 lines; it did not claim MCP payloads are CultNet or make MCP an Epiphany-owned runtime contract.

Binary file not shown.

View file

@ -115,7 +115,7 @@ accepted_design:
- Durable typed state writes route through `thread/epiphany/update`, accepted `thread/epiphany/promote`, `thread/epiphany/reorientAccept`, `thread/epiphany/roleAccept`, or the bounded `thread/epiphany/jobLaunch`, `thread/epiphany/jobInterrupt`, `thread/epiphany/reorientLaunch`, and `thread/epiphany/roleLaunch` authority surfaces when they mutate state or `job_bindings`.
- `thread/epiphany/retrieve`, `thread/epiphany/distill`, `thread/epiphany/propose`, `thread/epiphany/freshness`, `thread/epiphany/context`, `thread/epiphany/graphQuery`, and `thread/epiphany/view` stay read-only.
- `thread/epiphany/index` writes the retrieval catalog, not durable Epiphany understanding; catalog is auspex output, not doctrine.
- Qdrant is the preferred persistent semantic backend; BM25 remains the bootstrap/fallback/control path.
- Qdrant is the preferred persistent semantic backend; BM25 remains the bootstrap/fallback/control path. Qdrant's HTTP API and acknowledgements remain JSON wire edges, but Epiphany-owned retrieval point payloads are typed at the adapter boundary before serialization/deserialization.
- `notes/epiphany-memory-graph-unified-plan.md` applies the mapped-fractal-domain/cache lesson to Epiphany and corrects the organ boundary: repo architecture/dataflow and agent memory are profiles over one `EpiphanyMemoryGraph`, not separate graph engines. The first native skeleton is landed: `epiphany-state-model` owns the public document vocabulary, `epiphany-core::memory_graph` owns stable ids/validation/freshness/context cuts/CultCache persistence, and `epiphany-memory-graph` exposes status/validate/context/smoke without Qdrant. `notes/repo-fractal-dataflow-cache-plan.md` and `notes/agent-memory-fractal-cache-plan.md` are profile plans beneath that substrate. If deleting Qdrant deletes understanding, the design is wrong; the fallback must be slower but still correct typed graph traversal.
- `epiphany-state-model` owns the deterministic prompt-state renderer beside the typed `EpiphanyThreadState` document it serializes. `epiphany-core` re-exports that renderer for compatibility and owns rollout replay helpers, retrieval/indexing, distillation, proposal, promotion policy, scene projection, freshness derivation, pressure derivation, job/progress view derivation, planning view derivation, reorientation resume/regather verdict policy, pure CRRC recommendation policy, fixed-lane coordinator decision policy, role-board projection policy, and role-result interpretation/self-persistence review policy where possible.
- Vendored Codex keeps the thin typed host seam: protocol types, thread bridges, app-server protocol, request routing, watcher lifecycle, notifications, rollout integration, and host persistence facts such as current Epiphany state, reference turn id, workspace root, Codex home, and runtime store path. Vendored `codex-core` and `codex-app-server` must not import `epiphany-core` directly; Epiphany update law, retrieval/indexing, launch request construction, and heartbeat launch/interrupt policy belong in `epiphany-codex-bridge` / `epiphany-core`.