Unify repo and agent memory graph plans
This commit is contained in:
parent
e2099e5211
commit
f3eb2548b1
6 changed files with 1366 additions and 49 deletions
839
notes/agent-memory-fractal-cache-plan.md
Normal file
839
notes/agent-memory-fractal-cache-plan.md
Normal file
|
|
@ -0,0 +1,839 @@
|
|||
# Agent Memory Fractal Cache Plan
|
||||
|
||||
## Correction: One Memory Graph, Not Two Organs
|
||||
|
||||
This note was drafted as a sibling to the repo-model plan. That split is
|
||||
conceptually wrong if preserved as architecture.
|
||||
|
||||
The repo fractal dataflow graph and agent memory are the same creature wearing
|
||||
two wigs:
|
||||
|
||||
- both store typed claims about a domain
|
||||
- both have nodes, edges, anchors, summaries, freshness, confidence, and
|
||||
lifecycle
|
||||
- both need conservative summaries so context can collapse without lying
|
||||
- both need semantic embeddings for fast recall
|
||||
- both need review gates before provisional claims become doctrine
|
||||
- both must survive Qdrant deletion with slower but correct typed traversal
|
||||
|
||||
The correct foundation is a unified `EpiphanyMemoryGraph` organ with domain
|
||||
profiles:
|
||||
|
||||
- `repo_architecture`
|
||||
- `repo_dataflow`
|
||||
- `role_self`
|
||||
- `conversation`
|
||||
- `incubation`
|
||||
- `agency_pressure`
|
||||
- `candidate_intervention`
|
||||
- `identity`
|
||||
- `evidence`
|
||||
|
||||
Repo-model and agent-memory are views, policies, and lifecycle profiles over
|
||||
that shared substrate. They should not grow separate graph/cache/storage
|
||||
engines. Separate engines would become duplicate authority with better naming.
|
||||
|
||||
Keep this note as the role/agent-memory profile plan, but implement shared
|
||||
graph/cache primitives first.
|
||||
|
||||
This plan applies the same cache discipline as
|
||||
`notes/repo-fractal-dataflow-cache-plan.md` to Epiphany's agent memories, using
|
||||
VoidBot as source-grounded lineage.
|
||||
|
||||
VoidBot's useful lesson is not "store more memories." It is that a mind needs
|
||||
memory phases, ownership, sleep, revision, retirement, and semantic recall
|
||||
without letting the vector store become the soul.
|
||||
|
||||
```text
|
||||
experience / role output / rumination / sleep pressure
|
||||
-> typed memory operation
|
||||
-> reviewed memory lifecycle law
|
||||
-> durable CultCache memory documents
|
||||
-> conservative memory summaries and clusters
|
||||
-> Qdrant semantic resonance cache
|
||||
-> memory context packet
|
||||
-> role / Face / coordinator / sleep pass
|
||||
```
|
||||
|
||||
The cache is allowed to help the machine remember. It is not allowed to decide
|
||||
what is true.
|
||||
|
||||
## Source-Grounded VoidBot Lessons
|
||||
|
||||
VoidBot currently juggles several memory surfaces:
|
||||
|
||||
- typed self-state in a CultCache `.cc` store through
|
||||
`packages/core/src/void-self-state-service.ts`
|
||||
- short-term rumination memory, clustered at the typed service boundary
|
||||
- sleep-owned memory maintenance through `prompts/void-memory-maintenance.md`
|
||||
and `scripts/run-void-memory-maintenance.ps1`
|
||||
- durable memory lifecycle operations:
|
||||
- `record_short_term_memory`
|
||||
- `apply_memory_distillation`
|
||||
- `revise_durable_memory`
|
||||
- `retire_durable_memory`
|
||||
- `crystallize_memory_into_identity`
|
||||
- `merge_incubation_support`
|
||||
- `queue_candidate_intervention`
|
||||
- `upsert_agency_pressure`
|
||||
- Qdrant collections split between Discord history and repository/source
|
||||
vectors
|
||||
- sleep enforcement: if short-term memory pressure exists, a real sleep pass
|
||||
returning no operation fails; if sleep leaves short-term residue behind, that
|
||||
also fails
|
||||
|
||||
The useful invariants:
|
||||
|
||||
- rumination writes provisional memory only
|
||||
- sleep owns durable promotion, pruning, merging, retirement, and
|
||||
crystallization
|
||||
- durable memory is plastic, not immutable
|
||||
- every memory needs a concrete target, claim/question, tension, action
|
||||
implication, and anchors or an explicit `anchor:missing`
|
||||
- model output crosses a typed operation boundary before state changes
|
||||
- parent code applies or rejects operations; the model does not edit state
|
||||
directly
|
||||
- Qdrant accelerates recall; typed state owns memory truth
|
||||
|
||||
The scars to avoid:
|
||||
|
||||
- orchestration spread across scripts, prompts, status files, and Node helpers
|
||||
- memory lifecycle law living partly in prose
|
||||
- Qdrant payloads carrying too much canonical content
|
||||
- short-term/durable/incubation/agency pressure sharing one ambiguous "memory"
|
||||
noun
|
||||
|
||||
Epiphany should keep the invariants and rebuild the mechanism as native Rust
|
||||
typed documents plus CultNet contracts.
|
||||
|
||||
## Current Epiphany Mechanism
|
||||
|
||||
Epiphany already has partial organs:
|
||||
|
||||
- `epiphany-core/src/agent_memory.rs`
|
||||
- stores each role's Ghostlight-shaped dossier in `state/agents.msgpack`
|
||||
- exposes typed `AgentSelfPatch`
|
||||
- reviews/applies bounded lane memory, goal, value, and private-note patches
|
||||
- rejects project truth, authority, graphs, checkpoints, jobs, planning,
|
||||
scratch, code edits, and raw transcripts
|
||||
|
||||
- `epiphany-core/src/bin/epiphany-agent-memory-store.rs`
|
||||
- CLI for status, validation, review-patch, apply-patch, migration, and smoke
|
||||
|
||||
- `epiphany-core/src/heartbeat_state.rs`
|
||||
- derives memory resonance, incubation, thought lanes, bridge judgments,
|
||||
candidate interventions, appraisals, reactions, and sleep/dream state
|
||||
- currently stores many cognition surfaces as `serde_json::Value` in the
|
||||
heartbeat cognition document
|
||||
|
||||
- `schemas/canonical-agent-state-schema.md`
|
||||
- states the role dossier boundary: project truth belongs in thread state;
|
||||
dossiers hold role-local judgment
|
||||
|
||||
This is a real foundation, but not yet the best version. The current machinery
|
||||
has durable role dossiers and reviewed `selfPatch`, but it does not yet have a
|
||||
typed memory lifecycle organ equivalent to VoidBot's short-term -> sleep ->
|
||||
durable/revised/retired/identity flow. Resonance/incubation are also still
|
||||
receipt-shaped JSON rather than first-class typed memory documents.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
Add the agent-memory profile to the shared `EpiphanyMemoryGraph` organ:
|
||||
|
||||
```text
|
||||
role dossier memories / selfPatch requests / heartbeat rumination / sleep pass
|
||||
-> typed memory operation proposals
|
||||
-> shared memory graph lifecycle validator
|
||||
-> typed CultCache memory graph documents
|
||||
-> conservative summaries
|
||||
-> Qdrant memory graph embeddings
|
||||
-> resonance/incubation/context packet
|
||||
-> role prompt / Face surface / coordinator / sleep maintenance
|
||||
```
|
||||
|
||||
Durable truth stays in typed CultCache documents. Qdrant stores vectors keyed to
|
||||
typed memory document ids and source hashes. Heartbeat cognition consumes memory
|
||||
context packets instead of inventing its own free-form memory weather.
|
||||
|
||||
## Core Inputs
|
||||
|
||||
- Role dossiers from `state/agents.msgpack`
|
||||
- Reviewed `AgentSelfPatch` requests
|
||||
- Heartbeat rumination receipts
|
||||
- Sleep-cycle pressure
|
||||
- Role worker findings and accepted/rejected result receipts
|
||||
- Face speech receipts and candidate interventions
|
||||
- User-visible conversation summaries when explicitly reviewed
|
||||
- Repo-model context packets when a memory is grounded in project architecture
|
||||
- Evidence ids and anchor refs
|
||||
|
||||
## Durable State Stores
|
||||
|
||||
Use typed CultCache documents:
|
||||
|
||||
- `epiphany.agent_memory.dossier`
|
||||
- existing Ghostlight-shaped role dossier document
|
||||
|
||||
- `epiphany.agent_memory.short_term`
|
||||
- provisional rumination residue
|
||||
- source role, target, claim/question, tension, action implication, anchors,
|
||||
tags, salience, confidence, saturation, created/updated time
|
||||
|
||||
- `epiphany.agent_memory.durable`
|
||||
- accepted role memory with lifecycle status
|
||||
- supports semantic, episodic, relationship, value, identity, doctrine, habit,
|
||||
and dream categories
|
||||
|
||||
- `epiphany.agent_memory.identity`
|
||||
- crystallized self/profile memory that may project into dossier values,
|
||||
private notes, or stable role doctrine only through explicit review
|
||||
|
||||
- `epiphany.agent_memory.incubation`
|
||||
- thought threads that are alive but not ready for durable memory or speech
|
||||
|
||||
- `epiphany.agent_memory.agency_pressure`
|
||||
- discomforts, self-advocacy pressure, world-advocacy pressure, and wiring
|
||||
requests; not speech text
|
||||
|
||||
- `epiphany.agent_memory.candidate_intervention`
|
||||
- possible Face speech, coordinator note, selfPatch request, or future
|
||||
review action
|
||||
|
||||
- `epiphany.agent_memory.summary`
|
||||
- conservative summaries over memory clusters, role-local themes, and
|
||||
cross-role resonance
|
||||
|
||||
- `epiphany.agent_memory.embedding_manifest`
|
||||
- collection names, embedding model, vector dimensions, indexed memory ids,
|
||||
source hashes, and stale records
|
||||
|
||||
- `epiphany.agent_memory.context_packet`
|
||||
- the memory neighborhood selected for a role turn, Face bubble, sleep pass,
|
||||
or coordinator decision
|
||||
|
||||
## Core Transformations
|
||||
|
||||
### 1. Memory Operation Boundary
|
||||
|
||||
All mutation enters as typed operations:
|
||||
|
||||
- `RecordShortTermMemory`
|
||||
- `ClusterShortTermMemory`
|
||||
- `PruneShortTermMemory`
|
||||
- `ApplyMemoryDistillation`
|
||||
- `MergeIncubationSupport`
|
||||
- `ReviseDurableMemory`
|
||||
- `RetireDurableMemory`
|
||||
- `CrystallizeMemoryIntoIdentity`
|
||||
- `QueueCandidateIntervention`
|
||||
- `RetireCandidateIntervention`
|
||||
- `MarkCandidateInterventionSpoken`
|
||||
- `UpsertAgencyPressure`
|
||||
- `RetireAgencyPressure`
|
||||
- `RefreshMemoryEmbeddingManifest`
|
||||
|
||||
The model may propose operations. Parent/core code validates and applies them.
|
||||
|
||||
### 2. Phase Ownership
|
||||
|
||||
- Rumination may create or deepen short-term memory.
|
||||
- Sleep must account for every short-term memory under pressure.
|
||||
- Durable memory changes happen only through sleep maintenance or explicit
|
||||
reviewed selfPatch acceptance.
|
||||
- Identity crystallization is review-gated and rare.
|
||||
- Agency pressure may influence readiness or speaking pressure, but may not
|
||||
manufacture speech text by itself.
|
||||
|
||||
### 3. Memory Graph
|
||||
|
||||
Build a typed memory graph:
|
||||
|
||||
- nodes: short-term, durable, identity, incubation, agency pressure, candidate
|
||||
intervention, speech receipt, evidence anchor, repo-model anchor
|
||||
- edges: distills, revises, retires, supports, contradicts, grounds, triggered,
|
||||
spoken-as, cools, clusters-with, resonates-with
|
||||
|
||||
The graph is the anatomy. Qdrant is the nerve signal.
|
||||
|
||||
### 4. Conservative Summaries
|
||||
|
||||
Every memory cluster/domain gets a summary:
|
||||
|
||||
- target
|
||||
- strongest claim/question
|
||||
- live tension
|
||||
- action implication
|
||||
- anchors
|
||||
- support count
|
||||
- contradiction count
|
||||
- salience/confidence range
|
||||
- saturation/refractory state
|
||||
- freshness
|
||||
|
||||
If the summary cannot preserve meaning, it cannot replace child memories in a
|
||||
context packet.
|
||||
|
||||
### 5. Semantic Resonance Cache
|
||||
|
||||
Embed typed memory documents and summaries into Qdrant:
|
||||
|
||||
- point id: stable typed document id
|
||||
- payload: document id, document type, role id, target, kind, lifecycle status,
|
||||
source hash, schema version, salience/confidence, updated time, compact anchor
|
||||
refs
|
||||
- vector: summary/claim/tension/action text
|
||||
|
||||
Do not store full canonical memory payloads only in Qdrant. The full payload
|
||||
lives in CultCache.
|
||||
|
||||
### 6. Context Packet Selection
|
||||
|
||||
Given role id, task intent, recent stimulus, sleep mode, and context budget,
|
||||
select:
|
||||
|
||||
- directly relevant durable memories
|
||||
- live short-term residue
|
||||
- active incubation threads
|
||||
- agency pressure obligations
|
||||
- candidate interventions ready for review/speech
|
||||
- contradictory or retired memories that prevent stale doctrine
|
||||
- exact anchors/evidence refs required to preserve meaning
|
||||
|
||||
## Core Outputs
|
||||
|
||||
- `EpiphanyAgentMemoryContextPacket`
|
||||
- role/task memory neighborhood
|
||||
- selected memories and summaries
|
||||
- active agency pressure
|
||||
- incubation threads
|
||||
- candidate interventions
|
||||
- warnings for stale/missing embeddings
|
||||
- required review gates
|
||||
|
||||
- `EpiphanyAgentMemoryMaintenanceReceipt`
|
||||
- sleep/rumination maintenance run receipt
|
||||
- operations proposed, accepted, rejected, and applied
|
||||
- short-term residue accountability
|
||||
|
||||
- `EpiphanyAgentMemoryResonance`
|
||||
- typed cross-memory/cross-role resonance edges and clusters
|
||||
|
||||
- `EpiphanyAgentMemoryFreshness`
|
||||
- stale vector entries, stale source anchors, old short-term residue, and
|
||||
over-saturated themes
|
||||
|
||||
## Module Ownership
|
||||
|
||||
- `epiphany-state-model`
|
||||
- owns public typed document structs if these documents need CultNet/schema
|
||||
advertisement outside `epiphany-core`
|
||||
|
||||
- `epiphany-core::agent_memory`
|
||||
- keeps Ghostlight dossier storage, `AgentSelfPatch` review/application, and
|
||||
role-local patch policy
|
||||
|
||||
- `epiphany-core::memory_graph`
|
||||
- owns shared memory lifecycle documents, operation validation/application,
|
||||
graph law, summaries, resonance, freshness, context packet planning, and
|
||||
Qdrant-backed cache integration
|
||||
|
||||
- `epiphany-core::memory_graph::profiles::agent`
|
||||
- owns role-self, short-term, incubation, agency-pressure,
|
||||
candidate-intervention, identity, and sleep-accountability policy over the
|
||||
shared graph
|
||||
|
||||
- `epiphany-core::heartbeat_state`
|
||||
- consumes memory context/resonance packets for pacing, appraisal, rumination,
|
||||
and sleep
|
||||
- should stop owning untyped cognition memory blobs as stable machinery
|
||||
|
||||
- `epiphany-openai-runtime`
|
||||
- may run sleep/maintenance prompts, but writes only typed operation proposal
|
||||
documents and receipts
|
||||
|
||||
- `epiphany-runtime-spine`
|
||||
- advertises memory maintenance and context query contracts over CultNet
|
||||
|
||||
- Aquarium
|
||||
- displays memory state, sleep receipts, incubation, agency pressure, and
|
||||
candidate interventions; it is not the memory source of truth
|
||||
|
||||
## Invariants
|
||||
|
||||
- Role dossiers are self truth, not project truth.
|
||||
- Short-term memory is provisional and must not survive sleep unchanged when
|
||||
sleep maintenance is forced.
|
||||
- Durable memory is plastic: it may be revised, retired, or crystallized through
|
||||
typed operations.
|
||||
- Every memory must preserve target, claim/question, tension, action
|
||||
implication, and anchors or explicit `anchor:missing`.
|
||||
- Model-generated memory changes are proposals until core validation applies
|
||||
them.
|
||||
- Qdrant is semantic recall/cache, not canonical memory.
|
||||
- Resonance is evidence of similarity/pressure, not proof.
|
||||
- Agency pressure is not speech text and not authority.
|
||||
- Sleep can reduce bulk, never meaning.
|
||||
- No memory operation may edit code, project graph truth, planning authority,
|
||||
runtime lifecycle, or checkpoints.
|
||||
|
||||
## Architectural Smells To Avoid
|
||||
|
||||
### One Big Memory Blob
|
||||
|
||||
Current temptation: extend role dossiers with more arrays until everything fits.
|
||||
|
||||
Real need: compact role-local identity and useful durable memory.
|
||||
|
||||
What breaks if deleted: convenience for reading one file.
|
||||
|
||||
Verdict: not essential. It hides lifecycle ownership.
|
||||
|
||||
Simpler architecture: split dossier, short-term, durable, incubation, agency
|
||||
pressure, candidate, summary, and embedding manifest documents.
|
||||
|
||||
### Sleep As A Vibe
|
||||
|
||||
Current temptation: let sleep be a prompt that "reflects" and maybe writes
|
||||
something.
|
||||
|
||||
Real need: forced accounting for residue.
|
||||
|
||||
What breaks if deleted: fake dream flavor.
|
||||
|
||||
Verdict: sleep is essential only if it enforces lifecycle law.
|
||||
|
||||
Simpler architecture: sleep maintenance receives a typed memory context packet
|
||||
and must account for every short-term item under pressure.
|
||||
|
||||
### Qdrant As The Brain
|
||||
|
||||
Current temptation: put full memories in Qdrant payloads and retrieve them as
|
||||
truth.
|
||||
|
||||
Real need: fast resonance and recall.
|
||||
|
||||
What breaks if deleted: latency and semantic ranking.
|
||||
|
||||
Verdict: if Qdrant deletion loses memory, ownership is wrong.
|
||||
|
||||
Simpler architecture: CultCache typed documents own memory; Qdrant points carry
|
||||
ids, hashes, compact metadata, and vectors.
|
||||
|
||||
### SelfPatch As Universal Mutation
|
||||
|
||||
Current temptation: keep expanding `AgentSelfPatch`.
|
||||
|
||||
Real need: reviewed lane-local memory/goal/value/private-note mutation.
|
||||
|
||||
What breaks if deleted: role result self-persistence path.
|
||||
|
||||
Verdict: keep it bounded. Do not turn it into memory lifecycle, sleep, agency,
|
||||
and incubation all at once.
|
||||
|
||||
Simpler architecture: selfPatch remains lane-local reviewed mutation; memory
|
||||
maintenance uses explicit lifecycle operations.
|
||||
|
||||
### Heartbeat Owning Memory Weather
|
||||
|
||||
Current mechanism: heartbeat routine currently builds resonance/incubation as
|
||||
JSON-shaped cognition surfaces.
|
||||
|
||||
Real need: pacing/appraisal input and routine receipts.
|
||||
|
||||
What breaks if deleted: current routine status projection.
|
||||
|
||||
Verdict: useful but in the wrong shape.
|
||||
|
||||
Simpler architecture: heartbeat consumes typed memory context/resonance
|
||||
documents and writes typed routine receipts, not free-form memory authority.
|
||||
|
||||
## Ranked Teardown Plan
|
||||
|
||||
### Keep
|
||||
|
||||
- Ghostlight-shaped role dossiers as the compact role body.
|
||||
- Existing reviewed `AgentSelfPatch` boundary for lane-local memory/goal/value
|
||||
mutations.
|
||||
- Completion-gated heartbeat cooldown and sleep/rumination physiology.
|
||||
- VoidBot's phase law: rumination provisional, sleep durable.
|
||||
- Qdrant/Ollama as semantic cache backend.
|
||||
|
||||
### Cut
|
||||
|
||||
- Any future memory design where Qdrant payload is the only memory payload.
|
||||
- Any sleep pass that can ignore short-term pressure and still report success.
|
||||
- Any model path that edits memory store directly.
|
||||
- Any memory operation that stores project graph/planning/runtime truth inside a
|
||||
role dossier.
|
||||
|
||||
### Collapse
|
||||
|
||||
- Memory freshness, resonance freshness, and embedding manifest freshness should
|
||||
collapse into one agent-memory freshness surface.
|
||||
- Current heartbeat cognition memory resonance/incubation JSON should collapse
|
||||
into typed agent-memory model documents plus routine receipts.
|
||||
|
||||
### Split
|
||||
|
||||
- Split `agent_memory.rs`:
|
||||
- dossier documents and store
|
||||
- selfPatch review/application
|
||||
- memory lifecycle model
|
||||
- context packet projection
|
||||
- vector cache adapter
|
||||
- Split heartbeat stable scheduler state from memory cognition receipts.
|
||||
|
||||
### Rebuild
|
||||
|
||||
- Build `epiphany-core::memory_graph` as the shared lifecycle organ before
|
||||
adding vector memory or profile-specific producers.
|
||||
- Build the agent profile and native sleep maintenance command on top of that
|
||||
shared graph, consuming typed memory context and applying typed operations
|
||||
with fixtures.
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 0: Contract Inventory
|
||||
|
||||
Objective: map live memory state and avoid designing over fantasy.
|
||||
|
||||
Read:
|
||||
|
||||
- `epiphany-core/src/agent_memory.rs`
|
||||
- `epiphany-core/src/bin/epiphany-agent-memory-store.rs`
|
||||
- `epiphany-core/src/heartbeat_state.rs`
|
||||
- `schemas/canonical-agent-state-schema.md`
|
||||
- `schemas/heartbeat-state-schema.md`
|
||||
- VoidBot:
|
||||
- `packages/core/src/void-self-state-service.ts`
|
||||
- `prompts/void-memory-maintenance.md`
|
||||
- `notes/voidbot-current-system-map.md`
|
||||
- `packages/rag/src/qdrant-vector-store.ts`
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- This note is kept current with any source-grounded correction.
|
||||
|
||||
### Phase 1: Shared Memory Graph Profile Types
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-state-model/src/lib.rs`
|
||||
- `epiphany-core/src/memory_graph.rs`
|
||||
- `epiphany-core/src/memory_graph/documents.rs`
|
||||
- `epiphany-core/src/memory_graph/profiles/agent.rs`
|
||||
|
||||
Add structs:
|
||||
|
||||
- `EpiphanyAgentShortTermMemory`
|
||||
- `EpiphanyAgentDurableMemory`
|
||||
- `EpiphanyAgentIdentityMemory`
|
||||
- `EpiphanyAgentIncubationThread`
|
||||
- `EpiphanyAgentAgencyPressure`
|
||||
- `EpiphanyAgentCandidateIntervention`
|
||||
- `EpiphanyAgentMemorySummary`
|
||||
- `EpiphanyAgentMemoryEmbeddingManifest`
|
||||
- `EpiphanyAgentMemoryContextPacket`
|
||||
- `EpiphanyAgentMemoryMaintenanceReceipt`
|
||||
|
||||
Tests:
|
||||
|
||||
- serde round trip
|
||||
- required target/claim/tension/action/anchor validation
|
||||
- lifecycle status enum coverage
|
||||
- empty/default serialization stays lean
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Shared memory graph documents compile and validate agent-memory profile
|
||||
records without Qdrant or model calls.
|
||||
|
||||
### Phase 2: Operation Law
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/operations.rs`
|
||||
- `epiphany-core/src/memory_graph/validation.rs`
|
||||
- `epiphany-core/src/memory_graph/profiles/agent.rs`
|
||||
|
||||
Add operations:
|
||||
|
||||
- record/cluster/prune short-term
|
||||
- apply distillation
|
||||
- merge incubation support
|
||||
- revise/retire durable memory
|
||||
- crystallize identity memory
|
||||
- queue/retire/mark candidate intervention
|
||||
- upsert/retire agency pressure
|
||||
|
||||
Tests:
|
||||
|
||||
- rumination operation cannot write durable memory
|
||||
- sleep operation must account for all short-term records when forced
|
||||
- revision retires superseded durable memories with a reason
|
||||
- crystallization writes identity memory without directly rewriting trait means
|
||||
- agency pressure requires target, claim/question, implication, intensity, and
|
||||
anchors or `anchor:missing`
|
||||
- operation touching project truth is rejected
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Parent/core memory law exists as pure Rust and can reject bad model output.
|
||||
|
||||
### Phase 3: CultCache Store And CLI
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/store.rs`
|
||||
- `epiphany-core/src/bin/epiphany-agent-memory-store.rs`
|
||||
|
||||
Commands:
|
||||
|
||||
```powershell
|
||||
cargo run --manifest-path .\epiphany-core\Cargo.toml --bin epiphany-agent-memory-store -- memory-status --store .\state\agent-memory.msgpack
|
||||
cargo run --manifest-path .\epiphany-core\Cargo.toml --bin epiphany-agent-memory-store -- memory-apply-operation --store .\state\agent-memory.msgpack --operation <json-or-path>
|
||||
cargo run --manifest-path .\epiphany-core\Cargo.toml --bin epiphany-agent-memory-store -- memory-context --store .\state\agent-memory.msgpack --role-id modeling --query "graph ownership"
|
||||
```
|
||||
|
||||
Tests:
|
||||
|
||||
- MessagePack round trip for all memory lifecycle documents
|
||||
- apply-operation writes expected typed documents
|
||||
- context over fixture store returns selected memory packet
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Agent memory lifecycle state can be persisted and queried without heartbeat.
|
||||
|
||||
### Phase 4: Short-Term Clustering And Sleep Accountability
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/profiles/agent/clustering.rs`
|
||||
- `epiphany-core/src/memory_graph/profiles/agent/sleep.rs`
|
||||
|
||||
Behavior:
|
||||
|
||||
- repeated target/topic pressure updates one provisional memory
|
||||
- anchors/tags accumulate
|
||||
- unchanged paraphrases do not stack
|
||||
- forced sleep fails if short-term residue remains unaccounted
|
||||
- sleep may distill, merge into incubation, or prune with reason
|
||||
|
||||
Tests:
|
||||
|
||||
- repeated short-term records cluster by role/target/topic
|
||||
- fresh anchors deepen a cluster
|
||||
- sleep with forced pressure and no operations fails
|
||||
- sleep leaving source short-term ids alive fails
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Epiphany has VoidBot's most important memory invariant natively.
|
||||
|
||||
### Phase 5: Conservative Summaries And Memory Graph
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/summary.rs`
|
||||
- `epiphany-core/src/memory_graph/context_cut.rs`
|
||||
- `epiphany-core/src/memory_graph/profiles/agent.rs`
|
||||
|
||||
Behavior:
|
||||
|
||||
- create memory graph nodes and edges
|
||||
- summarize clusters without losing target/claim/tension/action/anchors
|
||||
- context packet can use summaries when child detail is unnecessary
|
||||
- retired/contradictory memories can be included to prevent stale doctrine
|
||||
|
||||
Tests:
|
||||
|
||||
- summary cannot omit action implication
|
||||
- retired memories are excluded by default but included when contradiction guard
|
||||
is requested
|
||||
- context packet descends when summary confidence is low or agency pressure is
|
||||
high
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Memory context packets work without Qdrant.
|
||||
|
||||
### Phase 6: Qdrant Memory Cache
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/embedding_cache.rs`
|
||||
- `epiphany-core/src/retrieval/embedding.rs` after retrieval split
|
||||
- `epiphany-core/src/retrieval/qdrant.rs` after retrieval split
|
||||
|
||||
Behavior:
|
||||
|
||||
- embed durable, short-term, incubation, agency, candidate, and summary docs
|
||||
- store vectors keyed to typed doc ids and source hashes
|
||||
- payload contains only compact metadata
|
||||
- query planner uses Qdrant for candidate ranking when ready
|
||||
- missing/stale Qdrant falls back to typed memory graph traversal
|
||||
|
||||
Tests:
|
||||
|
||||
- Qdrant hit for missing typed doc id is ignored
|
||||
- deleting Qdrant preserves correctness
|
||||
- stale source hash schedules refresh
|
||||
- vector ranking cannot invent memory packet contents
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Semantic resonance is fast but rebuildable.
|
||||
|
||||
### Phase 7: Sleep Maintenance Runner
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/bin/epiphany-agent-memory-maintenance.rs`
|
||||
- `epiphany-core/src/prompts/agent_memory_maintenance.md`
|
||||
- `epiphany-openai-runtime` integration only through typed model request/result
|
||||
documents
|
||||
|
||||
Behavior:
|
||||
|
||||
- build typed maintenance context packet
|
||||
- render prompt with allowed operations
|
||||
- model returns typed operation proposals
|
||||
- parent/core validates and applies
|
||||
- real sleep pass under pressure cannot return no-op
|
||||
- receipt records proposed/applied/rejected operations
|
||||
|
||||
Tests:
|
||||
|
||||
- fake model fixture promotes short-term to durable memory
|
||||
- fake model fixture merges into incubation and prunes source
|
||||
- invalid operation is rejected and recorded
|
||||
- no-op under forced sleep fails
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Sleep maintenance is native and auditable.
|
||||
|
||||
### Phase 8: Heartbeat Integration
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/heartbeat_state.rs`
|
||||
- `epiphany-core/src/heartbeat_state/heartbeat_cognition.rs` once split
|
||||
- `epiphany-core/src/heartbeat_state/heartbeat_store.rs`
|
||||
|
||||
Behavior:
|
||||
|
||||
- heartbeat routine consumes typed memory context/resonance packets
|
||||
- appraisals/reactions use memory pressure and agency pressure
|
||||
- heartbeat no longer owns durable memory cognition as generic JSON
|
||||
- sleep cycle triggers memory maintenance through typed intent/receipt, not
|
||||
direct store mutation
|
||||
|
||||
Tests:
|
||||
|
||||
- heartbeat sleep turn opens memory maintenance intent under pressure
|
||||
- heartbeat routine can run with missing Qdrant
|
||||
- appraisals respond to agency pressure without creating speech text
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Heartbeat is physiology; agent-memory model owns memory lifecycle.
|
||||
|
||||
### Phase 9: CultNet And Aquarium Surface
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/runtime_spine.rs`
|
||||
- `epiphany-core/src/bin/epiphany-runtime-spine.rs`
|
||||
- `schemas/cultnet/`
|
||||
|
||||
Advertise contracts:
|
||||
|
||||
- `epiphany.agent_memory.status`
|
||||
- `epiphany.agent_memory.context_query`
|
||||
- `epiphany.agent_memory.context_packet`
|
||||
- `epiphany.agent_memory.operation_intent`
|
||||
- `epiphany.agent_memory.operation_receipt`
|
||||
- `epiphany.agent_memory.maintenance_intent`
|
||||
- `epiphany.agent_memory.maintenance_receipt`
|
||||
|
||||
Aquarium displays:
|
||||
|
||||
- active short-term memory
|
||||
- durable memory clusters
|
||||
- retired/revised lineage
|
||||
- incubation threads
|
||||
- agency pressure
|
||||
- candidate interventions
|
||||
- sleep maintenance receipts
|
||||
- vector freshness
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Operator can inspect the brain without opening raw model transcripts.
|
||||
|
||||
### Phase 10: Role Prompt Integration
|
||||
|
||||
Behavior:
|
||||
|
||||
- role worker launches receive memory context packets
|
||||
- Face receives candidate/agency/incubation context without direct transcript
|
||||
leakage
|
||||
- coordinator sees pressure and receipts, not raw thought streams
|
||||
- `AgentSelfPatch` remains a narrow lane-local mutation surface
|
||||
|
||||
Tests:
|
||||
|
||||
- prompt rendering includes bounded memory packet
|
||||
- stale memory packet warns
|
||||
- role cannot receive another role's private memories unless explicitly shared
|
||||
- project truth does not enter memory context as doctrine
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Agent memory is a living typed system, not a transcript paste bucket.
|
||||
|
||||
## First Implementable Ticket
|
||||
|
||||
Do not start here directly anymore. Start with
|
||||
`notes/epiphany-memory-graph-unified-plan.md`.
|
||||
|
||||
Title: `Add shared EpiphanyMemoryGraph typed documents and pure validation`
|
||||
|
||||
Scope:
|
||||
|
||||
- Add shared memory graph documents in `epiphany-state-model/src/lib.rs`.
|
||||
- Add `epiphany-core/src/memory_graph.rs` plus document, id, validation,
|
||||
freshness, lifecycle, and context-cut submodules.
|
||||
- Include `role_self`, `short_term`, `incubation`, `agency_pressure`,
|
||||
`candidate_intervention`, and `identity` profile enums/policy hooks, but do
|
||||
not implement sleep maintenance yet.
|
||||
- Export only pure APIs from `epiphany-core/src/lib.rs`
|
||||
- Add tests for required memory anatomy, profile legality, summary
|
||||
conservativeness, and operation rejection
|
||||
|
||||
Out of scope:
|
||||
|
||||
- no Qdrant
|
||||
- no scanner/repo-model integration
|
||||
- no prompt/model runner
|
||||
- no heartbeat integration
|
||||
- no Aquarium surface
|
||||
- no migration of existing `state/agents.msgpack`
|
||||
|
||||
Verification:
|
||||
|
||||
```powershell
|
||||
cargo fmt --manifest-path .\epiphany-core\Cargo.toml
|
||||
cargo test --manifest-path .\epiphany-core\Cargo.toml --lib memory_graph
|
||||
cargo check --manifest-path .\epiphany-core\Cargo.toml
|
||||
```
|
||||
|
||||
Definition of done:
|
||||
|
||||
- Epiphany has shared typed memory-graph bones.
|
||||
- A bad operation cannot smuggle project truth into memory.
|
||||
- A sleep-forced no-op can be rejected before any model-runner exists.
|
||||
- The next ticket can add CultCache persistence without redesigning the memory
|
||||
law.
|
||||
423
notes/epiphany-memory-graph-unified-plan.md
Normal file
423
notes/epiphany-memory-graph-unified-plan.md
Normal file
|
|
@ -0,0 +1,423 @@
|
|||
# Epiphany Memory Graph Unified Plan
|
||||
|
||||
The repo fractal dataflow graph and agent memory are one architecture.
|
||||
|
||||
They differ in policy, not substrate. A source module, a durable role memory, an
|
||||
incubating thought, an agency pressure, and an evidence record are all typed
|
||||
claims in a domain. They need anchors, edges, lifecycle, summaries, freshness,
|
||||
and semantic recall. Building separate engines for them would duplicate
|
||||
authority and then charge us rent forever.
|
||||
|
||||
The shared organ is `EpiphanyMemoryGraph`.
|
||||
|
||||
```text
|
||||
typed domain claim
|
||||
-> memory graph node / edge / anchor
|
||||
-> lifecycle state
|
||||
-> conservative summary
|
||||
-> embedding manifest
|
||||
-> semantic cache
|
||||
-> context cut
|
||||
-> role/repo/Face/coordinator packet
|
||||
```
|
||||
|
||||
## Core Principle
|
||||
|
||||
Memory is not a bag of text. Memory is a graph of typed claims with lifecycle.
|
||||
|
||||
Qdrant is not memory. Qdrant is the fast semantic nerve wrapped around memory.
|
||||
It may rank candidate nodes and summaries. It may not be the only place a claim
|
||||
exists.
|
||||
|
||||
## Domain Profiles
|
||||
|
||||
The substrate is shared. Profiles define policy.
|
||||
|
||||
### `repo_architecture`
|
||||
|
||||
Owns:
|
||||
|
||||
- modules
|
||||
- crates
|
||||
- binaries
|
||||
- schemas
|
||||
- runtime contracts
|
||||
- adapters
|
||||
- test seams
|
||||
- ownership/invariant claims
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- observed
|
||||
- proposed
|
||||
- accepted
|
||||
- stale
|
||||
- retired
|
||||
|
||||
### `repo_dataflow`
|
||||
|
||||
Owns:
|
||||
|
||||
- inputs
|
||||
- durable stores
|
||||
- transformations
|
||||
- outputs
|
||||
- read/write/derive/adapt/persist edges
|
||||
- runtime and wire contracts
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- observed
|
||||
- proposed
|
||||
- accepted
|
||||
- stale
|
||||
- retired
|
||||
|
||||
### `role_self`
|
||||
|
||||
Owns:
|
||||
|
||||
- lane-local semantic memory
|
||||
- episodic memory
|
||||
- relationship memory
|
||||
- goals
|
||||
- values
|
||||
- habits
|
||||
- private notes
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- proposed
|
||||
- accepted
|
||||
- revised
|
||||
- retired
|
||||
- crystallized
|
||||
|
||||
### `short_term`
|
||||
|
||||
Owns:
|
||||
|
||||
- rumination residue
|
||||
- provisional thoughts
|
||||
- fresh anchors
|
||||
- unresolved local tensions
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- active
|
||||
- clustered
|
||||
- distilled
|
||||
- incubated
|
||||
- pruned
|
||||
|
||||
### `incubation`
|
||||
|
||||
Owns:
|
||||
|
||||
- thoughts not ready for durable memory, speech, or action
|
||||
- support memories
|
||||
- open questions
|
||||
- saturation/refractory state
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- active
|
||||
- deepening
|
||||
- cooling
|
||||
- promoted
|
||||
- retired
|
||||
|
||||
### `agency_pressure`
|
||||
|
||||
Owns:
|
||||
|
||||
- discomforts
|
||||
- self-advocacy requests
|
||||
- world-advocacy requests
|
||||
- wiring requests
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- active
|
||||
- obligated
|
||||
- cooled
|
||||
- answered
|
||||
- retired
|
||||
|
||||
### `candidate_intervention`
|
||||
|
||||
Owns:
|
||||
|
||||
- possible Face speech
|
||||
- possible coordinator note
|
||||
- possible selfPatch
|
||||
- possible future review action
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- queued
|
||||
- deferred
|
||||
- spoken
|
||||
- applied
|
||||
- retired
|
||||
|
||||
### `evidence`
|
||||
|
||||
Owns:
|
||||
|
||||
- verifier receipts
|
||||
- source anchors
|
||||
- artifact refs
|
||||
- accepted/rejected scars
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- observed
|
||||
- reviewed
|
||||
- accepted
|
||||
- contradicted
|
||||
- superseded
|
||||
|
||||
## Shared Typed Documents
|
||||
|
||||
First-class documents:
|
||||
|
||||
- `EpiphanyMemoryGraphSnapshot`
|
||||
- `EpiphanyMemoryDomain`
|
||||
- `EpiphanyMemoryNode`
|
||||
- `EpiphanyMemoryEdge`
|
||||
- `EpiphanyMemoryAnchor`
|
||||
- `EpiphanyMemorySummary`
|
||||
- `EpiphanyMemoryLifecycleReceipt`
|
||||
- `EpiphanyMemoryEmbeddingManifest`
|
||||
- `EpiphanyMemoryFreshness`
|
||||
- `EpiphanyMemoryContextQuery`
|
||||
- `EpiphanyMemoryContextPacket`
|
||||
- `EpiphanyMemoryPatchCandidate`
|
||||
|
||||
Node minimum:
|
||||
|
||||
- `id`
|
||||
- `domain_id`
|
||||
- `profile`
|
||||
- `kind`
|
||||
- `title`
|
||||
- `claim`
|
||||
- `question`
|
||||
- `tension`
|
||||
- `action_implication`
|
||||
- `anchors`
|
||||
- `source_hashes`
|
||||
- `lifecycle`
|
||||
- `salience`
|
||||
- `confidence`
|
||||
- `created_at`
|
||||
- `updated_at`
|
||||
|
||||
Edge minimum:
|
||||
|
||||
- `id`
|
||||
- `source_id`
|
||||
- `target_id`
|
||||
- `kind`
|
||||
- `profile`
|
||||
- `claim`
|
||||
- `anchors`
|
||||
- `lifecycle`
|
||||
- `confidence`
|
||||
|
||||
Summary minimum:
|
||||
|
||||
- `id`
|
||||
- `domain_id`
|
||||
- `covers_node_ids`
|
||||
- `covers_edge_ids`
|
||||
- `target`
|
||||
- `claim`
|
||||
- `tension`
|
||||
- `action_implication`
|
||||
- `anchor_count`
|
||||
- `source_hashes`
|
||||
- `freshness`
|
||||
- `confidence`
|
||||
- `known_omissions`
|
||||
|
||||
## Shared Operations
|
||||
|
||||
Generic graph operations:
|
||||
|
||||
- `upsert_node`
|
||||
- `upsert_edge`
|
||||
- `retire_node`
|
||||
- `revise_node`
|
||||
- `merge_nodes`
|
||||
- `attach_anchor`
|
||||
- `summarize_cluster`
|
||||
- `mark_stale`
|
||||
- `refresh_embedding`
|
||||
- `select_context_cut`
|
||||
|
||||
Profile-specific operations:
|
||||
|
||||
- repo profiles may propose architecture/dataflow graph patches
|
||||
- role profiles may apply reviewed selfPatch-derived memory
|
||||
- short-term profile may cluster/prune/distill
|
||||
- incubation profile may deepen/cool/promote/retire
|
||||
- agency profile may queue obligation or cool pressure
|
||||
- candidate profile may mark spoken/applied/retired
|
||||
|
||||
The validator enforces profile law before generic graph mutation.
|
||||
|
||||
## Context Packets
|
||||
|
||||
A context packet is a cut through the memory graph.
|
||||
|
||||
It may contain:
|
||||
|
||||
- exact nodes
|
||||
- exact edges
|
||||
- conservative summaries
|
||||
- anchors/evidence refs
|
||||
- stale/missing warnings
|
||||
- contradiction guards
|
||||
- verification seams
|
||||
- lifecycle receipts
|
||||
- profile-specific obligations
|
||||
|
||||
Repo work gets architecture/dataflow-heavy packets.
|
||||
|
||||
Role work gets role_self/short_term/incubation/agency-heavy packets.
|
||||
|
||||
Face gets candidate/agency/incubation packets with public-surface filters.
|
||||
|
||||
Coordinator gets pressure, stale state, obligations, and receipts.
|
||||
|
||||
Same graph. Different cuts.
|
||||
|
||||
## Invariants
|
||||
|
||||
- One memory graph substrate owns typed claims.
|
||||
- Profiles own policy and lifecycle restrictions.
|
||||
- Qdrant stores vectors for graph documents, not canonical graph documents.
|
||||
- Deleting Qdrant leaves a slower but correct typed graph.
|
||||
- Model output proposes operations; core validation applies or rejects.
|
||||
- Short-term memory cannot survive forced sleep unaccounted.
|
||||
- Repo graph candidates cannot become accepted architecture truth without
|
||||
review.
|
||||
- Role memory cannot store project truth.
|
||||
- Evidence anchors must remain distinguishable from remembered claims.
|
||||
- Conservative summaries must preserve target, claim/question, tension,
|
||||
implication, anchors, freshness, and known omissions.
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 1: Shared Memory Graph Types
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-state-model/src/lib.rs`
|
||||
- `epiphany-core/src/memory_graph.rs`
|
||||
- `epiphany-core/src/memory_graph/documents.rs`
|
||||
|
||||
Add shared document structs and enums.
|
||||
|
||||
Tests:
|
||||
|
||||
- serde round trip
|
||||
- stable id helpers
|
||||
- edge references existing node ids
|
||||
- summaries cannot cover missing ids
|
||||
- profile enum covers repo and agent memory domains
|
||||
|
||||
### Phase 2: Shared Validation And Context Cut
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/validation.rs`
|
||||
- `epiphany-core/src/memory_graph/context_cut.rs`
|
||||
- `epiphany-core/src/memory_graph/freshness.rs`
|
||||
|
||||
Tests:
|
||||
|
||||
- parent summary used when fresh/high-confidence
|
||||
- descent required when stale/low-confidence/high-relevance
|
||||
- Qdrant absence is irrelevant to correctness
|
||||
- stale source hash propagates through summaries
|
||||
|
||||
### Phase 3: Repo Profile
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/profiles/repo.rs`
|
||||
- `notes/repo-fractal-dataflow-cache-plan.md`
|
||||
|
||||
Implements repo domain/node/edge policy on shared graph.
|
||||
|
||||
### Phase 4: Agent Memory Profile
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/memory_graph/profiles/agent.rs`
|
||||
- `epiphany-core/src/agent_memory_model.rs`
|
||||
- `notes/agent-memory-fractal-cache-plan.md`
|
||||
|
||||
Implements short-term/durable/incubation/agency/candidate policy on shared
|
||||
graph.
|
||||
|
||||
### Phase 5: CultCache Store
|
||||
|
||||
Store shared graph docs as typed CultCache documents.
|
||||
|
||||
Commands should query memory graph status and context packets without Qdrant.
|
||||
|
||||
### Phase 6: Qdrant Cache
|
||||
|
||||
Embed shared graph nodes, edges, and summaries.
|
||||
|
||||
Qdrant candidate ranking feeds context-cut planning but never emits canonical
|
||||
memory payloads.
|
||||
|
||||
### Phase 7: Sleep And Repo Refresh
|
||||
|
||||
Sleep maintenance and repo scanning become profile-specific producers of typed
|
||||
memory graph operations.
|
||||
|
||||
### Phase 8: CultNet And Prompt Integration
|
||||
|
||||
Advertise memory graph read/mutation/context contracts through runtime-spine.
|
||||
Render context packets into role prompts and operator surfaces.
|
||||
|
||||
## First Implementable Ticket
|
||||
|
||||
Title: `Add shared EpiphanyMemoryGraph typed documents and pure validation`
|
||||
|
||||
Scope:
|
||||
|
||||
- Add shared memory graph document types.
|
||||
- Add stable id helpers.
|
||||
- Add pure validation for nodes, edges, summaries, anchors, and profile/lifecycle
|
||||
legality.
|
||||
- Add pure context-cut fixture tests.
|
||||
- Update repo/agent memory plans to point at this substrate.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- no Qdrant
|
||||
- no scanner
|
||||
- no sleep runner
|
||||
- no app-server route
|
||||
- no prompt integration
|
||||
|
||||
Verification:
|
||||
|
||||
```powershell
|
||||
cargo fmt --manifest-path .\epiphany-core\Cargo.toml
|
||||
cargo test --manifest-path .\epiphany-core\Cargo.toml --lib memory_graph
|
||||
cargo check --manifest-path .\epiphany-core\Cargo.toml
|
||||
```
|
||||
|
||||
Definition of done:
|
||||
|
||||
- The shared creature has a skeleton.
|
||||
- Repo modeling and agent memory can add profiles without duplicating graph,
|
||||
cache, freshness, or context-packet machinery.
|
||||
|
|
@ -39,7 +39,7 @@ remembers doctrine; the branch remembers the blade.
|
|||
- `epiphany-openai-runtime` worker result ingress now parses assistant output directly into typed role/reorient ingress structs selected from the durable launch document, stores nested `statePatch` and `selfPatch` as typed MessagePack in `EpiphanyRuntimeRoleWorkerResult`, and no longer passes worker result data through generic `serde_json::Value`. JSON survives there only as the OpenAI text-output edge before typed deserialization.
|
||||
- Agent self-memory review/application now has typed `AgentSelfPatch` core APIs. JSON-value selfPatch functions are compatibility wrappers for CLI/artifact ingress; native callers should parse once at the edge and then call `review_agent_self_patch_document` / `apply_agent_self_patch_document`. Role-result self-persistence no longer exports a JSON-value reviewer; the live API is `review_role_self_patch_document`.
|
||||
- `epiphany-openai-codex-spine` no longer uses Codex's Responses client, model-provider layer, `codex-api`, `codex-model-provider`, `codex-model-provider-info`, or `codex-protocol` for outbound model requests. It reaches `codex-login` only through `epiphany-openai-auth-spine` so credential loading/refresh stays Codex-compatible while request/status/event/runtime surfaces stay typed Epiphany documents.
|
||||
- `notes/repo-fractal-dataflow-cache-plan.md` is the current repo-model direction drawn from `Fractal Domains And The Cache That Bites`: the repo's semantic architecture/dataflow graph should be a typed CultCache document organ, with Qdrant storing rebuildable embeddings for nodes, edges, conservative summaries, and context cuts. Qdrant is a contribution cache, not truth. The roadmap's first implementable ticket is typed RepoModel documents plus pure `epiphany-core::repo_model` id/validation/freshness/context-cut tests; scanner, CultCache persistence, Qdrant, CultNet advertisement, and coding-lane integration come after the typed bones exist.
|
||||
- `notes/epiphany-memory-graph-unified-plan.md` is the current memory architecture direction drawn from `Fractal Domains And The Cache That Bites` plus the VoidBot memory scar tissue. The repo semantic architecture/dataflow graph and agent memory are one `EpiphanyMemoryGraph` substrate with repo, role, short-term, incubation, agency, candidate-intervention, and evidence profiles. Qdrant is a contribution cache for typed graph documents, not truth. The first implementable ticket is shared memory graph documents plus pure `epiphany-core::memory_graph` id/validation/freshness/context-cut tests; profile-specific scanners, sleep consolidation, CultCache persistence, Qdrant, CultNet advertisement, and prompt integration come after the shared skeleton exists. `notes/repo-fractal-dataflow-cache-plan.md` and `notes/agent-memory-fractal-cache-plan.md` are profile plans, not rival organs.
|
||||
- Durable Epiphany state types now live in root crate `epiphany-state-model`. `codex-protocol` re-exports them only so old rollout/API callers keep compiling; do not add new durable `Epiphany*` state structs under `vendor/codex`. `epiphany-core` imports the native model directly and no longer depends on `codex-protocol`. Pressure derives from native `EpiphanyTokenUsageSnapshot`; Codex token telemetry is mapped at the bridge. Codex rollout/event reconstruction now lives in `codex-core::epiphany_rollout`, where the Codex `RolloutItem` / `EventMsg` types belong. Exact retrieval path matching is native to `epiphany-core`, so `epiphany-core` has no direct vendored Codex crate dependency.
|
||||
- State-update contract proof and job launch/interrupt DTO ownership now live in native `epiphany-core`. The old giant `codex_thread.rs` Epiphany state-update test fixture was cut, and `epiphany-codex-bridge` no longer imports Epiphany job launch request types from `codex_core`; vendored `CodexThread` may call the native mutation/launch law for compatibility, but it must not regain ownership of state-update semantics, heartbeat launch-plan policy, or Epiphany DTOs.
|
||||
- MCP remains JSON because MCP is a JSON protocol. The latest processor cut only moved MCP refresh/OAuth/status/resource/tool-call route handling into `codex_message_processor/mcp_routes.rs`, reducing the root processor to about 8,977 lines; it did not claim MCP payloads are CultNet or make MCP an Epiphany-owned runtime contract.
|
||||
|
|
@ -96,6 +96,7 @@ remembers doctrine; the branch remembers the blade.
|
|||
- The heartbeat initiative pass is landed as a bounded tool seam. `state/agent-heartbeats.msgpack` tracks Self, Face, Imagination, Eyes, Body, Hands, Soul, and Life as Ghostlight-style initiative participants with arena, participant kind, speed, next-ready time, reaction bias, interrupt threshold, load, status, constraints, personality cooldown, mood cooldown, effective cooldown, adaptive pacing, history, and pending turns through `epiphany-core::EpiphanyHeartbeatStateEntry` and the native `epiphany-heartbeat-store` binary. Role dossiers set baseline timing; appraisal mood/anxiety bends it through urgency, arousal, thought pressure, guardedness, and reaction intensity, so Hands-like work pressure and high-need anxious lanes recover sooner without being heartbeaten while a prior turn is running. `epiphany-heartbeat-store pump` computes pressure from external urgency, mood/anxiety, reaction intensity, thought pressure, and current pending load, then chooses tempo plus target concurrency: calm can launch zero and sleep slow; alarm can fill most active lanes. Idle turns are for rumination: light thought shuffling, role-quality attention, and candidate selfPatch pressure. Sleep/dream cycle passes are the intended distillation window for durable self-memory and doctrine. JSON heartbeat state and Python wrapper state are gone; general CultCache schema sync, polyglot domain loading, and debug display tools belong in CultLib. This is a callable scheduler seam, not yet an always-on daemon; a heart valve, not a whole circulatory god.
|
||||
- The first Ghostlight-derived timing slice is landed in Epiphany, but Ghostlight is reference lineage rather than a sibling runtime to preserve. `epiphany-heartbeat-store init --profile ghostlight-scene --scene-id <id> --scene-participant <id|name|speed|reaction|threshold|constraints>` creates a typed CultCache MessagePack scene heartbeat store whose participants are `arena=scene`, `participantKind=character`; `tick` emits `ghostlight.initiative_schedule.v0` receipts with `scene_turn` actions and local-affordance basis. The generic Epiphany maintenance lanes are not auto-patched into scene stores.
|
||||
- The first Void-derived routine slice is landed in Epiphany, but VoidBot is reference lineage rather than a runtime dependency. `epiphany-heartbeat-store routine --store .\state\agent-heartbeats.msgpack --artifact-dir .\.epiphany-heartbeats --agent-store .\state\agents.msgpack` reads typed role dossiers, computes bounded memory resonance, maintains incubation themes, runs analytic and associative cognition lanes, writes bridge syntheses/saturation/tension, projects the active thought cluster through each role's Ghostlight-shaped personality vectors, derives participant-local reactions, applies personality/mood timing, advances the sleep/dream cycle, updates the typed heartbeat store, and emits an auditable `epiphany.void_routine.v0` receipt. The routine now also carries anti-loop physiology imported from Void's calmer brain: `noveltyToSelf`, `noveltyToRoom`, source coverage, saturation pressure, refractory cooling, and explicit permission to let a live unsaturated thought deepen without forcing novelty theater every pass. This mutates only heartbeat physiology fields: project truth and role memory mutation remain on their reviewed surfaces.
|
||||
- The next memory foundation must not build separate engines for repo graph and agent mind. Heartbeat cognition, role self-memory, repo architecture/dataflow, evidence scars, incubation, agency pressure, and context packets should converge through `EpiphanyMemoryGraph` profiles so one typed graph owns anchors, lifecycle, summaries, freshness, context cuts, and Qdrant embedding manifests.
|
||||
- Manual Codex-run rumination still has an explicit aftercare rule until Epiphany owns the full sleep consolidator. In the intended Epiphany cycle, sub-agents ruminate when idle and distill when they sleep; in this supervising Codex thread, a heartbeat/routine vigil is only physiology until a separate closing pass reviews the receipts and decides whether map, handoff, evidence, or role self-memory should change.
|
||||
- The Face public-surface pass is landed as a bounded lane. Face's role dossier lives in `state/agents.msgpack`; `epiphany_specialists.toml` gives it a VoidBot-heartbeat-derived prompt stripped of moderation authority; `state/face-discord.toml` and the native `epiphany-face-discord` binary enforce that Face may interact only through #aquarium. Missing channel id or token writes candidate chat artifacts instead of posting elsewhere. If a Face has a persona name/avatar, `post` uses the shared Discord webhook pipe so each Epiphany can speak with its own nickname and avatar without a separate bot identity.
|
||||
- The Void memory bridge is now native enough to keep the useful organs attached. `state/void-memory.toml` and `epiphany-void-memory` can check Void's Docker Postgres state spine, query Qdrant Discord-history and source/lore collections with the configured Ollama embedding model, and fetch raw archive context windows from Void's file-backed Discord archive. Native `epiphany-mvp-status` includes a `voidMemory` block so Aquarium/backend clients can inspect whether the mouth and memory are actually wired.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,33 @@
|
|||
# Repo Fractal Dataflow Cache Plan
|
||||
|
||||
## Correction: This Is A Memory Graph Profile
|
||||
|
||||
The repo fractal dataflow graph is not separate from agent memory. It is a
|
||||
domain profile of the same underlying machine.
|
||||
|
||||
The shared creature is an `EpiphanyMemoryGraph`:
|
||||
|
||||
```text
|
||||
typed domain claim
|
||||
-> node / edge / anchor / summary
|
||||
-> lifecycle state
|
||||
-> conservative context cut
|
||||
-> semantic embedding cache
|
||||
-> role/repo/Face/coordinator context packet
|
||||
```
|
||||
|
||||
The repo graph uses this substrate for architecture/dataflow/source domains.
|
||||
Agent memory uses it for role-local self memories, short-term residue,
|
||||
incubation, identity, agency pressure, and candidate interventions. Evidence
|
||||
and planning can also attach as graph domains.
|
||||
|
||||
Do not implement a standalone repo-model graph engine and then another
|
||||
agent-memory graph engine. That is duplicated authority. Build shared typed
|
||||
memory graph primitives, then add domain-specific policies.
|
||||
|
||||
This note remains the `repo_architecture` / `repo_dataflow` profile plan. The
|
||||
agent-memory profile plan lives in `notes/agent-memory-fractal-cache-plan.md`.
|
||||
|
||||
This note applies the lesson from
|
||||
`E:\Projects\gamecult-site\GameCult\Blog\fractal-domains-cache-that-bites.md`
|
||||
to Epiphany's repo-modeling architecture.
|
||||
|
|
@ -42,14 +70,15 @@ better coat.
|
|||
|
||||
## Desired Mechanism
|
||||
|
||||
Introduce an Epiphany-owned `RepoModel` organ:
|
||||
Add repo architecture/dataflow profiles to the shared
|
||||
`EpiphanyMemoryGraph` organ:
|
||||
|
||||
```text
|
||||
repo files / docs / manifests / build metadata / git history
|
||||
-> typed repo scanner
|
||||
-> typed code/document domains
|
||||
-> fractal architecture/dataflow graph documents
|
||||
-> conservative node and edge summaries
|
||||
-> shared memory graph architecture/dataflow nodes and edges
|
||||
-> conservative memory graph summaries
|
||||
-> Qdrant embeddings for nodes, edges, summaries, and candidate neighborhoods
|
||||
-> query planner chooses a stable context cut
|
||||
-> coding agent receives exact typed context packet
|
||||
|
|
@ -71,7 +100,8 @@ It makes selection fast. It does not become the source of truth.
|
|||
|
||||
## Durable State Stores
|
||||
|
||||
Keep truth in typed CultCache documents:
|
||||
Keep truth in typed CultCache documents. These are repo-profile projections of
|
||||
the shared memory graph documents, not a separate graph store:
|
||||
|
||||
- `epiphany.repo_model.snapshot`: repo identity, scan revision, source hashes,
|
||||
root domains, and model status.
|
||||
|
|
@ -156,12 +186,16 @@ must not be the only home of graph truth.
|
|||
Target module network:
|
||||
|
||||
- `epiphany-state-model`
|
||||
- owns typed document structs for repo model snapshots, nodes, edges,
|
||||
summaries, embedding manifests, context packets, freshness, and receipts.
|
||||
- owns shared memory graph document structs plus repo-profile public contract
|
||||
projections when they need CultNet/schema advertisement.
|
||||
|
||||
- `epiphany-core::repo_model`
|
||||
- owns domain mapping, graph grammar, summary construction, freshness,
|
||||
context-cut planning, validation, and patch proposal policy.
|
||||
- `epiphany-core::memory_graph`
|
||||
- owns shared graph grammar, lifecycle law, summary construction, freshness,
|
||||
context-cut planning, validation, and Qdrant embedding manifests.
|
||||
|
||||
- `epiphany-core::memory_graph::profiles::repo`
|
||||
- owns repo architecture/dataflow domain mapping, scanner policy, source-hash
|
||||
freshness, and reviewable patch proposal policy over the shared graph.
|
||||
|
||||
- `epiphany-core::retrieval`
|
||||
- remains text/chunk retrieval and low-level Qdrant/Ollama backend adapter
|
||||
|
|
@ -316,16 +350,16 @@ staleness, open questions, and required verification hooks.
|
|||
|
||||
### Rebuild
|
||||
|
||||
- Build `epiphany-core::repo_model` as the new architecture organ before adding
|
||||
more retrieval features.
|
||||
- Build typed repo-model CultNet contracts so Aquarium and coding lanes can ask
|
||||
for context packets without Codex JSON-RPC.
|
||||
- Build `epiphany-core::memory_graph` as the shared architecture/memory organ
|
||||
before adding more retrieval features.
|
||||
- Build typed memory-graph CultNet contracts so Aquarium and coding lanes can
|
||||
ask for repo-profile context packets without Codex JSON-RPC.
|
||||
|
||||
## First Slice
|
||||
|
||||
Do not start by embedding everything. First prove the typed shape.
|
||||
|
||||
1. Add typed repo-model document structs in `epiphany-state-model`:
|
||||
1. Add shared typed memory graph document structs in `epiphany-state-model`:
|
||||
- snapshot
|
||||
- domain
|
||||
- node
|
||||
|
|
@ -334,8 +368,9 @@ Do not start by embedding everything. First prove the typed shape.
|
|||
- embedding manifest
|
||||
- context packet
|
||||
- freshness
|
||||
2. Add `epiphany-core::repo_model` with pure in-memory validation and context
|
||||
cut planning over mocked documents.
|
||||
- repo profile kind/lifecycle hooks
|
||||
2. Add `epiphany-core::memory_graph` with repo-profile policy hooks plus pure
|
||||
in-memory validation and context cut planning over mocked documents.
|
||||
3. Add unit tests:
|
||||
- node ids are stable from domain/path/symbol
|
||||
- edges cannot reference missing nodes
|
||||
|
|
@ -389,7 +424,7 @@ The plan above is architecturally concrete, but not yet implementation-concrete.
|
|||
This roadmap is the cut order. Do not jump to Qdrant first. A fast lie is still
|
||||
a lie; it just gets to the wrong answer with better posture.
|
||||
|
||||
### Phase 0: Name The Contract
|
||||
### Phase 0: Name The Shared Contract
|
||||
|
||||
Objective: make the target type surface explicit before any scanner, model, or
|
||||
vector store can smuggle authority into the system.
|
||||
|
|
@ -402,6 +437,16 @@ Files:
|
|||
|
||||
Add typed state-model structs:
|
||||
|
||||
- `EpiphanyMemoryGraphSnapshot`
|
||||
- `EpiphanyMemoryDomain`
|
||||
- `EpiphanyMemoryNode`
|
||||
- `EpiphanyMemoryEdge`
|
||||
- `EpiphanyMemorySummary`
|
||||
- `EpiphanyMemoryEmbeddingManifest`
|
||||
- `EpiphanyMemoryFreshness`
|
||||
- `EpiphanyMemoryContextQuery`
|
||||
- `EpiphanyMemoryContextPacket`
|
||||
- `EpiphanyMemoryPatchCandidate`
|
||||
- `EpiphanyRepoModelSnapshot`
|
||||
- `EpiphanyRepoDomain`
|
||||
- `EpiphanyRepoNode`
|
||||
|
|
@ -436,31 +481,34 @@ Tests:
|
|||
|
||||
Exit criteria:
|
||||
|
||||
- The repo-model document vocabulary compiles without Qdrant.
|
||||
- The shared memory graph document vocabulary compiles without Qdrant.
|
||||
- Repo-profile aliases/projections can be represented without a second graph
|
||||
engine.
|
||||
- The contracts can be advertised without any scanner implementation.
|
||||
|
||||
### Phase 1: Pure Repo Model Core
|
||||
### Phase 1: Pure Memory Graph Core With Repo Profile
|
||||
|
||||
Objective: build the in-memory law engine that can validate and select graph
|
||||
context without touching disk, Qdrant, Ollama, Codex, or app-server routing.
|
||||
|
||||
Files:
|
||||
|
||||
- `epiphany-core/src/repo_model.rs`
|
||||
- `epiphany-core/src/repo_model/ids.rs`
|
||||
- `epiphany-core/src/repo_model/validation.rs`
|
||||
- `epiphany-core/src/repo_model/freshness.rs`
|
||||
- `epiphany-core/src/repo_model/context_cut.rs`
|
||||
- `epiphany-core/src/memory_graph.rs`
|
||||
- `epiphany-core/src/memory_graph/ids.rs`
|
||||
- `epiphany-core/src/memory_graph/validation.rs`
|
||||
- `epiphany-core/src/memory_graph/freshness.rs`
|
||||
- `epiphany-core/src/memory_graph/context_cut.rs`
|
||||
- `epiphany-core/src/memory_graph/profiles/repo.rs`
|
||||
- `epiphany-core/src/lib.rs`
|
||||
|
||||
Functions:
|
||||
|
||||
- `repo_model_domain_id(kind, path_or_name) -> String`
|
||||
- `repo_model_node_id(domain_id, kind, path, symbol) -> String`
|
||||
- `repo_model_edge_id(source_id, target_id, kind, code_refs) -> String`
|
||||
- `validate_repo_model_snapshot(snapshot) -> Vec<EpiphanyRepoModelValidationError>`
|
||||
- `derive_repo_model_freshness(snapshot, source_hashes, dirty_paths) -> EpiphanyRepoFreshness`
|
||||
- `plan_repo_context_cut(snapshot, query, budget) -> EpiphanyRepoContextPacket`
|
||||
- `memory_graph_domain_id(profile, kind, path_or_name) -> String`
|
||||
- `memory_graph_node_id(domain_id, kind, path, symbol) -> String`
|
||||
- `memory_graph_edge_id(source_id, target_id, kind, code_refs) -> String`
|
||||
- `validate_memory_graph_snapshot(snapshot) -> Vec<EpiphanyMemoryGraphValidationError>`
|
||||
- `derive_memory_graph_freshness(snapshot, source_hashes, dirty_paths) -> EpiphanyMemoryFreshness`
|
||||
- `plan_memory_graph_context_cut(snapshot, query, budget) -> EpiphanyMemoryContextPacket`
|
||||
|
||||
Tests:
|
||||
|
||||
|
|
@ -477,9 +525,10 @@ Tests:
|
|||
|
||||
Exit criteria:
|
||||
|
||||
- `cargo test --manifest-path .\epiphany-core\Cargo.toml --lib repo_model`
|
||||
- `cargo test --manifest-path .\epiphany-core\Cargo.toml --lib memory_graph`
|
||||
passes.
|
||||
- Repo context packets can be produced from fixture documents without Qdrant.
|
||||
- Repo-profile context packets can be produced from fixture documents without
|
||||
Qdrant.
|
||||
|
||||
### Phase 2: Deterministic Scanner
|
||||
|
||||
|
|
@ -816,15 +865,18 @@ objective
|
|||
|
||||
## First Implementable Ticket
|
||||
|
||||
Start here if the user says "build it."
|
||||
Do not start here directly anymore. Start with
|
||||
`notes/epiphany-memory-graph-unified-plan.md`.
|
||||
|
||||
Title: `Add typed RepoModel documents and pure context-cut validation`
|
||||
Title: `Add shared EpiphanyMemoryGraph typed documents and pure validation`
|
||||
|
||||
Scope:
|
||||
|
||||
- Add repo-model document structs to `epiphany-state-model/src/lib.rs`.
|
||||
- Add `epiphany-core/src/repo_model.rs` plus `ids`, `validation`,
|
||||
`freshness`, and `context_cut` submodules.
|
||||
- Add shared memory graph document structs to `epiphany-state-model/src/lib.rs`.
|
||||
- Add `epiphany-core/src/memory_graph.rs` plus document, id, validation,
|
||||
freshness, and context-cut submodules.
|
||||
- Include `repo_architecture` and `repo_dataflow` profile enums/policy hooks,
|
||||
but do not implement the scanner yet.
|
||||
- Export the pure APIs from `epiphany-core/src/lib.rs`.
|
||||
- Add fixture-only unit tests for stable ids, missing-edge validation, stale
|
||||
hash propagation, and parent-summary context cuts.
|
||||
|
|
@ -842,12 +894,12 @@ Verification:
|
|||
|
||||
```powershell
|
||||
cargo fmt --manifest-path .\epiphany-core\Cargo.toml
|
||||
cargo test --manifest-path .\epiphany-core\Cargo.toml --lib repo_model
|
||||
cargo test --manifest-path .\epiphany-core\Cargo.toml --lib memory_graph
|
||||
cargo check --manifest-path .\epiphany-core\Cargo.toml
|
||||
```
|
||||
|
||||
Definition of done:
|
||||
|
||||
- The repo-model organ has typed bones and pure tests.
|
||||
- The shared memory-graph organ has typed bones and pure tests.
|
||||
- The next ticket can add deterministic scanning without debating what a node,
|
||||
edge, summary, freshness record, or context packet is supposed to be.
|
||||
|
|
|
|||
Binary file not shown.
File diff suppressed because one or more lines are too long
Reference in a new issue