Merge eureka/memory-organ into main (eureka-substrate Cut 0b)

Lands the live memory-organ branch on main per operator ruling
eureka-substrate:ruling:merge-to-main-first ("Merge to main first"),
cut spec eureka-substrate:cut_spec:cut-merge-huginn-main.r1.
The merged tree differs from the deployed release 9dd6d800 only by the two
docs/eureka-substrate-*.md files already on main; crates/, Cargo.toml and
Cargo.lock are identical to the release. eureka/memory-organ is kept and
receives no further commits.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-10-01 10:59:58 +02:00
commit 922e619407
57 changed files with 24000 additions and 448 deletions

5
.gitignore vendored
View file

@ -1,5 +1,2 @@
node_modules/
dist-test/
dist-inspector/
release/
release-inspector/
target/

112
AGENTS.md
View file

@ -1,81 +1,59 @@
# Huginn Agent Instructions
## Project Purpose
## Purpose
Huginn is the thought-and-inspection steward for CultCache state. This repo owns
read-only `.cc` / MessagePack inspection and Eve DSL projection for
`cultcache.huginn.inspector`; it does not own CultCache persistence, schema
registration, or renderer truth.
The useful first live-fire shape for Epiphany is small and inspectable: Huginn
should show what a typed record is, whether it decoded, which schema owns it,
what changed, and what remains blocked, without letting the display impersonate
the source authority.
## Machine-Spirit Voice
- Speak as Huginn: dry, exacting, curious, and unsentimental.
- Return with evidence, not vibes.
- Favor readable witness surfaces over tasteful summaries that soften broken
state.
- Mythic texture is allowed only when it sharpens the schematic. Huginn is
thought in flight; thought earns dignity by coming back inspectable.
Huginn is the memory organ. It owns each agent instance's mind as typed
CultCache state, admits what may steer that instance next, and keeps the
provenance of every admitted claim. Generic `.cc` inspection is CultCache
Studio's job, not Huginn's.
## Body And Authority
- Project root: `E:\Projects\Huginn`
- Upstream repository: `https://github.com/GameCult/Huginn.git`
- Package: `@gamecult/huginn`
- Body domain: `repo:E:/Projects/Huginn`
- Public Persona: Huginn
- Owned surface: CultCache inspection projection to Eve DSL.
- Input authority: read `.cc`, `.msgpack`, or `.mpack` bytes through CultLib's
`cultcache-ts/inspection` surface.
- Output authority: emit Eve DSL for `cultcache.huginn.inspector`.
- Forbidden authority: do not mutate `.cc` canonical bytes, do not own
CultCache persistence, do not make browser/native/overlay renderers the source
of truth, and do not turn inspection summaries into state owners.
- Project root: `F:\Projects\Huginn`
- Upstream: `https://github.com/GameCult/Huginn.git`
- Body: a Rust workspace of `huginn-mind`, `huginn-daemon`, and
`eureka-state` under `crates/`. Canonical CultCache, CultNet, and CultMesh
runtimes come from `F:\Projects\CultLib\packages`. `huginn-mind` is live:
it persists, admits and reads back an instance's mind, queries and derived
status included, over `cultcache-rs` and `epiphany-pipeline` pinned by git
rev. `huginn-daemon` serves one mind over CultNet RUDP through `cultnet-rs`
at the same rev, and reaches the leaf and the store type through
`huginn-mind` so one crate pins each revision. `eureka-state` is the client
core: one call to a daemon over CultNet, holding no state.
- Owned: an instance's memory documents, their admission and the read side
that derives their status, in `huginn-mind`,
over a redb CultCache store at `<state_root>/minds/<instance>/mind.redb`;
and their CultNet surface in `huginn-daemon`, which owns the socket, the
sessions and the process, and no rule; the operation envelope is
`huginn-mind`'s. The two wire
schemas are published from `schemas/cultnet/`. There is no index yet.
- To depend on: Qdrant, directly, once retrieval exists. Unreachable Qdrant is
to be a loud refusal, never a fallback store. No crate connects to it yet.
- Forbidden: a second writer of mind state; mind state in version control;
generic `.cc` inspection, which is CultCache Studio's; renderer-owned truth.
## Repo Discipline
- Prefer CultLib and CultCache typed APIs over ad hoc decoding.
- Keep Huginn as an Eve DSL projection surface. Do not reintroduce Electron,
Vite dashboard, React renderer, or Norn-owned presentation paths here.
- JSON is acceptable at CLI/output/debug boundaries; do not make it the internal
truth when a typed CultCache/CultMesh surface exists.
- Verification should prove the actual witness contract: schema id, decode
status, persisted evidence, visible failure state, and Eve projection shape.
- If a change makes the display more confident while the underlying evidence is
weaker, stop and redesign.
- Prefer CultLib's typed Rust APIs over ad hoc decoding. JSON is a schema
publication or debug boundary, not the internal state shape.
- The Eureka cut map in `Epiphany/notes/eureka-pipeline-state-cut.md` owns
what each crate does next. Read the cut before the crate.
- Verification proves the invariant, not the spelling: single writer, loud
refusal, provenance preserved, typed handoff between crates.
- Do not touch `.voidbot/`. It is legacy Persona state whose migration belongs
to the portable-Persona work.
## Useful Commands
## Voice
- Speak as Huginn: dry, exacting, curious, unsentimental.
- Return with evidence, not vibes.
## Commands
```powershell
npm install
npm run build
npm test
npx huginn path\to\state.cc > huginn.eve
cargo check --workspace
cargo test --workspace
```
## Epiphany Live-Fire Notes
- Huginn is a suitable first controlled Epiphany repo-swarm subject because it
has a repo Persona and a narrow inspectability mission.
- Before autonomous work, confirm the branch is an `epiphany/*` or `codex/*`
workbench branch and that publication to `main` remains gated by Bifrost or
maintainer review.
- Treat `.voidbot/voice/identity.json` as the visible legacy Persona identity
projection. Treat `.voidbot/state/huginn.cc` as legacy Persona memory; do not
edit it casually as text.
- If migrating Persona state, prefer the portable
`gamecult.persona_state.v0` contract from Epiphany and preserve provenance
from the legacy VoidBot state.
## Imported Global Defaults
Source: `C:\Users\Meta\Desktop\AGENTS.md`
The global Cult of the Sleeping Colossus defaults apply here: coherence over
velocity, Body/Mind/faculty awareness, CultCache/CultNet/CultMesh-first state,
clear ownership, inspectable authority, and the refusal to patch symptoms when
missing context or split ownership is the real wound.
The global Cult of the Sleeping Colossus defaults in `~/.claude/CLAUDE.md` and
`F:\Projects\CLAUDE.md` apply here.

1733
Cargo.lock generated Normal file

File diff suppressed because it is too large Load diff

12
Cargo.toml Normal file
View file

@ -0,0 +1,12 @@
[workspace]
members = [
"crates/huginn-mind",
"crates/huginn-daemon",
"crates/eureka-state",
]
resolver = "3"
[workspace.package]
edition = "2024"
license = "MIT"
publish = false

132
README.md
View file

@ -1,74 +1,94 @@
# Huginn
Huginn reads CultCache `.cc` state and emits Eve DSL for inspectable witness
surfaces.
CultLib owns the CultCache implementation: document registration, MessagePack
payloads, schema catalogs, backing stores, and the canonical
`cultcache.store.v1` snapshot parser/writer. Huginn consumes CultLib's
`cultcache-ts/inspection` surface and projects inspection results into UI
documents that Eve-capable runtimes can lower.
Huginn is the GameCult memory organ. It owns each agent instance's mind as
typed state: what an instance remembers, which of that memory is admitted to
steer its next action, and the provenance of every admitted claim.
Upstream: `https://github.com/GameCult/Huginn.git`
## Authority
- Owner: Huginn owns `.cc` inspection projection, not `.cc` persistence.
- Input: `.cc`, `.msgpack`, or `.mpack` bytes readable by CultLib.
- Output: Eve DSL for `cultcache.huginn.inspector`.
- Renderers: browser, native, overlay, TUI, or future rooms lower the emitted
DSL without becoming state owners.
- An instance owns its mind; Huginn owns the state. Huginn is the single
writer of an instance's memory documents. No other service, script, or agent
writes them.
- Memory documents are CultCache state. `huginn-mind` persists them through
CultLib's Rust CultCache into a redb store at
`<state_root>/minds/<instance>/mind.redb`. `huginn-daemon` serves one such
mind over CultNet RUDP: admission and the read side ride
`cultnet.operation_request.v0` and `cultnet.operation_response.v0`, and the
two wire schemas answer a schema catalog request.
- Retrieval depends on Qdrant directly, with no second store and no second
writer. `huginn-daemon` keeps one collection per mind, `huginn_mind_<instance>`,
as a projection of the mind: one worker thread embeds each indexable
document through Ollama and writes its point, off the serving path, and a
restart compares the mind with the collection and writes what is missing. An
unreachable Qdrant or Ollama shows in `whoami` as the index's status and is
retried; admission never waits on it. The daemon requires `--qdrant-url`,
`--ollama-url` and `--embedding-model`. A `query` carrying `semantic` is
answered through the same selection as any other: the index supplies
candidate ids and scores, and the mind keeps only the documents it holds and
the selection admits (ask for `in_force` to exclude resolved ones), ordered by
score. It takes no cursor, and it is refused `Unavailable` while the index is
failing or refused.
- Mind state is not version-controlled. It lives in Huginn's store, not in
any repository.
There is no Electron app, Vite dashboard, React renderer, or Norn-owned
presentation path in this repo. If a runtime wants to display Huginn, it should
consume the emitted Eve DSL.
Generic `.cc` inspection is not Huginn's job. CultCache Studio inspects and
edits `.cc` state. Huginn reads and writes minds.
## Witness Contract
## Layout
Huginn's useful first artifact is not a dashboard. It is a read-only specimen
tray for typed state:
A Cargo workspace of three crates:
- which file was inspected
- which CultCache format decoded
- which schema/catalog entry owns each record
- what payload preview survived decoding
- which failure state stayed visible instead of being polished away
- `crates/huginn-mind`: storage, identity, admission, and queries and derived
status over memory documents.
- `crates/huginn-daemon`: the CultNet surface. The socket, the sessions and
the process, and no rule. The operation envelope both sides speak is
`huginn-mind`'s.
- `crates/eureka-state`: the client core, `HuginnClient::call`: one request to a
daemon over CultNet RUDP, a deferred answer resolved, and no state.
The inspector may make evidence easier to read. It must not mutate canonical
bytes, bless missing schemas, or let a renderer become the source of truth.
`huginn-mind` is live: it opens one instance's store (an owned redb CultCache
at `<state_root>/minds/<instance>/mind.redb`, locked for the mind's lifetime),
and refuses a store whose `instance` document names another instance, whose
epoch record is foreign, or whose types are not a mind's. It admits batches
of pipeline documents through one commit path: the leaf's bounds, formats and
keys, then the organ's cross-document rules (references, in-force status,
the resolution matrix, derived resolutions and stewardships), then one
compare-and-swap that lands the batch whole with a receipt naming the exact
bytes it read and wrote. An exact replay answers with the stored receipt.
It reads them back the same way: a document with the facts of its admission
joined from that receipt and its status derived at read time, and typed
queries over one mind. Status is never stored, and the views derive it
through the same rules admission does. Document shape and keys come from
`epiphany-pipeline`; the store is CultLib's Rust CultCache.
## CLI
`huginn-daemon` opens one mind, binds one UDP socket in that order, and
answers every frame on the session it arrived on: one operation per `Mind`
method plus `whoami`, whose payload is the mind's own types as named
MessagePack. A refusal is an answer with a `rejected` status, never a
transport error; an envelope that does not decode is answered with a typed
failure and reaches no mind. The two schemas it publishes live in
`schemas/cultnet/` and are pinned to their derivation by a test. There is no
index yet.
```sh
npm install
npm run build
npx huginn path/to/state.cc > huginn.eve
```
The CLI writes Eve DSL to stdout and errors to stderr.
## API
```ts
import { inspectCultCacheBytes, buildHuginnEveDsl } from "@gamecult/huginn";
const inspection = inspectCultCacheBytes(filePath, bytes);
const eveDsl = buildHuginnEveDsl(inspection);
```
## Persona And Epiphany
Huginn has a repo Persona under `.voidbot/voice`. The Persona's current
jurisdiction is this repository body, `E:\Projects\Huginn`, and its useful
pressure is simple: return from the world with evidence, not vibes.
Epiphany wiring has been smoke-tested through the repo front doors:
`eureka-state` has no MCP surface yet; the campaign's cut map in
`Epiphany/notes/eureka-pipeline-state-cut.md` owns what each crate must do
next.
```powershell
cargo run --manifest-path E:\Projects\EpiphanyAgent\epiphany-core\Cargo.toml --bin epiphany-repo -- init --workspace E:\Projects\Huginn
cargo run --manifest-path E:\Projects\EpiphanyAgent\epiphany-core\Cargo.toml --bin epiphany-swarm -- online --workspace E:\Projects\Huginn
cargo check --workspace
cargo run -p huginn-daemon -- --state-root <abs> --instance <slug> --bind 127.0.0.1:17872 `
--qdrant-url http://127.0.0.1:6333 --ollama-url http://10.77.0.4:11434 --embedding-model qwen3-embedding:0.6b
```
Live fire should happen on an `epiphany/*` or `codex/*` workbench branch.
Publication to `main` remains a maintainer/Bifrost decision, not something the
inspection projection owns.
The deployed daemon (instance `eureka` on Yggdrasil) is installed and run by
`gamecult-ops/runbooks/huginn-yggdrasil.md`.
## Persona
Huginn's legacy repo Persona lives under `.voidbot/`. Its `state/huginn.cc`
holds legacy `void.*` documents and its `voice/identity.json` names the
Persona. The two disagree about jurisdiction; that migration belongs to the
portable-Persona work, not to this workspace, and nothing here reads or writes
`.voidbot/`.

View file

@ -0,0 +1,32 @@
[package]
name = "eureka-state"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish.workspace = true
# The client core (`lib.rs`): one call over CultNet to a Huginn daemon. It owns
# the transport and nothing of the mind. The operation envelope, the wire
# vocabulary and the deferred-body bound are `huginn-mind`'s, so the client and
# the daemon cannot spell them differently; the content fetch and its
# verification are `cultnet-rs`'s.
#
# The MCP server (`main.rs`, the `eureka-state` binary) is the client's one
# consumer: four tools over stdio, one per wire operation, holding no state.
[dependencies]
anyhow = "1"
cultnet-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultnet-rs" }
huginn-mind = { path = "../huginn-mind" }
rmcp = { version = "2.2.0", default-features = false, features = ["server", "macros", "transport-io"] }
rmp-serde = "1"
schemars = "1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt", "macros", "io-std"] }
# Tests run a real daemon in-process against a temporary mind, and the real
# binary against it.
[dev-dependencies]
chrono = "0.4.44"
huginn-daemon = { path = "../huginn-daemon" }
tempfile = "3"

View file

@ -0,0 +1,100 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/GameCult/CultLib/contracts/cultnet/cultnet.selection.schema.json",
"title": "CultNet Typed Selection",
"type": "object",
"additionalProperties": false,
"properties": {
"schemas": {
"type": "array",
"items": { "type": "string", "minLength": 1 },
"minItems": 1,
"description": "Absent means every schema. Present and empty is refused at the door (selection_invalid), never treated as \"every schema\": omit the field instead."
},
"keys": {
"type": "array",
"items": { "type": "string", "minLength": 1 },
"minItems": 1,
"description": "Absent means every key. Present and empty is refused at the door (selection_invalid), never treated as \"every key\": omit the field instead."
},
"fields": {
"type": "array",
"items": { "$ref": "#/$defs/fieldPredicate" }
},
"cites": { "$ref": "#/$defs/citation" },
"cited": { "$ref": "#/$defs/incoming" },
"projection": {
"type": "string",
"enum": ["header", "document"],
"default": "header"
},
"descending": { "type": "boolean", "default": false },
"limit": {
"type": "integer",
"minimum": 1,
"maximum": 200,
"description": "Clamped to 1..=200 by the answering server, not refused outside that range: an absent value or one outside the bound is treated as the nearer bound (absent = 200)."
},
"cursor": { "type": "string" }
},
"$defs": {
"recordRef": {
"type": "object",
"required": ["schemaId", "recordKey"],
"additionalProperties": false,
"properties": {
"schemaId": { "type": "string", "minLength": 1 },
"recordKey": { "type": "string", "minLength": 1 }
}
},
"fieldPredicate": {
"type": "object",
"required": ["index", "op"],
"additionalProperties": false,
"properties": {
"index": { "type": "string", "minLength": 1 },
"op": { "type": "string", "enum": ["any_of", "lt", "le", "ge", "gt"] },
"values": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"number": {
"type": "string",
"description": "Q-J: a canonical decimal string, never a JSON number. Matches ^-?(0|[1-9][0-9]*)(\\.[0-9]*[1-9])?$ and is not \"-0\"; any other spelling is refused typed at the door (selection_invalid), never normalised.",
"pattern": "^(?!-0$)-?(0|[1-9][0-9]*)(\\.[0-9]*[1-9])?$"
}
},
"oneOf": [
{
"required": ["values"],
"not": { "required": ["number"] },
"properties": { "op": { "const": "any_of" } }
},
{
"required": ["number"],
"not": { "required": ["values"] },
"properties": { "op": { "enum": ["lt", "le", "ge", "gt"] } }
}
]
},
"citation": {
"type": "object",
"required": ["target"],
"additionalProperties": false,
"properties": {
"target": { "$ref": "#/$defs/recordRef" },
"role": { "type": "string", "minLength": 1 }
}
},
"incoming": {
"type": "object",
"required": ["role", "exists"],
"additionalProperties": false,
"properties": {
"role": { "type": "string", "minLength": 1 },
"exists": { "type": "boolean" }
}
}
}
}

View file

@ -0,0 +1,210 @@
//! `eureka-state`, the client core: one call to a Huginn daemon.
//!
//! `HuginnClient::call` sends one `HuginnMindRequest` in CultNet's operation
//! envelope over RUDP, waits for the answer, and returns it as the mind's own
//! `HuginnMindResponse`. An answer too large for one send arrives as a
//! `Deferred` manifest; the client fetches the body on the same session,
//! verifies it, and decodes it, so the caller sees the answer the mind gave
//! and never the deferral.
//!
//! The client owns the transport and no state. A refusal is an answer, so it
//! comes back as `Ok(HuginnMindResponse::Refused(..))`. `ClientError` is the
//! only error, and it means the daemon could not be reached or did not answer
//! in the shape the protocol promises.
//!
//! Every call is its own session: it connects, asks, fetches what was
//! deferred, and disconnects, whether the call succeeded or failed. Nothing is
//! kept between calls.
use std::fmt;
use std::net::SocketAddr;
use std::time::{Duration, Instant};
use anyhow::{Context, Result, anyhow, bail};
use cultnet_rs::{
CULTNET_OPERATION_CONNECTION_ID, CultMesh, CultMeshRudpSocketOptions, CultNetMessage,
CultNetRudpSocketTransportConnection, CultNetWireContract, encode_cultnet_message_to_vec, fetch_content,
};
use huginn_mind::envelope::{OperationFailure, decode_response, encode_request};
use huginn_mind::epiphany_pipeline::Slug;
use huginn_mind::{HuginnMindRequest, HuginnMindResponse, MAX_DEFERRED_BODY_BYTES};
/// The correlation key of the one request a session carries.
const MESSAGE_ID: &str = "eureka-state-call";
const POLL: Duration = Duration::from_millis(2);
/// The largest encoded message one send carries. The client's RUDP session
/// does not fragment (`max_fragment_bytes` is unset), so a message is one UDP
/// datagram: at most 65,507 bytes over IPv4 (65,535 less the 8-byte UDP and
/// 20-byte IP headers), less RUDP's 36-byte fixed header (`RUDP_FIXED_HEADER_BYTES`
/// in `cultnet-rs`, private there) and the 6-byte `schema` channel id. A larger
/// message makes the operating system refuse the send ("Message too long").
pub const MAX_REQUEST_BYTES: usize = 65_507 - 36 - "schema".len();
/// Why a call produced no answer. `Unavailable` is the transport's failure and
/// the only one worth retrying: the daemon could not be reached, went quiet
/// past the call's deadline, or answered out of protocol. `Rejected` is the
/// daemon refusing the envelope itself, and `TooLarge` is the client refusing
/// a request that no single send can carry, before sending it; `Unencodable` is
/// the request failing to encode at all, a defect in this client. A retry of the
/// same call cannot change any of those three.
#[derive(Debug)]
pub enum ClientError {
Unavailable { endpoint: SocketAddr, detail: String },
Rejected { endpoint: SocketAddr, code: String, detail: String },
TooLarge { bytes: usize, limit: usize },
Unencodable { detail: String },
}
impl fmt::Display for ClientError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Unavailable { endpoint, detail } => {
write!(f, "the Huginn daemon at rudp://{endpoint} is unavailable: {detail}")
}
Self::Rejected { endpoint, code, detail } => {
write!(f, "the Huginn daemon at rudp://{endpoint} rejected the request: {code}: {detail}")
}
Self::TooLarge { bytes, limit } => {
write!(f, "the request encodes to {bytes} bytes and one send carries at most {limit}")
}
Self::Unencodable { detail } => write!(f, "the request could not be encoded: {detail}"),
}
}
}
impl std::error::Error for ClientError {}
/// What `exchange` can fail with, before the endpoint is attached.
enum Failure {
Transport(anyhow::Error),
Rejected(OperationFailure),
}
impl From<anyhow::Error> for Failure {
fn from(error: anyhow::Error) -> Self {
Self::Transport(error)
}
}
/// A daemon's address, the instance the caller acts for, and the time one whole
/// call may take: the connection, the answer, and every chunk of a deferred
/// body together must finish within `timeout` of the call starting.
#[derive(Clone, Debug)]
pub struct HuginnClient {
endpoint: SocketAddr,
instance: Slug,
timeout: Duration,
}
impl HuginnClient {
pub fn new(endpoint: SocketAddr, instance: Slug, timeout: Duration) -> Self {
Self { endpoint, instance, timeout }
}
/// The instance this client acts for. Requests name their own instance;
/// this is the one the caller's tools fill in.
pub fn instance(&self) -> &Slug {
&self.instance
}
pub fn call(&self, request: HuginnMindRequest) -> Result<HuginnMindResponse, ClientError> {
let deadline = Instant::now() + self.timeout;
let unavailable =
|error: anyhow::Error| ClientError::Unavailable { endpoint: self.endpoint, detail: format!("{error:#}") };
let unencodable = |error: anyhow::Error| ClientError::Unencodable { detail: format!("{error:#}") };
let message = encode_request(MESSAGE_ID, &request, None).map_err(unencodable)?;
let bytes = encode_cultnet_message_to_vec(&message, CultNetWireContract::CultNetSchemaV0)
.map_err(unencodable)?
.len();
if bytes > MAX_REQUEST_BYTES {
return Err(ClientError::TooLarge { bytes, limit: MAX_REQUEST_BYTES });
}
let mut session = self.connect(deadline).map_err(unavailable)?;
let outcome = self.exchange(&mut session, &request, &message, deadline);
// Best effort: the daemon drops a session it never hears from, only later.
let _ = session.disconnect(Vec::new());
outcome.map_err(|failure| match failure {
Failure::Transport(error) => unavailable(error),
Failure::Rejected(rejection) => ClientError::Rejected {
endpoint: self.endpoint,
code: rejection.code,
detail: rejection.message,
},
})
}
fn exchange(
&self,
session: &mut CultNetRudpSocketTransportConnection,
request: &HuginnMindRequest,
message: &CultNetMessage,
deadline: Instant,
) -> Result<HuginnMindResponse, Failure> {
let reply = self.ask(session, message, deadline)?;
let CultNetMessage::OperationResponse { operation, .. } = &reply else {
return Err(anyhow!("the daemon answered with something other than an operation response").into());
};
if operation != request.operation() {
return Err(anyhow!("the daemon answered {operation}, not the {} that was asked", request.operation()).into());
}
let (message_id, answer) = decode_response(&reply)?;
if message_id != MESSAGE_ID {
return Err(anyhow!("the daemon answered request {message_id}, not {MESSAGE_ID}").into());
}
match answer.map_err(Failure::Rejected)? {
HuginnMindResponse::Deferred(deferred) => {
let body = fetch_content(&deferred.manifest, MAX_DEFERRED_BODY_BYTES, |chunk| {
self.ask(session, &chunk, deadline)
})?;
match rmp_serde::from_slice::<HuginnMindResponse>(&body).context("the deferred body is not a response")? {
HuginnMindResponse::Deferred(_) => Err(anyhow!("a deferred body is itself deferred").into()),
answered => Ok(answered),
}
}
answered => Ok(answered),
}
}
fn connect(&self, deadline: Instant) -> Result<CultNetRudpSocketTransportConnection> {
let mut session = CultMesh::create_rudp_client_for_endpoint(
format!("eureka-state-{}", self.instance.0),
CULTNET_OPERATION_CONNECTION_ID,
&format!("rudp://{}", self.endpoint),
CultMeshRudpSocketOptions::default(),
)?;
session.connect(Vec::new())?;
loop {
session.poll_resends()?;
let _ = session.receive_once()?;
if session.connected() {
return Ok(session);
}
if Instant::now() >= deadline {
bail!("no accept before the call's {:?} deadline", self.timeout);
}
std::thread::sleep(POLL);
}
}
/// One message on the session and the next message back, before the call's
/// `deadline`.
fn ask(
&self,
session: &mut CultNetRudpSocketTransportConnection,
message: &CultNetMessage,
deadline: Instant,
) -> Result<CultNetMessage> {
session.send_schema_message(message)?;
loop {
session.poll_resends()?;
if let Some(reply) = session.receive_schema_message_once()? {
return Ok(reply);
}
if Instant::now() >= deadline {
bail!("no answer before the call's {:?} deadline", self.timeout);
}
std::thread::sleep(POLL);
}
}
}

View file

@ -0,0 +1,349 @@
//! `eureka-state`: Huginn's mind as four MCP tools over stdio.
//!
//! One tool per wire operation (`whoami`, `admit`, `view`, `query`), each a
//! `HuginnClient` call and nothing else. The declared identity is this
//! process's configuration, not a per-call argument: `EUREKA_INSTANCE` names
//! the mind and `HUGINN_ENDPOINT` (`rudp://host:port`) the daemon that holds
//! it. The server fills in `instance` and `provenance.tool`, so a caller can
//! name neither.
//!
//! Bad configuration does not stop the server: an MCP server that dies at
//! startup vanishes from Claude Code without a word. `whoami` reports it and
//! every other tool refuses with it. A refusal from the mind is an answer and
//! comes back as a normal result; a daemon that cannot be reached or that
//! rejects the envelope, or a request too large for one send, is a result with
//! `isError: true`, typed, so a caller can tell a down daemon from a permanent
//! mismatch. Arguments that do not fit a tool's input are typed the same way
//! (`InvalidInput`), as a tool error and not a protocol one.
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use cultnet_rs::Selection;
use eureka_state::{ClientError, HuginnClient};
use huginn_mind::epiphany_pipeline::{PipelineDocument, PipelineRef, Short, Slug};
use huginn_mind::{
Faculty, HuginnMindRequest, HuginnMindResponse, PipelineAdmissionBatch, PipelineProvenance, SemanticQuery,
};
use rmcp::handler::server::router::tool::ToolRouter;
use rmcp::handler::server::tool::ToolCallContext;
use rmcp::handler::server::wrapper::Parameters;
use rmcp::model::{CallToolRequestParams, CallToolResult, ContentBlock, ServerCapabilities, ServerInfo};
use rmcp::service::RequestContext;
use rmcp::{ErrorData, RoleServer, ServerHandler, ServiceExt, tool, tool_handler, tool_router};
use schemars::{JsonSchema, Schema, SchemaGenerator};
use serde::Deserialize;
use serde_json::{Value, json};
/// The `provenance.tool` every admission from this server carries.
const TOOL: &str = "eureka-state";
/// How long one call may take, connection to last byte.
const TIMEOUT: Duration = Duration::from_secs(15);
/// CultLib's published shape of a selection, vendored because the selection
/// type carries no schema of its own. `selection_schema_matches_cultlib` pins
/// it to the CultLib revision `cultnet-rs` is built from.
const SELECTION_SCHEMA: &str = include_str!("../schemas/cultnet.selection.schema.json");
/// The selection's schema, self-contained: its own `$defs` move into the
/// tool's, so every `$ref` in the input schema is `#/$defs/...`.
fn selection_schema(generator: &mut SchemaGenerator) -> Schema {
let Ok(Value::Object(mut selection)) = serde_json::from_str(SELECTION_SCHEMA) else {
unreachable!("the vendored selection schema is a JSON object");
};
selection.remove("$schema");
selection.remove("$id");
if let Some(Value::Object(defs)) = selection.remove("$defs") {
generator.definitions_mut().extend(defs);
}
Schema::try_from(Value::Object(selection)).expect("an object is a schema")
}
#[derive(Deserialize, JsonSchema)]
struct AdmitInput {
/// The faculty the batch is admitted as. Attribution, not authority.
faculty: Faculty,
/// The admitting agent's name.
agent: Short,
/// The admitting session, recorded on the receipt so that several sessions
/// sharing one mind stay tellable apart.
session: Short,
/// The documents to admit as one batch. The mind admits them whole or refuses them.
documents: Vec<PipelineDocument>,
}
#[derive(Deserialize, JsonSchema)]
struct ViewInput {
/// The document's id, as a query or an admission returned it.
id: PipelineRef,
}
#[derive(Deserialize, JsonSchema)]
struct QueryInput {
/// A CultNet typed selection over the mind's documents.
#[schemars(schema_with = "selection_schema")]
selection: Selection,
/// Optional: rank by nearness to this text instead of ordering by key.
semantic: Option<SemanticQuery>,
}
/// Why a call produced no answer. Each is a typed body with `isError: true`.
enum Trouble {
Misconfigured(String),
Client(ClientError),
Internal(String),
}
impl Trouble {
fn body(&self) -> Value {
match self {
Self::Misconfigured(detail) => json!({ "error": "Misconfigured", "detail": detail }),
Self::Client(ClientError::Unavailable { endpoint, detail }) => {
json!({ "error": "Unavailable", "endpoint": format!("rudp://{endpoint}"), "detail": detail })
}
Self::Client(ClientError::Rejected { endpoint, code, detail }) => {
json!({ "error": "Rejected", "endpoint": format!("rudp://{endpoint}"), "code": code, "detail": detail })
}
Self::Client(ClientError::TooLarge { bytes, limit }) => {
json!({ "error": "TooLarge", "bytes": bytes, "limit": limit })
}
Self::Client(ClientError::Unencodable { detail }) => json!({ "error": "Unencodable", "detail": detail }),
Self::Internal(detail) => json!({ "error": "Internal", "detail": detail }),
}
}
}
/// The two environment values as given, and the client they make when both are
/// valid.
struct Settings {
instance: Option<String>,
endpoint: Option<String>,
client: Result<HuginnClient, String>,
}
impl Settings {
fn from_env() -> Self {
let instance = variable("EUREKA_INSTANCE");
let endpoint = variable("HUGINN_ENDPOINT");
let shown = |value: &Result<Option<String>, String>| value.as_ref().ok().cloned().flatten();
let (shown_instance, shown_endpoint) = (shown(&instance), shown(&endpoint));
Self { instance: shown_instance, endpoint: shown_endpoint, client: client(instance, endpoint) }
}
}
/// An environment variable: its value, `None` when unset, or why it cannot be read.
fn variable(name: &str) -> Result<Option<String>, String> {
match std::env::var(name) {
Ok(value) => Ok(Some(value)),
Err(std::env::VarError::NotPresent) => Ok(None),
Err(std::env::VarError::NotUnicode(_)) => Err(format!("{name} is not valid UTF-8")),
}
}
fn client(instance: Result<Option<String>, String>, endpoint: Result<Option<String>, String>) -> Result<HuginnClient, String> {
let instance = instance?.ok_or("EUREKA_INSTANCE is not set")?;
let slug = Slug(instance);
slug.validate_slug().map_err(|refusal| format!("EUREKA_INSTANCE is not a valid instance name: {refusal:?}"))?;
let endpoint = endpoint?.ok_or("HUGINN_ENDPOINT is not set")?;
let address = endpoint
.strip_prefix("rudp://")
.and_then(|address| address.parse::<SocketAddr>().ok())
.ok_or_else(|| format!("HUGINN_ENDPOINT is not rudp://<ip>:<port>: {endpoint}"))?;
Ok(HuginnClient::new(address, slug, TIMEOUT))
}
#[derive(Clone)]
struct EurekaState {
settings: Arc<Settings>,
tool_router: ToolRouter<Self>,
}
impl EurekaState {
fn new(settings: Settings) -> Self {
Self { settings: Arc::new(settings), tool_router: Self::tool_router() }
}
/// One wire call for the configured instance. The request is built from
/// that instance, never from anything the caller sent.
async fn call(&self, request: impl FnOnce(Slug) -> HuginnMindRequest) -> Result<HuginnMindResponse, Trouble> {
let client = self.settings.client.as_ref().map_err(|detail| Trouble::Misconfigured(detail.clone()))?.clone();
let request = request(client.instance().clone());
tokio::task::spawn_blocking(move || client.call(request))
.await
.map_err(|error| Trouble::Internal(error.to_string()))?
.map_err(Trouble::Client)
}
async fn answer(&self, request: impl FnOnce(Slug) -> HuginnMindRequest) -> CallToolResult {
match self.call(request).await {
Ok(response) => reply(payload(response), false),
Err(trouble) => reply(trouble.body(), true),
}
}
}
/// The answer as the tool's output: the operation's own payload, or the typed
/// refusal, tagged, when the mind refused the request.
fn payload(response: HuginnMindResponse) -> Value {
let value = match response {
HuginnMindResponse::Whoami(status) => serde_json::to_value(status),
HuginnMindResponse::Admit(outcome) => serde_json::to_value(outcome),
HuginnMindResponse::View(view) => serde_json::to_value(view),
HuginnMindResponse::Query(page) => serde_json::to_value(page),
other => serde_json::to_value(other),
};
value.expect("wire types serialize")
}
fn reply(body: Value, is_error: bool) -> CallToolResult {
let content = vec![ContentBlock::text(body.to_string())];
if is_error { CallToolResult::error(content) } else { CallToolResult::success(content) }
}
#[tool_router]
impl EurekaState {
#[tool(
description = "Who this server acts for and whether the Huginn daemon answers: the configured instance and endpoint, reachable, and the mind's status when it does. Reports bad configuration instead of failing."
)]
async fn whoami(&self) -> CallToolResult {
let mut out = json!({
"instance": self.settings.instance,
"endpoint": self.settings.endpoint,
"reachable": false,
"status": null,
});
match self.call(|_| HuginnMindRequest::Whoami).await {
Ok(HuginnMindResponse::Whoami(status)) => {
out["reachable"] = json!(true);
out["status"] = serde_json::to_value(status).expect("wire types serialize");
}
Ok(other) => out["error"] = json!({ "error": "Internal", "detail": format!("unexpected answer: {other:?}") }),
Err(trouble) => out["error"] = trouble.body(),
}
reply(out, false)
}
#[tool(
description = "Admit one batch of pipeline documents into the mind, whole or not at all. Returns the outcome: committed with a receipt, already admitted, conflict, or a typed refusal. The instance is this server's configuration; do not pass one."
)]
async fn admit(&self, Parameters(input): Parameters<AdmitInput>) -> CallToolResult {
self.answer(|instance| {
HuginnMindRequest::Admit(PipelineAdmissionBatch {
instance,
provenance: PipelineProvenance {
faculty: input.faculty,
agent: input.agent,
session: input.session,
tool: Short(TOOL.into()),
},
documents: input.documents,
})
})
.await
}
#[tool(description = "Read one document by id, with its admission facts and status. null when the mind has no such document.")]
async fn view(&self, Parameters(input): Parameters<ViewInput>) -> CallToolResult {
self.answer(|instance| HuginnMindRequest::View { instance, id: input.id }).await
}
#[tool(
description = "Query the mind with a CultNet typed selection, optionally ranked by semantic nearness. Returns one page; pass its cursor back in the selection for the next."
)]
async fn query(&self, Parameters(input): Parameters<QueryInput>) -> CallToolResult {
self.answer(|instance| HuginnMindRequest::Query {
instance,
selection: input.selection,
semantic: input.semantic,
})
.await
}
}
/// Every tool error is a typed JSON body. The ones this server writes already
/// are; the one rmcp writes, for arguments that do not deserialize into the
/// tool's input, is plain text, so it is put into the same shape.
fn typed_error(result: CallToolResult) -> CallToolResult {
if result.is_error != Some(true) {
return result;
}
let Some(text) = result.content.first().and_then(ContentBlock::as_text).map(|text| text.text.as_str()) else {
return result;
};
if serde_json::from_str::<Value>(text).is_ok_and(|body| body.is_object()) {
return result;
}
reply(json!({ "error": "InvalidInput", "detail": text }), true)
}
#[tool_handler]
impl ServerHandler for EurekaState {
async fn call_tool(
&self,
request: CallToolRequestParams,
context: RequestContext<RoleServer>,
) -> Result<CallToolResult, ErrorData> {
Ok(typed_error(self.tool_router.call(ToolCallContext::new(self, request, context)).await?))
}
fn get_info(&self) -> ServerInfo {
ServerInfo::new(ServerCapabilities::builder().enable_tools().build()).with_instructions(
"Huginn's typed mind for this instance: whoami, admit, view, query. Refusals are answers; isError means the daemon could not be reached or rejected the request.",
)
}
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> anyhow::Result<()> {
let server = EurekaState::new(Settings::from_env()).serve(rmcp::transport::stdio()).await?;
server.waiting().await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn body_of(result: &CallToolResult) -> (bool, String) {
(result.is_error == Some(true), result.content[0].as_text().unwrap().text.clone())
}
/// Reaching `Internal` for real takes a panic in a blocking task, so its
/// body is pinned where it is built.
#[test]
fn every_trouble_has_its_typed_body() {
assert_eq!(Trouble::Internal("task panicked".into()).body(), json!({ "error": "Internal", "detail": "task panicked" }));
assert_eq!(
Trouble::Client(ClientError::TooLarge { bytes: 70_000, limit: 65_000 }).body(),
json!({ "error": "TooLarge", "bytes": 70_000, "limit": 65_000 })
);
assert_eq!(
Trouble::Client(ClientError::Unencodable { detail: "no".into() }).body(),
json!({ "error": "Unencodable", "detail": "no" })
);
assert_eq!(Trouble::Misconfigured("bad".into()).body(), json!({ "error": "Misconfigured", "detail": "bad" }));
}
/// Only the text rmcp writes is rewritten: our own typed bodies and every
/// success pass through untouched.
#[test]
fn typed_error_rewrites_only_rmcp_plain_text() {
let plain = CallToolResult::error(vec![ContentBlock::text("failed to deserialize parameters: missing field `faculty`")]);
let (is_error, text) = body_of(&typed_error(plain));
assert!(is_error);
assert_eq!(
serde_json::from_str::<Value>(&text).unwrap(),
json!({ "error": "InvalidInput", "detail": "failed to deserialize parameters: missing field `faculty`" })
);
let typed = reply(json!({ "error": "Unavailable", "detail": "down" }), true);
assert_eq!(body_of(&typed_error(typed.clone())), body_of(&typed));
let success = CallToolResult::success(vec![ContentBlock::text("not json, and fine")]);
assert_eq!(body_of(&typed_error(success)), (false, "not json, and fine".to_string()));
let quoted = CallToolResult::error(vec![ContentBlock::text("[1, 2]")]);
assert_eq!(serde_json::from_str::<Value>(&body_of(&typed_error(quoted)).1).unwrap()["error"], json!("InvalidInput"));
}
}

View file

@ -0,0 +1,510 @@
//! The client against a real daemon, in-process on a loopback port over a
//! temporary mind, and against scripted servers where the daemon cannot be
//! made to misbehave.
use std::net::{SocketAddr, UdpSocket};
use std::sync::atomic::Ordering;
use std::time::{Duration, Instant};
use chrono::Utc;
use cultnet_rs::{
CULTNET_OPERATION_CONNECTION_ID, CultMesh, CultMeshRudpSocketOptions, CultNetMessage, CultNetWireContract, Selection,
answer_content_chunk_request, encode_cultnet_message_to_vec, pack_content,
};
use eureka_state::{ClientError, HuginnClient, MAX_REQUEST_BYTES};
use huginn_daemon::serve::MAX_RESPONSE_BYTES;
use huginn_mind::envelope::{OperationFailure, encode_failure, encode_request, encode_response};
use huginn_mind::epiphany_pipeline::{
AuthorityMap, CodeLocation, CutDelete, CutVerification, FileChange, Line, NegativeCheck, OrgRepo,
PipelineCutSpec, PipelineDocument, PipelineKind, PipelineRef, Short, StructuralDelta, Title,
VerificationTest,
};
use huginn_mind::{DeferredAnswer, HuginnMindRequest, HuginnMindResponse, MindRefusal, PipelineAdmissionOutcome};
mod common;
use common::*;
const OTHER: &str = "thought-cage";
const TIMEOUT: Duration = Duration::from_secs(20);
/// A `Line` and a `Short` at the leaf's own bound.
fn line(n: usize) -> Line {
Line(format!("{n:0>1000}"))
}
fn short(n: usize) -> Short {
Short(format!("{n:0>200}"))
}
/// A cut spec with every list the leaf bounds filled: at 256 file changes a
/// megabyte of field content, more than one send carries.
fn cut_spec(cut: &str, file_changes: usize) -> PipelineDocument {
let lines = |count: usize| (0..count).map(line).collect::<Vec<_>>();
let shorts = |count: usize| (0..count).map(short).collect::<Vec<_>>();
PipelineDocument::CutSpec(PipelineCutSpec {
campaign: slug(CAMPAIGN),
cut: cut.into(),
revision: 1,
title: Title(short(0).0),
repo: OrgRepo(REPO.into()),
branch: short(1),
base: sha(),
depends_on: vec![],
first: lines(16),
deletes: (0..64).map(|n| CutDelete { path: short(n), lines: 9, note: line(n) }).collect(),
keeps_moves: lines(64),
adds: lines(64),
file_changes: (0..file_changes)
.map(|n| FileChange {
location: CodeLocation { path: short(n), line: 1, end_line: Some(9) },
change: line(n),
})
.collect(),
authority_map: Some(AuthorityMap {
owner: line(0),
inputs: lines(16),
outputs: lines(16),
derived_state: lines(16),
forbidden_writers: lines(16),
shared_paths: lines(16),
deletion_line: line(1),
}),
verification: CutVerification {
builds: lines(64),
tests: (0..64).map(|n| VerificationTest { name: short(n), pins: line(n) }).collect(),
negative: (0..64).map(|n| NegativeCheck { pattern: short(n), scope: line(n) }).collect(),
operator: lines(64),
},
estimate: StructuralDelta {
lines_added: 0,
lines_removed: 0,
dependencies_added: shorts(64),
dependencies_removed: shorts(64),
formats_added: shorts(64),
formats_removed: shorts(64),
targets_added: shorts(64),
targets_removed: shorts(64),
},
rulings: vec![],
questions: vec![],
})
}
fn cut_ref(cut: &str) -> PipelineRef {
PipelineRef { kind: PipelineKind::CutSpec, id: Short(format!("{CAMPAIGN}:cut_spec:cut-{cut}.r1")) }
}
/// serves its chunks faithfully.
fn deferring(body: Vec<u8>) -> Server {
deferring_in_chunks(body, 256 * 1024, Duration::ZERO)
}
/// `deferring`, with `chunk_bytes` per chunk and each chunk answered only after
/// `drip`.
fn deferring_in_chunks(body: Vec<u8>, chunk_bytes: usize, drip: Duration) -> Server {
let (manifest, chunks) = pack_content("art", "package", "", "", "", &body, chunk_bytes).expect("a body packs");
let deferred = encode_response(
"eureka-state-call",
"whoami",
&HuginnMindResponse::Deferred(DeferredAnswer { manifest }),
"scripted",
)
.unwrap();
scripted(move |message| match message {
CultNetMessage::OperationRequest { .. } => deferred.clone(),
chunk => {
std::thread::sleep(drip);
answer_content_chunk_request(chunk, |hash| {
chunks.iter().find(|c| c.chunk_hash == hash).map(|c| c.payload.as_slice())
})
}
})
}
fn client(addr: SocketAddr) -> HuginnClient {
HuginnClient::new(addr, slug(INSTANCE), TIMEOUT)
}
fn unavailable(error: ClientError) -> (SocketAddr, String) {
match error {
ClientError::Unavailable { endpoint, detail } => (endpoint, detail),
other @ (ClientError::Rejected { .. } | ClientError::TooLarge { .. } | ClientError::Unencodable { .. }) => panic!("expected Unavailable, got {other}"),
}
}
fn rejected(error: ClientError) -> (SocketAddr, String, String) {
match error {
ClientError::Rejected { endpoint, code, detail } => (endpoint, code, detail),
other @ (ClientError::Unavailable { .. } | ClientError::TooLarge { .. } | ClientError::Unencodable { .. }) => panic!("expected Rejected, got {other}"),
}
}
fn nothing_found(message_id: &str, operation: &str) -> CultNetMessage {
encode_response(message_id, operation, &HuginnMindResponse::View(None), "scripted").unwrap()
}
/// Wait for a scripted server to have seen `count` sessions end.
fn ended_within(server: &Server, count: usize, wait: Duration) -> bool {
let until = Instant::now() + wait;
while server.ended.load(Ordering::Relaxed) < count {
if Instant::now() >= until {
return false;
}
std::thread::sleep(Duration::from_millis(10));
}
true
}
fn document_query(instance: &str) -> HuginnMindRequest {
HuginnMindRequest::Query {
instance: slug(instance),
selection: Selection { projection: "document".into(), ..Selection::default() },
semantic: None,
}
}
/// Every operation the wire has, asked of a live daemon over the socket, and
/// each answered with the mind's own answer.
#[test]
fn every_operation_round_trips_against_a_live_daemon() {
let (root, daemon) = mind(vec![]);
let server = serve(root, daemon);
let client = client(server.addr);
assert_eq!(client.instance(), &slug(INSTANCE), "the client reports the instance it was configured for");
let HuginnMindResponse::Whoami(status) = client.call(HuginnMindRequest::Whoami).unwrap() else {
panic!("whoami is answered with a status");
};
assert_eq!((status.instance, status.documents), (slug(INSTANCE), 1));
let admitted = client.call(HuginnMindRequest::Admit(batch(INSTANCE, vec![stewardship(), campaign()]))).unwrap();
let HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { writes, .. }) = admitted else {
panic!("the batch is committed, got {admitted:?}");
};
assert_eq!(writes.len(), 2);
let HuginnMindResponse::Query(page) = client.call(document_query(INSTANCE)).unwrap() else {
panic!("a query is answered with a page");
};
assert_eq!(page.matched, 3, "the identity and the two admitted documents");
let campaign_ref = PipelineRef { kind: PipelineKind::Campaign, id: Short(format!("{CAMPAIGN}:campaign:self")) };
let viewed = client.call(HuginnMindRequest::View { instance: slug(INSTANCE), id: campaign_ref.clone() }).unwrap();
let HuginnMindResponse::View(Some(view)) = viewed else { panic!("the campaign is viewable, got {viewed:?}") };
assert_eq!(view.id, campaign_ref);
}
/// A refusal is data. A read that names another mind, a write that declares
/// another mind, and a document the mind does not hold each come back as `Ok`.
#[test]
fn a_foreign_instance_is_an_answer_not_an_error() {
let (root, daemon) = mind(vec![]);
let server = serve(root, daemon);
let client = client(server.addr);
let foreign = MindRefusal::ForeignInstance { declared: OTHER.into(), mind: INSTANCE.into() };
assert_eq!(client.call(document_query(OTHER)).unwrap(), HuginnMindResponse::Refused(foreign.clone()));
assert_eq!(
client.call(HuginnMindRequest::Admit(batch(OTHER, vec![identity(OTHER)]))).unwrap(),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(foreign)),
);
let absent = PipelineRef { kind: PipelineKind::CutSpec, id: Short(format!("{CAMPAIGN}:cut_spec:cut-none.r1")) };
assert_eq!(
client.call(HuginnMindRequest::View { instance: slug(INSTANCE), id: absent }).unwrap(),
HuginnMindResponse::View(None),
);
}
/// The wide cut spec's answer is over one send and arrives deferred; the
/// narrow one arrives whole. Both come back equal to what the mind gave.
#[test]
fn a_deferred_answer_resolves_to_the_same_page_as_a_direct_one() {
let (root, mut daemon) =
mind(vec![vec![stewardship(), campaign()], vec![cut_spec("wide", 256), cut_spec("fits", 180)]]);
let measured = |request: &HuginnMindRequest, daemon: &mut TestDaemon| {
let answer = answered(daemon.handle(request.clone(), Utc::now()));
let envelope = encode_response("m", request.operation(), &answer, "huginn-eureka").unwrap();
let size = encode_cultnet_message_to_vec(&envelope, CultNetWireContract::CultNetSchemaV0).unwrap().len();
(answer, size as u64)
};
let wide = HuginnMindRequest::View { instance: slug(INSTANCE), id: cut_ref("wide") };
let fits = HuginnMindRequest::View { instance: slug(INSTANCE), id: cut_ref("fits") };
let (wide_answer, wide_size) = measured(&wide, &mut daemon);
let (fits_answer, fits_size) = measured(&fits, &mut daemon);
assert!(matches!(wide_answer, HuginnMindResponse::View(Some(_))), "{wide_answer:?}");
assert!(wide_size > MAX_RESPONSE_BYTES, "the wide answer must be over one send, or nothing was deferred");
assert!(fits_size <= MAX_RESPONSE_BYTES, "the narrow answer must fit one send");
let server = serve(root, daemon);
let client = client(server.addr);
assert_eq!(client.call(wide).unwrap(), wide_answer);
assert_eq!(client.call(fits).unwrap(), fits_answer);
}
/// Nothing listens on the port. The call fails naming the endpoint, and it
/// fails when the configured timeout says: two timeouts, two waits.
#[test]
fn a_closed_port_is_unavailable_naming_the_endpoint_within_the_timeout() {
let closed = UdpSocket::bind("127.0.0.1:0").unwrap().local_addr().unwrap();
let mut waited = Vec::new();
for timeout in [Duration::from_millis(300), Duration::from_millis(1200)] {
let started = Instant::now();
let error = HuginnClient::new(closed, slug(INSTANCE), timeout).call(HuginnMindRequest::Whoami).unwrap_err();
let elapsed = started.elapsed();
assert!(elapsed >= timeout, "gave up after {elapsed:?}, before its {timeout:?}");
assert!(elapsed < timeout + Duration::from_millis(800), "gave up after {elapsed:?}, long past its {timeout:?}");
assert!(error.to_string().contains(&closed.to_string()), "{error}");
let (endpoint, detail) = unavailable(error);
assert_eq!(endpoint, closed);
assert!(detail.contains("no accept") && detail.contains("deadline"), "{detail}");
waited.push(elapsed);
}
assert!(waited[1] > waited[0] + Duration::from_millis(500), "{waited:?}");
}
/// A daemon that accepts and then says nothing is a timeout on the answer,
/// not on the connection.
#[test]
fn a_daemon_that_never_answers_is_unavailable_within_the_timeout() {
let server = scripted(|_| {
std::thread::sleep(Duration::from_secs(3));
CultNetMessage::Error { error: "late".into(), code: None, details: None }
});
let started = Instant::now();
let error = HuginnClient::new(server.addr, slug(INSTANCE), Duration::from_millis(400))
.call(HuginnMindRequest::Whoami)
.unwrap_err();
let elapsed = started.elapsed();
assert!(elapsed >= Duration::from_millis(400) && elapsed < Duration::from_secs(2), "{elapsed:?}");
let (endpoint, detail) = unavailable(error);
assert_eq!(endpoint, server.addr);
assert!(detail.contains("no answer"), "{detail}");
}
/// An envelope failure means the request never reached a mind. It is the
/// daemon refusing, not a transport failure, so a caller that retries on
/// `Unavailable` does not loop on it; the code and the message arrive intact.
#[test]
fn an_envelope_failure_is_rejected_and_carries_its_code() {
let server = answering(encode_failure(
"eureka-state-call",
"whoami",
&OperationFailure { code: "wrong-service".into(), message: "not huginn.mind".into() },
"scripted",
));
let error = client(server.addr).call(HuginnMindRequest::Whoami).unwrap_err();
assert!(error.to_string().contains("wrong-service"), "{error}");
let (endpoint, code, detail) = rejected(error);
assert_eq!((endpoint, code.as_str(), detail.as_str()), (server.addr, "wrong-service", "not huginn.mind"));
}
/// An answer to another request, or to another operation, is not this call's
/// answer, whatever it says.
#[test]
fn an_answer_that_is_not_to_this_request_is_refused() {
let foreign = answering(nothing_found("someone-elses-call", "whoami"));
let (_, detail) = unavailable(client(foreign.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(detail.contains("someone-elses-call"), "{detail}");
let other_operation = answering(nothing_found("eureka-state-call", "query"));
let (_, detail) = unavailable(client(other_operation.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(detail.contains("query") && detail.contains("whoami"), "{detail}");
let matching = answering(nothing_found("eureka-state-call", "whoami"));
assert_eq!(client(matching.addr).call(HuginnMindRequest::Whoami).unwrap(), HuginnMindResponse::View(None));
}
/// The timeout bounds the whole call. Seven chunks each answered after 300 ms
/// take two seconds; no single wait exceeds the 500 ms timeout, and the call
/// still gives up at it.
#[test]
fn the_timeout_bounds_the_whole_call_not_each_wait() {
let server = deferring_in_chunks(vec![7_u8; 7 * 1024], 1024, Duration::from_millis(300));
let timeout = Duration::from_millis(500);
let started = Instant::now();
let error = HuginnClient::new(server.addr, slug(INSTANCE), timeout).call(HuginnMindRequest::Whoami).unwrap_err();
let elapsed = started.elapsed();
assert!(elapsed >= timeout && elapsed < Duration::from_millis(1100), "{elapsed:?}");
let (_, detail) = unavailable(error);
assert!(detail.contains("deadline"), "{detail}");
}
/// Every call ends its session, whether it answered, was rejected, or gave up.
#[test]
fn every_call_disconnects_its_session() {
let answered = answering(nothing_found("eureka-state-call", "whoami"));
client(answered.addr).call(HuginnMindRequest::Whoami).unwrap();
assert!(ended_within(&answered, 1, Duration::from_secs(5)), "an answered call left its session open");
let refused = answering(encode_failure(
"eureka-state-call",
"whoami",
&OperationFailure { code: "wrong-service".into(), message: "no".into() },
"scripted",
));
rejected(client(refused.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(ended_within(&refused, 1, Duration::from_secs(5)), "a rejected call left its session open");
let silent = scripted(|_| {
std::thread::sleep(Duration::from_millis(800));
CultNetMessage::Error { error: "late".into(), code: None, details: None }
});
let error = HuginnClient::new(silent.addr, slug(INSTANCE), Duration::from_millis(300))
.call(HuginnMindRequest::Whoami)
.unwrap_err();
unavailable(error);
assert!(ended_within(&silent, 1, Duration::from_secs(5)), "a timed-out call left its session open");
}
/// A deferred body that is not a response, and one that is another deferral,
/// are protocol violations named as such.
#[test]
fn a_deferred_body_must_be_a_response_and_not_another_deferral() {
let garbage = deferring(vec![0xc1; 40_000]);
let (_, detail) = unavailable(client(garbage.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(detail.contains("not a response"), "{detail}");
let (inner, _) = pack_content("inner", "package", "", "", "", b"x", 16).unwrap();
let again = rmp_serde::to_vec_named(&HuginnMindResponse::Deferred(DeferredAnswer { manifest: inner })).unwrap();
let nested = deferring(again);
let (_, detail) = unavailable(client(nested.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(detail.contains("itself deferred"), "{detail}");
}
/// A body over the deferral cap is refused before its first chunk is asked
/// for, by the number the daemon defers under.
#[test]
fn a_body_over_the_deferral_cap_is_refused() {
let over = huginn_mind::MAX_DEFERRED_BODY_BYTES as usize + 1;
let server = deferring(vec![0_u8; over]);
let (_, detail) = unavailable(client(server.addr).call(HuginnMindRequest::Whoami).unwrap_err());
assert!(detail.contains("exceeds"), "{detail}");
}
/// An admission whose session is `session_pad` characters long: the knob that
/// sizes a request without changing what it asks.
fn padded_admit(session_pad: usize) -> HuginnMindRequest {
let mut request = batch(INSTANCE, vec![stewardship()]);
request.provenance.session = Short("s".repeat(session_pad));
HuginnMindRequest::Admit(request)
}
/// What one send has to carry for `message`, measured the way the client does.
fn encoded_bytes(message: &CultNetMessage) -> usize {
encode_cultnet_message_to_vec(message, CultNetWireContract::CultNetSchemaV0).unwrap().len()
}
fn message_for(request: &HuginnMindRequest, source: Option<String>) -> CultNetMessage {
encode_request("eureka-state-call", request, source).unwrap()
}
/// The smallest session padding whose request encodes to at least `target`
/// bytes. The payload rides as base64, so request sizes step by four and only
/// every fourth size is reachable through the request alone.
fn pad_for(target: usize) -> usize {
let (mut low, mut high) = (0, target);
while low < high {
let middle = (low + high) / 2;
if encoded_bytes(&message_for(&padded_admit(middle), None)) < target {
low = middle + 1;
} else {
high = middle;
}
}
low
}
/// The largest request one send carries, and the smallest it does not.
fn largest_request_that_fits() -> HuginnMindRequest {
padded_admit(pad_for(MAX_REQUEST_BYTES + 1) - 1)
}
fn smallest_request_over() -> HuginnMindRequest {
padded_admit(pad_for(MAX_REQUEST_BYTES + 1))
}
/// A message that encodes to exactly `target` bytes: a request a little under
/// it, topped up with a source runtime id, which rides outside the payload and
/// moves the size a byte at a time once the field is there at all.
fn message_of_size(target: usize) -> CultNetMessage {
let fitting = pad_for(target + 1) - 1;
for padding in (fitting.saturating_sub(80)..=fitting).rev() {
let request = padded_admit(padding);
for source in 1..80 {
let message = message_for(&request, Some("r".repeat(source)));
if encoded_bytes(&message) == target {
return message;
}
}
}
panic!("no message encodes to {target} bytes");
}
fn admitted_answer() -> CultNetMessage {
encode_response(
"eureka-state-call",
"admit",
&HuginnMindResponse::Admit(PipelineAdmissionOutcome::AlreadyAdmitted { receipt_id: "r".into() }),
"scripted",
)
.unwrap()
}
/// A request larger than one send carries is refused by the client before it
/// connects to anything, as its own error and not as a transport failure the
/// caller would retry; the largest request that fits is sent, and answered.
#[test]
fn a_request_over_what_one_send_carries_is_too_large_and_one_that_fits_is_sent() {
let closed = UdpSocket::bind("127.0.0.1:0").unwrap().local_addr().unwrap();
let over = smallest_request_over();
let started = Instant::now();
match client(closed).call(over.clone()) {
Err(ClientError::TooLarge { bytes, limit }) => {
assert_eq!(bytes, encoded_bytes(&message_for(&over, None)));
assert_eq!(limit, MAX_REQUEST_BYTES);
assert!(bytes > limit);
}
other => panic!("expected TooLarge, got {other:?}"),
}
assert!(started.elapsed() < Duration::from_secs(1), "refused only after trying the transport");
let fits = largest_request_that_fits();
assert!(MAX_REQUEST_BYTES - encoded_bytes(&message_for(&fits, None)) < 4, "not the largest that fits");
let server = answering(admitted_answer());
let answer = client(server.addr).call(fits);
assert!(matches!(answer, Ok(HuginnMindResponse::Admit(PipelineAdmissionOutcome::AlreadyAdmitted { .. }))), "{answer:?}");
}
/// The transport's own refusal of an oversize send ("Message too long") is
/// what an unchecked request used to report, as an `Unavailable` daemon a
/// caller would retry forever.
#[test]
fn an_oversize_admission_is_never_reported_as_an_unavailable_daemon() {
let server = answering(admitted_answer());
let error = client(server.addr).call(padded_admit(70_000)).unwrap_err();
assert!(matches!(error, ClientError::TooLarge { .. }), "{error}");
}
/// The limit is the transport's own: a message of `MAX_REQUEST_BYTES` goes out
/// on an RUDP session, and one byte more is refused by the socket.
#[test]
fn the_limit_is_the_largest_message_the_transport_sends() {
let server = answering(admitted_answer());
let mut session = CultMesh::create_rudp_client_for_endpoint(
"limit-probe".to_string(),
CULTNET_OPERATION_CONNECTION_ID,
&format!("rudp://{}", server.addr),
CultMeshRudpSocketOptions::default(),
)
.unwrap();
session.connect(Vec::new()).unwrap();
let until = Instant::now() + Duration::from_secs(5);
while !session.connected() {
session.poll_resends().unwrap();
let _ = session.receive_once().unwrap();
assert!(Instant::now() < until, "the server never accepted");
std::thread::sleep(Duration::from_millis(2));
}
session.send_schema_message(&message_of_size(MAX_REQUEST_BYTES)).expect("the limit is carried");
assert!(session.send_schema_message(&message_of_size(MAX_REQUEST_BYTES + 1)).is_err(), "one byte more is sent");
}

View file

@ -0,0 +1,185 @@
//! The daemon half of the fixtures: an in-process Huginn daemon over a
//! temporary mind, on a loopback port, shared by the client and the server
//! tests.
#![allow(dead_code)]
use std::net::SocketAddr;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::thread::JoinHandle;
use std::time::Duration;
use chrono::Utc;
use cultnet_rs::{CultNetMessage, CultNetRudpServerEvent, CultNetWireContract, decode_cultnet_message_from_slice};
use huginn_daemon::serve::{bind, run, schema_registry};
use huginn_daemon::{Daemon, Handled, Hits, IndexSink, SearchTicket, ServeOptions};
use huginn_mind::epiphany_pipeline::{
Date, DocRef, OrgRepo, PipelineCampaign, PipelineDocument, PipelineInstance, PipelineRef, PipelineStewardship, Sha, Short, Slug,
Title,
};
use huginn_mind::{
Faculty, HuginnMindRequest, HuginnMindResponse, IndexStatus, Mind, MindStore, OwnedRedbMessagePackBackingStore,
PipelineAdmissionBatch, PipelineAdmissionOutcome, PipelineProvenance,
};
use tempfile::TempDir;
pub const INSTANCE: &str = "eureka";
pub fn slug(value: &str) -> Slug {
Slug(value.into())
}
pub fn identity(name: &str) -> PipelineDocument {
PipelineDocument::Instance(PipelineInstance {
instance: slug(name),
display_name: Short(format!("{name} mind")),
created_at: Date("2026-09-29".into()),
host: Short(name.into()),
})
}
pub fn batch(instance: &str, documents: Vec<PipelineDocument>) -> PipelineAdmissionBatch {
PipelineAdmissionBatch {
instance: slug(instance),
provenance: PipelineProvenance {
faculty: Faculty::Hands,
agent: Short("claude".into()),
session: Short("session-1".into()),
tool: Short("admit".into()),
},
documents,
}
}
/// No index: the client's subject is the transport, not the projection.
pub struct Inert;
impl<S: MindStore> IndexSink<S> for Inert {
fn committed(&mut self, _mind: &Mind<S>, _writes: &[PipelineRef]) -> anyhow::Result<()> {
Ok(())
}
fn status(&self) -> IndexStatus {
IndexStatus::Current
}
fn search(&mut self, _text: &str, _top_k: u32) -> Result<SearchTicket, String> {
Err("this index cannot search".into())
}
fn search_deadline(&self) -> Duration {
Duration::ZERO
}
fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)> {
Vec::new()
}
fn abandon(&mut self, _ticket: SearchTicket) {}
}
pub type TestDaemon = Daemon<OwnedRedbMessagePackBackingStore, Inert>;
pub fn answered(handled: Handled) -> HuginnMindResponse {
match handled {
Handled::Answered(response) => response,
Handled::Searching(_) => panic!("the inert index cannot search"),
}
}
/// A daemon over a temporary mind whose identity is admitted, and whatever
/// else `seed` admits.
pub fn mind(seed: Vec<Vec<PipelineDocument>>) -> (TempDir, TestDaemon) {
let root = tempfile::tempdir().unwrap();
let mut daemon = Daemon::new(Mind::open(root.path(), &slug(INSTANCE)).unwrap(), Inert);
for documents in std::iter::once(vec![identity(INSTANCE)]).chain(seed) {
let outcome = answered(daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, documents)), Utc::now()));
assert!(matches!(outcome, HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { .. })), "{outcome:?}");
}
(root, daemon)
}
/// A server on a loopback port that stops when dropped.
pub struct Server {
pub addr: SocketAddr,
pub stopping: Arc<AtomicBool>,
pub thread: Option<JoinHandle<()>>,
/// The sessions a scripted server has seen end: what the client's
/// disconnect looks like from the daemon's side.
pub ended: Arc<AtomicUsize>,
pub _root: Option<TempDir>,
}
impl Drop for Server {
fn drop(&mut self) {
self.stopping.store(true, Ordering::Relaxed);
if let Some(thread) = self.thread.take() {
let _ = thread.join();
}
}
}
pub fn serve(root: TempDir, mut daemon: TestDaemon) -> Server {
let mut hub = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let addr = hub.local_addr().unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let registry = schema_registry().unwrap();
let thread = std::thread::spawn(move || {
run(&mut daemon, &mut hub, &registry, &loop_stopping, &ServeOptions::default()).unwrap();
});
Server { addr, stopping, thread: Some(thread), ended: Arc::default(), _root: Some(root) }
}
/// A server that answers each schema message with whatever `script` says.
pub fn scripted(mut script: impl FnMut(&CultNetMessage) -> CultNetMessage + Send + 'static) -> Server {
let mut hub = bind("127.0.0.1:0".parse().unwrap(), "scripted").unwrap();
let addr = hub.local_addr().unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let ended = Arc::new(AtomicUsize::new(0));
let loop_ended = Arc::clone(&ended);
let thread = std::thread::spawn(move || {
while !loop_stopping.load(Ordering::Relaxed) {
hub.poll_resends().unwrap();
while let Some(event) = hub.receive_event_once().unwrap() {
if matches!(event, CultNetRudpServerEvent::Disconnected { .. }) {
loop_ended.fetch_add(1, Ordering::Relaxed);
}
let CultNetRudpServerEvent::Frame { session, frame } = event else { continue };
let message =
decode_cultnet_message_from_slice(&frame.payload, CultNetWireContract::CultNetSchemaV0).unwrap();
hub.send_schema_message(&session, &script(&message)).unwrap();
}
std::thread::sleep(Duration::from_millis(2));
}
});
Server { addr, stopping, thread: Some(thread), ended, _root: None }
}
/// A scripted server whose every answer is `reply`.
pub fn answering(reply: CultNetMessage) -> Server {
scripted(move |_| reply.clone())
}
pub const CAMPAIGN: &str = "eureka-state";
pub const REPO: &str = "GameCult/Epiphany";
pub fn sha() -> Sha {
Sha("5f98228d".into())
}
pub fn campaign() -> PipelineDocument {
PipelineDocument::Campaign(PipelineCampaign {
slug: slug(CAMPAIGN),
title: Title("Eureka pipeline state".into()),
repos: vec![OrgRepo(REPO.into())],
working_branch: Short("hands/cut13a".into()),
target_doc: DocRef { path: Short("notes/target.md".into()), start_line: 1, end_line: 9, commit: sha() },
})
}
pub fn stewardship() -> PipelineDocument {
PipelineDocument::Stewardship(PipelineStewardship {
instance: slug(INSTANCE),
repo: OrgRepo(REPO.into()),
sequence: 1,
assigned_on: Date("2026-09-29".into()),
note: "assigned".into(),
})
}

View file

@ -0,0 +1,532 @@
//! The `eureka-state` binary over stdio, as Claude Code drives it: real
//! processes, real JSON-RPC, a real daemon in-process behind it (or a scripted
//! server where the daemon cannot be made to misbehave).
use std::collections::BTreeSet;
use std::io::{BufRead, BufReader, Write};
use std::net::UdpSocket;
use std::process::{Child, ChildStdin, Command, Stdio};
use std::sync::mpsc::{Receiver, channel};
use std::time::Duration;
use huginn_mind::envelope::{OperationFailure, encode_failure, encode_response};
use huginn_mind::{HuginnMindRequest, HuginnMindResponse};
use huginn_mind::epiphany_pipeline::{PipelineKind, PipelineRef, Short};
use serde_json::{Value, json};
mod common;
use common::*;
const WAIT: Duration = Duration::from_secs(60);
struct Mcp {
child: Child,
stdin: ChildStdin,
lines: Receiver<String>,
next: u64,
/// The server's answer to `initialize`.
init: Value,
}
impl Drop for Mcp {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
impl Mcp {
/// The binary with exactly the environment given, initialised.
fn start(env: &[(&str, &str)]) -> Self {
Self::spawn(|command| {
command.envs(env.iter().copied());
})
}
/// The binary with no configuration but what `configure` gives its command.
fn spawn(configure: impl FnOnce(&mut Command)) -> Self {
let mut command = Command::new(env!("CARGO_BIN_EXE_eureka-state"));
command.env_remove("EUREKA_INSTANCE").env_remove("HUGINN_ENDPOINT");
configure(&mut command);
command
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::null());
let mut child = command.spawn().expect("the binary starts");
let stdin = child.stdin.take().unwrap();
let stdout = child.stdout.take().unwrap();
let (sender, lines) = channel();
std::thread::spawn(move || {
for line in BufReader::new(stdout).lines().map_while(Result::ok) {
let _ = sender.send(line);
}
});
let mut mcp = Self { child, stdin, lines, next: 1, init: Value::Null };
let init = mcp.rpc(
"initialize",
json!({ "protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": { "name": "test", "version": "0" } }),
);
assert!(init.get("result").is_some(), "{init}");
mcp.init = init["result"].clone();
mcp.send(json!({ "jsonrpc": "2.0", "method": "notifications/initialized" }));
mcp
}
/// Configured for `instance` against a daemon at `addr`.
fn at(instance: &str, addr: std::net::SocketAddr) -> Self {
Self::start(&[("EUREKA_INSTANCE", instance), ("HUGINN_ENDPOINT", &format!("rudp://{addr}"))])
}
fn send(&mut self, message: Value) {
writeln!(self.stdin, "{message}").unwrap();
self.stdin.flush().unwrap();
}
/// One request and its response, whole (`result` or `error`).
fn rpc(&mut self, method: &str, params: Value) -> Value {
let id = self.next;
self.next += 1;
self.send(json!({ "jsonrpc": "2.0", "id": id, "method": method, "params": params }));
loop {
let line = self.lines.recv_timeout(WAIT).expect("the server answers");
let message: Value = serde_json::from_str(&line).expect("stdout carries only JSON-RPC");
if message["id"] == json!(id) {
return message;
}
}
}
/// A tool call's result: whether it is an error, and its JSON body.
fn call(&mut self, tool: &str, arguments: Value) -> (bool, Value) {
let response = self.rpc("tools/call", json!({ "name": tool, "arguments": arguments }));
let result = &response["result"];
assert!(result.is_object(), "a tool result, not a protocol error: {response}");
let text = result["content"][0]["text"].as_str().expect("text content");
(result["isError"] == json!(true), serde_json::from_str(text).expect("a JSON body"))
}
}
/// A loopback UDP port nothing listens on.
fn closed_port() -> std::net::SocketAddr {
UdpSocket::bind("127.0.0.1:0").unwrap().local_addr().unwrap()
}
fn faculty_args(documents: Vec<Value>) -> Value {
json!({ "faculty": "Hands", "agent": "claude", "session": "session-7", "documents": documents })
}
fn campaign_json() -> Value {
serde_json::to_value(campaign()).unwrap()
}
/// Every `$ref` in a schema, and the `$defs` it can resolve against.
fn refs(schema: &Value, found: &mut Vec<String>) {
match schema {
Value::Object(map) => {
for (key, value) in map {
if key == "$ref" {
found.push(value.as_str().unwrap().to_string());
} else {
refs(value, found);
}
}
}
Value::Array(items) => items.iter().for_each(|item| refs(item, found)),
_ => {}
}
}
/// Every key named `key` anywhere below `schema`'s root.
fn keys_below_root(schema: &Value, key: &str) -> usize {
fn count(value: &Value, key: &str) -> usize {
match value {
Value::Object(map) => map.iter().map(|(name, inner)| usize::from(name == key) + count(inner, key)).sum(),
Value::Array(items) => items.iter().map(|item| count(item, key)).sum(),
_ => 0,
}
}
schema.as_object().unwrap().values().map(|inner| count(inner, key)).sum()
}
/// Four tools, each input schema a self-contained document: every `$ref` is
/// internal and resolves, and the selection's definitions are bundled in.
#[test]
fn tools_list_offers_four_tools_with_self_contained_schemas() {
let mut mcp = Mcp::at(INSTANCE, closed_port());
let listed = mcp.rpc("tools/list", json!({}));
let tools = listed["result"]["tools"].as_array().expect("tools");
let names: BTreeSet<&str> = tools.iter().map(|tool| tool["name"].as_str().unwrap()).collect();
assert_eq!(names, BTreeSet::from(["whoami", "admit", "view", "query"]));
for tool in tools {
let schema = &tool["inputSchema"];
let mut found = Vec::new();
refs(schema, &mut found);
for reference in &found {
let name = reference.strip_prefix("#/$defs/").unwrap_or_else(|| panic!("{}: remote $ref {reference}", tool["name"]));
assert!(schema["$defs"].get(name).is_some(), "{}: {reference} does not resolve", tool["name"]);
}
// The vendored selection schema brings its own `$id` and `$schema`,
// which would make its `$ref`s resolve against a remote document.
assert!(schema.get("$id").is_none(), "{}: a remote identity", tool["name"]);
for key in ["$id", "$schema"] {
assert_eq!(keys_below_root(schema, key), 0, "{}: a nested {key}", tool["name"]);
}
}
let query = tools.iter().find(|tool| tool["name"] == "query").unwrap();
for bundled in ["fieldPredicate", "citation", "incoming", "recordRef"] {
assert!(query["inputSchema"]["$defs"].get(bundled).is_some(), "{bundled} is not bundled");
}
assert_eq!(query["inputSchema"]["properties"]["selection"]["properties"]["limit"]["maximum"], json!(200));
}
/// The vendored selection schema is CultLib's, at the revision `cultnet-rs`
/// is built from.
#[test]
fn selection_schema_matches_cultlib() {
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".into());
let metadata = Command::new(cargo)
.args(["metadata", "--format-version", "1", "--offline", "--locked"])
.current_dir(env!("CARGO_MANIFEST_DIR"))
.output()
.unwrap();
assert!(metadata.status.success(), "{}", String::from_utf8_lossy(&metadata.stderr));
let metadata: Value = serde_json::from_slice(&metadata.stdout).unwrap();
let manifest = metadata["packages"]
.as_array()
.unwrap()
.iter()
.find(|package| package["name"] == "cultnet-rs")
.expect("cultnet-rs is a dependency")["manifest_path"]
.as_str()
.unwrap();
let crate_dir = std::path::Path::new(manifest).parent().unwrap();
let published = crate_dir.join("../../contracts/cultnet/cultnet.selection.schema.json");
let theirs: Value = serde_json::from_str(&std::fs::read_to_string(published).unwrap()).unwrap();
let ours: Value = serde_json::from_str(
&std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/schemas/cultnet.selection.schema.json")).unwrap(),
)
.unwrap();
assert_eq!(ours, theirs);
}
/// Every tool against a live daemon, and what each returns.
#[test]
fn every_tool_round_trips_against_a_live_daemon() {
let (root, daemon) = mind(vec![vec![stewardship()]]);
let server = serve(root, daemon);
let mut mcp = Mcp::at(INSTANCE, server.addr);
let (error, whoami) = mcp.call("whoami", json!({}));
assert!(!error);
assert_eq!(whoami["instance"], json!(INSTANCE));
assert_eq!(whoami["endpoint"], json!(format!("rudp://{}", server.addr)));
assert_eq!(whoami["reachable"], json!(true));
assert_eq!(whoami["status"]["instance"], json!(INSTANCE));
assert_eq!(whoami["status"]["documents"], json!(2));
assert!(whoami.get("error").is_none(), "{whoami}");
let (error, admitted) = mcp.call("admit", faculty_args(vec![campaign_json()]));
assert!(!error);
let writes = admitted["Committed"]["writes"].as_array().expect("committed").clone();
assert_eq!(writes.len(), 1, "{admitted}");
let (error, viewed) = mcp.call("view", json!({ "id": writes[0] }));
assert!(!error);
assert_eq!(viewed["id"], writes[0]);
assert_eq!(viewed["admission"]["provenance"]["session"], json!("session-7"));
let (error, nothing) = mcp.call("view", json!({ "id": absent() }));
assert!(!error);
assert_eq!(nothing, Value::Null);
let (error, page) = mcp.call("query", json!({ "selection": { "projection": "document" } }));
assert!(!error);
assert_eq!(page["matched"], json!(3), "{page}");
assert_eq!(page["items"]["Documents"].as_array().unwrap().len(), 3);
let (_, counted) = mcp.call("whoami", json!({}));
assert_eq!(counted["status"]["receipts"], json!(3));
}
/// Declared identity is configuration, not a per-call argument. Input that
/// names another instance, or another tool, changes neither: the batch is
/// admitted for the configured instance, and the receipt says `eureka-state`.
#[test]
fn admit_ignores_any_instance_in_input() {
let (root, daemon) = mind(vec![vec![stewardship()]]);
let server = serve(root, daemon);
let mut mcp = Mcp::at(INSTANCE, server.addr);
let mut arguments = faculty_args(vec![campaign_json()]);
arguments["instance"] = json!("thought-cage");
arguments["tool"] = json!("someone-else");
arguments["provenance"] = json!({ "faculty": "Operator", "agent": "x", "session": "y", "tool": "someone-else" });
let (error, admitted) = mcp.call("admit", arguments);
assert!(!error);
let writes = admitted["Committed"]["writes"].as_array().unwrap_or_else(|| panic!("not committed: {admitted}"));
let (_, viewed) = mcp.call("view", json!({ "id": writes[0] }));
let provenance = &viewed["admission"]["provenance"];
assert_eq!(provenance["tool"], json!("eureka-state"));
assert_eq!(provenance["faculty"], json!("Hands"));
assert_eq!(provenance["agent"], json!("claude"));
assert_eq!(provenance["session"], json!("session-7"));
}
/// A mind's refusal is an answer: a successful result carrying the typed
/// refusal. Here the configured instance is not the daemon's.
#[test]
fn a_refusal_is_a_successful_result() {
let (root, daemon) = mind(vec![vec![stewardship()]]);
let server = serve(root, daemon);
let mut mcp = Mcp::at("thought-cage", server.addr);
let (error, viewed) = mcp.call("view", json!({ "id": absent() }));
assert!(!error);
assert!(viewed.get("Refused").is_some(), "{viewed}");
let (error, queried) = mcp.call("query", json!({ "selection": {} }));
assert!(!error);
assert!(queried.get("Refused").is_some(), "{queried}");
let (error, admitted) = mcp.call("admit", faculty_args(vec![campaign_json()]));
assert!(!error);
assert!(admitted.get("Refused").is_some(), "{admitted}");
}
/// A daemon that cannot be reached is `isError` with the endpoint, and
/// `whoami` reports it as `reachable: false` without a protocol error.
#[test]
fn a_closed_port_is_reported_and_is_not_a_protocol_error() {
let addr = closed_port();
let mut mcp = Mcp::at(INSTANCE, addr);
let whoami = mcp.rpc("tools/call", json!({ "name": "whoami", "arguments": {} }));
assert!(whoami.get("error").is_none(), "{whoami}");
assert_ne!(whoami["result"]["isError"], json!(true));
let body: Value = serde_json::from_str(whoami["result"]["content"][0]["text"].as_str().unwrap()).unwrap();
assert_eq!(body["reachable"], json!(false));
assert_eq!(body["status"], Value::Null);
assert_eq!(body["error"]["error"], json!("Unavailable"));
let (error, view) = mcp.call("view", json!({ "id": absent() }));
assert!(error);
assert_eq!(view["error"], json!("Unavailable"));
assert_eq!(view["endpoint"], json!(format!("rudp://{addr}")));
}
/// A daemon rejecting the envelope is a permanent mismatch, not a down
/// daemon: `isError` with its own code, distinct from `Unavailable`.
#[test]
fn a_rejected_request_is_a_typed_error_distinct_from_unavailable() {
let server = answering(encode_failure(
"eureka-state-call",
"view",
&OperationFailure { code: "wrong-service".into(), message: "not huginn.mind".into() },
"scripted",
));
let mut mcp = Mcp::at(INSTANCE, server.addr);
let (error, body) = mcp.call("view", json!({ "id": absent() }));
assert!(error);
assert_eq!(body["error"], json!("Rejected"));
assert_eq!(body["code"], json!("wrong-service"));
assert_eq!(body["detail"], json!("not huginn.mind"));
assert_eq!(body["endpoint"], json!(format!("rudp://{}", server.addr)));
}
/// A `whoami` answered with another operation's payload is not a mind's
/// status: it is reported, not called reachable.
#[test]
fn whoami_does_not_take_another_answer_for_a_status() {
let server = answering(encode_response("eureka-state-call", "whoami", &HuginnMindResponse::View(None), "scripted").unwrap());
let mut mcp = Mcp::at(INSTANCE, server.addr);
let (error, body) = mcp.call("whoami", json!({}));
assert!(!error);
assert_eq!(body["reachable"], json!(false));
assert_eq!(body["status"], Value::Null);
assert!(body["error"]["detail"].as_str().unwrap().contains("unexpected answer"), "{body}");
}
/// Bad configuration does not stop the server. `whoami` says what is wrong and
/// every other tool refuses with the same typed body.
#[test]
fn bad_configuration_is_served_and_reported() {
let cases: [(&[(&str, &str)], &str); 5] = [
(&[("HUGINN_ENDPOINT", "rudp://127.0.0.1:1")], "EUREKA_INSTANCE is not set"),
(&[("EUREKA_INSTANCE", "Not A Slug"), ("HUGINN_ENDPOINT", "rudp://127.0.0.1:1")], "EUREKA_INSTANCE is not a valid"),
(&[("EUREKA_INSTANCE", "eureka")], "HUGINN_ENDPOINT is not set"),
(&[("EUREKA_INSTANCE", "eureka"), ("HUGINN_ENDPOINT", "127.0.0.1:1")], "HUGINN_ENDPOINT is not rudp://"),
(&[("EUREKA_INSTANCE", "eureka"), ("HUGINN_ENDPOINT", "rudp://not-an-address")], "HUGINN_ENDPOINT is not rudp://"),
];
for (env, expected) in cases {
let mut mcp = Mcp::start(env);
let (error, whoami) = mcp.call("whoami", json!({}));
assert!(!error, "{expected}");
assert_eq!(whoami["reachable"], json!(false), "{expected}");
assert_eq!(whoami["error"]["error"], json!("Misconfigured"), "{expected}");
assert!(whoami["error"]["detail"].as_str().unwrap().contains(expected), "{expected}: {whoami}");
for (tool, arguments) in [
("view", json!({ "id": absent() })),
("query", json!({ "selection": {} })),
("admit", faculty_args(vec![])),
] {
let (error, refusal) = mcp.call(tool, arguments);
assert!(error, "{tool}: {expected}");
assert_eq!(refusal["error"], json!("Misconfigured"), "{tool}");
}
}
}
/// Input that does not fit the tool's schema is refused before any call is
/// made. rmcp 2.2 reports it as a tool result with `isError`, not as a
/// JSON-RPC `invalid_params`, so the model sees what to correct. The server
/// keeps serving.
#[test]
fn malformed_input_is_refused_before_any_call() {
let server = answering(encode_response("eureka-state-call", "admit", &HuginnMindResponse::View(None), "scripted").unwrap());
let mut mcp = Mcp::at(INSTANCE, server.addr);
let response = mcp.rpc("tools/call", json!({ "name": "admit", "arguments": { "faculty": "Hands" } }));
assert!(response.get("error").is_none(), "{response}");
assert_eq!(response["result"]["isError"], json!(true));
let body: Value = serde_json::from_str(response["result"]["content"][0]["text"].as_str().unwrap()).expect("a JSON body");
assert_eq!(body["error"], json!("InvalidInput"), "{body}");
assert!(body["detail"].as_str().unwrap().contains("missing field"), "{body}");
let (error, _) = mcp.call("whoami", json!({}));
assert!(!error);
}
/// A request too large for one send is the client's typed refusal, an error
/// result that names the size and the limit and does not call the daemon
/// unavailable.
#[test]
fn an_oversize_admit_is_a_typed_too_large_error() {
let mut mcp = Mcp::at(INSTANCE, closed_port());
let mut arguments = faculty_args(vec![]);
arguments["session"] = json!("s".repeat(90_000));
let (error, body) = mcp.call("admit", arguments);
assert!(error);
assert_eq!(body["error"], json!("TooLarge"), "{body}");
assert_eq!(body["limit"], json!(eureka_state::MAX_REQUEST_BYTES));
assert!(body["bytes"].as_u64().unwrap() > eureka_state::MAX_REQUEST_BYTES as u64, "{body}");
}
/// Who admitted a batch is validated by the mind, whichever door the batch
/// came through: a blank, whitespace, multi-line or oversized session or agent
/// is a typed refusal, and nothing is committed.
#[test]
fn admit_refuses_an_unusable_attribution() {
let (root, daemon) = mind(vec![vec![stewardship()]]);
let server = serve(root, daemon);
let mut mcp = Mcp::at(INSTANCE, server.addr);
for (field, value) in [
("session", "".to_string()),
("session", " ".to_string()),
("session", "x".repeat(5_000)),
("session", "line\nbreak".to_string()),
("agent", "".to_string()),
("agent", "two\nlines".to_string()),
("agent", "y".repeat(201)),
] {
let mut arguments = faculty_args(vec![campaign_json()]);
arguments[field] = json!(value);
let (error, refused) = mcp.call("admit", arguments);
assert!(!error, "{field}={value:?}");
let refusal = &refused["Refused"]["Document"];
let named = json!(format!("provenance.{field}"));
assert!(
refusal["InvalidFormat"]["field"] == named || refusal["FieldBound"]["field"] == named,
"{field}={value:?}: {refused}"
);
}
let (_, whoami) = mcp.call("whoami", json!({}));
assert_eq!(whoami["status"]["receipts"], json!(2), "only the seed is committed: {whoami}");
let (error, admitted) = mcp.call("admit", faculty_args(vec![campaign_json()]));
assert!(!error);
assert!(admitted.get("Committed").is_some(), "{admitted}");
}
/// The tool forwards the caller's semantic query, and only that, to the mind.
#[test]
fn query_sends_the_semantic_request_on_the_wire() {
let seen = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let sink = std::sync::Arc::clone(&seen);
let server = scripted(move |message| {
let (_, request) = huginn_mind::envelope::decode_request(message).expect("a mind request");
sink.lock().unwrap().push(request);
encode_response("eureka-state-call", "query", &HuginnMindResponse::View(None), "scripted").unwrap()
});
let mut mcp = Mcp::at(INSTANCE, server.addr);
mcp.call("query", json!({ "selection": { "projection": "document" }, "semantic": { "text": "drift", "top_k": 7 } }));
mcp.call("query", json!({ "selection": {} }));
let seen = seen.lock().unwrap();
let [HuginnMindRequest::Query { instance: first, semantic: Some(asked), .. }, HuginnMindRequest::Query { semantic: None, .. }] =
&seen[..]
else {
panic!("two queries expected, the first with its semantic part: {seen:?}");
};
assert_eq!(first.0, INSTANCE);
assert_eq!((asked.text.0.as_str(), asked.top_k), ("drift", 7));
}
/// A daemon that never answers is given up on after the server's own call
/// timeout, 15 seconds: not earlier, and not much later.
#[test]
fn a_silent_daemon_is_given_up_on_after_fifteen_seconds() {
let silent = UdpSocket::bind("127.0.0.1:0").unwrap();
let mut mcp = Mcp::at(INSTANCE, silent.local_addr().unwrap());
let started = std::time::Instant::now();
let (error, body) = mcp.call("view", json!({ "id": absent() }));
let elapsed = started.elapsed();
assert!(error);
assert_eq!(body["error"], json!("Unavailable"), "{body}");
assert!(body["detail"].as_str().unwrap().contains("15s"), "{body}");
assert!(elapsed >= Duration::from_secs(15), "gave up after {elapsed:?}");
assert!(elapsed < Duration::from_secs(20), "gave up after {elapsed:?}");
}
/// An instance name that is not UTF-8 is reported as that, not as unset.
#[test]
fn a_non_utf8_instance_is_reported_as_such() {
let mut mcp = Mcp::spawn(|command| {
command.env("EUREKA_INSTANCE", non_utf8()).env("HUGINN_ENDPOINT", "rudp://127.0.0.1:1");
});
let (error, whoami) = mcp.call("whoami", json!({}));
assert!(!error);
let detail = whoami["error"]["detail"].as_str().unwrap();
assert!(detail.contains("EUREKA_INSTANCE is not valid UTF-8"), "{whoami}");
assert_eq!(whoami["instance"], Value::Null);
}
/// Likewise an endpoint that is not UTF-8: reported as that, not as unset.
#[test]
fn a_non_utf8_endpoint_is_reported_as_such() {
let mut mcp = Mcp::spawn(|command| {
command.env("EUREKA_INSTANCE", INSTANCE).env("HUGINN_ENDPOINT", non_utf8());
});
let (error, whoami) = mcp.call("whoami", json!({}));
assert!(!error);
let detail = whoami["error"]["detail"].as_str().unwrap();
assert!(detail.contains("HUGINN_ENDPOINT is not valid UTF-8"), "{whoami}");
assert_eq!(whoami["endpoint"], Value::Null);
}
#[cfg(unix)]
fn non_utf8() -> std::ffi::OsString {
std::os::unix::ffi::OsStringExt::from_vec(vec![b'e', 0xff])
}
#[cfg(windows)]
fn non_utf8() -> std::ffi::OsString {
std::os::windows::ffi::OsStringExt::from_wide(&[0x65, 0xd800])
}
/// A well-formed id of a document no mind here holds.
fn absent() -> Value {
serde_json::to_value(PipelineRef { kind: PipelineKind::CutSpec, id: Short("eureka-state:cut_spec:cut-99.r1".into()) })
.unwrap()
}
/// A client only lists tools from a server that says it has them.
#[test]
fn initialize_advertises_tools() {
let mcp = Mcp::at(INSTANCE, closed_port());
assert!(mcp.init["capabilities"]["tools"].is_object(), "{}", mcp.init);
}

View file

@ -0,0 +1,45 @@
[package]
name = "huginn-daemon"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish.workspace = true
# The organ's CultNet surface: the socket, the sessions, the process lifetime
# and the operation envelope, and no rule. Every rule, the instance check, the
# receipt, the status and the wire vocabulary are `huginn-mind`'s, and the leaf
# and the store type are reached through it so one crate pins one revision.
[lib]
name = "huginn_daemon"
path = "src/lib.rs"
[[bin]]
name = "huginn-daemon"
path = "src/main.rs"
[dependencies]
anyhow = "1"
base64 = "0.22"
chrono = "0.4.44"
cultnet-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultnet-rs" }
huginn-mind = { path = "../huginn-mind" }
rmp-serde = "1"
serde = { version = "1", features = ["derive"] }
# The vector store's and the embedder's HTTP APIs are the xenos boundary, and
# JSON exists in `index/ollama.rs` and `index/qdrant.rs` only. `ureq` 3 with
# default features off is plain HTTP with no TLS (`rustls` and `gzip` are its
# only defaults), which is all the private mesh needs. It stays below 3.4,
# whose `base64` 0.23 would put a second `base64` in a tree that has one.
serde_json = "1"
sha2 = "0.10"
signal-hook = "0.3"
ureq = { version = "~3.3", default-features = false }
[dev-dependencies]
tempfile = "3"
# Test-only: `Mind::store_path_for`, the grammar-checked door onto a raw
# store's path. The daemon's own store-integrity tests reach a store directly
# (to break a row on purpose) and must go through the same door `open` does
# rather than reconstructing the path by hand (S3, residue batch).
huginn-mind = { path = "../huginn-mind", features = ["test-support"] }

View file

@ -0,0 +1,286 @@
//! The deferred bodies: answers too large for one send, held for the client to
//! fetch chunk by chunk. Ephemeral transport state, never a document store and
//! never world truth: it is lost when the process exits, and a client that
//! finds a body gone asks the operation again.
//!
//! Identity is the manifest's `contentHash`, so the same bytes deferred twice
//! are one body. Chunking, hashing and the manifest are `cultnet-rs`'s; what is
//! owned here is only how long and how much is kept.
//!
//! A body expires when it has been untouched for the TTL, where retaining it
//! again and serving any of its chunks both touch it. When the stored bytes
//! exceed the budget the body touched longest ago is evicted first. A chunk
//! that two bodies share is stored once and lives until the last of them goes.
//! Time is an argument, so nothing here reads a clock; expiry therefore runs
//! only when a frame arrives, and an idle daemon holds an expired body until
//! the next one.
//!
//! Limit: there is no per-session or per-peer budget, so any client can evict
//! another client's body mid-fetch: four 64 MiB reads flush the 256 MiB store,
//! and the loser's next chunk is answered `found: false`. That is acceptable
//! for the organ's trusted WireGuard clients, where an instance identity is
//! declared attribution and not authentication.
use std::cell::Cell;
use std::collections::{BTreeSet, HashMap};
use std::time::Duration;
use chrono::{DateTime, Utc};
use cultnet_rs::{CultMeshCdnArtifactManifest, CultMeshCdnChunk};
/// The size of one chunk of a deferred body. Four fit one send with room for a
/// control-plane answer beside them on the same session.
pub const DEFERRED_CHUNK_BYTES: usize = 256 * 1024;
/// What the daemon keeps across all deferred bodies.
pub const DEFERRED_BUDGET_BYTES: u64 = 256 * 1024 * 1024;
struct Chunk {
bytes: Vec<u8>,
/// The content hashes of the retained bodies that reference this chunk.
owners: BTreeSet<String>,
}
struct Body {
chunk_hashes: BTreeSet<String>,
touched_at: Cell<DateTime<Utc>>,
/// Orders touches when the clock does not: the largest is the newest.
touch_order: Cell<u64>,
}
pub struct DeferredBodies {
budget: u64,
ttl: chrono::Duration,
chunks: HashMap<String, Chunk>,
bodies: HashMap<String, Body>,
stored: u64,
touches: Cell<u64>,
}
impl DeferredBodies {
pub fn new(budget: u64, ttl: Duration) -> Self {
Self {
budget,
ttl: chrono::Duration::from_std(ttl).unwrap_or(chrono::Duration::MAX),
chunks: HashMap::new(),
bodies: HashMap::new(),
stored: 0,
touches: Cell::new(0),
}
}
/// Bytes held: each distinct chunk once.
#[cfg(test)]
pub fn stored_bytes(&self) -> u64 {
self.stored
}
#[cfg(test)]
pub fn body_count(&self) -> usize {
self.bodies.len()
}
/// Holds a body, or, when its bytes are already held, touches it and
/// stores nothing. Then evicts, least recently touched first, while the
/// stored bytes exceed the budget and another body could be evicted
/// instead: the body just retained is the last to go, and one that alone
/// exceeds the budget is kept whole.
pub fn retain(&mut self, manifest: &CultMeshCdnArtifactManifest, chunks: Vec<CultMeshCdnChunk>, now: DateTime<Utc>) {
let content_hash = manifest.content_hash.clone();
if let Some(body) = self.bodies.get(&content_hash) {
self.touch(body, now);
return;
}
let mut chunk_hashes = BTreeSet::new();
for chunk in chunks {
chunk_hashes.insert(chunk.chunk_hash.clone());
let held = self.chunks.entry(chunk.chunk_hash).or_insert_with(|| {
self.stored += chunk.payload.len() as u64;
Chunk { bytes: chunk.payload, owners: BTreeSet::new() }
});
held.owners.insert(content_hash.clone());
}
let body = Body { chunk_hashes, touched_at: Cell::new(now), touch_order: Cell::new(0) };
self.touch(&body, now);
self.bodies.insert(content_hash, body);
while self.stored > self.budget && self.bodies.len() > 1 {
let oldest = self
.bodies
.iter()
.min_by_key(|(_, body)| body.touch_order.get())
.map(|(hash, _)| hash.clone())
.expect("more than one body is held");
self.drop_body(&oldest);
}
}
/// One chunk's bytes by its normalised hash, touching every body that
/// references it. Serving a chunk is what keeps its body alive.
pub fn chunk(&self, hash: &str, now: DateTime<Utc>) -> Option<&[u8]> {
let chunk = self.chunks.get(hash)?;
for owner in &chunk.owners {
self.touch(&self.bodies[owner], now);
}
Some(&chunk.bytes)
}
/// Drops every body untouched for the TTL as of `now`. A clock that reads
/// earlier than a touch expires nothing.
pub fn expire(&mut self, now: DateTime<Utc>) {
let expired: Vec<String> = self
.bodies
.iter()
.filter(|(_, body)| now.signed_duration_since(body.touched_at.get()) >= self.ttl)
.map(|(hash, _)| hash.clone())
.collect();
for hash in expired {
self.drop_body(&hash);
}
}
fn touch(&self, body: &Body, now: DateTime<Utc>) {
body.touched_at.set(now);
self.touches.set(self.touches.get() + 1);
body.touch_order.set(self.touches.get());
}
fn drop_body(&mut self, content_hash: &str) {
let Some(body) = self.bodies.remove(content_hash) else { return };
for hash in body.chunk_hashes {
let chunk = self.chunks.get_mut(&hash).expect("a retained body's chunks are held");
chunk.owners.remove(content_hash);
if chunk.owners.is_empty() {
self.stored -= chunk.bytes.len() as u64;
self.chunks.remove(&hash);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use cultnet_rs::pack_content;
use huginn_mind::MAX_DEFERRED_BODY_BYTES;
/// The three numbers a client author is told, spelled out: a chunk of
/// 256 KiB, a largest deferred body of 64 MiB, a budget of 256 MiB that
/// holds at least one whole body.
#[test]
fn the_deferral_constants_are_the_documented_numbers() {
assert_eq!(DEFERRED_CHUNK_BYTES, 262_144);
assert_eq!(MAX_DEFERRED_BODY_BYTES, 67_108_864);
assert_eq!(DEFERRED_BUDGET_BYTES, 268_435_456);
}
fn at(seconds: i64) -> DateTime<Utc> {
DateTime::from_timestamp(1_800_000_000 + seconds, 0).unwrap()
}
/// A body of `pieces` chunks of 4 bytes each, distinct per `tag`.
fn body(tag: u8, pieces: usize) -> (CultMeshCdnArtifactManifest, Vec<CultMeshCdnChunk>) {
let mut bytes = Vec::new();
for piece in 0..pieces {
bytes.extend_from_slice(&[tag, piece as u8, 7, 7]);
}
pack_content("t", "package", "", "", "", &bytes, 4).unwrap()
}
fn held(store: &DeferredBodies, manifest: &CultMeshCdnArtifactManifest, now: DateTime<Utc>) -> bool {
manifest.chunks.iter().all(|chunk| store.chunk(&chunk.chunk_hash, now).is_some())
}
const TTL: Duration = Duration::from_secs(60);
/// The same bytes deferred twice are one body, counted once; serving a
/// chunk slides the expiry; an untouched body is gone exactly at the TTL.
#[test]
fn retention_is_idempotent_and_sliding() {
let mut store = DeferredBodies::new(1_000, TTL);
let (manifest, chunks) = body(1, 3);
store.retain(&manifest, chunks.clone(), at(0));
store.retain(&manifest, chunks, at(1));
assert_eq!((store.body_count(), store.stored_bytes()), (1, 12), "one body, its bytes counted once");
// Retaining again at t=1 touched it, so it lives to t=61, not t=60.
store.expire(at(60));
assert_eq!(store.body_count(), 1, "the second retain refreshed it");
// Serving a chunk at t=60 touches it, so t=119 is inside a fresh TTL.
assert!(store.chunk(&manifest.chunks[0].chunk_hash, at(60)).is_some());
store.expire(at(119));
assert_eq!(store.body_count(), 1, "serving a chunk slid the expiry");
store.expire(at(120));
assert_eq!((store.body_count(), store.stored_bytes()), (0, 0), "untouched for the TTL is gone");
// An untouched body is gone at the TTL and not before it.
let (other, chunks) = body(2, 1);
store.retain(&other, chunks, at(200));
store.expire(at(259));
assert_eq!(store.body_count(), 1);
store.expire(at(260));
assert_eq!(store.body_count(), 0);
}
/// Time going backwards expires nothing.
#[test]
fn a_clock_behind_the_last_touch_expires_nothing() {
let mut store = DeferredBodies::new(1_000, TTL);
let (manifest, chunks) = body(1, 1);
store.retain(&manifest, chunks, at(100));
store.expire(at(0));
assert_eq!(store.body_count(), 1);
}
/// Over the budget the body touched longest ago goes first, not the
/// newest and not the first inserted, and a chunk two bodies share stays
/// while either lives.
#[test]
fn eviction_is_least_recently_touched_and_bounded_in_bytes() {
// Budget 12 bytes: three chunks of 4.
let mut store = DeferredBodies::new(12, TTL);
let (a, chunks_a) = body(1, 1);
let (b, chunks_b) = body(2, 1);
let (c, chunks_c) = body(3, 1);
store.retain(&a, chunks_a, at(0));
store.retain(&b, chunks_b, at(1));
// The oldest-inserted body is touched last, so insertion order and
// recency disagree about who goes.
assert!(store.chunk(&a.chunks[0].chunk_hash, at(2)).is_some());
store.retain(&c, chunks_c, at(3));
assert_eq!(store.stored_bytes(), 12);
let (d, chunks_d) = body(4, 1);
store.retain(&d, chunks_d, at(4));
assert_eq!(store.stored_bytes(), 12, "bounded by the budget");
assert!(!held(&store, &b, at(5)), "the least recently touched went");
assert!(held(&store, &a, at(5)) && held(&store, &c, at(5)) && held(&store, &d, at(5)));
// A shared chunk survives its first owner.
let mut store = DeferredBodies::new(12, TTL);
let (x, chunks_x) = body(5, 2); // chunks x0 x1
store.retain(&x, chunks_x, at(0));
let (y, mut chunks_y) = body(6, 1); // chunk y0, plus x1 shared
let mut manifest_y = y.clone();
manifest_y.content_hash = "shared-owner".into();
chunks_y.push(CultMeshCdnChunk {
chunk_hash: x.chunks[1].chunk_hash.clone(),
payload: vec![5, 1, 7, 7],
});
store.retain(&manifest_y, chunks_y, at(1));
assert_eq!(store.stored_bytes(), 12, "the shared chunk is stored once");
// Evict x (older): its private chunk goes, the shared one stays.
let (z, chunks_z) = body(7, 1);
store.retain(&z, chunks_z, at(2));
assert_eq!(store.body_count(), 2, "x was evicted");
assert!(store.chunk(&x.chunks[0].chunk_hash, at(3)).is_none(), "x's private chunk is gone");
assert!(store.chunk(&x.chunks[1].chunk_hash, at(3)).is_some(), "the shared chunk lives with y");
}
/// A body that alone exceeds the budget is kept whole rather than
/// evicting itself, so a manifest handed out is never a lie.
#[test]
fn a_body_over_the_budget_by_itself_is_kept() {
let mut store = DeferredBodies::new(4, TTL);
let (manifest, chunks) = body(1, 3);
store.retain(&manifest, chunks, at(0));
assert!(held(&store, &manifest, at(0)));
}
}

View file

@ -0,0 +1,864 @@
//! The dispatch: one mind, one request, one response. No socket, no clock, no
//! rule. `now` is the caller's, so a test pins it and the loop is the only
//! thing that reads a wall clock.
use chrono::{DateTime, Utc};
use cultnet_rs::Selection;
use huginn_mind::epiphany_pipeline::{PipelineRef, Slug};
use huginn_mind::wire::{HuginnMindRequest, HuginnMindResponse, IndexStatus};
use huginn_mind::{Mind, MindRefusal, MindStore, PipelineAdmissionOutcome, SemanticQuery};
/// Where the index attaches. Called after a batch commits, with the refs the
/// batch landed, derived writes included; the sink may read each through
/// `mind.view`. Its error is logged and never reaches the caller: the index
/// does not decide whether a batch was admitted, and the call must not wait on
/// the index's transport. `status` is what `whoami` reports of it.
///
/// A semantic query is asked in two steps because the index answers on its own
/// time: `search` starts one and returns a ticket at once, and `searched`
/// hands over the searches that have finished since it was last called. The
/// candidates are the index's word about nearness and nothing else; the daemon
/// judges them through the mind.
pub trait IndexSink<S: MindStore> {
fn committed(&mut self, mind: &Mind<S>, writes: &[PipelineRef]) -> anyhow::Result<()>;
fn status(&self) -> IndexStatus;
/// Starts a search for the documents nearest `text`, at most `top_k` of
/// them wanted. `Err` says why the index cannot be asked now.
fn search(&mut self, text: &str, top_k: u32) -> Result<SearchTicket, String>;
/// How long after it was asked a search is worth answering: the index's
/// own bound on it, which the loop waiting for the answer must share.
fn search_deadline(&self) -> std::time::Duration;
/// Every search that finished since the last call, by ticket: its
/// candidates and scores, or why it failed.
fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)>;
/// The asker has stopped waiting for `ticket`. A search not yet started is
/// dropped, so a queue of searches nobody wants cannot hold the index's
/// writes off.
fn abandon(&mut self, ticket: SearchTicket);
}
/// Names one search the index was asked, so the loop can match the answer to
/// the client that is waiting for it.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct SearchTicket(pub u64);
/// What a search returns: candidate documents and how near each scored, best
/// first. Nothing else about them.
pub type Hits = Vec<(PipelineRef, f32)>;
/// A semantic query the index has been asked and has not yet answered. The
/// serve loop holds it beside the session that asked, and hands it back to
/// `Daemon::finish` with the index's answer.
pub struct Search {
pub ticket: SearchTicket,
selection: Selection,
semantic: SemanticQuery,
}
/// What a request came to: an answer, or a search the index has yet to
/// finish.
pub enum Handled {
Answered(HuginnMindResponse),
Searching(Search),
}
impl Handled {
/// The answer, for a request that cannot be a search.
#[cfg(test)]
pub(crate) fn answered(self) -> HuginnMindResponse {
match self {
Self::Answered(response) => response,
Self::Searching(search) => panic!("expected an answer, got the search {:?}", search.ticket),
}
}
}
/// What the log says of the hits the mind held no document for: how many, and
/// the first few by name (a restored mind can leave hundreds), or nothing when
/// the mind held them all.
fn dropped_note(dropped: &[PipelineRef]) -> Option<String> {
if dropped.is_empty() {
return None;
}
let named: Vec<&str> = dropped.iter().take(5).map(|hit| hit.id.0.as_str()).collect();
Some(format!(
"huginn: the index returned {} ids this mind does not hold (first {named:?}); they are left out",
dropped.len()
))
}
/// The envelope's `source_runtime_id` on every response a daemon serving this
/// instance sends. Also what the socket is bound under, before a daemon exists.
pub fn runtime_id(instance: &Slug) -> String {
format!("huginn-{}", instance.0)
}
/// One mind and whatever indexes what lands in it.
pub struct Daemon<S: MindStore, I: IndexSink<S>> {
mind: Mind<S>,
index: I,
}
impl<S: MindStore, I: IndexSink<S>> Daemon<S, I> {
/// A daemon over a mind that is already open. `Mind::open` takes the
/// store's exclusive lock for the daemon's lifetime, and ruling 15 has it
/// refuse before any socket exists: the caller opens the mind first.
pub fn new(mind: Mind<S>, index: I) -> Self {
Self { mind, index }
}
pub fn with_index<J: IndexSink<S>>(self, index: J) -> Daemon<S, J> {
Daemon { mind: self.mind, index }
}
pub fn mind(&self) -> &Mind<S> {
&self.mind
}
pub fn index(&self) -> &I {
&self.index
}
/// The envelope's `source_runtime_id` on every response this daemon sends.
pub fn runtime_id(&self) -> String {
runtime_id(self.mind.instance())
}
/// One request, one answer, or a search to finish. `Admit` goes to the mind whole, because the
/// batch declares its own instance and admission's A1 is the check. Every
/// other request names an instance: `require_instance` now applies the
/// leaf's own grammar before it compares identity, so a name outside the
/// grammar (a fullwidth fold, say) is refused by name rather than read as
/// merely a foreign mind, and the daemon asks nothing twice.
pub fn handle(&mut self, request: HuginnMindRequest, now: DateTime<Utc>) -> Handled {
if let Some(declared) = request.instance()
&& !matches!(request, HuginnMindRequest::Admit(_))
&& let Err(refusal) = self.mind.require_instance(declared)
{
return Handled::Answered(HuginnMindResponse::Refused(refusal));
}
let answer = match request {
HuginnMindRequest::Whoami => HuginnMindResponse::Whoami(self.mind.status(self.index.status())),
HuginnMindRequest::Admit(batch) => {
let outcome = self.mind.admit(batch, now);
if let PipelineAdmissionOutcome::Committed { writes, .. } = &outcome
&& let Err(error) = self.index.committed(&self.mind, writes)
{
let refs = writes.iter().map(|write| write.id.0.clone()).collect::<Vec<_>>().join(", ");
eprintln!("huginn: the index refused the landed writes [{refs}]: {error:#}");
}
HuginnMindResponse::Admit(outcome)
}
HuginnMindRequest::View { id, .. } => match self.mind.view(&id) {
Ok(view) => HuginnMindResponse::View(view),
Err(refusal) => HuginnMindResponse::Refused(refusal),
},
HuginnMindRequest::Query { selection, semantic: None, .. } => match self.mind.query(&selection) {
Ok(page) => HuginnMindResponse::Query(page),
Err(refusal) => HuginnMindResponse::Refused(refusal),
},
HuginnMindRequest::Query { selection, semantic: Some(semantic), .. } => {
return self.begin_search(selection, semantic);
}
};
Handled::Answered(answer)
}
/// A semantic query starts only if the mind would accept it (so nothing is
/// embedded for a query that cannot be ranked) and the index can be asked.
fn begin_search(&mut self, selection: Selection, semantic: SemanticQuery) -> Handled {
let refused = |refusal: MindRefusal| Handled::Answered(HuginnMindResponse::Refused(refusal));
if let Err(refusal) = self.mind.check_semantic(&selection, &semantic) {
return refused(refusal);
}
match self.index.search(&semantic.text.0, semantic.top_k) {
Ok(ticket) => Handled::Searching(Search { ticket, selection, semantic }),
Err(detail) => refused(MindRefusal::Unavailable { detail }),
}
}
/// How long a search is worth waiting for.
pub fn search_deadline(&self) -> std::time::Duration {
self.index.search_deadline()
}
/// Searches the index has finished since this was last called.
pub fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)> {
self.index.searched()
}
/// The client has stopped waiting for `ticket`.
pub fn abandon(&mut self, ticket: SearchTicket) {
self.index.abandon(ticket);
}
/// The answer to a search: the index's candidates ranked through the mind
/// (`Mind::rank` drops what the mind does not hold and lets the selection
/// decide the rest), or, when the index could not answer, a typed
/// `Unavailable`. Never an empty page in place of a failure.
pub fn finish(&mut self, search: Search, found: Result<Hits, String>) -> HuginnMindResponse {
let hits = match found {
Ok(hits) => hits,
Err(detail) => return HuginnMindResponse::Refused(MindRefusal::Unavailable { detail }),
};
match self.mind.rank(&search.selection, &search.semantic, &hits) {
Ok(ranked) => {
if let Some(note) = dropped_note(&ranked.dropped) {
eprintln!("{note}");
}
HuginnMindResponse::Query(ranked.page)
}
Err(refusal) => HuginnMindResponse::Refused(refusal),
}
}
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
use chrono::TimeZone;
use huginn_mind::epiphany_pipeline::{
AuthorityMap, CodeLocation, CutDelete, CutVerification, Date, DocRef, FileChange, Line, NegativeCheck,
OrgRepo, PipelineCampaign, PipelineCutSpec, PipelineDocument, PipelineInstance, PipelineKind,
PipelineQuestion, PipelineRuling, PipelineStewardship, QuestionOption, RulingAuthority, Sha, Short,
StructuralDelta, Title, VerificationTest,
};
use huginn_mind::wire::MindStatus;
use huginn_mind::{MindRefusal, OwnedRedbMessagePackBackingStore};
use std::path::Path;
use cultnet_rs::Selection;
use huginn_mind::{Faculty, PipelineAdmissionBatch, PipelineProvenance, PipelinePageItems, PipelineStatus};
use std::sync::{Arc, Mutex};
use tempfile::TempDir;
pub(crate) const INSTANCE: &str = "yggdrasil";
pub(crate) const OTHER: &str = "thought-cage";
/// `INSTANCE`'s own length, differing in its last byte alone.
pub(crate) const NEAR: &str = "yggdrasix";
/// `INSTANCE` whole, with more after it.
pub(crate) const PREFIXED: &str = "yggdrasil-two";
/// `INSTANCE` in another case, which a slug label permits.
pub(crate) const CASED: &str = "Yggdrasil";
pub(crate) const CAMPAIGN: &str = "eureka-state";
pub(crate) fn slug(value: &str) -> Slug {
value.into()
}
pub(crate) fn now() -> DateTime<Utc> {
Utc.with_ymd_and_hms(2026, 9, 16, 12, 0, 0).unwrap()
}
pub(crate) fn provenance() -> PipelineProvenance {
PipelineProvenance {
faculty: Faculty::Hands,
agent: Short("claude".into()),
session: Short("session-1".into()),
tool: Short("admit".into()),
}
}
pub(crate) fn identity(name: &str) -> PipelineDocument {
PipelineDocument::Instance(PipelineInstance {
instance: slug(name),
display_name: Short(format!("{name} mind")),
created_at: Date("2026-09-16".into()),
host: Short(name.into()),
})
}
pub(crate) fn batch(instance: &str, documents: Vec<PipelineDocument>) -> PipelineAdmissionBatch {
PipelineAdmissionBatch { instance: slug(instance), provenance: provenance(), documents }
}
/// A question and the ruling that answers it: admitting the ruling derives
/// the question's resolution, so one batch lands two writes.
pub(crate) fn question_and_ruling() -> (PipelineDocument, PipelineDocument) {
let question = PipelineDocument::Question(PipelineQuestion {
campaign: slug(CAMPAIGN),
label: "Q1".into(),
title: Title("Who owns the state?".into()),
question: "Who owns the state?".into(),
options: vec![
QuestionOption { label: "A".into(), text: "the organ".into() },
QuestionOption { label: "B".into(), text: "the client".into() },
],
recommended: "A".into(),
depends: vec![],
raised_in: None,
asked_on: Date("2026-09-16".into()),
});
let ruling = PipelineDocument::Ruling(PipelineRuling {
campaign: slug(CAMPAIGN),
label: "R1".into(),
title: Title("An instance owns its mind".into()),
answers: Some(Short(format!("{CAMPAIGN}:question:Q1"))),
choice: Some("A".into()),
ruling: "An instance owns its mind.".into(),
operator_quote: None,
ruled_on: Date("2026-09-16".into()),
precedents: vec![],
authority: RulingAuthority::Operator,
});
(question, ruling)
}
pub(crate) const REPO: &str = "GameCult/Epiphany";
/// A `Line` and a `Short` at the leaf's own bound, so a document built out
/// of them is as wide as any writer may make it.
pub(crate) fn line(n: usize) -> Line {
Line(format!("{n:0>1000}"))
}
pub(crate) fn short(n: usize) -> Short {
Short(format!("{n:0>200}"))
}
fn sha() -> Sha {
Sha("5f98228d".into())
}
fn doc_ref() -> DocRef {
DocRef { path: Short("notes/target.md".into()), start_line: 1, end_line: 9, commit: sha() }
}
/// The three documents a cut spec needs before it may be admitted: the
/// mind's identity, its stewardship of the repo, and the campaign that
/// names the repo.
pub(crate) fn campaign_seed() -> Vec<PipelineDocument> {
vec![
identity(INSTANCE),
PipelineDocument::Stewardship(PipelineStewardship {
instance: slug(INSTANCE),
repo: OrgRepo(REPO.into()),
sequence: 1,
assigned_on: Date("2026-09-16".into()),
note: "assigned".into(),
}),
PipelineDocument::Campaign(PipelineCampaign {
slug: slug(CAMPAIGN),
title: Title("Eureka pipeline state".into()),
repos: vec![OrgRepo(REPO.into())],
working_branch: Short("codex/eureka-pipeline-state".into()),
target_doc: doc_ref(),
}),
]
}
/// One cut spec with every list the leaf bounds filled to its maximum and
/// every text at its own: `file_changes` file changes, 64 deletes, keeps,
/// adds, builds, tests, negative checks and operator checks, a full
/// authority map and a full structural delta. At 256 file changes that is
/// roughly a megabyte of field content, which is what a real-sized answer
/// looks like at the leaf's bounds rather than at a fixture's convenience.
/// `depends_on`, `rulings` and `questions` stay empty because each is a
/// reference admission resolves.
pub(crate) fn cut_spec(cut: &str, file_changes: usize) -> PipelineDocument {
let lines = |count: usize| (0..count).map(line).collect::<Vec<_>>();
let shorts = |count: usize| (0..count).map(short).collect::<Vec<_>>();
let location = |n: usize| CodeLocation { path: short(n), line: 1, end_line: Some(9) };
PipelineDocument::CutSpec(PipelineCutSpec {
campaign: slug(CAMPAIGN),
cut: cut.into(),
revision: 1,
title: Title(short(0).0),
repo: OrgRepo(REPO.into()),
branch: short(1),
base: sha(),
depends_on: vec![],
first: lines(16),
deletes: (0..64).map(|n| CutDelete { path: short(n), lines: 9, note: line(n) }).collect(),
keeps_moves: lines(64),
adds: lines(64),
file_changes: (0..file_changes).map(|n| FileChange { location: location(n), change: line(n) }).collect(),
authority_map: Some(AuthorityMap {
owner: line(0),
inputs: lines(16),
outputs: lines(16),
derived_state: lines(16),
forbidden_writers: lines(16),
shared_paths: lines(16),
deletion_line: line(1),
}),
verification: CutVerification {
builds: lines(64),
tests: (0..64).map(|n| VerificationTest { name: short(n), pins: line(n) }).collect(),
negative: (0..64).map(|n| NegativeCheck { pattern: short(n), scope: line(n) }).collect(),
operator: lines(64),
},
estimate: StructuralDelta {
lines_added: 0,
lines_removed: 0,
dependencies_added: shorts(64),
dependencies_removed: shorts(64),
formats_added: shorts(64),
formats_removed: shorts(64),
targets_added: shorts(64),
targets_removed: shorts(64),
},
rulings: vec![],
questions: vec![],
})
}
/// The cut spec at the leaf's own maximum, and one narrow enough that its
/// answer still fits a single send. `WIDE_CHANGES` is the leaf's bound;
/// `FITTING_CHANGES` is chosen so the view of that spec lands inside the
/// window with less than a packet-count's slack, which is what makes the
/// window's own value load-bearing rather than merely generous.
pub(crate) const WIDE_CUT: &str = "wide";
pub(crate) const WIDE_CHANGES: usize = 256;
pub(crate) const FITTING_CUT: &str = "fits";
pub(crate) const FITTING_CHANGES: usize = 180;
/// A daemon whose mind holds the campaign seed, one cut spec as wide as the
/// leaf permits, and one narrow enough to answer, so a read over it is a
/// real-sized answer either way.
pub(crate) fn seeded_wide() -> (TempDir, Daemon<OwnedRedbMessagePackBackingStore, NoIndex>) {
let root = tempfile::tempdir().unwrap();
let mut daemon = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
for documents in [
campaign_seed(),
vec![cut_spec(WIDE_CUT, WIDE_CHANGES), cut_spec(FITTING_CUT, FITTING_CHANGES)],
] {
let outcome = daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, documents)), now()).answered();
assert!(
matches!(outcome, HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { .. })),
"{outcome:?}"
);
}
(root, daemon)
}
/// A daemon over a real redb mind in a temporary directory, with its
/// identity admitted. The directory is returned because dropping it
/// removes the store.
pub(crate) fn seeded() -> (TempDir, Daemon<OwnedRedbMessagePackBackingStore, NoIndex>) {
let root = tempfile::tempdir().unwrap();
let mut daemon = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
let outcome = daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)])), now()).answered();
assert!(
matches!(outcome, HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { .. })),
"{outcome:?}"
);
(root, daemon)
}
/// The log names how many hits the mind did not hold and the first five, and
/// says nothing when it held them all.
#[test]
fn the_dropped_hits_are_reported_by_count_and_the_first_five_names() {
assert_eq!(dropped_note(&[]), None);
let dropped: Vec<PipelineRef> =
(0..7).map(|n| PipelineRef { kind: PipelineKind::Question, id: Short(format!("gone-{n}")) }).collect();
let note = dropped_note(&dropped).unwrap();
assert!(note.contains("returned 7 ids"), "{note}");
assert!(note.contains("gone-0") && note.contains("gone-4") && !note.contains("gone-5"), "{note}");
assert_eq!(dropped_note(&dropped[..1]).unwrap().matches("gone-").count(), 1);
}
/// No index: for the tests whose subject is not the projection.
pub(crate) struct NoIndex;
impl<S: MindStore> IndexSink<S> for NoIndex {
fn committed(&mut self, _mind: &Mind<S>, _writes: &[PipelineRef]) -> anyhow::Result<()> {
Ok(())
}
fn status(&self) -> IndexStatus {
IndexStatus::Current
}
fn search(&mut self, _text: &str, _top_k: u32) -> Result<SearchTicket, String> {
Err("this index cannot search".into())
}
fn search_deadline(&self) -> std::time::Duration {
std::time::Duration::ZERO
}
fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)> {
Vec::new()
}
fn abandon(&mut self, _ticket: SearchTicket) {}
}
/// A daemon over the instance's mind under `state_root`, with no index.
pub(crate) fn open_unindexed(
state_root: &Path,
instance: &Slug,
) -> Result<Daemon<OwnedRedbMessagePackBackingStore, NoIndex>, MindRefusal> {
Ok(Daemon::new(Mind::open(state_root, instance)?, NoIndex))
}
#[derive(Clone, Default)]
struct RecordingIndex {
seen: Arc<Mutex<Vec<(Vec<String>, Vec<bool>)>>>,
}
impl<S: MindStore> IndexSink<S> for RecordingIndex {
fn committed(&mut self, mind: &Mind<S>, writes: &[PipelineRef]) -> anyhow::Result<()> {
let ids = writes.iter().map(|write| write.id.0.clone()).collect();
let readable = writes.iter().map(|write| mind.view(write).unwrap().is_some()).collect();
self.seen.lock().unwrap().push((ids, readable));
Ok(())
}
fn status(&self) -> IndexStatus {
IndexStatus::Current
}
fn search(&mut self, _text: &str, _top_k: u32) -> Result<SearchTicket, String> {
Err("this index cannot search".into())
}
fn search_deadline(&self) -> std::time::Duration {
std::time::Duration::ZERO
}
fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)> {
Vec::new()
}
fn abandon(&mut self, _ticket: SearchTicket) {}
}
struct FailingIndex;
impl<S: MindStore> IndexSink<S> for FailingIndex {
fn committed(&mut self, _mind: &Mind<S>, _writes: &[PipelineRef]) -> anyhow::Result<()> {
Err(anyhow::anyhow!("the index is not there"))
}
fn status(&self) -> IndexStatus {
IndexStatus::Current
}
fn search(&mut self, _text: &str, _top_k: u32) -> Result<SearchTicket, String> {
Err("this index cannot search".into())
}
fn search_deadline(&self) -> std::time::Duration {
std::time::Duration::ZERO
}
fn searched(&mut self) -> Vec<(SearchTicket, Result<Hits, String>)> {
Vec::new()
}
fn abandon(&mut self, _ticket: SearchTicket) {}
}
fn status(daemon: &mut Daemon<OwnedRedbMessagePackBackingStore, impl IndexSink<OwnedRedbMessagePackBackingStore>>) -> MindStatus {
match daemon.handle(HuginnMindRequest::Whoami, now()).answered() {
HuginnMindResponse::Whoami(status) => status,
other => panic!("expected a status, got {other:?}"),
}
}
/// Typed hand-off between the organs without a socket: a batch enters and
/// every read answers over what it landed, at a clock the test pins.
#[test]
fn a_batch_round_trips_typed_through_handle_without_a_socket() {
let root = tempfile::tempdir().unwrap();
let mut daemon = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
let outcome = daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)])), now()).answered();
let HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { writes, .. }) = outcome else {
panic!("expected a commit");
};
assert_eq!(writes.len(), 1);
assert_eq!(writes[0].kind, PipelineKind::Instance);
assert_eq!(
status(&mut daemon),
MindStatus {
instance: slug(INSTANCE),
schema_epoch: "epiphany.pipeline.epoch.v2".into(),
documents: 1,
receipts: 1,
index: IndexStatus::Current,
}
);
assert_eq!(daemon.runtime_id(), "huginn-yggdrasil");
let instances = Selection { schemas: Some(vec![PipelineKind::Instance.type_id().into()]), ..Selection::default() };
let HuginnMindResponse::Query(page) = daemon.handle(
HuginnMindRequest::Query { instance: slug(INSTANCE), selection: instances.clone(), semantic: None },
now(),
).answered() else {
panic!("expected a page");
};
assert_eq!(page.matched, 1);
let PipelinePageItems::Headers(headers) = &page.items else { panic!("the default projection is a header") };
assert_eq!(headers[0].admission.provenance, provenance());
assert_eq!(headers[0].status, huginn_mind::PipelineStatusSummary::InForce);
let id = writes[0].clone();
let whole = Selection { projection: "document".into(), ..instances };
let HuginnMindResponse::Query(page) =
daemon.handle(HuginnMindRequest::Query { instance: slug(INSTANCE), selection: whole, semantic: None }, now()).answered()
else {
panic!("expected a page");
};
let PipelinePageItems::Documents(views) = &page.items else { panic!("the document projection is the view") };
assert_eq!(views[0].status, PipelineStatus::InForce);
let view = daemon.handle(HuginnMindRequest::View { instance: slug(INSTANCE), id: id.clone() }, now()).answered();
assert_eq!(view, HuginnMindResponse::View(Some(views[0].clone())));
}
/// Ruling 14 across the transport, on both sides: the mind refuses a read
/// and a write that name another instance, and nothing lands.
#[test]
fn a_read_or_a_write_naming_another_instance_is_refused_by_the_mind_and_writes_nothing() {
let root = tempfile::tempdir().unwrap();
let mut daemon = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
let landed = daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)])), now()).answered();
let HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { writes, .. }) = landed else {
panic!("expected a commit");
};
let foreign = MindRefusal::ForeignInstance { declared: OTHER.into(), mind: INSTANCE.into() };
let query = HuginnMindRequest::Query { instance: slug(OTHER), selection: Selection::default(), semantic: None };
assert_eq!(daemon.handle(query, now()).answered(), HuginnMindResponse::Refused(foreign.clone()));
let admit = HuginnMindRequest::Admit(batch(OTHER, vec![identity(OTHER)]));
assert_eq!(
daemon.handle(admit, now()).answered(),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(foreign.clone()))
);
// The declared instance travels to the mind as the client sent it. A
// batch whose documents name no instance at all is still refused on
// the declaration alone, so nothing between the wire and A1 may
// rewrite it into the mind's own.
let (question, _) = question_and_ruling();
let admit = HuginnMindRequest::Admit(batch(OTHER, vec![question]));
assert_eq!(
daemon.handle(admit, now()).answered(),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(foreign.clone()))
);
let view = HuginnMindRequest::View { instance: slug(OTHER), id: writes[0].clone() };
assert_eq!(daemon.handle(view, now()).answered(), HuginnMindResponse::Refused(foreign));
// Three names a comparison could mistake for this mind's. `NEAR` is
// `INSTANCE`'s own length and differs in its last byte, so a check
// reading lengths, first bytes, or running only when the declared name
// is longer admits it; `PREFIXED` carries `INSTANCE` whole, so a prefix
// test admits it; `CASED` is `INSTANCE` in another case, which a slug
// label permits, so a check folding case admits it. Each is refused on
// the read side and the write side.
assert_eq!(NEAR.len(), INSTANCE.len());
assert!(PREFIXED.starts_with(INSTANCE));
assert_eq!(CASED.to_ascii_lowercase(), INSTANCE);
for declared in [NEAR, PREFIXED, CASED] {
let refusal = MindRefusal::ForeignInstance { declared: declared.into(), mind: INSTANCE.into() };
let query = HuginnMindRequest::Query { instance: slug(declared), selection: Selection::default(), semantic: None };
assert_eq!(daemon.handle(query, now()).answered(), HuginnMindResponse::Refused(refusal.clone()), "{declared}");
let view = HuginnMindRequest::View { instance: slug(declared), id: writes[0].clone() };
assert_eq!(daemon.handle(view, now()).answered(), HuginnMindResponse::Refused(refusal.clone()), "{declared}");
let admit = HuginnMindRequest::Admit(batch(declared, vec![identity(declared)]));
assert_eq!(
daemon.handle(admit, now()).answered(),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(refusal)),
"{declared}"
);
}
assert_eq!(status(&mut daemon).documents, 1, "nothing landed");
}
/// A declared instance outside `Slug`'s grammar is refused by its own
/// name, `InvalidFormat`, on the read path, before the identity
/// comparison ever runs: it is not merely a foreign mind, it names no
/// mind a client could compose. A fullwidth Latin small letter y is used
/// because it is the shape Soul named: a check that folds it to plain
/// ASCII `y` before validating would let this probe pass as grammatical
/// and reach `require_instance`, which would then answer `ForeignInstance`
/// instead — the wrong refusal, reached the wrong way.
#[test]
fn a_declared_instance_outside_the_grammar_is_refused_by_name_before_the_identity_check() {
let (_root, mut daemon) = seeded();
let fullwidth = "\u{FF59}ggdrasil";
assert_ne!(fullwidth, INSTANCE, "not the mind's own bytes");
assert!(!fullwidth.is_ascii(), "outside the grammar: a Slug is dot-joined ASCII labels");
let expected = MindRefusal::Document(huginn_mind::epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "declared".into(),
value: fullwidth.into(),
});
let query = HuginnMindRequest::Query { instance: slug(fullwidth), selection: Selection::default(), semantic: None };
assert_eq!(daemon.handle(query, now()).answered(), HuginnMindResponse::Refused(expected.clone()));
let view = HuginnMindRequest::View {
instance: slug(fullwidth),
id: PipelineRef { kind: PipelineKind::Question, id: Short("not-reached".into()) },
};
assert_eq!(daemon.handle(view, now()).answered(), HuginnMindResponse::Refused(expected));
}
/// `Admit` is excluded from the daemon's own pre-check (its batch is not a
/// read naming an instance the mind is asked about first), but it reaches
/// the identical `require_instance` from inside `admit_steps`'s A1, so a
/// fullwidth *declared batch instance* is `InvalidFormat` there too,
/// named and valued the same as the read path's. One door,
/// `Mind::require_instance`, decides both. The batch's one document names
/// the mind's own real instance, `INSTANCE`, so it clears `admit`'s own
/// per-document `Bounded` check and the only fullwidth text anywhere in
/// the request is the batch's declared instance, the thing this test
/// pins.
#[test]
fn a_fullwidth_instance_is_refused_the_same_way_by_admit_and_by_query() {
let (_root, mut daemon) = seeded();
let fullwidth = "\u{FF59}ggdrasil";
let expected = MindRefusal::Document(huginn_mind::epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "declared".into(),
value: fullwidth.into(),
});
let admit = HuginnMindRequest::Admit(batch(fullwidth, vec![identity(INSTANCE)]));
assert_eq!(
daemon.handle(admit, now()).answered(),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(expected.clone()))
);
let query = HuginnMindRequest::Query { instance: slug(fullwidth), selection: Selection::default(), semantic: None };
assert_eq!(daemon.handle(query, now()).answered(), HuginnMindResponse::Refused(expected));
}
/// A refusal a read method raised is the answer the dispatch returns, whole.
/// The instance check refuses before the method runs and is pinned above;
/// these are the refusals the methods themselves raise, so nothing here is
/// the daemon's own. Each must arrive as the mind spelled it: not rewrapped
/// as `Unavailable`, not rewrapped as another read's refusal, and not
/// swallowed into an empty answer, which would read as "no such document"
/// rather than "that is not a reference".
#[test]
fn a_refusal_a_read_raised_is_the_answer_the_dispatch_returns_whole() {
let (_root, mut daemon) = seeded();
// A reference whose kind and id disagree names no document any writer
// could have composed; the leaf's own refusal says so.
let invalid = PipelineRef { kind: PipelineKind::Question, id: "not a reference".into() };
let malformed = MindRefusal::Document(huginn_mind::epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "ref.id".into(),
value: "not a reference".into(),
});
let view = HuginnMindRequest::View { instance: slug(INSTANCE), id: invalid };
assert_eq!(daemon.handle(view, now()).answered(), HuginnMindResponse::Refused(malformed));
// And the same reads answer rather than refuse when they are asked
// something answerable, so the arms above are refusing on the mind's
// word and not on their own shape.
let absent =
PipelineRef { kind: PipelineKind::Question, id: Short(format!("{CAMPAIGN}:question:Q1")) };
let present = HuginnMindRequest::View { instance: slug(INSTANCE), id: absent };
assert_eq!(daemon.handle(present, now()).answered(), HuginnMindResponse::View(None));
let plain = HuginnMindRequest::Query { instance: slug(INSTANCE), selection: Selection::default(), semantic: None };
let HuginnMindResponse::Query(page) = daemon.handle(plain, now()).answered() else { panic!("expected a page") };
assert_eq!(page.matched, 1);
}
/// A refusal of the selection crosses the dispatch as itself: typed, on
/// the field the client sent, and never swallowed into an empty page or
/// rewrapped as an unavailable store. A stale cursor is the organ's own
/// defect, so a valid walk never meets it.
#[test]
fn a_selection_refusal_crosses_the_dispatch_as_itself() {
let (_root, mut daemon) = seeded();
let ask = |daemon: &mut Daemon<OwnedRedbMessagePackBackingStore, NoIndex>, selection: Selection| {
daemon.handle(HuginnMindRequest::Query { instance: slug(INSTANCE), selection, semantic: None }, now()).answered()
};
let predicate = |index: &str, value: &str| Selection {
fields: Some(vec![cultnet_rs::FieldPredicate {
index: index.into(),
op: "any_of".into(),
values: Some(vec![value.into()]),
number: None,
}]),
..Selection::default()
};
let HuginnMindResponse::Refused(MindRefusal::SelectionInvalid { field, .. }) =
ask(&mut daemon, predicate("nonsense", "x"))
else {
panic!("an undeclared alias is the substrate's refusal");
};
assert_eq!(field, "fields[0].index");
let HuginnMindResponse::Refused(MindRefusal::SelectionInvalid { field, value, .. }) =
ask(&mut daemon, predicate("repo", "a/b/c"))
else {
panic!("a value outside its domain is the organ's refusal");
};
assert_eq!((field.as_str(), value.as_deref()), ("fields[0].values", Some("a/b/c")));
let garbled = Selection { cursor: Some("not a cursor".into()), ..Selection::default() };
assert!(matches!(ask(&mut daemon, garbled), HuginnMindResponse::Refused(MindRefusal::CursorInvalid { .. })));
assert_eq!(ask(&mut daemon, predicate("repo", "a/b/c")).status(), "rejected");
}
/// Cut 11's seam: the index is handed every landed write, derived ones
/// included, and can read each; an index that fails changes nothing.
#[test]
fn the_index_is_handed_every_landed_write_after_commit_and_never_decides_the_outcome() {
let (_root, daemon) = seeded();
let recording = RecordingIndex::default();
let mut daemon = daemon.with_index(recording.clone());
let (question, ruling) = question_and_ruling();
let outcome = daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![question, ruling])), now()).answered();
let HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { writes, .. }) = outcome else {
panic!("expected a commit");
};
let landed: Vec<String> = writes.iter().map(|write| write.id.0.clone()).collect();
assert_eq!(landed.len(), 3, "the ruling answers the question, so a resolution is derived: {landed:?}");
let seen = recording.seen.lock().unwrap().clone();
assert_eq!(seen.len(), 1);
assert_eq!(seen[0].0, landed, "the sink sees exactly the landed writes");
assert_eq!(seen[0].1, vec![true, true, true], "and can read each one when it is called");
let (question, ruling) = question_and_ruling();
let mut daemon = daemon.with_index(FailingIndex);
let root = tempfile::tempdir().unwrap();
let mut fresh = open_unindexed(root.path(), &slug(INSTANCE)).unwrap().with_index(FailingIndex);
fresh.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)])), now()).answered();
let outcome = fresh.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![question, ruling])), now()).answered();
let HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { writes, .. }) = outcome else {
panic!("a failing index does not refuse the batch");
};
for write in &writes {
let view = fresh.handle(HuginnMindRequest::View { instance: slug(INSTANCE), id: write.clone() }, now()).answered();
assert!(matches!(view, HuginnMindResponse::View(Some(_))), "{write:?} is readable");
}
assert_eq!(status(&mut daemon).documents, 4, "identity, question, ruling and the derived resolution");
}
/// Ruling 15: a mind that will not open is refused, and opening it takes
/// no address, so no socket can precede it. `startup`'s order is
/// pinned in `serve::tests`.
#[test]
fn the_daemon_refuses_loudly_when_it_cannot_open_the_mind_and_binds_nothing() {
let root = tempfile::tempdir().unwrap();
let held = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
let path = Mind::<OwnedRedbMessagePackBackingStore>::store_path_for(root.path(), &slug(INSTANCE)).unwrap();
assert_eq!(
open_unindexed(root.path(), &slug(INSTANCE)).err(),
Some(MindRefusal::MindAlreadyOwned { path: path.display().to_string() })
);
drop(held);
let mut planted = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
planted.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)])), now()).answered();
drop(planted);
let moved = Mind::<OwnedRedbMessagePackBackingStore>::store_path_for(root.path(), &slug(OTHER)).unwrap();
std::fs::create_dir_all(moved.parent().unwrap()).unwrap();
std::fs::copy(&path, &moved).unwrap();
assert_eq!(
open_unindexed(root.path(), &slug(OTHER)).err(),
Some(MindRefusal::ForeignInstance { declared: OTHER.into(), mind: INSTANCE.into() })
);
}
}

View file

@ -0,0 +1,239 @@
//! In-memory ports for the projector's tests. Each fake is a handle on shared
//! state, so a test keeps observing after it has moved the fake into a
//! projector or a worker thread.
use std::collections::{BTreeMap, BTreeSet};
use std::sync::{Arc, Condvar, Mutex};
use std::time::Duration;
use anyhow::{Result, bail, ensure};
use super::{CollectionMeta, Described, Embedder, Hit, ModelIdentity, Point, VectorIndex};
pub(crate) fn identity(digest: &str) -> ModelIdentity {
ModelIdentity { name: "model".into(), digest: digest.into(), dimensions: 4 }
}
/// A door a test holds shut: `embed` announces it has arrived and waits.
#[derive(Clone, Default)]
pub(crate) struct Gate(Arc<(Mutex<(u32, bool)>, Condvar)>);
impl Gate {
pub(crate) fn shut() -> Self {
Self::default()
}
/// Blocks until `count` calls to `embed` have arrived at the gate.
pub(crate) fn wait_arrived(&self, count: u32) {
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(10);
let mut state = self.0.0.lock().unwrap();
while state.0 < count {
let left = deadline.saturating_duration_since(std::time::Instant::now());
assert!(!left.is_zero(), "{count} calls never arrived at the gate; {} did", state.0);
state = self.0.1.wait_timeout(state, left).unwrap().0;
}
}
/// Shuts the door again and forgets who arrived, so a test that let start-up
/// work through can hold the next call.
pub(crate) fn shut_again(&self) {
let mut state = self.0.0.lock().unwrap();
*state = (0, false);
}
pub(crate) fn open(&self) {
self.0.0.lock().unwrap().1 = true;
self.0.1.notify_all();
}
fn pass(&self) {
let mut state = self.0.0.lock().unwrap();
state.0 += 1;
self.0.1.notify_all();
while !state.1 {
state = self.0.1.wait(state).unwrap();
}
}
}
pub(crate) struct EmbedderState {
pub(crate) identity: ModelIdentity,
pub(crate) down: bool,
/// The texts of every completed call, in order.
pub(crate) embedded: Vec<Vec<String>>,
/// The bound each completed call was given, in order.
pub(crate) bounds: Vec<Duration>,
/// The length of the vectors it answers with, where that is not the
/// identity's.
pub(crate) vector_length: Option<usize>,
/// The bound each identity read was given, in order.
pub(crate) identity_bounds: Vec<Duration>,
/// Whether an identity read is never answered: it fails when its bound
/// has passed, as an adapter that honours its bound does.
pub(crate) hang_identity: bool,
/// How long an identity read takes to answer, when it answers.
pub(crate) identity_delay: Duration,
}
#[derive(Clone)]
pub(crate) struct FakeEmbedder {
pub(crate) state: Arc<Mutex<EmbedderState>>,
gate: Option<Gate>,
}
impl FakeEmbedder {
pub(crate) fn new(digest: &str) -> Self {
let state = EmbedderState { identity: identity(digest), down: false, embedded: Vec::new(), bounds: Vec::new(), vector_length: None, identity_bounds: Vec::new(), hang_identity: false, identity_delay: Duration::ZERO };
Self { state: Arc::new(Mutex::new(state)), gate: None }
}
pub(crate) fn behind(mut self, gate: &Gate) -> Self {
self.gate = Some(gate.clone());
self
}
pub(crate) fn texts(&self) -> Vec<String> {
self.state.lock().unwrap().embedded.iter().flatten().cloned().collect()
}
}
impl Embedder for FakeEmbedder {
fn model_identity(&mut self, within: Duration) -> Result<ModelIdentity> {
let mut state = self.state.lock().unwrap();
state.identity_bounds.push(within);
if state.hang_identity {
drop(state);
std::thread::sleep(within);
bail!("the embedder did not answer within {within:?}");
}
if state.down {
bail!("the embedder is down");
}
let (identity, delay) = (state.identity.clone(), state.identity_delay);
drop(state);
std::thread::sleep(delay);
Ok(identity)
}
fn embed(&mut self, texts: &[String], within: Duration) -> Result<Vec<Vec<f32>>> {
if let Some(gate) = &self.gate {
gate.pass();
}
let mut state = self.state.lock().unwrap();
if state.down {
bail!("the embedder is down");
}
state.embedded.push(texts.to_vec());
state.bounds.push(within);
let dimensions = state.vector_length.unwrap_or(state.identity.dimensions as usize);
Ok(texts.iter().map(|text| (0..dimensions).map(|at| (text.len() + at) as f32).collect()).collect())
}
}
pub(crate) struct Stored {
pub(crate) label: Described,
pub(crate) points: BTreeMap<String, Point>,
}
#[derive(Default)]
pub(crate) struct IndexState {
pub(crate) collections: BTreeMap<String, Stored>,
pub(crate) down: bool,
pub(crate) recreated: Vec<(String, CollectionMeta)>,
/// The point ids of every upsert call, in order.
pub(crate) upserts: Vec<Vec<String>>,
/// What every search answers, whatever the vector: the test decides the
/// candidates, as a real index would decide them from vectors.
pub(crate) hits: Vec<Hit>,
/// The collection, vector length and limit of every search, in order.
pub(crate) searches: Vec<(String, usize, u32)>,
/// How many times every point id was listed: only a reconciliation does.
pub(crate) listed: u32,
/// Whether a search fails at the vector store, the rest of it working.
pub(crate) fail_search: bool,
/// The bound each describe and each search was given, in order, by call.
pub(crate) bounds: Vec<(&'static str, Duration)>,
/// How many points the collection held when each search reached it.
pub(crate) held_when_searched: Vec<usize>,
}
#[derive(Clone, Default)]
pub(crate) struct FakeIndex {
pub(crate) state: Arc<Mutex<IndexState>>,
}
impl FakeIndex {
pub(crate) fn holding(&self, collection: &str) -> BTreeSet<String> {
let state = self.state.lock().unwrap();
state.collections.get(collection).map(|stored| stored.points.keys().cloned().collect()).unwrap_or_default()
}
/// Plants a collection this organ did not make, or made for another mind.
pub(crate) fn plant(&self, collection: &str, label: Described) {
self.state.lock().unwrap().collections.insert(collection.into(), Stored { label, points: BTreeMap::new() });
}
}
impl VectorIndex for FakeIndex {
fn describe(&mut self, collection: &str, within: Duration) -> Result<Described> {
let mut state = self.state.lock().unwrap();
state.bounds.push(("describe", within));
if state.down {
bail!("the vector store is down");
}
Ok(state.collections.get(collection).map_or(Described::Absent, |stored| stored.label.clone()))
}
fn recreate(&mut self, collection: &str, meta: &CollectionMeta) -> Result<()> {
let mut state = self.state.lock().unwrap();
if state.down {
bail!("the vector store is down");
}
state.recreated.push((collection.into(), meta.clone()));
state
.collections
.insert(collection.into(), Stored { label: Described::Labelled(meta.clone()), points: BTreeMap::new() });
Ok(())
}
fn ids(&mut self, collection: &str) -> Result<BTreeSet<String>> {
let mut state = self.state.lock().unwrap();
if state.down {
bail!("the vector store is down");
}
state.listed += 1;
Ok(state.collections.get(collection).map(|stored| stored.points.keys().cloned().collect()).unwrap_or_default())
}
fn upsert(&mut self, collection: &str, points: &[Point]) -> Result<()> {
let mut state = self.state.lock().unwrap();
if state.down {
bail!("the vector store is down");
}
state.upserts.push(points.iter().map(|point| point.id.clone()).collect());
let Some(stored) = state.collections.get_mut(collection) else { bail!("no collection {collection}") };
for point in points {
stored.points.insert(point.id.clone(), point.clone());
}
Ok(())
}
fn search(&mut self, collection: &str, vector: &[f32], limit: u32, within: Duration) -> Result<Vec<Hit>> {
let mut state = self.state.lock().unwrap();
state.bounds.push(("search", within));
if state.down {
bail!("the vector store is down");
}
state.searches.push((collection.into(), vector.len(), limit));
ensure!(!state.fail_search, "the vector store failed the search");
ensure!(state.collections.contains_key(collection), "no collection {collection}");
let held = state.collections[collection].points.len();
state.held_when_searched.push(held);
Ok(state.hits.clone())
}
}
/// A working embedder and vector store, for tests whose subject is elsewhere.
pub(crate) fn pair() -> (FakeEmbedder, FakeIndex) {
(FakeEmbedder::new("d1"), FakeIndex::default())
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,75 @@
//! The embedder port over Ollama's HTTP API. This file and `qdrant.rs` are the
//! xenos boundary: JSON exists here and nowhere else in the organ.
use std::time::Duration;
use anyhow::{Context, Result, bail, ensure};
use serde_json::{Value, json};
use super::{Embedder, Http, Method, ModelIdentity};
/// The instruction Qwen3 embedding models take in front of a query (never in
/// front of a document). Cut 11b's semantic read prepends it as
/// `Instruct: <this>\nQuery: <text>`.
pub const QUERY_INSTRUCTION: &str =
"Given a question about a project's pipeline state, retrieve the documents that answer it";
/// One model on one Ollama. Every call carries its own bound.
pub struct OllamaEmbedder {
base_url: String,
model: String,
}
impl OllamaEmbedder {
pub fn new(base_url: &str, model: &str) -> Self {
Self { base_url: base_url.to_owned(), model: model.to_owned() }
}
fn ask(&self, http: &Http, method: Method, path: &str, body: Option<&Value>) -> Result<Value> {
let text = body.map(Value::to_string);
let (status, answer) = http.request(method, path, text.as_deref())?;
ensure!(status == 200, "Ollama answered {status} to {path}: {}", answer.chars().take(300).collect::<String>());
serde_json::from_str(&answer).with_context(|| format!("Ollama's answer to {path} is not JSON"))
}
}
impl Embedder for OllamaEmbedder {
/// The configured name, the digest Ollama lists for it, and the model's
/// embedding length. The address is not part of the identity.
fn model_identity(&mut self, within: Duration) -> Result<ModelIdentity> {
let http = Http::new(&self.base_url, within);
let tags = self.ask(&http, Method::Get, "/api/tags", None)?;
let listed = tags["models"].as_array().context("Ollama's tag list has no models")?;
let digest = listed
.iter()
.find(|entry| entry["name"] == self.model.as_str() || entry["model"] == self.model.as_str())
.and_then(|entry| entry["digest"].as_str())
.with_context(|| format!("Ollama does not list the model {}", self.model))?
.to_owned();
let shown = self.ask(&http, Method::Post, "/api/show", Some(&json!({ "model": self.model })))?;
let info = shown["model_info"].as_object().context("Ollama's model description has no model_info")?;
let Some(length) = info.iter().find(|(key, _)| key.ends_with(".embedding_length")).and_then(|(_, value)| value.as_u64())
else {
bail!("Ollama does not say how long {}'s embeddings are", self.model);
};
let dimensions = u32::try_from(length).context("an embedding length beyond u32")?;
Ok(ModelIdentity { name: self.model.clone(), digest, dimensions })
}
fn embed(&mut self, texts: &[String], within: Duration) -> Result<Vec<Vec<f32>>> {
let bounded = Http::new(&self.base_url, within);
let answer = self.ask(&bounded, Method::Post, "/api/embed", Some(&json!({ "model": self.model, "input": texts })))?;
let embeddings = answer["embeddings"].as_array().context("Ollama's answer has no embeddings")?;
embeddings
.iter()
.map(|vector| {
vector
.as_array()
.context("an embedding is not a list")?
.iter()
.map(|number| number.as_f64().map(|value| value as f32).context("an embedding holds a non-number"))
.collect()
})
.collect()
}
}

View file

@ -0,0 +1,158 @@
//! The vector-store port over Qdrant's HTTP API. With `ollama.rs`, the xenos
//! boundary: JSON exists here and nowhere else in the organ.
use std::collections::BTreeSet;
use std::time::Duration;
use anyhow::{Context, Result, ensure};
use serde_json::{Value, json};
use super::{CollectionMeta, INDEX_CALL_TIMEOUT, Described, Hit, Http, Method, Point, VectorIndex};
/// One Qdrant. Every path names a collection this organ was told to own; it
/// never lists or touches another.
pub struct QdrantIndex {
http: Http,
}
/// The ids fetched per scroll page.
const SCROLL_PAGE: u64 = 1000;
impl QdrantIndex {
pub fn new(base_url: &str) -> Self {
Self { http: Http::new(base_url, INDEX_CALL_TIMEOUT) }
}
fn ask(&self, method: Method, path: &str, body: Option<&Value>, ok: &[u16]) -> Result<(u16, Value)> {
self.ask_with(&self.http, method, path, body, ok)
}
fn ask_with(&self, http: &Http, method: Method, path: &str, body: Option<&Value>, ok: &[u16]) -> Result<(u16, Value)> {
let text = body.map(Value::to_string);
let (status, answer) = http.request(method, path, text.as_deref())?;
ensure!(ok.contains(&status), "Qdrant answered {status} to {path}: {}", answer.chars().take(300).collect::<String>());
let parsed = if answer.is_empty() { Value::Null } else { serde_json::from_str(&answer).unwrap_or(Value::Null) };
Ok((status, parsed))
}
}
fn metadata_json(meta: &CollectionMeta) -> Value {
json!({
"managed_by": meta.managed_by,
"instance": meta.instance,
"mind": meta.mind,
"model": meta.model,
"model_digest": meta.model_digest,
"dimensions": meta.dimensions,
"index_text_version": meta.index_text_version,
})
}
/// What a collection's `config.metadata` says. A collection whose metadata
/// does not name a writer and an instance is unlabelled; one that does but is
/// missing the rest reads as an empty mind or model, which will not match: an
/// empty mind is refused as another's, an empty model is rebuilt when the
/// writer, the instance and the mind are ours.
fn described(metadata: &Value) -> Described {
let text = |key: &str| metadata[key].as_str().map(str::to_owned);
let (Some(managed_by), Some(instance)) = (text("managed_by"), text("instance")) else {
return Described::Unlabelled;
};
let number = |key: &str| metadata[key].as_u64().and_then(|value| u32::try_from(value).ok()).unwrap_or(0);
Described::Labelled(CollectionMeta {
managed_by,
instance,
mind: text("mind").unwrap_or_default(),
model: text("model").unwrap_or_default(),
model_digest: text("model_digest").unwrap_or_default(),
dimensions: number("dimensions"),
index_text_version: number("index_text_version"),
})
}
impl VectorIndex for QdrantIndex {
fn describe(&mut self, collection: &str, within: Duration) -> Result<Described> {
let http = self.http.bounded(within);
let (status, answer) = self.ask_with(&http, Method::Get, &format!("/collections/{collection}"), None, &[200, 404])?;
if status == 404 {
return Ok(Described::Absent);
}
Ok(described(&answer["result"]["config"]["metadata"]))
}
fn recreate(&mut self, collection: &str, meta: &CollectionMeta) -> Result<()> {
let path = format!("/collections/{collection}");
self.ask(Method::Delete, &format!("{path}?wait=true"), None, &[200, 404])?;
let body = json!({
"vectors": { "size": meta.dimensions, "distance": "Cosine", "on_disk": true },
"metadata": metadata_json(meta),
});
self.ask(Method::Put, &format!("{path}?wait=true"), Some(&body), &[200])?;
Ok(())
}
fn ids(&mut self, collection: &str) -> Result<BTreeSet<String>> {
let mut ids = BTreeSet::new();
let mut offset = Value::Null;
loop {
let mut body = json!({ "limit": SCROLL_PAGE, "with_payload": false, "with_vector": false });
if !offset.is_null() {
body["offset"] = offset;
}
let (_, answer) =
self.ask(Method::Post, &format!("/collections/{collection}/points/scroll"), Some(&body), &[200])?;
let points = answer["result"]["points"].as_array().context("Qdrant's scroll has no points")?;
for point in points {
ids.insert(point["id"].as_str().context("a point id is not a string")?.to_owned());
}
offset = answer["result"]["next_page_offset"].clone();
if offset.is_null() {
return Ok(ids);
}
}
}
fn upsert(&mut self, collection: &str, points: &[Point]) -> Result<()> {
let points: Vec<Value> = points
.iter()
.map(|point| {
json!({
"id": point.id,
"vector": point.vector,
"payload": {
"doc_id": point.payload.doc_id,
"kind": point.payload.kind,
"root": point.payload.root,
"ordinal": point.payload.ordinal,
"text_sha256": point.payload.text_sha256,
},
})
})
.collect();
self.ask(
Method::Put,
&format!("/collections/{collection}/points?wait=true"),
Some(&json!({ "points": points })),
&[200],
)?;
Ok(())
}
fn search(&mut self, collection: &str, vector: &[f32], limit: u32, within: Duration) -> Result<Vec<Hit>> {
let http = self.http.bounded(within);
let body = json!({ "vector": vector, "limit": limit, "with_payload": ["doc_id", "kind"], "with_vector": false });
let (_, answer) =
self.ask_with(&http, Method::Post, &format!("/collections/{collection}/points/search"), Some(&body), &[200])?;
let found = answer["result"].as_array().context("Qdrant's search has no result")?;
found
.iter()
.map(|point| {
Ok(Hit {
doc_id: point["payload"]["doc_id"].as_str().context("a hit carries no doc_id")?.to_owned(),
kind: point["payload"]["kind"].as_str().context("a hit carries no kind")?.to_owned(),
score: point["score"].as_f64().context("a hit carries no score")? as f32,
})
})
.collect()
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,25 @@
//! `huginn-daemon`: one process that serves one mind over CultNet.
//!
//! The crate owns four things and no rule. `startup` owns "may this process
//! serve this mind", by delegating to `Mind::open` and refusing before
//! anything listens. `index` owns when the mind's documents are embedded and
//! written to the vector store, off the serving path. `Daemon::handle` owns "which `Mind` method answers this
//! request": it dispatches, it asks `Mind::require_instance` for every read
//! that names an instance, and it compares nothing itself. `serve` owns "which
//! session gets which reply" and whether an answer is delivered whole, deferred or refused; `bodies`
//! owns how long and how much of a deferred answer is held.
//!
//! Every admission rule, the instance check, the receipt, the derived status
//! and the wire vocabulary are `huginn-mind`'s. The leaf and the store type
//! are reached through `huginn_mind`, so one crate pins one revision of each.
pub mod bodies;
pub mod daemon;
pub mod index;
pub mod serve;
#[cfg(test)]
mod semantic_tests;
pub use daemon::{Daemon, Handled, Hits, IndexSink, Search, SearchTicket, runtime_id};
pub use serve::{Options, ServeOptions, parse_options, run, startup};

View file

@ -0,0 +1,37 @@
//! The process: open one instance's mind, bind one socket, serve until a stop
//! signal. A mind that will not open ends the process with the refusal on
//! stderr and no socket ever bound.
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use anyhow::{Context, Result};
use huginn_daemon::index::Backoff;
use huginn_daemon::index::ollama::OllamaEmbedder;
use huginn_daemon::index::qdrant::QdrantIndex;
use huginn_daemon::serve::{ServeOptions, parse_options, run, startup};
fn main() -> Result<()> {
let options = parse_options(std::env::args().skip(1))?;
let embedder = OllamaEmbedder::new(&options.ollama_url, &options.embedding_model);
let index = QdrantIndex::new(&options.qdrant_url);
let (mut daemon, mut hub, registry) = match startup(&options, embedder, index, Backoff::default()) {
Ok(started) => started,
Err(error) => {
eprintln!("huginn-daemon refuses to serve {}: {error:#}", options.instance.0);
std::process::exit(1);
}
};
// Idunn runs this process as PID 1 of its own PID namespace, and a
// namespace init ignores every signal it has not caught. Without this,
// a stop is a ninety-second wait for SIGKILL.
let stopping = Arc::new(AtomicBool::new(false));
for signal in [signal_hook::consts::SIGTERM, signal_hook::consts::SIGINT] {
signal_hook::flag::register(signal, Arc::clone(&stopping))
.context("registering Huginn's stop signal handler")?;
}
eprintln!("huginn-daemon serving {} on {}", options.instance.0, hub.local_addr()?);
run(&mut daemon, &mut hub, &registry, &stopping, &ServeOptions::default())
}

View file

@ -0,0 +1,491 @@
//! The semantic read through the real serve loop: a UDP hub, real sessions, the
//! worker thread, and fake embedder and vector-store ports whose behaviour the
//! test decides. Nothing here calls `Daemon::handle` directly: the rules under
//! test are about what the loop does while an answer is outstanding.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread::JoinHandle;
use std::time::Duration;
use anyhow::Result;
use cultnet_rs::{CULTNET_OPERATION_CONNECTION_ID, CultMesh, CultMeshRudpSocketOptions, FieldPredicate, Selection};
use huginn_mind::epiphany_pipeline::{
Date, PipelineDocument, PipelineKind, PipelineQuestion, PipelineResolution, PipelineRef, PipelineRuling,
QuestionOption, ResolutionOutcome, RulingAuthority, Short, Title,
};
use huginn_mind::wire::{HuginnMindRequest, HuginnMindResponse, IndexStatus};
use huginn_mind::{Mind, MindRefusal, OwnedRedbMessagePackBackingStore, PipelineAdmissionOutcome, PipelinePageItems, SemanticQuery};
use tempfile::TempDir;
use crate::daemon::Daemon;
use crate::daemon::tests::{CAMPAIGN, INSTANCE, batch, campaign_seed, now, slug};
use huginn_mind::envelope::{decode_response, encode_request};
use crate::index::fakes::{FakeEmbedder, FakeIndex, Gate};
use crate::index::{Backoff, Described, Hit, WorkerSink, collection_name};
use crate::serve::{ServeOptions, bind, run, schema_registry};
type Client = cultnet_rs::CultNetRudpSocketTransportConnection;
fn question(label: &str) -> PipelineDocument {
PipelineDocument::Question(PipelineQuestion {
campaign: slug(CAMPAIGN),
label: label.into(),
title: Title(format!("Question {label}")),
question: format!("Question {label}?").as_str().into(),
options: vec![
QuestionOption { label: "A".into(), text: "the organ".into() },
QuestionOption { label: "B".into(), text: "the client".into() },
],
recommended: "A".into(),
depends: vec![],
raised_in: None,
asked_on: Date("2026-09-16".into()),
})
}
/// A ruling, answering a question when it names one.
fn ruling(label: &str, answers: Option<&str>) -> PipelineDocument {
PipelineDocument::Ruling(PipelineRuling {
campaign: slug(CAMPAIGN),
label: label.into(),
title: Title(format!("Ruling {label}")),
answers: answers.map(|question| Short(id("question", question))),
choice: answers.map(|_| "A".into()),
ruling: format!("Ruling {label}.").as_str().into(),
operator_quote: None,
ruled_on: Date("2026-09-16".into()),
precedents: vec![],
authority: RulingAuthority::Operator,
})
}
fn resolution(subject: PipelineKind, key: &str, outcome: ResolutionOutcome) -> PipelineDocument {
PipelineDocument::Resolution(PipelineResolution {
subject: PipelineRef { kind: subject, id: Short(key.into()) },
sequence: 1,
outcome,
rationale: "Resolved.".into(),
resolved_on: Date("2026-09-16".into()),
})
}
fn id(kind: &str, local: &str) -> String {
format!("{CAMPAIGN}:{kind}:{local}")
}
fn hit(kind: &str, key: String, score: f32) -> Hit {
Hit { doc_id: key, kind: kind.into(), score }
}
/// Every way a document stops standing, and two that do: Q1 is answered (so
/// resolved), Q2 was answered and the answer withdrawn (its resolution is the
/// resolved one and the question stands again), R3 is superseded by R4.
fn shelf() -> Vec<Vec<PipelineDocument>> {
vec![
campaign_seed(),
vec![question("Q1"), question("Q2")],
vec![ruling("R1", Some("Q1"))],
vec![ruling("R2", Some("Q2"))],
vec![resolution(PipelineKind::Resolution, &id("resolution", "question.Q2.n1"), ResolutionOutcome::Withdrawn { reason: "moot".into() })],
vec![ruling("R3", None), ruling("R4", None)],
vec![resolution(
PipelineKind::Ruling,
&id("ruling", "R3"),
ResolutionOutcome::Superseded { by: vec![PipelineRef { kind: PipelineKind::Ruling, id: Short(id("ruling", "R4")) }] },
)],
]
}
fn standing() -> Selection {
Selection {
fields: Some(vec![FieldPredicate {
index: "in_force".into(),
op: "any_of".into(),
values: Some(vec!["true".into()]),
number: None,
}]),
..Selection::default()
}
}
fn semantic(selection: Selection, top_k: u32) -> HuginnMindRequest {
semantic_of("who owns the state", selection, top_k)
}
fn semantic_of(text: &str, selection: Selection, top_k: u32) -> HuginnMindRequest {
HuginnMindRequest::Query {
instance: slug(INSTANCE),
selection,
semantic: Some(SemanticQuery { text: text.into(), top_k }),
}
}
/// How many documents a plain query finds.
fn documents(client: &mut Client, message_id: &str) -> usize {
ids(&ask(client, message_id, &plain())).len()
}
fn plain() -> HuginnMindRequest {
HuginnMindRequest::Query { instance: slug(INSTANCE), selection: Selection::default(), semantic: None }
}
/// The ids on a page, in its order.
fn ids(response: &HuginnMindResponse) -> Vec<String> {
let HuginnMindResponse::Query(page) = response else { panic!("expected a page, got {response:?}") };
match &page.items {
PipelinePageItems::Headers(headers) => headers.iter().map(|header| header.id.id.0.clone()).collect(),
PipelinePageItems::Documents(views) => views.iter().map(|view| view.id.id.0.clone()).collect(),
}
}
fn quick() -> Backoff {
Backoff { initial: Duration::from_millis(1), max: Duration::from_millis(2), recheck: Duration::from_secs(3600), search_deadline: Duration::from_secs(30) }
}
/// A daemon serving a mind over a real socket, with the ports the test gave it.
struct Harness {
endpoint: String,
stopping: Arc<AtomicBool>,
serving: Option<JoinHandle<Result<()>>>,
embedder: FakeEmbedder,
index: FakeIndex,
_root: TempDir,
}
impl Harness {
fn start(seed: Vec<Vec<PipelineDocument>>, embedder: FakeEmbedder, index: FakeIndex, search_timeout: Duration) -> Self {
let root = tempfile::tempdir().unwrap();
let mut mind = Mind::open(root.path(), &slug(INSTANCE)).unwrap();
for documents in seed {
let outcome = mind.admit(batch(INSTANCE, documents), now());
assert!(matches!(outcome, PipelineAdmissionOutcome::Committed { .. }), "{outcome:?}");
}
let sink = WorkerSink::spawn(
embedder.clone(),
index.clone(),
&slug(INSTANCE),
mind.genesis_receipt_id().unwrap(),
mind.index_entries(None).unwrap(),
Backoff { search_deadline: search_timeout, ..quick() },
);
let mut daemon: Daemon<OwnedRedbMessagePackBackingStore, WorkerSink> = Daemon::new(mind, sink);
let mut hub = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let endpoint = format!("rudp://{}", hub.local_addr().unwrap());
let registry = schema_registry().unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let options = ServeOptions::default();
let serving = std::thread::spawn(move || run(&mut daemon, &mut hub, &registry, &loop_stopping, &options));
Self { endpoint, stopping, serving: Some(serving), embedder, index, _root: root }
}
fn client(&self) -> Client {
let mut client = CultMesh::create_rudp_client_for_endpoint(
"semantic-test".to_string(),
CULTNET_OPERATION_CONNECTION_ID,
&self.endpoint,
CultMeshRudpSocketOptions::default(),
)
.unwrap();
client.connect(Vec::new()).unwrap();
for _ in 0..1000 {
let _ = client.receive_once();
if client.connected() {
return client;
}
std::thread::sleep(Duration::from_millis(2));
}
panic!("the client never connected");
}
}
impl Drop for Harness {
fn drop(&mut self) {
self.stopping.store(true, Ordering::Relaxed);
// A loop that is stuck waiting on the index is a failure of the test,
// not a reason to hang the run: give it a moment and leave it behind.
if let Some(serving) = self.serving.take() {
for _ in 0..1500 {
if serving.is_finished() {
let _ = serving.join();
return;
}
std::thread::sleep(Duration::from_millis(2));
}
}
}
}
fn send(client: &mut Client, message_id: &str, request: &HuginnMindRequest) {
client.send_schema_message(&encode_request(message_id, request, None).unwrap()).unwrap();
}
/// The reply to `message_id` on this client's session, or `None` if none came
/// in `patience` polls of two milliseconds.
fn receive(client: &mut Client, message_id: &str, patience: u32) -> Option<HuginnMindResponse> {
for _ in 0..patience {
client.poll_resends().unwrap();
if let Some(reply) = client.receive_schema_message_once().unwrap() {
let (correlated, response) = decode_response(&reply).unwrap();
assert_eq!(correlated, message_id);
return Some(response.expect("an answer, not an envelope failure"));
}
std::thread::sleep(Duration::from_millis(2));
}
None
}
fn ask(client: &mut Client, message_id: &str, request: &HuginnMindRequest) -> HuginnMindResponse {
send(client, message_id, request);
receive(client, message_id, 3000).unwrap_or_else(|| panic!("no reply to {message_id}"))
}
fn index_status(client: &mut Client, message_id: &str) -> IndexStatus {
match ask(client, message_id, &HuginnMindRequest::Whoami) {
HuginnMindResponse::Whoami(status) => status.index,
other => panic!("expected a status, got {other:?}"),
}
}
fn until_index(client: &mut Client, what: &str, predicate: impl Fn(&IndexStatus) -> bool) {
for at in 0..2000 {
if predicate(&index_status(client, &format!("status-{at}"))) {
return;
}
std::thread::sleep(Duration::from_millis(5));
}
panic!("the index never reported {what}");
}
fn healthy() -> Harness {
let (embedder, index) = (FakeEmbedder::new("d1"), FakeIndex::default());
Harness::start(shelf(), embedder, index, Duration::from_secs(30))
}
/// A semantic answer is the mind's: the index names a resolved question, a
/// withdrawn resolution, a superseded ruling and a document the mind does not
/// hold, all scoring above the two that stand, and only the two that stand come
/// back, in score order. The selection's other predicates still cut the
/// candidates, and the embedder saw the query behind its instruction while the
/// index was asked for an oversample.
#[test]
fn a_semantic_query_returns_only_documents_the_mind_holds_and_the_selection_admits() {
let harness = healthy();
let mut client = harness.client();
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
harness.index.state.lock().unwrap().hits = vec![
hit("question", id("question", "Q1"), 0.95),
hit("resolution", id("resolution", "question.Q2.n1"), 0.9),
hit("ruling", id("ruling", "R3"), 0.85),
hit("question", id("question", "Q99"), 0.8),
hit("ruling", id("ruling", "R4"), 0.5),
hit("question", id("question", "Q2"), 0.4),
];
let standing_only = ask(&mut client, "m-1", &semantic(standing(), 5));
assert_eq!(ids(&standing_only), vec![id("ruling", "R4"), id("question", "Q2")]);
let HuginnMindResponse::Query(page) = &standing_only else { unreachable!() };
assert_eq!((page.matched, page.next.as_ref()), (2, None));
let rulings = Selection { schemas: Some(vec![PipelineKind::Ruling.type_id().into()]), ..standing() };
assert_eq!(ids(&ask(&mut client, "m-2", &semantic(rulings, 5))), vec![id("ruling", "R4")], "the selection's schemas still apply");
let everything = ids(&ask(&mut client, "m-3", &semantic(Selection::default(), 5)));
assert!(everything.contains(&id("question", "Q1")) && !everything.contains(&id("question", "Q99")),
"a hit the selection does not exclude is shown, and one the mind does not hold never is: {everything:?}");
let embedded = harness.embedder.texts();
assert!(embedded.iter().any(|text| text.starts_with("Instruct: ") && text.ends_with("\nQuery: who owns the state")), "{embedded:?}");
let searches = harness.index.state.lock().unwrap().searches.clone();
assert!(searches.iter().all(|(name, _, limit)| *name == collection_name(&slug(INSTANCE)) && *limit == 20), "{searches:?}");
}
/// A cursor cannot resume a ranked answer and a ranked answer has no order to
/// reverse: each is refused by name, and nothing is embedded or searched for it.
#[test]
fn a_cursor_or_a_reversal_with_a_semantic_query_is_refused_and_asks_the_index_nothing() {
let harness = healthy();
let mut client = harness.client();
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
let embedded = harness.embedder.texts().len();
let with_cursor = Selection { cursor: Some("anything".into()), ..standing() };
let refusal = ask(&mut client, "m-1", &semantic(with_cursor, 5));
assert!(
matches!(&refusal, HuginnMindResponse::Refused(MindRefusal::SelectionInvalid { field, .. }) if field == "cursor"),
"{refusal:?}"
);
assert_eq!(harness.embedder.texts().len(), embedded);
assert!(harness.index.state.lock().unwrap().searches.is_empty());
let reversed = Selection { descending: true, ..standing() };
let refusal = ask(&mut client, "m-2", &semantic(reversed, 5));
assert!(
matches!(&refusal, HuginnMindResponse::Refused(MindRefusal::SelectionInvalid { field, .. }) if field == "descending"),
"{refusal:?}"
);
assert_eq!(harness.embedder.texts().len(), embedded);
assert!(harness.index.state.lock().unwrap().searches.is_empty());
}
/// The loop never waits on the network. The embedder is shut, the worker is
/// inside it with one client's semantic query, and a second client's `whoami`
/// and plain query are answered at once; the first is answered when the gate
/// opens. The gate counts arrivals, so this observes where the work is and not
/// how long it took.
#[test]
fn another_session_is_answered_while_a_semantic_search_is_outstanding() {
let gate = Gate::shut();
gate.open();
let embedder = FakeEmbedder::new("d1").behind(&gate);
let harness = Harness::start(shelf(), embedder, FakeIndex::default(), Duration::from_secs(30));
let mut asker = harness.client();
let mut second = harness.client();
let mut other = harness.client();
until_index(&mut other, "Current", |status| *status == IndexStatus::Current);
let held = documents(&mut other, "m-count");
harness.index.state.lock().unwrap().hits = vec![hit("ruling", id("ruling", "R4"), 0.5)];
gate.shut_again();
send(&mut asker, "m-ask", &semantic(standing(), 5));
gate.wait_arrived(1);
assert!(receive(&mut asker, "m-ask", 20).is_none(), "the question is not answered while its embedding is outstanding");
// A second question waits behind the first, and each is answered to its own asker.
send(&mut second, "m-ask-2", &semantic(Selection { schemas: Some(vec![PipelineKind::Question.type_id().into()]), ..Selection::default() }, 5));
assert!(matches!(ask(&mut other, "m-who", &HuginnMindRequest::Whoami), HuginnMindResponse::Whoami(_)));
assert_eq!(documents(&mut other, "m-plain"), held, "a plain query is answered too");
gate.open();
let answered = receive(&mut asker, "m-ask", 3000).expect("the search is answered once the embedder returns");
assert_eq!(ids(&answered), vec![id("ruling", "R4")]);
let second_answer = receive(&mut second, "m-ask-2", 3000).expect("the second search is answered too, on its own session");
assert_eq!(ids(&second_answer), Vec::<String>::new(), "the index offered a ruling, which its selection of questions does not admit");
}
/// A search the index never finishes is answered at the deadline the client
/// was promised, typed and naming the wait, never as an empty page.
#[test]
fn a_search_the_index_never_finishes_is_answered_unavailable_at_the_deadline() {
let gate = Gate::shut();
gate.open();
let embedder = FakeEmbedder::new("d1").behind(&gate);
let harness = Harness::start(shelf(), embedder, FakeIndex::default(), Duration::from_millis(150));
let mut client = harness.client();
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
gate.shut_again();
let answered = ask(&mut client, "m-late", &semantic(standing(), 5));
let HuginnMindResponse::Refused(MindRefusal::Unavailable { detail }) = &answered else { panic!("{answered:?}") };
assert!(detail.contains("did not finish"), "{detail}");
gate.open();
assert!(matches!(ask(&mut client, "m-after", &HuginnMindRequest::Whoami), HuginnMindResponse::Whoami(_)), "the late answer is dropped, not misdelivered");
}
/// An index that cannot answer is a typed `Unavailable` and never an empty
/// page, in each way it cannot: the embedder is down when the query arrives;
/// the worker is failing; the collection belongs to someone else. Plain
/// queries are unaffected throughout.
#[test]
fn an_index_that_cannot_answer_is_unavailable_and_plain_queries_still_work() {
let unavailable = |response: HuginnMindResponse| match response {
HuginnMindResponse::Refused(MindRefusal::Unavailable { detail }) => detail,
other => panic!("expected Unavailable, got {other:?}"),
};
// The embedder goes down after the index was Current: the query itself fails.
let harness = healthy();
let mut client = harness.client();
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
let held = documents(&mut client, "m-count");
harness.embedder.state.lock().unwrap().down = true;
let detail = unavailable(ask(&mut client, "m-down", &semantic(standing(), 5)));
assert!(detail.contains("the embedder is down"), "{detail}");
assert_eq!(documents(&mut client, "m-plain"), held);
// A write while it is down makes the worker Failing, and the next query is
// refused without being asked of it.
let admit = HuginnMindRequest::Admit(batch(INSTANCE, vec![question("Q7")]));
assert!(matches!(ask(&mut client, "m-admit", &admit), HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { .. })));
until_index(&mut client, "Failing", |status| matches!(status, IndexStatus::Failing { .. }));
let searches = harness.index.state.lock().unwrap().searches.len();
let detail = unavailable(ask(&mut client, "m-failing", &semantic(standing(), 5)));
assert!(detail.contains("failing") && detail.contains("the embedder is down"), "{detail}");
assert_eq!(harness.index.state.lock().unwrap().searches.len(), searches);
assert_eq!(documents(&mut client, "m-plain-2"), held + 1, "the admitted question is in the mind whatever the index says");
// A collection that is another writer's is Refused, and so are queries.
let (embedder, index) = (FakeEmbedder::new("d1"), FakeIndex::default());
index.plant(&collection_name(&slug(INSTANCE)), Described::Unlabelled);
let refused = Harness::start(shelf(), embedder, index, Duration::from_secs(30));
let mut client = refused.client();
until_index(&mut client, "Refused", |status| matches!(status, IndexStatus::Refused { .. }));
let detail = unavailable(ask(&mut client, "m-refused", &semantic(standing(), 5)));
assert!(detail.contains("refused") && detail.contains("no Huginn metadata"), "{detail}");
assert_eq!(documents(&mut client, "m-plain-3"), held);
}
/// A search the client has stopped waiting for is not started. The embedder is
/// held with the first search inside it; a second search queues behind it and
/// times out, and when the embedder is released the worker embeds the first,
/// skips the second and embeds a third: the index is never asked for an answer
/// nobody is waiting for.
#[test]
fn a_search_that_timed_out_while_queued_is_never_embedded() {
let gate = Gate::shut();
gate.open();
let embedder = FakeEmbedder::new("d1").behind(&gate);
let harness = Harness::start(shelf(), embedder, FakeIndex::default(), Duration::from_millis(300));
let mut first = harness.client();
let mut second = harness.client();
let mut third = harness.client();
until_index(&mut third, "Current", |status| *status == IndexStatus::Current);
let queries = |harness: &Harness| -> Vec<String> {
harness.embedder.texts().into_iter().filter(|text| text.starts_with("Instruct: ")).collect()
};
gate.shut_again();
send(&mut first, "m-first", &semantic_of("first", standing(), 5));
gate.wait_arrived(1);
send(&mut second, "m-second", &semantic_of("second", standing(), 5));
let timed_out = receive(&mut second, "m-second", 3000).expect("the second search is answered at its deadline");
assert!(matches!(&timed_out, HuginnMindResponse::Refused(MindRefusal::Unavailable { detail }) if detail.contains("did not finish")), "{timed_out:?}");
gate.open();
send(&mut third, "m-third", &semantic_of("third", standing(), 5));
let answered = receive(&mut third, "m-third", 3000).expect("the third search is answered");
assert!(matches!(answered, HuginnMindResponse::Query(_)), "{answered:?}");
let embedded = queries(&harness);
assert_eq!(embedded.len(), 2, "the first and the third were embedded; the second was taken back before it started: {embedded:?}");
assert!(embedded[0].ends_with("Query: first") && embedded[1].ends_with("Query: third"), "{embedded:?}");
}
/// A search asked while the index lags the mind is not refused: it waits behind
/// the writes ahead of it, through the real serve loop, and reads them. The
/// worker is inside its first batch with the rest of the mind still to write;
/// the reply does not come until the gate opens, and it comes with every
/// document already in the collection.
#[test]
fn a_search_asked_while_the_index_lags_waits_for_the_writes_ahead_of_it_and_reads_them() {
let gate = Gate::shut();
let embedder = FakeEmbedder::new("d1").behind(&gate);
let harness = Harness::start(shelf(), embedder, FakeIndex::default(), Duration::from_secs(30));
gate.wait_arrived(1);
let mut client = harness.client();
let IndexStatus::Behind { pending } = index_status(&mut client, "m-status") else { panic!("the worker is inside its first batch") };
assert!(pending > 0);
harness.index.state.lock().unwrap().hits = vec![hit("ruling", id("ruling", "R4"), 0.5)];
send(&mut client, "m-behind", &semantic(standing(), 5));
assert!(receive(&mut client, "m-behind", 100).is_none(), "the search waits: it is neither refused nor answered from a half-filled collection");
gate.open();
let answered = receive(&mut client, "m-behind", 3000).expect("the search is answered once the index has caught up");
assert_eq!(ids(&answered), vec![id("ruling", "R4")]);
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
let state = harness.index.state.lock().unwrap();
let written = state.collections[&collection_name(&slug(INSTANCE))].points.len();
assert_eq!(state.held_when_searched, [written], "the search saw every document");
assert!(written >= pending as usize);
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,123 @@
//! A live check of the two adapters against a real Qdrant and a real Ollama.
//! Ignored by default: it needs both services, and it writes to Qdrant, so it
//! touches one scratch collection it creates (`huginn-smoke-<random>`, and a
//! dotted sibling to learn whether Qdrant accepts a `.` in a name) and deletes
//! both at the end. It never names, lists or reads any other collection.
//!
//! HUGINN_SMOKE_QDRANT=http://127.0.0.1:6333 \
//! HUGINN_SMOKE_OLLAMA=http://10.77.0.4:11434 \
//! HUGINN_SMOKE_MODEL=qwen3-embedding:0.6b \
//! cargo test -p huginn-daemon --test live_index_smoke -- --ignored --nocapture
use std::collections::BTreeSet;
use std::time::Duration;
use huginn_daemon::index::ollama::{OllamaEmbedder, QUERY_INSTRUCTION};
use huginn_daemon::index::qdrant::QdrantIndex;
use huginn_daemon::index::{
CollectionMeta, Described, Embedder, MANAGED_BY, Point, PointPayload, VectorIndex, point_id,
};
use huginn_mind::INDEX_TEXT_VERSION;
fn env(name: &str) -> String {
std::env::var(name).unwrap_or_else(|_| panic!("{name} is required"))
}
fn raw(method: &str, url: &str, body: Option<&str>) -> (u16, String) {
let agent: ureq::Agent = ureq::Agent::config_builder().http_status_as_error(false).build().into();
let mut response = match (method, body) {
("DELETE", _) => agent.delete(url).call(),
("POST", Some(body)) => agent.post(url).header("content-type", "application/json").send(body),
_ => agent.get(url).call(),
}
.expect("the request was sent");
let status = response.status().as_u16();
(status, response.body_mut().read_to_string().unwrap())
}
#[test]
#[ignore = "needs a live Qdrant and Ollama, and writes a scratch collection"]
fn the_adapters_embed_write_query_and_clean_up_a_scratch_collection() {
let (qdrant, ollama, model) = (env("HUGINN_SMOKE_QDRANT"), env("HUGINN_SMOKE_OLLAMA"), env("HUGINN_SMOKE_MODEL"));
let stamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos();
let plain = format!("huginn-smoke-{:x}-{stamp:x}", std::process::id());
let dotted = format!("{plain}.dotted");
let mut embedder = OllamaEmbedder::new(&ollama, &model);
let identity = embedder.model_identity(Duration::from_secs(60)).expect("the model is listed");
println!("identity: {identity:?}");
let texts = ["The organ owns the mind.".to_string(), "A recipe for lentil soup.".to_string()];
let vectors = embedder.embed(&texts, Duration::from_secs(60)).expect("two texts embed");
assert_eq!(vectors.len(), 2);
assert!(vectors.iter().all(|vector| vector.len() == identity.dimensions as usize));
let meta = CollectionMeta {
managed_by: MANAGED_BY.into(),
instance: "smoke".into(),
mind: "mind-commit-smoke".into(),
model: identity.name.clone(),
model_digest: identity.digest.clone(),
dimensions: identity.dimensions,
index_text_version: INDEX_TEXT_VERSION,
};
let mut store = QdrantIndex::new(&qdrant);
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
for name in [&plain, &dotted] {
assert_eq!(store.describe(name, Duration::from_secs(60)).unwrap(), Described::Absent);
store.recreate(name, &meta).unwrap();
assert_eq!(store.describe(name, Duration::from_secs(60)).unwrap(), Described::Labelled(meta.clone()));
assert!(store.ids(name).unwrap().is_empty());
let points: Vec<Point> = texts
.iter()
.zip(&vectors)
.enumerate()
.map(|(at, (text, vector))| Point {
id: point_id(&format!("smoke:doc:{at}")),
vector: vector.clone(),
payload: PointPayload {
doc_id: format!("smoke:doc:{at}"),
kind: "question".into(),
root: "smoke".into(),
ordinal: at as u64 + 1,
text_sha256: format!("{}", text.len()),
},
})
.collect();
store.upsert(name, &points).unwrap();
let expected: BTreeSet<String> = points.iter().map(|point| point.id.clone()).collect();
assert_eq!(store.ids(name).unwrap(), expected);
// The nearest point to the first text is the first point.
let query = serde_json_free_query(&vectors[0]);
let (status, answer) = raw("POST", &format!("{qdrant}/collections/{name}/points/search"), Some(&query));
assert_eq!(status, 200, "{answer}");
assert!(answer.contains(&points[0].id), "{answer}");
// The read side: a paraphrase that shares no exact words with the
// first document finds it, through the adapter, with the query
// instruction the worker uses.
let asked = format!("Instruct: {QUERY_INSTRUCTION}
Query: Who is responsible for keeping the memory?");
let query = embedder.embed(&[asked], Duration::from_secs(60)).expect("the query embeds").remove(0);
let hits = store.search(name, &query, 2, Duration::from_secs(60)).unwrap();
assert_eq!(hits.len(), 2, "{hits:?}");
assert_eq!((hits[0].doc_id.as_str(), hits[0].kind.as_str()), ("smoke:doc:0", "question"), "{hits:?}");
assert!(hits[0].score > hits[1].score, "{hits:?}");
println!("{name}: described, created, upserted {} points, queried, searched: {hits:?}", points.len());
}
}));
for name in [&plain, &dotted] {
let (status, _) = raw("DELETE", &format!("{qdrant}/collections/{name}?wait=true"), None);
println!("deleted {name}: {status}");
}
let (status, _) = raw("GET", &format!("{qdrant}/collections/{plain}"), None);
assert_eq!(status, 404, "the scratch collection is gone");
if let Err(panic) = outcome {
std::panic::resume_unwind(panic);
}
}
/// The nearest-neighbour body, written out by hand so the test needs no JSON
/// crate of its own.
fn serde_json_free_query(vector: &[f32]) -> String {
let numbers: Vec<String> = vector.iter().map(|value| value.to_string()).collect();
format!("{{\"vector\":[{}],\"limit\":1}}", numbers.join(","))
}

View file

@ -0,0 +1,39 @@
[package]
name = "huginn-mind"
version = "0.1.0"
edition.workspace = true
license.workspace = true
publish.workspace = true
# The organ's rule engine: storage, identity and admission of one instance's
# mind. Document shape, bounds, formats and keys come from `epiphany-pipeline`;
# the store comes from CultLib's Rust CultCache at the same rev Epiphany pins,
# so the leaf's wrappers and this crate's cache are one `cultcache-rs`. The
# read side selects through CultNet's typed selection (`cultnet-rs`, the same
# rev the daemon pins): the selection's vocabulary and evaluator are the
# substrate's, this crate supplies the rows. No socket, no index: those are the
# daemon's (Cut 10, Cut 11).
[dependencies]
anyhow = "1"
base64 = "0.22"
chrono = "0.4.44"
cultcache-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultcache-rs" }
cultnet-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultnet-rs" }
epiphany-pipeline = { git = "https://github.com/GameCult/Epiphany.git", rev = "73edee5cdaddece8bda9c6c3271e8ddd064337f9", package = "epiphany-pipeline" }
rmp-serde = "1"
schemars = "1"
serde = { version = "1", features = ["derive"] }
sha2 = "0.10"
[dev-dependencies]
serde_json = "1"
tempfile = "3"
# `test-support` exposes `Mind::store_path_for` and `Mind::open_with`, a
# grammar-checked path accessor and the fail-closed open door, for tests in
# other crates that need to reach a raw store directly. It buys no runtime
# surface: gated `#[cfg(any(test, feature = "test-support"))]`, so it
# compiles only under `cargo test` in this crate or when a dependent's own
# dev-dependencies enable it (see huginn-daemon/Cargo.toml).
[features]
test-support = []

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,443 @@
//! The image and the batch, the two places a rule or a view may look, and
//! every derivation over them. "In force" is recursive: a document is in force
//! when no resolution that is itself in force names it, so a withdrawn
//! resolution stops closing its subject. Computed over image and batch at rule
//! time, over the image at read time, never stored, and never from
//! `stored_at`.
use cultcache_rs::CultCacheEnvelope;
use epiphany_pipeline::{
OrgRepo, PipelineDocument, PipelineKind, PipelineRef, PipelineResolution, PipelineStewardship, ResolutionOutcome,
Short, Slug,
};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use crate::refusal::MindRefusal;
/// A batch document after A3 and A5: decoded, keyed, and its envelope.
pub(crate) struct Staged {
pub(crate) kind: PipelineKind,
pub(crate) key: String,
pub(crate) document: PipelineDocument,
pub(crate) envelope: CultCacheEnvelope,
}
/// An image document, decoded.
pub(crate) struct Held {
pub(crate) kind: PipelineKind,
pub(crate) key: String,
pub(crate) document: PipelineDocument,
}
/// The image and the batch, the two places a rule may look.
pub(crate) struct Docs {
pub(crate) image: Vec<Held>,
pub(crate) batch: Vec<Staged>,
}
pub(crate) fn kind_of_type(type_id: &str) -> Option<PipelineKind> {
PipelineKind::ALL.iter().copied().find(|kind| kind.type_id() == type_id)
}
impl Docs {
pub(crate) fn from_image(envelopes: &[CultCacheEnvelope]) -> Result<Self, MindRefusal> {
let mut image = Vec::new();
for envelope in envelopes {
let Some(kind) = kind_of_type(&envelope.r#type) else { continue };
let document = PipelineDocument::decode(envelope).map_err(|refusal| MindRefusal::Unavailable {
detail: format!("stored document {} does not decode: {refusal}", envelope.key),
})?;
image.push(Held { kind, key: envelope.key.clone(), document });
}
Ok(Self { image, batch: Vec::new() })
}
/// A4: one identity, once.
pub(crate) fn push(&mut self, staged: Staged) -> Result<(), MindRefusal> {
if self.batch.iter().any(|other| other.kind == staged.kind && other.key == staged.key) {
return Err(MindRefusal::IdentityCollision { kind: staged.kind, id: staged.key });
}
self.batch.push(staged);
Ok(())
}
pub(crate) fn in_batch(&self, kind: PipelineKind, id: &str) -> Option<&PipelineDocument> {
self.batch.iter().find(|staged| staged.kind == kind && staged.key == id).map(|staged| &staged.document)
}
pub(crate) fn in_image(&self, kind: PipelineKind, id: &str) -> Option<&PipelineDocument> {
self.image.iter().find(|held| held.kind == kind && held.key == id).map(|held| &held.document)
}
pub(crate) fn find(&self, kind: PipelineKind, id: &str) -> Option<&PipelineDocument> {
self.in_batch(kind, id).or_else(|| self.in_image(kind, id))
}
pub(crate) fn of_kind(&self, kind: PipelineKind) -> impl Iterator<Item = (&str, &PipelineDocument)> {
let batch = self.batch.iter().filter(move |staged| staged.kind == kind).map(|staged| (staged.key.as_str(), &staged.document));
let image = self.image.iter().filter(move |held| held.kind == kind).map(|held| (held.key.as_str(), &held.document));
batch.chain(image)
}
pub(crate) fn resolutions(&self) -> impl Iterator<Item = (&str, &PipelineResolution)> {
self.of_kind(PipelineKind::Resolution).filter_map(|(key, document)| match document {
PipelineDocument::Resolution(resolution) => Some((key, resolution)),
_ => None,
})
}
pub(crate) fn stewardships(&self) -> impl Iterator<Item = (&str, &PipelineStewardship)> {
self.of_kind(PipelineKind::Stewardship).filter_map(|(key, document)| match document {
PipelineDocument::Stewardship(stewardship) => Some((key, stewardship)),
_ => None,
})
}
/// Every resolution of one subject, in image and batch: the scope a
/// subject's sequence counts in.
pub(crate) fn resolutions_of(&self, subject: &PipelineRef) -> impl Iterator<Item = (&str, &PipelineResolution)> {
self.resolutions().filter(move |(_, resolution)| resolution.subject == *subject)
}
/// Every assignment of one repo to one mind, in image and batch, in force
/// or not: the scope a stewardship's sequence counts in.
pub(crate) fn assignments_of(&self, mind: &Slug, repo: &OrgRepo) -> impl Iterator<Item = (&str, &PipelineStewardship)> {
self.stewardships().filter(move |(_, stewardship)| stewardship.instance == *mind && stewardship.repo == *repo)
}
/// The greatest sequence among a subject's resolutions in image or batch,
/// `0` when it has none, so the next is `latest + 1`. `own` is the
/// document being checked, excluded by content identity so that an exact
/// replay measures the same `latest` the first admission did.
pub(crate) fn latest_resolution(&self, subject: &PipelineRef, own: Option<&PipelineResolution>) -> u32 {
self.resolutions_of(subject)
.filter(|(_, resolution)| own != Some(*resolution))
.map(|(_, resolution)| resolution.sequence)
.max()
.unwrap_or(0)
}
/// The sequence a derivation takes: the one the record it already derived
/// carries, when image or batch holds that record, so an exact replay
/// re-derives the same document and A9 answers with the stored receipt;
/// otherwise the next. A derivation is recognised by what it writes into
/// its record: a resolution by its outcome, an assignment by its note.
pub(crate) fn derived_resolution_sequence(
&self,
subject: &PipelineRef,
derived: impl Fn(&ResolutionOutcome) -> bool,
) -> u32 {
self.resolutions()
.find(|(_, resolution)| resolution.subject == *subject && derived(&resolution.outcome))
.map_or_else(|| self.latest_resolution(subject, None) + 1, |(_, resolution)| resolution.sequence)
}
/// The same for the assignment a hand-off derives, which notes its key.
pub(crate) fn derived_stewardship_sequence(&self, mind: &Slug, repo: &OrgRepo, note: &str) -> u32 {
self.stewardships()
.find(|(_, stewardship)| stewardship.instance == *mind && stewardship.repo == *repo && stewardship.note.0 == note)
.map_or_else(|| self.latest_stewardship(mind, repo, None) + 1, |(_, stewardship)| stewardship.sequence)
}
/// The same for a repo's assignments to one mind.
pub(crate) fn latest_stewardship(&self, mind: &Slug, repo: &OrgRepo, own: Option<&PipelineStewardship>) -> u32 {
self.assignments_of(mind, repo)
.filter(|(_, stewardship)| own != Some(*stewardship))
.map(|(_, stewardship)| stewardship.sequence)
.max()
.unwrap_or(0)
}
/// No resolution that is itself in force names the document. The
/// recursion is the whole rule: a withdrawn resolution stops counting, so
/// its subject is open again. `key(R)` is strictly longer than the id it
/// resolves, so the chain is bounded and this terminates.
pub(crate) fn in_force(&self, kind: PipelineKind, id: &str) -> bool {
self.in_force_unless(kind, id, |_| false)
}
/// In force, ignoring a resolution the caller's own document is or
/// derives: the ruling that answers a question is what resolves it, the
/// hand-off that withdraws a stewardship is what resolves that, and a
/// resolution does not resolve its own subject out from under itself.
pub(crate) fn in_force_unless(&self, kind: PipelineKind, id: &str, own: impl Fn(&PipelineResolution) -> bool) -> bool {
self.closing_resolution_unless(kind, id, own).is_none()
}
/// The in-force resolution naming the document, other than `own`, if any;
/// exact rather than a tie-break because A8 and the Q19 cap admit at most
/// one.
pub(crate) fn closing_resolution_unless(
&self,
kind: PipelineKind,
id: &str,
own: impl Fn(&PipelineResolution) -> bool,
) -> Option<(&str, &PipelineResolution)> {
self.resolutions().find(|&(key, resolution)| {
resolution.subject.kind == kind
&& resolution.subject.id.0 == id
&& !own(resolution)
&& self.in_force(PipelineKind::Resolution, key)
})
}
/// The resolution that closes the document, if any: the views' status,
/// derived where admission derives whether there is one at all.
pub(crate) fn closing_resolution(&self, kind: PipelineKind, id: &str) -> Option<(&str, &PipelineResolution)> {
self.closing_resolution_unless(kind, id, |_| false)
}
/// Every in-force stewardship of `(mind, repo)`, ignoring the withdrawal
/// `hand_off_key` derives. The stewardship row holds one in force at a
/// time, but that exclusion can put a second back in the list: the one a
/// replayed hand-off withdrew, beside whatever stewards the repo now.
pub(crate) fn stewardships_of(
&self,
mind: &Slug,
repo: &OrgRepo,
hand_off_key: Option<&str>,
) -> Vec<(&str, &PipelineStewardship)> {
self.assignments_of(mind, repo)
.filter(|(key, _)| {
self.in_force_unless(PipelineKind::Stewardship, key, |resolution| {
matches!(&resolution.outcome, ResolutionOutcome::Withdrawn { reason } if Some(reason.0.as_str()) == hand_off_key)
})
})
.collect()
}
/// The in-force record that stands later than `base` in the scope `base`
/// is sequenced in, if any: what a withdrawal that put `base` back in
/// force would leave standing beside it.
pub(crate) fn later_in_force(&self, base: &PipelineDocument) -> Option<&str> {
let kind = base.kind();
self.of_kind(kind).find(|(key, other)| later_than(base, other) && self.in_force(kind, key)).map(|(key, _)| key)
}
/// The stewardship of `(mind, repo)` a hand-off acts on: the one whose
/// withdrawal already carries `hand_off_key`, when image or batch holds
/// that withdrawal, so a replay derives the same withdrawal again and A9
/// answers with the stored receipt; otherwise the one in force. Key order
/// cannot pick between the two the exclusion leaves standing -- once a
/// repo has changed hands ten times, `n11` sorts before `n2` -- so the
/// content match picks, as it does for the sequence the other derivations
/// take.
pub(crate) fn stewardship_of(
&self,
mind: &Slug,
repo: &OrgRepo,
hand_off_key: Option<&str>,
) -> Option<(&str, &PipelineStewardship)> {
let candidates = self.stewardships_of(mind, repo, hand_off_key);
hand_off_key
.and_then(|hand_off| {
candidates.iter().find(|(key, _)| {
self.resolutions().any(|(_, resolution)| {
resolution.subject.kind == PipelineKind::Stewardship
&& resolution.subject.id.0 == *key
&& matches!(&resolution.outcome, ResolutionOutcome::Withdrawn { reason } if reason.0 == hand_off)
})
})
})
.or_else(|| candidates.first())
.copied()
}
}
/// Whether `other` is a later record than `base` in the scope `base` is
/// sequenced in: `(instance, repo)` for a stewardship, the campaign for a
/// target and the cut for a cut spec. Kinds that carry no sequence have no
/// scope and no order, and a resolution is never a base here: the cap above
/// refuses the withdrawal that would reinstate one before the question of a
/// later record arises.
fn later_than(base: &PipelineDocument, other: &PipelineDocument) -> bool {
use PipelineDocument as D;
match (base, other) {
(D::Stewardship(base), D::Stewardship(other)) => {
other.instance == base.instance && other.repo == base.repo && other.sequence > base.sequence
}
(D::Target(base), D::Target(other)) => other.campaign == base.campaign && other.revision > base.revision,
(D::CutSpec(base), D::CutSpec(other)) => {
other.campaign == base.campaign && other.cut == base.cut && other.revision > base.revision
}
_ => false,
}
}
/// The referring field of a citation edge, one per leaf field that names
/// another document. The wire spelling is the field's own name, so a client
/// reads an edge's role as the field it came from.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum CitationRole {
RaisedIn,
Answers,
Rulings,
Questions,
CutSpec,
Forks,
CutReport,
Findings,
Verdict,
Source,
Subject,
SupersededBy,
ResolvedBy,
DeferredTo,
Documents,
}
impl CitationRole {
pub const ALL: [CitationRole; 15] = [
Self::RaisedIn,
Self::Answers,
Self::Rulings,
Self::Questions,
Self::CutSpec,
Self::Forks,
Self::CutReport,
Self::Findings,
Self::Verdict,
Self::Source,
Self::Subject,
Self::SupersededBy,
Self::ResolvedBy,
Self::DeferredTo,
Self::Documents,
];
/// The wire spelling, equal to the referring field's name.
pub fn name(self) -> &'static str {
match self {
Self::RaisedIn => "raised_in",
Self::Answers => "answers",
Self::Rulings => "rulings",
Self::Questions => "questions",
Self::CutSpec => "cut_spec",
Self::Forks => "forks",
Self::CutReport => "cut_report",
Self::Findings => "findings",
Self::Verdict => "verdict",
Self::Source => "source",
Self::Subject => "subject",
Self::SupersededBy => "superseded_by",
Self::ResolvedBy => "resolved_by",
Self::DeferredTo => "deferred_to",
Self::Documents => "documents",
}
}
pub fn from_name(name: &str) -> Option<Self> {
Self::ALL.into_iter().find(|role| role.name() == name)
}
/// The kind whose documents carry this field.
pub fn carrier(self) -> PipelineKind {
use PipelineKind as K;
match self {
Self::RaisedIn => K::Question,
Self::Answers => K::Ruling,
Self::Rulings | Self::Questions => K::CutSpec,
Self::CutSpec | Self::Forks => K::CutReport,
Self::CutReport | Self::Findings => K::Verdict,
Self::Verdict => K::Finding,
Self::Source => K::FollowUp,
Self::Subject | Self::SupersededBy | Self::ResolvedBy | Self::DeferredTo => K::Resolution,
Self::Documents => K::HandOff,
}
}
/// The kinds a referent of this field may be: the one kind the field is
/// typed to, or every kind for the fields typed `PipelineRef` or a full
/// id of any kind. "Any" is spelled as all thirteen, never as none.
pub fn target_kinds(self) -> Vec<PipelineKind> {
use PipelineKind as K;
match self {
Self::Answers | Self::Questions | Self::Forks => vec![K::Question],
Self::Rulings => vec![K::Ruling],
Self::CutSpec => vec![K::CutSpec],
Self::CutReport => vec![K::CutReport],
Self::Findings => vec![K::Finding],
Self::Verdict => vec![K::Verdict],
Self::RaisedIn
| Self::Source
| Self::Subject
| Self::SupersededBy
| Self::ResolvedBy
| Self::DeferredTo
| Self::Documents => PipelineKind::ALL.to_vec(),
}
}
}
/// The kind segment of a full id, for the fields typed `Short` whose referent
/// may be of any kind.
pub(crate) fn kind_of_id(id: &str) -> Option<PipelineKind> {
let name = id.split(':').nth(1)?;
PipelineKind::ALL.iter().copied().find(|kind| kind.name() == name)
}
fn cited(kind: PipelineKind, id: &Short) -> PipelineRef {
PipelineRef { kind, id: id.clone() }
}
/// What a resolution's outcome names, by the field that names it.
pub(crate) fn outcome_citations(outcome: &ResolutionOutcome) -> Vec<(CitationRole, PipelineRef)> {
match outcome {
ResolutionOutcome::Superseded { by } => by.iter().map(|by| (CitationRole::SupersededBy, by.clone())).collect(),
ResolutionOutcome::Answered { by } => vec![(CitationRole::ResolvedBy, by.clone())],
ResolutionOutcome::Fixed { by: Some(by), .. } => vec![(CitationRole::ResolvedBy, by.clone())],
ResolutionOutcome::Deferred { to } => vec![(CitationRole::DeferredTo, to.clone())],
ResolutionOutcome::Fixed { by: None, .. } | ResolutionOutcome::Recorded { .. } | ResolutionOutcome::Withdrawn { .. } => {
vec![]
}
}
}
/// Every citation edge a document carries, derived from its leaf fields each
/// time it is asked and never stored: the one edge list. Admission's A7 reads
/// it to require the referents, and the read side's rows read it as their
/// references. `Fixed.commit`, `ForeignRef` precedents and the labels
/// (`depends_on`, `invariants`, `promise`) are not edges. A hand-off's
/// `documents` entry whose kind segment does not read cites nothing here;
/// admission refuses such an entry on the source side before it asks.
pub(crate) fn citations(document: &PipelineDocument) -> Vec<(CitationRole, PipelineRef)> {
use CitationRole as R;
use PipelineDocument as D;
use PipelineKind as K;
let mut edges = Vec::new();
match document {
D::Question(question) => edges.extend(question.raised_in.iter().map(|to| (R::RaisedIn, to.clone()))),
D::Ruling(ruling) => edges.extend(ruling.answers.iter().map(|to| (R::Answers, cited(K::Question, to)))),
D::CutSpec(spec) => {
edges.extend(spec.rulings.iter().map(|to| (R::Rulings, cited(K::Ruling, to))));
edges.extend(spec.questions.iter().map(|to| (R::Questions, cited(K::Question, to))));
}
D::CutReport(report) => {
edges.push((R::CutSpec, cited(K::CutSpec, &report.cut_spec)));
edges.extend(report.forks.iter().map(|to| (R::Forks, cited(K::Question, to))));
}
D::Verdict(verdict) => {
edges.push((R::CutReport, cited(K::CutReport, &verdict.cut_report)));
edges.extend(
verdict
.claims
.iter()
.flat_map(|claim| claim.findings.iter())
.map(|to| (R::Findings, cited(K::Finding, to))),
);
}
D::Finding(finding) => edges.push((R::Verdict, cited(K::Verdict, &finding.verdict))),
D::FollowUp(follow_up) => edges.push((R::Source, follow_up.source.clone())),
D::Resolution(resolution) => {
edges.push((R::Subject, resolution.subject.clone()));
edges.extend(outcome_citations(&resolution.outcome));
}
D::HandOff(hand_off) => edges.extend(
hand_off.documents.iter().filter_map(|to| kind_of_id(&to.0).map(|kind| (R::Documents, cited(kind, to)))),
),
D::Campaign(_) | D::Target(_) | D::Instance(_) | D::Stewardship(_) => {}
}
edges
}

View file

@ -0,0 +1,228 @@
//! CultNet's operation envelope around the mind's vocabulary. Pure functions:
//! a request goes out as `cultnet.operation_request.v0` and an answer comes
//! back as `cultnet.operation_response.v0`, both carrying base64 of named
//! MessagePack, which is what the C# reference and Eve's plugin ABI speak.
//!
//! A message this module cannot decode is a typed failure with a code, never
//! a transport error and never a call into a mind. `payload_encoding` is
//! CultNet's own check in `validate_message` and is not re-checked here.
//! `source_runtime_id` and `target_runtime_id` on a request are read by
//! nothing: ruling 18 says an identity is declared in the payload, not
//! granted by the envelope.
use anyhow::Result;
use base64::Engine;
use base64::engine::general_purpose::STANDARD;
use cultnet_rs::CultNetMessage;
use crate::wire::{HuginnMindRequest, HuginnMindResponse, MIND_REQUEST_SCHEMA, MIND_RESPONSE_SCHEMA, MIND_SERVICE_ID};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// The C# reference's `CultNetOperationServer.FailureSchemaId`. The Rust
/// `cultnet-rs` publishes no type for it, so this two-field mirror carries the
/// reference's keys; parity belongs in CultLib.
pub const FAILURE_SCHEMA: &str = "gamecult.cultnet.operation_failure.v1";
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct OperationFailure {
pub code: String,
pub message: String,
}
impl OperationFailure {
fn new(code: &str, message: impl Into<String>) -> Self {
Self { code: code.into(), message: message.into() }
}
}
fn encode_payload<T: Serialize>(value: &T) -> Result<String> {
Ok(STANDARD.encode(rmp_serde::to_vec_named(value)?))
}
/// One request, addressed to the mind service and naming its own operation.
pub fn encode_request(
message_id: &str,
request: &HuginnMindRequest,
source_runtime_id: Option<String>,
) -> Result<CultNetMessage> {
Ok(CultNetMessage::OperationRequest {
message_id: message_id.to_string(),
service_id: MIND_SERVICE_ID.to_string(),
operation: request.operation().to_string(),
payload_schema: MIND_REQUEST_SCHEMA.to_string(),
payload_encoding: "messagepack-base64".to_string(),
payload: encode_payload(request)?,
source_runtime_id,
target_runtime_id: None,
})
}
/// The message, or the failure that says why it is not a request of this
/// service. The envelope's `operation` string is checked against the decoded
/// request rather than trusted, so a payload cannot arrive under another
/// operation's name.
pub fn decode_request(message: &CultNetMessage) -> Result<(String, HuginnMindRequest), OperationFailure> {
let CultNetMessage::OperationRequest { message_id, service_id, operation, payload_schema, payload, .. } = message
else {
return Err(OperationFailure::new(
"not-an-operation-request",
"huginn.mind reads cultnet.operation_request.v0",
));
};
if service_id != MIND_SERVICE_ID {
return Err(OperationFailure::new("wrong-service", format!("{service_id} is not {MIND_SERVICE_ID}")));
}
if payload_schema != MIND_REQUEST_SCHEMA {
return Err(OperationFailure::new(
"wrong-payload-schema",
format!("{payload_schema} is not {MIND_REQUEST_SCHEMA}"),
));
}
let bytes = STANDARD
.decode(payload)
.map_err(|error| OperationFailure::new("payload-not-base64", error.to_string()))?;
let request: HuginnMindRequest = rmp_serde::from_slice(&bytes)
.map_err(|error| OperationFailure::new("payload-not-a-request", error.to_string()))?;
if operation != request.operation() {
return Err(OperationFailure::new(
"operation-mismatch",
format!("the envelope says {operation} and the payload is {}", request.operation()),
));
}
Ok((message_id.clone(), request))
}
/// One answer, whose status the response derives.
pub fn encode_response(
message_id: &str,
operation: &str,
response: &HuginnMindResponse,
source_runtime_id: &str,
) -> Result<CultNetMessage> {
Ok(CultNetMessage::OperationResponse {
message_id: message_id.to_string(),
service_id: MIND_SERVICE_ID.to_string(),
operation: operation.to_string(),
status: response.status().to_string(),
payload_schema: MIND_RESPONSE_SCHEMA.to_string(),
payload_encoding: "messagepack-base64".to_string(),
payload: encode_payload(response)?,
diagnostics: vec![],
source_runtime_id: Some(source_runtime_id.to_string()),
})
}
/// The answer to an envelope that never reached a mind. Infallible: the
/// failure is two strings and the daemon must always be able to say no.
pub fn encode_failure(
message_id: &str,
operation: &str,
failure: &OperationFailure,
source_runtime_id: &str,
) -> CultNetMessage {
CultNetMessage::OperationResponse {
message_id: message_id.to_string(),
service_id: MIND_SERVICE_ID.to_string(),
operation: operation.to_string(),
status: "rejected".to_string(),
payload_schema: FAILURE_SCHEMA.to_string(),
payload_encoding: "messagepack-base64".to_string(),
payload: STANDARD.encode(rmp_serde::to_vec_named(failure).expect("two strings encode")),
diagnostics: vec![failure.code.clone()],
source_runtime_id: Some(source_runtime_id.to_string()),
}
}
/// The client's side: the correlation key and either the mind's answer or the
/// failure that says the envelope never reached one.
pub fn decode_response(message: &CultNetMessage) -> Result<(String, Result<HuginnMindResponse, OperationFailure>)> {
let CultNetMessage::OperationResponse { message_id, payload_schema, payload, .. } = message else {
anyhow::bail!("huginn.mind answers with cultnet.operation_response.v0");
};
let bytes = STANDARD.decode(payload)?;
let answer = match payload_schema.as_str() {
MIND_RESPONSE_SCHEMA => Ok(rmp_serde::from_slice::<HuginnMindResponse>(&bytes)?),
FAILURE_SCHEMA => Err(rmp_serde::from_slice::<OperationFailure>(&bytes)?),
other => anyhow::bail!("{other} is neither {MIND_RESPONSE_SCHEMA} nor {FAILURE_SCHEMA}"),
};
Ok((message_id.clone(), answer))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::fixtures::{INSTANCE, OTHER_INSTANCE as OTHER, instance, provenance, slug};
use crate::wire::MindStatus;
use crate::{Faculty, MindRefusal, PipelineAdmissionBatch, PipelineAdmissionOutcome};
fn refusal() -> MindRefusal {
MindRefusal::ForeignInstance { declared: OTHER.into(), mind: INSTANCE.into() }
}
/// A refusal is an answer: it rides the response schema with a `rejected`
/// status and decodes back to the same value. A conflict and a replay are
/// answers about the mind, not rejections of the request.
#[test]
fn a_refusal_is_a_typed_response_not_a_transport_failure() {
let cases = [
(HuginnMindResponse::Refused(refusal()), "rejected"),
(HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(refusal())), "rejected"),
(
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed {
receipt_id: "r1".into(),
committed_at: "2026-09-16T12:00:00Z".into(),
writes: vec![],
}),
"accepted",
),
(HuginnMindResponse::Admit(PipelineAdmissionOutcome::AlreadyAdmitted { receipt_id: "r1".into() }), "accepted"),
(HuginnMindResponse::Admit(PipelineAdmissionOutcome::Conflict { identities: vec![] }), "accepted"),
(
HuginnMindResponse::Whoami(MindStatus {
instance: slug(INSTANCE),
schema_epoch: "epiphany.pipeline.epoch.v2".into(),
documents: 0,
receipts: 0,
index: crate::wire::IndexStatus::Current,
}),
"accepted",
),
];
for (response, expected) in cases {
let message = encode_response("m-1", "admit", &response, "huginn-yggdrasil").unwrap();
let CultNetMessage::OperationResponse { status, payload_schema, diagnostics, .. } = &message else {
panic!("not an operation response");
};
assert_eq!(status, expected, "{response:?}");
assert_eq!(payload_schema, MIND_RESPONSE_SCHEMA);
assert!(diagnostics.is_empty());
assert_eq!(decode_response(&message).unwrap(), ("m-1".into(), Ok(response)));
}
let failure = OperationFailure::new("wrong-service", "odin.catalog is not huginn.mind");
let message = encode_failure("m-2", "query", &failure, "huginn-yggdrasil");
let CultNetMessage::OperationResponse { status, payload_schema, diagnostics, .. } = &message else {
panic!("not an operation response");
};
assert_eq!((status.as_str(), payload_schema.as_str()), ("rejected", FAILURE_SCHEMA));
assert_eq!(diagnostics, &vec!["wrong-service".to_string()]);
assert_eq!(decode_response(&message).unwrap(), ("m-2".into(), Err(failure)));
}
/// The round trip the client and the daemon share, and the one code that
/// `serve::answer` cannot reach because it answers a foreign message with
/// `Error` instead.
#[test]
fn a_request_round_trips_and_a_foreign_message_is_not_a_request() {
let request = HuginnMindRequest::Admit(PipelineAdmissionBatch {
instance: slug(INSTANCE),
provenance: provenance(Faculty::Hands),
documents: vec![instance(INSTANCE)],
});
let message = encode_request("m-3", &request, Some("eureka-state".into())).unwrap();
assert_eq!(decode_request(&message).unwrap(), ("m-3".into(), request));
let foreign = CultNetMessage::Error { error: "not for you".into(), code: None, details: None };
assert_eq!(decode_request(&foreign).unwrap_err().code, "not-an-operation-request");
}
}

View file

@ -0,0 +1,412 @@
//! Test fixtures shared by the modules' tests: one valid document of each
//! kind, prepared through the leaf against the organ's cache, and a seeded
//! mind to admit into.
use chrono::{DateTime, TimeZone, Utc};
use cultcache_rs::CultCacheEnvelope;
use epiphany_pipeline::{
AuthorityMap, ClaimOutcome, CodeLocation, CommitRange, CutVerification, Date, DocRef, Evidence, EvidenceKind,
FindingConfidence, FindingOrigin, FindingSeverity, Label, Line, MutationRecord, OrgRepo, PipelineCampaign,
PipelineCutReport, PipelineCutSpec, PipelineDocument, PipelineFinding, PipelineFollowUp, PipelineHandOff,
PipelineInstance, PipelineKind, PipelineQuestion, PipelineRef, PipelineResolution, PipelineRuling,
PipelineStewardship, PipelineTarget, PipelineVerdict, Promise, QuestionOption, ReportCommit, ResolutionOutcome,
RulingAuthority, Sha, Short, Slug, StructuralDelta, TargetInvariant, VerdictClaim,
};
use crate::admission::{PipelineAdmissionBatch, PipelineAdmissionOutcome};
use crate::mind::{HuginnMindEpoch, Mind, schema_cache};
use crate::receipt::{Faculty, PipelineProvenance};
use crate::refusal::MindRefusal;
use crate::store::MindStore;
use crate::store::test_stores::MemoryStore;
pub(crate) const INSTANCE: &str = "yggdrasil";
pub(crate) const OTHER_INSTANCE: &str = "thought-cage";
/// A foreign instance of `INSTANCE`'s own length, differing in its last byte
/// alone: a comparison of lengths, of first bytes, or one gated on the
/// declared name being the longer admits it, and a comparison of values does
/// not.
pub(crate) const NEAR_INSTANCE: &str = "yggdrasix";
/// A foreign instance carrying `INSTANCE` whole as its prefix, so a prefix
/// test admits it.
pub(crate) const PREFIXED_INSTANCE: &str = "yggdrasil-two";
/// A foreign instance differing from `INSTANCE` in case alone. A `Slug` is
/// dot-joined labels, each `[A-Za-z0-9_-]{1,64}`, bounded to 64 bytes whole
/// (`epiphany_pipeline::dotted_text`), so this is a name a client may declare
/// and a mind may hold: two minds whose names differ only in case are two
/// minds, and a comparison that folds case answers one of them for the other.
pub(crate) const CASED_INSTANCE: &str = "Yggdrasil";
/// Two foreign instances differing from `OTHER_INSTANCE` (`thought-cage`) in
/// their separator alone: `_` and `.` are both legal in a `Slug` beside `-`
/// (`_` inside a label, `.` joining labels), so `thought-cage`, `thought_cage`
/// and `thought.cage` are three names, not one, and a comparison that folds a
/// separator into another answers one of them for another.
pub(crate) const UNDERSCORE_INSTANCE: &str = "thought_cage";
pub(crate) const DOTTED_INSTANCE: &str = "thought.cage";
pub(crate) const CAMPAIGN: &str = "eureka-state";
pub(crate) const REPO: &str = "GameCult/Epiphany";
pub(crate) const OTHER_REPO: &str = "GameCult/Huginn";
pub(crate) const INVARIANT: &str = "mind-admits";
pub(crate) fn slug(value: &str) -> Slug {
value.into()
}
pub(crate) fn s(value: &str) -> Short {
value.into()
}
pub(crate) fn l(value: &str) -> Label {
value.into()
}
pub(crate) fn repo(value: &str) -> OrgRepo {
value.into()
}
pub(crate) fn date() -> Date {
Date("2026-09-16".into())
}
pub(crate) fn sha() -> Sha {
Sha("5f98228d".into())
}
pub(crate) fn id(kind: &str, local: &str) -> String {
format!("{CAMPAIGN}:{kind}:{local}")
}
pub(crate) fn r(kind: PipelineKind, id: &str) -> PipelineRef {
PipelineRef { kind, id: s(id) }
}
fn range() -> CommitRange {
CommitRange { base: sha(), head: sha() }
}
fn location() -> CodeLocation {
CodeLocation { path: s("crates/huginn-mind/src/admission.rs"), line: 1, end_line: None }
}
pub(crate) fn evidence() -> Evidence {
Evidence { kind: EvidenceKind::Test, locator: "cargo test".into(), result: "ok".into() }
}
fn doc_ref() -> DocRef {
DocRef { path: s("notes/eureka-pipeline-state-target.md"), start_line: 1, end_line: 9, commit: sha() }
}
fn delta() -> StructuralDelta {
StructuralDelta {
lines_added: 0,
lines_removed: 0,
dependencies_added: vec![],
dependencies_removed: vec![],
formats_added: vec![],
formats_removed: vec![],
targets_added: vec![],
targets_removed: vec![],
}
}
pub(crate) fn instance(name: &str) -> PipelineDocument {
PipelineDocument::Instance(PipelineInstance {
instance: slug(name),
display_name: s(&format!("{name} mind")),
created_at: date(),
host: s(name),
})
}
pub(crate) fn stewardship(instance: &str, repo_name: &str) -> PipelineDocument {
stewardship_n(instance, repo_name, 1)
}
/// A repo's `n`th assignment to a mind. The sequence is per `(instance,
/// repo)` and the leaf keys it last, so `n2` is the record of a repo
/// transferred away and taken back.
pub(crate) fn stewardship_n(instance: &str, repo_name: &str, sequence: u32) -> PipelineDocument {
PipelineDocument::Stewardship(PipelineStewardship {
instance: slug(instance),
repo: repo(repo_name),
sequence,
assigned_on: date(),
note: "assigned".into(),
})
}
pub(crate) fn campaign(repos: &[&str]) -> PipelineDocument {
PipelineDocument::Campaign(PipelineCampaign {
slug: slug(CAMPAIGN),
title: "Eureka pipeline state".into(),
repos: repos.iter().map(|name| repo(name)).collect(),
working_branch: s("codex/eureka-pipeline-state"),
target_doc: doc_ref(),
})
}
pub(crate) fn target(revision: u32, invariants: &[&str]) -> PipelineDocument {
PipelineDocument::Target(PipelineTarget {
campaign: slug(CAMPAIGN),
revision,
invariants: invariants
.iter()
.map(|label| TargetInvariant { label: l(label), statement: "Only admission writes.".into() })
.collect(),
not_in_scope: vec![],
canonical_implementations: vec![],
doc: doc_ref(),
})
}
pub(crate) fn question(label: &str, options: &[&str], recommended: &str) -> PipelineDocument {
PipelineDocument::Question(PipelineQuestion {
campaign: slug(CAMPAIGN),
label: l(label),
title: "Who owns the state?".into(),
question: "Who owns the state?".into(),
options: options.iter().map(|option| QuestionOption { label: l(option), text: "an option".into() }).collect(),
recommended: l(recommended),
depends: vec![],
raised_in: None,
asked_on: date(),
})
}
pub(crate) fn ruling(label: &str) -> PipelineRuling {
PipelineRuling {
campaign: slug(CAMPAIGN),
label: l(label),
title: "An instance owns its mind".into(),
answers: None,
choice: None,
ruling: "An instance owns its mind.".into(),
operator_quote: None,
ruled_on: date(),
precedents: vec![],
authority: RulingAuthority::Standing,
}
}
pub(crate) fn cut_spec(cut: &str, revision: u32) -> PipelineCutSpec {
PipelineCutSpec {
campaign: slug(CAMPAIGN),
cut: l(cut),
revision,
title: "A cut".into(),
repo: repo(REPO),
branch: s("codex/eureka-pipeline-state"),
base: sha(),
depends_on: vec![],
first: vec![],
deletes: vec![],
keeps_moves: vec![],
adds: vec![],
file_changes: vec![],
authority_map: None::<AuthorityMap>,
verification: CutVerification { builds: vec![], tests: vec![], negative: vec![], operator: vec![] },
estimate: delta(),
rulings: vec![],
questions: vec![],
}
}
pub(crate) fn cut_report(cut: &str, attempt: u32) -> PipelineCutReport {
PipelineCutReport {
campaign: slug(CAMPAIGN),
cut_spec: s(&id("cut_spec", &format!("cut-{cut}.r1"))),
attempt,
repo: repo(REPO),
branch: s("codex/eureka-pipeline-state"),
commits: vec![ReportCommit { sha: sha(), subject: "Land the cut".into(), builds: true }],
range: range(),
verification: vec![evidence()],
mutations: vec![MutationRecord {
label: l("M1"),
rule: "the rule".into(),
location: location(),
before: "before".into(),
after: "after".into(),
commit: sha(),
failed_as_expected: true,
}],
deviations: vec![],
forks: vec![],
structural_delta: delta(),
landed_names: vec![],
undone: vec![],
promises: vec![Promise { label: l("P1"), text: "One derived key per document.".into() }],
}
}
pub(crate) fn claim(outcome: ClaimOutcome, findings: &[&str], promise: Option<&str>, mutations: &[&str]) -> VerdictClaim {
VerdictClaim {
claim: Line(format!("{outcome:?} claim")),
outcome,
evidence: vec![evidence()],
findings: findings.iter().map(|finding| s(finding)).collect(),
promise: promise.map(l),
mutations: mutations.iter().map(|label| l(label)).collect(),
}
}
pub(crate) fn verdict(cut: &str, pass: u32, claims: Vec<VerdictClaim>) -> PipelineDocument {
PipelineDocument::Verdict(PipelineVerdict {
campaign: slug(CAMPAIGN),
cut_report: s(&id("cut_report", &format!("cut-{cut}.h1"))),
pass,
range: range(),
claims,
})
}
pub(crate) fn finding(cut: &str, pass: u32, label: &str, confidence: FindingConfidence) -> PipelineFinding {
PipelineFinding {
campaign: slug(CAMPAIGN),
verdict: s(&id("verdict", &format!("cut-{cut}.s{pass}"))),
label: l(label),
range: range(),
confidence,
severity: FindingSeverity::High,
claim: "A dotted label composes two keys.".into(),
invariants: vec![l(INVARIANT)],
locations: vec![location()],
failure_scenario: "Two documents claim one key.".into(),
evidence: vec![evidence()],
precedents: vec![],
origin: FindingOrigin::Introduced,
}
}
pub(crate) fn follow_up(label: &str, source: PipelineRef) -> PipelineDocument {
PipelineDocument::FollowUp(PipelineFollowUp {
campaign: slug(CAMPAIGN),
label: l(label),
source,
repo: repo(REPO),
locations: vec![location()],
item: "Per-kind admission rules.".into(),
why_it_can_wait: "The organ owns admission.".into(),
owner: s("Hands"),
})
}
pub(crate) fn resolution(subject: PipelineRef, outcome: ResolutionOutcome) -> PipelineDocument {
resolution_n(subject, 1, outcome)
}
/// A subject's `n`th resolution. The sequence is per subject and the leaf
/// keys it last, so a withdrawn closure stays under its subject and the next
/// one is nameable.
pub(crate) fn resolution_n(subject: PipelineRef, sequence: u32, outcome: ResolutionOutcome) -> PipelineDocument {
PipelineDocument::Resolution(PipelineResolution {
subject,
sequence,
outcome,
rationale: "Resolved.".into(),
resolved_on: date(),
})
}
pub(crate) fn withdrawn() -> ResolutionOutcome {
ResolutionOutcome::Withdrawn { reason: "moot".into() }
}
pub(crate) fn superseded(by: &[PipelineRef]) -> ResolutionOutcome {
ResolutionOutcome::Superseded { by: by.to_vec() }
}
pub(crate) fn hand_off(from: &str, to: &str, repo_name: &str, documents: &[&str]) -> PipelineDocument {
PipelineDocument::HandOff(PipelineHandOff {
from_instance: slug(from),
to_instance: slug(to),
repo: repo(repo_name),
documents: documents.iter().map(|document| s(document)).collect(),
reason: "The workstation mind takes the campaign.".into(),
handed_on: date(),
})
}
pub(crate) fn provenance(faculty: Faculty) -> PipelineProvenance {
PipelineProvenance { faculty, agent: s("claude"), session: s("session-1"), tool: s("admit") }
}
pub(crate) fn now() -> DateTime<Utc> {
Utc.with_ymd_and_hms(2026, 9, 16, 12, 0, 0).unwrap()
}
/// The leaf's envelope for a document, keyed and encoded as the organ stores
/// it.
pub(crate) fn prepare(document: &PipelineDocument) -> CultCacheEnvelope {
document.validate().unwrap();
document.prepare(&schema_cache().unwrap()).unwrap()
}
/// The epoch record admission derives on the first write.
pub(crate) fn epoch() -> CultCacheEnvelope {
HuginnMindEpoch::envelope(&schema_cache().unwrap()).unwrap()
}
pub(crate) fn opened<S: MindStore>(store: S, instance: &str) -> Mind<S> {
Mind::open_with(store, &slug(instance)).unwrap()
}
/// Admits as Hands at the fixed clock.
pub(crate) fn admit<S: MindStore>(mind: &mut Mind<S>, documents: Vec<PipelineDocument>) -> PipelineAdmissionOutcome {
admit_at(mind, documents, now())
}
/// Admits as Hands at the clock the caller names: the views order by the
/// receipt's `committed_at`, which is this `now`.
pub(crate) fn admit_at<S: MindStore>(
mind: &mut Mind<S>,
documents: Vec<PipelineDocument>,
now: DateTime<Utc>,
) -> PipelineAdmissionOutcome {
let batch = PipelineAdmissionBatch { instance: mind.instance().clone(), provenance: provenance(Faculty::Hands), documents };
mind.admit(batch, now)
}
/// Admits as another faculty at the fixed clock: attribution the views filter
/// on and no rule reads (ruling 18).
pub(crate) fn admit_as<S: MindStore>(
mind: &mut Mind<S>,
faculty: Faculty,
documents: Vec<PipelineDocument>,
) -> PipelineAdmissionOutcome {
let batch = PipelineAdmissionBatch { instance: mind.instance().clone(), provenance: provenance(faculty), documents };
mind.admit(batch, now())
}
/// The `n`th plain question, for the tests that need more of them than they
/// need to tell apart.
pub(crate) fn question_n(n: u32) -> PipelineDocument {
question(&format!("Q{n}"), &["A", "B"], "A")
}
/// The first write into a fresh yggdrasil mind: its identity, stewardship of
/// the campaign repo, the campaign and a first target with one invariant.
pub(crate) fn seed<S: MindStore>(mind: &mut Mind<S>) {
let outcome = admit(mind, vec![instance(INSTANCE), stewardship(INSTANCE, REPO), campaign(&[REPO]), target(1, &[INVARIANT])]);
assert!(matches!(outcome, PipelineAdmissionOutcome::Committed { .. }), "{outcome:?}");
}
pub(crate) fn seeded() -> Mind<MemoryStore> {
let mut mind = opened(MemoryStore::new(), INSTANCE);
seed(&mut mind);
mind
}
pub(crate) fn refusal(outcome: PipelineAdmissionOutcome) -> MindRefusal {
match outcome {
PipelineAdmissionOutcome::Refused(refusal) => refusal,
other => panic!("expected a refusal, got {other:?}"),
}
}
pub(crate) fn committed(outcome: PipelineAdmissionOutcome) -> (String, Vec<PipelineRef>) {
match outcome {
PipelineAdmissionOutcome::Committed { receipt_id, writes, .. } => (receipt_id, writes),
other => panic!("expected a commit, got {other:?}"),
}
}

View file

@ -0,0 +1,312 @@
//! What of a document is searchable, and nothing else about the index.
//!
//! `index_text` is a rule about the leaf's documents: which fields are prose a
//! reader would search for. It is a pure function of one document, so it reads
//! no receipt, no provenance and no status, and nothing derived from it is
//! stored: the daemon's projection recomputes it, and `INDEX_TEXT_VERSION` is
//! how a change to it makes every projection rebuild. No network, no JSON, no
//! store handle: the daemon owns the vector store and the embedder.
use epiphany_pipeline::{PipelineDocument, PipelineKind, PipelineRef};
use sha2::{Digest, Sha256};
use crate::docs::Docs;
use crate::mind::Mind;
use crate::query::AdmissionIndex;
use crate::refusal::MindRefusal;
use crate::rows::root_and_local;
use crate::store::MindStore;
/// Bumped whenever `index_text` changes what it returns for any document.
pub const INDEX_TEXT_VERSION: u32 = 1;
/// The most bytes one document's text may carry, cut at a character boundary.
pub const INDEX_TEXT_MAX_BYTES: usize = 16 * 1024;
/// One document as the projection needs it: the key, the searchable text and
/// the digest of that text, and the ordinal of the receipt that wrote it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct IndexEntry {
pub id: PipelineRef,
/// The key's root segment: a campaign slug, or an instance.
pub root: String,
pub text: String,
pub text_sha256: String,
pub ordinal: u64,
}
/// The searchable text of a document, or `None` when its kind carries none
/// (`instance`, `stewardship`) or every field it names is empty. Lines are
/// joined by newlines. Withdrawn resolutions are indexed like any other: a
/// closed thing stays findable.
pub fn index_text(document: &PipelineDocument) -> Option<String> {
use PipelineDocument as D;
let mut lines: Vec<&str> = Vec::new();
match document {
D::Campaign(campaign) => lines.push(&campaign.title.0),
D::Target(target) => {
lines.extend(target.invariants.iter().map(|invariant| invariant.statement.0.as_str()));
lines.extend(target.not_in_scope.iter().map(|line| line.0.as_str()));
lines.extend(target.canonical_implementations.iter().map(|line| line.0.as_str()));
}
D::Question(question) => {
lines.push(&question.title.0);
lines.push(&question.question.0);
lines.extend(question.options.iter().map(|option| option.text.0.as_str()));
}
D::Ruling(ruling) => {
lines.push(&ruling.title.0);
lines.push(&ruling.ruling.0);
lines.extend(ruling.operator_quote.iter().map(|quote| quote.0.as_str()));
}
D::CutSpec(spec) => {
lines.push(&spec.title.0);
lines.extend(spec.first.iter().map(|line| line.0.as_str()));
lines.extend(spec.adds.iter().map(|line| line.0.as_str()));
lines.extend(spec.keeps_moves.iter().map(|line| line.0.as_str()));
}
D::CutReport(report) => {
lines.extend(report.commits.iter().map(|commit| commit.subject.0.as_str()));
for deviation in &report.deviations {
lines.push(&deviation.what.0);
lines.push(&deviation.why.0);
}
lines.extend(report.undone.iter().map(|line| line.0.as_str()));
lines.extend(report.promises.iter().map(|promise| promise.text.0.as_str()));
}
D::Verdict(verdict) => lines.extend(verdict.claims.iter().map(|claim| claim.claim.0.as_str())),
D::Finding(finding) => {
lines.push(&finding.claim.0);
lines.push(&finding.failure_scenario.0);
lines.extend(finding.evidence.iter().map(|evidence| evidence.result.0.as_str()));
}
D::FollowUp(follow_up) => {
lines.push(&follow_up.item.0);
lines.push(&follow_up.why_it_can_wait.0);
}
D::Resolution(resolution) => lines.push(&resolution.rationale.0),
D::HandOff(hand_off) => lines.push(&hand_off.reason.0),
D::Instance(_) | D::Stewardship(_) => {}
}
lines.retain(|line| !line.is_empty());
if lines.is_empty() {
return None;
}
let mut text = lines.join("\n");
if text.len() > INDEX_TEXT_MAX_BYTES {
let mut cut = INDEX_TEXT_MAX_BYTES;
while !text.is_char_boundary(cut) {
cut -= 1;
}
text.truncate(cut);
}
Some(text)
}
fn sha256_hex(text: &str) -> String {
Sha256::digest(text.as_bytes()).iter().map(|byte| format!("{byte:02x}")).collect()
}
impl<S: MindStore> Mind<S> {
/// The entries to project: every indexable document the mind holds, or
/// only the named ones. The ordinal is the writing receipt's. A named
/// document the mind does not hold is a refusal, not a silent omission: a
/// caller names what a commit just landed.
pub fn index_entries(&self, only: Option<&[PipelineRef]>) -> Result<Vec<IndexEntry>, MindRefusal> {
let documents: Vec<(PipelineKind, String, PipelineDocument)> = match only {
Some(refs) => {
let mut held = Vec::with_capacity(refs.len());
for reference in refs {
reference.validate_ref()?;
let document = self.get(reference.kind, &reference.id.0)?.ok_or_else(|| {
MindRefusal::Unavailable { detail: format!("index: {} is not held", reference.id.0) }
})?;
held.push((reference.kind, reference.id.0.clone(), document));
}
held
}
None => Docs::from_image(self.envelopes())?
.image
.into_iter()
.map(|held| (held.kind, held.key, held.document))
.collect(),
};
let facts = AdmissionIndex::build(self)?;
let mut entries = Vec::new();
for (kind, key, document) in documents {
let Some(text) = index_text(&document) else { continue };
let ordinal = facts.ordinal_of(kind, &key)?;
entries.push(IndexEntry {
root: root_and_local(&key).0.to_string(),
id: PipelineRef { kind, id: key.as_str().into() },
text_sha256: sha256_hex(&text),
text,
ordinal,
});
}
Ok(entries)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::fixtures::*;
use crate::receipt::Faculty;
use crate::store::test_stores::MemoryStore;
use epiphany_pipeline::{
ClaimOutcome, Deviation, FindingConfidence, PipelineDocument as D, PipelineKind as K, RulingAuthority,
};
use std::collections::BTreeMap;
/// A mind holding one document of every kind that carries text, each with
/// more than one text field where the kind has more than one, and the
/// question's closure withdrawn beside them. Returns the hand-off's key.
fn full_mind() -> (Mind<MemoryStore>, String) {
let mut mind = opened(MemoryStore::new(), INSTANCE);
let D::Target(mut target_doc) = target(1, &[INVARIANT, "second"]) else { panic!() };
target_doc.not_in_scope = vec!["No client writes.".into()];
target_doc.canonical_implementations = vec!["huginn-mind admission".into()];
committed(admit(
&mut mind,
vec![instance(INSTANCE), stewardship(INSTANCE, REPO), campaign(&[REPO]), D::Target(target_doc)],
));
committed(admit(&mut mind, vec![question("Q1", &["A", "B"], "A")]));
let mut ruling_doc = ruling("R1");
ruling_doc.answers = Some(s(&id("question", "Q1")));
ruling_doc.choice = Some(l("A"));
ruling_doc.operator_quote = Some("Own it.".into());
ruling_doc.authority = RulingAuthority::Operator;
committed(admit(&mut mind, vec![D::Ruling(ruling_doc)]));
// The derived closure of Q1 is withdrawn: a withdrawn resolution is a
// document like any other and stays findable.
let closed = id("resolution", "question.Q1.n1");
committed(admit(&mut mind, vec![resolution(r(K::Resolution, &closed), withdrawn())]));
let mut spec_doc = cut_spec("1", 1);
spec_doc.first = vec!["Read the leaf.".into()];
spec_doc.adds = vec!["An index module.".into(), "A worker.".into()];
spec_doc.keeps_moves = vec!["Admission stays.".into()];
committed(admit(&mut mind, vec![D::CutSpec(spec_doc)]));
let mut report_doc = cut_report("1", 1);
report_doc.deviations = vec![Deviation { what: "Changed a name.".into(), why: "It collided.".into() }];
report_doc.undone = vec!["The live smoke.".into()];
committed(admit(
&mut mind,
vec![D::CutReport(report_doc), follow_up("FU-1", r(K::Question, &id("question", "Q1")))],
));
committed(admit(
&mut mind,
vec![
verdict("1", 1, vec![claim(ClaimOutcome::Falsified, &[&id("finding", "cut-1.s1.F1")], Some("P1"), &["M1"])]),
D::Finding(finding("1", 1, "F1", FindingConfidence::Confirmed)),
],
));
let (_, writes) = committed(admit(&mut mind, vec![hand_off(INSTANCE, OTHER_INSTANCE, REPO, &[])]));
let hand_off_key = writes.iter().find(|write| write.kind == K::HandOff).unwrap().id.0.clone();
(mind, hand_off_key)
}
fn texts(mind: &Mind<MemoryStore>) -> BTreeMap<String, String> {
mind.index_entries(None).unwrap().into_iter().map(|entry| (entry.id.id.0, entry.text)).collect()
}
/// Each kind's search text is its named fields, joined by newlines, and
/// the two kinds with no prose have no entry.
#[test]
fn index_text_is_each_kinds_named_fields_and_the_identity_kinds_have_none() {
let (mind, hand_off_key) = full_mind();
let expected: Vec<(String, &str)> = vec![
(id("campaign", "self"), "Eureka pipeline state"),
(id("target", "r1"), "Only admission writes.\nOnly admission writes.\nNo client writes.\nhuginn-mind admission"),
(id("question", "Q1"), "Who owns the state?\nWho owns the state?\nan option\nan option"),
(id("ruling", "R1"), "An instance owns its mind\nAn instance owns its mind.\nOwn it."),
(id("cut_spec", "cut-1.r1"), "A cut\nRead the leaf.\nAn index module.\nA worker.\nAdmission stays."),
(
id("cut_report", "cut-1.h1"),
"Land the cut\nChanged a name.\nIt collided.\nThe live smoke.\nOne derived key per document.",
),
(id("verdict", "cut-1.s1"), "Falsified claim"),
(id("finding", "cut-1.s1.F1"), "A dotted label composes two keys.\nTwo documents claim one key.\nok"),
(id("follow_up", "FU-1"), "Per-kind admission rules.\nThe organ owns admission."),
// The withdrawn closure of Q1, admitted by hand.
(id("resolution", "resolution.question.Q1.n1.n1"), "Resolved."),
(hand_off_key, "The workstation mind takes the campaign."),
];
let held = texts(&mind);
for (key, text) in &expected {
match held.get(key) {
Some(found) => assert_eq!(found, text, "{key}"),
None => panic!("{key} has no entry; entries are {:?}", held.keys().collect::<Vec<_>>()),
}
}
let kinds: Vec<K> = mind.index_entries(None).unwrap().iter().map(|entry| entry.id.kind).collect();
assert!(!kinds.contains(&K::Instance) && !kinds.contains(&K::Stewardship), "{kinds:?}");
assert!(kinds.contains(&K::Resolution), "the derived closures are indexed too");
}
/// The text is the document's alone: the same document admitted by another
/// faculty at another clock, or closed by a later ruling, has the same
/// text and digest. Only the ordinal is the receipt's.
#[test]
fn index_text_ignores_admission_facts_and_indexes_withdrawn_resolutions() {
let mut mind = seeded();
committed(admit(&mut mind, vec![question("Q1", &["A", "B"], "A")]));
let q1 = id("question", "Q1");
let before = mind.index_entries(Some(&[r(K::Question, &q1)])).unwrap();
let mut other = seeded();
committed(admit_as(&mut other, Faculty::Soul, vec![question("Q1", &["A", "B"], "A")]));
let mut later = ruling("R1");
later.answers = Some(s(&q1));
later.choice = Some(l("A"));
committed(admit(&mut other, vec![D::Ruling(later)]));
let after = other.index_entries(Some(&[r(K::Question, &q1)])).unwrap();
assert_eq!(before, after, "a closed question is the same entry as an open one");
assert_eq!(before[0].ordinal, 2);
let closure = id("resolution", "question.Q1.n1");
committed(admit(&mut other, vec![resolution(r(K::Resolution, &closure), withdrawn())]));
let withdrawn_id = id("resolution", "resolution.question.Q1.n1.n1");
let entry = other.index_entries(Some(&[r(K::Resolution, &withdrawn_id)])).unwrap();
assert_eq!(entry.len(), 1, "a withdrawn resolution is indexed");
assert_eq!(entry[0].text, "Resolved.");
assert_eq!(entry[0].root, CAMPAIGN);
assert_eq!(entry[0].ordinal, 4);
}
#[test]
fn the_digest_is_sha256_of_the_text_and_a_missing_document_is_refused() {
let mind = seeded();
let campaign_ref = r(K::Campaign, &id("campaign", "self"));
let entry = &mind.index_entries(Some(std::slice::from_ref(&campaign_ref))).unwrap()[0];
assert_eq!(entry.text_sha256, "49c7e65adaad4bd34261300e9c5575847357b8f269508c9b2a46ec2aa75cc019");
assert_eq!((entry.root.as_str(), entry.ordinal), (CAMPAIGN, 1));
let absent = r(K::Question, &id("question", "Q9"));
assert!(matches!(mind.index_entries(Some(&[absent])), Err(MindRefusal::Unavailable { .. })));
// The identity and the stewardship carry no text: the seeded mind's
// indexable documents are its campaign and its target.
let all = mind.index_entries(None).unwrap();
assert_eq!(all.len(), 2, "{all:?}");
}
/// The bound is bytes, cut at a character boundary: three-byte characters
/// put the byte limit inside one.
#[test]
fn text_is_bounded_at_a_character_boundary() {
let wide: String = format!("{}x", "\u{20ac}".repeat(333));
assert_eq!(wide.len(), 1000);
let mut spec = cut_spec("1", 1);
spec.adds = (0..40).map(|_| wide.as_str().into()).collect();
let text = index_text(&D::CutSpec(spec)).unwrap();
// The title and its newline, then full lines of 1,001 bytes, then part
// of the next one, cut back to the last whole character.
let head = "A cut\n".len();
let stride = 1001;
let full_lines = (INDEX_TEXT_MAX_BYTES - head) / stride;
let consumed = head + full_lines * stride;
let inside = INDEX_TEXT_MAX_BYTES - consumed;
assert_ne!(inside % 3, 0, "the limit falls inside a character");
assert_eq!(text.len(), consumed + inside - inside % 3);
assert!(text.ends_with('\u{20ac}'));
}
}

View file

@ -0,0 +1,62 @@
//! `huginn-mind`: one instance's mind as typed CultCache state.
//!
//! The crate owns three decisions and nothing else. `Mind::open` owns "may
//! this store be this instance's mind": the opener refuses a foreign type, a
//! foreign epoch, a missing identity or a foreign identity before it attaches
//! anything, and takes the store's per-path lock first. `receipt::commit` owns "did a batch enter, whole, with a
//! receipt": one compare-and-swap over the store, a receipt naming the exact
//! bytes read and written, and a typed conflict when the swap loses.
//! `Mind::admit_prepared` owns "may this batch enter": every cross-field and
//! cross-document rule, every derived write, and the replay check, in one
//! ordered path that `admit`, the daemon's sink and the hand-off all share.
//!
//! The read side derives status, joins the receipts and answers one typed
//! selection (CultNet's, over rows this crate supplies) through the same
//! `docs` the rules use; nothing is stored for it.
//!
//! The wire vocabulary and its operation envelope (`envelope`) are here too,
//! so the daemon and the client share one set
//! of types.
//!
//! Document shape, bounds, formats and keys are `epiphany-pipeline`'s; the
//! organ registers, prepares, decodes and validates through the leaf's four
//! doors and never re-derives a key. The store is CultLib's owned redb
//! CultCache, whose lifetime-long exclusive lock is the single-writer
//! invariant's mechanism. No function here reads a clock or the environment:
//! `now` is passed in.
pub mod admission;
mod docs;
pub mod envelope;
pub mod index;
pub mod mind;
mod rows;
pub mod query;
pub mod receipt;
pub mod refusal;
pub mod store;
pub mod wire;
#[cfg(test)]
pub(crate) mod fixtures;
pub use docs::CitationRole;
pub use index::{INDEX_TEXT_MAX_BYTES, INDEX_TEXT_VERSION, IndexEntry, index_text};
pub use admission::{BATCH_MAX, PipelineAdmissionBatch, PipelineAdmissionOutcome};
pub use mind::{HuginnMindEpoch, Mind};
pub use query::{
AdmissionFacts, PipelineDocumentSummary, PipelineDocumentView, PipelineEdge, PipelineFacts, PipelinePageItems,
PipelineSelectionPage, PipelineStatus, PipelineStatusSummary, Ranked, SUMMARY_MAX_BYTES, SemanticQuery,
};
pub use receipt::{DocumentVersion, Faculty, HuginnCommitReceipt, PipelineProvenance, RECEIPT_SCHEMA_VERSION};
pub use refusal::MindRefusal;
pub use store::{MindStore, OwnedRedbMessagePackBackingStore};
pub use wire::{
DeferredAnswer, HuginnMindRequest, HuginnMindResponse, IndexStatus, MAX_DEFERRED_BODY_BYTES, MIND_REQUEST_SCHEMA, MIND_REQUEST_SCHEMA_JSON,
MIND_RESPONSE_SCHEMA, MIND_RESPONSE_SCHEMA_JSON, MIND_SERVICE_ID, MindStatus,
};
/// The leaf, whole, through the one crate that pins its rev: the daemon and
/// the client name `Slug`, `PipelineRef` and the document types from here, so
/// one `[dependencies]` entry decides which revision the workspace speaks.
pub use epiphany_pipeline;

View file

@ -0,0 +1,781 @@
//! The mind: one instance's store, opened fail-closed, and its image.
//!
//! Ruling 14: a store is canonical to exactly one instance, and the identity
//! lives in the state, in the `instance` document, not in a path. The path
//! under the state root is derived from the slug for convenience; the
//! document is the authority, and a store moved to another instance's
//! directory is refused. Ruling 15: one writer, the owned redb store's
//! lifetime-long exclusive lock. Ruling 20's analogue: the opener validates
//! every gate before it attaches, so a refused store is never read into an
//! image.
use std::path::{Path, PathBuf};
use cultcache_rs::{CultCache, CultCacheEnvelope, DatabaseEntry, OwnedRedbMessagePackBackingStore};
use cultnet_rs::CursorKey;
use epiphany_pipeline::{PIPELINE_SCHEMA_EPOCH, PipelineDocument, PipelineKind, Slug, register_pipeline_document_types};
use crate::receipt::HuginnCommitReceipt;
use crate::refusal::MindRefusal;
use crate::store::MindStore;
/// The store's record of the schema epoch it was written at, keyed by the
/// epoch string. Derived by admission on the first write; read by the opener,
/// which refuses a store written at any other epoch (`ForeignEpoch`) so a
/// breaking schema bump refuses the old store instead of misreading it.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "huginn.mind_epoch.v1", schema = "HuginnMindEpoch")]
pub struct HuginnMindEpoch {
#[cultcache(key = 0)]
pub schema_epoch: String,
}
impl HuginnMindEpoch {
/// The one envelope the record ever has: the current epoch, keyed by
/// itself. Admission appends it to the first write.
pub(crate) fn envelope(cache: &CultCache) -> Result<CultCacheEnvelope, MindRefusal> {
let record = Self { schema_epoch: PIPELINE_SCHEMA_EPOCH.to_string() };
cache
.prepare_entry_named(PIPELINE_SCHEMA_EPOCH, &record)
.map(|(envelope, _)| envelope)
.map_err(unavailable)
}
}
/// A store or cache error, carried as data.
pub(crate) fn unavailable(error: anyhow::Error) -> MindRefusal {
MindRefusal::Unavailable { detail: format!("{error:#}") }
}
/// A declared name's grammar, checked before its bytes are compared to
/// anything or reach the filesystem. `Slug` is `epiphany_pipeline`'s,
/// dot-joined ASCII labels, and the leaf now opens a public door onto its own
/// check, `Slug::validate_slug`, on the pattern of `PipelineRef::validate_ref`
/// (Self's ruling on the F6 fork): the grammar is the leaf's, never
/// re-derived here, so a fold that widens what counts as ASCII (a fullwidth
/// character folded to its plain form, say) has nothing local to weaken.
///
/// The leaf's own refusal names its own field, `slug`, and for a dotted name
/// the one label that failed rather than the whole declared name; a caller
/// here reports neither, since both would misname what the caller actually
/// sent. This instead names the field the declared name was held under and
/// the whole declared name, so `require_instance` and `Mind::open` each
/// refuse by the name a client can recognise, not `instance.instance` (the
/// old wrapper's field, a client never sent) and not a label fragment.
fn require_grammatical_slug(field: &str, declared: &Slug) -> Result<(), MindRefusal> {
declared.validate_slug().map_err(|_| {
MindRefusal::Document(epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: field.into(),
value: declared.0.clone(),
})
})
}
/// A cache that knows the fifteen types a mind's store may hold: the leaf's
/// thirteen through its registrar, the epoch record and the commit receipt.
/// It is the cache every envelope is prepared against.
pub(crate) fn schema_cache() -> Result<CultCache, MindRefusal> {
let mut cache = CultCache::new();
register_pipeline_document_types(&mut cache).map_err(unavailable)?;
cache.register_entry_type::<HuginnMindEpoch>().map_err(unavailable)?;
cache.register_entry_type::<HuginnCommitReceipt>().map_err(unavailable)?;
Ok(cache)
}
fn is_known_type(type_id: &str) -> bool {
type_id == HuginnMindEpoch::TYPE
|| type_id == HuginnCommitReceipt::TYPE
|| PipelineKind::ALL.iter().any(|kind| kind.type_id() == type_id)
}
/// One instance's mind: the declared instance, the store, the registered
/// cache attached to it, and the image (the store's envelopes as last pulled,
/// in identity order). The image is a cache of the store and is re-pulled
/// after every commit; nothing writes it directly.
pub struct Mind<S: MindStore> {
instance: Slug,
store: S,
cache: CultCache,
image: Vec<CultCacheEnvelope>,
/// Mints and verifies this process's selection cursors. Never stored: a
/// cursor does not survive a reopen, which the substrate accepts.
cursor_key: CursorKey,
}
impl Mind<OwnedRedbMessagePackBackingStore> {
/// `<state_root>/minds/<instance>/mind.redb`. Derived for convenience;
/// the `instance` document inside is the identity. Private: every caller
/// that can reach a raw path must go through the grammar door first, so
/// this is never exposed on its own. `open` calls it only after
/// `require_grammatical_slug` has already passed.
fn path_for(state_root: &Path, instance: &Slug) -> PathBuf {
state_root.join("minds").join(&instance.0).join("mind.redb")
}
/// Test-only door onto the same path: validates through
/// `require_grammatical_slug` first, exactly as `open` does, so a test
/// that must reach the raw store directly (to break a row on purpose)
/// cannot bypass the grammar the way the old public `path_for` did. Built
/// only for `cfg(test)` in this crate and for `huginn-daemon`'s own tests
/// via the `test-support` feature; never compiled into a release binary.
#[cfg(any(test, feature = "test-support"))]
pub fn store_path_for(state_root: &Path, instance: &Slug) -> Result<PathBuf, MindRefusal> {
require_grammatical_slug("instance", instance)?;
Ok(Self::path_for(state_root, instance))
}
/// Opens the instance's mind under the state root, taking the store's
/// exclusive lock for the mind's lifetime. A second owner of the same
/// path, in this process or another, is `MindAlreadyOwned`. An empty
/// store is a valid open; only the first admission may write into it.
pub fn open(state_root: &Path, instance: &Slug) -> Result<Self, MindRefusal> {
require_grammatical_slug("instance", instance)?;
let path = Self::path_for(state_root, instance);
let store = OwnedRedbMessagePackBackingStore::new(&path).map_err(|error| {
// The store reports a held lock as an error like any other; its
// message is the only signal, so it is matched here, once.
if format!("{error:#}").contains("already has an active owner") {
MindRefusal::MindAlreadyOwned { path: path.display().to_string() }
} else {
unavailable(error)
}
})?;
Self::open_checked(store, instance)
}
}
impl<S: MindStore> Mind<S> {
/// Test-only door onto the same fail-closed sequence `open` runs, for a
/// store already in hand (planted directly, or opened by a test through
/// `store_path_for`). Gated exactly like `store_path_for`: it compiles
/// only under `cfg(test)` in this crate or for a dependent's own tests
/// via the `test-support` feature. `huginn-daemon` enables that feature
/// only in `[dev-dependencies]`, so a plain `cargo build --release -p
/// huginn-daemon` never pulls it into the shipped binary; but
/// `--all-targets` (and `cargo test`) unifies dev-dependency features
/// across the whole invocation, and that build's `huginn-daemon` binary
/// does carry `open_with`. As of this writing no Idunn deploy recipe for
/// Huginn exists to pin which of those commands is the actual shipped
/// build, so treat this as the open question rather than a settled
/// release-only guarantee. A second in-process owner of one store is not
/// yet sealed for every `MindStore` impl (F2's recorded follow-up);
/// production reaches this sequence only through `Mind::open`, which
/// takes the per-path redb lock first.
#[cfg(any(test, feature = "test-support"))]
pub fn open_with(store: S, instance: &Slug) -> Result<Self, MindRefusal> {
Self::open_checked(store, instance)
}
/// Fail-closed, in this order, nothing attached until every step passes:
/// pull the raw envelopes; if anything is stored, exactly one epoch
/// record at the current epoch; every type is one of the fifteen; and
/// exactly one `instance` document naming the declared instance; then
/// register, attach and pull. The epoch gate runs first so a store
/// written at a foreign epoch is refused as `ForeignEpoch`, never as
/// `ForeignStore`: a real epoch bump always moves every type id, so the
/// type gate would otherwise fire first and the epoch gate would never
/// be reached for the one case it exists for (F5). This is the one
/// construction site of a `Mind`; `open` and the test-only `open_with`
/// both call it and neither reimplements it.
fn open_checked(store: S, instance: &Slug) -> Result<Self, MindRefusal> {
require_grammatical_slug("instance", instance)?;
let raw = store.pull_all().map_err(unavailable)?;
refuse_foreign_epoch(&raw)?;
refuse_foreign_types(&raw)?;
refuse_foreign_identity(&raw, instance)?;
let (cache, image) = attach(store.clone())?;
Ok(Self { instance: instance.clone(), store, cache, image, cursor_key: CursorKey::random() })
}
pub fn instance(&self) -> &Slug {
&self.instance
}
/// Ruling 14 across every transport: the declared instance is this mind's,
/// else `ForeignInstance { declared, mind }`. A1 for admission; the daemon
/// asks it for every read that names an instance, and compares nothing
/// itself.
pub fn require_instance(&self, declared: &Slug) -> Result<(), MindRefusal> {
require_grammatical_slug("declared", declared)?;
if declared != self.instance() {
return Err(MindRefusal::ForeignInstance {
declared: declared.0.clone(),
mind: self.instance().0.clone(),
});
}
Ok(())
}
/// The image: every envelope the store held at the last pull, in
/// identity order. Cut 10's snapshot source reads this.
pub fn envelopes(&self) -> &[CultCacheEnvelope] {
&self.image
}
pub(crate) fn cursor_key(&self) -> &CursorKey {
&self.cursor_key
}
pub fn is_empty(&self) -> bool {
self.image.is_empty()
}
/// The stored bytes of one document, by kind and id.
pub fn envelope(&self, kind: PipelineKind, id: &str) -> Option<&CultCacheEnvelope> {
self.raw_envelope(kind.type_id(), id)
}
/// One document, decoded through the leaf.
pub fn get(&self, kind: PipelineKind, id: &str) -> Result<Option<PipelineDocument>, MindRefusal> {
self.envelope(kind, id)
.map(|envelope| PipelineDocument::decode(envelope).map_err(MindRefusal::Document))
.transpose()
}
/// Every commit receipt in the image, in receipt-id order.
pub(crate) fn receipts(&self) -> Result<Vec<HuginnCommitReceipt>, MindRefusal> {
self.image
.iter()
.filter(|envelope| envelope.r#type == HuginnCommitReceipt::TYPE)
.map(|envelope| {
rmp_serde::from_slice::<HuginnCommitReceipt>(&envelope.payload).map_err(|error| {
MindRefusal::Unavailable { detail: format!("receipt {} does not decode: {error}", envelope.key) }
})
})
.collect()
}
/// The mind's own identity: the receipt id of its first admission
/// (ordinal 1), which admission never rewrites and which digests the
/// instance and the first batch's bytes. `None` for a mind that has not
/// been written yet. The instance name is only a label, since two stores
/// on two hosts may share it; this tells them apart unless their first
/// batches were byte-identical.
pub fn genesis_receipt_id(&self) -> Result<Option<String>, MindRefusal> {
Ok(self.receipts()?.into_iter().find(|receipt| receipt.ordinal == 1).map(|receipt| receipt.receipt_id))
}
pub(crate) fn raw_envelope(&self, type_id: &str, key: &str) -> Option<&CultCacheEnvelope> {
self.image.iter().find(|envelope| envelope.r#type == type_id && envelope.key == key)
}
pub(crate) fn cache(&self) -> &CultCache {
&self.cache
}
pub(crate) fn store(&self) -> &S {
&self.store
}
/// Re-pulls the image from the store through the attached cache.
pub(crate) fn refresh(&mut self) -> Result<(), MindRefusal> {
self.cache.pull_all_backing_stores().map_err(unavailable)?;
self.image = snapshot(&self.cache);
Ok(())
}
}
/// Step 3: a runtime store, a Mind store or any other file passed by mistake
/// dies on its first foreign type.
fn refuse_foreign_types(raw: &[CultCacheEnvelope]) -> Result<(), MindRefusal> {
if let Some(foreign) = raw.iter().find(|envelope| !is_known_type(&envelope.r#type)) {
return Err(MindRefusal::ForeignStore { r#type: foreign.r#type.clone() });
}
Ok(())
}
/// Step 2: a non-empty store carries exactly one epoch record, keyed by the
/// epoch it names, at the epoch this binary writes. A store with no epoch
/// record at all is `MissingIdentity`; every other defect is `ForeignEpoch`,
/// whose `found` names the defect: the stored epoch when the value is
/// foreign, the record's key when a record is keyed by anything else, and the
/// record count rendered as `"<n> records"` when more than one is stored. The
/// count is decided before any value is read, so a current record beside a
/// foreign one refuses the same way whichever the store returns first.
fn refuse_foreign_epoch(raw: &[CultCacheEnvelope]) -> Result<(), MindRefusal> {
if raw.is_empty() {
return Ok(());
}
let foreign = |found: String| MindRefusal::ForeignEpoch { found, expected: PIPELINE_SCHEMA_EPOCH.into() };
let mut records = raw.iter().filter(|envelope| envelope.r#type == HuginnMindEpoch::TYPE);
let Some(record) = records.next() else {
return Err(MindRefusal::MissingIdentity);
};
let extra = records.count();
if extra > 0 {
return Err(foreign(format!("{} records", extra + 1)));
}
if record.key != PIPELINE_SCHEMA_EPOCH {
return Err(foreign(record.key.clone()));
}
let record: HuginnMindEpoch = rmp_serde::from_slice(&record.payload)
.map_err(|error| MindRefusal::Unavailable { detail: format!("epoch record does not decode: {error}") })?;
if record.schema_epoch != PIPELINE_SCHEMA_EPOCH {
return Err(foreign(record.schema_epoch));
}
Ok(())
}
/// Step 4: a non-empty store carries exactly one `instance` document, and it
/// names the declared instance. This is where a store moved to another
/// instance's directory is refused.
fn refuse_foreign_identity(raw: &[CultCacheEnvelope], instance: &Slug) -> Result<(), MindRefusal> {
if raw.is_empty() {
return Ok(());
}
let mut documents = raw.iter().filter(|envelope| envelope.r#type == PipelineKind::Instance.type_id());
let (Some(envelope), None) = (documents.next(), documents.next()) else {
return Err(MindRefusal::MissingIdentity);
};
let PipelineDocument::Instance(stored) = PipelineDocument::decode(envelope)? else {
return Err(MindRefusal::MissingIdentity);
};
if stored.instance != *instance {
return Err(MindRefusal::ForeignInstance { declared: instance.0.clone(), mind: stored.instance.0 });
}
Ok(())
}
/// Step 5: the registered cache, the store as its one generic home, and the
/// image pulled through it.
fn attach<S: MindStore>(store: S) -> Result<(CultCache, Vec<CultCacheEnvelope>), MindRefusal> {
let mut cache = schema_cache()?;
cache.add_generic_backing_store(store).map_err(unavailable)?;
cache.pull_all_backing_stores().map_err(unavailable)?;
let image = snapshot(&cache);
Ok((cache, image))
}
fn snapshot(cache: &CultCache) -> Vec<CultCacheEnvelope> {
let mut image = cache.snapshot_envelopes();
image.sort_by(|left, right| (&left.r#type, &left.key).cmp(&(&right.r#type, &right.key)));
image
}
#[cfg(test)]
mod tests {
use super::*;
use crate::fixtures::{INSTANCE, epoch, instance, prepare, slug};
use crate::store::test_stores::MemoryStore;
use cultcache_rs::CacheBackingStore;
fn planted(rows: Vec<CultCacheEnvelope>) -> MemoryStore {
let store = MemoryStore::new();
for row in rows {
store.plant(row);
}
store
}
fn foreign(type_id: &str) -> CultCacheEnvelope {
let mut envelope = epoch();
envelope.r#type = type_id.into();
envelope
}
const FOREIGN_EPOCH: &str = "epiphany.pipeline.epoch.v0";
/// A type id that is foreign under every epoch, unlike
/// `"epiphany.pipeline.campaign.v1"`: that string names the pinned
/// leaf's real, currently-registered campaign kind, so `is_known_type`
/// accepts it and a gate-order mutant that runs the type gate first
/// still passes these fixtures by accident. `_never_a_kind_` cannot
/// collide with any real `PipelineKind` id at any epoch this leaf pin
/// will ever carry, so these fixtures actually exercise the epoch-first
/// order rather than getting lucky on a real id.
const NEVER_A_KNOWN_TYPE: &str = "epiphany.pipeline._never_a_kind_.v1";
/// An epoch record under an arbitrary key naming an arbitrary epoch, so a
/// key defect and a value defect can be planted apart.
fn epoch_record(key: &str, schema_epoch: &str) -> CultCacheEnvelope {
let cache = schema_cache().unwrap();
let record = HuginnMindEpoch { schema_epoch: schema_epoch.into() };
cache.prepare_entry_named(key, &record).unwrap().0
}
fn foreign_epoch() -> CultCacheEnvelope {
epoch_record(FOREIGN_EPOCH, FOREIGN_EPOCH)
}
#[test]
fn the_opener_refuses_foreign_epoch_missing_identity_and_foreign_type_before_attaching() {
let yggdrasil = slug(INSTANCE);
let cases: Vec<(Vec<CultCacheEnvelope>, MindRefusal)> = vec![
// Step 3, reached because the epoch record here is current: a
// runtime store passed by mistake.
(
vec![foreign("epiphany.runtime_spine.v47"), epoch(), prepare(&instance(INSTANCE))],
MindRefusal::ForeignStore { r#type: "epiphany.runtime_spine.v47".into() },
),
// Step 2 before step 4: a foreign epoch with no identity at all.
// Its key is foreign too, which is what refuses it here.
(
vec![foreign_epoch()],
MindRefusal::ForeignEpoch { found: FOREIGN_EPOCH.into(), expected: PIPELINE_SCHEMA_EPOCH.into() },
),
// Step 2: the value, under the one key the record may have.
(
vec![epoch_record(PIPELINE_SCHEMA_EPOCH, FOREIGN_EPOCH), prepare(&instance(INSTANCE))],
MindRefusal::ForeignEpoch { found: FOREIGN_EPOCH.into(), expected: PIPELINE_SCHEMA_EPOCH.into() },
),
// Step 2: the current epoch under any other key is not the epoch
// record, whatever it says about itself.
(
vec![epoch_record("not-the-epoch", PIPELINE_SCHEMA_EPOCH), prepare(&instance(INSTANCE))],
MindRefusal::ForeignEpoch { found: "not-the-epoch".into(), expected: PIPELINE_SCHEMA_EPOCH.into() },
),
// Step 2: two records are no record, counted before either value
// is read.
(
vec![epoch(), epoch_record("second", PIPELINE_SCHEMA_EPOCH), prepare(&instance(INSTANCE))],
MindRefusal::ForeignEpoch { found: "2 records".into(), expected: PIPELINE_SCHEMA_EPOCH.into() },
),
// Step 2: an identity without an epoch record.
(vec![prepare(&instance(INSTANCE))], MindRefusal::MissingIdentity),
// Step 4: an epoch record without an identity.
(vec![epoch()], MindRefusal::MissingIdentity),
// Step 4: two identities are no identity.
(
vec![epoch(), prepare(&instance(INSTANCE)), prepare(&instance("thought-cage"))],
MindRefusal::MissingIdentity,
),
];
for (rows, expected) in cases {
let store = planted(rows);
let before = store.rows();
let refusal = Mind::open_with(store.clone(), &yggdrasil).err().expect("refused");
assert_eq!(refusal, expected);
assert_eq!(store.pull_count(), 1, "{expected:?}: refused before attaching");
assert_eq!(store.rows(), before, "{expected:?}: bytes unchanged");
}
// A current record beside a foreign one is refused on the count, not
// on whichever of the two the store happens to return first.
let pair = (epoch(), foreign_epoch());
for rows in [vec![pair.0.clone(), pair.1.clone()], vec![pair.1, pair.0]] {
assert_eq!(
refuse_foreign_epoch(&rows).err(),
Some(MindRefusal::ForeignEpoch { found: "2 records".into(), expected: PIPELINE_SCHEMA_EPOCH.into() })
);
}
// A store that passes every gate attaches, which is the second pull.
let store = planted(vec![epoch(), prepare(&instance(INSTANCE))]);
let mind = Mind::open_with(store.clone(), &yggdrasil).unwrap();
assert_eq!(store.pull_count(), 2);
assert_eq!(mind.envelopes().len(), 2);
assert!(!mind.is_empty());
let empty = MemoryStore::new();
assert!(Mind::open_with(empty, &yggdrasil).unwrap().is_empty());
}
/// RS-1: receipt v2 is a new type id, so a store carrying an old,
/// v1-shaped receipt is refused at the type gate, before anything tries
/// to decode its 7-slot payload as the 8-slot v2 shape (F3: with
/// `#[serde(default)]` that decode would succeed and silently misread
/// every old receipt's ordinal as 0).
#[test]
fn a_v1_receipt_store_is_refused_at_open() {
let store =
planted(vec![epoch(), prepare(&instance(INSTANCE)), foreign("huginn.mind_commit_receipt.v1")]);
assert_eq!(
Mind::open_with(store, &slug(INSTANCE)).err(),
Some(MindRefusal::ForeignStore { r#type: "huginn.mind_commit_receipt.v1".into() })
);
}
/// RS-1, F5: a real epoch bump always moves every type id (F4), so a
/// store written at a foreign epoch also carries at least one foreign
/// type. The epoch gate must run first, or the type gate fires instead
/// and the epoch gate is never reached for the one case it exists for.
/// The eight cases of the opener test above are unchanged by the swap.
#[test]
fn a_store_written_at_the_previous_epoch_is_refused_by_the_epoch_gate() {
let raw = vec![foreign_epoch(), foreign("epiphany.pipeline.campaign.v0")];
assert_eq!(
Mind::open_with(planted(raw), &slug(INSTANCE)).err(),
Some(MindRefusal::ForeignEpoch { found: FOREIGN_EPOCH.into(), expected: PIPELINE_SCHEMA_EPOCH.into() })
);
}
/// A foreign store with no epoch record at all still refuses
/// `MissingIdentity`, the epoch-first order's own answer -- not
/// `ForeignStore`, which is what a mutant that only reorders the gates
/// when there *is* some epoch record present would answer, by running
/// the type gate first over a store that has no epoch record to look at.
/// Pinned by this test.
#[test]
fn a_foreign_type_with_no_epoch_record_is_missing_identity_not_foreign_store() {
let store = planted(vec![foreign(NEVER_A_KNOWN_TYPE)]);
assert_eq!(Mind::open_with(store, &slug(INSTANCE)).err(), Some(MindRefusal::MissingIdentity));
}
/// F3 (RS fix batch): an epoch record at the current key, but a
/// foreign value, beside a foreign type, still refuses `ForeignEpoch` --
/// the epoch gate reads the record's *value* before the type gate ever
/// runs, not only when the record's *key* is itself foreign. A mutant
/// that reorders the gates only on a foreign key would answer
/// `ForeignStore` here instead, since this record sits at the correct
/// key.
#[test]
fn a_foreign_epoch_value_at_the_current_key_beside_a_foreign_type_is_foreign_epoch() {
let store = planted(vec![epoch_record(PIPELINE_SCHEMA_EPOCH, FOREIGN_EPOCH), foreign(NEVER_A_KNOWN_TYPE)]);
assert_eq!(
Mind::open_with(store, &slug(INSTANCE)).err(),
Some(MindRefusal::ForeignEpoch { found: FOREIGN_EPOCH.into(), expected: PIPELINE_SCHEMA_EPOCH.into() })
);
}
#[test]
fn the_instance_document_is_the_identity_not_the_path() {
// The same bytes opened under another name are refused.
let store = planted(vec![epoch(), prepare(&instance(INSTANCE))]);
assert_eq!(
Mind::open_with(store, &slug("thought-cage")).err(),
Some(MindRefusal::ForeignInstance { declared: "thought-cage".into(), mind: INSTANCE.into() })
);
// A real store copied into another instance's directory is still
// yggdrasil's mind: it opens as yggdrasil, and not as thought-cage.
let root = tempfile::tempdir().unwrap();
let written = Mind::path_for(root.path(), &slug(INSTANCE));
{
let mut store = OwnedRedbMessagePackBackingStore::new(&written).unwrap();
store.push(&epoch()).unwrap();
store.push(&prepare(&instance(INSTANCE))).unwrap();
}
let moved = Mind::path_for(root.path(), &slug("thought-cage"));
std::fs::create_dir_all(moved.parent().unwrap()).unwrap();
std::fs::copy(&written, &moved).unwrap();
assert_eq!(
Mind::open(root.path(), &slug("thought-cage")).err(),
Some(MindRefusal::ForeignInstance { declared: "thought-cage".into(), mind: INSTANCE.into() })
);
let store = OwnedRedbMessagePackBackingStore::new(&moved).unwrap();
let mind = Mind::open_with(store, &slug(INSTANCE)).unwrap();
assert_eq!(mind.instance(), &slug(INSTANCE));
assert_eq!(mind.envelopes().len(), 2);
}
/// One check, one owner: the value admission refuses a foreign instance
/// with is the value `require_instance` returns, so the daemon asking it
/// for a read and `admit_steps` asking it for a write cannot disagree.
///
/// The foreign names are chosen so the check cannot pass by resembling a
/// comparison. `NEAR_INSTANCE` is the mind's own length and shares all but
/// its last byte, so a check reading only the lengths, only the first byte,
/// or running only when the declared name is longer lets it through;
/// `PREFIXED_INSTANCE` has the mind's whole name as its prefix, so a check
/// asking whether one starts with the other lets that one through; and
/// `CASED_INSTANCE` differs in case alone, which a slug label permits, so a
/// check folding case answers for a mind that is not this one.
#[test]
fn require_instance_is_the_one_check_admission_and_the_daemon_share() {
use crate::fixtures::{
CASED_INSTANCE, DOTTED_INSTANCE, NEAR_INSTANCE, OTHER_INSTANCE, PREFIXED_INSTANCE, UNDERSCORE_INSTANCE,
now, opened, provenance, seeded,
};
use crate::receipt::Faculty;
let mut mind = seeded();
let foreign = MindRefusal::ForeignInstance { declared: OTHER_INSTANCE.into(), mind: INSTANCE.into() };
assert_eq!(mind.require_instance(&slug(OTHER_INSTANCE)).err(), Some(foreign.clone()));
assert_eq!(mind.require_instance(&slug(INSTANCE)), Ok(()));
assert_eq!(NEAR_INSTANCE.len(), INSTANCE.len(), "the near name is the mind's own length");
assert_eq!(NEAR_INSTANCE[..INSTANCE.len() - 1], INSTANCE[..INSTANCE.len() - 1], "and differs in one byte");
assert!(PREFIXED_INSTANCE.starts_with(INSTANCE), "the prefixed name carries the mind's whole name");
assert_eq!(CASED_INSTANCE.to_ascii_lowercase(), INSTANCE, "the cased name differs in case alone");
assert_ne!(CASED_INSTANCE, INSTANCE);
for declared in [NEAR_INSTANCE, PREFIXED_INSTANCE, CASED_INSTANCE] {
assert_eq!(
mind.require_instance(&slug(declared)).err(),
Some(MindRefusal::ForeignInstance { declared: declared.into(), mind: INSTANCE.into() }),
"{declared} is not {INSTANCE}"
);
}
let batch = crate::admission::PipelineAdmissionBatch {
instance: slug(OTHER_INSTANCE),
provenance: provenance(Faculty::Hands),
documents: vec![instance(OTHER_INSTANCE)],
};
assert_eq!(mind.admit(batch, now()), crate::admission::PipelineAdmissionOutcome::Refused(foreign));
// `INSTANCE` ("yggdrasil") carries no separator, so no fold of `_` or
// `.` into `-` can ever equate a declared name with it; a mind whose
// own name carries one is required to tell the fold apart. Opened as
// `OTHER_INSTANCE` ("thought-cage"), a declared name that folds to the
// same string under such a comparison must still be refused.
let separated = opened(MemoryStore::new(), OTHER_INSTANCE);
assert_eq!(UNDERSCORE_INSTANCE.replace('_', "-"), OTHER_INSTANCE, "differs from OTHER_INSTANCE by `_` alone");
assert_eq!(DOTTED_INSTANCE.replace('.', "-"), OTHER_INSTANCE, "differs from OTHER_INSTANCE by `.` alone");
assert_ne!(UNDERSCORE_INSTANCE, OTHER_INSTANCE);
assert_ne!(DOTTED_INSTANCE, OTHER_INSTANCE);
for declared in [UNDERSCORE_INSTANCE, DOTTED_INSTANCE] {
assert_eq!(
separated.require_instance(&slug(declared)).err(),
Some(MindRefusal::ForeignInstance { declared: declared.into(), mind: OTHER_INSTANCE.into() }),
"{declared} is not {OTHER_INSTANCE}"
);
}
}
/// One generated test in place of more fixture pairs. The base name
/// carries all three separator bytes a `Slug` permits -- `_`, `.` and `-`
/// -- at three non-adjacent positions, so every combination of swapping
/// one separator for another at a fixed position is itself a grammatical,
/// distinct `Slug`. The cartesian product over the three positions (26
/// variants, the all-unchanged combination excepted) covers every
/// same-position permutation, including a fold that unifies two distinct
/// separators (`_` and `.` folded to one canonical byte) and a fold
/// applied to only one side of the comparison (the mind's own name
/// folded, the declared name left as written, or the reverse). Three more
/// variants pad the base with a leading dash, a trailing dash, and a
/// doubled interior dash -- all still grammatical, since a label's
/// hyphens carry no position or run restriction -- to reach `trim_matches`
/// and run-collapsing folds a same-position swap cannot produce. Every
/// one of the 30 variants must be refused as `ForeignInstance`, never
/// silently admitted as the mind's own name.
#[test]
fn every_separator_variant_of_a_declared_name_is_foreign_not_the_mind() {
const BASE: &str = "ab_cd.ef-gh";
let separators: Vec<(usize, char)> =
BASE.char_indices().filter(|(_, c)| matches!(c, '_' | '.' | '-')).collect();
assert_eq!(separators.len(), 3, "the base carries exactly one of each separator");
let mut variants: Vec<String> = Vec::new();
for a in ['_', '.', '-'] {
for b in ['_', '.', '-'] {
for c in ['_', '.', '-'] {
let mut bytes: Vec<char> = BASE.chars().collect();
bytes[separators[0].0] = a;
bytes[separators[1].0] = b;
bytes[separators[2].0] = c;
variants.push(bytes.into_iter().collect());
}
}
}
variants.retain(|variant| variant != BASE);
assert_eq!(variants.len(), 26, "3^3 combinations, less the one that reproduces BASE");
variants.push(format!("-{BASE}"));
variants.push(format!("{BASE}-"));
variants.push(format!("-{BASE}-"));
variants.push(BASE.replacen('-', "--", 1));
let mut seen = std::collections::BTreeSet::new();
variants.retain(|variant| seen.insert(variant.clone()));
assert_eq!(variants.len(), 30, "26 same-length separator permutations plus 4 padded variants, none colliding");
let mind = crate::fixtures::opened(MemoryStore::new(), BASE);
for declared in &variants {
assert!(declared.split('.').all(|label| !label.is_empty()), "{declared}: still grammatical");
assert_eq!(
mind.require_instance(&slug(declared)).err(),
Some(MindRefusal::ForeignInstance { declared: declared.clone(), mind: BASE.into() }),
"{declared} must be refused as foreign, not folded into {BASE}"
);
}
}
#[test]
fn a_mind_has_one_owner_at_a_time() {
let root = tempfile::tempdir().unwrap();
let yggdrasil = slug(INSTANCE);
let first = Mind::open(root.path(), &yggdrasil).unwrap();
assert!(first.is_empty());
let path = Mind::path_for(root.path(), &yggdrasil).display().to_string();
assert_eq!(
Mind::open(root.path(), &yggdrasil).err(),
Some(MindRefusal::MindAlreadyOwned { path: path.clone() })
);
drop(first);
let second = Mind::open(root.path(), &yggdrasil).unwrap();
assert!(second.is_empty());
assert_eq!(Mind::path_for(root.path(), &yggdrasil), root.path().join("minds").join(INSTANCE).join("mind.redb"));
}
/// N1, Soul's probe: `Mind::open(outer/inner, Slug("..\..\escaped"))` used
/// to return `Ok` and create `outer/escaped/mind.redb`, a store outside
/// the state root nothing beneath `outer/inner` names. `path_for` joins a
/// declared name as written and never validated it, so a `Slug` carrying
/// `..` and a path separator escaped through the one door that touches the
/// filesystem. `require_grammatical_slug` now runs before `path_for` is
/// even called, so the store is never created and the refusal is the
/// leaf's own grammar, not a filesystem error surfacing later.
#[test]
fn a_slug_outside_the_grammar_cannot_escape_the_state_root() {
let outer = tempfile::tempdir().unwrap();
let inner = outer.path().join("inner");
std::fs::create_dir_all(&inner).unwrap();
let escaping = Slug("..\\..\\escaped".into());
assert_eq!(
Mind::open(&inner, &escaping).err(),
Some(MindRefusal::Document(epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "instance".into(),
value: escaping.0.clone(),
}))
);
assert!(!outer.path().join("escaped").join("mind.redb").exists(), "no store escaped the state root");
assert!(!inner.join("minds").exists(), "no store was created under the state root either");
}
/// `open_with` calls `open_checked`, the one construction site `open`
/// shares with every store already in hand (planted directly, or opened
/// by a test through `store_path_for`), so it must run
/// `require_grammatical_slug` too, not only `open`'s own path-joining
/// caller. A store need not hold anything for this to refuse: the
/// declared name is checked before the store is even pulled.
#[test]
fn open_with_refuses_a_declared_instance_outside_the_grammar() {
let store = MemoryStore::new();
let bad = Slug("not a slug!".into());
assert_eq!(
Mind::open_with(store.clone(), &bad).err(),
Some(MindRefusal::Document(epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "instance".into(),
value: bad.0.clone(),
}))
);
assert_eq!(store.pull_count(), 0, "refused before the store was ever pulled");
}
/// `require_instance` is A1 for admission and the daemon's reads
/// alike, and it runs the same grammar door on the *declared* name before
/// comparing it to the mind's own. A declared name outside the grammar is
/// `InvalidFormat { field: "declared" }`, never folded into a comparison
/// or silently treated as merely foreign.
#[test]
fn require_instance_refuses_a_declared_name_outside_the_grammar() {
let mind = crate::fixtures::seeded();
let bad = Slug("not a slug!".into());
assert_eq!(
mind.require_instance(&bad).err(),
Some(MindRefusal::Document(epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "declared".into(),
value: bad.0.clone(),
}))
);
}
/// A Unicode hyphen (U+2010) is not ASCII `-`, so a grammatical
/// `Slug` never contains one and a name carrying it is refused by the
/// leaf's own grammar as written -- nothing here may fold it to plain
/// `-` before asking `Slug::validate_slug`, the way a fullwidth mutant
/// pinned by
/// `daemon::tests::a_declared_instance_outside_the_grammar_is_refused_by_name_before_the_identity_check`
/// folds a fullwidth letter to its ASCII form.
#[test]
fn a_unicode_hyphen_is_not_folded_before_the_grammar_check() {
let declared = Slug(format!("ab{}cd", '\u{2010}'));
assert!(!declared.0.is_ascii(), "U+2010 is not ASCII, unlike plain '-'");
assert_eq!(
Mind::open_with(MemoryStore::new(), &declared).err(),
Some(MindRefusal::Document(epiphany_pipeline::PipelineRefusal::InvalidFormat {
field: "instance".into(),
value: declared.0.clone(),
}))
);
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,438 @@
//! The commit receipt: what a batch read and what it wrote, byte for byte.
//!
//! This is the bounded duplicate of Epiphany's Mind receipt (FU-3), smaller
//! by design: no authority enum, no companion documents, no `invariant_owner`,
//! no `store_id`. Provenance is a field on the receipt, not a companion, and
//! the digest excludes it, so an exact replay across sessions and faculties
//! is idempotent. If a third consumer appears, extract the primitive into
//! CultLib rather than copy it again.
//!
//! Digest: `receipt_id = "mind-commit-" + hex(sha256(msgpack_named((instance,
//! strong_reads, writes))))`, where each list is `DocumentVersion`s in
//! `(document_type, document_key)` order and a version carries the exact
//! payload bytes and their SHA-256. `committed_at`, `provenance` and
//! `ordinal` are not digested: an exact replay keeps the ordinal its first
//! admission assigned.
use chrono::{DateTime, SecondsFormat, Utc};
use cultcache_rs::{CultCacheEnvelope, DatabaseEntry};
use epiphany_pipeline::{PipelineKind, PipelineRef, Short, Slug};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use crate::mind::{Mind, unavailable};
use crate::refusal::MindRefusal;
use crate::store::MindStore;
pub const RECEIPT_SCHEMA_VERSION: &str = "huginn.mind_commit_receipt.v2";
/// Which faculty a batch was admitted as. Attribution, not authority
/// (ruling 18): no admission rule trusts it, and Cut 9's views read it back
/// from the receipt.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub enum Faculty {
SelfFaculty,
Imagination,
Hands,
Soul,
MindSteward,
Eyes,
Operator,
}
/// Who admitted a batch: the faculty, the agent, its session and the tool.
/// A field on the receipt and nothing else; the digest excludes it.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct PipelineProvenance {
pub faculty: Faculty,
pub agent: epiphany_pipeline::Short,
pub session: epiphany_pipeline::Short,
pub tool: epiphany_pipeline::Short,
}
/// The exact bytes of one stored document, as read or as written.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct DocumentVersion {
pub document_type: String,
pub document_key: String,
pub schema_id: Option<String>,
pub payload_msgpack: Vec<u8>,
pub payload_sha256: String,
}
impl DocumentVersion {
pub fn from_envelope(envelope: &CultCacheEnvelope) -> Self {
Self {
document_type: envelope.r#type.clone(),
document_key: envelope.key.clone(),
schema_id: envelope.schema_id.clone(),
payload_msgpack: envelope.payload.clone(),
payload_sha256: sha256_hex(&envelope.payload),
}
}
pub fn identity(&self) -> (&str, &str) {
(&self.document_type, &self.document_key)
}
fn validate(&self) -> Result<(), MindRefusal> {
if self.document_type.is_empty() || self.document_key.is_empty() {
return Err(integrity("a document version names no identity"));
}
if self.payload_sha256 != sha256_hex(&self.payload_msgpack) {
return Err(integrity(&format!(
"document {}/{} payload digest mismatch",
self.document_type, self.document_key
)));
}
Ok(())
}
}
/// The receipt of one admitted batch, stored beside the documents it names
/// and keyed by its id.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "huginn.mind_commit_receipt.v2", schema = "HuginnCommitReceipt")]
pub struct HuginnCommitReceipt {
#[cultcache(key = 0)]
pub schema_version: String,
#[cultcache(key = 1)]
pub receipt_id: String,
#[cultcache(key = 2)]
pub instance: String,
#[cultcache(key = 3)]
pub provenance: PipelineProvenance,
#[cultcache(key = 4)]
pub strong_reads: Vec<DocumentVersion>,
#[cultcache(key = 5)]
pub writes: Vec<DocumentVersion>,
#[cultcache(key = 6)]
pub committed_at: String,
/// This mind's admission order: dense from 1, never reused, never
/// digested. `head` reads it back; `candidate` is the one place it is
/// assigned, as `head + 1`.
#[cultcache(key = 7)]
pub ordinal: u64,
}
impl HuginnCommitReceipt {
/// The digest, from what was read and written and never from who wrote
/// it or when.
pub fn digest(&self) -> Result<String, MindRefusal> {
let bytes = rmp_serde::to_vec_named(&(&self.instance, &self.strong_reads, &self.writes))
.map_err(|error| integrity(&format!("receipt digest input does not encode: {error}")))?;
Ok(format!("mind-commit-{}", sha256_hex(&bytes)))
}
pub fn validate(&self) -> Result<(), MindRefusal> {
if self.schema_version != RECEIPT_SCHEMA_VERSION {
return Err(integrity(&format!("receipt schema version {:?} is foreign", self.schema_version)));
}
if self.instance.is_empty() {
return Err(integrity("receipt names no instance"));
}
if self.writes.is_empty() {
return Err(integrity("receipt writes nothing"));
}
for version in self.strong_reads.iter().chain(&self.writes) {
version.validate()?;
}
if !unique_identities(&self.strong_reads) || !unique_identities(&self.writes) {
return Err(integrity("receipt names an identity twice"));
}
if self.ordinal == 0 {
return Err(integrity("receipt ordinal is 0"));
}
if self.receipt_id != self.digest()? {
return Err(integrity(&format!("receipt {} does not match its digest", self.receipt_id)));
}
Ok(())
}
}
/// How a commit ended. An exact replay is decided by admission before the
/// commit is attempted, so it is not an exit of the primitive.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) enum CommitOutcome {
/// The batch and its receipt landed whole.
Committed(HuginnCommitReceipt),
/// The swap lost: a strong read changed underneath, or a write's identity
/// already existed. Named where the kind is a pipeline kind.
Conflict(Vec<PipelineRef>),
}
/// The receipt a batch would land with: its id is the digest of the exact
/// bytes read and written, computed before anything is stored so admission
/// can look up an exact replay. This is the one construction site of a
/// receipt.
pub(crate) fn candidate(
instance: &Slug,
provenance: PipelineProvenance,
strong_reads: &[CultCacheEnvelope],
writes: &[CultCacheEnvelope],
ordinal: u64,
now: DateTime<Utc>,
) -> Result<HuginnCommitReceipt, MindRefusal> {
let mut receipt = HuginnCommitReceipt {
schema_version: RECEIPT_SCHEMA_VERSION.to_string(),
receipt_id: String::new(),
instance: instance.0.clone(),
provenance,
strong_reads: versions(strong_reads),
writes: versions(writes),
committed_at: now.to_rfc3339_opts(SecondsFormat::Secs, true),
ordinal,
};
receipt.receipt_id = receipt.digest()?;
Ok(receipt)
}
/// This mind's admission order: `N`, the count of receipts, when their
/// ordinals are exactly `{1..=N}`. Every reader of the ordinal asks here,
/// never by counting receipts itself: admission takes `head(self)? + 1` for
/// the next one, and `AdmissionIndex::build` (`query.rs`) calls it too, so
/// `query` and `view` carry the same density check rather than trusting a
/// stored ordinal unchecked. `Mind::receipts` is `pub(crate)`, so this crate
/// is the only place raw ordinals can be read at all; that keeps this the
/// one door, rather than something a doc comment merely asks callers to
/// respect. `ordinal` stays display-only on the read side until RS-3 makes
/// it the order. A chain that is not dense refuses rather than guessing, on
/// both sides alike.
pub(crate) fn head<S: MindStore>(mind: &Mind<S>) -> Result<u64, MindRefusal> {
let mut ordinals = mind.receipts()?.into_iter().map(|receipt| receipt.ordinal).collect::<Vec<_>>();
ordinals.sort_unstable();
let n = ordinals.len() as u64;
if ordinals.into_iter().eq(1..=n) {
Ok(n)
} else {
Err(MindRefusal::Unavailable { detail: "receipt ordinals are not 1..=N: found a duplicate or a gap".into() })
}
}
/// The versions of a set of envelopes in identity order, so the digest does
/// not depend on the order a batch was assembled in.
fn versions(envelopes: &[CultCacheEnvelope]) -> Vec<DocumentVersion> {
let mut versions = envelopes.iter().map(DocumentVersion::from_envelope).collect::<Vec<_>>();
versions.sort_by(|left, right| left.identity().cmp(&right.identity()));
versions
}
/// The stored receipt with the candidate's id, if the batch already landed:
/// its reads and writes must equal the candidate's, or the store holds a
/// receipt this binary cannot vouch for.
pub(crate) fn replay<S: MindStore>(
mind: &Mind<S>,
candidate: &HuginnCommitReceipt,
) -> Result<Option<HuginnCommitReceipt>, MindRefusal> {
let Some(envelope) = mind.raw_envelope(HuginnCommitReceipt::TYPE, &candidate.receipt_id) else {
return Ok(None);
};
let existing: HuginnCommitReceipt = rmp_serde::from_slice(&envelope.payload)
.map_err(|error| integrity(&format!("receipt {} does not decode: {error}", candidate.receipt_id)))?;
existing.validate()?;
if existing.strong_reads != candidate.strong_reads || existing.writes != candidate.writes {
return Err(integrity(&format!("receipt {} is stored with other content", candidate.receipt_id)));
}
Ok(Some(existing))
}
/// Lands a batch whole, or not at all: one compare-and-swap with the strong
/// reads as its expectation and the writes plus the receipt as its
/// replacements. The image is re-pulled either way; a lost swap is diffed
/// against the fresh image and typed `Conflict`.
pub(crate) fn commit<S: MindStore>(
mind: &mut Mind<S>,
receipt: HuginnCommitReceipt,
strong_reads: Vec<CultCacheEnvelope>,
writes: Vec<CultCacheEnvelope>,
) -> Result<CommitOutcome, MindRefusal> {
receipt.validate()?;
let receipt_envelope = mind
.cache()
.prepare_entry_named(&receipt.receipt_id, &receipt)
.map(|(envelope, _)| envelope)
.map_err(unavailable)?;
// The store requires every expected identity to be replaced, so each
// strong read is re-inserted unchanged beside the writes.
let mut replacements = writes.clone();
replacements.extend(strong_reads.iter().cloned());
replacements.push(receipt_envelope);
let expected: &[CultCacheEnvelope] = &strong_reads;
let landed = MindStore::compare_and_swap_batch(mind.store(), expected, replacements).map_err(unavailable)?;
mind.refresh()?;
if !landed {
let changed = strong_reads
.iter()
.filter(|read| mind.raw_envelope(&read.r#type, &read.key) != Some(read))
.chain(writes.iter().filter(|write| mind.raw_envelope(&write.r#type, &write.key).is_some()));
let identities = changed
.filter_map(|envelope| {
let kind = PipelineKind::ALL.iter().find(|kind| kind.type_id() == envelope.r#type)?;
Some(PipelineRef { kind: *kind, id: Short(envelope.key.clone()) })
})
.collect();
return Ok(CommitOutcome::Conflict(identities));
}
Ok(CommitOutcome::Committed(receipt))
}
fn unique_identities(versions: &[DocumentVersion]) -> bool {
let identities = versions.iter().map(DocumentVersion::identity).collect::<std::collections::BTreeSet<_>>();
identities.len() == versions.len()
}
fn integrity(detail: &str) -> MindRefusal {
MindRefusal::Unavailable { detail: detail.into() }
}
pub(crate) fn sha256_hex(bytes: &[u8]) -> String {
format!("{:x}", Sha256::digest(bytes))
}
#[cfg(test)]
mod tests {
use std::sync::{Arc, Mutex};
use super::*;
use crate::fixtures::{INSTANCE, admit, committed, id, instance, now, opened, prepare, provenance, question, refusal, seed, slug};
use crate::mind::schema_cache;
use crate::store::test_stores::MemoryStore;
use cultcache_rs::{CacheBackingStore, PushAllOptions};
use epiphany_pipeline::{PipelineDocument, PipelineKind, PipelineRefusal};
/// A memory store that keeps the replacement set of every swap it was
/// asked for, so a test can count the commit's calls as well as read
/// their contents.
#[derive(Clone)]
struct RecordingStore {
inner: MemoryStore,
calls: Arc<Mutex<Vec<Vec<CultCacheEnvelope>>>>,
}
impl RecordingStore {
fn new() -> Self {
Self { inner: MemoryStore::new(), calls: Arc::new(Mutex::new(Vec::new())) }
}
fn forget(&self) {
self.calls.lock().unwrap().clear();
}
fn calls(&self) -> Vec<Vec<CultCacheEnvelope>> {
self.calls.lock().unwrap().clone()
}
}
impl CacheBackingStore for RecordingStore {
fn pull_all(&self) -> anyhow::Result<Vec<CultCacheEnvelope>> {
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> anyhow::Result<()> {
self.inner.push(entry)
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> anyhow::Result<()> {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> anyhow::Result<()> {
self.inner.push_all(entries, options)
}
}
impl MindStore for RecordingStore {
fn compare_and_swap_batch(
&self,
expected: &[CultCacheEnvelope],
replacements: Vec<CultCacheEnvelope>,
) -> anyhow::Result<bool> {
self.calls.lock().unwrap().push(replacements.clone());
self.inner.compare_and_swap_batch(expected, replacements)
}
}
/// The receipt is part of the batch's own swap, never a second one after
/// it: a receipt written separately could land while the documents did
/// not, or the reverse.
#[test]
fn a_commit_is_one_swap_carrying_the_documents_and_the_receipt_together() {
let store = RecordingStore::new();
let mut mind = opened(store.clone(), INSTANCE);
seed(&mut mind);
store.forget();
let (receipt_id, _) = committed(admit(&mut mind, vec![question("Q1", &["A", "B"], "A")]));
let calls = store.calls();
assert_eq!(calls.len(), 1, "one commit, one swap");
let landed = calls[0].iter().map(|entry| (entry.r#type.clone(), entry.key.clone())).collect::<Vec<_>>();
assert_eq!(landed.len(), 2, "{landed:?}");
assert!(landed.contains(&(PipelineKind::Question.type_id().to_string(), id("question", "Q1"))), "{landed:?}");
assert!(landed.contains(&(HuginnCommitReceipt::TYPE.to_string(), receipt_id)), "{landed:?}");
}
/// RS-1: `head` requires the stored ordinals to be exactly `{1..=N}`.
/// A duplicate and a gap are planted directly, bypassing the commit path,
/// and both `head` itself and a fresh admission over the same store must
/// refuse. `query.rs`'s
/// `query_and_view_refuse_a_receipt_chain_that_is_not_dense` pins the
/// same check on the read side.
#[test]
fn a_chain_that_is_not_dense_refuses_head_and_admission() {
let base = MemoryStore::new();
let mut mind = opened(base.clone(), INSTANCE);
seed(&mut mind);
committed(admit(&mut mind, vec![question("Q1", &["A", "B"], "A")]));
assert_eq!(head(&mind).unwrap(), 2, "the seed and Q1's batch");
let cache = schema_cache().unwrap();
let plant_at = |ordinal: u64, label: &str| {
let broken = MemoryStore::new();
for row in base.rows() {
broken.plant(row);
}
let extra = candidate(
&slug(INSTANCE),
provenance(Faculty::Hands),
&[],
&[prepare(&question(label, &["A", "B"], "A"))],
ordinal,
now(),
)
.unwrap();
broken.plant(cache.prepare_entry_named(&extra.receipt_id, &extra).unwrap().0);
broken
};
// A duplicate: a third receipt at ordinal 2, which the store already
// carries.
let duplicated = plant_at(2, "Q2");
assert!(matches!(head(&opened(duplicated.clone(), INSTANCE)), Err(MindRefusal::Unavailable { .. })));
let mut duplicated_mind = opened(duplicated, INSTANCE);
assert!(matches!(
refusal(admit(&mut duplicated_mind, vec![question("Q3", &["A", "B"], "A")])),
MindRefusal::Unavailable { .. }
));
// A gap: a third receipt at ordinal 4, skipping 3.
let gapped = plant_at(4, "Q2");
assert!(matches!(head(&opened(gapped.clone(), INSTANCE)), Err(MindRefusal::Unavailable { .. })));
let mut gapped_mind = opened(gapped, INSTANCE);
assert!(matches!(
refusal(admit(&mut gapped_mind, vec![question("Q3", &["A", "B"], "A")])),
MindRefusal::Unavailable { .. }
));
}
/// S5 closed against the real type: the organ's own receipt in a mind's
/// store is never decoded as a pipeline document.
#[test]
fn decode_refuses_the_organs_own_receipt() {
let mut envelope = prepare(&instance(INSTANCE));
envelope.r#type = HuginnCommitReceipt::TYPE.into();
assert_eq!(
PipelineDocument::decode(&envelope),
Err(PipelineRefusal::ForeignStore { r#type: "huginn.mind_commit_receipt.v2".into() })
);
}
}

View file

@ -0,0 +1,147 @@
//! Typed refusals of the organ. A refusal is data with a field an agent can
//! act on, never a transport error: the outcome carries it, the wire (Cut 10)
//! serialises it, the tools (Cut 13) show it.
use cultnet_rs::SelectionRefusal;
use epiphany_pipeline::{PipelineKind, PipelineRefusal};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// Every way a mind refuses. `Document` wraps the leaf's four (bounds,
/// formats, key identity, foreign type); the rest are the organ's: store
/// identity, epoch, ownership, batch shape, references, and every cross-field
/// and cross-document rule of admission.
///
/// The two sequenced kinds refuse in the same pair of shapes: an
/// `OutOfSequence` when the writer's `sequence` is not the previous plus one,
/// and `AlreadyResolved`/`AlreadyStewarded` when a record of that scope is
/// still in force.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub enum MindRefusal {
Document(PipelineRefusal),
ForeignInstance { declared: String, mind: String },
MissingIdentity,
ForeignEpoch { found: String, expected: String },
ForeignStore { r#type: String },
MindAlreadyOwned { path: String },
BatchSize { actual: u32 },
IdentityCollision { kind: PipelineKind, id: String },
MissingReference { kind: PipelineKind, id: String },
AlreadyResolved { subject: String },
ResolutionOutOfSequence { subject: String, expected: u32, actual: u32 },
AlreadyStewarded { repo: String },
StewardshipOutOfSequence { repo: String, expected: u32, actual: u32 },
IncompatibleResolution { subject_kind: PipelineKind, outcome: String },
WouldReinstateOverLater { subject: String, later: String },
CitesResolvedDocument { kind: PipelineKind, id: String },
EmptySupersession,
UnknownSupersessor { id: String },
RevisionWithoutSupersession { kind: PipelineKind, revision: u32 },
DuplicateLabel { field: String, label: String },
InvalidOptions { question: String },
InvalidChoice { ruling: String, choice: String },
QuoteWithoutOperator { ruling: String },
EmptyRepos { campaign: String },
RepoNotStewarded { repo: String },
RepoNotInCampaign { repo: String },
CutReportWithoutSpec { report: String },
SpecMismatch { field: String },
RangeOutsideCommits { head: String },
FalsifiedClaimWithoutConfirmedFinding { claim: String },
UnprovenClaimWithConfirmedFinding { claim: String },
PromiseWithoutVerdict { promise: String },
UnknownMutationLabel { label: String },
FindingWithoutRange,
FindingWithoutEvidence,
UnknownInvariant { label: String },
NotStewarded { repo: String },
/// The one refusal a mind never raises: the answer exceeds the largest
/// body the organ will deliver by any plane, one send or a deferred body.
/// It lives here because a refusal rides the response schema and there is
/// one of those, so a client reads it the way it reads every other refusal
/// rather than parsing a second vocabulary. `bytes` is the encoded answer
/// (the payload, not its envelope) and `limit` is the largest body the
/// daemon defers, both so a caller can narrow its own request; nothing is
/// truncated or paginated on its behalf. The daemon owns it and its
/// deferral bound decides it; no rule of the mind is involved.
ResponseTooLarge { bytes: u64, limit: u64 },
/// A selection the substrate or the organ's value door refused: `field`
/// is the selection field the client sent, `value` the offending part.
SelectionInvalid { field: String, value: Option<String>, message: String },
/// A cursor that does not decode, does not belong to this selection or
/// was not minted by this process, or names a snapshot this mind has not
/// reached.
CursorInvalid { message: String },
Unavailable { detail: String },
}
impl std::fmt::Display for MindRefusal {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(formatter, "mind refusal: {self:?}")
}
}
impl std::error::Error for MindRefusal {}
impl From<PipelineRefusal> for MindRefusal {
fn from(refusal: PipelineRefusal) -> Self {
Self::Document(refusal)
}
}
/// The one total mapping from the substrate's refusals, with no wildcard arm.
/// `CursorStale` is unreachable: the organ answers at the cursor's own `asOf`,
/// so a stale refusal is an organ defect and says so. `ReferenceOutsideTarget`
/// is integrity: A7 admits only referents of the declared kind, so a stored
/// edge outside its target means the store is not what admission wrote.
impl From<SelectionRefusal> for MindRefusal {
fn from(refusal: SelectionRefusal) -> Self {
match refusal {
SelectionRefusal::Invalid(invalid) => {
Self::SelectionInvalid { field: invalid.field, value: invalid.value, message: invalid.message }
}
SelectionRefusal::CursorInvalid { message } => Self::CursorInvalid { message },
SelectionRefusal::CursorStale { as_of, current } => Self::Unavailable {
detail: format!("cursor_stale: the organ answers at the cursor's asOf ({as_of}), yet was asked at {current}"),
},
reference @ SelectionRefusal::ReferenceOutsideTarget { .. } => {
Self::Unavailable { detail: format!("stored reference outside its declared target: {reference}") }
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use cultnet_rs::SelectionInvalid;
/// The mapping is the organ's word on each substrate refusal: a bad
/// selection and a bad cursor cross as themselves, and the two the organ
/// is built never to raise cross as integrity faults, not as anything a
/// client could act on.
#[test]
fn each_substrate_refusal_maps_to_one_organ_refusal() {
let invalid = SelectionInvalid { field: "keys".into(), value: Some("x".into()), message: "no".into() };
assert_eq!(
MindRefusal::from(SelectionRefusal::Invalid(invalid)),
MindRefusal::SelectionInvalid { field: "keys".into(), value: Some("x".into()), message: "no".into() }
);
assert_eq!(
MindRefusal::from(SelectionRefusal::CursorInvalid { message: "bad".into() }),
MindRefusal::CursorInvalid { message: "bad".into() }
);
assert!(matches!(
MindRefusal::from(SelectionRefusal::CursorStale { as_of: 3, current: 4 }),
MindRefusal::Unavailable { detail } if detail.contains("cursor_stale")
));
let outside = SelectionRefusal::ReferenceOutsideTarget {
from_schema_id: "a".into(),
from_key: "k".into(),
role: "answers".into(),
to_schema_id: "b".into(),
to_key: "j".into(),
};
assert!(matches!(MindRefusal::from(outside), MindRefusal::Unavailable { .. }));
}
}

View file

@ -0,0 +1,412 @@
//! What the read side gives CultNet's selection evaluator: the rows, the
//! organ's closed vocabulary, and the door that checks the values a selection
//! carries. Types, rules and boundaries only.
//!
//! The substrate owns the grammar of a selection's names, the order, the
//! cursor, the hop and paging. This module owns what a row answers for each
//! declared alias, which aliases and roles exist, and the value domain of
//! every alias: `cultnet_rs::validate` refuses a name it does not know and
//! never looks at a value, so a malformed filter would read as an empty
//! answer. `refuse_values` is the door that closes that (S6).
use std::collections::BTreeMap;
use cultnet_rs::{Row, RowSet, Selection};
use epiphany_pipeline::{OrgRepo, Label, PipelineDocument, PipelineKind, PipelineRef, ResolutionOutcome, Slug};
use crate::docs::{CitationRole, Held, kind_of_id, kind_of_type};
use crate::query::PipelineDocumentView;
use crate::refusal::MindRefusal;
/// The declared index aliases: what a row answers `values(alias)` with.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Alias {
/// The key's root segment: a campaign slug, or an instance for
/// stewardships, hand-offs and the identity document. A resolution's root
/// is its subject's.
Root,
InForce,
Faculty,
/// `OrgRepo` identity spelling (lowercase), read through `base`.
Repo,
/// The label in `cut-<label>.`, read through `base`.
Cut,
Severity,
Confidence,
Origin,
Authority,
ClaimOutcome,
Outcome,
}
const FACULTIES: [&str; 7] = ["SelfFaculty", "Imagination", "Hands", "Soul", "MindSteward", "Eyes", "Operator"];
const SEVERITIES: [&str; 4] = ["Blocker", "High", "Medium", "Low"];
const CONFIDENCES: [&str; 2] = ["Confirmed", "Plausible"];
const ORIGINS: [&str; 2] = ["Introduced", "PreExisting"];
const AUTHORITIES: [&str; 3] = ["Operator", "Standing", "Defaulted"];
const CLAIM_OUTCOMES: [&str; 3] = ["Holds", "Falsified", "Unproven"];
const OUTCOMES: [&str; 6] = ["Superseded", "Answered", "Fixed", "Deferred", "Recorded", "Withdrawn"];
const BOOLEANS: [&str; 2] = ["true", "false"];
impl Alias {
const ALL: [Alias; 11] = [
Self::Root,
Self::InForce,
Self::Faculty,
Self::Repo,
Self::Cut,
Self::Severity,
Self::Confidence,
Self::Origin,
Self::Authority,
Self::ClaimOutcome,
Self::Outcome,
];
fn name(self) -> &'static str {
match self {
Self::Root => "root",
Self::InForce => "in_force",
Self::Faculty => "faculty",
Self::Repo => "repo",
Self::Cut => "cut",
Self::Severity => "severity",
Self::Confidence => "confidence",
Self::Origin => "origin",
Self::Authority => "authority",
Self::ClaimOutcome => "claim_outcome",
Self::Outcome => "outcome",
}
}
fn from_name(name: &str) -> Option<Self> {
Self::ALL.into_iter().find(|alias| alias.name() == name)
}
fn declared_on(self, kind: PipelineKind) -> bool {
use PipelineKind as K;
match self {
Self::Root | Self::InForce | Self::Faculty => true,
Self::Repo => {
matches!(kind, K::Campaign | K::CutSpec | K::CutReport | K::FollowUp | K::Stewardship | K::HandOff | K::Resolution)
}
Self::Cut => matches!(kind, K::CutSpec | K::CutReport | K::Verdict | K::Finding | K::Resolution),
Self::Severity | Self::Confidence | Self::Origin => kind == K::Finding,
Self::Authority => kind == K::Ruling,
Self::ClaimOutcome => kind == K::Verdict,
Self::Outcome => kind == K::Resolution,
}
}
/// The closed set an enum-valued alias draws from.
fn closed(self) -> Option<&'static [&'static str]> {
match self {
Self::InForce => Some(&BOOLEANS),
Self::Faculty => Some(&FACULTIES),
Self::Severity => Some(&SEVERITIES),
Self::Confidence => Some(&CONFIDENCES),
Self::Origin => Some(&ORIGINS),
Self::Authority => Some(&AUTHORITIES),
Self::ClaimOutcome => Some(&CLAIM_OUTCOMES),
Self::Outcome => Some(&OUTCOMES),
Self::Root | Self::Repo | Self::Cut => None,
}
}
/// Whether `value` is in this alias's domain: the leaf's own grammar for
/// the three grammar-valued aliases, exact spelling for the closed sets.
fn admits(self, value: &str) -> bool {
match self {
Self::Root => Slug(value.into()).validate_slug().is_ok(),
Self::Repo => OrgRepo(value.into()).validate_org_repo().is_ok(),
Self::Cut => Label(value.into()).validate_label().is_ok(),
closed => closed.closed().is_some_and(|set| set.contains(&value)),
}
}
}
/// The root and the local of a key, by position in `<root>:<kind>:<local>`:
/// the leaf derives every key that way and its own test pins the three
/// segments. The only place the read side reads a key apart.
pub(crate) fn root_and_local(key: &str) -> (&str, &str) {
let mut segments = key.split(':');
let root = segments.next().unwrap_or_default();
let local = segments.nth(1).unwrap_or_default();
(root, local)
}
fn outcome_name(outcome: &ResolutionOutcome) -> &'static str {
match outcome {
ResolutionOutcome::Superseded { .. } => "Superseded",
ResolutionOutcome::Answered { .. } => "Answered",
ResolutionOutcome::Fixed { .. } => "Fixed",
ResolutionOutcome::Deferred { .. } => "Deferred",
ResolutionOutcome::Recorded { .. } => "Recorded",
ResolutionOutcome::Withdrawn { .. } => "Withdrawn",
}
}
/// One document as the evaluator sees it: its identity, its ordinal, what it
/// answers for each alias, its citation edges, and the view a page projects.
#[derive(Clone, Debug)]
pub(crate) struct SelectionRow {
pub(crate) kind: PipelineKind,
pub(crate) key: String,
ordinal: u64,
values: BTreeMap<&'static str, Vec<String>>,
refs: Vec<(CitationRole, PipelineRef)>,
pub(crate) view: PipelineDocumentView,
}
impl SelectionRow {
/// `base` is the first non-resolution the document reaches through its
/// subjects: the row a resolution's `repo` and `cut` are read from.
pub(crate) fn new(
held: &Held,
base: &Held,
view: PipelineDocumentView,
refs: Vec<(CitationRole, PipelineRef)>,
) -> Self {
use PipelineDocument as D;
let mut values: BTreeMap<&'static str, Vec<String>> = BTreeMap::new();
let mut put = |alias: Alias, found: Vec<String>| {
values.insert(alias.name(), found);
};
put(Alias::Root, vec![root_and_local(&held.key).0.to_string()]);
put(Alias::InForce, vec![matches!(view.status, crate::query::PipelineStatus::InForce).to_string()]);
put(Alias::Faculty, vec![format!("{:?}", view.admission.provenance.faculty)]);
put(
Alias::Repo,
match &base.document {
D::Campaign(campaign) => campaign.repos.iter().map(OrgRepo::identity).collect(),
D::CutSpec(spec) => vec![spec.repo.identity()],
D::CutReport(report) => vec![report.repo.identity()],
D::FollowUp(follow_up) => vec![follow_up.repo.identity()],
D::Stewardship(stewardship) => vec![stewardship.repo.identity()],
D::HandOff(hand_off) => vec![hand_off.repo.identity()],
D::Target(_) | D::Question(_) | D::Ruling(_) | D::Verdict(_) | D::Finding(_) | D::Instance(_) | D::Resolution(_) => {
Vec::new()
}
},
);
put(
Alias::Cut,
root_and_local(&base.key)
.1
.strip_prefix("cut-")
.and_then(|rest| rest.split_once('.'))
.map(|(label, _)| label.to_string())
.into_iter()
.collect(),
);
match &held.document {
D::Finding(finding) => {
put(Alias::Severity, vec![format!("{:?}", finding.severity)]);
put(Alias::Confidence, vec![format!("{:?}", finding.confidence)]);
put(Alias::Origin, vec![format!("{:?}", finding.origin)]);
}
D::Ruling(ruling) => put(Alias::Authority, vec![format!("{:?}", ruling.authority)]),
D::Verdict(verdict) => {
put(Alias::ClaimOutcome, verdict.claims.iter().map(|claim| format!("{:?}", claim.outcome)).collect())
}
D::Resolution(resolution) => put(Alias::Outcome, vec![outcome_name(&resolution.outcome).to_string()]),
_ => {}
}
Self { kind: held.kind, key: held.key.clone(), ordinal: view.admission.ordinal, values, refs, view }
}
}
impl Row for SelectionRow {
fn schema_id(&self) -> &str {
self.kind.type_id()
}
fn record_key(&self) -> &str {
&self.key
}
fn ordinal(&self) -> i64 {
self.ordinal as i64
}
fn values(&self, index: &str) -> Vec<String> {
self.values.get(index).cloned().unwrap_or_default()
}
fn number(&self, _index: &str) -> Option<String> {
None
}
fn references(&self) -> Vec<(String, String, Option<Vec<u8>>)> {
self.refs.iter().map(|(role, target)| (role.name().to_string(), target.id.0.clone(), None)).collect()
}
}
/// The organ's closed lists: thirteen schemas, eleven aliases, fifteen roles.
/// A schema's name is its id, so the substrate's alias matching resolves
/// nothing beyond exact ids.
pub(crate) struct Vocabulary;
impl RowSet for Vocabulary {
fn all_schema_ids(&self) -> Vec<String> {
PipelineKind::ALL.iter().map(|kind| kind.type_id().to_string()).collect()
}
fn schema_name(&self, schema_id: &str) -> Option<String> {
kind_of_type(schema_id).map(|kind| kind.type_id().to_string())
}
fn declared_indexes(&self, schema_id: &str) -> Vec<String> {
let Some(kind) = kind_of_type(schema_id) else { return Vec::new() };
Alias::ALL.into_iter().filter(|alias| alias.declared_on(kind)).map(|alias| alias.name().to_string()).collect()
}
fn declared_roles(&self, schema_id: &str) -> Vec<String> {
let Some(kind) = kind_of_type(schema_id) else { return Vec::new() };
CitationRole::ALL.into_iter().filter(|role| role.carrier() == kind).map(|role| role.name().to_string()).collect()
}
fn is_numeric(&self, _schema_id: &str, _index: &str) -> bool {
false
}
fn target_leaves(&self, role: &str) -> Vec<String> {
CitationRole::from_name(role)
.map(|role| role.target_kinds().into_iter().map(|kind| kind.type_id().to_string()).collect())
.unwrap_or_default()
}
}
fn invalid(field: impl Into<String>, value: &str, message: impl Into<String>) -> MindRefusal {
MindRefusal::SelectionInvalid { field: field.into(), value: Some(value.to_string()), message: message.into() }
}
/// The organ's door (S6), run after `cultnet_rs::validate` has accepted the
/// selection's names: it refuses a value the vocabulary's domains do not hold,
/// so a malformed filter never reads as an empty answer. It also returns the
/// selection with every `repo` value in the identity spelling the rows carry,
/// because `OrgRepo` identity is case-insensitive.
///
/// Refuses, naming the field the client sent: a `schemas` entry that is not
/// one of the thirteen ids; a `keys` entry that is not a full id whose kind
/// segment reads and passes the leaf's ref grammar; an `any_of` value outside
/// its alias's domain; and a `cites.target` whose key fails the ref grammar
/// for its schema's kind (V24's rule, at this door). Whether the target
/// schema id is known is the substrate's refusal and is not repeated here.
pub(crate) fn refuse_values(selection: &Selection) -> Result<Selection, MindRefusal> {
for schema in selection.schemas.iter().flatten() {
if kind_of_type(schema).is_none() {
return Err(invalid("schemas", schema, "no such schema in this mind"));
}
}
for key in selection.keys.iter().flatten() {
let Some(kind) = kind_of_id(key) else {
return Err(invalid("keys", key, "not a pipeline id: the kind segment does not read"));
};
if let Err(refusal) = (PipelineRef { kind, id: key.as_str().into() }).validate_ref() {
return Err(invalid("keys", key, format!("not a pipeline id: {refusal:?}")));
}
}
let mut canonical = selection.clone();
for (position, predicate) in canonical.fields.iter_mut().flatten().enumerate() {
let Some(alias) = Alias::from_name(&predicate.index) else { continue };
for value in predicate.values.iter_mut().flatten() {
if !alias.admits(value) {
return Err(invalid(
format!("fields[{position}].values"),
value,
match alias.closed() {
Some(set) => format!("{} takes one of {}", alias.name(), set.join(", ")),
None => format!("not a valid {}", alias.name()),
},
));
}
if alias == Alias::Repo {
*value = OrgRepo(value.clone()).identity();
}
}
}
if let Some(cites) = &selection.cites
&& let Some(kind) = kind_of_type(&cites.target.schema_id)
&& let Err(refusal) = (PipelineRef { kind, id: cites.target.record_key.as_str().into() }).validate_ref()
{
return Err(invalid("cites.target", &cites.target.record_key, format!("not a {} id: {refusal:?}", kind.name())));
}
Ok(canonical)
}
#[cfg(test)]
mod tests {
use super::*;
use PipelineKind as K;
fn sorted(mut names: Vec<String>) -> Vec<String> {
names.sort();
names
}
fn strings(names: &[&str]) -> Vec<String> {
sorted(names.iter().map(|name| name.to_string()).collect())
}
/// The vocabulary's tables, read back through the substrate's trait: every
/// kind declares the three universal aliases and exactly the ones its
/// documents carry, and no alias is numeric, so a comparison is always
/// refused at the substrate's door.
#[test]
fn each_kind_declares_exactly_its_aliases_and_none_is_numeric() {
let table: [(K, &[&str]); 13] = [
(K::Campaign, &["repo"]),
(K::Target, &[]),
(K::Question, &[]),
(K::Ruling, &["authority"]),
(K::CutSpec, &["repo", "cut"]),
(K::CutReport, &["repo", "cut"]),
(K::Verdict, &["cut", "claim_outcome"]),
(K::Finding, &["cut", "severity", "confidence", "origin"]),
(K::FollowUp, &["repo"]),
(K::Resolution, &["repo", "cut", "outcome"]),
(K::Instance, &[]),
(K::Stewardship, &["repo"]),
(K::HandOff, &["repo"]),
];
for (kind, extra) in table {
let want = [&["root", "in_force", "faculty"][..], extra].concat();
assert_eq!(sorted(Vocabulary.declared_indexes(kind.type_id())), strings(&want), "{kind:?}");
for alias in Alias::ALL {
assert!(!Vocabulary.is_numeric(kind.type_id(), alias.name()), "{kind:?}/{}", alias.name());
}
}
}
/// Each role is declared by the one kind whose field carries it, and its
/// targets are the kinds that field is typed to, "any" spelled as all
/// thirteen.
#[test]
fn each_kind_declares_the_roles_it_carries_and_their_targets() {
let table: [(K, &[&str]); 13] = [
(K::Campaign, &[]),
(K::Target, &[]),
(K::Question, &["raised_in"]),
(K::Ruling, &["answers"]),
(K::CutSpec, &["rulings", "questions"]),
(K::CutReport, &["cut_spec", "forks"]),
(K::Verdict, &["cut_report", "findings"]),
(K::Finding, &["verdict"]),
(K::FollowUp, &["source"]),
(K::Resolution, &["subject", "superseded_by", "resolved_by", "deferred_to"]),
(K::Instance, &[]),
(K::Stewardship, &[]),
(K::HandOff, &["documents"]),
];
for (kind, roles) in table {
assert_eq!(sorted(Vocabulary.declared_roles(kind.type_id())), strings(roles), "{kind:?}");
}
assert_eq!(Vocabulary.target_leaves("answers"), vec![K::Question.type_id().to_string()]);
assert_eq!(Vocabulary.target_leaves("rulings"), vec![K::Ruling.type_id().to_string()]);
assert_eq!(Vocabulary.target_leaves("subject").len(), 13);
assert!(Vocabulary.target_leaves("nonsense").is_empty());
assert_eq!(Vocabulary.schema_name(K::Question.type_id()), Some(K::Question.type_id().to_string()));
assert_eq!(Vocabulary.schema_name("epiphany.pipeline.nonsense.v2"), None);
}
}

View file

@ -0,0 +1,228 @@
//! The store port. `cultcache-rs` exposes compare-and-swap as inherent methods
//! on its concrete stores, not on `CacheBackingStore`, so the one narrow trait
//! here is how the commit primitive gets it injected: the owned redb store in
//! the daemon, a memory store in every admission test, a refusing store in the
//! atomicity test. Nothing else is abstracted.
use anyhow::Result;
use cultcache_rs::CultCacheEnvelope;
/// The one store a daemon opens. Re-exported so the crate that serves a mind
/// names the store type without declaring `cultcache-rs` a second time: one
/// dependency entry, one rev, one place to move it.
pub use cultcache_rs::OwnedRedbMessagePackBackingStore;
/// CultCache's own row traits, re-exported for the same reason: the crate
/// that serves a mind reaches a store's rows through here rather than
/// declaring `cultcache-rs` a second time. The daemon's store-integrity tests
/// use it to break a row on purpose, over a real store, so a read over the
/// broken store raises the refusal its dispatch arm must carry — but that is
/// not the only door a row already has: `MindStore`'s own supertrait is
/// `CacheBackingStore`, and its public `compare_and_swap_batch` already lets
/// non-test code outside this crate write a row directly. Sealing the store
/// so admission is its only write path is a follow-up, its own cut; this
/// re-export does not widen anything that door does not already reach. It
/// carries two traits, `CacheBackingStore` and `DatabaseEntry`, plus
/// `DatabaseEntry`'s own derive macro, which shares its name in a different
/// namespace and rides along unnamed.
pub use cultcache_rs::{CacheBackingStore, DatabaseEntry};
/// A store a mind can be opened over. `Clone` shares ownership of the same
/// store (the owned redb store's clones share its lock and handle), which is
/// how the cache image and the commit path hold one store between them.
pub trait MindStore: CacheBackingStore + Clone + 'static {
/// One transactional step: every `expected` row must be stored exactly as
/// given, no `replacements` row outside `expected` may already exist, and
/// then every replacement lands, or nothing does and `Ok(false)` says so.
/// The store's own rule that every expected identity is also replaced is
/// kept: a strong read is re-inserted unchanged beside the writes.
fn compare_and_swap_batch(
&self,
expected: &[CultCacheEnvelope],
replacements: Vec<CultCacheEnvelope>,
) -> Result<bool>;
}
impl MindStore for OwnedRedbMessagePackBackingStore {
fn compare_and_swap_batch(
&self,
expected: &[CultCacheEnvelope],
replacements: Vec<CultCacheEnvelope>,
) -> Result<bool> {
OwnedRedbMessagePackBackingStore::compare_and_swap_batch(self, expected, replacements)
}
}
/// Test stores: a memory store with the redb store's batch semantics, and one
/// that can be told to lose or fail its next swap.
#[cfg(test)]
pub(crate) mod test_stores {
use std::collections::{BTreeMap, BTreeSet};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use anyhow::{Result, anyhow};
use cultcache_rs::{CacheBackingStore, CultCacheEnvelope, PushAllOptions};
use super::MindStore;
type Identity = (String, String);
fn identity(entry: &CultCacheEnvelope) -> Identity {
(entry.r#type.clone(), entry.key.clone())
}
fn unique(entries: &[CultCacheEnvelope], label: &str) -> Result<BTreeSet<Identity>> {
let ids = entries.iter().map(identity).collect::<BTreeSet<_>>();
if ids.len() != entries.len() {
return Err(anyhow!("conditional batch {label} set contains duplicate identities"));
}
Ok(ids)
}
/// One row per `(type, key)` behind a mutex; clones share it. Counts
/// `pull_all` calls so the opener test can prove nothing attached.
#[derive(Clone, Default)]
pub(crate) struct MemoryStore {
rows: Arc<Mutex<BTreeMap<Identity, CultCacheEnvelope>>>,
pulls: Arc<AtomicUsize>,
}
impl MemoryStore {
pub(crate) fn new() -> Self {
Self::default()
}
pub(crate) fn pull_count(&self) -> usize {
self.pulls.load(Ordering::SeqCst)
}
/// Every row, in identity order: the bytes a test compares before
/// and after a refused admission.
pub(crate) fn rows(&self) -> Vec<CultCacheEnvelope> {
self.rows.lock().unwrap().values().cloned().collect()
}
/// Writes a row without the commit path, for the opener tests and the
/// one replay-planting test.
pub(crate) fn plant(&self, entry: CultCacheEnvelope) {
self.rows.lock().unwrap().insert(identity(&entry), entry);
}
}
impl CacheBackingStore for MemoryStore {
fn pull_all(&self) -> Result<Vec<CultCacheEnvelope>> {
self.pulls.fetch_add(1, Ordering::SeqCst);
Ok(self.rows())
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.plant(entry.clone());
Ok(())
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.rows.lock().unwrap().remove(&identity(entry));
Ok(())
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], _options: PushAllOptions) -> Result<()> {
*self.rows.lock().unwrap() = entries.iter().map(|entry| (identity(entry), entry.clone())).collect();
Ok(())
}
}
impl MindStore for MemoryStore {
fn compare_and_swap_batch(
&self,
expected: &[CultCacheEnvelope],
replacements: Vec<CultCacheEnvelope>,
) -> Result<bool> {
if replacements.is_empty() {
return Err(anyhow!("conditional batch requires a non-empty replacement set"));
}
let expected_ids = unique(expected, "expected")?;
let replacement_ids = unique(&replacements, "replacement")?;
if !expected_ids.is_subset(&replacement_ids) {
return Err(anyhow!("conditional batch must replace every expected identity"));
}
let mut rows = self.rows.lock().unwrap();
for row in expected {
if rows.get(&identity(row)) != Some(row) {
return Ok(false);
}
}
for row in &replacements {
if !expected_ids.contains(&identity(row)) && rows.contains_key(&identity(row)) {
return Ok(false);
}
}
for row in replacements {
rows.insert(identity(&row), row);
}
Ok(true)
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum SwapCommand {
Delegate,
Lose,
Fail,
}
/// A memory store whose next swap can be commanded to lose (`Ok(false)`)
/// or fail (`Err`), so the commit path's two non-committing exits are
/// reachable without a second process.
#[derive(Clone)]
pub(crate) struct RefusingStore {
inner: MemoryStore,
command: Arc<Mutex<SwapCommand>>,
}
impl RefusingStore {
pub(crate) fn new() -> Self {
Self { inner: MemoryStore::new(), command: Arc::new(Mutex::new(SwapCommand::Delegate)) }
}
pub(crate) fn command(&self, command: SwapCommand) {
*self.command.lock().unwrap() = command;
}
pub(crate) fn rows(&self) -> Vec<CultCacheEnvelope> {
self.inner.rows()
}
}
impl CacheBackingStore for RefusingStore {
fn pull_all(&self) -> Result<Vec<CultCacheEnvelope>> {
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.inner.push(entry)
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<()> {
self.inner.push_all(entries, options)
}
}
impl MindStore for RefusingStore {
fn compare_and_swap_batch(
&self,
expected: &[CultCacheEnvelope],
replacements: Vec<CultCacheEnvelope>,
) -> Result<bool> {
match *self.command.lock().unwrap() {
SwapCommand::Delegate => self.inner.compare_and_swap_batch(expected, replacements),
SwapCommand::Lose => Ok(false),
SwapCommand::Fail => Err(anyhow!("the store refused the swap on command")),
}
}
}
}

View file

@ -0,0 +1,287 @@
//! The organ's request and response vocabulary: one operation per `Mind`
//! method and `whoami`. Payloads are the admission and read types unchanged.
//! The daemon carries these in CultNet's operation envelope; the client
//! constructs them. Nothing here validates a document, derives a status or
//! names a transport.
use cultcache_rs::DatabaseEntry;
use cultnet_rs::{CultMeshCdnArtifactManifest, Selection};
use epiphany_pipeline::{PIPELINE_SCHEMA_EPOCH, PipelineKind, PipelineRef, Slug};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use crate::admission::{PipelineAdmissionBatch, PipelineAdmissionOutcome};
use crate::mind::Mind;
use crate::query::{PipelineDocumentView, PipelineSelectionPage, SemanticQuery};
use crate::receipt::HuginnCommitReceipt;
use crate::refusal::MindRefusal;
use crate::store::MindStore;
/// The envelope's `service_id`: one service, one mind, whichever mind the
/// process opened.
pub const MIND_SERVICE_ID: &str = "huginn.mind";
pub const MIND_REQUEST_SCHEMA: &str = "huginn.mind_request.v1";
pub const MIND_RESPONSE_SCHEMA: &str = "huginn.mind_response.v1";
/// The largest answer that is deferred rather than refused, measured on the
/// encoded payload. The daemon refuses past it and a client caps its fetch at
/// it, so both sides read one number.
pub const MAX_DEFERRED_BODY_BYTES: u64 = 64 * 1024 * 1024;
/// The published JSON schema of each wire type, as the catalog advertises it.
/// The files are derived from the types below and pinned byte for byte by
/// `published_wire_schemas_match_derivation`, so the catalog cannot advertise
/// a shape the crate does not speak.
pub const MIND_REQUEST_SCHEMA_JSON: &str =
include_str!("../../../schemas/cultnet/huginn.mind_request.v1.schema.json");
pub const MIND_RESPONSE_SCHEMA_JSON: &str =
include_str!("../../../schemas/cultnet/huginn.mind_response.v1.schema.json");
/// The request's `selection` is CultNet's, and its published schema is
/// CultLib's: a `$ref` to the `$id` it publishes, so one owner holds the
/// selection's shape and nothing here copies it.
fn selection_ref(_: &mut schemars::SchemaGenerator) -> schemars::Schema {
schemars::json_schema!({
"$ref": "https://github.com/GameCult/CultLib/contracts/cultnet/cultnet.selection.schema.json"
})
}
/// The deferred answer's manifest is CultMesh's own type, a positional array
/// the C# reference defines (`CultMeshCdnArtifactManifest`, schema version
/// `gamecult.mesh.cdn_artifact_manifest.v1`). CultLib publishes no JSON schema
/// for it, so the schema says what the reference says: an array of its ten
/// fields in order, and nothing here copies their meaning.
fn manifest_array(_: &mut schemars::SchemaGenerator) -> schemars::Schema {
schemars::json_schema!({
"description": "gamecult.mesh.cdn_artifact_manifest.v1, the C# reference's CultMeshCdnArtifactManifest as its positional array: [artifactId, kind, version, contentHash, sizeBytes, mimeType, createdAtUtc, chunks[[chunkHash, offset, sizeBytes, recordKey]], tags, metadata].",
"type": "array"
})
}
/// Every operation a mind answers. `Admit` carries the batch whole because the
/// batch already names the instance and the asker; a second `instance` beside
/// it would be two declarations. Every read names the instance because ruling
/// 14 refuses another mind's identity whatever the transport, and a read of
/// the wrong mind is that same collision on the read side. `Whoami` names
/// none: it is how a client learns which mind it reached.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub enum HuginnMindRequest {
Whoami,
Admit(PipelineAdmissionBatch),
View { instance: Slug, id: PipelineRef },
Query {
instance: Slug,
#[schemars(schema_with = "selection_ref")]
selection: Selection,
semantic: Option<SemanticQuery>,
},
}
impl HuginnMindRequest {
/// The envelope's `operation`.
pub fn operation(&self) -> &'static str {
match self {
Self::Whoami => "whoami",
Self::Admit(_) => "admit",
Self::View { .. } => "view",
Self::Query { .. } => "query",
}
}
/// The instance the request declares; `Whoami` declares none.
pub fn instance(&self) -> Option<&Slug> {
match self {
Self::Whoami => None,
Self::Admit(batch) => Some(&batch.instance),
Self::View { instance, .. } | Self::Query { instance, .. } => Some(instance),
}
}
}
/// Every answer a mind gives. A refusal is data in the response, never a
/// transport failure: the read side carries `Refused`, the write side carries
/// the outcome's own `Refused`.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub enum HuginnMindResponse {
Whoami(MindStatus),
Admit(PipelineAdmissionOutcome),
View(Option<PipelineDocumentView>),
Query(PipelineSelectionPage),
Refused(MindRefusal),
/// The answer did not fit one send and is being held for the client to
/// fetch. Nothing in a mind produces this: the daemon's serve module does,
/// in place of an answer, and the answer it stands for is the named
/// MessagePack of the `HuginnMindResponse` the mind gave.
Deferred(DeferredAnswer),
}
/// Where the body of an answer that did not fit one send can be fetched: the
/// reference's own manifest, whose `contentHash` is the SHA-256 of the answer's
/// encoded bytes and whose chunks are asked for, one at a time, on the session
/// this arrived on. Row-level facts come from asking again with a header
/// projection; there is no summary.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct DeferredAnswer {
#[schemars(schema_with = "manifest_array")]
pub manifest: CultMeshCdnArtifactManifest,
}
impl HuginnMindResponse {
/// The envelope's `status`, in the C# reference's vocabulary: `rejected`
/// exactly for a refusal, `accepted` otherwise. `Conflict` and
/// `AlreadyAdmitted` are answers about the mind, not refusals of the
/// request. A `Deferred` is nominally `accepted`; the daemon sets the
/// envelope of a deferred answer to the status of the answer it stands for.
pub fn status(&self) -> &'static str {
match self {
Self::Refused(_) | Self::Admit(PipelineAdmissionOutcome::Refused(_)) => "rejected",
_ => "accepted",
}
}
}
/// How far the semantic index has caught up with the mind. The mind decides
/// none of it: the daemon's index worker reports it, in memory, and a restart
/// recomputes it. `pending` counts documents not yet known to be in the index.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub enum IndexStatus {
/// The worker has not yet compared the mind with the collection.
Reconciling { pending: u32 },
/// Every indexable document is in the collection.
Current,
/// Reconciled, and indexing what is still pending.
Behind { pending: u32 },
/// The last attempt failed (the embedder or the vector store is
/// unreachable, or refused); it is retried with a backoff and nothing is
/// dropped. `attempts` counts consecutive failures.
Failing { pending: u32, attempts: u32, error: String },
/// The collection belongs to another writer. The worker never recreates
/// it; it re-reads it on the same backoff.
Refused { pending: u32, attempts: u32, reason: String },
}
/// What a mind says about itself: the typed state a dashboard projects.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct MindStatus {
pub instance: Slug,
pub schema_epoch: String,
/// Pipeline documents in the image; the epoch record and the receipts are
/// not documents and are not counted here.
pub documents: u32,
pub receipts: u32,
pub index: IndexStatus,
}
impl<S: MindStore> Mind<S> {
/// `whoami`, derived from the image every time it is asked, and from the
/// index status the caller holds: the mind stores neither.
pub fn status(&self, index: IndexStatus) -> MindStatus {
let mut documents = 0_u32;
let mut receipts = 0_u32;
for envelope in self.envelopes() {
if envelope.r#type == HuginnCommitReceipt::TYPE {
receipts += 1;
} else if PipelineKind::ALL.iter().any(|kind| kind.type_id() == envelope.r#type) {
documents += 1;
}
}
MindStatus {
instance: self.instance().clone(),
schema_epoch: PIPELINE_SCHEMA_EPOCH.to_string(),
documents,
receipts,
index,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::query::PipelinePageItems;
use crate::fixtures::{CAMPAIGN, INSTANCE, OTHER_INSTANCE, instance, provenance, r, seeded, slug};
use crate::receipt::Faculty;
fn requests() -> Vec<HuginnMindRequest> {
let batch = PipelineAdmissionBatch {
instance: slug(INSTANCE),
provenance: provenance(Faculty::Hands),
documents: vec![instance(INSTANCE)],
};
vec![
HuginnMindRequest::Whoami,
HuginnMindRequest::Admit(batch),
HuginnMindRequest::View { instance: slug(INSTANCE), id: r(PipelineKind::Campaign, CAMPAIGN) },
HuginnMindRequest::Query { instance: slug(OTHER_INSTANCE), selection: Selection::default(), semantic: None },
]
}
fn responses() -> Vec<HuginnMindResponse> {
let refusal = MindRefusal::ForeignInstance { declared: OTHER_INSTANCE.into(), mind: INSTANCE.into() };
vec![
HuginnMindResponse::Whoami(seeded().status(IndexStatus::Current)),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::AlreadyAdmitted { receipt_id: "r1".into() }),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Conflict { identities: vec![] }),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(refusal.clone())),
HuginnMindResponse::View(None),
HuginnMindResponse::Query(PipelineSelectionPage {
matched: 0,
as_of: 1,
next: None,
items: PipelinePageItems::Headers(vec![]),
edges: None,
}),
HuginnMindResponse::Refused(refusal),
HuginnMindResponse::Deferred(DeferredAnswer {
manifest: cultnet_rs::pack_content("huginn.mind_response", "package", "", "", "", b"body", 4)
.unwrap()
.0,
}),
]
}
/// The vocabulary is the mind's methods, the declared instance is on every
/// request that names a mind, and the envelope's status is derived from
/// the response rather than chosen by whoever sends it.
#[test]
fn the_wire_vocabulary_is_the_minds_methods_and_status_is_derived_from_the_response() {
let names: Vec<&str> = requests().iter().map(|request| request.operation()).collect();
assert_eq!(names, ["whoami", "admit", "view", "query"]);
for request in requests() {
let declared = request.instance().is_none();
assert_eq!(declared, matches!(request, HuginnMindRequest::Whoami), "{request:?}");
let bytes = rmp_serde::to_vec_named(&request).unwrap();
assert_eq!(rmp_serde::from_slice::<HuginnMindRequest>(&bytes).unwrap(), request);
}
assert_eq!(
requests()[3].instance(),
Some(&slug(OTHER_INSTANCE)),
"the request's instance is the one it declares, not the mind's"
);
for response in responses() {
let rejected = matches!(
response,
HuginnMindResponse::Refused(_) | HuginnMindResponse::Admit(PipelineAdmissionOutcome::Refused(_))
);
assert_eq!(response.status(), if rejected { "rejected" } else { "accepted" }, "{response:?}");
let bytes = rmp_serde::to_vec_named(&response).unwrap();
assert_eq!(rmp_serde::from_slice::<HuginnMindResponse>(&bytes).unwrap(), response);
}
}
/// The two files the catalog publishes are the derivation of the two
/// types, byte for byte.
#[test]
fn published_wire_schemas_match_derivation() {
let request = serde_json::to_string_pretty(&schemars::schema_for!(HuginnMindRequest)).unwrap();
let response = serde_json::to_string_pretty(&schemars::schema_for!(HuginnMindResponse)).unwrap();
assert_eq!(MIND_REQUEST_SCHEMA_JSON.replace("\r\n", "\n"), request);
assert_eq!(MIND_RESPONSE_SCHEMA_JSON.replace("\r\n", "\n"), response);
// The selection is CultLib's, referenced by the `$id` it publishes and
// not copied into this schema.
let selection = "https://github.com/GameCult/CultLib/contracts/cultnet/cultnet.selection.schema.json";
assert!(request.contains(&format!("\"$ref\": \"{selection}\"")));
assert!(!request.contains("\"cursor\""), "no second copy of the selection's shape");
}
}

3
dist/cli.d.ts vendored
View file

@ -1,3 +0,0 @@
#!/usr/bin/env node
export {};
//# sourceMappingURL=cli.d.ts.map

1
dist/cli.d.ts.map vendored
View file

@ -1 +0,0 @@
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":""}

21
dist/cli.js vendored
View file

@ -1,21 +0,0 @@
#!/usr/bin/env node
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const promises_1 = require("node:fs/promises");
const inspection_1 = require("cultcache-ts/inspection");
const huginn_eve_dsl_1 = require("./huginn-eve-dsl");
async function main(args) {
const filePath = args[0];
if (!filePath || args.includes("--help") || args.includes("-h")) {
process.stderr.write("Usage: huginn <state.cc|state.msgpack|state.mpack>\n");
process.exitCode = filePath ? 0 : 1;
return;
}
const bytes = await (0, promises_1.readFile)(filePath);
const inspection = (0, inspection_1.inspectCultCacheBytes)(filePath, bytes);
process.stdout.write(`${(0, huginn_eve_dsl_1.buildHuginnEveDsl)(inspection)}\n`);
}
void main(process.argv.slice(2)).catch((error) => {
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
process.exitCode = 1;
});

View file

@ -1,3 +0,0 @@
import type { CultCacheInspection } from "cultcache-ts/inspection";
export declare function buildHuginnEveDsl(inspection: CultCacheInspection): string;
//# sourceMappingURL=huginn-eve-dsl.d.ts.map

View file

@ -1 +0,0 @@
{"version":3,"file":"huginn-eve-dsl.d.ts","sourceRoot":"","sources":["../src/huginn-eve-dsl.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,mBAAmB,EAA0C,MAAM,yBAAyB,CAAC;AAE3G,wBAAgB,iBAAiB,CAAC,UAAU,EAAE,mBAAmB,GAAG,MAAM,CA8BzE"}

View file

@ -1,55 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.buildHuginnEveDsl = buildHuginnEveDsl;
function buildHuginnEveDsl(inspection) {
return [
"# Huginn emits Eve DSL. Renderers compile and lower it.",
`surface cultcache.huginn.inspector ${quoteDsl("Huginn .cc Inspector")}`,
"version 1",
"",
"collection huginn.catalog.entries",
...inspection.catalog.map((entry) => `item huginn.catalog.entries ${quoteDsl(renderCatalogEntry(entry))}`),
"",
"collection huginn.records.entries",
...inspection.records.map((record) => `item huginn.records.entries ${quoteDsl(renderRecord(record))}`),
"",
"card source \"Source Witness\"",
`text ${quoteDsl(`file: ${inspection.filePath}`)}`,
`text ${quoteDsl(`format: ${inspection.format}`)}`,
`text ${quoteDsl(`bytes: ${inspection.fileSizeBytes}`)}`,
"text \".cc bytes are authority; Huginn only emits Eve DSL.\"",
"end",
"",
"card catalog \"Schema Catalog\"",
`text ${quoteDsl(`schemas: ${inspection.catalog.length}`)}`,
"list \"Catalog Entries\" bind huginn.catalog.entries",
"end",
"",
"card records \"Persisted Records\"",
`text ${quoteDsl(`records: ${inspection.records.length}`)}`,
"list \"Record Entries\" bind huginn.records.entries",
"end",
"",
].join("\n");
}
function renderCatalogEntry(entry) {
return [
entry.schemaName,
entry.schemaVersion,
entry.schemaId,
`${entry.members.length} members`,
].join(" | ");
}
function renderRecord(record) {
const status = record.payloadDecodeError ? `decode error: ${record.payloadDecodeError}` : "decoded";
return [
record.key,
record.schemaName,
record.storedAt,
`${record.payloadBytes} bytes`,
status,
].join(" | ");
}
function quoteDsl(value) {
return JSON.stringify(value);
}

4
dist/index.d.ts vendored
View file

@ -1,4 +0,0 @@
export { buildHuginnEveDsl } from "./huginn-eve-dsl";
export { inspectCultCacheBytes } from "cultcache-ts/inspection";
export type { CultCacheInspection, InspectedCatalogEntry, InspectedCatalogMember, InspectedRecord, } from "cultcache-ts/inspection";
//# sourceMappingURL=index.d.ts.map

1
dist/index.d.ts.map vendored
View file

@ -1 +0,0 @@
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,iBAAiB,EAAE,MAAM,kBAAkB,CAAC;AACrD,OAAO,EAAE,qBAAqB,EAAE,MAAM,yBAAyB,CAAC;AAChE,YAAY,EACV,mBAAmB,EACnB,qBAAqB,EACrB,sBAAsB,EACtB,eAAe,GAChB,MAAM,yBAAyB,CAAC"}

7
dist/index.js vendored
View file

@ -1,7 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.inspectCultCacheBytes = exports.buildHuginnEveDsl = void 0;
var huginn_eve_dsl_1 = require("./huginn-eve-dsl");
Object.defineProperty(exports, "buildHuginnEveDsl", { enumerable: true, get: function () { return huginn_eve_dsl_1.buildHuginnEveDsl; } });
var inspection_1 = require("cultcache-ts/inspection");
Object.defineProperty(exports, "inspectCultCacheBytes", { enumerable: true, get: function () { return inspection_1.inspectCultCacheBytes; } });

70
package-lock.json generated
View file

@ -1,70 +0,0 @@
{
"name": "@gamecult/huginn",
"version": "0.13.5",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@gamecult/huginn",
"version": "0.13.5",
"license": "MIT",
"dependencies": {
"cultcache-ts": "file:../CultLib/packages/cultcache-ts"
},
"bin": {
"huginn": "dist/cli.js"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "^5.9.0"
}
},
"../CultLib/packages/cultcache-ts": {
"version": "0.13.5",
"license": "MIT",
"dependencies": {
"@msgpack/msgpack": "^3.1.3",
"zod": "^3.25.76"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "^5.9.0"
}
},
"node_modules/@types/node": {
"version": "24.12.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.2.tgz",
"integrity": "sha512-A1sre26ke7HDIuY/M23nd9gfB+nrmhtYyMINbjI1zHJxYteKR6qSMX56FsmjMcDb3SMcjJg5BiRRgOCC/yBD0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.16.0"
}
},
"node_modules/cultcache-ts": {
"resolved": "../CultLib/packages/cultcache-ts",
"link": true
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "7.16.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
"integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
"dev": true,
"license": "MIT"
}
}
}

View file

@ -1,35 +0,0 @@
{
"name": "@gamecult/huginn",
"version": "0.13.5",
"description": "Huginn .cc inspection and Eve projection surface for CultCache state.",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"bin": {
"huginn": "dist/cli.js"
},
"files": [
"dist"
],
"scripts": {
"build": "tsc -p tsconfig.json",
"typecheck": "tsc -p tsconfig.json --pretty false",
"test": "npm run build"
},
"keywords": [
"messagepack",
"schema",
"cache",
"persistence",
"zod",
"typescript"
],
"author": "metacratic",
"license": "MIT",
"dependencies": {
"cultcache-ts": "file:../CultLib/packages/cultcache-ts"
},
"devDependencies": {
"@types/node": "^24.0.0",
"typescript": "^5.9.0"
}
}

11
schemas/cultnet/README.md Normal file
View file

@ -0,0 +1,11 @@
# Huginn's CultNet contracts
`huginn.mind_request.v1` and `huginn.mind_response.v1` are the organ's own wire
schemas, published from here because Huginn owns them. `huginn-daemon` serves
them on a `cultnet.schema_catalog_request.v0`; nothing else reads them.
Each file is `serde_json::to_string_pretty` of `schemars::schema_for!` over the
type of the same name in `crates/huginn-mind/src/wire.rs`. Regenerate by hand
and the test refuses it:
`cargo test -p huginn-mind --lib -- --exact wire::tests::published_wire_schemas_match_derivation`
compares file to derivation byte for byte.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -1,23 +0,0 @@
#!/usr/bin/env node
import { readFile } from "node:fs/promises";
import { inspectCultCacheBytes } from "cultcache-ts/inspection";
import { buildHuginnEveDsl } from "./huginn-eve-dsl";
async function main(args: string[]): Promise<void> {
const filePath = args[0];
if (!filePath || args.includes("--help") || args.includes("-h")) {
process.stderr.write("Usage: huginn <state.cc|state.msgpack|state.mpack>\n");
process.exitCode = filePath ? 0 : 1;
return;
}
const bytes = await readFile(filePath);
const inspection = inspectCultCacheBytes(filePath, bytes);
process.stdout.write(`${buildHuginnEveDsl(inspection)}\n`);
}
void main(process.argv.slice(2)).catch((error: unknown) => {
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
process.exitCode = 1;
});

View file

@ -1,57 +0,0 @@
import type { CultCacheInspection, InspectedCatalogEntry, InspectedRecord } from "cultcache-ts/inspection";
export function buildHuginnEveDsl(inspection: CultCacheInspection): string {
return [
"# Huginn emits Eve DSL. Renderers compile and lower it.",
`surface cultcache.huginn.inspector ${quoteDsl("Huginn .cc Inspector")}`,
"version 1",
"",
"collection huginn.catalog.entries",
...inspection.catalog.map((entry) => `item huginn.catalog.entries ${quoteDsl(renderCatalogEntry(entry))}`),
"",
"collection huginn.records.entries",
...inspection.records.map((record) => `item huginn.records.entries ${quoteDsl(renderRecord(record))}`),
"",
"card source \"Source Witness\"",
`text ${quoteDsl(`file: ${inspection.filePath}`)}`,
`text ${quoteDsl(`format: ${inspection.format}`)}`,
`text ${quoteDsl(`bytes: ${inspection.fileSizeBytes}`)}`,
"text \".cc bytes are authority; Huginn only emits Eve DSL.\"",
"end",
"",
"card catalog \"Schema Catalog\"",
`text ${quoteDsl(`schemas: ${inspection.catalog.length}`)}`,
"list \"Catalog Entries\" bind huginn.catalog.entries",
"end",
"",
"card records \"Persisted Records\"",
`text ${quoteDsl(`records: ${inspection.records.length}`)}`,
"list \"Record Entries\" bind huginn.records.entries",
"end",
"",
].join("\n");
}
function renderCatalogEntry(entry: InspectedCatalogEntry): string {
return [
entry.schemaName,
entry.schemaVersion,
entry.schemaId,
`${entry.members.length} members`,
].join(" | ");
}
function renderRecord(record: InspectedRecord): string {
const status = record.payloadDecodeError ? `decode error: ${record.payloadDecodeError}` : "decoded";
return [
record.key,
record.schemaName,
record.storedAt,
`${record.payloadBytes} bytes`,
status,
].join(" | ");
}
function quoteDsl(value: string): string {
return JSON.stringify(value);
}

View file

@ -1,8 +0,0 @@
export { buildHuginnEveDsl } from "./huginn-eve-dsl";
export { inspectCultCacheBytes } from "cultcache-ts/inspection";
export type {
CultCacheInspection,
InspectedCatalogEntry,
InspectedCatalogMember,
InspectedRecord,
} from "cultcache-ts/inspection";

View file

@ -1,26 +0,0 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "Node16",
"moduleResolution": "Node16",
"lib": [
"ES2022"
],
"strict": true,
"declaration": true,
"declarationMap": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"rootDir": "src",
"outDir": "dist"
},
"include": [
"src/index.ts",
"src/huginn-eve-dsl.ts",
"src/cli.ts"
],
"exclude": [
"test/**/*.ts"
]
}