Compare commits

...
Sign in to create a new pull request.
Author SHA1 Message Date
51fc7c3576 Merge pull request 'Speech binds to her words; Huginn's chat Interpreter; she speaks only in the turn's room' (#82) from hands/speech-binding into main 2026-11-20 05:27:32 +00:00
Meta Vens
7ed31c6f75 Test prose speaks in standalone quotations: a dialogue tag frames its quotation (Soul F1)
"what a day" he says and "Missed you too, friend," he types are framed under F1, so the test doubles' lines went silent. The prose now quotes whole sentences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:25:23 +01:00
Meta Vens
0f3f76a8ab Simulation: one turn a round per Mind, in the room of her earliest waking line; only the room she read is seen (Soul F5, replacing R7)
Who wakes is decided from the room as the round opens, as one heartbeat poll decides from the pending it read, so a line spoken this round counts toward a cap from the next round. A room that woke her but was not read stays unseen and wakes her next round. Tests: a DM and a #minds address answered in rounds 1 and 2; Soul's three rooms answered in three rounds, one turn a round, under ruled and names_with_cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:23:19 +01:00
Meta Vens
be87931b53 Person keys: a Mind's full Matrix id is her handle and the operator's is operator, everywhere a key is read; a value carrying a mark is refused (Soul F6, F7)
People::person_of is the one key reader for senders, targets, about: tags and bond objects. A value has no mark in the schema; one that carries one is not rendered. Tests: both spellings of a Mind behave alike for target and about:, a bond keyed by full ids is found, a marked value never renders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:23:18 +01:00
Meta Vens
484d4f65ba Speech: no line starts or ends inside a quotation her sentence frames; the private word match is deleted; an unclosed opening mark is not part of its sentence (Soul F1, F2, F4)
- A quotation is standalone when it, marks included, is a run of whole top-level sentences; any other is framed, and a line may not start or end strictly inside it. A line holding a framed quotation whole stays sayable.
- The three-word refusal against private items is deleted: whether a line tells a confidence is meaning, the Interpreter's judgement over the private list it is shown.
- An opening mark left unclosed (the budget cut her prose) is stripped from the sentence it starts, so the sentence is citable and the line carries no stray mark.
- Tests: every F1 probe refused (stray inch mark and scare-quote plan included); Weksa's hedge and a whole scare-quote sentence said; Weksa r6 and r10 said; the Q5d test keeps the prompt list and shows no word match refuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:23:18 +01:00
Meta Vens
8881b3f4b5 Simulation: one turn per room that woke her, each in its own room (Soul R7)
woken() returns one wake per room; every turn of hers in a round reads from where the round found her, so a direct message is never marked seen without a turn. Test: a DM and a #minds address in one round give Libby a turn in each, each reading only its own room's line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:01:11 +01:00
Meta Vens
f7decde001 Speech is whole consecutive sentences of her prose; private items are named to the Interpreter and refused in code; echo needs 8 words and 80% overlap (Soul R1, R4, R5, R8, R9)
- Every cited span starts and ends on a UAX#29 sentence boundary; the spans of one line are consecutive sentences, and the line is that unbroken run of her prose. A quotation's inside is segmented as a text of its own, so quoted speech stays sayable. She cannot say half a sentence.
- The prompt lists what she knows privately and allows plain speech of a private matter only to its person, in a DM with exactly them. Code refuses a line that shares a run of three words with a listed item.
- Tests: Soul's Q1a-Q1d refused, Q3d refused on a beat, Q5d refused in a DM with Libby, R9 said, P3 still refused; Hands' "Fine," + "winter" test rewritten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:01:11 +01:00
Meta Vens
56c45967ef Privacy has one owner: concerns() reads who a private item is about, scope() decides how it renders, for every pool and the bonds (Soul R2, R3, R6, R8, R10)
- concerns(): private is extensions.privacy, a private tag, or a room:! tag; it is about target/object person, else an about: tag, else the DM's partner, else everyone. private_elsewhere is deleted.
- scope(): plain only in a DM whose members are exactly her and that person (a partner-only item: in its own DM, still two; an everyone item: her own room); left out where the person is in the room, a group DM included; framed elsewhere.
- Applied to memories, shortTerm, incubation, needs, doctrine stances and bonds; history.said in a DM is private about its partner.
- Packet.private carries the framed items' words for the Interpreter; a beat's bodies are the tail lines its stimulus quoted.
- Tests: Soul's Q5a-Q5c, the DM-memory test rewritten for the room-tag rule, the said pool; the golden packet moves because dm-secret is now framed in public.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 06:01:11 +01:00
Meta Vens
947d8f7a40 Scoped privacy: only a bond or memory marked private is framed or left out; unmarked renders as written
Self's decision: the blanket framing smothered relationship texture in public rooms. A bond or memory is private only
when marked (`extensions.privacy = "private"`, or a memory's `private` tag); the schema is unchanged. A marked item
renders as written in a direct message with the person it is about, is left out about someone present in any other
room, and is framed "she knows privately" about someone absent. Unmarked items render as written everywhere. The
Interpreter's rule about private knowledge stays. The convention is documented in the Persona State Standard
(gamecult-ops `docs/persona-state-standard.md`, GameCult/gamecult-ops#129).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 05:34:11 +01:00
Meta Vens
02e78e1427 Speech binds to her words: a spoken line is her prose verbatim; Huginn's chat Interpreter prompt; she speaks only in the turn's room
Soul's hold on #81 and Self's privacy decision.

- F1: a spoken line is exactly the spans of her prose the Interpreter cites, located in order by Ghostlight's
  SourceSpan::locate and joined with one space. The model never writes the words; `text` is gone from the contract.
  The Persona writes the words she would type, never a description of them. Given up: indirect speech and any
  paraphrase by the Interpreter.
- F3: Huginn owns a short chat Interpreter prompt (what, where, to whom; gaps conditional, addressee, elsewhere);
  Ghostlight's membrane and the INTERPRETER_GUIDANCE patch are gone. `[interpreter]` keeps only the model.
- F2: an echo is a line of four words or more whose word trigrams are mostly the heard message bodies
  (heard_overlap); short answers are never echoes.
- F4: mentions and reply targets only for people this packet showed her.
- F5: a line said once; at most two lines a turn.
- F6: stale docs on the Mind run, the exchange and the policies.
- F7: a human's line ends a Mind's run only in a room the run has lines in.
- Privacy: she speaks only in the turn's room; the Interpreter is told the room, its kind (public, her own, a direct
  message) and who is in it, and not to say in a room what she knows privately; the packet frames bond summaries and
  memories about a person as known privately everywhere but a direct message with that person.
- The simulation's turn is in the room of the line that woke her.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 05:30:00 +01:00
14be61a167 Merge pull request 'The Interpreter's speech (C3b): what she says is decided from her prose, not parsed from quotes' (#81) from hands/interpreter-speech into main 2026-11-20 04:05:32 +00:00
Meta Vens
ea38f5f17d The Interpreter's speech (C3b): what she says is decided from her prose, not parsed from quotes; Soul's F2 to F9
Speech (Soul F1): `projection::interpret` is the Interpreter's speech step, one tool-less structured call through the
connector under Ghostlight's Interpreter membrane, over her prose, her lived stream, the stimulus, the rooms she may
speak in and the people she may address by Matrix id. It proposes lines, each with a room, structured mentions and a
reply target, and the verbatim span of her prose it rests on. Code keeps a line only when the span locates in her prose
(Ghostlight's SourceSpan::locate, map 4.3), the room is hers this turn, and its words are not what she heard; it drops
mentions or a reply target naming anyone not shown, and records every refusal and drop. An answer that is not the
contract is a failure, never a silence. `--through interpreter`; the batch and the simulation run through it, and in
the simulation the only explicit addressees are its structured fields. `would_say`, `inner` and the quote parsing are
gone; the Persona stage writes her thought with no speech convention. Tuning gains `[interpreter]`.

- F2: she is in a room's exchange when she spoke there within `exchange_minutes` (default 30).
- F3: the Projector guidance names no duty, not even to deny one; "X is waiting on her" only when true.
- F4: the narration metric is a lexical overlap, labeled so.
- F5: a scripted direct message in one of the plan's rooms is refused.
- F6: the Mind-run cap counts one run per Mind across her rooms; LineFacts::default() is at the cap (fail-safe).
- F7: an unbonded sender is labeled by display name and full id, or the id: never a bare handle.
- F8: a room is named by the key the tail carries (alias, or a direct message's id); a public room's id is reported,
  not mapped, since no table in her body maps it.
- F9: group-state parts are sentences, the person waiting on her first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 04:54:37 +01:00
e817b67abd Merge pull request 'Harness fixes: one wake rule in the heartbeat, the heartbeat's beat, group state in the Projector' (#80) from hands/harness-fixes into main 2026-11-20 03:34:37 +00:00
Meta Vens
63606efddd Projector and Persona after the operator's reading; humans_pace per ruling; Soul's hold on #80
Projector (the operator: "Sounds like a Projector issue"):
- The group state is the conversation as it stands, in one room: who is talking with whom, a human's question to the
  room that others answered as covered (one nobody answered goes unmentioned), when a human last spoke, Mind lines in a
  row, and a human waiting on her by name. Never what she owes. Every count covers the window the line describes (S4);
  never several rooms folded together, never a direct message into a public room (S3). Default group_lines 12.
- Absent state is left out: no "mood: none recorded"; a bond, need, memory, stance, wondering or value without content
  is not rendered, and a bond without trust or tension says neither.
- Guidance: never retell what she heard (it reaches her verbatim), render what it stirs; never a debt; no options; leave
  out what the context does not carry; use pronouns as given. Metrics, not filters: heard_overlap (word trigrams shared
  with the heard lines) and offers_options (a narrow lexical flag), per render and per run.
- Pronouns: `[pronouns]` in the tuning by Matrix id, the swarm's people as the default, they/them for anyone else; the
  persona schema has no pronoun field for another person. A pronoun line names everyone the packet shows from the room.
  The packet speaks of her in her own pronouns throughout ("Said to him:") (S5).
- Bond lines put the kind after a dash and lists use semicolons: a label may hold a comma ("Meta, the operator").

Persona stage: continues her inner stream in the first person; speech only where her thought moves her to it, in double
quotes; others' words she recalls in single quotes; @handle tags someone. The report shows her inner prose apart from
what she would say.

humans_pace (ruling persona-only-humans-pace; the operator's words that Minds' explicit mentions wake too):
`HumansPace(cap)`: a human's line wakes her in her own room or a direct message always, and in a public room when it
addresses her or continues an exchange she is in (an exchange ends at a gap over exchange_gap_minutes, default 20); a
Mind's line wakes her only by a Matrix mention or a reply, in any room, while fewer than `cap` Mind lines in a row came
before it. Own rooms come from the plan: neither the heartbeat's table nor persona.env records ownership. Live is
unchanged. In the simulation a said line's @handle of a cast member stands in for a Matrix mention.

Soul's hold:
- S1: Config::new refuses an operator id that is also a persona's.
- S2: the simulator's beat asks heartbeat::addressed_by_member over the deliveries the mouth would give her.
- S3: --in given as a room id is read by its alias.
- S6: a scripted direct message names its recipient (`to`), is seen by sender and recipient alone, and wakes only her.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 04:23:06 +01:00
Meta Vens
b90c21e695 Harness fixes after Soul's hold on #79: one owner of the wake rule, the heartbeat's beat, group state in the Projector
- F1: `heartbeat::wakes(policy, addressee, delivery, speaker, mind_run)` is the one wake rule. The poll asks it with
  `WakePolicy::Live` (behaviour unchanged until `po-mind-wakes-mind` is answered); the simulator asks it with the plan's
  policy: `ruled`, `humans_pace`, `names_with_cap = N`, `all_messages`. The simulator reads each persona's addressee from
  the heartbeat's addressing table (stewarded project names included) when the plan names `heartbeat.cc`.
- F2: the beat falls on `beat_s` of simulated time and wakes one Persona, the winner of `tempo::pick`.
- F5: Mind-addressed evidence is counted per wake (its strongest evidence), not per line.
- F6: reply chains are per room and count Mind lines spoken in turns another Mind's line woke.
- F7: a failed call is a failure, apart from silence; silence rate is per woken turn that did not fail.
- F8: a test that another seed changes the order.
- F9: `--state` docs say P4 and P5 are not checked.
- Group state (`select.group_state`, default on): one packet line over the room's last `group_lines` lines: who spoke
  to whom, when a human last spoke, how many Mind lines in a row, lines to her, an open question to everyone. Inside the
  context budget, narrated by the Projector like every other line; golden fixtures for the four scenario rooms.
- Reports: wakes by cause and by round, per-room chains, failures, 18 representative lines, the explicit-addressee gap.
- Tuning: `persona.words` and `persona.guidance` get the upper bounds the Projector's knobs have.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 03:40:38 +01:00
fe723867f7 Merge pull request 'mind-persona project: the persona seed harness (--room, --through persona, --batch, --simulate)' (#79) from hands/persona-harness into main 2026-11-20 01:53:34 +00:00
Meta Vens
19e0ee5285 simulate: wakes per round split human-caused and Mind-addressed; Mind-addressed evidence by mention, id as text or name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 01:14:38 +01:00
Meta Vens
8d69d3bded mind-persona project: the persona seed harness (--room, --through persona, --batch, --simulate)
The harness judges candidate persona states before any Mind holds them, through the
same `project` the live turn will use: every case is one `project` call.

- `--room <fixture.toml>`: a mocked room in place of the mouth and the room tail:
  her id, rooms, messages (sender, display name, room, ago, body; new, tag,
  mentions, dm, follow_up, reply_to), a mocked now and a "last heard from" table.
  The fixture is a MouthReader, so addressing is still `address::is_for`. The live
  room filter that `--room` used to name is now `--in`.
- `--state` is checked as a put checks content: huginn-mind's P1 to P3 now live in
  one owner, `persona::validate_document`, which the put calls.
- `--through persona`: the Persona stage (C3b's first piece). One tool-less
  connector call under Ghostlight's persona membrane over identity, narration and
  the heard lines; prose capped at 180 words; would-say lines are the double-quoted
  spans; usage per call. Tuning gains `[persona]` (model, guidance, words).
- `--batch <plan> --out <md>`: states x scenarios x samples, one Markdown report
  with packets, narration, prose, would-say lines, word counts, distinct-2 and
  cross-state word similarity, and a total-usage footer.
- `--simulate <plan> --out <md>`: up to eight personas taking turns in one room.
  Wake is is_for plus the ruled set (addressed, any human in her rooms, the beat);
  `wake = "all_messages"` is the old loop as a control. Woken personas run in a
  seeded order; quoted lines join the room as from her id. State does not evolve
  (`state_after_turn` is the seam for C3b/C3c). Metrics: messages, silence,
  who addresses whom, Mind-to-Mind chains, lexical convergence, a work-talk
  heuristic, usage.
- With a state file and a fixture no daemon and no mouth is needed (bin test).
- Request ids take the wall clock, so a fixture's fixed now cannot replay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 01:13:51 +01:00
c4bd83b026 Merge pull request 'Projector (C3a): Select stage, Projector narration, persona-tuning.toml, mind-persona project' (#76) from hands/persona-projector into main 2026-11-20 00:04:05 +00:00
Meta Vens
c43385962f Projector: a table without her row says so, whether or not its age is known
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 00:34:03 +01:00
Meta Vens
a2c59462e0 mind-body public surface: Failed, and the heartbeat view's written_at
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 00:32:43 +01:00
Meta Vens
078cefc024 Projector: tests read the third pass's rules (quoted display names, the 60% floor, the part bounds); Soul's list is 69 forms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 00:31:01 +01:00
Meta Vens
f146be2e65 Projector after Soul's third pass: a glued-name boundary rule; one selection for both renderings of the messages to her; JSON escapes for bodies and display names; failures keep the report and their usage; a 60% floor and CJK words in the cap; addressing age; bounds under the connector frame
T1: address.rs names_whole_word: a name is glued only by a letter or digit, or
a single hyphen with one beyond it; every other character is a boundary, so
_Eureka_, Eureka/Libby, ok...Eureka? and Huginn.toml name her again and
mind-eureka does not. Soul's 72 forms pinned.
T2: said_to_her decides once; stimulus and heard both carry the count.
T4: Model::ask fails with Failed { reason, usage }; a failed render or judge
call is recorded with its usage and the report goes on.
T5: the sentence cut keeps at least 60% of the budget; 。!? end sentences;
CJK characters are words. T6/T7: spoken is a JSON string; a display name is
quoted, capped at 80 bytes and followed by its Matrix id. T8: the addressing
report gives the table's age and why it was not used. T9: own beat lines
excluded (tested); words, stimulus, item and the other parts bounded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 00:28:55 +01:00
Meta Vens
754f28c550 mind-body public surface: drop a stray runner line
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:50:50 +01:00
Meta Vens
a19ea126ed mind-body public surface: Model answers with its usage
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:49:13 +01:00
Meta Vens
432d76e994 Projector: an overrun ends on the last whole sentence inside the budget; every render and judge call reports the connector's usage
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:47:32 +01:00
Meta Vens
33b42226f5 Projector guard: "short-term" is ordinary English, its one kebab exemption; shortTerm and short_term are still caught
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
47f2e6c5d3 Projector: a section heading over nothing is left out
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
57a6556968 Projector: golden packet and public surface after the second pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
12b777e67f Projector: clippy; others counted when messages to her take the room; the e^-21 floor is a short-term item's
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
fd56ea1442 address: the backslash joint as a character literal
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
c2cf38904e Projector after Soul's second pass: is_for decides addressing; escaped bodies; a ceiling on messages to her; crystallized memories by relevance, not age; lens from what is held; full-id people; tighter tuning and guard
S1: to_her is address::is_for over the heartbeat's addressing row (own names
when there is none); a name inside a path or identifier no longer addresses
her, in address.rs itself. S2: room talk is headed Also in the room. S3:
addressed_bytes (8 KiB) bounds the messages to her, newest first; one larger
message is clipped between graphemes with its remainder counted. S4: one
escape (spoken) renders every body. S5: only short-term items decay; memories,
values and stances come forward by lexical relevance to what the room said;
the lens is drawn from included items. Lows: people keyed by full id except
the operator and the Minds; unbonded speakers shown with their id; tuning
refuses an empty model, a temperature outside 0.01..10, all-zero caps and an
overflowing identity; the guard folds kebab and run-together forms; a beat
respects small caps; project refuses without MIND_PERSONA_SELF_SENDER.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
f4f02b841c mind-body public surface: projection's read-only organs; recall gone
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
0522ded4c3 Projector after Soul: golden packet, clippy, testkit helpers orphaned by the recall cut
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
dd039e55b9 Projector after Soul: no Mind recall; messages to her always whole; sampled inclusion; effective tuning hash and validation; schema-derived guard; budget edges; read-only organs; grapheme clips
F1/F2: the Projector shows her persona state and her room; recall.rs and the
recall pool, recall_sorts and its cap are deleted. Reading the Mind is the
Interpreter's (C3b).
F3: messages to her go in whole and first; others fill what is left, with an
exact note; overflow past the cap is reported.
F4: candidates are included with probability 1 - exp(-score / inclusion_score);
caps are maxima. F5: the hash is of the effective tuning. F6: the guard reads
the persona schema and the tuning keys. F7: bounded fixed slots and relatedness
cue; the redundant final clip is gone. F8: tuning values are validated.
F9: project's organs are read-only traits. F10: clips land on graphemes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:50 +01:00
Meta Vens
fb5ea6b924 Projector: Decided leaves the default recall_sorts (her Mind files mission decisions there); default is Learned, Idea, Rapport
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
f363d3b230 recall test: distinct labels
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
12ec0a5aa5 Projector: recall tests for memories only; public surface carries recall_sorts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
97961f3c0e Projector: recall offers only persona-tuning.toml's recall_sorts (default Learned, Idea, Decided, Rapport), one item a packet
Recall now reads only memories of the cued sorts: follow-ups, rulings and the
open-item filter leave it, with the tests that pinned them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
ca0703cce8 Lock: ghostlight-persona-projection, unicode-segmentation and toml for the Projector
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
a7a0862640 recall: cut the old prompt's recall section (render, lane headings); the Select stage renders its own lines
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
Meta Vens
8d52a44883 Projector (C3a): Select stage, Projector narration, persona-tuning.toml, mind-persona project
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:43:49 +01:00
f20a1533c8 Merge pull request 'Body map after C1: the persona door and the fence are live' (#78) from modeling/body-map-after-c1 into main 2026-11-19 22:40:08 +00:00
Meta Vens
71024a0f28 Body map after C1: the persona door and the fence are live; huginn.service is huginn@eureka
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-19 23:39:50 +01:00
57b931fdc3 Merge pull request 'Persona door: a second listener that answers reads and her persona, never an admission (C1)' (#74) from hands/persona-door into main 2026-11-19 22:25:39 +00:00
Meta Vens
04ebc4b0c5 Read-back gate: HUGINN_MIND_INSTANCE names the store it opens (default eureka)
The upgrade gate opened instance eureka only, and Mind::open refuses a store
whose instance document names another Mind, so no other Mind's store could be
gated. gamecult-ops now gates every store before a release moves any daemon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 23:15:15 +01:00
5b1a1e2cf4 Merge pull request 'Body map after C2: the persona-only body' (#77) from modeling/body-map-after-c2 into main 2026-11-19 21:40:28 +00:00
Meta Vens
e011c2aa21 Body map after C2: the persona-only body, stale rows marked, READMEs corrected
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-19 22:40:02 +01:00
66b85c8b67 Merge pull request 'C2: cut the mission organ and every Persona-side Mind write' (#75) from hands/persona-only-subtraction into main 2026-11-19 21:35:59 +00:00
Meta Vens
8ebe5de822 self_run: drop the doc pointer to the deleted launch::run_label
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:31:15 +01:00
Meta Vens
e9a883f732 mind-body public surface: regenerated after the F3/F4 follow-ups
Removed: launch's run opener and its types, queue's queue and breaker,
heartbeat's Night; changed: heartbeat Config and HeartbeatState.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:30:50 +01:00
Meta Vens
8fe9fc351b Heartbeat: no night wake until C4; stale docs and empty tests (Soul F4, F6, F7)
F4: the night wake started a plain mind-persona@ unit and traced it as
Sleep and "asleep". Deleted the night branch, Night, Config.night, the
HEARTBEAT_SLEEP_UTC and HEARTBEAT_SLEEP_MINDS settings and
HeartbeatState.last_night_at (added by this PR, never deployed, so nothing
must decode it). A wake's trace entry is her slug. Reflection and
consolidation return in C4.

F6: deleted a_flag_overrides_the_environment (asserted exit 0 with the
config discarded) and the heartbeat test that compared two identical rigs.

F7: Cargo.toml no longer says the Persona drains her chat or that launch
opens runs; heartbeat.rs no longer says Persona units read its addressing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:30:50 +01:00
Meta Vens
6df3255330 Delete the run opener, the work queue and the breaker (C2 follow-up, Soul F3)
Map section 7 keeps no dormant switch in main: Self-run launching is parked
at parked/self-run-launch. open_and_launch was the only code that opened a
SelfRun run and nothing called it in production.

launch.rs keeps the systemd door (Launcher, SystemdLauncher, unit names,
the brake's stop) and the two reads self_run uses, holders and closure.
Deleted: open_and_launch, Ports, LaunchRequest, LaunchOutcome, Declined,
slot, START_GRACE_S, run_label, the dead-run closer and the breaker
question, with their tests.

queue.rs keeps MindPort, HuginnMind, the paged reads and on_hold, which
self_run's inbox reads. Deleted: queue, breaker, Breaker, CONSECUTIVE,
WINDOW, EMPTY_SHARE, live_claims, claims_of, with their tests.

control: Grant::for_test and the launch-gate tests went with the gate.
testkit: the helpers only those tests used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:30:50 +01:00
Meta Vens
3e6554ca5a Cargo.lock as cargo writes it for the baf4ecf7 pin (duplicate CultLib entries reordered)
Soul C1 F5: the lock from the pin merge was accepted by --locked but
rewritten by an unlocked cargo; this is cargo's own rewrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:19:45 +01:00
Meta Vens
cf27291471 Persona door: tests pin every read at the door and a semantic answer by its hub
Soul C1 F4: the source, chat, chat-window, activity and roots reads are
handed to the mind at the persona door (none refused NotThisDoor, each
answered the same at both doors); the semantic wire test also runs through
the persona door, so a held search answers by the persona hub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:19:15 +01:00
Meta Vens
7b9fa7d33f mind-body public surface: regenerated after the C2 subtraction
Taken from the test's printed full current surface. Two lines are main's
drift, not this cut (chat_client's Basis gained FollowUp, and
FOLLOW_UP_BASIS); the rest is the subtraction: 83 lines removed (persona's
reply and session organ, stream, sleep's inbox, tempo's tally and PID,
heartbeat's AGENT and HeartbeatView::latest) and 11 changed or added
(persona Outcome/carry/Config, heartbeat Config.beat_s, DEFAULT_BEAT_S,
HeartbeatState's turn and night maps, FileStore::latest, Organs without
host, Standing without self_permille).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:07:56 +01:00
Meta Vens
0fd12325bd Cut the mission organ and every Persona-side Mind write (C2)
Map: docs/persona-only-heartbeat-map.md (imagination/persona-only-heartbeat,
d861574), cut C2. Rulings: eureka-body:ruling:persona-only-heartbeat (she
reads the Mind and never writes it; no Self runs from the Persona).

Persona organ (persona.rs 3983 -> ~420 lines): deleted the live session
organ whole: TURN_INSTRUCTION, TURN_REMINDER, MESSAGES_HEADER, WORK_QUESTION,
ANSWER_MESSAGES, WAKE_ASK, WORK_TAIL; PersonaReply with intent, remember,
rule, settle, start_self_run, tempo vote and urgency, and reply_schema; the
doctrine and persona-JSON system prompt (read_doctrine, render_instructions,
campaigns); start_session, persona_args, PersonaInbox; act, decode_reply and
every admission (memories, rulings, settles, pulses, follow-ups, run closure);
start_self_run; the responsibilities, recent-runs and wear renderers;
render_input and render_tempo. carry reads the brakes and answers Dead or
NoOrgan; a start writes nothing and records no run. Kept per the map's keep
list for C3: Config (doctrine, image store and fetch fields cut with their
consumers), the brake read, the chat-addressing helpers, the tail renderer
and FileTrace; uncalled ones carry allow(dead_code) naming C3.

Night: sleep.rs is deleted. It was a Mind-writing session built on the
deleted prelude; C4 retargets its contract from history.
attachment.rs is deleted with the reply's save_attachment, its one caller.
chat_image's loader and recall.rs stay for C3 under allow(dead_code).

Heartbeat: no Mind writes. A wake is a plain unit start through the
Launcher (brake, then alive, then start); busy is the unit's own state.
Deleted the Self-run branch (claimable briefs), the self-ended wake, tempo
votes and declared urgency (standing_vote, pending_urgency, last_self_end)
and the run-based last_turn and last_sleep reads; HeartbeatState gains
last_turn_at and last_night_at (default slots, so a state written before
this cut still decodes). The beat is HEARTBEAT_BEAT_S, default 1800 s.
InstanceTrace.vote_beat_s stays as an always-none slot so heartbeat.cc's
older trace records still decode. HeartbeatView loses latest; the
timeline reads FileStore::latest.

tempo.rs: tally, fallback_beat and the PID controller and its sim are
deleted; pick and score stay, without self_permille.

Tests that pinned deleted behaviour are deleted with it; surviving ones are
kept, and two heartbeat tests pin the cut: a Mind with claimable briefs gets
a Persona wake and no Self run, and no poll admits anything to any mind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:06:11 +01:00
Meta Vens
609696f43e Park the divergence stream organ (C2, persona-only-divergence-parked)
Deletes stream.rs, its tests, the mind-stream binary and its integration
tests, and the dependencies only it used (ghostlight-persona-projection,
ureq, the model-connector daemon dev feature). The organ is kept at tag
parked/divergence-stream, whose note names the ruling's return condition.
A brake hold still stops mind-stream@ units a host has installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:03:17 +01:00
Meta Vens
e5d1984dc1 Persona door: startup binds it when asked; the desk test names no door
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:01:26 +01:00
Meta Vens
df3272d2f2 desk_door integration test: run takes the persona door's hub
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:01:26 +01:00
Meta Vens
af7315b98b Persona door: a second RUDP listener that answers reads and her persona, never an admission
HuginnMindRequest::reaches_persona_door classifies every request with no
wildcard; Daemon::handle_at refuses what the persona door does not answer
with MindRefusal::NotThisDoor before the mind is asked; --persona-bind binds
the second hub, and run serves both, a search answering by the door it came
in at. Map: docs/persona-only-heartbeat-map.md (d861574), cut C1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 22:01:26 +01:00
58a320c6ee Merge pull request 'Pin CultLib baf4ecf7 (cultcache-rs delete refuses like the put, #178)' (#73) from hands/cultlib-pin-baf4ecf7 into main 2026-11-19 21:00:05 +00:00
Meta Vens
0373175270 Pin CultLib baf4ecf7 (cultcache-rs delete refuses like the put, #178)
Huginn calls no CultCache::delete; the CacheBackingStore::delete impls are test fakes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-19 21:45:39 +01:00
917430ce99 Merge pull request 'Chat search: multi-room listing no longer advises a chunk-losing --after' (#71) from eureka/chat-listing-continuation into main 2026-11-17 03:46:25 +00:00
4b276dfebc Merge pull request 'admission: clippy 1.95.0 nonminimal_bool at admission.rs:674' (#72) from eureka/clippy-195-nonminimal-bool into main 2026-11-17 03:46:21 +00:00
556324017e ChatQuery docs: --after continues a listing within one room only
Soul found the wire comment (and the published schema carrying it) still told every
caller to narrow with the last hit's ts_end_ms + 1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 03:46:19 +00:00
c9a33f7ed5 admission: simplify the historic-ruling guard (clippy 1.95.0 nonminimal_bool)
The pinned 1.95.0 toolchain rejects the negated conjunction at admission.rs:674 under
-D warnings; the rewrite is its own suggestion and De Morgan-equal.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 03:33:34 +00:00
41fa789d6f chat search: a multi-room listing no longer advises an --after that loses chunks
A truncated listing printed --after <last.ts_end+1>. Listing is ordered by chunk
start and chunks of different rooms overlap in time, so the next page skipped any
chunk starting before that instant and ending before it. Inside one room chunks do
not overlap, so the advice stays; with several rooms (or none named) the CLI now
says to list one room at a time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 03:33:32 +00:00
416e385c97 Clippy-clean huginn-daemon and mind-body under -D warnings (#70) 2026-11-17 02:22:30 +00:00
a424c1c4d5 huginn-sources: every run takes the lock (chat-off included) (#68) 2026-11-17 02:02:25 +00:00
2ca3105322 Clear all clippy -D warnings errors in huginn-mind (#69) 2026-11-17 01:58:12 +00:00
07942c1a83 Chat ingest job: huginn-sources chat flags, Matrix reader, lock, chat-purge (#67) 2026-11-17 01:52:38 +00:00
64e88966ad Merge pull request 'Pin CultLib 75188fde (stamp hardening #175)' (#66) from hands/cultlib-pin-75188fde into main 2026-11-14 09:35:41 +00:00
211cfe5e82 SwarmWriter::open refuses a store holding a record it does not read
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-14 09:35:31 +00:00
0c2d1e2cab ControlWriter::open refuses a store holding a record it does not read
The CultLib pin makes a pull carry unregistered records instead of refusing; the writer guards like the heartbeat store does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-14 09:31:17 +00:00
477482b1a0 Pin CultLib 75188fde (stamp hardening #175; includes #171 #172 #174)
The workspace's one CultLib pin moves 8fc74c70 -> 75188fdefa0b4baf131dbfd94e77d06a965f0cc6
(cultcache-rs 0.2.0 -> 0.3.1, cultnet-rs 0.1.0 -> 0.3.0 in the lock; the lock drops
the crates the old revision alone needed and gains sha2/stacker for the new one).

Adaptations the new cultcache-rs forces:
- CacheBackingStore::push/push_all return the stamps written; the five test
  stores follow (store.rs, receipt.rs, read_tests.rs).
- A pull now carries a record no registered type claims instead of refusing it,
  so the heartbeat writer's open() checks foreign_records() and keeps its
  fail-closed refusal of an old initiative store.
- A cache refuses writes until it has pulled, and a zero-byte file is not a
  store: two test fixtures pull first / plant a real store.

Not deployed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-14 09:14:30 +00:00
db9605fbf2 Desk door: listing gate and account-id grammar at decode (PR B of cut-desk-feed-listing.r8; stacked on #62) (#63) 2026-11-14 02:15:20 +00:00
f09bc56e77 Chat: basis follow (conversation-following), counted by is_for (#65)
Wire string exactly `follow` (Bifrost #74); one constant FOLLOW_UP_BASIS; Soul pass at eeaaad9 PASS, finding fixed at c71be9d. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 12:24:20 +00:00
232764bf78 Persona saves a named attachment into her sandbox (BIFROST_CHAT_FETCH, off by default) (#64)
Specs cut-persona-save-attachment.r7; Soul s1, s2; flag off, not deployed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-12 02:07:13 +00:00
Meta Vens
ecf147dbd7 Merge hands/eureka-body-flush-keeps-unseen-revision: a flush spends no unseen revision (cut-flush-keeps-unseen-revision h1; Soul s1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 23:55:45 +01:00
Meta Vens
b721c3eac7 Merge hands/eureka-body-usage-per-turn: one usage line per turn (cut-usage-per-turn h1-h2; Soul s1-s2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 23:37:30 +01:00
Meta Vens
51a13b7e82 mind-body: only an answer to a revision line spends a waiting stale-send revision
A flush answer never showed her the revision, so spending it acked a batch she
had not answered. final_reply stays the send-basis switch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 23:28:19 +01:00
Meta Vens
8b18434282 mind-body: pin that a flush answer does not spend a stale-send revision she was never told
Fails at base: the flush acks delivery 1 and drops the revision.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 23:23:36 +01:00
Meta Vens
43c3b1c047 usage-per-turn: the capture reports both TTL cache writes (0), so its lines carry them
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 23:18:41 +01:00
Meta Vens
65642dcf3c usage-per-turn: model ids are field-safe and the cache-write TTL split is carried
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 23:18:41 +01:00
Meta Vens
b920f3f81f usage-per-turn: the captured error result carries result_index 0 and no usage
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 22:16:57 +01:00
Meta Vens
fbedfdce2f usage-per-turn: the three journal-exact tests expect the usage line after each result
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 22:13:16 +01:00
Meta Vens
e719aa3766 usage-per-turn: every result writes one usage line to the unit's journal, from its per-turn usage
usage_line reads model_usage_since_last_result (never the cumulative modelUsage);
write_result writes it after the result line, so Self runs and Persona turns share it.
Unreported usage says "not reported", never zeros.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 22:06:15 +01:00
160 changed files with 11568 additions and 22129 deletions

278
Cargo.lock generated
View file

@ -37,17 +37,6 @@ dependencies = [
"subtle",
]
[[package]]
name = "ahash"
version = "0.7.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891477e0c6a8957309ee5c45a6368af3ae14bb510732d2684ffa19af310920f9"
dependencies = [
"getrandom 0.2.17",
"once_cell",
"version_check",
]
[[package]]
name = "ahash"
version = "0.8.12"
@ -195,12 +184,6 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bitflags"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
[[package]]
name = "bitflags"
version = "2.13.2"
@ -367,24 +350,6 @@ dependencies = [
"cipher",
]
[[package]]
name = "cultcache-rs"
version = "0.2.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=8fc74c709444b884a2bd339e0c306bf9484d99d2#8fc74c709444b884a2bd339e0c306bf9484d99d2"
dependencies = [
"anyhow",
"chrono",
"cultcache-rs-derive 0.1.0",
"fs2",
"redb",
"rmp-serde",
"serde",
"serde_bytes",
"serde_json",
"uuid",
"windows-sys 0.59.0",
]
[[package]]
name = "cultcache-rs"
version = "0.3.1"
@ -392,7 +357,7 @@ source = "git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f111
dependencies = [
"anyhow",
"chrono",
"cultcache-rs-derive 0.2.0",
"cultcache-rs-derive 0.2.0 (git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f1115f97bf35fb00ab6d23)",
"fs2",
"redb",
"rmp-serde",
@ -406,13 +371,24 @@ dependencies = [
]
[[package]]
name = "cultcache-rs-derive"
version = "0.1.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=8fc74c709444b884a2bd339e0c306bf9484d99d2#8fc74c709444b884a2bd339e0c306bf9484d99d2"
name = "cultcache-rs"
version = "0.3.1"
source = "git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb#baf4ecf78fae634fafef956d3b85420162c074bb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
"anyhow",
"chrono",
"cultcache-rs-derive 0.2.0 (git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb)",
"fs2",
"redb",
"rmp-serde",
"serde",
"serde_bytes",
"serde_json",
"serde_stacker",
"sha2",
"stacker",
"uuid",
"windows-sys 0.59.0",
]
[[package]]
@ -425,20 +401,29 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "cultcache-rs-derive"
version = "0.2.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb#baf4ecf78fae634fafef956d3b85420162c074bb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "cultnet-rs"
version = "0.1.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=8fc74c709444b884a2bd339e0c306bf9484d99d2#8fc74c709444b884a2bd339e0c306bf9484d99d2"
version = "0.2.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f1115f97bf35fb00ab6d23#a4eae9056152335662f1115f97bf35fb00ab6d23"
dependencies = [
"aes-gcm",
"anyhow",
"base64",
"chrono",
"cultcache-rs 0.2.0",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f1115f97bf35fb00ab6d23)",
"ed25519-dalek",
"hmac",
"rand",
"reed-solomon-erasure",
"rmp-serde",
"rmpv",
"serde",
@ -453,16 +438,18 @@ dependencies = [
[[package]]
name = "cultnet-rs"
version = "0.2.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f1115f97bf35fb00ab6d23#a4eae9056152335662f1115f97bf35fb00ab6d23"
version = "0.3.0"
source = "git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb#baf4ecf78fae634fafef956d3b85420162c074bb"
dependencies = [
"aes-gcm",
"anyhow",
"base64",
"chrono",
"cultcache-rs 0.3.1",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb)",
"ed25519-dalek",
"fs2",
"hmac",
"libc",
"rand",
"rmp-serde",
"rmpv",
@ -625,7 +612,7 @@ version = "0.1.0"
dependencies = [
"anyhow",
"chrono",
"cultcache-rs 0.2.0",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb)",
"rmp-serde",
"schemars",
"serde",
@ -639,7 +626,7 @@ dependencies = [
"anyhow",
"chrono",
"clap",
"cultnet-rs 0.1.0",
"cultnet-rs 0.3.0",
"huginn-daemon",
"huginn-mind",
"rmcp",
@ -874,15 +861,6 @@ dependencies = [
"unicode-segmentation",
]
[[package]]
name = "hashbrown"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
dependencies = [
"ahash 0.7.8",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
@ -932,7 +910,7 @@ dependencies = [
"anyhow",
"base64",
"chrono",
"cultnet-rs 0.1.0",
"cultnet-rs 0.3.0",
"huginn-mind",
"rmp-serde",
"rustix",
@ -951,8 +929,8 @@ dependencies = [
"anyhow",
"base64",
"chrono",
"cultcache-rs 0.2.0",
"cultnet-rs 0.1.0",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb)",
"cultnet-rs 0.3.0",
"eureka-pipeline",
"jsonschema",
"rmp-serde",
@ -995,6 +973,16 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39"
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "inout"
version = "0.1.4"
@ -1004,15 +992,6 @@ dependencies = [
"generic-array",
]
[[package]]
name = "instant"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0242819d153cba4b4b05a5a8f2a7e9bbf97b6055b2a002b395c96b5ff3c0222"
dependencies = [
"cfg-if",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
@ -1042,7 +1021,7 @@ version = "0.58.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f622c574a29fb3034a0a90645afee194dfedc75fc124228801b6e77041075bd1"
dependencies = [
"ahash 0.8.12",
"ahash",
"bytecount",
"data-encoding",
"email_address",
@ -1079,7 +1058,7 @@ version = "0.58.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39b7670345e5ce87371339d13705dbb9276b7e3e934bbe1efa2c6734a1f65a96"
dependencies = [
"ahash 0.8.12",
"ahash",
"bytecount",
"fraction",
"getrandom 0.3.4",
@ -1095,12 +1074,6 @@ version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libm"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
@ -1122,15 +1095,6 @@ version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "lru"
version = "0.7.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e999beba7b6e8345721bd280141ed958096a2e4abdf74f67ff4ce49b4b54e47a"
dependencies = [
"hashbrown 0.12.3",
]
[[package]]
name = "memchr"
version = "2.8.3"
@ -1150,8 +1114,8 @@ dependencies = [
"anyhow",
"base64",
"chrono",
"cultcache-rs 0.2.0",
"cultnet-rs 0.1.0",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=baf4ecf78fae634fafef956d3b85420162c074bb)",
"cultnet-rs 0.3.0",
"eureka-pipeline",
"eureka-state",
"ghostlight-persona-projection",
@ -1167,8 +1131,9 @@ dependencies = [
"sha2",
"syn 2.0.119",
"tempfile",
"toml",
"unicode-general-category",
"ureq",
"unicode-segmentation",
"uuid",
]
@ -1179,7 +1144,7 @@ source = "git+https://eureka.gamecult.org/forge/GameCult/ModelConnector.git?rev=
dependencies = [
"aes-gcm",
"base64",
"cultcache-rs 0.3.1",
"cultcache-rs 0.3.1 (git+https://github.com/GameCult/CultLib.git?rev=a4eae9056152335662f1115f97bf35fb00ab6d23)",
"cultnet-rs 0.2.0",
"rand_core 0.6.4",
"rmp-serde",
@ -1304,17 +1269,6 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a80800c0488c3a21695ea981a54918fbb37abf04f4d0720c453632255e2ff0e"
[[package]]
name = "parking_lot"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d17b78036a60663b797adeaee46f5c9dfebb86948d1255007a1d6be0271ff99"
dependencies = [
"instant",
"lock_api",
"parking_lot_core 0.8.6",
]
[[package]]
name = "parking_lot"
version = "0.12.5"
@ -1322,21 +1276,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
dependencies = [
"lock_api",
"parking_lot_core 0.9.12",
]
[[package]]
name = "parking_lot_core"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60a2cfe6f0ad2bfc16aefa463b497d5c7a5ecd44a23efa72aa342d90177356dc"
dependencies = [
"cfg-if",
"instant",
"libc",
"redox_syscall 0.2.16",
"smallvec",
"winapi",
"parking_lot_core",
]
[[package]]
@ -1347,7 +1287,7 @@ checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
dependencies = [
"cfg-if",
"libc",
"redox_syscall 0.5.18",
"redox_syscall",
"smallvec",
"windows-link",
]
@ -1488,35 +1428,13 @@ dependencies = [
"libc",
]
[[package]]
name = "redox_syscall"
version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb5a58c1855b4b6819d59012155603f0b22ad30cad752600aadfcb695265519a"
dependencies = [
"bitflags 1.3.2",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [
"bitflags 2.13.2",
]
[[package]]
name = "reed-solomon-erasure"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7263373d500d4d4f505d43a2a662d475a894aa94503a1ee28e9188b5f3960d4f"
dependencies = [
"libm",
"lru",
"parking_lot 0.11.2",
"smallvec",
"spin",
"bitflags",
]
[[package]]
@ -1545,13 +1463,13 @@ version = "0.58.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "558f3855529c0a7e78a48e409fa653456e55f1b167eab4109c16a255b7727e3f"
dependencies = [
"ahash 0.8.12",
"ahash",
"fluent-uri",
"getrandom 0.3.4",
"hashbrown 0.17.1",
"hashbrown",
"itoa",
"micromap",
"parking_lot 0.12.5",
"parking_lot",
"percent-encoding",
"serde_json",
]
@ -1664,7 +1582,7 @@ version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags 2.13.2",
"bitflags",
"errno",
"libc",
"linux-raw-sys",
@ -1790,6 +1708,15 @@ dependencies = [
"serde_core",
]
[[package]]
name = "serde_spanned"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c187e0a369471c550af028cba56d4f4fa2a489ad28151ae95a4d340a2a8b782"
dependencies = [
"serde_core",
]
[[package]]
name = "serde_stacker"
version = "0.1.14"
@ -1875,12 +1802,6 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "spin"
version = "0.9.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e"
[[package]]
name = "spki"
version = "0.7.3"
@ -2034,6 +1955,45 @@ dependencies = [
"tokio",
]
[[package]]
name = "toml"
version = "0.9.12+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf92845e79fc2e2def6a5d828f0801e29a2f8acc037becc5ab08595c7d5e9863"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"toml_datetime",
"toml_parser",
"toml_writer",
"winnow 0.7.15",
]
[[package]]
name = "toml_datetime"
version = "0.7.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347"
dependencies = [
"serde_core",
]
[[package]]
name = "toml_parser"
version = "1.1.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baa693a8032d7e1cada7d0041e96126df243179ff061456783ac7f12bda4744c"
dependencies = [
"winnow 1.0.5",
]
[[package]]
name = "toml_writer"
version = "1.2.0+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "009dbb7700f11c55ef83aebbe4c0e9cf1f1ac0844bd8d6f00dc38f302026e13a"
[[package]]
name = "tracing"
version = "0.1.44"
@ -2411,6 +2371,18 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "winnow"
version = "0.7.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945"
[[package]]
name = "winnow"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c3b93199df0ab874cdf0c6a94f0a0dc3423fe41f1cafb62b8bd7b1deb277aef"
[[package]]
name = "wit-bindgen"
version = "0.57.1"

View file

@ -16,5 +16,5 @@ publish = false
# The one CultLib revision the workspace speaks. Every crate inherits these
# entries; a bump is a change to this table and nowhere else.
[workspace.dependencies]
cultcache-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultcache-rs" }
cultnet-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "8fc74c709444b884a2bd339e0c306bf9484d99d2", package = "cultnet-rs" }
cultcache-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "baf4ecf78fae634fafef956d3b85420162c074bb", package = "cultcache-rs" }
cultnet-rs = { git = "https://github.com/GameCult/CultLib.git", rev = "baf4ecf78fae634fafef956d3b85420162c074bb", package = "cultnet-rs" }

View file

@ -42,7 +42,7 @@ edits `.cc` state. Huginn reads and writes minds.
## Layout
A Cargo workspace of five crates (`crates/eureka-pipeline` holds the document shapes and keys, `crates/mind-body` the control store, launch primitive and `mind-heartbeat` timeline; the three below are the mind, the daemon and the client):
A Cargo workspace of five crates (`crates/eureka-pipeline` holds the document shapes and keys, `crates/mind-body` the control store, unit launcher, `mind-heartbeat`, the inert `mind-persona` organ and the `mind-self-run` actuator; the three below are the mind, the daemon and the client):
- `crates/huginn-mind`: storage, identity, admission, and queries and derived
status over memory documents.

View file

@ -32,7 +32,8 @@ sample.research eureka-state:research:p1.scope epiphany.pipeline.research.v2 Epi
sample.idea eureka-state:idea:p1.i1 epiphany.pipeline.idea.v2 EpiphanyPipelineIdeaDocument 9189a863616d706169676eac657572656b612d7374617465a470617373a27031a56c6162656ca26931a57469746c65b24f6e65206e6f74652070657220636c61696da469646561d9204b65657020612070617373206173206d616e7920736d616c6c206e6f7465732ea974726164656f666673b74d6f726520646f63756d656e747320746f206c6973742ea872657374735f6f6e9182a46b696e64a85265736561726368a26964be657572656b612d73746174653a72657365617263683a70312e73636f7065a6746f776172649182a46b696e64a85175657374696f6ea26964b8657572656b612d73746174653a7175657374696f6e3a5131ab70726f706f7365645f6f6eaa323032362d30392d3135
sample.reply yggdrasil:reply:mind-2026-10-09-1-r1 epiphany.pipeline.reply.v2 EpiphanyPipelineReplyDocument 9185a8696e7374616e6365a979676764726173696ca56c6162656cb46d696e642d323032362d31302d30392d312d7231a372756e82a46b696e64a352756ea26964bf79676764726173696c3a72756e3a6d696e642d323032362d31302d30392d31a7616e73776572739182a46b696e64a8466f6c6c6f775570a26964d923657572656b612d73746174653a666f6c6c6f775f75703a61736b2d7468652d73656c66a474657874d93054686520736861646572207061737320697320646f6e653b2074686520736f756e647320617265204c6962627927732e
sample.thought yggdrasil:thought:t1793000000 epiphany.pipeline.thought.v2 EpiphanyPipelineThoughtDocument 91de0014a8696e7374616e6365a979676764726173696ca56c6162656cab7431373933303030303030a474657874d93d54686520736f756e647320617265204c6962627927733b2074686520736861646572732077657265206e6576657220746865206861726420706172742ea572656164739182a46b696e64a64d656d6f7279a26964d92879676764726173696c3a6d656d6f72793a323032362d31302d3331743034313230302d74302d6d30af706572736f6e615f72656365697074b06d696e642d636f6d6d69742d30613162a86964656e74697479be457572656b612c207368652f6865722c20706c61696e2d73706f6b656e2ea7636f6e74657874d92056616c7565733a20636f686572656e6365206f7665722076656c6f636974792ea87374696d756c7573bd4c696262792061736b65642061626f75742074686520736f756e64732eaf70726f6a6563746f725f6d6f64656cb070726f6a6563746f723a6c6174657374aa70726f6a656374696f6ed92949206b656570207475726e696e67206f7665722077686f20686f6c64732074686520736f756e64732eaa626173655f6d6f64656cab626173653a6c6174657374b474656d70657261747572655f7065726d696c6c65cd03e8ae6d696e5f705f7065726d696c6c6532aa6d61785f746f6b656e73cca0aa63616e6469646174657392d93d54686520736f756e647320617265204c6962627927733b2074686520736861646572732077657265206e6576657220746865206861726420706172742eaf41207365636f6e642064726966742eb06e6561726573745f7065726d696c6c6592cd0136cd0366a663686f73656e00ac70726f6a6563746f725f6d73cd1068ac636f6e74696e7565725f6d73cd238cae70726f6d70745f76657273696f6ebd67686f73746c696768743d633433656235343b20776f7264733d313830
sample.desk_answer eureka-state:desk_answer:Q1.n1 epiphany.pipeline.desk_answer.v2 EpiphanyPipelineDeskAnswerDocument 918ea863616d706169676eac657572656b612d7374617465a4666f726bb8657572656b612d73746174653a7175657374696f6e3a5131a873657175656e636501ac6f7074696f6e735f68617368d94063636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363a663686f696365a161a46e6f7465b853686172652069742c207769746820746865206d61726b2eab616e7377657265645f6174b4323032362d31312d31305431353a30303a30305aab72656365697665645f6174b4323032362d31312d31305431353a30303a30335aaf636c61696d5f696e5f667574757265c2aa726571756573745f6964d92436663163306135322d336237652d346431612d396335382d306532643761346239663133ab626f64795f736861323536d94064646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464a8616e737765726572a6757365722d37a9617574686f72697479a64d656d626572a76f7574636f6d6581a85265766973696f6e81a26f66be657572656b612d73746174653a6465736b5f616e737765723a51312e6e31
sample.desk_answer eureka-state:desk_answer:Q1.n1 epiphany.pipeline.desk_answer.v2 EpiphanyPipelineDeskAnswerDocument 918ea863616d706169676eac657572656b612d7374617465a4666f726bb8657572656b612d73746174653a7175657374696f6e3a5131a873657175656e636501ac6f7074696f6e735f68617368d94063636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363636363a663686f696365a161a46e6f7465b853686172652069742c207769746820746865206d61726b2eab616e7377657265645f6174b4323032362d31312d31305431353a30303a30305aab72656365697665645f6174b4323032362d31312d31305431353a30303a30335aaf636c61696d5f696e5f667574757265c2aa726571756573745f6964d92436663163306135322d336237652d346431612d396335382d306532643761346239663133ab626f64795f736861323536d94064646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464646464a8616e737765726572d92435663063326231652d386134642d346333622d396537662d316132623363346435653666a9617574686f72697479a64d656d626572a76f7574636f6d6581a85265766973696f6e81a26f66be657572656b612d73746174653a6465736b5f616e737765723a51312e6e31
sample.desk_listing eureka-state:desk_listing:Q1 epiphany.pipeline.desk_listing.v2 EpiphanyPipelineDeskListingDocument 9183a863616d706169676eac657572656b612d7374617465a66c6973746564b8657572656b612d73746174653a7175657374696f6e3a5131a6726561736f6ed9324d656d62657273206d617920726561642077687920746865204d696e64206c6973747320697473206f776e20666f726b732e
resolution.superseded eureka-state:resolution:question.Q1.n1 epiphany.pipeline.resolution.v2 EpiphanyPipelineResolutionDocument 9185a77375626a65637482a46b696e64a85175657374696f6ea26964b8657572656b612d73746174653a7175657374696f6e3a5131a873657175656e636501a76f7574636f6d6581aa5375706572736564656481a262799282a46b696e64a652756c696e67a26964b6657572656b612d73746174653a72756c696e673a523882a46b696e64a85175657374696f6ea26964b8657572656b612d73746174653a7175657374696f6e3a5131a9726174696f6e616c65a852756c656420412eab7265736f6c7665645f6f6eaa323032362d30392d3135
resolution.fixed-by-none eureka-state:resolution:question.Q1.n1 epiphany.pipeline.resolution.v2 EpiphanyPipelineResolutionDocument 9185a77375626a65637482a46b696e64a85175657374696f6ea26964b8657572656b612d73746174653a7175657374696f6e3a5131a873657175656e636501a76f7574636f6d6581a5466978656482a6636f6d6d6974a83566393832323864a26279c0a9726174696f6e616c65a852756c656420412eab7265736f6c7665645f6f6eaa323032362d30392d3135
resolution.fixed-by-some eureka-state:resolution:question.Q1.n1 epiphany.pipeline.resolution.v2 EpiphanyPipelineResolutionDocument 9185a77375626a65637482a46b696e64a85175657374696f6ea26964b8657572656b612d73746174653a7175657374696f6e3a5131a873657175656e636501a76f7574636f6d6581a5466978656482a6636f6d6d6974a83566393832323864a2627982a46b696e64a94375745265706f7274a26964d921657572656b612d73746174653a6375745f7265706f72743a6375742d33612e6831a9726174696f6e616c65a852756c656420412eab7265736f6c7665645f6f6eaa323032362d30392d3135

View file

@ -316,6 +316,10 @@ bounded_text! {
Words = 4000, words_text;
/// A client's retry token: a lowercase D-format UUID.
RequestId = 36, uuid_text;
/// A Bifrost account id: a lowercase D-format UUID (`UserAccount.Id`). A Matrix id, an email or a
/// handle is never one.
#[schemars(extend("pattern" = "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"))]
AccountId = 36, uuid_text;
/// A key label: `[A-Za-z0-9_-]{1,64}`.
Label = 64, label_text;
/// A campaign slug or key segment: label parts joined by `.`.
@ -885,8 +889,13 @@ value_types! {
pub struct PipelineDeskAnswer {
campaign: Slug, fork: Short, sequence: u32, options_hash: Sha256Hex, choice: Option<Short>, note: Option<Words>,
answered_at: Short, received_at: Short, claim_in_future: bool, request_id: RequestId, body_sha256: Sha256Hex,
answerer: Short, authority: DeskAuthority, outcome: DeskOutcome,
answerer: AccountId, authority: DeskAuthority, outcome: DeskOutcome,
}
/// A fork the Mind lists: members who are not on a call's decider list may read the question
/// while this document is in force (unlist = a `Withdrawn` resolution on it; relist = withdraw
/// that). One per question, keyed by the question's label; the receipt says who listed it and
/// when, `reason` why members may read it. Admitted on the Mind's own door only, never the desk's.
pub struct PipelineDeskListing { campaign: Slug, listed: Short, reason: Line }
pub struct PipelineVerdict { campaign: Slug, cut_report: Short, pass: u32, range: CommitRange, claims: Vec<VerdictClaim>[64] }
pub struct PipelineFinding {
campaign: Slug, verdict: Short, label: Label, range: CommitRange, confidence: FindingConfidence,
@ -1425,6 +1434,16 @@ pipeline_kinds! {
"epiphany.pipeline.thought.v2", "EpiphanyPipelineThoughtDocument";
DeskAnswer(PipelineDeskAnswer) => EpiphanyPipelineDeskAnswerDocument, "desk_answer",
"epiphany.pipeline.desk_answer.v2", "EpiphanyPipelineDeskAnswerDocument";
DeskListing(PipelineDeskListing) => EpiphanyPipelineDeskListingDocument, "desk_listing",
"epiphany.pipeline.desk_listing.v2", "EpiphanyPipelineDeskListingDocument";
}
impl AccountId {
/// Whether this is a lowercase D-format UUID. For the callers outside the crate, which have no way
/// to reach `Bounded`; on the pattern of `Slug::validate_slug`.
pub fn validate_account(&self) -> Result<(), PipelineRefusal> {
self.validate("account")
}
}
/// Parses a full document id: the grammar read backwards. A key and an id are
@ -1516,7 +1535,7 @@ const SUBJECT_LOCAL_MAX: usize = 64;
/// The longest kind name a resolution can name as its subject (`stewardship`),
/// checked against `PipelineKind::ALL` by a test.
const LONGEST_KIND_NAME: usize = "stewardship".len();
const LONGEST_KIND_NAME: usize = "desk_listing".len();
/// A `.n<sequence>` part at `u32::MAX`.
const SEQUENCE_PART_MAX: usize = ".n".len() + "4294967295".len();
@ -1651,6 +1670,10 @@ pub fn pipeline_key(document: &PipelineDocument) -> Result<String, PipelineRefus
let sequence = format!("n{}", value.sequence);
(campaign_field.as_str(), value.campaign.0.as_str(), local(&key_field, document_kind, &[label, &sequence])?)
}
D::DeskListing(value) => {
let label = parent_local("desk_listing.listed", &value.listed.0, &value.campaign.0, PipelineKind::Question)?;
(campaign_field.as_str(), value.campaign.0.as_str(), local(&key_field, document_kind, &[label])?)
}
D::Verdict(value) => {
let report = parent_local("verdict.cut_report", &value.cut_report.0, &value.campaign.0, PipelineKind::CutReport)?;
let cut = parent_cut("verdict.cut_report", report, 'h')?;
@ -1927,9 +1950,12 @@ mod tests {
campaign: slug(CAMPAIGN), fork: id("question", "Q1"), sequence: 1, options_hash: Sha256Hex("c".repeat(64)),
choice: Some(s("a")), note: Some("Share it, with the mark.".into()), answered_at: s("2026-11-10T15:00:00Z"),
received_at: s("2026-11-10T15:00:03Z"), claim_in_future: false, request_id: RequestId("6f1c0a52-3b7e-4d1a-9c58-0e2d7a4b9f13".into()), body_sha256: Sha256Hex("d".repeat(64)),
answerer: s("user-7"), authority: DeskAuthority::Member,
answerer: AccountId("5f0c2b1e-8a4d-4c3b-9e7f-1a2b3c4d5e6f".into()), authority: DeskAuthority::Member,
outcome: DeskOutcome::Revision { of: id("desk_answer", "Q1.n1") },
}), format!("{CAMPAIGN}:desk_answer:Q1.n1")),
(D::DeskListing(PipelineDeskListing {
campaign: slug(CAMPAIGN), listed: id("question", "Q1"), reason: "Members may read why the Mind lists its own forks.".into(),
}), format!("{CAMPAIGN}:desk_listing:Q1")),
]
}
@ -3587,7 +3613,7 @@ mod tests {
);
}
assert_eq!(widest, RESOLUTION_LOCAL_MAX, "the bound is exactly the widest depth-two chain");
assert_eq!(RESOLUTION_LOCAL_MAX, 111);
assert_eq!(RESOLUTION_LOCAL_MAX, 112);
}
#[test]
@ -4109,7 +4135,7 @@ mod tests {
};
let document_schema = serde_json::to_value(schemars::schema_for!(PipelineDocument))?;
let variants = document_schema["oneOf"].as_array().expect("the document schema is a oneOf");
assert_eq!(PipelineKind::ALL.len(), 21, "twenty-one kinds");
assert_eq!(PipelineKind::ALL.len(), 22, "twenty-two kinds");
assert_eq!(
variants.iter().map(tag).collect::<Vec<_>>(),
PipelineKind::ALL.iter().map(|kind| kind.name().to_owned()).collect::<Vec<_>>(),
@ -4639,7 +4665,7 @@ mod tests {
}
fn desk_answer_sample() -> PipelineDeskAnswer {
let PipelineDocument::DeskAnswer(answer) = samples().pop().unwrap().0 else { unreachable!() };
let PipelineDocument::DeskAnswer(answer) = samples().remove(20).0 else { unreachable!() };
answer
}

View file

@ -536,6 +536,7 @@ fn chat_search(
json: bool,
) -> Result<u8, Trouble> {
let now = chrono::Utc::now();
let one_room = rooms.len() == 1;
let query = ChatQuery {
texts: texts.into_iter().map(Line).collect(),
rooms: rooms.into_iter().map(Short).collect(),
@ -548,7 +549,7 @@ fn chat_search(
if json {
say(serde_json::to_string_pretty(&found).map_err(|error| Trouble::Internal(error.to_string()))?);
} else {
say(render::chat_hits(&found));
say(render::chat_hits(&found, one_room));
}
Ok(0)
}

View file

@ -152,8 +152,11 @@ pub fn source_hits(found: &SourceHits) -> String {
/// when there are some), then one `time sender event_id body` line per message,
/// the body as the room held it with line breaks written `\n`: quoted data
/// under our header, never a message framed as if sent to the reader. A blank line between hits; a
/// truncated listing ends with the `--after` that continues it.
pub fn chat_hits(found: &ChatHits) -> String {
/// truncated listing of one room ends with the `--after` that continues it. A
/// listing of several rooms (`one_room` false) cannot be continued by an instant,
/// because chunks of different rooms overlap in time and a later start can hold an
/// earlier end; it says to page one room at a time.
pub fn chat_hits(found: &ChatHits, one_room: bool) -> String {
let mut out: Vec<String> = found
.hits
.iter()
@ -173,10 +176,11 @@ pub fn chat_hits(found: &ChatHits) -> String {
text
})
.collect();
if found.truncated
&& let Some(last) = found.hits.last()
{
out.push(format!("truncated: more follow, continue with --after {}\n", crate::chat::shown(last.ts_end_ms + 1)));
if found.truncated {
match found.hits.last() {
Some(last) if one_room => out.push(format!("truncated: more follow, continue with --after {}\n", crate::chat::shown(last.ts_end_ms + 1))),
_ => out.push("truncated: more follow; --after cannot continue a listing of several rooms, list one --room at a time\n".to_string()),
}
}
out.join("\n")
}

View file

@ -20,7 +20,7 @@ macro_rules! kinds {
kinds!(
campaign, target, question, ruling, cut_spec, cut_report, verdict, finding, follow_up, resolution, instance, stewardship,
hand_off, run, pulse, memory, research, idea, reply, thought, desk_answer,
hand_off, run, pulse, memory, research, idea, reply, thought, desk_answer, desk_listing,
);
/// The schema named, or the names when none is: the exit code is 2 for a name

View file

@ -1145,12 +1145,12 @@ fn schema_prints_the_published_files() {
let names = run(&[], &["schema"], None);
assert_eq!(names.code, 0);
let names: Vec<&str> = names.out.lines().collect();
assert_eq!(names.len(), 22, "{names:?}");
assert_eq!(names.len(), 23, "{names:?}");
assert_eq!(names[0], "campaign");
assert_eq!(names[14], "pulse");
assert_eq!(names[15], "memory");
assert_eq!((names[16], names[17]), ("research", "idea"));
assert_eq!((names[18], names[19], names[20], names[21]), ("reply", "thought", "desk_answer", "selection"));
assert_eq!((names[18], names[19], names[20], names[21], names[22]), ("reply", "thought", "desk_answer", "desk_listing", "selection"));
for kind in PipelineKind::ALL {
assert!(names.contains(&kind.name()), "{}", kind.name());
}
@ -2998,6 +2998,26 @@ fn chat_search_prints_cited_messages_and_sends_its_selectors() {
let rooms: Vec<&str> = query.rooms.iter().map(|room| room.0.as_str()).collect();
assert_eq!((texts, rooms, query.after_ms, query.before_ms, query.top_k), (vec!["a", "b"], vec!["!x:y", "#z:y"], Some(1_700_000_000_000), Some(1_700_000_060_000), 3));
// A truncated listing is continued by `--after` only inside one room; across rooms the spans overlap and an instant loses chunks.
let hit = |end| huginn_mind::ChatHit {
room_id: "!r:y".into(),
room_alias: None,
first_event_id: "$a".into(),
last_event_id: "$b".into(),
ts_start_ms: 1_700_000_000_000,
ts_end_ms: end,
score: None,
messages: vec![],
};
let cut_short = HuginnMindResponse::Chat(huginn_mind::ChatHits { indexed_under: huginn_mind::SourceModel { name: "m".into(), digest: "d".into() }, truncated: true, hits: vec![hit(1_700_000_060_000)] });
let (listing, _) = replying("search_chat", cut_short);
let one = at(listing.addr, &["chat", "search", "--room", "!r:y"], None);
assert!(one.out.contains("continue with --after 2023-11-14T22:14:20.001Z"), "{}", one.out);
for rooms in [&["chat", "search"][..], &["chat", "search", "--room", "!r:y", "--room", "!s:y"]] {
let many = at(listing.addr, rooms, None);
assert!(many.out.contains("--after cannot continue a listing of several rooms") && !many.out.contains("continue with --after"), "{rooms:?}: {}", many.out);
}
assert_ne!(at(spy.addr, &["chat", "search", "--after", "yesterday"], None).code, 0);
assert!(seen.lock().unwrap().is_empty(), "a bad instant never left the CLI");
for bad in [&["chat", "search", "--room", "swarm"][..], &["chat", "search", "--top-k", "0"]] {

View file

@ -172,7 +172,7 @@ pub fn serve<I: IndexSink<OwnedRedbMessagePackBackingStore> + Send + 'static>(
let loop_stopping = Arc::clone(&stopping);
let registry = schema_registry().unwrap();
let thread = std::thread::spawn(move || {
run(&mut daemon, &mut hub, &registry, None, &loop_stopping, &ServeOptions::default()).unwrap();
run(&mut daemon, &mut hub, None, &registry, None, &loop_stopping, &ServeOptions::default()).unwrap();
});
Server { addr, stopping, thread: Some(thread), ended: Arc::default(), _root: Some(root) }
}

View file

@ -18,7 +18,7 @@ name = "huginn-daemon"
path = "src/main.rs"
# The source index's one writer: a oneshot job that brings `huginn_sources` to
# the forge's current state and exits. No daemon calls any of its writes.
# the forge's and the chat reader's current state and exits. No daemon calls any of its writes.
[[bin]]
name = "huginn-sources"
path = "src/bin/huginn-sources.rs"
@ -34,7 +34,8 @@ rmp-serde = "1"
rustix = { version = "1", default-features = false, features = ["std", "net"] }
serde = { version = "1", features = ["derive"] }
# The vector store's and the embedder's HTTP APIs are the xenos boundary, and
# JSON exists in `index/ollama.rs` and `index/qdrant.rs` only. `ureq` 3 with
# JSON exists in `index/ollama.rs`, `index/qdrant.rs`, `sources/forge.rs` and
# `sources/matrix.rs` only. `ureq` 3 with
# default features off is plain HTTP with no TLS (`rustls` and `gzip` are its
# only defaults), which is all the private mesh needs. It stays below 3.4,
# whose `base64` 0.23 would put a second `base64` in a tree that has one.

View file

@ -1,21 +1,38 @@
//! The source index's refresh job: one run brings the `huginn_sources`
//! collection to the forge's current state, journals one line per repo, and
//! exits nonzero if any repo failed or the collection could not be used.
//! The source index's refresh job: one run brings the forge corpus and, when
//! configured, the rooms the chat reader has joined (minus the estate and DM
//! exclusion) to current state, journals one line per repo and room, and exits
//! nonzero if either failed or the collection could not be used. The
//! `chat-purge` subcommand writes a steward's purge tombstone and nothing else.
//!
//! Exit status: 0 clean, 1 a run failed, 2 bad arguments, 3 another run (or a
//! purge) holds the lock, which every run and every purge takes. The Matrix
//! token is read from a file (a systemd credential), never from the environment or argv.
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::collections::{BTreeMap, BTreeSet};
use std::fs::{File, OpenOptions, TryLockError};
use std::path::{Path, PathBuf};
use std::process::ExitCode;
use anyhow::{Context, Result, ensure};
use chrono::{DateTime, Utc};
use huginn_daemon::index::ollama::pool;
use huginn_daemon::index::qdrant::QdrantIndex;
use huginn_daemon::sources::forge::HttpForge;
use huginn_daemon::sources::refresh;
use huginn_daemon::sources::job::{self, Chat, Purge};
use huginn_daemon::sources::matrix::HttpMatrix;
use huginn_daemon::sources::room_scope::{ESTATE_ALIASES, Estate};
/// Where the forge answers when the flag is absent: the loopback address of the
/// forge on the host that runs this job.
const DEFAULT_FORGE_URL: &str = "http://127.0.0.1:3310";
/// The chat flags, which come all together or not at all.
struct ChatOptions {
matrix_url: String,
token: String,
estate: Estate,
}
struct Options {
state_root: PathBuf,
qdrant_url: String,
@ -25,75 +42,335 @@ struct Options {
reserve_urls: Vec<String>,
embedding_model: String,
forge_url: String,
chat: Option<ChatOptions>,
/// Every run takes it: the cursors object is replaced whole, with or without chat.
lock_file: PathBuf,
}
fn parse(args: impl Iterator<Item = String>) -> Result<Options> {
enum Command {
Run(Options),
Purge { qdrant_url: String, lock_file: PathBuf, purge: Purge },
}
/// How a run ends other than by finishing.
enum Stop {
Usage(anyhow::Error),
Locked,
}
/// The single-valued options and the repeated ones, by name.
type Collected = (BTreeMap<String, String>, BTreeMap<String, Vec<String>>);
/// The options named, each once, except those in `repeated`, which keep every value.
fn collect(args: impl Iterator<Item = String>, once: &[&str], repeated: &[&str]) -> Result<Collected> {
let mut args = args;
let mut values: BTreeMap<String, String> = BTreeMap::new();
let mut ollama_urls = Vec::new();
let mut reserve_urls = Vec::new();
let mut singles = BTreeMap::new();
let mut many: BTreeMap<String, Vec<String>> = BTreeMap::new();
while let Some(name) = args.next() {
let name = name.strip_prefix("--").with_context(|| format!("expected --option, got {name:?}"))?;
ensure!(
matches!(name, "state-root" | "qdrant-url" | "ollama-url" | "reserve-ollama-url" | "embedding-model" | "forge-url"),
"unsupported huginn-sources option --{name}"
);
ensure!(once.contains(&name) || repeated.contains(&name), "unsupported huginn-sources option --{name}");
let value = args.next().with_context(|| format!("missing value for --{name}"))?;
if name == "ollama-url" || name == "reserve-ollama-url" {
ensure!(value.starts_with("http://"), "--{name} must be a plain http:// address, got {value:?}");
if name == "ollama-url" { ollama_urls.push(value) } else { reserve_urls.push(value) }
if repeated.contains(&name) {
many.entry(name.to_owned()).or_default().push(value);
} else {
ensure!(values.insert(name.to_owned(), value).is_none(), "duplicate option --{name}");
ensure!(singles.insert(name.to_owned(), value).is_none(), "duplicate option --{name}");
}
}
let take = |name: &str| values.get(name).cloned().with_context(|| format!("--{name} is required"));
let url = |name: &str, value: String| {
ensure!(value.starts_with("http://"), "--{name} must be a plain http:// address, got {value:?}");
Ok(value)
};
let state_root = PathBuf::from(take("state-root")?);
ensure!(state_root.is_absolute(), "--state-root must be an absolute path");
let embedding_model = take("embedding-model")?;
ensure!(!embedding_model.is_empty(), "--embedding-model must name a model");
Ok(Options {
state_root,
qdrant_url: url("qdrant-url", take("qdrant-url")?)?,
ollama_urls: {
ensure!(!ollama_urls.is_empty(), "at least one --ollama-url is required; repeat it for each GPU host");
ollama_urls
Ok((singles, many))
}
fn http(name: &str, value: String) -> Result<String> {
ensure!(value.starts_with("http://"), "--{name} must be a plain http:// address, got {value:?}");
Ok(value)
}
/// The Matrix reader's bearer travels in clear, and this build speaks no TLS: only a loopback host may be named.
fn loopback_http(name: &str, value: String) -> Result<String> {
let value = http(name, value)?;
let authority = value["http://".len()..].split(['/', '?', '#']).next().unwrap_or("");
let host = match authority.strip_prefix('[') {
// The bracket must end the authority or be followed by the port, so `[::1].evil.example` names no loopback.
Some(inner) => match inner.split_once(']') {
Some((host, "")) => host,
Some((host, port)) if port.starts_with(':') => host,
_ => "",
},
None => authority.split(':').next().unwrap_or(""),
};
let loopback = !authority.contains('@') && (host.eq_ignore_ascii_case("localhost") || host.parse::<std::net::IpAddr>().is_ok_and(|ip| ip.is_loopback()));
ensure!(loopback, "--{name} must name a loopback host (localhost, 127.0.0.0/8 or [::1]): the bearer token travels in clear and this build has no TLS, got {value:?}");
Ok(value)
}
fn absolute(name: &str, value: &str) -> Result<PathBuf> {
let path = PathBuf::from(value);
ensure!(path.is_absolute(), "--{name} must be an absolute path");
Ok(path)
}
fn chat_options(singles: &BTreeMap<String, String>, excluded: Vec<String>) -> Result<Option<ChatOptions>> {
let given = [singles.contains_key("matrix-url"), singles.contains_key("matrix-token-file"), !excluded.is_empty()];
if !given.contains(&true) {
return Ok(None);
}
for (present, name) in given.iter().zip(["matrix-url", "matrix-token-file", "chat-exclude-room"]) {
ensure!(*present, "--{name} is missing: the chat flags (--matrix-url, --matrix-token-file, --chat-exclude-room) come all together or not at all");
}
let ids: BTreeSet<String> = excluded.iter().cloned().collect();
ensure!(ids.len() == excluded.len(), "--chat-exclude-room must not repeat a room id");
ensure!(ids.len() == ESTATE_ALIASES.len(), "--chat-exclude-room must be given exactly {} times (the ids of {})", ESTATE_ALIASES.len(), ESTATE_ALIASES.join(", "));
ensure!(ids.iter().all(|id| id.starts_with('!')), "--chat-exclude-room takes room ids beginning with !, not aliases");
let path = absolute("matrix-token-file", &singles["matrix-token-file"])?;
let token = std::fs::read_to_string(&path).with_context(|| format!("reading --matrix-token-file {}", path.display()))?.trim().to_owned();
ensure!(!token.is_empty(), "--matrix-token-file {} is empty", path.display());
Ok(Some(ChatOptions { matrix_url: loopback_http("matrix-url", singles["matrix-url"].clone())?, token, estate: Estate::new(ids)? }))
}
fn parse(mut args: impl Iterator<Item = String>, now: DateTime<Utc>) -> Result<Command> {
let mut rest: Vec<String> = Vec::new();
let first = args.next();
if first.as_deref() == Some("chat-purge") {
let (singles, _) = collect(args, &["qdrant-url", "room", "from", "to", "request", "lock-file"], &[])?;
let take = |name: &str| singles.get(name).cloned().with_context(|| format!("--{name} is required"));
return Ok(Command::Purge {
qdrant_url: http("qdrant-url", take("qdrant-url")?)?,
lock_file: absolute("lock-file", &take("lock-file")?)?,
purge: Purge::check(&take("room")?, &take("from")?, &take("to")?, &take("request")?, now)?,
});
}
rest.extend(first);
let (mut singles, mut many) = collect(
rest.into_iter().chain(args),
&["state-root", "qdrant-url", "embedding-model", "forge-url", "matrix-url", "matrix-token-file", "lock-file"],
&["ollama-url", "reserve-ollama-url", "chat-exclude-room"],
)?;
let mut hosts = |name: &str| -> Result<Vec<String>> { many.remove(name).unwrap_or_default().into_iter().map(|url| http(name, url)).collect() };
let ollama_urls = hosts("ollama-url")?;
let reserve_urls = hosts("reserve-ollama-url")?;
let excluded = many.remove("chat-exclude-room").unwrap_or_default();
let take = |singles: &mut BTreeMap<String, String>, name: &str| singles.remove(name).with_context(|| format!("--{name} is required"));
let state_root = absolute("state-root", &take(&mut singles, "state-root")?)?;
let embedding_model = take(&mut singles, "embedding-model")?;
ensure!(!embedding_model.is_empty(), "--embedding-model must name a model");
ensure!(!ollama_urls.is_empty(), "at least one --ollama-url is required; repeat it for each GPU host");
let chat = chat_options(&singles, excluded)?;
let lock_file = absolute("lock-file", &take(&mut singles, "lock-file")?)?;
Ok(Command::Run(Options {
state_root,
qdrant_url: http("qdrant-url", take(&mut singles, "qdrant-url")?)?,
ollama_urls,
reserve_urls,
embedding_model,
forge_url: url("forge-url", values.get("forge-url").cloned().unwrap_or_else(|| DEFAULT_FORGE_URL.to_owned()))?,
})
forge_url: http("forge-url", singles.remove("forge-url").unwrap_or_else(|| DEFAULT_FORGE_URL.to_owned()))?,
chat,
lock_file,
}))
}
/// Takes the job's lock for the life of the returned file. A held lock means a run (or a
/// purge request) is in progress and nothing may change: the cursors object is replaced whole.
fn take_lock(path: &Path) -> Result<File, Stop> {
let file = OpenOptions::new().create(true).truncate(false).write(true).open(path).with_context(|| format!("opening --lock-file {}", path.display())).map_err(Stop::Usage)?;
match file.try_lock() {
Ok(()) => Ok(file),
Err(TryLockError::WouldBlock) => Err(Stop::Locked),
Err(TryLockError::Error(error)) => Err(Stop::Usage(anyhow::Error::new(error).context(format!("locking --lock-file {}", path.display())))),
}
}
fn execute(command: Command) -> Result<u8, Stop> {
match command {
Command::Purge { qdrant_url, lock_file, purge } => {
let _lock = take_lock(&lock_file)?;
let mut index = QdrantIndex::new(&qdrant_url);
Ok(match job::purge(&mut index, &purge) {
Ok(line) => {
println!("{line}");
0
}
Err(error) => {
eprintln!("huginn-sources: {error:#}");
1
}
})
}
Command::Run(options) => {
let _lock = take_lock(&options.lock_file)?;
let mut embedder = pool(&options.ollama_urls, &options.reserve_urls, &options.embedding_model);
let mut index = QdrantIndex::new(&options.qdrant_url);
let mut forge = HttpForge::new(&options.forge_url);
let mut matrix = options.chat.as_ref().map(|chat| HttpMatrix::new(&chat.matrix_url, &chat.token));
let chat = matrix.as_mut().zip(options.chat.as_ref()).map(|(history, chat)| Chat { history, estate: &chat.estate });
Ok(job::run(&mut embedder, &mut index, &mut forge, &options.state_root, chat, &mut |line| println!("{line}"), &mut |line| eprintln!("{line}")))
}
}
}
fn main() -> ExitCode {
let options = match parse(std::env::args().skip(1)) {
Ok(options) => options,
Err(error) => {
match parse(std::env::args().skip(1), Utc::now()).map_err(Stop::Usage).and_then(execute) {
Ok(status) => ExitCode::from(status),
Err(Stop::Usage(error)) => {
eprintln!("huginn-sources: {error:#}");
return ExitCode::from(2);
ExitCode::from(2)
}
};
let mut embedder = pool(&options.ollama_urls, &options.reserve_urls, &options.embedding_model);
let mut index = QdrantIndex::new(&options.qdrant_url);
let mut forge = HttpForge::new(&options.forge_url);
let mut journal = |report: &huginn_daemon::sources::Report| println!("{report}");
match refresh(&mut embedder, &mut index, &mut forge, &options.state_root, &mut journal) {
Ok(refreshed) if refreshed.is_clean() => ExitCode::SUCCESS,
Ok(refreshed) => {
if !refreshed.failed.is_empty() {
eprintln!("huginn-sources: {} repos failed: {}", refreshed.failed.len(), refreshed.failed.join(", "));
}
if let Some(why) = &refreshed.retirement_withheld {
eprintln!("huginn-sources: no repo was retired: {why}");
}
ExitCode::from(1)
}
Err(error) => {
eprintln!("huginn-sources: {error:#}");
ExitCode::from(1)
Err(Stop::Locked) => {
eprintln!("huginn-sources: huginn-sources is running");
ExitCode::from(3)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
const NOW: &str = "2026-11-17T00:00:00Z";
fn args(text: &str) -> impl Iterator<Item = String> {
text.split_whitespace().map(str::to_owned).collect::<Vec<_>>().into_iter()
}
fn now() -> DateTime<Utc> {
DateTime::parse_from_rfc3339(NOW).unwrap().with_timezone(&Utc)
}
fn error(text: &str) -> String {
match parse(args(text), now()) {
Ok(_) => panic!("parsed: {text}"),
Err(error) => format!("{error:#}"),
}
}
const BASE: &str = "--state-root /s --qdrant-url http://q --ollama-url http://o --embedding-model m";
const IDS: &str = "--chat-exclude-room !a:s --chat-exclude-room !b:s --chat-exclude-room !c:s";
fn with_token(secret: &str) -> (tempfile::TempDir, String) {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("token");
std::fs::write(&path, secret).unwrap();
let path = path.display().to_string();
(dir, path)
}
#[test]
fn chat_flags_come_all_or_none() {
let (_dir, token) = with_token(" sekret-token\n");
let lock = "--lock-file /l";
let base = format!("{BASE} {lock}");
let full = format!("{BASE} --matrix-url http://localhost --matrix-token-file {token} {IDS} {lock}");
let Ok(Command::Run(options)) = parse(args(&full), now()) else { panic!("full chat flags must parse") };
assert_eq!(options.chat.as_ref().unwrap().token, "sekret-token", "trimmed");
assert!(matches!(parse(args(&base), now()), Ok(Command::Run(Options { chat: None, .. }))));
assert!(error(BASE).contains("--lock-file is required"), "a chat-off run needs the lock too");
assert!(error(&format!("{base} --matrix-url http://localhost")).contains("--matrix-token-file is missing"));
assert!(error(&format!("{base} --matrix-token-file {token} {IDS}")).contains("--matrix-url is missing"));
assert!(error(&format!("{base} --matrix-url http://localhost --matrix-token-file {token}")).contains("--chat-exclude-room is missing"));
assert!(error(&format!("{BASE} --matrix-url http://localhost --matrix-token-file {token} {IDS}")).contains("--lock-file is required"));
}
#[test]
fn chat_flag_values_are_checked_and_never_print_the_token() {
let (_dir, token) = with_token("sekret-token");
let on = |extra: &str| format!("{BASE} --matrix-url http://localhost --matrix-token-file {token} --lock-file /l {extra}");
assert!(error(&on("--chat-exclude-room !a:s --chat-exclude-room !b:s")).contains("exactly 3"));
assert!(error(&on(&format!("{IDS} --chat-exclude-room !d:s"))).contains("exactly 3"));
assert!(error(&on("--chat-exclude-room !a:s --chat-exclude-room !a:s --chat-exclude-room !b:s")).contains("repeat"));
assert!(error(&on("--chat-exclude-room !a:s --chat-exclude-room !b:s --chat-exclude-room #c:s")).contains("beginning with !"));
assert!(error(&format!("{BASE} --matrix-url https://m --matrix-token-file {token} --lock-file /l {IDS}")).contains("plain http://"));
assert!(error(&format!("{BASE} --matrix-url http://localhost --matrix-token-file rel --lock-file /l {IDS}")).contains("absolute"));
assert!(error(&format!("{BASE} --matrix-url http://localhost --matrix-token-file {token} --lock-file rel {IDS}")).contains("absolute"));
let (_empty, empty) = with_token(" \n");
let message = error(&format!("{BASE} --matrix-url http://localhost --matrix-token-file {empty} --lock-file /l {IDS}"));
assert!(message.contains(&empty) && message.contains("empty"));
let missing = error(&format!("{BASE} --matrix-url http://localhost --matrix-token-file /nowhere/t --lock-file /l {IDS}"));
assert!(missing.contains("/nowhere/t"));
for text in [on(IDS), on("--chat-exclude-room !a:s")] {
assert!(!error(&format!("{text} --nope x")).contains("sekret-token"));
}
let dead = ["--chat", "room"].join("-");
assert!(error(&format!("{BASE} {dead} !x:s")).contains(&format!("unsupported huginn-sources option {dead}")));
}
fn purge(extra: &str) -> String {
error(&format!("chat-purge --qdrant-url http://q --lock-file /l {extra}"))
}
#[test]
fn chat_purge_arguments() {
let good = "--room !r:s --from 2026-11-10T00:00:00Z --to 2026-11-11T00:00:00Z --request ref-1";
let Ok(Command::Purge { purge: asked, .. }) = parse(args(&format!("chat-purge --qdrant-url http://q --lock-file /l {good}")), now()) else { panic!() };
assert_eq!((asked.window.from_ms, asked.window.to_ms), (1_794_268_800_000, 1_794_355_200_000));
let bad = |from: &str, to: &str, room: &str, request: &str| purge(&format!("--room {room} --from {from} --to {to} --request {request}"));
let (f, t) = ("2026-11-10T00:00:00Z", "2026-11-11T00:00:00Z");
assert!(bad(f, t, "#a:s", "r").contains("--room"));
assert!(bad("2026-11-10T00:00:00+00:00", t, "!r:s", "r").contains("--from"));
assert!(bad(f, "2026-11-11T00:00:00", "!r:s", "r").contains("--to"));
assert!(bad(t, f, "!r:s", "r").contains("before"));
assert!(bad(f, "2026-11-18T00:00:00Z", "!r:s", "r").contains("after now"));
assert!(bad("2025-11-09T00:00:00Z", t, "!r:s", "r").contains("366"));
assert!(purge(&format!("--room !r:s --from {f} --to {t} --request {}", "x".repeat(121))).contains("--request"));
assert!(purge("--room !r:s --from 2026-11-10T00:00:00Z --to 2026-11-11T00:00:00Z").contains("--request is required"));
assert!(parse(args(&format!("chat-purge --qdrant-url http://q --lock-file /l {good} --matrix-url http://localhost")), now()).is_err(), "no Matrix flags");
}
#[test]
fn matrix_url_must_be_loopback_http() {
let (_dir, token) = with_token("sekret-token");
let with = |url: &str| format!("{BASE} --matrix-url {url} --matrix-token-file {token} --lock-file /l {IDS}");
for good in ["http://localhost", "http://127.0.0.1:8008", "http://127.9.9.9/", "http://[::1]:8008", "http://LOCALHOST:1/x"] {
assert!(parse(args(&with(good)), now()).is_ok(), "{good}");
}
for bad in ["http://matrix.example", "http://10.0.0.5:8008", "http://localhost@evil.example", "http://127.0.0.1.evil.example", "http://localhost.evil.example", "http://[::2]", "https://localhost", "http://127.0.0.1:pw@evil.example", "http://[::1].evil.example", "http://[::1]x"] {
assert!(error(&with(bad)).contains("--matrix-url"), "{bad}");
}
}
#[test]
fn a_held_lock_exits_3_through_execute_for_both_modes() {
let dir = tempfile::tempdir().unwrap();
let lock = dir.path().join("lock");
let held = take_lock(&lock).ok().unwrap();
let lock = lock.display().to_string();
let good = "--room !r:s --from 2026-11-10T00:00:00Z --to 2026-11-11T00:00:00Z --request r";
let purge = parse(args(&format!("chat-purge --qdrant-url http://127.0.0.1:1 --lock-file {lock} {good}")), now()).unwrap();
assert!(matches!(execute(purge), Err(Stop::Locked)));
let (_t, token) = with_token("sekret-token");
let run = parse(args(&format!("{BASE} --matrix-url http://localhost --matrix-token-file {token} {IDS} --lock-file {lock}")), now()).unwrap();
assert!(matches!(execute(run), Err(Stop::Locked)));
let off = parse(args(&format!("{BASE} --lock-file {lock}")), now()).unwrap();
assert!(matches!(execute(off), Err(Stop::Locked)), "a chat-off run must not write cursors under a held lock");
drop(held);
}
#[test]
fn the_lock_is_held_for_the_whole_run() {
let dir = tempfile::tempdir().unwrap();
let lock = dir.path().join("lock");
// A qdrant that accepts the run's first request and says nothing: the run is mid-flight while we probe the lock.
let qdrant = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let url = format!("http://{}", qdrant.local_addr().unwrap());
let run = parse(args(&format!("--state-root /s --qdrant-url {url} --ollama-url http://127.0.0.1:1 --embedding-model m --lock-file {}", lock.display())), now()).unwrap();
let worker = std::thread::spawn(move || execute(run));
let (connection, _) = qdrant.accept().unwrap();
assert!(matches!(take_lock(&lock), Err(Stop::Locked)), "the run must hold the lock while it works");
drop(connection);
// Refuse any retry the same way until the run gives up.
qdrant.set_nonblocking(true).unwrap();
while !worker.is_finished() {
drop(qdrant.accept());
std::thread::sleep(std::time::Duration::from_millis(5));
}
assert!(worker.join().unwrap().is_ok());
assert!(take_lock(&lock).is_ok(), "the lock is released when the run ends");
}
#[test]
fn a_held_lock_stops_the_run_untouched() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("lock");
let held = take_lock(&path).ok().unwrap();
assert!(matches!(take_lock(&path), Err(Stop::Locked)));
drop(held);
assert!(take_lock(&path).is_ok());
}
}

View file

@ -6,7 +6,7 @@ use chrono::{DateTime, Utc};
use cultnet_rs::Selection;
use crate::sources::forge::HttpForge;
use huginn_mind::desk_wire::{DeskRequest, DeskResponse};
use huginn_mind::eureka_pipeline::{PipelineRef, Slug};
use huginn_mind::eureka_pipeline::{AccountId, PipelineRef, Short, Slug};
use huginn_mind::wire::{ACTIVITY_REPOS_MAX, HuginnMindRequest, HuginnMindResponse, IndexStatus};
use huginn_mind::{ChatHits, ChatQuery, ChatWindow, ChatWindowQuery, Members, Mind, MindRefusal, MindStore, PipelineAdmissionOutcome, SemanticQuery, SourceHits, SourceQuery};
@ -109,6 +109,26 @@ impl Handled {
}
}
/// Which RUDP listener a request arrived on. `Full` is `--bind`, which the
/// faculties that write the mind reach. `Persona` is `--persona-bind`, the
/// only listener a Persona's uid can reach (the host fence decides that): it
/// answers what `HuginnMindRequest::reaches_persona_door` accepts and refuses
/// the rest with `MindRefusal::NotThisDoor` before the mind is asked.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Door {
Full,
Persona,
}
/// The door checks an account id; the Mind's desk calls take it as the text it is.
fn short_id(id: AccountId) -> Short {
Short(id.0)
}
fn short_ids(ids: &[AccountId]) -> Vec<Short> {
ids.iter().map(|id| Short(id.0.clone())).collect()
}
/// What the log says of the hits the mind held no document for: how many, and
/// the first few by name (a restored mind can leave hundreds), or nothing when
/// the mind held them all.
@ -168,6 +188,16 @@ impl<S: MindStore, I: IndexSink<S>> Daemon<S, I> {
runtime_id(self.mind.instance())
}
/// One request as it arrived at `door`: a request the door does not
/// answer is refused before the mind sees it, and every other goes to
/// `handle`. The wire reaches the mind only through here.
pub fn handle_at(&mut self, door: Door, request: HuginnMindRequest, now: DateTime<Utc>) -> Handled {
if door == Door::Persona && !request.reaches_persona_door() {
return Handled::Answered(HuginnMindResponse::Refused(MindRefusal::NotThisDoor));
}
self.handle(request, now)
}
/// One request, one answer, or a search to finish. `Admit` goes to the mind whole, because the
/// batch declares its own instance and admission's A1 is the check. Every
/// other request names an instance: `require_instance` now applies the
@ -271,10 +301,10 @@ impl<S: MindStore, I: IndexSink<S>> Daemon<S, I> {
}
match request {
DeskRequest::Answer { subject, authority, deciders, request, .. } => {
self.mind.desk_answer(&deciders, subject, authority, request, now)
self.mind.desk_answer(&short_ids(&deciders), short_id(subject), authority, request, now)
}
DeskRequest::Forks { subject, deciders, status, cursor, limit, .. } => self.mind.desk_forks(&subject, &deciders, status, cursor, limit),
DeskRequest::Fork { subject, deciders, fork_id, .. } => self.mind.desk_fork(&subject, &deciders, &fork_id),
DeskRequest::Forks { subject, deciders, status, cursor, limit, .. } => self.mind.desk_forks(&short_id(subject), &short_ids(&deciders), status, cursor, limit),
DeskRequest::Fork { subject, deciders, fork_id, .. } => self.mind.desk_fork(&short_id(subject), &short_ids(&deciders), &fork_id),
}
}
@ -1002,6 +1032,117 @@ pub(crate) mod tests {
assert_eq!(status(&mut daemon).persona, huginn_mind::wire::PersonaStatus::Absent);
}
/// The persona door refuses an admission before the mind sees it, so the
/// store is unchanged, and answers the reads and her own persona put; the
/// full door admits the same batch.
#[test]
fn the_persona_door_refuses_an_admission_and_answers_reads_and_her_persona() {
let (_root, mut daemon) = seeded();
let before = status(&mut daemon);
let admit = || HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)]));
assert_eq!(
daemon.handle_at(Door::Persona, admit(), now()).answered(),
HuginnMindResponse::Refused(MindRefusal::NotThisDoor)
);
assert_eq!(status(&mut daemon), before, "nothing was written");
let state = serde_json::json!({
"schemaVersion": "gamecult.persona_state.v0",
"provenance": {
"sourceSystem": "huginn", "sourceDocumentId": INSTANCE, "sourceUpdatedAt": "2026-11-19T10:00:00Z",
"exportedAt": "2026-11-19T10:00:00Z", "authority": "canonical"
},
"personaId": INSTANCE,
"publicName": "Test Persona",
"presentation": { "voiceSummary": "plain" },
"activationProfile": {
"underlyingOrganization": {}, "stableDispositions": {}, "behavioralDimensions": {},
"presentationStrategy": {}, "voiceStyle": {}, "situationalState": {}
},
"thoughtMemory": { "shortTerm": [], "memories": [], "incubation": [] },
"agencyPressure": { "pressures": [] },
"candidateActions": { "actions": [] },
"affect": {
"needs": [], "socialBonds": [], "statusReads": [], "moodDimensions": [],
"socialBiases": [], "doctrineStances": []
},
"updatedAt": "2026-11-19T10:00:00Z"
});
let put = HuginnMindRequest::PersonaPut {
instance: slug(INSTANCE),
provenance: provenance(),
state,
expected_updated_at: None,
};
assert!(matches!(
daemon.handle_at(Door::Persona, put, now()).answered(),
HuginnMindResponse::PersonaPut(huginn_mind::PersonaPutOutcome::Committed { .. })
));
assert!(matches!(
daemon.handle_at(Door::Persona, HuginnMindRequest::PersonaGet { instance: slug(INSTANCE) }, now()).answered(),
HuginnMindResponse::Persona(Some(_))
));
let query = HuginnMindRequest::Query { instance: slug(INSTANCE), selection: Selection::default(), semantic: None };
let HuginnMindResponse::Query(page) = daemon.handle_at(Door::Persona, query, now()).answered() else {
panic!("a query answers a page at the persona door")
};
let id = match page.items {
PipelinePageItems::Headers(items) => items[0].id.clone(),
PipelinePageItems::Documents(items) => items[0].id.clone(),
};
assert!(matches!(
daemon.handle_at(Door::Persona, HuginnMindRequest::View { instance: slug(INSTANCE), id }, now()).answered(),
HuginnMindResponse::View(Some(_))
));
assert_eq!(status(&mut daemon).documents, before.documents, "a persona put writes no pipeline document");
assert!(
matches!(daemon.handle_at(Door::Full, admit(), now()).answered(), HuginnMindResponse::Admit(_)),
"the full door hands the same batch to admission"
);
}
/// Every read the persona door answers reaches the mind there: none of the
/// source, chat, activity and roots reads is refused as `NotThisDoor`.
/// What the mind then says (a search, or `Unavailable` with no index or
/// forge) is the mind's, the same at either door.
#[test]
fn the_persona_door_hands_every_other_read_to_the_mind() {
use huginn_mind::wire::{ChatWindowQuery, SourceQuery};
let (_root, mut daemon) = seeded();
let reads = [
HuginnMindRequest::SearchSources {
instance: slug(INSTANCE),
query: SourceQuery { text: Line("reconcile".into()), texts: vec![], top_k: 5, repo: None, front: false },
},
HuginnMindRequest::SearchChat {
instance: slug(INSTANCE),
query: ChatQuery { texts: vec![Line("release".into())], rooms: vec![], after_ms: None, before_ms: None, top_k: 5 },
},
HuginnMindRequest::ChatWindow {
instance: slug(INSTANCE),
query: ChatWindowQuery { event_id: Short("$a".into()), before: 5, after: 0 },
},
HuginnMindRequest::StewardedActivity { instance: slug(INSTANCE), repos: vec![], after: None },
HuginnMindRequest::RootsTouched { instance: slug(INSTANCE) },
];
for read in reads {
let operation = read.operation();
let at_door = daemon.handle_at(Door::Persona, read.clone(), now());
assert!(
!matches!(at_door, Handled::Answered(HuginnMindResponse::Refused(MindRefusal::NotThisDoor))),
"{operation} is refused at the persona door"
);
if let (Handled::Answered(at_door), Handled::Answered(at_full)) = (at_door, daemon.handle_at(Door::Full, read, now())) {
assert_eq!(at_door, at_full, "{operation} answers the same at either door");
}
}
assert!(matches!(
daemon.handle_at(Door::Persona, HuginnMindRequest::RootsTouched { instance: slug(INSTANCE) }, now()).answered(),
HuginnMindResponse::RootsTouched(_)
));
}
/// A declared instance outside `Slug`'s grammar is refused by its own
/// name, `InvalidFormat`, on the read path, before the identity
/// comparison ever runs: it is not merely a foreign mind, it names no

View file

@ -20,6 +20,13 @@
//! gets `deciders` empty, `answerer` nil and the counts instead. The reads carry no `authority` (a
//! payload that names one is refused whole), and the door has no other way to ask who is reading.
//!
//! The listing gate: a reader who is not a decider sees a fork only while a `desk_listing` for it is
//! in force, and a fork it may not see reads `fork_unknown`, the same bytes as a missing one, on
//! every desk operation, a replay of a stored answer included. The door never writes a listing; it
//! arrives by the RUDP Admit door only. Unlisting is a `Withdrawn` resolution on the listing, and
//! admitting the listing again after that returns `AlreadyAdmitted` and the fork stays hidden:
//! relisting is withdrawing the withdrawal, not a second admission.
//!
//! It owns no rule. Which Mind method answers is `Daemon::handle_desk`'s; when the door is
//! served is `serve::run`'s, once per pass in the thread that owns the Mind. A peer that is
//! slow costs the loop at most `DESK_ACCEPTS_PER_PASS` connections of `DESK_CONNECTION_DEADLINE`

View file

@ -20,7 +20,7 @@ use cultnet_rs::{
decode_cultnet_message_from_slice, encode_cultnet_message_to_vec, encode_frame,
};
use huginn_mind::desk_wire::{
DESK_MAX_RESPONSE_BYTES, DESK_REQUEST_SCHEMA, DESK_SERVICE_ID, DeskRequest, DeskResponse, ForkAnswer, ForkAnswerCode, ForkAnswerRequest, ForkCursor,
DESK_REQUEST_SCHEMA, DESK_SERVICE_ID, DeskRequest, DeskResponse, ForkAnswer, ForkAnswerCode, ForkAnswerRequest, ForkCursor,
ForkEntry, ForkLimit, ForkList, ListStatus, RequestSchema,
};
use huginn_mind::envelope::{OperationFailure, decode_desk_response, decode_response, encode_desk_request, encode_request};
@ -85,20 +85,26 @@ fn wire(hash: &str, request_id: &str) -> ForkAnswerRequest {
}
}
/// A test's name for someone as the Bifrost account id the door requires: a deterministic lowercase D-format UUID.
fn acct(name: &str) -> huginn_mind::eureka_pipeline::AccountId {
let hash = name.bytes().fold(0xcbf2_9ce4_8422_2325_u64, |hash, byte| (hash ^ u64::from(byte)).wrapping_mul(0x0100_0000_01b3));
huginn_mind::eureka_pipeline::AccountId(format!("{:08x}-{:04x}-4000-8000-{:012x}", hash >> 32, (hash >> 16) & 0xffff, hash & 0xffff_ffff_ffff))
}
const R1: &str = "00000000-0000-4000-8000-000000000001";
const R2: &str = "00000000-0000-4000-8000-000000000002";
const R3: &str = "00000000-0000-4000-8000-000000000003";
fn answer(who: &str, authority: DeskAuthority, deciders: &[&str], request: ForkAnswerRequest) -> DeskRequest {
DeskRequest::Answer { mind: slug(INSTANCE), subject: short(who), authority, deciders: deciders.iter().map(|d| short(d)).collect(), request }
DeskRequest::Answer { mind: slug(INSTANCE), subject: acct(who), authority, deciders: deciders.iter().map(|d| acct(d)).collect(), request }
}
/// A list read by `who` against `deciders`.
fn forks(who: &str, deciders: &[&str], status: Option<ListStatus>, cursor: Option<ForkCursor>, limit: u8) -> DeskRequest {
DeskRequest::Forks {
mind: slug(INSTANCE),
subject: short(who),
deciders: deciders.iter().map(|d| short(d)).collect(),
subject: acct(who),
deciders: deciders.iter().map(|d| acct(d)).collect(),
status,
cursor,
limit: ForkLimit::new(limit).unwrap(),
@ -110,7 +116,7 @@ fn forks_all(deciders: &[&str]) -> DeskRequest {
}
fn fork_read(who: &str, deciders: &[&str], fork_id: &str) -> DeskRequest {
DeskRequest::Fork { mind: slug(INSTANCE), subject: short(who), deciders: deciders.iter().map(|d| short(d)).collect(), fork_id: fork_id.into() }
DeskRequest::Fork { mind: slug(INSTANCE), subject: acct(who), deciders: deciders.iter().map(|d| acct(d)).collect(), fork_id: fork_id.into() }
}
fn page_of(response: DeskResponse) -> (ForkList, u64) {
@ -205,6 +211,13 @@ fn rig(peer_uid: u32) -> Rig {
}
impl Rig {
/// The Mind lists the seeded fork, on its own door, so a non-decider reaches it.
fn list_the_fork(&mut self) {
let listing = PipelineDocument::DeskListing(huginn_mind::eureka_pipeline::PipelineDeskListing { campaign: slug("eureka-state"), listed: short(FORK), reason: "Listed.".into() });
let admitted = self.daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![listing])), now()).answered();
assert!(matches!(admitted, HuginnMindResponse::Admit(huginn_mind::PipelineAdmissionOutcome::Committed { .. })), "{admitted:?}");
}
/// Runs `client` on its own thread while this one serves the door, then drains what is still waiting.
fn serving<T: Send>(&mut self, client: impl FnOnce(&Path) -> T + Send) -> T {
let path = self.path.clone();
@ -454,7 +467,7 @@ mod door_smoke {
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
run(&mut daemon, &mut hub, &registry, Some(&mut door), &loop_stopping, &ServeOptions::default()).unwrap();
run(&mut daemon, &mut hub, None, &registry, Some(&mut door), &loop_stopping, &ServeOptions::default()).unwrap();
});
let mut stalled = connect(&path);
@ -567,7 +580,7 @@ mod door_smoke {
claim_in_future: false,
request_id: RequestId(R1.into()),
body_sha256: Sha256Hex("0".repeat(64)),
answerer: short("x"),
answerer: acct("x"),
authority: DeskAuthority::Operator,
outcome: DeskOutcome::Counted,
});
@ -578,7 +591,7 @@ mod door_smoke {
let registry = schema_registry().unwrap();
let mut bodies = crate::bodies::DeferredBodies::new(1 << 20, Duration::from_secs(60));
let desk = raw(MIND_SERVICE_ID, DESK_REQUEST_SCHEMA, "answer", "m-1", rmp_serde::to_vec_named(&answer("x", DeskAuthority::Operator, &["x"], wire(&hash, R1))).unwrap());
let CultNetMessage::OperationResponse { payload_schema, diagnostics, .. } = crate::serve::answer(&mut daemon, &registry, &mut bodies, desk, now()).reply() else {
let CultNetMessage::OperationResponse { payload_schema, diagnostics, .. } = crate::serve::answer(&mut daemon, crate::daemon::Door::Full, &registry, &mut bodies, desk, now()).reply() else {
panic!("expected a response")
};
assert_eq!(payload_schema, huginn_mind::envelope::FAILURE_SCHEMA);
@ -601,6 +614,7 @@ mod door_smoke {
#[test]
fn deciders_ride_each_request() {
let mut rig = rig(my_uid());
rig.list_the_fork();
let (before, hash) = (rig.state(), rig.hash.clone());
let out = rig.serving(|path| {
let ask = |id: &str, who: &str, deciders: &[&str], request_id: &str| {
@ -672,6 +686,7 @@ mod door_smoke {
state_root: root.path().to_path_buf(),
instance: slug(INSTANCE),
bind: "127.0.0.1:0".parse().unwrap(),
persona_bind: None,
qdrant_url: "http://127.0.0.1:1".into(),
ollama_urls: vec!["http://127.0.0.1:1".into()],
reserve_ollama_urls: Vec::new(),
@ -776,6 +791,7 @@ mod door_smoke {
#[test]
fn a_mind_refusal_and_a_stored_refusal_are_typed_replies() {
let mut rig = rig(my_uid());
rig.list_the_fork();
let (before, hash) = (rig.state(), rig.hash.clone());
let mut foreign = answer("x", DeskAuthority::Member, &["x"], wire(&hash, R1));
let DeskRequest::Answer { mind, .. } = &mut foreign else { unreachable!() };
@ -801,7 +817,7 @@ mod door_smoke {
assert_eq!((listed_at, read_at), (before.1 as u64, before.1 as u64), "both read at the head the mind held");
assert_eq!(list.stamp, format!("s{listed_at}"));
assert_eq!(list.forks[0], entry);
assert_eq!((entry.card.fork_id.as_str(), entry.card.options_hash.as_str(), entry.card.deciders.as_slice()), (FORK, hash.as_str(), ["x".to_string(), "y".to_string()].as_slice()));
assert_eq!((entry.card.fork_id.as_str(), entry.card.options_hash.as_str(), entry.card.deciders.as_slice()), (FORK, hash.as_str(), [acct("x").0, acct("y").0].as_slice()));
assert_eq!(rig.state(), before, "a read stores nothing");
assert!(rig.index.seen.lock().unwrap().is_empty());
// An unknown fork is the answer path's refusal.
@ -864,11 +880,19 @@ mod door_smoke {
let note = "Keep it local: the author's call.";
rig.serving(|path| counted(reply_of(path, "m-1", &DeskRequest::Answer {
mind: slug(INSTANCE),
subject: short("y"),
subject: acct("y"),
authority: DeskAuthority::Member,
deciders: vec![short("y")],
deciders: vec![acct("y")],
request: noted(&hash, R1, note),
})));
// Unlisted: the outsider is shown nothing at all, and a read is `fork_unknown`.
let (hidden, unknown) = rig.serving(|path| (reply_of(path, "m-1", &forks("outsider", &["x"], None, None, 50)), reply_of(path, "m-1b", &fork_read("outsider", &["x"], FORK))));
assert_eq!(page_of(hidden).0.forks.len(), 0);
assert!(matches!(unknown, DeskResponse::NotStored(_)), "{unknown:?}");
// The Mind lists it, on its own door.
let listing = PipelineDocument::DeskListing(huginn_mind::eureka_pipeline::PipelineDeskListing { campaign: slug("eureka-state"), listed: short(FORK), reason: "Members may read why.".into() });
let admitted = rig.daemon.handle(HuginnMindRequest::Admit(batch(INSTANCE, vec![listing])), now()).answered();
assert!(matches!(admitted, HuginnMindResponse::Admit(huginn_mind::PipelineAdmissionOutcome::Committed { .. })), "{admitted:?}");
let (list, read) = rig.serving(|path| (reply_of(path, "m-2", &forks("outsider", &["x"], None, None, 50)), reply_of(path, "m-3", &fork_read("outsider", &["x"], FORK))));
let (list, _) = page_of(list);
let (read, _) = entry_of(read);
@ -882,8 +906,8 @@ mod door_smoke {
}
let seen = rig.serving(|path| reply_of(path, "m-4", &fork_read("x", &["x"], FORK)));
let (seen, _) = entry_of(seen);
assert_eq!(seen.card.deciders, ["x"]);
assert_eq!(seen.record.answers[0].answerer.as_deref(), Some("y"));
assert_eq!(seen.card.deciders, [acct("x").0]);
assert_eq!(seen.record.answers[0].answerer.as_deref(), Some(acct("y").0.as_str()));
}
#[test]
@ -902,7 +926,7 @@ mod door_smoke {
let refused = rig.serving(|path| failure_of(path, &bytes));
assert_eq!(refused.code, "payload-not-a-request", "{what}");
assert!(refused.message.contains("deciders"), "{what}: {}", refused.message);
assert!(!refused.message.contains("d64"), "{what} echoes a decider: {}", refused.message);
assert!(!refused.message.contains(&acct("d64").0), "{what} echoes a decider: {}", refused.message);
}
assert_eq!(rig.state(), before, "nothing stored, head unchanged");
}

View file

@ -1,5 +1,5 @@
//! The process: open one instance's mind, bind one socket, serve until a stop
//! signal. A mind that will not open ends the process with the refusal on
//! The process: open one instance's mind, bind its socket (and the persona
//! door's, when asked), serve until a stop signal. A mind that will not open ends the process with the refusal on
//! stderr and no socket ever bound.
use std::sync::Arc;
@ -15,7 +15,7 @@ fn main() -> Result<()> {
let options = parse_options(std::env::args().skip(1))?;
let embedder = pool(&options.ollama_urls, &options.reserve_ollama_urls, &options.embedding_model);
let index = QdrantIndex::new(&options.qdrant_url);
let Started { mut daemon, mut hub, registry, mut desk } = match startup(&options, embedder, index, Backoff::default()) {
let Started { mut daemon, mut hub, mut persona, registry, mut desk } = match startup(&options, embedder, index, Backoff::default()) {
Ok(started) => started,
Err(error) => {
eprintln!("huginn-daemon refuses to serve {}: {error:#}", options.instance.0);
@ -33,8 +33,11 @@ fn main() -> Result<()> {
}
eprintln!("huginn-daemon serving {} on {}", options.instance.0, hub.local_addr()?);
if let Some(door) = &persona {
eprintln!("huginn-daemon serving the persona door on {}", door.local_addr()?);
}
if let Some(door) = &desk {
eprintln!("huginn-daemon serving the desk door on {}", door.path().display());
}
run(&mut daemon, &mut hub, &registry, desk.as_mut(), &stopping, &ServeOptions::default())
run(&mut daemon, &mut hub, persona.as_mut(), &registry, desk.as_mut(), &stopping, &ServeOptions::default())
}

View file

@ -143,9 +143,10 @@ fn quick() -> Backoff {
Backoff { initial: Duration::from_millis(1), max: Duration::from_millis(2), recheck: Duration::from_secs(3600), search_deadline: Duration::from_secs(30) }
}
/// A daemon serving a mind over a real socket, with the ports the test gave it.
/// A daemon serving a mind over real sockets, its full door and its persona door, with the ports the test gave it.
struct Harness {
endpoint: String,
persona_endpoint: String,
stopping: Arc<AtomicBool>,
serving: Option<JoinHandle<Result<()>>>,
embedder: FakeEmbedder,
@ -172,19 +173,31 @@ impl Harness {
let mut daemon: Daemon<OwnedRedbMessagePackBackingStore, WorkerSink> = Daemon::new(mind, sink);
let mut hub = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let endpoint = format!("rudp://{}", hub.local_addr().unwrap());
let mut door = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let persona_endpoint = format!("rudp://{}", door.local_addr().unwrap());
let registry = schema_registry().unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let options = ServeOptions::default();
let serving = std::thread::spawn(move || run(&mut daemon, &mut hub, &registry, None, &loop_stopping, &options));
Self { endpoint, stopping, serving: Some(serving), embedder, index, _root: root }
let serving =
std::thread::spawn(move || run(&mut daemon, &mut hub, Some(&mut door), &registry, None, &loop_stopping, &options));
Self { endpoint, persona_endpoint, stopping, serving: Some(serving), embedder, index, _root: root }
}
fn client(&self) -> Client {
Self::client_at(&self.endpoint)
}
/// A client of the persona door.
fn persona_client(&self) -> Client {
Self::client_at(&self.persona_endpoint)
}
fn client_at(endpoint: &str) -> Client {
let mut client = CultMesh::create_rudp_client_for_endpoint(
"semantic-test".to_string(),
CULTNET_OPERATION_CONNECTION_ID,
&self.endpoint,
endpoint,
CultMeshRudpSocketOptions::default(),
)
.unwrap();
@ -272,8 +285,19 @@ fn healthy() -> Harness {
/// the members.
#[test]
fn a_semantic_query_returns_only_documents_the_mind_holds_and_the_selection_admits() {
a_semantic_query_returns_only_documents_the_mind_holds_and_the_selection_admits_at(Harness::client);
}
/// The same semantic read asked at the persona door: the search is held while the index works, and its answer leaves by
/// the persona door's own hub to the session that asked there.
#[test]
fn a_semantic_query_at_the_persona_door_is_answered_through_the_persona_door() {
a_semantic_query_returns_only_documents_the_mind_holds_and_the_selection_admits_at(Harness::persona_client);
}
fn a_semantic_query_returns_only_documents_the_mind_holds_and_the_selection_admits_at(client_of: fn(&Harness) -> Client) {
let harness = healthy();
let mut client = harness.client();
let mut client = client_of(&harness);
until_index(&mut client, "Current", |status| *status == IndexStatus::Current);
harness.index.state.lock().unwrap().hits = vec![
hit("question", id("question", "Q1"), 0.95),

View file

@ -78,7 +78,7 @@ use huginn_mind::{Mind, MindRefusal, MindStore, OwnedRedbMessagePackBackingStore
use crate::bodies::{
DEFERRED_BUDGET_BYTES, DEFERRED_CHUNK_BYTES, DeferredBodies,
};
use crate::daemon::{Daemon, Handled, IndexSink, Search, SearchTicket, runtime_id};
use crate::daemon::{Daemon, Door, Handled, IndexSink, Search, SearchTicket, runtime_id};
use crate::desk_door::{DeskDoor, bind_desk};
use huginn_mind::envelope::{OperationFailure, decode_request, encode_failure, encode_response};
use huginn_mind::MAX_DEFERRED_BODY_BYTES;
@ -111,6 +111,10 @@ pub struct Options {
pub state_root: PathBuf,
pub instance: Slug,
pub bind: SocketAddr,
/// The persona door (`--persona-bind`): a second RUDP listener answering
/// only what `HuginnMindRequest::reaches_persona_door` accepts. Absent,
/// the daemon has no persona door.
pub persona_bind: Option<SocketAddr>,
pub qdrant_url: String,
/// The GPU hosts: all take embedding at once. One is the plain case.
pub ollama_urls: Vec<String>,
@ -134,7 +138,7 @@ pub struct DeskOptions {
/// Exactly `--state-root`, `--instance`, `--bind`, `--qdrant-url`,
/// `--ollama-url` and `--embedding-model`, each required, each once, except
/// `--ollama-url`, which repeats for each GPU host, and the optional
/// `--reserve-ollama-url`, which repeats for each CPU host, and the optional `--forge-url`, once, and the optional pair `--desk-socket` (an absolute path) and `--desk-peer-uid` (a uid other than 0), both or neither, each once. The URLs are plain
/// `--reserve-ollama-url`, which repeats for each CPU host, and the optional `--forge-url` and `--persona-bind` (an ip:port other than `--bind`), each once, and the optional pair `--desk-socket` (an absolute path) and `--desk-peer-uid` (a uid other than 0), both or neither, each once. The URLs are plain
/// `http://`: this build carries no TLS.
pub fn parse_options(args: impl Iterator<Item = String>) -> Result<Options> {
let mut args = args;
@ -143,7 +147,7 @@ pub fn parse_options(args: impl Iterator<Item = String>) -> Result<Options> {
while let Some(name) = args.next() {
let name = name.strip_prefix("--").with_context(|| format!("expected --option, got {name:?}"))?;
ensure!(
matches!(name, "state-root" | "instance" | "bind" | "qdrant-url" | "ollama-url" | "reserve-ollama-url" | "embedding-model" | "forge-url" | "desk-socket" | "desk-peer-uid"),
matches!(name, "state-root" | "instance" | "bind" | "persona-bind" | "qdrant-url" | "ollama-url" | "reserve-ollama-url" | "embedding-model" | "forge-url" | "desk-socket" | "desk-peer-uid"),
"unsupported Huginn option --{name}"
);
let value = args.next().with_context(|| format!("missing value for --{name}"))?;
@ -167,10 +171,20 @@ pub fn parse_options(args: impl Iterator<Item = String>) -> Result<Options> {
};
let embedding_model = take("embedding-model")?;
ensure!(!embedding_model.is_empty(), "--embedding-model must name a model");
let bind: SocketAddr = bind.parse().with_context(|| format!("--bind must be an ip:port, got {bind:?}"))?;
let persona_bind = match values.get("persona-bind") {
None => None,
Some(door) => {
let door: SocketAddr = door.parse().with_context(|| format!("--persona-bind must be an ip:port, got {door:?}"))?;
ensure!(door != bind, "--persona-bind must not be --bind: the persona door is its own listener");
Some(door)
}
};
Ok(Options {
state_root,
instance: Slug(take("instance")?),
bind: bind.parse().with_context(|| format!("--bind must be an ip:port, got {bind:?}"))?,
bind,
persona_bind,
qdrant_url: url("qdrant-url")?,
ollama_urls: {
ensure!(!ollama_urls.is_empty(), "--ollama-url is required; repeat it for each GPU host");
@ -206,6 +220,7 @@ where
{
let mind = Mind::open(&options.state_root, &options.instance)?;
let hub = bind(options.bind, &runtime_id(&options.instance))?;
let persona = options.persona_bind.map(|door| bind(door, &runtime_id(&options.instance))).transpose()?;
let desk = options.desk.as_ref().map(|desk| bind_desk(&desk.socket, desk.peer_uid)).transpose()?;
let startup_entries = mind.index_entries(None)?;
let sink = WorkerSink::spawn(embedder, index, &options.instance, mind.genesis_receipt_id()?, startup_entries, backoff);
@ -214,14 +229,16 @@ where
Some(url) => daemon.with_forge(crate::sources::forge::HttpForge::new(url)),
None => daemon,
};
Ok(Started { daemon, hub, registry: schema_registry()?, desk })
Ok(Started { daemon, hub, persona, registry: schema_registry()?, desk })
}
/// What `startup` hands the loop: the daemon over the opened mind, the bound RUDP hub, the
/// catalog the RUDP door advertises, and the desk door when one was asked for.
/// persona door's hub when one was asked for, the catalog both RUDP doors advertise, and the
/// desk door when one was asked for.
pub struct Started {
pub daemon: Daemon<OwnedRedbMessagePackBackingStore, WorkerSink>,
pub hub: CultNetRudpServerHub,
pub persona: Option<CultNetRudpServerHub>,
pub registry: CultNetSchemaRegistry,
pub desk: Option<DeskDoor>,
}
@ -321,12 +338,14 @@ impl Routed {
}
/// One message in, one message out, or a search to wait for. Nothing here
/// reaches a mind except through `Daemon::handle`, and an envelope that does
/// reaches a mind except through `Daemon::handle_at`, at the `door` the message
/// arrived on, and an envelope that does
/// not decode never gets that far. A chunk request is answered from `bodies`
/// alone, on the session it came in on; every operation's reply leaves through
/// `respond`.
pub fn answer<S: MindStore, I: IndexSink<S>>(
daemon: &mut Daemon<S, I>,
door: Door,
registry: &CultNetSchemaRegistry,
bodies: &mut DeferredBodies,
message: CultNetMessage,
@ -345,7 +364,7 @@ pub fn answer<S: MindStore, I: IndexSink<S>>(
CultNetMessage::OperationRequest { message_id, operation, .. } => match decode_request(&message) {
Ok((message_id, request)) => {
let operation = request.operation();
match daemon.handle(request, now) {
match daemon.handle_at(door, request, now) {
Handled::Answered(response) => respond(&response, &message_id, operation, &runtime_id, bodies, now),
Handled::Searching(search) => return Routed::Search { message_id, operation, search },
}
@ -483,9 +502,11 @@ fn deliver(
deferred
}
/// A semantic query on its way through the index: who asked, what to call the
/// answer, the search to finish, and when the client stops being promised one.
/// A semantic query on its way through the index: who asked, at which door,
/// what to call the answer, the search to finish, and when the client stops
/// being promised one.
struct Waiting {
door: Door,
session: CultNetRudpServerSessionContext,
message_id: String,
operation: &'static str,
@ -494,9 +515,11 @@ struct Waiting {
}
/// Until `stopping`: expire what timed out, resend what was not acknowledged,
/// answer every frame waiting on the socket, then answer every semantic query
/// the index has finished or that has waited out the index's own
/// `search_deadline`. A semantic
/// answer every frame waiting on either RUDP door's socket, then answer every
/// semantic query the index has finished or that has waited out the index's
/// own `search_deadline`. A frame on `hub` is asked at `Door::Full`, one on
/// `persona` at `Door::Persona`, and a search's answer leaves by the door it
/// came in at. A semantic
/// query is never answered inline: `answer` hands back a `Search`, the loop
/// holds it beside its session and goes on serving, and the embedder and the
/// vector store are the worker's to wait on. A hostile datagram and a departed
@ -505,51 +528,18 @@ struct Waiting {
pub fn run<S: MindStore, I: IndexSink<S>>(
daemon: &mut Daemon<S, I>,
hub: &mut CultNetRudpServerHub,
mut persona: Option<&mut CultNetRudpServerHub>,
registry: &CultNetSchemaRegistry,
mut desk: Option<&mut DeskDoor>,
stopping: &AtomicBool,
options: &ServeOptions,
) -> Result<()> {
let timeout_ms = options.session_timeout.as_millis() as u64;
let mut bodies = DeferredBodies::new(DEFERRED_BUDGET_BYTES, options.deferred_ttl);
let mut waiting: BTreeMap<SearchTicket, Waiting> = BTreeMap::new();
while !stopping.load(Ordering::Relaxed) {
hub.remove_timed_out_sessions(timeout_ms);
if let Err(error) = hub.poll_resends() {
eprintln!("huginn: resending to a peer failed: {error:#}");
}
let mut served = 0_u32;
loop {
let event = match hub.receive_event_once() {
Ok(Some(event)) => event,
Ok(None) => break,
Err(error) => {
eprintln!("huginn: a datagram was not readable and was discarded: {error:#}");
break;
}
};
let CultNetRudpServerEvent::Frame { session, frame } = event else {
continue;
};
if frame.channel_id != "schema" {
continue;
}
let message = match decode_cultnet_message_from_slice(&frame.payload, CultNetWireContract::CultNetSchemaV0)
{
Ok(message) => message,
Err(error) => {
eprintln!("huginn: a frame was not a CultNet message and was discarded: {error:#}");
continue;
}
};
match answer(daemon, registry, &mut bodies, message, Utc::now()) {
Routed::Reply(reply) => send(hub, &session, &reply),
Routed::Search { message_id, operation, search } => {
let until = Instant::now() + daemon.search_deadline();
waiting.insert(search.ticket, Waiting { session, message_id, operation, search, until });
}
}
served += 1;
let mut served = serve_frames(daemon, hub, Door::Full, registry, &mut bodies, &mut waiting, options);
if let Some(door) = persona.as_deref_mut() {
served += serve_frames(daemon, door, Door::Persona, registry, &mut bodies, &mut waiting, options);
}
let desk_served = desk.as_deref_mut().map_or(0, |door| door.serve_ready(daemon, Utc::now()));
let finished = collect_searches(daemon, &waiting);
@ -558,7 +548,14 @@ pub fn run<S: MindStore, I: IndexSink<S>>(
let Some(asked) = waiting.remove(&ticket) else { continue };
let response = daemon.finish(asked.search, found);
let reply = respond(&response, &asked.message_id, asked.operation, &daemon.runtime_id(), &mut bodies, Utc::now());
send(hub, &asked.session, &reply);
let out = match asked.door {
Door::Full => &mut *hub,
Door::Persona => match persona.as_deref_mut() {
Some(door) => door,
None => continue,
},
};
send(out, &asked.session, &reply);
}
if served == 0 && desk_served == 0 && !answered {
std::thread::sleep(options.idle_sleep);
@ -567,6 +564,57 @@ pub fn run<S: MindStore, I: IndexSink<S>>(
Ok(())
}
/// One pass over one RUDP door: expire its idle sessions, resend what it has
/// unacknowledged, and answer every frame waiting on its socket as asked at
/// `door`. Returns how many frames it answered or queued.
fn serve_frames<S: MindStore, I: IndexSink<S>>(
daemon: &mut Daemon<S, I>,
hub: &mut CultNetRudpServerHub,
door: Door,
registry: &CultNetSchemaRegistry,
bodies: &mut DeferredBodies,
waiting: &mut BTreeMap<SearchTicket, Waiting>,
options: &ServeOptions,
) -> u32 {
hub.remove_timed_out_sessions(options.session_timeout.as_millis() as u64);
if let Err(error) = hub.poll_resends() {
eprintln!("huginn: resending to a peer failed: {error:#}");
}
let mut served = 0_u32;
loop {
let event = match hub.receive_event_once() {
Ok(Some(event)) => event,
Ok(None) => break,
Err(error) => {
eprintln!("huginn: a datagram was not readable and was discarded: {error:#}");
break;
}
};
let CultNetRudpServerEvent::Frame { session, frame } = event else {
continue;
};
if frame.channel_id != "schema" {
continue;
}
let message = match decode_cultnet_message_from_slice(&frame.payload, CultNetWireContract::CultNetSchemaV0) {
Ok(message) => message,
Err(error) => {
eprintln!("huginn: a frame was not a CultNet message and was discarded: {error:#}");
continue;
}
};
match answer(daemon, door, registry, bodies, message, Utc::now()) {
Routed::Reply(reply) => send(hub, &session, &reply),
Routed::Search { message_id, operation, search } => {
let until = Instant::now() + daemon.search_deadline();
waiting.insert(search.ticket, Waiting { door, session, message_id, operation, search, until });
}
}
served += 1;
}
served
}
/// What the index has finished, then what has waited too long: the latter as
/// an `Unavailable` detail, and abandoned, so the index does not start a search
/// nobody is waiting for. A search that finishes after its deadline finds
@ -680,14 +728,14 @@ mod tests {
("m-5b", request("m-5b", MIND_SERVICE_ID, "Admit", MIND_REQUEST_SCHEMA, admit), "operation-mismatch"),
];
for (id, message, code) in cases {
let reply = answer(&mut daemon, &registry, &mut fresh_bodies(), message, now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut fresh_bodies(), message, now()).reply();
assert_eq!(rejected(&reply).0, id, "the client's own correlation key is echoed");
assert_eq!(rejected(&reply).1, code);
}
// The sixth code, `not-an-operation-request`, is the codec's and is
// pinned in `huginn_mind::envelope::tests`: `answer` never produces it, because a
// message of another family is answered with `Error` instead.
let reply = answer(&mut daemon, &registry, &mut fresh_bodies(), CultNetMessage::Error { error: "hello".into(), code: None, details: None }, now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut fresh_bodies(), CultNetMessage::Error { error: "hello".into(), code: None, details: None }, now()).reply();
let CultNetMessage::Error { error, .. } = &reply else { panic!("expected an error, got {reply:?}") };
assert!(
!error.is_empty() && error.contains("cultnet.operation_request.v0") && error.contains("cultnet.schema_catalog_request.v0"),
@ -699,7 +747,7 @@ mod tests {
// A mind's refusal is an answer on the response schema, not a failure
// of the envelope: the client decodes it as the typed refusal it is.
let read = HuginnMindRequest::Query { instance: slug(OTHER), selection: Selection::default(), semantic: None };
let reply = answer(&mut daemon, &registry, &mut fresh_bodies(), encode_request("m-r", &read, None).unwrap(), now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut fresh_bodies(), encode_request("m-r", &read, None).unwrap(), now()).reply();
let CultNetMessage::OperationResponse { status, payload_schema, .. } = &reply else {
panic!("expected an operation response, got {reply:?}");
};
@ -717,7 +765,7 @@ mod tests {
schema_ids: None,
kinds: None,
};
let reply = answer(&mut daemon, &registry, &mut fresh_bodies(), catalog, now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut fresh_bodies(), catalog, now()).reply();
let CultNetMessage::SchemaCatalogResponse { message_id, schemas } = reply else {
panic!("expected a catalog response");
};
@ -785,7 +833,7 @@ mod tests {
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
let result = run(&mut daemon, &mut hub, &registry, None, &loop_stopping, &ServeOptions::default());
let result = run(&mut daemon, &mut hub, None, &registry, None, &loop_stopping, &ServeOptions::default());
(result, daemon)
});
@ -874,6 +922,43 @@ mod tests {
panic!("{name} never connected");
}
/// Over loopback, the persona door's listener refuses an admission by name and answers a
/// read, and the full door's listener admits the same batch: only that admission lands.
#[test]
fn the_persona_listener_refuses_an_admission_the_full_listener_takes() {
let root = tempfile::tempdir().unwrap();
let mut daemon = open_unindexed(root.path(), &slug(INSTANCE)).unwrap();
let mut hub = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let mut door = bind("127.0.0.1:0".parse().unwrap(), &daemon.runtime_id()).unwrap();
let full = format!("rudp://{}", hub.local_addr().unwrap());
let persona = format!("rudp://{}", door.local_addr().unwrap());
let registry = schema_registry().unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
let result = run(&mut daemon, &mut hub, Some(&mut door), &registry, None, &loop_stopping, &ServeOptions::default());
(result, daemon)
});
let admit = HuginnMindRequest::Admit(batch(INSTANCE, vec![identity(INSTANCE)]));
let mut her = connect(&persona, "persona");
assert_eq!(ask(&mut her, "p-admit", &admit), HuginnMindResponse::Refused(MindRefusal::NotThisDoor));
let HuginnMindResponse::Whoami(status) = ask(&mut her, "p-whoami", &HuginnMindRequest::Whoami) else {
panic!("the persona door answers whoami")
};
assert_eq!(status.documents, 0, "the refused admission wrote nothing");
let mut self_faculty = connect(&full, "full");
assert!(matches!(
ask(&mut self_faculty, "f-admit", &admit),
HuginnMindResponse::Admit(PipelineAdmissionOutcome::Committed { .. })
));
stopping.store(true, Ordering::Relaxed);
let (result, mut daemon) = serving.join().unwrap();
result.unwrap();
assert_eq!(documents(&mut daemon), 1, "only the full door's admission landed");
}
/// One client admits batches that close, withdraw and re-open records while three others read
/// the same mind: fresh queries, cursor walks across commits, a hop and whoami. The loop is
/// serial, so every answer is one snapshot's; at quiesce each answer must equal what a fresh
@ -892,7 +977,7 @@ mod tests {
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
let result = run(&mut daemon, &mut hub, &registry, None, &loop_stopping, &ServeOptions::default());
let result = run(&mut daemon, &mut hub, None, &registry, None, &loop_stopping, &ServeOptions::default());
(result, daemon)
});
@ -1057,7 +1142,7 @@ mod tests {
at: DateTime<Utc>,
) -> CultMeshCdnArtifactManifest {
let registry = schema_registry().unwrap();
deferred_manifest(&answer(daemon, &registry, bodies, encode_request(message_id, read, None).unwrap(), at).reply())
deferred_manifest(&answer(daemon, Door::Full, &registry, bodies, encode_request(message_id, read, None).unwrap(), at).reply())
}
/// Whether the first chunk of a manifest is still served at an injected time.
@ -1068,7 +1153,7 @@ mod tests {
at: DateTime<Utc>,
) -> bool {
let registry = schema_registry().unwrap();
found(&answer(daemon, &registry, bodies, chunk_request(manifest, 0), at).reply())
found(&answer(daemon, Door::Full, &registry, bodies, chunk_request(manifest, 0), at).reply())
}
/// An answer larger than one send can carry is deferred to the body plane
@ -1133,7 +1218,7 @@ mod tests {
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
let result = run(&mut daemon, &mut hub, &registry, None, &loop_stopping, &ServeOptions::default());
let result = run(&mut daemon, &mut hub, None, &registry, None, &loop_stopping, &ServeOptions::default());
(result, daemon)
});
@ -1381,7 +1466,7 @@ mod tests {
let manifest = ask_deferred(&mut daemon, &mut bodies, "m-v", &read, t0);
assert!(manifest.chunks.len() >= 2, "the fixture needs a second chunk to serve");
let mut chunk = |index: usize, at: i64| {
answer(&mut daemon, &registry, &mut bodies, chunk_request(&manifest, index), after(at)).reply()
answer(&mut daemon, Door::Full, &registry, &mut bodies, chunk_request(&manifest, index), after(at)).reply()
};
assert!(found(&chunk(0, 59)), "inside the TTL");
assert!(found(&chunk(1, 118)), "serving a chunk slid the expiry past the first TTL");
@ -1394,7 +1479,7 @@ mod tests {
let again = ask_deferred(&mut daemon, &mut bodies, "m-v2", &read, after(179));
assert_eq!(again.content_hash, manifest.content_hash, "the same answer is the same body");
let bytes = fetch_content(&again, MAX_DEFERRED_BODY_BYTES, |request| {
Ok(answer(&mut daemon, &registry, &mut bodies, request, after(180)).reply())
Ok(answer(&mut daemon, Door::Full, &registry, &mut bodies, request, after(180)).reply())
})
.unwrap();
assert_eq!(bytes, expected);
@ -1494,7 +1579,7 @@ mod tests {
("record_key", chunk("m", "abc", &huge)),
];
for (case, message) in refused {
let reply = answer(&mut daemon, &registry, &mut bodies, message, now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut bodies, message, now()).reply();
let CultNetMessage::Error { error, .. } = &reply else {
panic!("{case}: expected a transport error, got {reply:?}");
};
@ -1503,9 +1588,9 @@ mod tests {
}
assert_eq!(documents(&mut daemon), 0);
let reply = answer(&mut daemon, &registry, &mut bodies, whoami(&at, MIND_SERVICE_ID, "whoami", MIND_REQUEST_SCHEMA), now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut bodies, whoami(&at, MIND_SERVICE_ID, "whoami", MIND_REQUEST_SCHEMA), now()).reply();
assert_eq!(decode_response(&reply).unwrap().0, at, "a field of exactly the bound is served");
let reply = answer(&mut daemon, &registry, &mut bodies, chunk(&at, &at, &at), now()).reply();
let reply = answer(&mut daemon, Door::Full, &registry, &mut bodies, chunk(&at, &at, &at), now()).reply();
assert!(!found(&reply), "an unknown chunk is answered found: false, not refused at the door");
}
@ -1565,6 +1650,7 @@ mod tests {
state_root: root.path().to_path_buf(),
instance: slug(INSTANCE),
bind: port,
persona_bind: Some("127.0.0.1:0".parse().unwrap()),
qdrant_url: "http://127.0.0.1:1".into(),
ollama_urls: vec!["http://127.0.0.1:1".into()],
reserve_ollama_urls: Vec::new(),
@ -1588,6 +1674,8 @@ mod tests {
drop(occupied);
let started = started(&options).expect("both gates open");
assert_eq!(started.hub.local_addr().unwrap(), port);
let door = started.persona.as_ref().expect("the persona door was asked for and bound");
assert_ne!(door.local_addr().unwrap(), port, "the persona door is its own listener");
}
#[test]
@ -1604,6 +1692,7 @@ mod tests {
state_root: PathBuf::from(root),
instance: slug("yggdrasil"),
bind: "127.0.0.1:17872".parse().unwrap(),
persona_bind: None,
qdrant_url: "http://127.0.0.1:6333".into(),
ollama_urls: vec!["http://10.77.0.4:11434".into()],
reserve_ollama_urls: Vec::new(),
@ -1647,5 +1736,11 @@ mod tests {
assert_eq!(parse_options(args(&forged)).unwrap().forge_url.as_deref(), Some("http://127.0.0.1:3310"));
assert!(parse_options(args(&[good.to_vec(), vec!["--forge-url", "https://127.0.0.1:3310"]].concat())).is_err());
assert!(parse_options(args(&[forged.clone(), vec!["--forge-url", "http://127.0.0.1:3311"]].concat())).is_err());
// The persona door is optional, once, an ip:port, and never the full door's.
let door = [good.to_vec(), vec!["--persona-bind", "127.0.0.1:17862"]].concat();
assert_eq!(parse_options(args(&door)).unwrap().persona_bind, Some("127.0.0.1:17862".parse().unwrap()));
assert!(parse_options(args(&[door.clone(), vec!["--persona-bind", "127.0.0.1:17863"]].concat())).is_err());
assert!(parse_options(args(&[good.to_vec(), vec!["--persona-bind", "not-an-address"]].concat())).is_err());
assert!(parse_options(args(&[good.to_vec(), vec!["--persona-bind", "127.0.0.1:17872"]].concat())).is_err());
}
}

View file

@ -134,6 +134,39 @@ impl std::fmt::Display for ChatReport {
}
}
/// One line of a chat run's journal, typed so the job routes by what happened and not by how the line reads.
#[derive(Clone, Debug, PartialEq)]
pub enum Entry {
/// A room's report; a failed one is an alarm.
Report(ChatReport),
Refused(RefusedRoom),
Scrubbed(ScrubHit),
/// A failure outside any one room's report: retiring rooms.
Trouble(String),
}
impl Entry {
/// The operator must see this on stderr: a failure, a refusal, a scrub hit.
pub fn is_alarm(&self) -> bool {
match self {
Self::Report(report) => report.failure.is_some(),
Self::Refused(_) | Self::Scrubbed(_) | Self::Trouble(_) => true,
}
}
}
impl std::fmt::Display for Entry {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Report(report) => report.fmt(formatter),
Self::Refused(RefusedRoom { room_id, reason, event_id: None }) => write!(formatter, "refused: {room_id} {reason}"),
Self::Refused(RefusedRoom { room_id, reason, event_id: Some(event) }) => write!(formatter, "refused: {room_id} {reason} [{event}]"),
Self::Scrubbed(hit) => write!(formatter, "scrubbed: {} {} {}", hit.room_id, hit.event_id, hit.kind),
Self::Trouble(why) => write!(formatter, "huginn-sources: {why}"),
}
}
}
/// A joined room the scope refused, whole, and why; `event_id` when an event did it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct RefusedRoom {
@ -562,7 +595,7 @@ pub fn refresh_chat<E: Embedder, S: SourceIndex, H: RoomHistory>(
index: &mut S,
history: &mut H,
estate: &Estate,
journal: &mut dyn FnMut(String),
journal: &mut dyn FnMut(Entry),
) -> Result<ChatRefreshed> {
let DescribedSources::Labelled { meta, .. } = index.describe_sources(INDEX_CALL_TIMEOUT).context("describing the source collection")? else {
bail!("the source collection is absent or not Huginn's; the forge refresh creates it and chat never does");
@ -589,13 +622,14 @@ pub fn refresh_chat<E: Embedder, S: SourceIndex, H: RoomHistory>(
Ok(Err(reason)) => {
// Not kept, so the retirement below purges whatever it held.
keep.remove(room_id);
journal(format!("refused: {room_id} {reason}"));
out.refused.push(RefusedRoom { room_id: room_id.clone(), reason, event_id: None });
let refused = RefusedRoom { room_id: room_id.clone(), reason, event_id: None };
journal(Entry::Refused(refused.clone()));
out.refused.push(refused);
}
// An error is not a reclassification: the room keeps its points and cursor, and the report says why.
Err(error) => {
let report = ChatReport::failed(room_id, None, &error.context("reading the room's facts"));
journal(report.to_string());
journal(Entry::Report(report.clone()));
out.failed.push(room_id.clone());
out.rooms.push(report);
}
@ -604,35 +638,35 @@ pub fn refresh_chat<E: Embedder, S: SourceIndex, H: RoomHistory>(
match retire_chat(index, &keep) {
Ok(done) => {
for (room, why) in done.failed {
journal(format!("huginn-sources: retiring {room} failed, the others go on: {why}"));
journal(Entry::Trouble(format!("retiring {room} failed, the others go on: {why}")));
out.failed.push(room);
}
}
Err(error) => {
journal(format!("huginn-sources: retiring rooms failed: {error:#}"));
journal(Entry::Trouble(format!("retiring rooms failed: {error:#}")));
out.failed.push("retire".to_owned());
}
}
for facts in admitted {
match ingest_room(embedder, index, history, identity.dimensions, &facts, &mut out) {
Ok(Outcome::Done(report)) => {
journal(report.to_string());
journal(Entry::Report(report.clone()));
out.rooms.push(report);
}
Ok(Outcome::Refused(refused)) => {
journal(format!("refused: {} {} [{}]", refused.room_id, refused.reason, refused.event_id.as_deref().unwrap_or("")));
journal(Entry::Refused(refused.clone()));
out.refused.push(refused);
}
Err(error) => {
let report = ChatReport::failed(&facts.room_id, facts.alias.clone(), &error);
journal(report.to_string());
journal(Entry::Report(report.clone()));
out.failed.push(facts.room_id.clone());
out.rooms.push(report);
}
}
}
for hit in &out.scrubbed {
journal(format!("scrubbed: {} {} {}", hit.room_id, hit.event_id, hit.kind));
journal(Entry::Scrubbed(hit.clone()));
}
Ok(out)
}

View file

@ -141,7 +141,7 @@ impl Rig {
fn run(&mut self, history: &mut FakeHistory) -> (Result<ChatRefreshed>, Vec<String>) {
let mut journal = Vec::new();
let out = refresh_chat(&mut self.embedder, &mut self.index, history, &self.estate, &mut |line| journal.push(line));
let out = refresh_chat(&mut self.embedder, &mut self.index, history, &self.estate, &mut |line| journal.push(line.to_string()));
(out, journal)
}

View file

@ -180,7 +180,7 @@ impl HttpForge {
capped.push(repo.clone());
}
}
events.sort_by(|a, b| b.id.cmp(&a.id));
events.sort_by_key(|event| std::cmp::Reverse(event.id));
// The cursor never moves backwards: a read that saw nothing new keeps the one it was given.
head = head.max(after.unwrap_or(0));
let (mut open_prs, mut unseen) = (Vec::new(), dark.clone());
@ -251,7 +251,7 @@ fn redirect_target(base: &str, location: &str) -> Option<OrgRepo> {
let at = segments.windows(3).position(|window| window == ["api", "v1", "repos"])?;
let plain = |segment: &&str| !segment.is_empty() && *segment != "." && *segment != ".." && segment.chars().all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-'));
let [owner, name, "activities", "feeds"] = segments[at + 3..] else { return None };
if !segments[..at].iter().all(|segment| plain(segment)) || !plain(&owner) || !plain(&name) {
if !segments[..at].iter().all(plain) || !plain(&owner) || !plain(&name) {
return None;
}
let repo = OrgRepo(format!("{owner}/{name}"));
@ -322,6 +322,7 @@ mod tests {
use serde_json::json;
use super::*;
use crate::sources::test_http::{Reply, forge_of};
/// Only public, live, original, non-empty repos are the corpus; a repo the
/// forge does not say is safe is not indexed.
@ -400,60 +401,6 @@ mod tests {
assert!(HttpForge::new(&unstated).catalog().unwrap().shortfall.unwrap().contains("X-Total-Count"));
}
/// What a fake forge answers: a status, the `Location` header when there is one, and a body.
struct Reply {
status: u16,
location: Option<String>,
body: String,
}
impl From<String> for Reply {
fn from(body: String) -> Self {
Self { status: 200, location: None, body }
}
}
impl Reply {
/// What the live forge answers an anonymous reader for a private repo.
fn not_found() -> Self {
Self { status: 404, location: None, body: json!({ "message": "The target couldn't be found." }).to_string() }
}
/// What the live forge answers for a renamed repo: a 301 to its public address, `/forge` prefix and all.
fn moved_to(location: &str) -> Self {
Self { status: 301, location: Some(location.to_owned()), body: String::new() }
}
}
/// Answers each request by its path from `routes`; a path it has no answer for is held open and never answered, as a hung forge
/// does. Every path asked is recorded in order.
fn forge_of<R: Into<Reply>>(routes: impl Fn(&str) -> Option<R> + Send + 'static) -> (String, std::sync::Arc<std::sync::Mutex<Vec<String>>>) {
use std::io::{Read, Write};
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}", listener.local_addr().unwrap());
let asked = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let record = std::sync::Arc::clone(&asked);
std::thread::spawn(move || {
let mut held = Vec::new();
for stream in listener.incoming() {
let mut stream = stream.unwrap();
let mut buffer = [0_u8; 4096];
let read = stream.read(&mut buffer).unwrap();
let request = String::from_utf8_lossy(&buffer[..read]).into_owned();
let path = request.split_whitespace().nth(1).unwrap_or_default().to_owned();
record.lock().unwrap().push(path.clone());
match routes(&path).map(Into::into) {
Some(Reply { status, location, body }) => {
let location = location.map(|location| format!("location: {location}\r\n")).unwrap_or_default();
write!(stream, "HTTP/1.1 {status} X\r\ncontent-type: application/json\r\n{location}content-length: {}\r\nconnection: close\r\n\r\n{body}", body.len()).unwrap();
}
None => held.push(stream),
}
}
});
(base, asked)
}
const FEED_SAMPLE: &str = include_str!("../../tests/fixtures/feed.json");
const NOW: &str = "2026-11-03T12:00:00Z";
@ -523,7 +470,7 @@ mod tests {
assert!(read.open_prs.is_empty() && read.unseen.is_empty() && read.capped.is_empty() && read.moved.is_empty(), "a cursor asks for neither list");
let wire = serde_json::to_string(&read).unwrap();
assert!(!wire.contains("CANARY") && !wire.contains("obey") && !wire.contains("Message"), "no forge text leaves the read: {wire}");
let asked = asked.lock().unwrap().clone();
let asked = asked.paths();
assert!(!asked.is_empty() && asked.iter().all(|path| names.iter().any(|name| path.starts_with(&format!("/api/v1/repos/{name}/activities/feeds")))), "{asked:?}");
}
@ -555,13 +502,13 @@ mod tests {
let far = HttpForge::new(&base).activity(&wanted, &me, Some(5), now).unwrap();
assert!(far.capped == repos(&["GameCult/Huginn"]), "stopped at the cap with the cursor not reached");
assert_eq!((far.events.len(), far.head), (200, 2500));
assert_eq!(asked.lock().unwrap().len(), 4, "four pages and no more");
assert_eq!(asked.paths().len(), 4, "four pages and no more");
let (base, asked) = forge_of(feed("nibu"));
let near = HttpForge::new(&base).activity(&wanted, &me, Some(2300), now).unwrap();
assert!(near.capped.is_empty());
assert_eq!((near.events.len(), near.head), (20, 2500), "ids above the cursor only");
assert_eq!(asked.lock().unwrap().len(), 1);
assert_eq!(asked.paths().len(), 1);
let (base, _) = forge_of(feed("eureka"));
let hers = HttpForge::new(&base).activity(&wanted, &me, Some(2300), now).unwrap();
@ -583,7 +530,7 @@ mod tests {
let read = HttpForge::new(&base).activity(&repos(&["GameCult/Ratatoskr"]), &me(), None, at(NOW)).unwrap();
assert_eq!(read.events.iter().map(|event| event.id).collect::<Vec<_>>(), [300]);
assert!(read.capped.is_empty());
assert!(asked.lock().unwrap().iter().all(|path| !path.contains("/orgs/GameCult/activities")));
assert!(asked.paths().iter().all(|path| !path.contains("/orgs/GameCult/activities")));
}
/// A first read keeps what is inside 72 hours and nothing older, and its cursor is the newest id it saw, an entry past the window
@ -619,13 +566,13 @@ mod tests {
let (base, asked) = forge_of(move |path| if is_feed(path) { repo_feed(&entries, path) } else { Some("[]".into()) });
let read = HttpForge::new(&base).activity(&wanted, &me(), None, at(NOW)).unwrap();
assert_eq!((read.events.len(), read.capped.len()), (53, 0));
assert_eq!(asked.lock().unwrap().iter().filter(|path| is_feed(path)).count(), 2, "page 2 was asked, page 3 was not");
assert_eq!(asked.paths().iter().filter(|path| is_feed(path)).count(), 2, "page 2 was asked, page 3 was not");
let entries: Vec<Value> = (0..400).map(|n| push_at(5000 - n, "GameCult/Huginn", "Nibu", 5)).collect();
let (base, asked) = forge_of(move |path| if is_feed(path) { repo_feed(&entries, path) } else { Some("[]".into()) });
let read = HttpForge::new(&base).activity(&wanted, &me(), None, at(NOW)).unwrap();
assert_eq!((read.events.len(), read.capped), (REPO_FEED_PAGES * PAGE, repos(&["GameCult/Huginn"])), "200 read and the rest disclosed");
assert_eq!(asked.lock().unwrap().iter().filter(|path| is_feed(path)).count(), REPO_FEED_PAGES);
assert_eq!(asked.paths().iter().filter(|path| is_feed(path)).count(), REPO_FEED_PAGES);
}
/// With no cursor the read also lists the open PRs in her repos and hers anywhere, and the repos the forge's listing does not hold.
@ -636,9 +583,9 @@ mod tests {
let mut first_page: Vec<Value> = (0..PAGE as u32).map(|n| pr(n + 100, "kiko", "GameCult/Elsewhere")).collect();
first_page[0] = pr(8, "nibu", "GameCult/Ratatoskr");
first_page[1] = pr(42, "Eureka", "GameCult/Eureka");
let search = vec![Value::Array(first_page).to_string(), Value::Array(vec![pr(9, "void", "GameCult/Huginn")]).to_string()];
let search = [Value::Array(first_page).to_string(), Value::Array(vec![pr(9, "void", "GameCult/Huginn")]).to_string()];
let names: Vec<Value> = (0..61).map(|n| json!({ "full_name": match n { 0 => "GameCult/ratatoskr".to_owned(), 1 => "GameCult/Huginn".to_owned(), n => format!("GameCult/r{n}") } })).collect();
let listing = vec![Value::Array(names[..PAGE].to_vec()).to_string(), Value::Array(names[PAGE..].to_vec()).to_string()];
let listing = [Value::Array(names[..PAGE].to_vec()).to_string(), Value::Array(names[PAGE..].to_vec()).to_string()];
let sample = sample();
let (base, asked) = forge_of(move |path| {
if is_feed(path) && path.contains("/GameCult/Ghost/") {
@ -660,14 +607,14 @@ mod tests {
assert_eq!(first.unseen, repos(&["GameCult/Ghost"]), "r60 is on page 2 of the listing");
let window = at("2026-10-31T12:00:00Z").timestamp_millis() as u64;
assert!(!first.events.is_empty() && first.events.iter().all(|event| event.at >= window), "inside 72 h: {:?}", first.events);
let paths = asked.lock().unwrap().clone();
let paths = asked.paths();
let count = |what: &str| paths.iter().filter(|path| path.contains(what)).count();
assert_eq!((count("/issues/search"), count("/orgs/GameCult/repos")), (2, 2), "a short page ends each: {paths:?}");
// A cursor asks for neither list.
let before = paths.len();
HttpForge::new(&base).activity(&wanted, &me(), Some(1), at(NOW)).unwrap();
let after = asked.lock().unwrap().clone();
let after = asked.paths();
assert!(after[before..].iter().all(|path| is_feed(path)), "only the feeds: {after:?}");
}
@ -679,7 +626,7 @@ mod tests {
let full = |from: u32| Value::Array((0..PAGE as u32).map(|n| pr(from + n, "GameCult/Elsewhere")).collect()).to_string();
let mut third: Vec<Value> = (0..PAGE as u32).map(|n| pr(300 + n, "GameCult/Elsewhere")).collect();
third[PAGE - 1] = pr(7, "GameCult/Huginn");
let search = vec![full(100), full(200), Value::Array(third).to_string(), "[]".to_owned()];
let search = [full(100), full(200), Value::Array(third).to_string(), "[]".to_owned()];
let (base, asked) = forge_of(move |path| {
if is_feed(path) {
Some("[]".to_owned())
@ -693,12 +640,12 @@ mod tests {
});
let read = HttpForge::new(&base).activity(&repos(&["GameCult/Huginn"]), &me(), None, at(NOW)).unwrap();
assert_eq!(read.open_prs.iter().map(|pr| pr.number).collect::<Vec<_>>(), [7], "page 3 was read");
assert_eq!(asked.lock().unwrap().iter().filter(|path| path.contains("/issues/search")).count(), 4, "to the short page");
assert_eq!(asked.paths().iter().filter(|path| path.contains("/issues/search")).count(), 4, "to the short page");
}
/// The forge as it answers an anonymous reader: public repos give their feed (`[]` when quiet), a private repo answers 404, a
/// renamed repo answers 301 to a path under `/forge` that this loopback does not serve (asking it gives 404).
fn live_forge(entries: Vec<Value>, public: &'static [&'static str], listing: String) -> (String, std::sync::Arc<std::sync::Mutex<Vec<String>>>) {
fn live_forge(entries: Vec<Value>, public: &'static [&'static str], listing: String) -> (String, crate::sources::test_http::Asked) {
forge_of(move |path| {
if path.starts_with("/forge/") {
return Some(Reply::not_found());
@ -733,7 +680,7 @@ mod tests {
assert_eq!(first.unseen, repos(&["GameCult/gamecult-ops", "GameCult/cinny"]));
assert_eq!(first.moved, [RepoMoved { from: "GameCult/cinny".into(), to: "GameCult/Ratatoskr".into() }]);
assert!(first.capped.is_empty());
assert!(asked.lock().unwrap().iter().all(|path| !path.starts_with("/forge/")), "a redirect is not followed: {:?}", asked.lock().unwrap());
assert!(asked.paths().iter().all(|path| !path.starts_with("/forge/")), "a redirect is not followed: {:?}", asked.paths());
let later = forge.activity(&wanted, &me(), Some(250), at(NOW)).unwrap();
assert_eq!((later.events.iter().map(|event| event.id).collect::<Vec<_>>(), later.head), (vec![300], 300), "a later read goes on past the dark repos");
@ -799,9 +746,7 @@ mod tests {
let (base, _) = forge_of(move |path| {
if is_feed(path) && path.contains("/GameCult/cinny/") {
Some(dark())
} else if is_feed(path) {
Some("[]".to_owned().into())
} else if path.contains("/issues/search") {
} else if is_feed(path) || path.contains("/issues/search") {
Some("[]".to_owned().into())
} else {
Some(listing.clone().into())

View file

@ -0,0 +1,128 @@
//! What one `huginn-sources` run does with the pieces it is handed: the forge
//! refresh, then the chat refresh (or, with chat off, the retirement of every
//! chat point), one exit status for both, and the steward-purge request. The
//! binary supplies the real forge, embedder, vector store and Matrix client and
//! owns the flags and the lock; this file decides nothing about scope or
//! scrubbing (`room_scope` and `redact` do) and never sees a credential.
use std::collections::BTreeSet;
use std::path::Path;
use anyhow::{Context, Result, bail, ensure};
use chrono::{DateTime, Utc};
use super::chat::{Entry, RoomHistory, refresh_chat, retire_chat};
use super::chat_purge::{PurgeWindow, parse_windows, purge_key, request_purge};
use super::room_scope::Estate;
use super::{DescribedSources, Forge, SourceIndex, refresh};
use crate::index::{Embedder, INDEX_CALL_TIMEOUT};
/// Chat on: the reader's rooms and the estate ids the scope refuses.
pub struct Chat<'a, H> {
pub history: &'a mut H,
pub estate: &'a Estate,
}
/// One run: the forge refresh, then chat (`Some`) or the retirement of every chat point (`None`).
/// `say` takes the journal's report lines (stdout), `complain` the failures, refusals and scrub hits (stderr).
/// A forge failure never skips the chat run and a chat failure never hides a forge one.
/// Returns the exit status: 0 only when both were clean.
pub fn run<E: Embedder, S: SourceIndex, F: Forge, H: RoomHistory>(
embedder: &mut E,
index: &mut S,
forge: &mut F,
state_root: &Path,
chat: Option<Chat<'_, H>>,
say: &mut dyn FnMut(String),
complain: &mut dyn FnMut(String),
) -> u8 {
let mut clean = true;
match refresh(embedder, index, forge, state_root, &mut |report| say(report.to_string())) {
Ok(refreshed) if refreshed.is_clean() => {}
Ok(refreshed) => {
clean = false;
if !refreshed.failed.is_empty() {
complain(format!("huginn-sources: {} repos failed: {}", refreshed.failed.len(), refreshed.failed.join(", ")));
}
if let Some(why) = &refreshed.retirement_withheld {
complain(format!("huginn-sources: no repo was retired: {why}"));
}
}
Err(error) => {
clean = false;
complain(format!("huginn-sources: {error:#}"));
}
}
match chat {
Some(Chat { history, estate }) => {
let mut line = |entry: Entry| if entry.is_alarm() { complain(entry.to_string()) } else { say(entry.to_string()) };
match refresh_chat(embedder, index, history, estate, &mut line) {
Ok(refreshed) => clean &= refreshed.is_clean(),
Err(error) => {
clean = false;
complain(format!("huginn-sources: chat: {error:#}"));
}
}
}
None => match retire_chat(index, &BTreeSet::new()) {
Ok(done) => {
if !done.retired.is_empty() {
say(format!("chat retired: {} rooms", done.retired.len()));
}
for (room, why) in done.failed {
clean = false;
complain(format!("huginn-sources: retiring {room} failed: {why}"));
}
}
Err(error) => {
clean = false;
complain(format!("huginn-sources: chat: {error:#}"));
}
},
}
u8::from(!clean)
}
/// The longest span one request may name.
const PURGE_SPAN_DAYS: i64 = 366;
const REQUEST_BYTES_MAX: usize = 120;
/// A steward's purge request as the admin typed it, checked. Each error names its option.
#[derive(Debug, PartialEq, Eq)]
pub struct Purge {
pub room_id: String,
pub from: String,
pub to: String,
pub request: String,
pub window: PurgeWindow,
}
fn instant(option: &str, text: &str) -> Result<DateTime<Utc>> {
ensure!(text.ends_with('Z'), "--{option} must be an RFC 3339 UTC time ending in Z, got {text:?}");
let at = DateTime::parse_from_rfc3339(text).map_err(|_| anyhow::anyhow!("--{option} must be an RFC 3339 UTC time ending in Z, got {text:?}"))?;
Ok(at.with_timezone(&Utc))
}
impl Purge {
pub fn check(room: &str, from: &str, to: &str, request: &str, now: DateTime<Utc>) -> Result<Self> {
ensure!(room.starts_with('!'), "--room must be a room id beginning with !, got {room:?}: an alias needs Matrix, and the journal prints both #alias (!id)");
let (start, end) = (instant("from", from)?, instant("to", to)?);
ensure!(start < end, "--from must be before --to");
ensure!(end <= now, "--to must not be after now");
ensure!(end - start <= chrono::Duration::days(PURGE_SPAN_DAYS), "the window must be at most {PURGE_SPAN_DAYS} days");
ensure!(!request.is_empty() && request.len() <= REQUEST_BYTES_MAX, "--request must be 1 to {REQUEST_BYTES_MAX} bytes");
let ms = |at: DateTime<Utc>| u64::try_from(at.timestamp_millis()).context("a purge time before 1970");
Ok(Self { room_id: room.to_owned(), from: from.to_owned(), to: to.to_owned(), request: request.to_owned(), window: PurgeWindow { from_ms: ms(start)?, to_ms: ms(end)? } })
}
}
/// Writes the tombstone and returns the line the admin sees; the free-text request is quoted and escaped so it cannot forge a journal line. Touches no point and makes no Matrix call;
/// the caller holds the job's lock. The next run applies it.
pub fn purge<S: SourceIndex>(index: &mut S, purge: &Purge) -> Result<String> {
request_purge(index, &purge.room_id, purge.window)?;
let DescribedSources::Labelled { cursors, .. } = index.describe_sources(INDEX_CALL_TIMEOUT).context("describing the source collection")? else {
bail!("the source collection is absent or not Huginn's");
};
let windows = cursors.get(&purge_key(&purge.room_id)).map(|held| parse_windows(held)).transpose()?.map_or(0, |held| held.len());
Ok(format!("chat-purge requested: {} {}..{} request={:?} windows={windows}", purge.room_id, purge.from, purge.to, purge.request))
}

View file

@ -0,0 +1,287 @@
//! One run of the job: forge refresh and chat refresh over a fake Synapse reached through the real client,
//! a fake vector store and the fake embedder; the exit status; the chat-off retirement; the purge request.
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
use anyhow::{Result, bail};
use chrono::{DateTime, Utc};
use serde_json::json;
use super::chat::{CHAT_PURGE_PREFIX, RoomHistory};
use super::job::{Chat, Purge, purge, run};
use super::matrix::HttpMatrix;
use super::matrix_tests::{Room, Rooms, TOKEN, message, state, synapse};
use super::room_scope::Estate;
use super::tests::{FakeForge, Remote};
use super::test_http::forge_of;
use super::*;
use crate::index::fakes::{FakeEmbedder, FakeSources};
const DAY0: u64 = 1_794_268_800_000; // 2026-11-10T00:00:00Z
fn now() -> DateTime<Utc> {
DateTime::parse_from_rfc3339("2026-11-17T00:00:00Z").unwrap().with_timezone(&Utc)
}
fn estate() -> Estate {
Estate::new(["!e1:s", "!e2:s", "!e3:s"].map(str::to_owned).into()).unwrap()
}
struct Out {
said: Vec<String>,
complained: Vec<String>,
status: u8,
}
struct Rig {
embedder: FakeEmbedder,
index: FakeSources,
forge: FakeForge,
_remote: Remote,
root: tempfile::TempDir,
}
impl Rig {
fn new() -> Self {
let remote = Remote::new();
remote.commit(&[("README.md", b"# Demo\n")], &[]);
Self { embedder: FakeEmbedder::new("d1"), index: FakeSources::default(), forge: FakeForge::of(&[("Demo", &remote)]), _remote: remote, root: tempfile::tempdir().unwrap() }
}
fn go<H: RoomHistory>(&mut self, history: Option<&mut H>) -> Out {
let (mut said, mut complained) = (Vec::new(), Vec::new());
let estate = estate();
let chat = history.map(|history| Chat { history, estate: &estate });
let status = run(&mut self.embedder, &mut self.index, &mut self.forge, self.root.path(), chat, &mut |line| said.push(line), &mut |line| complained.push(line));
Out { said, complained, status }
}
fn chat_bodies(&self) -> Vec<String> {
let mut bodies: Vec<String> = self.index.state.lock().unwrap().chat.values().flat_map(|point| point.payload.messages.iter().map(|m| m.body.clone())).collect();
bodies.sort();
bodies
}
fn chat_rooms(&self) -> std::collections::BTreeSet<String> {
self.index.state.lock().unwrap().chat.values().map(|point| point.payload.room_id.clone()).collect()
}
}
fn rooms(list: Vec<(&str, Vec<serde_json::Value>, Vec<serde_json::Value>)>) -> Rooms {
Arc::new(Mutex::new(list.into_iter().map(|(id, state, timeline)| (id.to_owned(), Room { state, timeline })).collect::<BTreeMap<_, _>>()))
}
fn platform() -> String {
["disc", "ord"].concat()
}
fn world() -> Rooms {
let relay = format!("@_{}_1:s", platform());
rooms(vec![
("!swarm:s", vec![state("m.room.canonical_alias", json!({ "alias": "#swarm:gamecult.org" })), state("m.room.join_rules", json!({ "join_rule": "invite" }))], vec![
message("$m1", "@a:s", DAY0 - 5000, "first words"),
message("$m2", "@b:s", DAY0 - 4000, "to be redacted"),
message("$m3", "@a:s", DAY0 + 1000, "inside the window"),
]),
("!bridged:s", vec![state("m.bridge", json!({}))], vec![message("$b1", "@a:s", DAY0, "bridged talk")]),
("!relay:s", vec![], vec![message("$d1", &relay, DAY0, "relayed talk")]),
("!aliased:s", vec![state("m.room.canonical_alias", json!({ "alias": "#Users:gamecult.org" }))], vec![message("$u1", "@a:s", DAY0, "estate talk")]),
])
}
fn all_lines(out: &Out) -> String {
out.said.iter().chain(&out.complained).cloned().collect::<Vec<_>>().join("\n")
}
#[test]
fn a_full_run_over_a_fake_synapse() {
let world = world();
let (base, asked) = synapse(&world);
let mut matrix = HttpMatrix::new(&base, TOKEN);
let mut rig = Rig::new();
let first = rig.go(Some(&mut matrix));
assert_eq!(first.status, 1, "joined bridged, relay and estate rooms are refused: {:?}", first.complained);
assert_eq!(rig.chat_rooms(), ["!swarm:s".to_owned()].into(), "an invite-only room is ingested; nothing of the refused rooms is");
assert_eq!(rig.chat_bodies(), ["first words", "inside the window", "to be redacted"]);
for refused in ["!bridged:s", "!relay:s", "!aliased:s"] {
assert!(first.complained.iter().any(|line| line.starts_with("refused:") && line.contains(refused)), "{refused}: {:?}", first.complained);
}
assert!(!rig.embedder.texts().iter().any(|text| text.contains("bridged talk") || text.contains("relayed talk") || text.contains("estate talk")));
assert!(first.said.iter().any(|line| line.starts_with("Demo ")), "the forge's line is journaled too");
assert!(first.said.iter().any(|line| line.starts_with("#swarm:gamecult.org (!swarm:s)") && line.contains("events=3")), "{:?}", first.said);
assert!(asked.seen().iter().all(|seen| seen.method == "GET"));
// The steward asks for the window; then the room edits, redacts and leaks a key.
let request = Purge::check("!swarm:s", "2026-11-10T00:00:00Z", "2026-11-11T00:00:00Z", "follow_up:x", now()).unwrap();
assert_eq!(purge(&mut rig.index, &request).unwrap(), "chat-purge requested: !swarm:s 2026-11-10T00:00:00Z..2026-11-11T00:00:00Z request=\"follow_up:x\" windows=1");
let before = rig.index.cursors();
assert!(purge(&mut rig.index, &request).unwrap().ends_with("windows=1"));
assert_eq!(rig.index.cursors(), before, "a repeated request writes nothing");
let secret = format!("key ghp_{}", "k2L9".repeat(9));
{
let mut held = world.lock().unwrap();
let room = held.get_mut("!swarm:s").unwrap();
room.timeline.push(json!({ "type": "m.room.message", "event_id": "$edit", "sender": "@a:s", "origin_server_ts": DAY0 + 90_000_000, "content": {
"msgtype": "m.text", "body": "* first words, amended", "m.new_content": { "msgtype": "m.text", "body": "first words, amended" }, "m.relates_to": { "rel_type": "m.replace", "event_id": "$m1" } } }));
room.timeline.push(json!({ "type": "m.room.redaction", "event_id": "$red", "sender": "@b:s", "origin_server_ts": DAY0 + 91_000_000, "content": { "redacts": "$m2" } }));
room.timeline.push(message("$m4", "@a:s", DAY0 + 92_000_000, &secret));
}
let second = rig.go(Some(&mut matrix));
assert_eq!(rig.chat_bodies().len(), 2, "{:?}", rig.chat_bodies());
let bodies = rig.chat_bodies().join("\n");
assert!(bodies.contains("first words, amended") && !bodies.contains("to be redacted") && !bodies.contains("inside the window"), "{bodies}");
assert!(!bodies.contains("ghp_k2L9") && !rig.embedder.texts().iter().any(|text| text.contains("ghp_k2L9")), "the key was scrubbed before it was embedded");
let text = all_lines(&second);
assert!(second.said.iter().any(|line| line.contains("purged=")), "{text}");
assert!(second.complained.iter().any(|line| line == "scrubbed: !swarm:s $m4 key-shape" || line.starts_with("scrubbed: !swarm:s $m4 ")), "{text}");
for body in ["first words", "to be redacted", "inside the window", "relayed talk", "ghp_"] {
assert!(!text.contains(body), "the journal carries ids and counts, not bodies: {body}");
}
let third = rig.go(Some(&mut matrix));
assert!(!rig.chat_bodies().join("\n").contains("inside the window"), "a purged window does not come back");
assert!(third.complained.iter().all(|line| line.starts_with("refused:")), "{:?}", third.complained);
}
#[test]
fn no_chat_flags_purge_the_corpus_and_make_no_matrix_call() {
let world = world();
let (base, _) = synapse(&world);
let mut matrix = HttpMatrix::new(&base, TOKEN);
let mut rig = Rig::new();
rig.go(Some(&mut matrix));
let request = Purge::check("!swarm:s", "2026-11-10T00:00:00Z", "2026-11-11T00:00:00Z", "r", now()).unwrap();
purge(&mut rig.index, &request).unwrap();
assert!(!rig.chat_bodies().is_empty());
let (hung, asked) = forge_of(|_| None::<String>);
drop(hung);
let out = rig.go::<HttpMatrix>(None);
assert_eq!(out.status, 0, "{:?}", out.complained);
assert!(rig.chat_bodies().is_empty() && rig.chat_rooms().is_empty());
assert!(rig.index.cursors().keys().any(|key| key.starts_with(CHAT_PURGE_PREFIX)), "tombstones stay");
assert!(!rig.index.cursors().keys().any(|key| key.starts_with("chat:")), "cursors go");
assert!(asked.seen().is_empty());
}
struct BrokenForge;
impl Forge for BrokenForge {
fn catalog(&mut self) -> Result<Catalog> {
bail!("the forge is down")
}
fn clone_url(&self, _name: &str) -> String {
String::new()
}
}
struct DeadSynapse;
impl RoomHistory for DeadSynapse {
fn joined_rooms(&mut self) -> Result<Vec<String>> {
bail!("synapse is down")
}
fn facts(&mut self, _room_id: &str) -> Result<super::room_scope::RoomFacts> {
bail!("synapse is down")
}
fn events_after(&mut self, _room_id: &str, _after: Option<&str>, _limit: u32) -> Result<Option<super::chat::EventPage>> {
bail!("synapse is down")
}
}
#[test]
fn exit_status_is_clean_only_when_forge_and_chat_both_are() {
let clean = rooms(vec![("!swarm:s", vec![], vec![message("$m1", "@a:s", 1, "words")])]);
let (base, _) = synapse(&clean);
let mut rig = Rig::new();
let out = rig.go(Some(&mut HttpMatrix::new(&base, TOKEN)));
assert_eq!((out.status, out.complained.len()), (0, 0));
// Scrub hits alone are not a failure; they are reported.
clean.lock().unwrap().get_mut("!swarm:s").unwrap().timeline.push(message("$m2", "@a:s", 2, &format!("ghp_{}", "k2L9".repeat(9))));
let out = rig.go(Some(&mut HttpMatrix::new(&base, TOKEN)));
assert_eq!(out.status, 0, "{:?}", out.complained);
assert!(out.complained.iter().any(|line| line.starts_with("scrubbed: !swarm:s $m2")));
// A chat failure still reports the forge, and a forge failure still runs chat.
let out = rig.go(Some(&mut DeadSynapse));
assert_eq!(out.status, 1);
assert!(out.complained.iter().any(|line| line.contains("synapse is down")));
let (mut said, mut complained) = (Vec::new(), Vec::new());
let (estate, mut ok) = (estate(), HttpMatrix::new(&base, TOKEN));
let status = run(&mut rig.embedder, &mut rig.index, &mut BrokenForge, rig.root.path(), Some(Chat { history: &mut ok, estate: &estate }), &mut |l| said.push(l), &mut |l| complained.push(l));
assert_eq!(status, 1);
assert!(complained.iter().any(|line| line.contains("the forge is down")), "{complained:?}");
assert!(said.iter().any(|line| line.contains("(!swarm:s)")), "chat ran after the forge failed: {said:?}");
}
#[test]
fn a_request_cannot_forge_a_journal_line() {
let mut rig = Rig::new();
rig.go::<HttpMatrix>(None);
let request = Purge::check("!swarm:s", "2026-11-10T00:00:00Z", "2026-11-11T00:00:00Z", "x\nrefused: !forged:s", now()).unwrap();
let line = purge(&mut rig.index, &request).unwrap();
assert_eq!(line.lines().count(), 1, "{line:?}");
}
#[test]
fn a_failed_room_report_goes_to_the_complaint_stream_and_fails_the_run() {
let mut rig = Rig::new();
struct FactsDown;
impl RoomHistory for FactsDown {
fn joined_rooms(&mut self) -> Result<Vec<String>> {
Ok(vec!["!swarm:s".to_owned()])
}
fn facts(&mut self, _room_id: &str) -> Result<super::room_scope::RoomFacts> {
bail!("facts are down")
}
fn events_after(&mut self, _room_id: &str, _after: Option<&str>, _limit: u32) -> Result<Option<super::chat::EventPage>> {
bail!("unreachable")
}
}
let out = rig.go(Some(&mut FactsDown));
assert_eq!(out.status, 1);
assert!(out.complained.iter().any(|line| line.contains("!swarm:s") && line.contains("facts are down")), "{:?}", out.complained);
assert!(!out.said.iter().any(|line| line.contains("facts are down")), "{:?}", out.said);
}
#[test]
fn chat_purge_makes_no_matrix_request() {
let world = world();
let (base, asked) = synapse(&world);
let mut rig = Rig::new();
rig.go(Some(&mut HttpMatrix::new(&base, TOKEN)));
let made = asked.seen().len();
let request = Purge::check("!swarm:s", "2026-11-10T00:00:00Z", "2026-11-11T00:00:00Z", "r", now()).unwrap();
purge(&mut rig.index, &request).unwrap();
assert_eq!(asked.seen().len(), made);
}
#[test]
fn a_retirement_failure_goes_to_the_complaint_stream() {
struct Left;
impl RoomHistory for Left {
fn joined_rooms(&mut self) -> Result<Vec<String>> {
Ok(vec![])
}
fn facts(&mut self, _room_id: &str) -> Result<super::room_scope::RoomFacts> {
bail!("unreachable")
}
fn events_after(&mut self, _room_id: &str, _after: Option<&str>, _limit: u32) -> Result<Option<super::chat::EventPage>> {
bail!("unreachable")
}
}
let world = rooms(vec![("!swarm:s", vec![], vec![message("$m1", "@a:s", 1, "words")])]);
let (base, _) = synapse(&world);
let mut rig = Rig::new();
rig.go(Some(&mut HttpMatrix::new(&base, TOKEN)));
rig.index.state.lock().unwrap().fail_chat_deletes = true;
let out = rig.go(Some(&mut Left));
assert_eq!(out.status, 1);
assert!(out.complained.iter().any(|line| line.starts_with("huginn-sources: retiring !swarm:s failed")), "{:?}", out.complained);
assert!(!out.said.iter().any(|line| line.contains("retiring")), "{:?}", out.said);
}

View file

@ -0,0 +1,159 @@
//! Synapse's client API as the reader user sees it: Matrix JSON in, `RoomEvent`
//! and `RoomFacts` out. JSON for Matrix is spoken here and nowhere else. This
//! client reads with GET only and holds no policy: it filters nothing, decides
//! no room's or sender's scope, and names no platform. What is in scope is
//! `room_scope`'s; the token is read at one place, the `Authorization` header.
use std::time::Duration;
use anyhow::{Context, Result, bail};
use serde_json::Value;
use super::chat::{EventKind, EventPage, RoomEvent, RoomHistory};
use super::room_scope::RoomFacts;
const CALL_TIMEOUT: Duration = Duration::from_secs(60);
/// A bridge's state event types: either marks a room as relaying another platform.
const BRIDGE_TYPES: [&str; 2] = ["m.bridge", "uk.half-shot.bridge"];
pub struct HttpMatrix {
agent: ureq::Agent,
base: String,
token: String,
}
impl std::fmt::Debug for HttpMatrix {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.debug_struct("HttpMatrix").field("base", &self.base).finish_non_exhaustive()
}
}
/// A path segment with everything but unreserved characters escaped.
fn segment(raw: &str) -> String {
let mut out = String::with_capacity(raw.len());
for byte in raw.bytes() {
if byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.' | b'~') {
out.push(char::from(byte));
} else {
out.push_str(&format!("%{byte:02X}"));
}
}
out
}
impl HttpMatrix {
pub fn new(base_url: &str, token: &str) -> Self {
let agent: ureq::Agent =
ureq::Agent::config_builder().timeout_global(Some(CALL_TIMEOUT)).http_status_as_error(false).build().into();
Self { agent, base: base_url.trim_end_matches('/').to_owned(), token: token.to_owned() }
}
/// One GET of `path` (already escaped): the status and the JSON body. No message made here carries the token.
fn get(&self, path: &str) -> Result<(u16, Value)> {
let url = format!("{}{path}", self.base);
let mut response = self
.agent
.get(&url)
.header("Authorization", format!("Bearer {}", self.token))
.call()
.with_context(|| format!("no answer from {url}"))?;
let status = response.status().as_u16();
let body = response.body_mut().read_to_string().with_context(|| format!("reading the answer from {url}"))?;
Ok((status, serde_json::from_str(&body).unwrap_or(Value::Null)))
}
fn ok(&self, path: &str) -> Result<Value> {
let (status, body) = self.get(path)?;
if status != 200 {
bail!("synapse answered {status} for {path}");
}
Ok(body)
}
}
fn event_of(raw: &Value) -> Result<RoomEvent> {
let text = |key: &str| raw[key].as_str().map(str::to_owned).with_context(|| format!("an event has no {key}"));
let content = &raw["content"];
let content_keys = content.as_object().map(|map| map.keys().cloned().collect()).unwrap_or_default();
Ok(RoomEvent { event_id: text("event_id")?, sender: text("sender")?, ts_ms: raw["origin_server_ts"].as_u64().context("an event has no origin_server_ts")?, content_keys, kind: kind_of(raw) })
}
fn kind_of(raw: &Value) -> EventKind {
let content = &raw["content"];
if !raw["unsigned"]["redacted_because"].is_null() {
return EventKind::Other;
}
match raw["type"].as_str() {
Some("m.room.redaction") => match content["redacts"].as_str().or_else(|| raw["redacts"].as_str()) {
Some(target) => EventKind::Redaction { target: target.to_owned() },
None => EventKind::Other,
},
Some("m.room.message") if matches!(content["msgtype"].as_str(), Some("m.text" | "m.notice" | "m.emote")) => {
let Some(body) = content["body"].as_str() else { return EventKind::Other };
let relation = &content["m.relates_to"];
if relation["rel_type"].as_str() == Some("m.replace") {
return match (relation["event_id"].as_str(), content["m.new_content"]["body"].as_str()) {
(Some(target), Some(new)) => EventKind::Edit { target: target.to_owned(), body: new.to_owned() },
_ => EventKind::Other,
};
}
EventKind::Message { body: body.to_owned() }
}
_ => EventKind::Other,
}
}
impl RoomHistory for HttpMatrix {
fn joined_rooms(&mut self) -> Result<Vec<String>> {
let body = self.ok("/_matrix/client/v3/joined_rooms")?;
let rooms = body["joined_rooms"].as_array().context("the joined rooms answer is not a list")?;
Ok(rooms.iter().filter_map(|room| room.as_str().map(str::to_owned)).collect())
}
fn facts(&mut self, room_id: &str) -> Result<RoomFacts> {
let body = self.ok(&format!("/_matrix/client/v3/rooms/{}/state", segment(room_id)))?;
let state = body.as_array().context("the room state is not a list")?;
let mut facts = RoomFacts { room_id: room_id.to_owned(), alias: None, alt_aliases: Vec::new(), name: None, encrypted: false, direct: false, bridge: None };
for event in state {
let content = &event["content"];
match event["type"].as_str() {
Some("m.room.canonical_alias") => {
facts.alias = content["alias"].as_str().map(str::to_owned);
facts.alt_aliases = content["alt_aliases"].as_array().map(|list| list.iter().filter_map(|alias| alias.as_str().map(str::to_owned)).collect()).unwrap_or_default();
}
Some("m.room.name") => facts.name = content["name"].as_str().map(str::to_owned),
Some("m.room.encryption") => facts.encrypted = true,
Some("m.room.member") if content["is_direct"].as_bool() == Some(true) => facts.direct = true,
Some(kind) if BRIDGE_TYPES.contains(&kind) => {
facts.bridge.get_or_insert_with(|| kind.to_owned());
}
_ => {}
}
}
Ok(facts)
}
fn events_after(&mut self, room_id: &str, after: Option<&str>, limit: u32) -> Result<Option<EventPage>> {
let room = segment(room_id);
let from = match after {
None => String::new(),
Some(event) => {
let (status, body) = self.get(&format!("/_matrix/client/v3/rooms/{room}/context/{}?limit=0", segment(event)))?;
match status {
200 => match body["end"].as_str() {
Some(end) => format!("&from={}", segment(end)),
None => bail!("synapse gave no end token for the context of {event} in {room_id}"),
},
404 => return Ok(None),
other => bail!("synapse answered {other} for the context of {event} in {room_id}"),
}
}
};
let body = self.ok(&format!("/_matrix/client/v3/rooms/{room}/messages?dir=f&limit={limit}{from}"))?;
let chunk = body["chunk"].as_array().context("the messages answer has no chunk")?;
let events = chunk.iter().map(event_of).collect::<Result<Vec<_>>>()?;
let more = body["end"].as_str().is_some() && !events.is_empty();
Ok(Some(EventPage { events, more }))
}
}

View file

@ -0,0 +1,207 @@
//! The Matrix client against a fake Synapse that speaks the client API's JSON.
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
use serde_json::{Value, json};
use super::chat::{EventKind, RoomHistory};
use super::matrix::HttpMatrix;
use super::test_http::{Asked, Reply, forge_of};
pub(super) const TOKEN: &str = "syt_reader_9f8e7d6c5b4a";
/// A room as the fake server holds it: its state events and its timeline.
pub(super) struct Room {
pub state: Vec<Value>,
pub timeline: Vec<Value>,
}
pub(super) type Rooms = Arc<Mutex<BTreeMap<String, Room>>>;
pub(super) fn decode(raw: &str) -> String {
let bytes = raw.as_bytes();
let mut out = Vec::new();
let mut at = 0;
while at < bytes.len() {
if bytes[at] == b'%' {
out.push(u8::from_str_radix(&raw[at + 1..at + 3], 16).unwrap());
at += 3;
} else {
out.push(bytes[at]);
at += 1;
}
}
String::from_utf8(out).unwrap()
}
pub(super) fn message(id: &str, sender: &str, ts: u64, body: &str) -> Value {
json!({ "type": "m.room.message", "event_id": id, "sender": sender, "origin_server_ts": ts, "content": { "msgtype": "m.text", "body": body } })
}
pub(super) fn state(kind: &str, content: Value) -> Value {
json!({ "type": kind, "state_key": "", "content": content })
}
/// A Synapse over `rooms`, joined to the rooms in the map: GET only, bearer required.
pub(super) fn synapse(rooms: &Rooms) -> (String, Asked) {
let rooms = Arc::clone(rooms);
forge_of(move |raw| {
let (path, query) = raw.split_once('?').unwrap_or((raw, ""));
let path = decode(path);
let rooms = rooms.lock().unwrap();
let rest = path.strip_prefix("/_matrix/client/v3/")?;
if rest == "joined_rooms" {
return Some(Reply::from(json!({ "joined_rooms": rooms.keys().collect::<Vec<_>>() }).to_string()));
}
let rest = rest.strip_prefix("rooms/")?;
let (id, rest) = rest.split_once('/')?;
let Some(room) = rooms.get(id) else { return Some(Reply::status(404, r#"{"errcode":"M_NOT_FOUND"}"#)) };
let query: BTreeMap<String, String> = query.split('&').filter_map(|pair| pair.split_once('=')).map(|(k, v)| (k.to_owned(), decode(v))).collect();
if rest == "state" {
return Some(Reply::from(Value::Array(room.state.clone()).to_string()));
}
if let Some(event) = rest.strip_prefix("context/") {
return Some(match room.timeline.iter().any(|held| held["event_id"] == event) {
true => Reply::from(json!({ "end": format!("t_{event}"), "events_before": [], "events_after": [] }).to_string()),
false => Reply::status(404, r#"{"errcode":"M_NOT_FOUND"}"#),
});
}
if rest == "messages" {
let start = match query.get("from") {
Some(from) => 1 + room.timeline.iter().position(|held| format!("t_{}", held["event_id"].as_str().unwrap()) == *from)?,
None => 0,
};
let limit: usize = query["limit"].parse().unwrap();
let chunk: Vec<&Value> = room.timeline.iter().skip(start).take(limit).collect();
let end = chunk.last().map(|last| format!("t_{}", last["event_id"].as_str().unwrap()));
return Some(Reply::from(json!({ "chunk": chunk, "end": end }).to_string()));
}
None
})
}
fn one_room(state: Vec<Value>, timeline: Vec<Value>) -> Rooms {
Arc::new(Mutex::new(BTreeMap::from([("!r:s".to_owned(), Room { state, timeline })])))
}
#[test]
fn state_maps_to_facts_and_the_join_rule_is_not_read() {
let plain = one_room(vec![state("m.room.join_rules", json!({ "join_rule": "invite" }))], vec![]);
let (base, _) = synapse(&plain);
let facts = HttpMatrix::new(&base, TOKEN).facts("!r:s").unwrap();
assert_eq!((facts.alias, facts.name, facts.encrypted, facts.direct, facts.bridge), (None, None, false, false, None));
for rule in ["public", "knock", "invite"] {
let rooms = one_room(vec![state("m.room.join_rules", json!({ "join_rule": rule }))], vec![]);
let (base, _) = synapse(&rooms);
let facts = HttpMatrix::new(&base, TOKEN).facts("!r:s").unwrap();
assert!(!facts.direct && !facts.encrypted && facts.bridge.is_none(), "{rule}");
}
let full = one_room(
vec![
state("m.room.canonical_alias", json!({ "alias": "#a:s", "alt_aliases": ["#b:s", "#c:s"] })),
state("m.room.name", json!({ "name": "Swarm" })),
state("m.room.encryption", json!({ "algorithm": "m.megolm.v1.aes-sha2" })),
json!({ "type": "m.room.member", "state_key": "@x:s", "content": { "membership": "join" } }),
json!({ "type": "m.room.member", "state_key": "@y:s", "content": { "membership": "join", "is_direct": true } }),
state("uk.half-shot.bridge", json!({})),
],
vec![],
);
let (base, _) = synapse(&full);
let facts = HttpMatrix::new(&base, TOKEN).facts("!r:s").unwrap();
assert_eq!(facts.alias.as_deref(), Some("#a:s"));
assert_eq!(facts.alt_aliases, ["#b:s", "#c:s"]);
assert_eq!(facts.name.as_deref(), Some("Swarm"));
assert!(facts.encrypted && facts.direct);
assert_eq!(facts.bridge.as_deref(), Some("uk.half-shot.bridge"));
let relay = one_room(vec![state("m.bridge", json!({}))], vec![]);
let (base, _) = synapse(&relay);
assert_eq!(HttpMatrix::new(&base, TOKEN).facts("!r:s").unwrap().bridge.as_deref(), Some("m.bridge"));
}
#[test]
fn events_map_to_kinds_and_content_keys_are_names_only() {
let mut other = message("$img", "@a:s", 3, "pic.png");
other["content"]["msgtype"] = json!("m.image");
let mut pruned = message("$pruned", "@a:s", 4, "was here");
pruned["unsigned"] = json!({ "redacted_because": { "event_id": "$x" } });
let mut notice = message("$n", "@a:s", 5, "n");
notice["content"]["msgtype"] = json!("m.notice");
let mut emote = message("$e", "@a:s", 6, "e");
emote["content"]["msgtype"] = json!("m.emote");
let edit = json!({ "type": "m.room.message", "event_id": "$edit", "sender": "@a:s", "origin_server_ts": 7, "content": {
"msgtype": "m.text", "body": "* new", "m.new_content": { "msgtype": "m.text", "body": "new" }, "m.relates_to": { "rel_type": "m.replace", "event_id": "$m1" } } });
let redaction = json!({ "type": "m.room.redaction", "event_id": "$red", "sender": "@a:s", "origin_server_ts": 8, "content": { "redacts": "$m1" } });
let old_redaction = json!({ "type": "m.room.redaction", "event_id": "$red2", "sender": "@a:s", "origin_server_ts": 9, "redacts": "$m2", "content": {} });
let encrypted = json!({ "type": "m.room.encrypted", "event_id": "$enc", "sender": "@a:s", "origin_server_ts": 10, "content": { "ciphertext": "xx" } });
let timeline = vec![message("$m1", "@a:s", 1, "hello"), other, pruned, notice, emote, edit, redaction, old_redaction, encrypted];
let rooms = one_room(vec![], timeline);
let (base, _) = synapse(&rooms);
let page = HttpMatrix::new(&base, TOKEN).events_after("!r:s", None, 50).unwrap().unwrap();
let kinds: Vec<EventKind> = page.events.iter().map(|event| event.kind.clone()).collect();
assert_eq!(
kinds,
[
EventKind::Message { body: "hello".into() },
EventKind::Other,
EventKind::Other,
EventKind::Message { body: "n".into() },
EventKind::Message { body: "e".into() },
EventKind::Edit { target: "$m1".into(), body: "new".into() },
EventKind::Redaction { target: "$m1".into() },
EventKind::Redaction { target: "$m2".into() },
EventKind::Other,
]
);
assert_eq!(page.events[0].content_keys, ["body", "msgtype"]);
assert_eq!(page.events[5].content_keys, ["body", "m.new_content", "m.relates_to", "msgtype"], "names, never values");
assert_eq!((page.events[0].event_id.as_str(), page.events[0].sender.as_str(), page.events[0].ts_ms), ("$m1", "@a:s", 1));
}
#[test]
fn resume_uses_the_cursor_events_end_token_and_a_missing_event_is_none() {
let rooms = one_room(vec![], vec![message("$1", "@a:s", 1, "a"), message("$2", "@a:s", 2, "b"), message("$3", "@a:s", 3, "c")]);
let (base, asked) = synapse(&rooms);
let mut matrix = HttpMatrix::new(&base, TOKEN);
let page = matrix.events_after("!r:s", Some("$1"), 1).unwrap().unwrap();
assert_eq!(page.events.iter().map(|event| event.event_id.as_str()).collect::<Vec<_>>(), ["$2"]);
assert!(page.more);
assert_eq!(asked.paths(), ["/_matrix/client/v3/rooms/%21r%3As/context/%241?limit=0", "/_matrix/client/v3/rooms/%21r%3As/messages?dir=f&limit=1&from=t_%241"]);
assert!(matrix.events_after("!r:s", Some("$unknown"), 5).unwrap().is_none());
let tail = matrix.events_after("!r:s", Some("$3"), 5).unwrap().unwrap();
assert!(tail.events.is_empty() && !tail.more, "an empty chunk ends the run");
let first = matrix.events_after("!r:s", None, 2).unwrap().unwrap();
assert_eq!(first.events.len(), 2);
assert_eq!(asked.paths().last().unwrap(), "/_matrix/client/v3/rooms/%21r%3As/messages?dir=f&limit=2", "no from, no context call");
}
#[test]
fn only_get_always_the_bearer_and_the_token_is_never_printed() {
let rooms = one_room(vec![], vec![message("$1", "@a:s", 1, "a")]);
let (base, asked) = synapse(&rooms);
let mut matrix = HttpMatrix::new(&base, TOKEN);
matrix.joined_rooms().unwrap();
matrix.facts("!r:s").unwrap();
matrix.events_after("!r:s", Some("$1"), 3).unwrap();
let seen = asked.seen();
assert!(seen.len() >= 4 && seen.iter().all(|seen| seen.method == "GET" && seen.authorization.as_deref() == Some(&format!("Bearer {TOKEN}"))), "{seen:?}");
let (base, _) = forge_of(|_| Some(Reply::status(401, &format!(r#"{{"errcode":"M_UNKNOWN_TOKEN","error":"bad {TOKEN}"}}"#))));
let mut matrix = HttpMatrix::new(&base, TOKEN);
let error = matrix.joined_rooms().unwrap_err();
assert!(format!("{error:#}").contains("401"));
assert!(!format!("{error:#}{error:?}").contains(TOKEN));
assert!(!format!("{matrix:?}").contains(TOKEN));
}
#[test]
fn ids_are_percent_encoded_in_paths() {
let weird = "$a+b/c=d:s";
let rooms: Rooms = Arc::new(Mutex::new(BTreeMap::from([("!ro om:s".to_owned(), Room { state: vec![], timeline: vec![message(weird, "@a:s", 1, "x"), message("$2", "@a:s", 2, "y")] })])));
let (base, asked) = synapse(&rooms);
let mut matrix = HttpMatrix::new(&base, TOKEN);
let page = matrix.events_after("!ro om:s", Some(weird), 5).unwrap().unwrap();
assert_eq!(page.events.len(), 1, "the server decoded the room and event ids to the originals");
let raw = asked.paths();
assert!(raw.iter().all(|path| !path.contains('+') && !path.contains(' ') && !path.contains("!ro")), "{raw:?}");
}

View file

@ -30,11 +30,20 @@ pub mod chat_purge;
pub mod chunk;
pub mod forge;
pub mod git;
pub mod job;
pub mod matrix;
pub mod room_scope;
#[cfg(test)]
mod test_http;
#[cfg(test)]
mod chat_tests;
#[cfg(test)]
mod job_tests;
#[cfg(test)]
mod matrix_tests;
#[cfg(test)]
mod tests;
use std::collections::hash_map::Entry;

View file

@ -0,0 +1,92 @@
//! The one fake HTTP server the source tests talk to (the forge's and Matrix's):
//! it answers by path, holds open a path it has no answer for as a hung server
//! does, and records each request's method, path and `Authorization` header.
use std::io::{Read, Write};
use std::sync::{Arc, Mutex};
use serde_json::json;
/// What a fake server answers: a status, the `Location` header when there is one, and a body.
pub struct Reply {
pub status: u16,
pub location: Option<String>,
pub body: String,
}
impl From<String> for Reply {
fn from(body: String) -> Self {
Self { status: 200, location: None, body }
}
}
impl Reply {
/// What the live forge answers an anonymous reader for a private repo.
pub fn not_found() -> Self {
Self { status: 404, location: None, body: json!({ "message": "The target couldn't be found." }).to_string() }
}
/// What the live forge answers for a renamed repo: a 301 to its public address, `/forge` prefix and all.
pub fn moved_to(location: &str) -> Self {
Self { status: 301, location: Some(location.to_owned()), body: String::new() }
}
pub fn status(status: u16, body: &str) -> Self {
Self { status, location: None, body: body.to_owned() }
}
}
/// One request as the server saw it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Seen {
pub method: String,
pub path: String,
pub authorization: Option<String>,
}
/// Every request a fake server has seen, in order.
#[derive(Clone, Default)]
pub struct Asked(Arc<Mutex<Vec<Seen>>>);
impl Asked {
pub fn seen(&self) -> Vec<Seen> {
self.0.lock().unwrap().clone()
}
pub fn paths(&self) -> Vec<String> {
self.0.lock().unwrap().iter().map(|seen| seen.path.clone()).collect()
}
}
/// Answers each request by its path from `routes`; a path it has no answer for is held open and never answered.
pub fn forge_of<R: Into<Reply>>(routes: impl Fn(&str) -> Option<R> + Send + 'static) -> (String, Asked) {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}", listener.local_addr().unwrap());
let asked = Asked::default();
let record = asked.clone();
std::thread::spawn(move || {
let mut held = Vec::new();
for stream in listener.incoming() {
let mut stream = stream.unwrap();
let mut buffer = [0_u8; 4096];
let read = stream.read(&mut buffer).unwrap();
let request = String::from_utf8_lossy(&buffer[..read]).into_owned();
let mut words = request.split_whitespace();
let method = words.next().unwrap_or_default().to_owned();
let path = words.next().unwrap_or_default().to_owned();
let authorization = request
.lines()
.find_map(|line| line.split_once(':').filter(|(name, _)| name.eq_ignore_ascii_case("authorization")))
.map(|(_, value)| value.trim().to_owned());
record.0.lock().unwrap().push(Seen { method, path: path.clone(), authorization });
match routes(&path).map(Into::into) {
Some(Reply { status, location, body }) => {
let location = location.map(|location| format!("location: {location}\r\n")).unwrap_or_default();
write!(stream, "HTTP/1.1 {status} X\r\ncontent-type: application/json\r\n{location}content-length: {}\r\nconnection: close\r\n\r\n{body}", body.len()).unwrap();
}
None => held.push(stream),
}
}
});
(base, asked)
}

View file

@ -25,23 +25,23 @@ fn git_in(dir: &Path, args: &[&str]) -> String {
}
/// A working repo on disk, committed to by the test.
struct Remote {
pub(super) struct Remote {
dir: tempfile::TempDir,
}
impl Remote {
fn new() -> Self {
pub(super) fn new() -> Self {
let dir = tempfile::tempdir().unwrap();
git_in(dir.path(), &["init", "--quiet", "-b", "main"]);
Self { dir }
}
fn path(&self) -> PathBuf {
pub(super) fn path(&self) -> PathBuf {
self.dir.path().to_path_buf()
}
/// Writes files, removes paths, commits everything and returns the commit.
fn commit(&self, write: &[(&str, &[u8])], remove: &[&str]) -> String {
pub(super) fn commit(&self, write: &[(&str, &[u8])], remove: &[&str]) -> String {
for (path, bytes) in write {
let full = self.dir.path().join(path);
std::fs::create_dir_all(full.parent().unwrap()).unwrap();
@ -56,13 +56,13 @@ impl Remote {
}
}
struct FakeForge {
pub(super) struct FakeForge {
repos: Vec<(SourceRepo, PathBuf)>,
shortfall: Option<String>,
}
impl FakeForge {
fn of(repos: &[(&str, &Remote)]) -> Self {
pub(super) fn of(repos: &[(&str, &Remote)]) -> Self {
Self {
repos: repos
.iter()

View file

@ -191,12 +191,18 @@ impl Drop for Daemon {
}
}
/// A test's name for someone as the Bifrost account id the door requires: a deterministic lowercase D-format UUID.
fn acct(name: &str) -> huginn_mind::eureka_pipeline::AccountId {
let hash = name.bytes().fold(0xcbf2_9ce4_8422_2325_u64, |hash, byte| (hash ^ u64::from(byte)).wrapping_mul(0x0100_0000_01b3));
huginn_mind::eureka_pipeline::AccountId(format!("{:08x}-{:04x}-4000-8000-{:012x}", hash >> 32, (hash >> 16) & 0xffff, hash & 0xffff_ffff_ffff))
}
fn answer(who: &str, hash: &str, request_id: &str) -> DeskRequest {
DeskRequest::Answer {
mind: INSTANCE.into(),
subject: short(who),
subject: acct(who),
authority: DeskAuthority::Operator,
deciders: vec![short(who)],
deciders: vec![acct(who)],
request: ForkAnswerRequest {
schema_version: RequestSchema::V0,
fork_id: FORK.into(),
@ -213,8 +219,8 @@ fn answer(who: &str, hash: &str, request_id: &str) -> DeskRequest {
fn forks_request(who: &str, deciders: &[&str]) -> DeskRequest {
DeskRequest::Forks {
mind: INSTANCE.into(),
subject: short(who),
deciders: deciders.iter().map(|d| short(d)).collect(),
subject: acct(who),
deciders: deciders.iter().map(|d| acct(d)).collect(),
status: None,
cursor: None,
limit: huginn_mind::desk_wire::ForkLimit::new(50).unwrap(),
@ -284,14 +290,14 @@ fn the_binary_serves_both_doors_and_survives_restart() {
};
assert_eq!((list.open_count, list.recorded_count, list.forks.len(), list.stamp.clone()), (1, 0, 1, format!("s{listed_at}")));
assert_eq!(list.forks[0].card.options_hash, hash);
let DeskResponse::Fork { entry, as_of } = ask(&desk, "m-r2", &DeskRequest::Fork { mind: INSTANCE.into(), subject: short("x"), deciders: vec![short("x")], fork_id: FORK.into() })
let DeskResponse::Fork { entry, as_of } = ask(&desk, "m-r2", &DeskRequest::Fork { mind: INSTANCE.into(), subject: acct("x"), deciders: vec![acct("x")], fork_id: FORK.into() })
else {
panic!("a read of the seeded fork")
};
assert_eq!((entry, as_of), (list.forks[0].clone(), listed_at));
assert_eq!(daemon.documents(), seeded);
// `authority` is no field of a read: the payload is refused whole and nothing is answered or stored.
for (operation, request) in [("forks", forks_request("x", &["x"])), ("fork", DeskRequest::Fork { mind: INSTANCE.into(), subject: short("x"), deciders: vec![short("x")], fork_id: FORK.into() })] {
for (operation, request) in [("forks", forks_request("x", &["x"])), ("fork", DeskRequest::Fork { mind: INSTANCE.into(), subject: acct("x"), deciders: vec![acct("x")], fork_id: FORK.into() })] {
let mut value = serde_json::to_value(&request).unwrap();
let variant = if operation == "forks" { "Forks" } else { "Fork" };
value[variant]["authority"] = serde_json::json!("Operator");
@ -319,16 +325,20 @@ fn the_binary_serves_both_doors_and_survives_restart() {
assert_eq!((stale.code, counted_answer_id.as_deref()), (Some(ForkAnswerCode::StaleFork), Some("n1")));
let stored = daemon.documents();
assert_eq!(stored, seeded + 2);
// A subject the call's deciders do not name reads both answers, the first with its choice and the stale one with its code, and is told no account id.
// A subject the call's deciders do not name is a non-decider: the fork is unlisted, so it reads as a missing one.
let outsider = ask(&desk, "m-r3", &DeskRequest::Fork { mind: INSTANCE.into(), subject: acct("outsider"), deciders: vec![acct("y")], fork_id: FORK.into() });
assert!(matches!(outsider, DeskResponse::NotStored(ref refusal) if refusal.code == huginn_mind::desk_wire::ForkRefusalCode::ForkUnknown), "{outsider:?}");
// A decider reads both answers, the first with its choice and the stale one with its code, and is told who answered.
let DeskResponse::Fork { entry, .. } =
ask(&desk, "m-r3", &DeskRequest::Fork { mind: INSTANCE.into(), subject: short("outsider"), deciders: vec![short("y")], fork_id: FORK.into() })
ask(&desk, "m-r4", &DeskRequest::Fork { mind: INSTANCE.into(), subject: acct("y"), deciders: vec![acct("y")], fork_id: FORK.into() })
else {
panic!("a read")
};
let seen: Vec<_> = entry.record.answers.iter().map(|answer| (answer.answer_id.as_str(), answer.answerer.as_deref(), answer.code)).collect();
assert_eq!(seen, [("n1", None, None), ("n2", None, Some(ForkAnswerCode::StaleFork))]);
let x = acct("x").0;
assert_eq!(seen, [("n1", Some(x.as_str()), None), ("n2", Some(x.as_str()), Some(ForkAnswerCode::StaleFork))]);
assert_eq!((entry.record.counted_answer_id.as_deref(), entry.record.answerer_count), (Some("n1"), 1));
assert_eq!((entry.card.deciders.as_slice(), entry.card.decider_count), (&[][..], 1));
assert_eq!((entry.card.deciders.as_slice(), entry.card.decider_count), (&[acct("y").0][..], 1));
assert_eq!(daemon.documents(), stored);
// A stop takes the socket with it; the stored answers are the retry's memory.
@ -539,13 +549,13 @@ fn a_busy_pass_is_not_followed_by_the_idle_sleep() {
];
let options = parse_options(args.iter().map(|arg| arg.to_string())).unwrap();
let embedder = pool(&options.ollama_urls, &options.reserve_ollama_urls, &options.embedding_model);
let Started { mut daemon, mut hub, registry, desk: mut door } =
let Started { mut daemon, mut hub, persona: _, registry, desk: mut door } =
startup(&options, embedder, QdrantIndex::new(&options.qdrant_url), Backoff::default()).unwrap();
let stopping = Arc::new(AtomicBool::new(false));
let loop_stopping = Arc::clone(&stopping);
let serving = std::thread::spawn(move || {
let serve = ServeOptions { idle_sleep: Duration::from_millis(600), ..ServeOptions::default() };
run(&mut daemon, &mut hub, &registry, door.as_mut(), &loop_stopping, &serve)
run(&mut daemon, &mut hub, None, &registry, door.as_mut(), &loop_stopping, &serve)
});
std::thread::sleep(Duration::from_millis(100));
let frame = framed(&catalog_request());
@ -580,7 +590,7 @@ fn forks_page(who: &str, cursor: Option<&str>) -> DeskRequest {
}
fn fork_read(who: &str, fork_id: &str) -> DeskRequest {
DeskRequest::Fork { mind: INSTANCE.into(), subject: short(who), deciders: vec![short(who)], fork_id: fork_id.into() }
DeskRequest::Fork { mind: INSTANCE.into(), subject: acct(who), deciders: vec![acct(who)], fork_id: fork_id.into() }
}
/// The wire spelling of a not-stored refusal's code, and its text.

View file

@ -62,12 +62,18 @@ pub(crate) enum Door {
Desk,
}
/// Every staged `desk_answer`, whether the caller sent it or a derivation
/// produced it, is refused unless it came by the desk door.
fn refuse_desk_answers_outside_desk(door: Door, docs: &Docs<'_>) -> Result<(), MindRefusal> {
match docs.batch.iter().find(|staged| staged.kind == PipelineKind::DeskAnswer) {
Some(staged) if door != Door::Desk => Err(MindRefusal::DeskAnswerOutsideDesk { id: staged.key.clone() }),
_ => Ok(()),
/// The door split. Every staged `desk_answer`, whether the caller sent it or a derivation produced it,
/// is refused unless it came by the desk door; and a staged `desk_listing` is refused if it came by the
/// desk door: listing is the Mind's own act, on the Admit door, with a receipt saying who.
fn refuse_documents_at_the_wrong_door(door: Door, docs: &Docs<'_>) -> Result<(), MindRefusal> {
match docs.batch.iter().find(|staged| match staged.kind {
PipelineKind::DeskAnswer => door != Door::Desk,
PipelineKind::DeskListing => door == Door::Desk,
_ => false,
}) {
Some(staged) if staged.kind == PipelineKind::DeskListing => Err(MindRefusal::DeskRequestInvalid { field: "desk_listing".into() }),
Some(staged) => Err(MindRefusal::DeskAnswerOutsideDesk { id: staged.key.clone() }),
None => Ok(()),
}
}
@ -160,7 +166,7 @@ impl<S: MindStore> Mind<S> {
for envelope in envelopes {
docs.push(stage(envelope, &mind)?)?;
}
refuse_desk_answers_outside_desk(door, &docs)?;
refuse_documents_at_the_wrong_door(door, &docs)?;
// A6
let carries_instance = docs.batch.iter().filter(|staged| staged.kind == PipelineKind::Instance).count() == 1;
if self.is_empty() && !carries_instance {
@ -184,7 +190,7 @@ impl<S: MindStore> Mind<S> {
docs.push(stage(envelope, &mind)?)?;
}
}
refuse_desk_answers_outside_desk(door, &docs)?;
refuse_documents_at_the_wrong_door(door, &docs)?;
for staged in &docs.batch[originals..] {
resolve(&docs, staged, &mind, external, &mut strong)?;
}
@ -262,17 +268,9 @@ impl<S: MindStore> Mind<S> {
if self.is_empty() {
return Err(MindRefusal::MissingIdentity);
}
// P1
persona::validate_shape(&state)?;
// P2
// P1 to P3: the document alone, the one check an offline reader of a state file shares.
let mind = self.instance().clone();
if state.get("personaId").and_then(Value::as_str) != Some(mind.0.as_str()) {
return Err(MindRefusal::PersonaForeign { instance: mind.0 });
}
// P3
if state.pointer("/provenance/authority").and_then(Value::as_str) != Some("canonical") {
return Err(MindRefusal::PersonaNotCanonical);
}
persona::validate_document(&mind, &state)?;
// P4
let stored = self.stored_updated_at()?;
if stored != expected_updated_at {
@ -627,6 +625,9 @@ fn check(docs: &Docs<'_>, staged: &Staged, mind: &Slug, import: Option<&ImportSo
}
// Decided in `answer_desk` before the document exists; no other door can stage one.
D::DeskAnswer(_) => Ok(()),
// The question is a citation (`listed`), so a missing one is refused with every other missing referent, and
// the key derivation holds it to the listing's own campaign.
D::DeskListing(_) => Ok(()),
D::Ruling(ruling) => {
if ruling.operator_quote.is_some() && ruling.authority != RulingAuthority::Operator {
return Err(MindRefusal::QuoteWithoutOperator { ruling: key.into() });
@ -662,7 +663,7 @@ fn check(docs: &Docs<'_>, staged: &Staged, mind: &Slug, import: Option<&ImportSo
return Err(MindRefusal::RepoNotInCampaign { repo: spec.repo.0.clone() });
}
for ruling in &spec.rulings {
if !(historic && root_and_local(&ruling.0).0 != root) && !docs.in_force(K::Ruling, &ruling.0) {
if !(docs.in_force(K::Ruling, &ruling.0) || historic && root_and_local(&ruling.0).0 != root) {
return Err(MindRefusal::CitesResolvedDocument { kind: K::Ruling, id: ruling.0.clone() });
}
}
@ -905,6 +906,7 @@ fn matrix(subject: PipelineKind, outcome: &ResolutionOutcome) -> bool {
(K::Target, O::Superseded { by }) => all(by, K::Target),
(K::Question, O::Answered { by }) => by.kind == K::Ruling,
(K::Question, O::Withdrawn { .. }) => true,
(K::DeskListing, O::Withdrawn { .. }) => true,
(K::Ruling, O::Superseded { by }) => all(by, K::Ruling),
(K::CutSpec, O::Superseded { by }) => all(by, K::CutSpec),
(K::CutSpec, O::Withdrawn { .. }) => true,

View file

@ -19,6 +19,7 @@
use chrono::{DateTime, Duration, SecondsFormat, SubsecRound, Utc};
use eureka_pipeline::{
AccountId,
DeskAuthority, DeskOutcome, DeskRefusal, PipelineDeskAnswer, PipelineDocument, PipelineKind, PipelineQuestion, PipelineRefusal,
PipelineResolution, RequestId, ResolutionOutcome, Sha256Hex, Short, Slug, Words,
};
@ -145,7 +146,7 @@ fn counts_under(answer: &PipelineDeskAnswer, current: &Sha256Hex) -> bool {
/// `answer_desk` runs before it reads the fork and the door runs before its showability gate.
pub(crate) fn bound_answer<'a>(docs: &'a Docs<'_>, answerer: &Short, request_id: &RequestId) -> Option<(&'a str, &'a PipelineDeskAnswer)> {
docs.of_kind(PipelineKind::DeskAnswer).find_map(|(key, document)| match document {
PipelineDocument::DeskAnswer(held) if held.answerer == *answerer && held.request_id == *request_id => Some((key, held)),
PipelineDocument::DeskAnswer(held) if held.answerer.0 == answerer.0 && held.request_id == *request_id => Some((key, held)),
_ => None,
})
}
@ -314,7 +315,7 @@ impl<S: MindStore> Mind<S> {
claim_in_future,
request_id: request.request_id.clone(),
body_sha256: body_sha256.clone(),
answerer: stamp.answerer.clone(),
answerer: AccountId(stamp.answerer.0.clone()),
authority: stamp.authority,
outcome,
};
@ -425,7 +426,7 @@ mod tests {
const LIST: [&str; 6] = ["x", "y", "z", "a", "op", "dec"];
fn stamp_as(who: &str, authority: DeskAuthority, list: &[&str]) -> DeskStamp {
DeskStamp { answerer: s(who), authority, deciders: list.iter().map(|id| s(id)).collect() }
DeskStamp { answerer: account(who), authority, deciders: list.iter().map(|id| account(id)).collect() }
}
fn decider(who: &str) -> DeskStamp {
@ -531,7 +532,7 @@ mod tests {
let mut mind = desk();
stored(ask(&mut mind, &decider("z"), pick(FORK, "Q1", "A", "2026-11-10T12:00:00Z")));
stored(ask(&mut mind, &decider("a"), pick(FORK, "Q1", "A", "2026-11-10T12:00:00Z")));
assert_eq!(held(&mind, &counted(&mind, FORK).unwrap()).answerer, s("z"));
assert_eq!(held(&mind, &counted(&mind, FORK).unwrap()).answerer, acct("z"));
}
#[test]
@ -780,7 +781,7 @@ mod tests {
let record = held(&mind, &answer);
assert_eq!(record.outcome, outcome);
assert_eq!(outcome, DeskOutcome::Refused { code: DeskRefusal::NotADecider, retracts: None });
assert_eq!((record.answerer, record.authority, record.answered_at), (s("guest"), DeskAuthority::Steward, s(at)));
assert_eq!((record.answerer, record.authority, record.answered_at), (acct("guest"), DeskAuthority::Steward, s(at)));
}
#[test]
@ -790,7 +791,7 @@ mod tests {
for (who, authority) in [("op", DeskAuthority::Operator), ("dec", DeskAuthority::Steward), ("x", DeskAuthority::Member)] {
let (answer, _) = stored(ask(&mut mind, &stamp_as(who, authority, &LIST), pick(FORK, "Q1", "A", at)));
let record = held(&mind, &answer);
assert_eq!((record.answerer, record.authority), (s(who), authority));
assert_eq!((record.answerer, record.authority), (acct(who), authority));
}
}
@ -1039,7 +1040,7 @@ mod tests {
let DeskAnswerOutcome::Stored { receipt_id, .. } = outcome else { panic!("{outcome:?}") };
let receipt = mind.receipts().unwrap().into_iter().find(|receipt| receipt.receipt_id == receipt_id).unwrap();
assert_eq!(receipt.provenance.faculty, faculty, "{authority:?}");
assert_eq!(receipt.provenance.agent, s("x"));
assert_eq!(receipt.provenance.agent, account("x"));
}
}
@ -1206,10 +1207,10 @@ mod tests {
let image = first.image().unwrap();
let found = Docs::at(image, image.head).find(K::DeskAnswer, &key("Q1", 1)).cloned();
let Some(PipelineDocument::DeskAnswer(mut held)) = found else { panic!("stored") };
held.answerer = s(&long);
held.answerer = eureka_pipeline::AccountId(long.clone());
store.plant(PipelineDocument::DeskAnswer(held).prepare(&cache).unwrap());
let mut mind = opened(store, INSTANCE);
let stamp = stamp_as(&long, DeskAuthority::Member, &LIST);
let stamp = DeskStamp { answerer: s(&long), ..stamp_as("x", DeskAuthority::Member, &LIST) };
let before = head(&mind);
assert_eq!(replayed(ask(&mut mind, &stamp, request)), (key("Q1", 1), DeskOutcome::Counted));
assert_eq!(head(&mind), before);
@ -1262,8 +1263,8 @@ mod tests {
assert_eq!(stored(ask(&mut mind, &decider("x"), sent.clone())), (key("Q1", 1), DeskOutcome::Counted));
let (y_answer, y_outcome) = stored(ask(&mut mind, &decider("y"), sent.clone()));
assert_eq!((y_answer.as_str(), &y_outcome), (key("Q1", 2).as_str(), &DeskOutcome::Revision { of: s(&key("Q1", 1)) }));
assert_eq!(held(&mind, &y_answer).answerer, s("y"));
assert_eq!(held(&mind, &key("Q1", 1)).answerer, s("x"));
assert_eq!(held(&mind, &y_answer).answerer, acct("y"));
assert_eq!(held(&mind, &key("Q1", 1)).answerer, acct("x"));
// x's later retry still returns n1; y's returns n2.
assert_eq!(replayed(ask(&mut mind, &decider("x"), sent.clone())).0, key("Q1", 1));
assert_eq!(replayed(ask(&mut mind, &decider("y"), sent)).0, key("Q1", 2));
@ -1467,7 +1468,7 @@ mod tests {
let before = head(&mind);
let id = request.request_id.clone();
let (answer, outcome) = stored(ask(&mut mind, &decider("y"), request.clone()));
assert_eq!(outcome, DeskOutcome::Refused { code: DeskRefusal::BadRetraction, retracts: sent.2.map(|target| s(target)) }, "{what}");
assert_eq!(outcome, DeskOutcome::Refused { code: DeskRefusal::BadRetraction, retracts: sent.2.map(s) }, "{what}");
assert_eq!(head(&mind), before + 1, "{what} is stored");
// The record keeps what was sent: the choice, the note and the target it named.
let record = held(&mind, &answer);

View file

@ -20,7 +20,7 @@ pub use reads::{CardSchema, CardStatus, EntrySchema, ForkCard, ForkCursor, ForkE
use chrono::{DateTime, Utc};
use eureka_pipeline::{
DeskAuthority, DeskOutcome, DeskRefusal, PipelineDeskAnswer, PipelineDocument, PipelineKind, PipelineQuestion, RequestId, Sha256Hex, Short, Slug, Words,
AccountId, DeskAuthority, DeskOutcome, DeskRefusal, PipelineDeskAnswer, PipelineDocument, PipelineKind, PipelineQuestion, RequestId, Sha256Hex, Short, Slug, Words,
};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
@ -232,18 +232,33 @@ pub const DESK_PAGE_BUDGET: usize = DESK_MAX_RESPONSE_BYTES - DESK_ENVELOPE_MARG
/// Most deciders one call carries (CultLib `card.deciders` `maxItems`).
pub const DESK_DECIDERS_MAX: usize = 64;
/// A decider list, read whole or not at all: one to `DESK_DECIDERS_MAX` subjects, none twice. A
/// bad list is the decoder's refusal, which names the field and never quotes a subject.
fn deciders<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<Vec<Short>, D::Error> {
/// An account id read from the wire: a lowercase D-format UUID, or the decoder's refusal, which names the
/// field and never quotes the value.
fn account_id<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<AccountId, D::Error> {
let id = AccountId(String::deserialize(deserializer)?);
id.validate_account().map_err(|_| serde::de::Error::custom("not an account id: a lowercase D-format UUID"))?;
Ok(id)
}
/// A decider list, read whole or not at all: one to `DESK_DECIDERS_MAX` account ids, none twice. A
/// bad list is the decoder's refusal, which names the field and never quotes an id.
fn deciders<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<Vec<AccountId>, D::Error> {
struct List;
/// One list entry, through the same grammar as `subject`.
struct Account(AccountId);
impl<'de> Deserialize<'de> for Account {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
account_id(deserializer).map(Self)
}
}
impl<'de> serde::de::Visitor<'de> for List {
type Value = Vec<Short>;
type Value = Vec<AccountId>;
fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
formatter.write_str("a list of distinct deciders")
}
fn visit_seq<A: serde::de::SeqAccess<'de>>(self, mut seq: A) -> Result<Vec<Short>, A::Error> {
fn visit_seq<A: serde::de::SeqAccess<'de>>(self, mut seq: A) -> Result<Vec<AccountId>, A::Error> {
let mut list = Vec::new();
while let Some(subject) = seq.next_element::<Short>()? {
while let Some(Account(subject)) = seq.next_element::<Account>()? {
if list.len() == DESK_DECIDERS_MAX {
return Err(serde::de::Error::custom("too many deciders"));
}
@ -264,7 +279,7 @@ fn deciders<'de, D: serde::Deserializer<'de>>(deserializer: D) -> Result<Vec<Sho
fn deciders_schema(generator: &mut schemars::SchemaGenerator) -> schemars::Schema {
schemars::json_schema!({
"type": "array",
"items": generator.subschema_for::<Short>(),
"items": generator.subschema_for::<AccountId>(),
"minItems": 1,
"maxItems": DESK_DECIDERS_MAX,
"uniqueItems": true
@ -279,8 +294,9 @@ fn deciders_schema(generator: &mut schemars::SchemaGenerator) -> schemars::Schem
///
/// The reads, `Forks` and `Fork`, carry NO `authority` (CultLib `list_request` and
/// `read_request` are closed and have none; one that names it is refused whole). Their `deciders`
/// feed the showability gate, and `subject` decides the view: a subject in `deciders` is a decider
/// and is told the ids; any other is told counts in their place. `Forks` takes the contract's `list_request` fields flat
/// feed the showability gate, and `subject` decides the view: a subject in `deciders` is a decider,
/// sees every showable fork and is told the ids; any other reads only the forks the Mind lists
/// (a `desk_listing` in force) and is told counts in the ids' place. `Forks` takes the contract's `list_request` fields flat
/// beside them (`status`, `cursor`, `limit`), each key written, `null` where absent. Bifrost
/// strips notes; see `DeskResponse` for the trust boundary. These docs and the doc of `Mind::desk_forks`/`Mind::desk_fork` are the
/// reads spec.
@ -289,19 +305,21 @@ fn deciders_schema(generator: &mut schemars::SchemaGenerator) -> schemars::Schem
pub enum DeskRequest {
Answer {
mind: Slug,
subject: Short,
#[serde(deserialize_with = "account_id")]
subject: AccountId,
authority: DeskAuthority,
#[serde(deserialize_with = "deciders")]
#[schemars(schema_with = "deciders_schema")]
deciders: Vec<Short>,
deciders: Vec<AccountId>,
request: ForkAnswerRequest,
},
Forks {
mind: Slug,
subject: Short,
#[serde(deserialize_with = "account_id")]
subject: AccountId,
#[serde(deserialize_with = "deciders")]
#[schemars(schema_with = "deciders_schema")]
deciders: Vec<Short>,
deciders: Vec<AccountId>,
#[serde(deserialize_with = "required")]
#[schemars(required)]
status: Option<ListStatus>,
@ -312,10 +330,11 @@ pub enum DeskRequest {
},
Fork {
mind: Slug,
subject: Short,
#[serde(deserialize_with = "account_id")]
subject: AccountId,
#[serde(deserialize_with = "deciders")]
#[schemars(schema_with = "deciders_schema")]
deciders: Vec<Short>,
deciders: Vec<AccountId>,
fork_id: String,
},
}
@ -337,7 +356,7 @@ impl DeskRequest {
}
}
pub fn deciders(&self) -> &[Short] {
pub fn deciders(&self) -> &[AccountId] {
match self {
Self::Answer { deciders, .. } | Self::Forks { deciders, .. } | Self::Fork { deciders, .. } => deciders,
}
@ -530,10 +549,13 @@ impl<S: MindStore> Mind<S> {
};
let image = self.image()?;
let docs = Docs::at(image, image.head);
// A request already stored replays whatever the list is now: the lookup is `answer_desk`'s,
// asked here only so the gate does not shadow it.
let unshowable = bound_answer(&docs, &subject, &sent.request_id).is_none()
&& matches!(docs.find(PipelineKind::Question, &sent.fork.0), Some(PipelineDocument::Question(question)) if !showable(question, deciders));
// A fork this reader may not see is unshowable here too, by the read side's own `visible`: the answer path
// must not tell a hidden fork from a missing one, nor store a refusal for it. A request already stored replays
// whatever the decider list is now (the lookup is `answer_desk`'s, asked here only so `showable` does not
// shadow it), but never to a reader who can no longer see the fork: the counted answer id and `as_of` it
// would carry are the fork's current state, so that replay is `fork_unknown` too (ruling clause 3).
let replay = bound_answer(&docs, &subject, &sent.request_id).is_some();
let unshowable = matches!(docs.find(PipelineKind::Question, &sent.fork.0), Some(PipelineDocument::Question(question)) if (!replay && !showable(question, deciders)) || !reads::visible(&docs, &sent.fork.0, deciders.contains(&subject)));
if unshowable {
return Ok(DeskResponse::NotStored(fork_unknown()));
}
@ -547,7 +569,7 @@ impl<S: MindStore> Mind<S> {
let Some(PipelineDocument::DeskAnswer(stored)) = docs.find(PipelineKind::DeskAnswer, &key) else { return Err(unreadable_key()) };
let fork = stored.fork.0.as_str();
let counted_answer_id = counted_answer(&docs, fork).map(|counted| answer_seq(fork, &counted).map(n_text).ok_or_else(unreadable_key)).transpose()?;
Ok((DeskResponse::Answered { answer: project(&key, stored)?, counted_answer_id, as_of: image.head }))
Ok(DeskResponse::Answered { answer: project(&key, stored)?, counted_answer_id, as_of: image.head })
}
}
@ -597,6 +619,11 @@ mod tests {
fn desk() -> Mind<MemoryStore> {
let mut mind = seeded();
committed(admit(&mut mind, vec![question_n(1), question_n(2)]));
// Listed, so a reader who is not a decider reaches them (an unlisted fork is the reads' own tests').
for fork in [FORK, FORK2] {
let listing = D::DeskListing(eureka_pipeline::PipelineDeskListing { campaign: slug("eureka-state"), listed: s(fork), reason: "Listed.".into() });
committed(admit(&mut mind, vec![listing]));
}
mind
}
@ -632,14 +659,14 @@ mod tests {
}
fn deciders(list: &[&str]) -> Vec<Short> {
list.iter().map(|who| s(who)).collect()
list.iter().map(|who| account(who)).collect()
}
/// The response and whether the call committed anything: a replay, a refused body and a gate
/// refusal store nothing, so the head does not move.
fn ask_as(mind: &mut Mind<MemoryStore>, who: &str, list: &[&str], request: ForkAnswerRequest) -> (DeskResponse, bool) {
let before = head(mind);
let response = mind.desk_answer(&deciders(list), s(who), DeskAuthority::Member, request, now());
let response = mind.desk_answer(&deciders(list), account(who), DeskAuthority::Member, request, now());
(response, head(mind) != before)
}
@ -779,7 +806,7 @@ mod tests {
request_id: RequestId(text("request_id").unwrap()),
body_sha256: Sha256Hex("0".repeat(64)),
// A withheld vector names no answerer: store a placeholder, the vector is the non-decider's view of it.
answerer: Short(text("answerer").unwrap_or_else(|| "e0d1c2b3-a495-4687-8879-6a5b4c3d2e1f".into())),
answerer: AccountId(text("answerer").unwrap_or_else(|| "e0d1c2b3-a495-4687-8879-6a5b4c3d2e1f".into())),
authority,
outcome,
};
@ -1286,6 +1313,54 @@ mod tests {
}
}
/// Ruling clause 3: a stored request replays whatever the decider list is now, but never to a reader who can no
/// longer see the fork. The replay would carry the fork's current counted answer id and head, so it is the same
/// bytes a missing fork answers with; the stored word itself comes back once the fork is listed again.
#[test]
fn a_stored_answer_does_not_replay_to_a_reader_who_can_no_longer_see_the_fork() {
let mut mind = desk();
let hash = hash_of(&mind, FORK);
// Dan is not a decider: his answer to the listed fork is a stored refusal and counts nothing.
let dan_request = wire(FORK, &hash);
let (first, wrote) = ask(&mut mind, "d", dan_request.clone());
let (refused, counted, _) = answered(first);
assert!((wrote, refused.outcome.as_str(), refused.code.is_some(), counted) == (true, "Refused", true, None), "{refused:?}");
// A decider answers, so the fork now has a counted answer a replay could name.
let x_request = wire(FORK, &hash);
let (counted_by_x, _) = ask(&mut mind, "x", x_request.clone());
assert!(answered(counted_by_x).1.is_some());
// Still listed: the replay is the stored word, with the fork's current state.
let before = head(&mind);
let (listed_replay, wrote) = ask(&mut mind, "d", dan_request.clone());
assert!(!wrote);
let (_, counted, as_of) = answered(listed_replay);
assert_eq!((counted.is_some(), as_of), (true, before));
// Unlisted: the replay is fork_unknown, byte for byte a fork that does not exist, and writes nothing.
let withdrawal = crate::fixtures::resolution_n(
eureka_pipeline::PipelineRef { kind: PipelineKind::DeskListing, id: s("eureka-state:desk_listing:Q1") },
1,
eureka_pipeline::ResolutionOutcome::Withdrawn { reason: "unlisted".into() },
);
let withdrawal_key = eureka_pipeline::pipeline_key(&withdrawal).unwrap();
committed(admit(&mut mind, vec![withdrawal]));
let before = head(&mind);
let (hidden, wrote) = ask(&mut mind, "d", dan_request.clone());
assert!(!wrote);
let (missing, _) = ask(&mut mind, "d", ForkAnswerRequest { fork_id: "eureka-state:question:Q99".into(), ..wire(FORK, &hash) });
assert_eq!(not_stored(hidden.clone()).code, ForkRefusalCode::ForkUnknown);
assert_eq!(encode(&hidden), encode(&missing));
// A decider is told the fork whether or not it is listed.
let (decider_replay, wrote) = ask(&mut mind, "x", x_request);
assert!(!wrote && answered(decider_replay).1.is_some());
assert_eq!(head(&mind), before);
// Relisted: the stored word replays again.
let subject = eureka_pipeline::PipelineRef { kind: PipelineKind::Resolution, id: s(&withdrawal_key) };
committed(admit(&mut mind, vec![crate::fixtures::resolution(subject, eureka_pipeline::ResolutionOutcome::Withdrawn { reason: "listed again".into() })]));
let (relisted, wrote) = ask(&mut mind, "d", dan_request);
assert!(!wrote);
assert_eq!(answered(relisted).0.outcome, "Refused");
}
#[test]
fn showable_needs_two_options_text_depends_and_a_decider() {
let D::Question(plain) = question("Q9", &["A", "B"], "A") else { unreachable!() };
@ -1355,9 +1430,9 @@ mod tests {
let (other, committed) = ask(&mut mind, "y", request.clone());
let (other, _, _) = answered(other);
assert!(committed);
assert_eq!((other.answer_id.as_str(), other.answerer.as_deref().unwrap(), other.outcome.as_str(), other.request_id.as_str()), ("n2", "y", "Revision", request.request_id.as_str()));
assert_eq!((other.answer_id.as_str(), other.answerer.as_deref().unwrap(), other.outcome.as_str(), other.request_id.as_str()), ("n2", account("y").0.as_str(), "Revision", request.request_id.as_str()));
let (original, _, _) = answered(ask(&mut mind, "x", request.clone()).0);
assert_eq!((original.answer_id.as_str(), original.answerer.as_deref().unwrap()), ("n1", "x"));
assert_eq!((original.answer_id.as_str(), original.answerer.as_deref().unwrap()), ("n1", account("x").0.as_str()));
// V17 and V18: an id that is no lowercase D-format UUID is body_invalid naming request_id, and binds nothing.
let before = head(&mind);
for bad in ["retry-1", "0000000A-0000-4000-8000-00000000000B", "{00000000-0000-4000-8000-000000000001}", "00000000-0000-4000-8000-0000000000010", "", " 00000000-0000-4000-8000-000000000001", "00000000-0000-4000-8000-000000000001 ", "00000000-0000-4000-8000-000000000001\n", "00000000-0000-4000-8000-000000000001x"] {
@ -1472,6 +1547,7 @@ mod tests {
assert_eq!(shown_hash(&question).0, her_hash, "{name}: the card's hash");
let mut mind = seeded();
committed(admit(&mut mind, vec![D::Question(question)]));
committed(admit(&mut mind, vec![D::DeskListing(eureka_pipeline::PipelineDeskListing { campaign: slug("eureka-state"), listed: s(FORK), reason: "Listed.".into() })]));
let list: Vec<&str> = card["deciders"].as_array().unwrap().iter().map(|who| who.as_str().unwrap()).collect();
if card["status"] == "closed" {
close(&mut mind, FORK, "RQ1", "no");
@ -1568,27 +1644,27 @@ mod tests {
let mut mind = desk();
let hash = hash_of(&mind, FORK);
// The operator's own account, off the list, is no decider; a member on the list is.
let off = mind.desk_answer(&deciders(&["x"]), s("op"), DeskAuthority::Operator, wire(FORK, &hash), now());
let off = mind.desk_answer(&deciders(&["x"]), account("op"), DeskAuthority::Operator, wire(FORK, &hash), now());
assert_eq!(answered(off).0.code, Some(ForkAnswerCode::NotADecider));
let on = mind.desk_answer(&deciders(&["m"]), s("m"), DeskAuthority::Member, wire(FORK, &hash), now());
let on = mind.desk_answer(&deciders(&["m"]), account("m"), DeskAuthority::Member, wire(FORK, &hash), now());
let (answer, _, _) = answered(on);
assert_eq!((answer.outcome.as_str(), answer.authority, answer.answerer.as_deref().unwrap()), ("Counted", DeskAuthority::Member, "m"));
assert_eq!((answer.outcome.as_str(), answer.authority, answer.answerer.as_deref().unwrap()), ("Counted", DeskAuthority::Member, account("m").0.as_str()));
}
// ---- the desk request on the wire: deciders ride each call, at most 64, never stored ----
use crate::envelope::{OperationFailure, decode_desk_request, encode_desk_request};
fn list_of(n: usize) -> Vec<Short> {
(0..n).map(|at| s(&format!("d{at}"))).collect()
fn list_of(n: usize) -> Vec<AccountId> {
(0..n).map(|at| acct(&format!("d{at}"))).collect()
}
/// Each request kind of the desk, carrying `list`.
fn desk_requests(mind: &str, list: &[Short], hash: &str) -> [DeskRequest; 3] {
fn desk_requests(mind: &str, list: &[AccountId], hash: &str) -> [DeskRequest; 3] {
[
DeskRequest::Answer { mind: slug(mind), subject: s("d0"), authority: DeskAuthority::Member, deciders: list.to_vec(), request: wire(FORK, hash) },
DeskRequest::Forks { mind: slug(mind), subject: s("d0"), deciders: list.to_vec(), status: None, cursor: None, limit: ForkLimit::new(10).unwrap() },
DeskRequest::Fork { mind: slug(mind), subject: s("d0"), deciders: list.to_vec(), fork_id: FORK.into() },
DeskRequest::Answer { mind: slug(mind), subject: acct("d0"), authority: DeskAuthority::Member, deciders: list.to_vec(), request: wire(FORK, hash) },
DeskRequest::Forks { mind: slug(mind), subject: acct("d0"), deciders: list.to_vec(), status: None, cursor: None, limit: ForkLimit::new(10).unwrap() },
DeskRequest::Fork { mind: slug(mind), subject: acct("d0"), deciders: list.to_vec(), fork_id: FORK.into() },
]
}
@ -1611,7 +1687,7 @@ mod tests {
// 64 deciders through to the Mind: the subject is listed and counts.
let DeskRequest::Answer { subject, authority, deciders, request, .. } = through_the_door(&desk_requests(INSTANCE, &list_of(64), &hash)[0]).unwrap() else { panic!("not an answer") };
let before = head(&mind);
let response = mind.desk_answer(&deciders, subject, authority, request, now());
let response = mind.desk_answer(&deciders.iter().map(|id| s(&id.0)).collect::<Vec<_>>(), s(&subject.0), authority, request, now());
assert!(head(&mind) > before);
assert_eq!(answered(response).0.outcome, "Counted");
@ -1631,7 +1707,7 @@ mod tests {
let hash = hash_of(&mind, FORK);
let before = (head(&mind), stored_count(&mind, FORK));
let mut twice = list_of(3);
twice.push(s("d1"));
twice.push(acct("d1"));
for (what, list) in [("empty", vec![]), ("duplicate", twice)] {
for sent in desk_requests(INSTANCE, &list, &hash) {
let failure = through_the_door(&sent).unwrap_err();
@ -1667,6 +1743,6 @@ mod tests {
// The same request to the mind it names is answered.
let DeskRequest::Answer { mind: named, subject, authority, deciders, request } = through_the_door(&desk_requests(INSTANCE, &list_of(4), &hash)[0]).unwrap() else { panic!("not an answer") };
mind.require_instance(&named).unwrap();
assert_eq!(answered(mind.desk_answer(&deciders, subject, authority, request, now())).0.outcome, "Counted");
assert_eq!(answered(mind.desk_answer(&deciders.iter().map(|id| s(&id.0)).collect::<Vec<_>>(), s(&subject.0), authority, request, now())).0.outcome, "Counted");
}
}

View file

@ -352,6 +352,18 @@ fn card_of(docs: &Docs<'_>, key: &str, order: u64, question: &PipelineQuestion,
}
}
/// Whether a reader may see this fork at all: a decider sees every showable fork (they must, to answer
/// it); anyone else only while the Mind's `desk_listing` for it is in force (derived from the
/// resolutions on every read; no status is stored). A fork nobody may see reads as a missing one.
pub(super) fn visible(docs: &Docs<'_>, key: &str, decider: bool) -> bool {
decider || {
let (root, label) = root_and_local(key);
let listing = format!("{root}:desk_listing:{label}");
// `in_force` is true of a document nobody wrote: a listing counts only if the Mind holds it.
docs.in_image(PipelineKind::DeskListing, &listing).is_some() && docs.in_force(PipelineKind::DeskListing, &listing)
}
}
/// Every question the Mind holds at the snapshot, showable or not, with its rank: the 0-based position
/// in ascending (admission ordinal, key) order. A rank is unique, and admissions only append, so a held
/// question's rank never moves; a question the deciders do not show leaves a gap and shifts no other.
@ -427,15 +439,17 @@ impl<S: MindStore> Mind<S> {
/// The fork itself is still its whole entry on `desk_fork`; if that entry is past the reply cap the door
/// answers the read with `fork_too_large` naming the fork (no order, no cursor), not `ResponseTooLarge`. Nothing is dropped silently.
///
/// The call's `deciders` feed the showability gate; the call's `subject` decides which view of each entry
/// is built (a decider's, with ids, or a non-decider's, with counts). It gates no fork. Notes stay whole
/// (Bifrost strips them).
/// The call's `deciders` feed the showability gate; the call's `subject` decides which forks are visible and
/// which view of each entry is built. A decider sees every showable fork, with ids; any other reader sees a
/// showable fork only while the Mind's `desk_listing` for it is in force, with counts. The counts, the
/// pages and the cursor are over the visible set; `order` and `stamp` are not (a non-decider sees gaps).
/// Notes stay whole (Bifrost strips them).
pub fn desk_forks(&self, subject: &Short, deciders: &[Short], status: Option<ListStatus>, cursor: Option<ForkCursor>, limit: ForkLimit) -> DeskResponse {
self.forks_page(subject, deciders, status, cursor, limit).unwrap_or_else(door_refusal)
}
/// One fork, as the entry a page carries, read at one snapshot. A fork that is not held, is not a
/// question or is not showable is `fork_unknown`, the answer path's refusal. The view is the reader's, as `desk_forks`.
/// question, is not showable or is not visible to the reader is `fork_unknown`, the answer path's refusal, with the same bytes as a missing fork. The view is the reader's, as `desk_forks`.
pub fn desk_fork(&self, subject: &Short, deciders: &[Short], fork_id: &str) -> DeskResponse {
self.fork_entry(subject, deciders, fork_id).unwrap_or_else(door_refusal)
}
@ -444,7 +458,7 @@ impl<S: MindStore> Mind<S> {
let image = self.image()?;
let docs = Docs::at(image, image.head);
let decider = deciders.contains(subject);
let mut shown: Vec<_> = ranked(&docs)?.into_iter().filter(|(_, _, question)| showable(question, deciders)).map(|(order, key, question)| (order, key, question, docs.in_force(PipelineKind::Question, key))).collect();
let mut shown: Vec<_> = ranked(&docs)?.into_iter().filter(|(_, key, question)| showable(question, deciders) && visible(&docs, key, decider)).map(|(order, key, question)| (order, key, question, docs.in_force(PipelineKind::Question, key))).collect();
shown.reverse();
let open_count = shown.iter().filter(|fork| fork.3).count() as u64;
let recorded_count = shown.len() as u64 - open_count;
@ -499,7 +513,7 @@ impl<S: MindStore> Mind<S> {
let Some((order, key, question)) = ranked(&docs)?.into_iter().find(|(_, key, _)| *key == fork_id) else {
return Ok(DeskResponse::NotStored(fork_unknown()));
};
if !showable(question, deciders) {
if !showable(question, deciders) || !visible(&docs, key, decider) {
return Ok(DeskResponse::NotStored(fork_unknown()));
}
Ok(DeskResponse::Fork { entry: entry_of(&docs, key, order, question, deciders, decider)?, as_of: image.head })
@ -536,7 +550,7 @@ mod tests {
}
fn list(who: &[&str]) -> Vec<Short> {
who.iter().map(|id| s(id)).collect()
who.iter().map(|id| account(id)).collect()
}
fn head(mind: &Mind<MemoryStore>) -> u64 {
@ -568,7 +582,7 @@ mod tests {
answered_at: "2026-09-16T11:00:00Z".into(),
request_id: request_id(),
};
mind.desk_answer(&list(deciders), s(who), DeskAuthority::Member, request, now())
mind.desk_answer(&list(deciders), account(who), DeskAuthority::Member, request, now())
}
fn close(mind: &mut Mind<MemoryStore>, fork: &str, label: &str) {
@ -605,7 +619,7 @@ mod tests {
}
fn all(mind: &Mind<MemoryStore>, n: u8) -> ForkList {
page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(n))).0
page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(n))).0
}
// ---- the reads, one snapshot ----
@ -616,7 +630,7 @@ mod tests {
let stale = answer(&mut mind, "x", &LIST, &fork(1), &"0".repeat(64), None);
let DeskResponse::Answered { answer: stale, .. } = stale else { panic!("a stale answer is stored") };
assert_eq!(stale.code, Some(ForkAnswerCode::StaleFork));
let (shown, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (shown, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
assert_eq!(shown.card.options_hash, hash_of(&mind, &fork(1)));
assert_ne!(shown.card.options_hash, "0".repeat(64));
assert_eq!((shown.record.answers.len(), shown.record.counted_answer_id.clone()), (1, None));
@ -627,7 +641,7 @@ mod tests {
};
assert_eq!((counted.outcome.as_str(), counted.answer_id.as_str()), ("Counted", "n2"));
assert_eq!(counted_answer_id.as_deref(), Some("n2"));
let (after, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (after, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
assert_eq!(after.record.counted_answer_id.as_deref(), Some("n2"));
let names: Vec<_> = after.record.answers.iter().map(|answer| answer.answer_id.as_str()).collect();
assert_eq!(names, ["n1", "n2"], "answers in admission order, named n<seq>");
@ -636,23 +650,23 @@ mod tests {
#[test]
fn list_then_read_at_one_stamp() {
let mut mind = desk(3);
let (first, as_of) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(2)));
let (first, as_of) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(2)));
assert_eq!(as_of, head(&mind));
assert_eq!(first.stamp, format!("s{as_of}"));
assert_eq!(first.mind, INSTANCE);
assert_eq!(ids(&first), [fork(3), fork(2)]);
let next = first.next_cursor.expect("one more fork waits");
assert_eq!(next.order(), first.forks[1].card.order);
let (second, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, Some(next), limit(2)));
let (second, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, Some(next), limit(2)));
assert_eq!((ids(&second), second.next_cursor), (vec![fork(1)], None));
// The read at that stamp is the entry the list carried.
let (read, read_at) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(2)));
let (read, read_at) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(2)));
assert_eq!(read_at, as_of);
assert_eq!(read, first.forks[1]);
// Writing moves the stamp, and only then.
let hash = hash_of(&mind, &fork(2));
answer(&mut mind, "x", &LIST, &fork(2), &hash, None);
let (moved, moved_at) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(2)));
let (moved, moved_at) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(2)));
assert_eq!((moved.stamp.clone(), moved_at), (format!("s{}", head(&mind)), head(&mind)));
assert_ne!(moved.stamp, first.stamp);
assert_eq!(moved.forks[1].record.answers.len(), 1);
@ -671,17 +685,17 @@ mod tests {
assert_eq!(ForkCursor::new(0), None, "no cursor names rank 0: nothing is below it");
for (at, expected) in [(0, 3), (1, 2), (2, 1)] {
let below = ForkCursor::new(orders[at]).unwrap();
let (rest, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, Some(below), limit(50)));
let (rest, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, Some(below), limit(50)));
assert_eq!(rest.forks.len(), expected, "below order {}", orders[at]);
assert!(rest.forks.iter().all(|entry| entry.card.order < orders[at]));
}
// A page that stops short names the last order it returned.
let (short, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(1)));
let (short, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(1)));
assert_eq!((ids(&short), short.next_cursor.map(ForkCursor::order)), (vec![fork(4)], Some(orders[0])));
// A page that ends exactly on the last fork has no cursor.
let (exact, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(4)));
let (exact, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(4)));
assert_eq!((exact.forks.len(), exact.next_cursor), (4, None));
let (over, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50)));
let (over, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50)));
assert_eq!(over.forks.len(), 4);
}
@ -694,7 +708,7 @@ mod tests {
for status in [None, Some(ListStatus::Open), Some(ListStatus::Recorded)] {
for cursor in [None, ForkCursor::new(1), ForkCursor::new(3), ForkCursor::new(1000)] {
for size in [0, 1, 50] {
let (shown, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), status, cursor, limit(size)));
let (shown, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), status, cursor, limit(size)));
assert_eq!((shown.open_count, shown.recorded_count), expected, "{status:?} {cursor:?} {size}");
assert!(shown.forks.len() <= size as usize);
for entry in &shown.forks {
@ -708,31 +722,31 @@ mod tests {
}
}
}
let (open, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), Some(ListStatus::Open), None, limit(50)));
let (open, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), Some(ListStatus::Open), None, limit(50)));
assert_eq!(ids(&open), [fork(5), fork(3), fork(2)]);
let (recorded, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), Some(ListStatus::Recorded), None, limit(50)));
let (recorded, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), Some(ListStatus::Recorded), None, limit(50)));
assert_eq!(ids(&recorded), [fork(4), fork(1)]);
// A cursor and a status together: the cards below the cursor that have the status.
let below_four = ForkCursor::new(recorded.forks[0].card.order);
let (rest, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), Some(ListStatus::Open), below_four, limit(50)));
let (rest, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), Some(ListStatus::Open), below_four, limit(50)));
assert_eq!(ids(&rest), [fork(3), fork(2)]);
}
#[test]
fn limit_zero_is_the_counts_and_the_stamp_and_no_entries() {
let mind = desk(3);
let (counts, as_of) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(0)));
let (counts, as_of) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(0)));
assert_eq!((counts.forks.len(), counts.next_cursor), (0, None));
assert_eq!((counts.open_count, counts.recorded_count), (3, 0));
assert_eq!((counts.stamp, as_of), (format!("s{}", head(&mind)), head(&mind)));
let (one, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(1)));
let (one, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(1)));
assert_eq!(one.forks.len(), 1, "limit 1 is one entry, not none");
}
#[test]
fn a_desk_with_no_forks_is_an_empty_page() {
let mind = seeded();
let (empty, as_of) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(10)));
let (empty, as_of) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(10)));
assert_eq!((empty.open_count, empty.recorded_count, empty.forks.len(), empty.next_cursor), (0, 0, 0, None));
assert_eq!(empty.stamp, format!("s{as_of}"));
}
@ -757,11 +771,11 @@ mod tests {
let shown = all(&mind, 50);
assert_eq!(ids(&shown), [fork(2), fork(1)]);
assert_eq!((shown.open_count, shown.recorded_count), (2, 0), "the count is over the showable set");
unknown(mind.desk_fork(&s(LIST[0]), &list(&LIST), &unshowable));
unknown(mind.desk_fork(&account(LIST[0]), &list(&LIST), &unshowable));
// No decider, nothing is shown at all: the gate needs one.
let (none, _) = page(mind.desk_forks(&s(LIST[0]), &[], None, None, limit(50)));
let (none, _) = page(mind.desk_forks(&account(LIST[0]), &[], None, None, limit(50)));
assert_eq!((none.forks.len(), none.open_count, none.recorded_count), (0, 0, 0));
unknown(mind.desk_fork(&s(LIST[0]), &[], &fork(1)));
unknown(mind.desk_fork(&account(LIST[0]), &[], &fork(1)));
}
#[test]
@ -770,12 +784,12 @@ mod tests {
close(&mut mind, &fork(1), "R1");
let ruling_key = format!("{CAMPAIGN_ROOT}:ruling:R1");
for id in [fork(9).as_str(), "", "Q1", "no-colons", "eureka-state:question:", ruling_key.as_str(), &"x".repeat(5000), "other-campaign:question:Q1"] {
unknown(mind.desk_fork(&s(LIST[0]), &list(&LIST), id));
unknown(mind.desk_fork(&account(LIST[0]), &list(&LIST), id));
}
// And nothing was written by looking.
let before = head(&mind);
let _ = mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1));
let _ = mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50));
let _ = mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1));
let _ = mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50));
assert_eq!(head(&mind), before);
}
@ -790,6 +804,9 @@ mod tests {
/// and Ben answers a second time (a retraction): four stored answers, three answerers.
fn answered_desk() -> Mind<MemoryStore> {
let mut mind = desk(2);
// Listed: a reader who is not a decider sees these forks (and their counts) at all.
list_it(&mut mind, 1);
list_it(&mut mind, 2);
let hash = hash_of(&mind, &fork(1));
let deciders = [ANNA, BEN];
answer(&mut mind, ANNA, &deciders, &fork(1), &hash, Some("Anna's note"));
@ -812,8 +829,8 @@ mod tests {
}
fn reader(mind: &Mind<MemoryStore>, subject: &str, deciders: &[&str]) -> (ForkList, ForkEntry, Vec<u8>) {
let (page, _) = page(mind.desk_forks(&s(subject), &list(deciders), None, None, limit(50)));
let response = mind.desk_fork(&s(subject), &list(deciders), &fork(1));
let (page, _) = page(mind.desk_forks(&account(subject), &list(deciders), None, None, limit(50)));
let response = mind.desk_fork(&account(subject), &list(deciders), &fork(1));
let bytes = encode_desk_response("m", "fork", &response, "r").unwrap();
let bytes = cultnet_rs::encode_cultnet_message_to_vec(&bytes, cultnet_rs::CultNetWireContract::CultNetSchemaV0).unwrap();
let (entry, _) = entry(response);
@ -890,7 +907,7 @@ mod tests {
// ---- raised_at: the question's first admission ----
fn raised_at(mind: &Mind<MemoryStore>, fork_id: &str) -> Option<String> {
entry(mind.desk_fork(&s(ANNA), &list(&[ANNA]), fork_id)).0.card.raised_at
entry(mind.desk_fork(&account(ANNA), &list(&[ANNA]), fork_id)).0.card.raised_at
}
#[test]
@ -964,18 +981,246 @@ mod tests {
#[test]
fn raised_at_is_outside_options_hash() {
let mind = desk(1);
let (read, _) = entry(mind.desk_fork(&s(ANNA), &list(&[ANNA]), &fork(1)));
let (read, _) = entry(mind.desk_fork(&account(ANNA), &list(&[ANNA]), &fork(1)));
let moved = ForkCard { raised_at: Some("2020-01-01T00:00:00Z".into()), ..read.card.clone() };
assert_ne!(moved.raised_at, read.card.raised_at);
assert_eq!(moved.options_hash, read.card.options_hash);
assert_eq!(read.card.options_hash, hash_of(&mind, &fork(1)));
}
// ---- the listing gate: a non-decider reads a fork only while the Mind lists it ----
fn listing_doc(n: u32) -> D {
D::DeskListing(eureka_pipeline::PipelineDeskListing { campaign: slug(CAMPAIGN_ROOT), listed: s(&fork(n)), reason: "Members may read why.".into() })
}
fn listing_ref(n: u32) -> PipelineRef {
PipelineRef { kind: PipelineKind::DeskListing, id: s(&format!("{CAMPAIGN_ROOT}:desk_listing:Q{n}")) }
}
fn list_it(mind: &mut Mind<MemoryStore>, n: u32) {
committed(admit(mind, vec![listing_doc(n)]));
}
/// Unlist: a `Withdrawn` resolution on the listing, `sequence` its per-subject sequence.
fn unlist(mind: &mut Mind<MemoryStore>, n: u32, sequence: u32) -> String {
let withdrawal = crate::fixtures::resolution_n(listing_ref(n), sequence, eureka_pipeline::ResolutionOutcome::Withdrawn { reason: "unlisted".into() });
let key = eureka_pipeline::pipeline_key(&withdrawal).unwrap();
committed(admit(mind, vec![withdrawal]));
key
}
/// Relist: withdraw the withdrawal.
fn relist(mind: &mut Mind<MemoryStore>, withdrawal: &str) {
let subject = PipelineRef { kind: PipelineKind::Resolution, id: s(withdrawal) };
committed(admit(mind, vec![crate::fixtures::resolution(subject, eureka_pipeline::ResolutionOutcome::Withdrawn { reason: "listed again".into() })]));
}
fn seen_by(mind: &Mind<MemoryStore>, subject: &str) -> Vec<String> {
ids(&page(mind.desk_forks(&account(subject), &list(&[ANNA, BEN]), None, None, limit(50))).0)
}
fn read_by(mind: &Mind<MemoryStore>, subject: &str, n: u32) -> DeskResponse {
mind.desk_fork(&account(subject), &list(&[ANNA, BEN]), &fork(n))
}
/// Negative first: with nothing listed, a non-decider is shown nothing, and a read of a real fork
/// is the same bytes as a read of one that does not exist.
#[test]
fn an_unlisted_fork_is_invisible_to_a_non_decider() {
let mind = desk(3);
let (shown, _) = page(mind.desk_forks(&account(DAN), &list(&[ANNA, BEN]), None, None, limit(50)));
assert_eq!((shown.forks.len(), shown.open_count, shown.recorded_count, shown.next_cursor), (0, 0, 0, None));
let (counts, _) = page(mind.desk_forks(&account(DAN), &list(&[ANNA, BEN]), None, None, limit(0)));
assert_eq!((counts.open_count, counts.recorded_count), (0, 0));
let real = read_by(&mind, DAN, 1);
unknown(real.clone());
assert_eq!(real, read_by(&mind, DAN, 99), "a hidden fork reads as a missing one");
let bytes = |response: &DeskResponse| rmp_serde::to_vec_named(response).unwrap();
assert_eq!(bytes(&real), bytes(&read_by(&mind, DAN, 99)));
}
#[test]
fn a_decider_sees_unlisted_forks() {
let mut mind = desk(3);
list_it(&mut mind, 2);
assert_eq!(seen_by(&mind, ANNA), [fork(3), fork(2), fork(1)]);
assert_eq!(seen_by(&mind, DAN), [fork(2)]);
// Listing is not read for a decider: unlisting changes nothing for them.
unlist(&mut mind, 2, 1);
assert_eq!(seen_by(&mind, ANNA), [fork(3), fork(2), fork(1)]);
assert!(seen_by(&mind, DAN).is_empty());
}
#[test]
fn listing_flips_and_is_audited() {
let mut mind = desk(2);
let mut states = vec![seen_by(&mind, DAN)];
let before = head(&mind);
list_it(&mut mind, 1);
states.push(seen_by(&mind, DAN));
let withdrawal = unlist(&mut mind, 1, 1);
states.push(seen_by(&mind, DAN));
relist(&mut mind, &withdrawal);
states.push(seen_by(&mind, DAN));
unlist(&mut mind, 1, 2);
states.push(seen_by(&mind, DAN));
let q1 = vec![fork(1)];
assert_eq!(states, [vec![], q1.clone(), vec![], q1, vec![]]);
assert_eq!(head(&mind), before + 4, "every flip is a document with its own receipt");
assert!(matches!(read_by(&mind, DAN, 1), DeskResponse::NotStored(_)));
}
/// Re-admitting the listing after unlisting is the same document: AlreadyAdmitted, and the fork stays hidden.
#[test]
fn admitting_the_listing_again_after_unlisting_does_not_relist() {
let mut mind = desk(1);
list_it(&mut mind, 1);
let withdrawal = unlist(&mut mind, 1, 1);
let before = head(&mind);
let again = admit(&mut mind, vec![listing_doc(1)]);
assert!(matches!(again, PipelineAdmissionOutcome::AlreadyAdmitted { .. }), "{again:?}");
assert_eq!((head(&mind), seen_by(&mind, DAN)), (before, Vec::<String>::new()));
relist(&mut mind, &withdrawal);
assert_eq!(seen_by(&mind, DAN), [fork(1)]);
}
#[test]
fn counts_and_cursor_cover_the_visible_set() {
let mut mind = desk(5);
for n in [2, 4] {
list_it(&mut mind, n);
}
close(&mut mind, &fork(4), "R4");
let list_for = |subject: &str, cursor, size| page(mind.desk_forks(&account(subject), &list(&[ANNA, BEN]), None, cursor, limit(size))).0;
let counts = list_for(DAN, None, 0);
assert_eq!((counts.open_count, counts.recorded_count), (1, 1));
// Pages are gapless over the visible forks and the cursor never names a hidden one.
let first = list_for(DAN, None, 1);
assert_eq!(ids(&first), [fork(4)]);
let cursor = first.next_cursor.expect("one more visible fork");
let second = list_for(DAN, Some(cursor), 1);
assert_eq!((ids(&second), second.next_cursor), (vec![fork(2)], None));
// Order keeps its rank among ALL the Mind's questions: the gap is visible, the text is not.
assert_eq!(orders(&first).into_iter().chain(orders(&second)).collect::<Vec<_>>(), [3, 1]);
}
#[test]
fn a_listing_needs_a_question_of_its_campaign() {
let mut mind = desk(1);
let missing = refusal(admit(&mut mind, vec![listing_doc(9)]));
assert!(matches!(missing, MindRefusal::MissingReference { kind: PipelineKind::Question, .. }), "{missing:?}");
let D::DeskListing(mut other) = listing_doc(1) else { unreachable!() };
other.campaign = slug("other-campaign");
let before = head(&mind);
assert!(matches!(admit(&mut mind, vec![D::DeskListing(other)]), PipelineAdmissionOutcome::Refused(_)));
assert_eq!(head(&mind), before);
}
/// The answer path is no oracle: a non-decider answering a hidden fork gets the bytes of answering
/// one that never existed, and nothing is stored. A listed fork still stores the NotADecider refusal.
#[test]
fn a_non_decider_answering_a_hidden_fork_learns_nothing_and_stores_nothing() {
let mut mind = desk(2);
let hash = hash_of(&mind, &fork(1));
let before = (head(&mind), stored_answers(&mind));
let hidden = answer(&mut mind, DAN, &[ANNA, BEN], &fork(1), &hash, None);
let missing = answer(&mut mind, DAN, &[ANNA, BEN], &fork(99), &hash, None);
unknown(hidden.clone());
assert_eq!(rmp_serde::to_vec_named(&hidden).unwrap(), rmp_serde::to_vec_named(&missing).unwrap());
assert_eq!((head(&mind), stored_answers(&mind)), before, "nothing stored");
list_it(&mut mind, 1);
let listed = answer(&mut mind, DAN, &[ANNA, BEN], &fork(1), &hash, None);
let DeskResponse::Answered { answer: refused, .. } = listed else { panic!("a listed fork stores the refusal: {listed:?}") };
assert_eq!(refused.code, Some(ForkAnswerCode::NotADecider));
assert_eq!(stored_answers(&mind), before.1 + 1);
}
fn stored_answers(mind: &Mind<MemoryStore>) -> usize {
let image = mind.image().unwrap();
Docs::at(image, image.head).of_kind(PipelineKind::DeskAnswer).count()
}
#[test]
fn the_desk_door_cannot_admit_a_listing() {
let mut mind = desk(1);
let envelope = crate::fixtures::prepare(&listing_doc(1));
let instance = mind.instance().clone();
let outcome = mind.admit_steps(crate::admission::Door::Desk, &instance, provenance(Faculty::Operator), vec![envelope], now());
assert_eq!(outcome.unwrap_err(), MindRefusal::DeskRequestInvalid { field: "desk_listing".into() });
assert!(seen_by(&mind, DAN).is_empty());
}
#[test]
fn listed_role_is_carried_by_desk_listing_only() {
use crate::docs::CitationRole as R;
assert_eq!((R::Listed.carrier(), R::Listed.target_kinds(), R::Listed.name()), (PipelineKind::DeskListing, vec![PipelineKind::Question], "listed"));
assert_eq!(R::Fork.carrier(), PipelineKind::DeskAnswer);
assert_eq!(R::ALL.len(), 27);
}
// ---- the account grammar at decode ----
#[test]
fn a_decider_or_subject_outside_the_account_grammar_is_refused_at_decode() {
let ask = ForkAnswerRequest {
schema_version: RequestSchema::V0,
fork_id: fork(1),
options_hash: "h".into(),
answer: Some(ForkAnswer { choice: Some("A".into()), note: None }),
retracted: false,
retracts: None,
answered_at: "2026-09-16T11:00:00Z".into(),
request_id: request_id(),
};
let (mind_slug, who) = (slug(INSTANCE), acct("a"));
let requests = [
DeskRequest::Answer { mind: mind_slug.clone(), subject: who.clone(), authority: DeskAuthority::Member, deciders: vec![who.clone()], request: ask },
DeskRequest::Forks { mind: mind_slug.clone(), subject: who.clone(), deciders: vec![who.clone()], status: None, cursor: None, limit: ForkLimit::new(10).unwrap() },
DeskRequest::Fork { mind: mind_slug, subject: who.clone(), deciders: vec![who.clone()], fork_id: fork(1) },
];
for request in &requests {
let operation = request.operation();
let variant = match operation {
"answer" => "Answer",
"forks" => "Forks",
_ => "Fork",
};
let good = encode_desk_request_value(operation, &serde_json::to_value(request).unwrap());
decode_desk_request(&good).unwrap_or_else(|failure| panic!("{operation}: {failure:?}"));
let bad_ids = ["@x:server", "meta@gamecult.org", "someone", "7", &who.0.to_uppercase(), &format!("{{{}}}", who.0), ""];
for bad in bad_ids {
for field in ["subject", "deciders"] {
let mut value = serde_json::to_value(request).unwrap();
value[variant][field] = if field == "subject" { bad.into() } else { json!([bad]) };
let failure = decode_desk_request(&encode_desk_request_value(operation, &value)).unwrap_err();
assert_eq!(failure.code, "payload-not-a-request", "{operation} {field}: {bad:?}");
assert!(failure.message.contains(field), "{operation}: {}", failure.message);
assert!(bad.is_empty() || !failure.message.contains(bad), "the refusal quotes {bad:?}: {}", failure.message);
}
}
}
}
fn encode_desk_request_value(operation: &str, value: &Value) -> cultnet_rs::CultNetMessage {
use base64::Engine;
cultnet_rs::CultNetMessage::OperationRequest {
message_id: "m".into(),
service_id: DESK_SERVICE_ID.into(),
operation: operation.into(),
payload_schema: DESK_REQUEST_SCHEMA.into(),
payload_encoding: "messagepack-base64".into(),
payload: base64::engine::general_purpose::STANDARD.encode(rmp_serde::to_vec_named(value).unwrap()),
source_runtime_id: None,
target_runtime_id: None,
}
}
// ---- the card refuses a count that does not match ----
fn card_with(deciders: usize, decider_count: u64) -> Vec<u8> {
let mind = desk(1);
let (read, _) = entry(mind.desk_fork(&s(ANNA), &list(&[ANNA]), &fork(1)));
let (read, _) = entry(mind.desk_fork(&account(ANNA), &list(&[ANNA]), &fork(1)));
let ids: Vec<String> = (0..deciders).map(|n| format!("00000000-0000-4000-8000-{n:012x}")).collect();
rmp_serde::to_vec_named(&ForkCard { deciders: ids, decider_count, ..read.card }).unwrap()
}
@ -1015,7 +1260,7 @@ mod tests {
held.raised_in = Some(PipelineRef { kind: PipelineKind::Ruling, id: s("eureka-state:ruling:R0") });
committed(admit(&mut mind, vec![D::Ruling(ruling("R0"))]));
committed(admit(&mut mind, vec![shown]));
let (read, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (read, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
let card = &read.card;
assert_eq!((card.fork_id.as_str(), card.title.as_str(), card.context.as_str()), (fork(1).as_str(), "Share the mark?", "Should a slot mark be shared?"));
let options: Vec<_> = card.options.iter().map(|option| (option.id.as_str(), option.text.as_str())).collect();
@ -1024,7 +1269,7 @@ mod tests {
assert_eq!(card.depends, ["eureka-state:question:other", "a prose line"]);
assert_eq!(card.raised_in, Some(ForkRef { kind: "Ruling".into(), id: "eureka-state:ruling:R0".into() }));
assert_eq!((card.leans.clone(), card.flag.clone(), card.status, card.ruling.clone(), card.ruled_from.clone()), (None, None, CardStatus::Open, None, vec![]));
assert_eq!(card.deciders, LIST);
assert_eq!(card.deciders, LIST.iter().map(|id| account(id).0).collect::<Vec<_>>());
assert_eq!(card.options_hash, hash_of(&mind, &fork(1)));
assert_eq!(card.order, 0, "the only question is rank 0");
assert_eq!(read.record.fork_id, fork(1));
@ -1035,9 +1280,9 @@ mod tests {
fn a_ruling_closes_the_card_and_names_itself() {
let mut mind = desk(2);
close(&mut mind, &fork(1), "R1");
let (closed, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (closed, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
assert_eq!((closed.card.status, closed.card.ruling.as_deref()), (CardStatus::Closed, Some("eureka-state:ruling:R1")));
let (open, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(2)));
let (open, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(2)));
assert_eq!((open.card.status, open.card.ruling), (CardStatus::Open, None));
}
@ -1047,7 +1292,7 @@ mod tests {
let hash = hash_of(&mind, &fork(1));
answer(&mut mind, "x", &LIST, &fork(1), &hash, None);
answer(&mut mind, "y", &LIST, &fork(1), &hash, None);
let (read, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (read, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
assert_eq!(read.record.counted_answer_id.as_deref(), Some("n1"));
let outcomes: Vec<_> = read.record.answers.iter().map(|answer| (answer.outcome.as_str(), answer.of.as_deref())).collect();
assert_eq!(outcomes, [("Counted", None), ("Revision", Some("n1"))]);
@ -1062,9 +1307,9 @@ mod tests {
answered_at: "2026-09-16T11:00:00Z".into(),
request_id: request_id(),
};
let stored = mind.desk_answer(&list(&LIST), s("x"), DeskAuthority::Member, retract, now());
let stored = mind.desk_answer(&list(&LIST), account("x"), DeskAuthority::Member, retract, now());
assert!(matches!(stored, DeskResponse::Answered { .. }));
let (after, _) = entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(1)));
let (after, _) = entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(1)));
assert_eq!(after.record.counted_answer_id.as_deref(), Some("n2"));
}
@ -1236,7 +1481,7 @@ mod tests {
let file = transport();
let flat = |message: &Value| {
json!({ "Forks": {
"mind": message["mind"], "subject": "s", "deciders": ["s"],
"mind": message["mind"], "subject": "00000000-0000-4000-8000-0000000000e5", "deciders": ["00000000-0000-4000-8000-0000000000e5"],
"status": message["status"], "cursor": message["cursor"], "limit": message["limit"]
}})
};
@ -1261,7 +1506,7 @@ mod tests {
}
fn forks_json(status: Value, cursor: Value, limit: Value) -> Value {
json!({ "Forks": { "mind": INSTANCE, "subject": "s", "deciders": ["s"], "status": status, "cursor": cursor, "limit": limit } })
json!({ "Forks": { "mind": INSTANCE, "subject": "00000000-0000-4000-8000-0000000000e5", "deciders": ["00000000-0000-4000-8000-0000000000e5"], "status": status, "cursor": cursor, "limit": limit } })
}
#[test]
@ -1308,7 +1553,7 @@ mod tests {
#[test]
fn a_read_that_names_an_authority_is_not_a_request() {
let mut forks = forks_json(Value::Null, Value::Null, json!(5));
let mut fork = json!({ "Fork": { "mind": INSTANCE, "subject": "s", "deciders": ["s"], "fork_id": fork(1) } });
let mut fork = json!({ "Fork": { "mind": INSTANCE, "subject": "00000000-0000-4000-8000-0000000000e5", "deciders": ["00000000-0000-4000-8000-0000000000e5"], "fork_id": fork(1) } });
assert!(serde_json::from_value::<DeskRequest>(forks.clone()).is_ok() && serde_json::from_value::<DeskRequest>(fork.clone()).is_ok());
for authority in [json!("Operator"), json!("Member"), Value::Null] {
forks["Forks"]["authority"] = authority.clone();
@ -1317,7 +1562,7 @@ mod tests {
assert!(serde_json::from_value::<DeskRequest>(fork.clone()).is_err(), "Fork with {authority}");
}
// And through the envelope the door reads.
let typed = DeskRequest::Forks { mind: slug(INSTANCE), subject: s("s"), deciders: list(&["s"]), status: None, cursor: None, limit: limit(5) };
let typed = DeskRequest::Forks { mind: slug(INSTANCE), subject: acct("s"), deciders: vec![acct("s")], status: None, cursor: None, limit: limit(5) };
let message = encode_desk_request("m-1", &typed, None).unwrap();
assert_eq!(decode_desk_request(&message).unwrap().1, typed);
}
@ -1355,7 +1600,7 @@ mod tests {
fn pages(mind: &Mind<MemoryStore>, size: u8) -> Vec<(DeskResponse, ForkList)> {
let (mut cursor, mut out) = (None, Vec::new());
loop {
let response = mind.desk_forks(&s(LIST[0]), &list(&LIST), None, cursor, limit(size));
let response = mind.desk_forks(&account(LIST[0]), &list(&LIST), None, cursor, limit(size));
let (page, _) = page(response.clone());
cursor = page.next_cursor;
out.push((response, page));
@ -1406,7 +1651,7 @@ mod tests {
}
mind
};
let first_page = |mind: &Mind<MemoryStore>| page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50))).0;
let first_page = |mind: &Mind<MemoryStore>| page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50))).0;
// The most forks that fit one page with the oldest one's text short.
let fits = (2..=50).take_while(|forks| first_page(&build(*forks, 300)).forks.len() == *forks as usize).last().expect("two fit");
assert!(fits < 50, "the desk outgrows a page");
@ -1417,18 +1662,18 @@ mod tests {
if first_page(&build(fits, mid)).forks.len() == fits as usize { low = mid } else { high = mid }
}
let (in_page, over) = (build(fits, low), build(fits, low + 1));
let held = page(in_page.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50)));
let held = page(in_page.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50)));
assert_eq!((held.0.forks.len(), held.0.next_cursor), (fits as usize, None));
let size = bare(&DeskResponse::Forks { list: held.0, as_of: held.1 });
// The wire grows in steps of up to three bytes per text byte, so "exact" is within one step.
assert!(size <= DESK_PAGE_BUDGET && DESK_PAGE_BUDGET - size < 4, "{size} against {DESK_PAGE_BUDGET}");
let (moved, _) = page(over.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50)));
let (moved, _) = page(over.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50)));
assert_eq!(moved.forks.len(), fits as usize - 1, "one byte more and the oldest fork goes to the next page");
// The page it would have been, composed from the single reads, is past the budget.
let would_be = ForkList { forks: (1..=fits).rev().map(|n| entry(over.desk_fork(&s(LIST[0]), &list(&LIST), &fork(n))).0).collect(), next_cursor: None, ..moved.clone() };
let would_be = ForkList { forks: (1..=fits).rev().map(|n| entry(over.desk_fork(&account(LIST[0]), &list(&LIST), &fork(n))).0).collect(), next_cursor: None, ..moved.clone() };
assert!(bare(&DeskResponse::Forks { list: would_be, as_of: head(&over) }) > DESK_PAGE_BUDGET);
assert_eq!(moved.next_cursor.map(ForkCursor::order), moved.forks.last().map(|entry| entry.card.order));
let (rest, _) = page(over.desk_forks(&s(LIST[0]), &list(&LIST), None, moved.next_cursor, limit(50)));
let (rest, _) = page(over.desk_forks(&account(LIST[0]), &list(&LIST), None, moved.next_cursor, limit(50)));
assert_eq!(ids(&rest), [fork(1)]);
}
@ -1437,7 +1682,7 @@ mod tests {
let hash = hash_of(mind, &fork(n));
let note = "a long reason, ".repeat(60);
let mut added = 0;
while bare(&mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(n))) <= bytes {
while bare(&mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(n))) <= bytes {
let DeskResponse::Answered { .. } = answer(mind, "x", &LIST, &fork(n), &hash, Some(&note)) else { panic!("the note was not stored") };
added += 1;
assert!(added < 200, "the record does not grow");
@ -1457,18 +1702,18 @@ mod tests {
let mut mind = desk(3);
grown_past(&mut mind, 2, DESK_PAGE_BUDGET);
// The newest fork pages alone: the oversize one would bust the page beside it.
let (first, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50)));
let (first, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50)));
assert_eq!(ids(&first), [fork(3)]);
assert_eq!(first.next_cursor.map(ForkCursor::order), Some(2));
let response = mind.desk_forks(&s(LIST[0]), &list(&LIST), None, first.next_cursor, limit(50));
let response = mind.desk_forks(&account(LIST[0]), &list(&LIST), None, first.next_cursor, limit(50));
let DeskResponse::NotStored(refusal) = response else { panic!("{response:?}") };
assert_eq!(refusal.code, ForkRefusalCode::ForkTooLarge);
let order = 1;
assert_eq!(refusal.error, fork_too_large(&fork(2), order).error);
assert!(refusal.error.contains(&fork(2)) && refusal.error.contains("order 1"), "{refusal:?}");
// The order the refusal names is the order of the fork it names; its cursor goes on past it.
assert_eq!(entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(2))).0.card.order, order);
let (rest, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, ForkCursor::new(order), limit(50)));
assert_eq!(entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(2))).0.card.order, order);
let (rest, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, ForkCursor::new(order), limit(50)));
assert_eq!((ids(&rest), rest.next_cursor), (vec![fork(1)], None));
}
@ -1499,13 +1744,13 @@ mod tests {
fn a_rank_is_stable_when_more_questions_are_admitted() {
let mut mind = seeded();
committed(admit(&mut mind, (1..=5).map(question_n).collect()));
let (first, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(2)));
let (first, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(2)));
assert_eq!((orders(&first), first.next_cursor.map(ForkCursor::order)), (vec![4, 3], Some(3)));
let held = first.next_cursor;
let (before, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, held, limit(50)));
let (before, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, held, limit(50)));
committed(admit(&mut mind, vec![question_n(6)]));
committed(admit(&mut mind, vec![question_n(0)]));
let (after, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, held, limit(50)));
let (after, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, held, limit(50)));
assert_eq!((after.forks, after.next_cursor), (before.forks, before.next_cursor), "the cursor from before resumes the same forks");
let everything = all(&mind, 50);
assert_eq!(ids(&everything), [fork(0), fork(6), fork(5), fork(4), fork(3), fork(2), fork(1)]);
@ -1534,9 +1779,9 @@ mod tests {
admit_unshowable(&mut mind, "Q2");
committed(admit(&mut mind, vec![question_n(3)]));
for who in [&LIST[..], &["x"][..], &["z"][..]] {
let (shown, _) = page(mind.desk_forks(&s(LIST[0]), &list(who), None, None, limit(50)));
let (shown, _) = page(mind.desk_forks(&account(who[0]), &list(who), None, None, limit(50)));
assert_eq!((ids(&shown), orders(&shown)), (vec![fork(3), fork(1)], vec![2, 0]), "{who:?}: rank 1 is the gap");
assert_eq!(entry(mind.desk_fork(&s(LIST[0]), &list(who), &fork(3))).0.card.order, 2, "{who:?}");
assert_eq!(entry(mind.desk_fork(&account(who[0]), &list(who), &fork(3))).0.card.order, 2, "{who:?}");
}
committed(admit(&mut mind, vec![question_n(4)]));
assert_eq!(orders(&all(&mind, 50)), [3, 2, 0]);
@ -1549,11 +1794,11 @@ mod tests {
let mut mind = desk(2);
for n in [1, 2] {
grown_past(&mut mind, n, DESK_PAGE_BUDGET * 55 / 100);
assert!(bare(&mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(n))) <= DESK_PAGE_BUDGET, "each fork alone fits");
assert!(bare(&mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(n))) <= DESK_PAGE_BUDGET, "each fork alone fits");
}
let (one, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, None, limit(50)));
let (one, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, None, limit(50)));
assert_eq!((ids(&one), one.next_cursor.map(ForkCursor::order)), (vec![fork(2)], Some(1)));
let (rest, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, one.next_cursor, limit(50)));
let (rest, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, one.next_cursor, limit(50)));
assert_eq!((ids(&rest), rest.next_cursor), (vec![fork(1)], None));
}
@ -1584,9 +1829,9 @@ mod tests {
assert_eq!(after.len(), 70);
assert_eq!(after.iter().map(|(_, order)| *order).collect::<std::collections::BTreeSet<_>>(), (0..70).collect());
// The page boundary at c1: the page ending at order 1 names c1, and the next page is order 0 alone.
let (upper, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, ForkCursor::new(3), limit(2)));
let (upper, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, ForkCursor::new(3), limit(2)));
assert_eq!((orders(&upper), upper.next_cursor.map(ForkCursor::order)), (vec![2, 1], Some(1)));
let (last, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), None, upper.next_cursor, limit(50)));
let (last, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), None, upper.next_cursor, limit(50)));
assert_eq!((orders(&last), last.next_cursor), (vec![0], None));
}
@ -1670,7 +1915,7 @@ mod tests {
let mut mind = desk(3);
close(&mut mind, &fork(1), "R9");
close(&mut mind, &fork(2), "R2");
let named = |n: u32| entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(n))).0.card.ruling;
let named = |n: u32| entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(n))).0.card.ruling;
assert_eq!(named(1).as_deref(), Some("eureka-state:ruling:R9"));
assert_eq!(named(2).as_deref(), Some("eureka-state:ruling:R2"));
assert_eq!(named(3), None);
@ -1721,6 +1966,9 @@ mod tests {
}
for (order, entry) in &held {
let key = fork(*order as u32);
if scenario.get("twin").is_some() {
list_it(&mut mind, *order as u32);
}
let answers = entry["record"]["answers"].as_array().unwrap();
let hash = hash_of(&mind, &key);
if !answers.is_empty() {
@ -1741,7 +1989,7 @@ mod tests {
answered_at: "2026-09-16T11:00:00Z".into(),
request_id: request_id(),
};
mind.desk_answer(&list(&LIST), s("x"), DeskAuthority::Member, retraction, now());
mind.desk_answer(&list(&LIST), account("x"), DeskAuthority::Member, retraction, now());
} else {
answer(&mut mind, "y", &LIST, &key, &hash, None);
}
@ -1753,7 +2001,7 @@ mod tests {
let (request, want) = (&step["request"], &step["response"]);
let status: Option<ListStatus> = serde_json::from_value(request["status"].clone()).unwrap();
let cursor: Option<ForkCursor> = serde_json::from_value(request["cursor"].clone()).unwrap();
let (got, _) = page(mind.desk_forks(&s(subject), &list(&["x"]), status, cursor, limit(request["limit"].as_u64().unwrap() as u8)));
let (got, _) = page(mind.desk_forks(&account(subject), &list(&["x"]), status, cursor, limit(request["limit"].as_u64().unwrap() as u8)));
let name = &scenario["name"];
assert_eq!((got.open_count, got.recorded_count), (want["open_count"].as_u64().unwrap(), want["recorded_count"].as_u64().unwrap()), "{name}");
assert_eq!(serde_json::to_value(got.next_cursor).unwrap(), want["next_cursor"], "{name}");
@ -1802,7 +2050,7 @@ mod tests {
// Every third fork is closed; the batch's ranks follow its keys, so the closed ranks are read, not computed.
let mut closed = std::collections::BTreeSet::new();
for n in (1..=24).step_by(3) {
closed.insert(entry(mind.desk_fork(&s(LIST[0]), &list(&LIST), &fork(n))).0.card.order);
closed.insert(entry(mind.desk_fork(&account(LIST[0]), &list(&LIST), &fork(n))).0.card.order);
close(&mut mind, &fork(n), &format!("R{n}"));
}
for status in [None, Some(ListStatus::Open), Some(ListStatus::Recorded)] {
@ -1815,7 +2063,7 @@ mod tests {
for size in 1..=24_u8 {
let (mut cursor, mut walked, mut pages) = (None, Vec::new(), 0);
loop {
let (page, _) = page(mind.desk_forks(&s(LIST[0]), &list(&LIST), status, cursor, limit(size)));
let (page, _) = page(mind.desk_forks(&account(LIST[0]), &list(&LIST), status, cursor, limit(size)));
pages += 1;
assert!(page.forks.len() <= size as usize && pages < 100);
walked.extend(orders(&page));

View file

@ -401,7 +401,8 @@ pub(crate) fn line_of(document: &PipelineDocument) -> Option<Lineage<'_>> {
| D::Pulse(_)
| D::Reply(_)
| D::Thought(_)
| D::DeskAnswer(_) => None,
| D::DeskAnswer(_)
| D::DeskListing(_) => None,
}
}
@ -455,10 +456,11 @@ pub enum CitationRole {
DayThrough,
Fork,
Answer,
Listed,
}
impl CitationRole {
pub const ALL: [CitationRole; 26] = [
pub const ALL: [CitationRole; 27] = [
Self::RaisedIn,
Self::Answers,
Self::Rulings,
@ -485,6 +487,7 @@ impl CitationRole {
Self::DayThrough,
Self::Fork,
Self::Answer,
Self::Listed,
];
/// The wire spelling, equal to the referring field's name.
@ -516,6 +519,7 @@ impl CitationRole {
Self::DayThrough => "day_through",
Self::Fork => "fork",
Self::Answer => "answer",
Self::Listed => "listed",
}
}
@ -543,6 +547,7 @@ impl CitationRole {
Self::Run => K::Reply,
Self::Reads => K::Thought,
Self::Fork | Self::Answer => K::DeskAnswer,
Self::Listed => K::DeskListing,
}
}
@ -552,7 +557,7 @@ impl CitationRole {
pub fn target_kinds(self) -> Vec<PipelineKind> {
use PipelineKind as K;
match self {
Self::Answers | Self::Questions | Self::Forks | Self::Fork => vec![K::Question],
Self::Answers | Self::Questions | Self::Forks | Self::Fork | Self::Listed => vec![K::Question],
Self::Rulings => vec![K::Ruling],
Self::CutSpec => vec![K::CutSpec],
Self::CutReport => vec![K::CutReport],
@ -670,6 +675,7 @@ pub(crate) fn citations(document: &PipelineDocument) -> Vec<(CitationRole, Pipel
DeskOutcome::Counted | DeskOutcome::Refused { .. } => {}
}
}
D::DeskListing(listing) => edges.push((R::Listed, cited(K::Question, &listing.listed))),
D::Campaign(_) | D::Target(_) | D::Instance(_) | D::Stewardship(_) | D::Pulse(_) => {}
}
edges

View file

@ -408,9 +408,9 @@ mod tests {
fn desk_answer() -> DeskRequest {
DeskRequest::Answer {
mind: slug(INSTANCE),
subject: crate::fixtures::s("user-7"),
subject: crate::fixtures::acct("user-7"),
authority: DeskAuthority::Member,
deciders: vec![crate::fixtures::s("user-7")],
deciders: vec![crate::fixtures::acct("user-7")],
request: ForkAnswerRequest {
schema_version: crate::desk_wire::RequestSchema::V0,
fork_id: "eureka-state:question:Q1".into(),
@ -425,7 +425,7 @@ mod tests {
}
fn desk_requests() -> [(DeskRequest, &'static str); 3] {
let (mind, subject, deciders) = (slug(INSTANCE), crate::fixtures::s("user-7"), vec![crate::fixtures::s("user-7")]);
let (mind, subject, deciders) = (slug(INSTANCE), crate::fixtures::acct("user-7"), vec![crate::fixtures::acct("user-7")]);
[
(desk_answer(), "Answer"),
(DeskRequest::Forks { mind: mind.clone(), subject: subject.clone(), deciders: deciders.clone(), status: Some(crate::desk_wire::ListStatus::Open), cursor: crate::desk_wire::ForkCursor::new(7), limit: crate::desk_wire::ForkLimit::new(50).unwrap() }, "Forks"),
@ -579,13 +579,13 @@ mod tests {
fn a_maximal_desk_message_fits_the_frame() {
let long = |n: usize| "x".repeat(n);
let wire = |message: &CultNetMessage| encode_cultnet_message_to_vec(message, CultNetWireContract::CultNetSchemaV0).unwrap().len();
let deciders: Vec<_> = (0..64).map(|at| crate::fixtures::s(&format!("{at:0>200}"))).collect();
let deciders: Vec<_> = (0..64).map(|at| crate::fixtures::acct(&format!("decider {at}"))).collect();
let mut sized = vec![];
for (request, _) in desk_requests() {
let widest = match request {
DeskRequest::Answer { mind, request, authority, .. } => DeskRequest::Answer {
mind,
subject: crate::fixtures::s(&long(200)),
subject: crate::fixtures::acct("the widest subject"),
authority,
deciders: deciders.clone(),
request: ForkAnswerRequest {
@ -596,10 +596,10 @@ mod tests {
},
},
DeskRequest::Forks { mind, status, cursor, limit, .. } => {
DeskRequest::Forks { mind, subject: crate::fixtures::s(&long(200)), deciders: deciders.clone(), status, cursor, limit }
DeskRequest::Forks { mind, subject: crate::fixtures::acct("the widest subject"), deciders: deciders.clone(), status, cursor, limit }
}
DeskRequest::Fork { mind, .. } => {
DeskRequest::Fork { mind, subject: crate::fixtures::s(&long(200)), deciders: deciders.clone(), fork_id: long(200) }
DeskRequest::Fork { mind, subject: crate::fixtures::acct("the widest subject"), deciders: deciders.clone(), fork_id: long(200) }
}
};
let message = encode_desk_request("m-10", &widest, Some(long(200))).unwrap();
@ -607,7 +607,8 @@ mod tests {
sized.push(wire(&message));
}
assert!(sized.iter().all(|size| *size <= DESK_MAX_REQUEST_BYTES), "{sized:?} against {DESK_MAX_REQUEST_BYTES}");
assert!(sized[0] > DESK_MAX_REQUEST_BYTES / 4, "the test builds a request that is actually large: {sized:?}");
// 64 account ids of 36 bytes and a 4,000-byte note: the widest answer a request can carry.
assert!(sized[0] > 4000 + 64 * 36, "the test builds a request that is actually large: {sized:?}");
let answer = ForkDeskAnswer {
schema_version: crate::desk_wire::AnswerSchema::V0,

View file

@ -56,6 +56,21 @@ pub(crate) fn s(value: &str) -> Short {
value.into()
}
/// A deterministic Bifrost account id for a test's name for someone: a lowercase D-format UUID.
pub(crate) fn account(name: &str) -> Short {
// An id already in account form stands for itself.
if name.len() == 36 && name.bytes().all(|byte| matches!(byte, b'0'..=b'9' | b'a'..=b'f' | b'-')) {
return Short(name.into());
}
let hash = name.bytes().fold(0xcbf2_9ce4_8422_2325_u64, |hash, byte| (hash ^ u64::from(byte)).wrapping_mul(0x0100_0000_01b3));
Short(format!("{:08x}-{:04x}-4000-8000-{:012x}", hash >> 32, (hash >> 16) & 0xffff, hash & 0xffff_ffff_ffff))
}
/// The same, as the stored type.
pub(crate) fn acct(name: &str) -> eureka_pipeline::AccountId {
eureka_pipeline::AccountId(account(name).0)
}
pub(crate) fn l(value: &str) -> Label {
value.into()
}

View file

@ -150,7 +150,7 @@ mod tests {
fn view(document: PipelineDocument, ordinal: u64) -> PipelineDocumentView {
let key = pipeline_key(&document).unwrap();
let id = PipelineRef { kind: document.kind(), id: s(&key) };
let faculty = if ordinal % 2 == 0 { Faculty::Hands } else { Faculty::Soul };
let faculty = if ordinal.is_multiple_of(2) { Faculty::Hands } else { Faculty::Soul };
let admission = AdmissionFacts {
receipt_id: format!("receipt-{ordinal}"),
admitted_at: format!("2026-11-08T16:{ordinal:02}:00Z"),
@ -226,7 +226,7 @@ mod tests {
let to_source = [held.clone(), vec![moved(RECEIVER, 4)]].concat();
assert_eq!(plan(&slug(CAMPAIGN), &slug(RECEIVER), &slug(RECEIVER), to_source), Err(ImportFault::SourceIsReceiver));
let other_root = view(
resolution(r(K::Campaign, &format!("elsewhere:campaign:self")), ResolutionOutcome::Moved { to: slug(RECEIVER) }),
resolution(r(K::Campaign, "elsewhere:campaign:self"), ResolutionOutcome::Moved { to: slug(RECEIVER) }),
4,
);
assert_eq!(plan_of([held, vec![other_root]].concat()), Err(ImportFault::NotMoved), "another root's move does not move this one");

View file

@ -105,8 +105,8 @@ pub fn index_text(document: &PipelineDocument) -> Option<String> {
lines.push(&idea.idea.0);
lines.push(&idea.tradeoffs.0);
}
// A desk answer is a person's words, not indexed.
D::Instance(_) | D::Stewardship(_) | D::Run(_) | D::Pulse(_) | D::Reply(_) | D::DeskAnswer(_) => {}
// A desk answer is a person's words and a listing's reason is no retrieval text: neither is indexed.
D::Instance(_) | D::Stewardship(_) | D::Run(_) | D::Pulse(_) | D::Reply(_) | D::DeskAnswer(_) | D::DeskListing(_) => {}
}
lines.retain(|line| !line.is_empty());
if lines.is_empty() {

View file

@ -663,7 +663,6 @@ fn an_import_never_cites_or_resolves_a_follow_up_the_receiver_wrote() {
let native = follow_up("FU-N", r(K::Question, &q1));
let native_ref = r(K::FollowUp, &pipeline_key(&native).unwrap());
committed(admit(&mut recv, vec![native]));
let before = head(&recv);
// An imported record that withdraws the receiver's own follow-up mixes a replay into her work.
assert_eq!(
refusal(import_from_kiko(&mut recv, vec![resolution(native_ref.clone(), ResolutionOutcome::Withdrawn { reason: "x".into() })])),

View file

@ -92,7 +92,7 @@ fn declared_names(node: &Value, names: &mut BTreeSet<String>) {
/// Rule 1: the document is valid against the published schema. The first
/// failure is the refusal; it names where and which keyword, never the value.
pub(crate) fn validate_shape(state: &Value) -> Result<(), MindRefusal> {
fn validate_shape(state: &Value) -> Result<(), MindRefusal> {
let compiled = compiled()?;
let Err(error) = compiled.validator.validate(state) else { return Ok(()) };
let mut path = error
@ -115,6 +115,21 @@ pub(crate) fn validate_shape(state: &Value) -> Result<(), MindRefusal> {
})
}
/// Rules P1 to P3 of a put, the ones that read the document alone: it is valid against the published schema (P1), it is
/// this mind's (`personaId` is `instance`, P2), and its provenance is canonical (P3). A put applies these first and then
/// the rules that read the store; a reader of a state file outside the mind (`mind-persona project --state`) applies
/// these and nothing else, so a file it accepts is one a put would not refuse for its content.
pub fn validate_document(instance: &Slug, state: &Value) -> Result<(), MindRefusal> {
validate_shape(state)?;
if state.get("personaId").and_then(Value::as_str) != Some(instance.0.as_str()) {
return Err(MindRefusal::PersonaForeign { instance: instance.0.clone() });
}
if state.pointer("/provenance/authority").and_then(Value::as_str) != Some("canonical") {
return Err(MindRefusal::PersonaNotCanonical);
}
Ok(())
}
/// A stored persona envelope's document.
pub(crate) fn decode(envelope: &CultCacheEnvelope) -> Result<Value, MindRefusal> {
rmp_serde::from_slice::<HuginnPersonaEntry>(&envelope.payload)

View file

@ -166,6 +166,8 @@ pub enum PipelineFacts {
/// the one that counts for its fork at the read's snapshot, derived by `counted_answer` and never
/// stored: a retracted `Counted` answer reads `counts: false`, the promoted next answer `true`.
DeskAnswer { fork: Short, sequence: u32, outcome: Short, counts: bool, authority: DeskAuthority },
/// The question this listing lets members read.
DeskListing { listed: Short },
}
impl PipelineFacts {
@ -306,6 +308,7 @@ impl PipelineFacts {
counts: counted_of(&value.fork.0).as_deref() == Some(id.id.0.as_str()),
authority: value.authority,
},
D::DeskListing(value) => Self::DeskListing { listed: value.listed.clone() },
}
}
}
@ -1849,7 +1852,7 @@ mod tests {
// Two desk answers on the open fork Q2: the second is a Revision of the first.
let D::Question(shown) = question("Q2", &["A", "B"], "A") else { unreachable!() };
let stamp = crate::desk::DeskStamp { answerer: s("x"), authority: eureka_pipeline::DeskAuthority::Member, deciders: vec![s("x")] };
let stamp = crate::desk::DeskStamp { answerer: crate::fixtures::account("x"), authority: eureka_pipeline::DeskAuthority::Member, deciders: vec![crate::fixtures::account("x")] };
let pick = |n: u32| crate::desk::DeskAnswerRequest {
fork: s(&q2),
options_hash: crate::desk::options_hash(&shown),
@ -1862,6 +1865,10 @@ mod tests {
assert!(matches!(mind.answer_desk(stamp.clone(), pick(n), now()), crate::desk::DeskAnswerOutcome::Stored { .. }));
}
// The Mind lists Q2: one `listed` edge.
committed(admit(&mut mind, vec![D::DeskListing(eureka_pipeline::PipelineDeskListing { campaign: slug(CAMPAIGN), listed: s(&q2), reason: "Members may read why.".into() })]));
let listing = id("desk_listing", "Q2");
let (n1, f1_closed, f2_closed) = (
id("resolution", "question.Q1.n1"),
id("resolution", &format!("finding.{}.n1", f1.rsplit(':').next().unwrap())),
@ -1910,8 +1917,9 @@ mod tests {
(CitationRole::Fork, &answer_1, &q2),
(CitationRole::Fork, &answer_2, &q2),
(CitationRole::Answer, &answer_2, &answer_1),
(CitationRole::Listed, &listing, &q2),
];
assert_eq!(CitationRole::ALL.len(), 26);
assert_eq!(CitationRole::ALL.len(), 27);
for role in CitationRole::ALL {
let selection =
Selection { cited: Some(Incoming { role: role.name().into(), exists: true }), ..Selection::default() };
@ -2784,13 +2792,16 @@ mod tests {
/// The upgrade gate on a real store (ruling readback-gate-and-materialized-image):
/// the one function `Mind::read_back` over a scratch copy of a state root.
/// Opens the store read-write, so run it on a copy, never the live file.
/// Measured about 30 s in debug at 11,328 documents.
/// `HUGINN_MIND_INSTANCE` names the instance whose store it opens, `eureka`
/// when unset, so every Mind's store is gated the same way before a release
/// moves her daemon. Measured about 30 s in debug at 11,328 documents.
#[test]
#[ignore = "needs HUGINN_MIND_SNAPSHOT, a scratch copy of a Huginn state root"]
fn stored_documents_read_back() {
let root = std::env::var("HUGINN_MIND_SNAPSHOT").expect("HUGINN_MIND_SNAPSHOT names a scratch copy of the state root");
let instance = std::env::var("HUGINN_MIND_INSTANCE").unwrap_or_else(|_| "eureka".into());
let started = std::time::Instant::now();
let mind = Mind::open(std::path::Path::new(&root), &Slug("eureka".into())).unwrap();
let mind = Mind::open(std::path::Path::new(&root), &Slug(instance)).unwrap();
let count = mind.read_back().unwrap();
println!("read back {count} stored documents in {:?}", started.elapsed());
}

View file

@ -330,13 +330,13 @@ impl CacheBackingStore for LandsThenFails {
fn pull_all(&self) -> Result<Vec<CultCacheEnvelope>> {
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<String> {
self.inner.push(entry)
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<()> {
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<Vec<String>> {
self.inner.push_all(entries, options)
}
}
@ -547,13 +547,13 @@ impl CacheBackingStore for FailsTheLook {
}
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<String> {
self.inner.push(entry)
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<()> {
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<Vec<String>> {
self.inner.push_all(entries, options)
}
}

View file

@ -374,7 +374,7 @@ mod tests {
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> anyhow::Result<()> {
fn push(&mut self, entry: &CultCacheEnvelope) -> anyhow::Result<String> {
self.inner.push(entry)
}
@ -382,7 +382,7 @@ mod tests {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> anyhow::Result<()> {
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> anyhow::Result<Vec<String>> {
self.inner.push_all(entries, options)
}
}

View file

@ -1,13 +1,11 @@
//! Credential redaction: the only code that knows the shape of a secret in text.
//!
//! `redact_url_userinfo` redacts URL credentials (the Self run's journal and reply,
//! the Persona's structured output). `scrub` is chat ingest's door: every message body
//! `redact_url_userinfo` redacts URL credentials (the Self run's journal and reply). `scrub` is chat ingest's door: every message body
//! passes it before it is chunked, embedded or stored, and it also runs the URL rule.
//! Shapes only; nothing here reads meaning, and a hit is a kind, never a value.
/// The one redaction of URL credentials. Call it at each place a string the CLI produced enters the journal, her mind or chat: today
/// the Self result (`result_text`: journal line and Reply) and the structured output of a Persona or Sleep result
/// (`persona::structured_output`, string by string). It does not cover claude's stderr (inherited) or the CLI's transcript files.
/// the Self result (`result_text`: journal line and Reply). It does not cover claude's stderr (inherited) or the CLI's transcript files.
///
/// A forge token rides in a remote URL, so every `scheme://` and every protocol-relative `//` opens an authority, read as git, curl and
/// RFC 3986 read it: it ends at the first `/`, `?`, `#`, backslash, quote, backtick, parenthesis, bracket, angle bracket or ASCII whitespace
@ -159,7 +157,7 @@ fn zero_width(c: char) -> bool {
pub fn scrub(text: &str) -> Scrubbed {
let mut hits = Vec::new();
let mut stripped: String = text.chars().filter(|c| !zero_width(*c)).collect();
let passes: [(ScrubKind, fn(&str, usize) -> Option<usize>); 6] = [
let passes: [(ScrubKind, Finder); 6] = [
(ScrubKind::PemBlock, pem_block),
(ScrubKind::AuthorizationHeader, authorization_line),
(ScrubKind::AccessToken, access_token),
@ -176,7 +174,10 @@ pub fn scrub(text: &str) -> Scrubbed {
if hits.is_empty() { Scrubbed { text: text.to_owned(), hits } } else { Scrubbed { text: urls, hits } }
}
fn replace_matches(text: &str, kind: ScrubKind, find: fn(&str, usize) -> Option<usize>, hits: &mut Vec<ScrubKind>) -> String {
/// A scrub pass: finds the end of one secret starting at a byte offset, if one starts there.
type Finder = fn(&str, usize) -> Option<usize>;
fn replace_matches(text: &str, kind: ScrubKind, find: Finder, hits: &mut Vec<ScrubKind>) -> String {
let mut out = String::with_capacity(text.len());
let mut at = 0;
while at < text.len() {
@ -463,7 +464,7 @@ mod tests {
"123e4567-e89b-12d3-a456-426614174000", "!abc:s", "$Ab3dEf-9_zQ", "user@example.com", "ssh://git@host/r.git",
"https://github.com/@user", "pkg@1.2.3", "the bearer of bad news", "Authorization is a header",
"padding like dGVzdA== in prose", "```\nlet x = 1;\n```", "the BEGIN PRIVATE KEY header", "a token of thanks",
"AKIAshort", "ghp_short", "eyJ is a prefix", "naïve café — 日本語 with Bearer in it",
"AKIAshort", "ghp_short", "eyJ is a prefix", "naïve café — 日本語 with Bearer in it",
] {
let out = scrub(same);
assert_eq!(out.text, same);
@ -473,9 +474,9 @@ mod tests {
#[test]
fn multibyte_text_around_a_secret_never_panics_the_scrub() {
let body = format!("日本 \u{a0}é Bearer {} → ünï", "A1b2".repeat(6));
assert_eq!(scrub(&body).text, "日本 \u{a0}é <redacted> → ünï");
let seed = "é日 sk-aaaaaaaaaaaaaaaaaaaa -----BEGIN PRIVATE KEY----- authorization: é eyJa.b.c https://u:p@h/x";
let body = format!("日本 \u{a0}é Bearer {} → ünï", "A1b2".repeat(6));
assert_eq!(scrub(&body).text, "日本 \u{a0}é <redacted> → ünï");
let seed = "é日 sk-aaaaaaaaaaaaaaaaaaaa -----BEGIN PRIVATE KEY----- authorization: é eyJa.b.c https://u:p@h/x";
let chars: Vec<char> = seed.chars().collect();
for start in 0..chars.len() {
for end in start..=chars.len() {

View file

@ -117,6 +117,10 @@ pub enum MindRefusal {
/// reached.
CursorInvalid { message: String },
Unavailable { detail: String },
/// The request reached the persona door, which answers reads and the
/// mind's own persona document and nothing that writes the mind
/// (`HuginnMindRequest::reaches_persona_door`). Nothing was written.
NotThisDoor,
}
impl std::fmt::Display for MindRefusal {

View file

@ -248,7 +248,7 @@ impl SelectionRow {
D::Stewardship(stewardship) => vec![stewardship.repo.identity()],
D::HandOff(hand_off) => vec![hand_off.repo.identity()],
D::Ruling(PipelineRuling { authority: RulingAuthority::Steward { repo, .. }, .. }) => vec![repo.identity()],
D::Target(_) | D::Question(_) | D::Ruling(_) | D::Verdict(_) | D::Finding(_) | D::Instance(_) | D::Resolution(_) | D::Run(_) | D::Pulse(_) | D::Reply(_) | D::Thought(_) | D::Memory(_) | D::Research(_) | D::Idea(_) | D::DeskAnswer(_) => {
D::Target(_) | D::Question(_) | D::Ruling(_) | D::Verdict(_) | D::Finding(_) | D::Instance(_) | D::Resolution(_) | D::Run(_) | D::Pulse(_) | D::Reply(_) | D::Thought(_) | D::Memory(_) | D::Research(_) | D::Idea(_) | D::DeskAnswer(_) | D::DeskListing(_) => {
Vec::new()
}
},
@ -466,7 +466,7 @@ mod tests {
/// refused at the substrate's door.
#[test]
fn each_kind_declares_exactly_its_aliases_and_none_is_numeric() {
let table: [(K, &[&str]); 21] = [
let table: [(K, &[&str]); 22] = [
(K::Campaign, &["repo"]),
(K::Target, &[]),
(K::Question, &[]),
@ -488,6 +488,7 @@ mod tests {
(K::Reply, &[]),
(K::Thought, &[]),
(K::DeskAnswer, &[]),
(K::DeskListing, &[]),
];
for (kind, extra) in table {
let want = [&["root", "in_force", "faculty"][..], extra].concat();
@ -503,7 +504,7 @@ mod tests {
/// twenty-one.
#[test]
fn each_kind_declares_the_roles_it_carries_and_their_targets() {
let table: [(K, &[&str]); 21] = [
let table: [(K, &[&str]); 22] = [
(K::Campaign, &[]),
(K::Target, &[]),
(K::Question, &["raised_in"]),
@ -525,13 +526,14 @@ mod tests {
(K::Reply, &["run"]),
(K::Thought, &["reads"]),
(K::DeskAnswer, &["fork", "answer"]),
(K::DeskListing, &["listed"]),
];
for (kind, roles) in table {
assert_eq!(sorted(Vocabulary.declared_roles(kind.type_id())), strings(roles), "{kind:?}");
}
assert_eq!(Vocabulary.target_leaves("answers"), vec![K::Question.type_id().to_string()]);
assert_eq!(Vocabulary.target_leaves("rulings"), vec![K::Ruling.type_id().to_string()]);
assert_eq!(Vocabulary.target_leaves("subject").len(), 21);
assert_eq!(Vocabulary.target_leaves("subject").len(), 22);
assert_eq!(
sorted(Vocabulary.target_leaves("claims")),
sorted([K::CutSpec, K::Finding, K::FollowUp].iter().map(|kind| kind.type_id().to_string()).collect())

View file

@ -116,9 +116,9 @@ pub(crate) mod test_stores {
Ok(self.rows())
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<String> {
self.plant(entry.clone());
Ok(())
Ok(entry.stored_at.clone())
}
fn delete(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
@ -126,9 +126,9 @@ pub(crate) mod test_stores {
Ok(())
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], _options: PushAllOptions) -> Result<()> {
fn push_all(&mut self, entries: &[CultCacheEnvelope], _options: PushAllOptions) -> Result<Vec<String>> {
*self.rows.lock().unwrap() = entries.iter().map(|entry| (identity(entry), entry.clone())).collect();
Ok(())
Ok(entries.iter().map(|entry| entry.stored_at.clone()).collect())
}
}
@ -199,7 +199,7 @@ pub(crate) mod test_stores {
self.inner.pull_all()
}
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<String> {
self.inner.push(entry)
}
@ -207,7 +207,7 @@ pub(crate) mod test_stores {
self.inner.delete(entry)
}
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<()> {
fn push_all(&mut self, entries: &[CultCacheEnvelope], options: PushAllOptions) -> Result<Vec<String>> {
self.inner.push_all(entries, options)
}
}

View file

@ -286,8 +286,9 @@ pub const CHAT_ROOMS_MAX: usize = 16;
/// [after, before]. `texts` (none to `SOURCE_TEXTS_MAX`) is the meaning match,
/// several fused as in a source search; with none the search is a listing:
/// chunks oldest first, at most `top_k`, `ChatHits.truncated` saying more
/// exist, and the caller narrows with `after_ms` = the last hit's
/// `ts_end_ms + 1`.
/// exist. Within one room the caller narrows with `after_ms` = the last hit's
/// `ts_end_ms + 1`; across rooms spans overlap and that loses chunks, so list
/// one room at a time.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct ChatQuery {
pub texts: Vec<Line>,
@ -429,6 +430,28 @@ impl HuginnMindRequest {
}
}
/// Whether the persona door answers this request. The persona door is the
/// daemon's second listener, the only one a Persona's uid can reach: it
/// serves the mind's reads and her own persona document, and refuses
/// every write to the mind (`MindRefusal::NotThisDoor`). The match has no
/// wildcard, so a new request does not compile until it is classified
/// here.
pub fn reaches_persona_door(&self) -> bool {
match self {
Self::Whoami
| Self::View { .. }
| Self::Query { .. }
| Self::PersonaGet { .. }
| Self::PersonaPut { .. }
| Self::SearchSources { .. }
| Self::SearchChat { .. }
| Self::ChatWindow { .. }
| Self::StewardedActivity { .. }
| Self::RootsTouched { .. } => true,
Self::Admit(_) => false,
}
}
/// The instance the request declares; `Whoami` declares none.
pub fn instance(&self) -> Option<&Slug> {
match self {
@ -632,6 +655,7 @@ mod tests {
scores: Some(vec![0.5]),
}),
HuginnMindResponse::Refused(refusal),
HuginnMindResponse::Refused(MindRefusal::NotThisDoor),
HuginnMindResponse::Persona(None),
HuginnMindResponse::Persona(Some(PersonaStateView {
value: serde_json::json!({ "personaId": INSTANCE }),

View file

@ -21,8 +21,8 @@ use huginn_mind::desk_wire::{
fork_too_large_read, AnswerSchema, CardSchema, CardStatus, DeskRequest, DeskResponse, EntrySchema, ForkCard, ForkCursor, ForkDeskAnswer, ForkEntry, ForkLimit,
ForkList, ForkOption, ForkRecord, ForkRefusal, ForkRefusalCode, ForkRef, ListSchema, ListStatus, RecordSchema,
};
use huginn_mind::envelope::{OperationFailure, encode_desk_failure, encode_desk_request, encode_desk_response};
use huginn_mind::eureka_pipeline::{DeskAuthority, Short, Slug};
use huginn_mind::envelope::{OperationFailure, decode_desk_request, encode_desk_failure, encode_desk_request, encode_desk_response};
use huginn_mind::eureka_pipeline::{AccountId, DeskAuthority, Slug};
const RUNTIME: &str = "huginn-eureka";
/// Account ids as Bifrost sends them: `UserAccount.Id` rendered lowercase D-format
@ -38,15 +38,15 @@ const NON_DECIDER: &str = "9e8d7c6b-5a49-4382-b1a0-f9e8d7c6b5a4";
/// synthetic: the hash is CultLib's sample digest, not this question's.
const FORK: &str = "eureka-substrate:question:desk-listing-authority";
fn s(text: &str) -> Short {
Short(text.into())
fn a(text: &str) -> AccountId {
AccountId(text.into())
}
fn forks(status: Option<ListStatus>, cursor: Option<u64>, limit: u8) -> DeskRequest {
DeskRequest::Forks {
mind: Slug("eureka".into()),
subject: s(READER),
deciders: vec![s(READER), s(OTHER_DECIDER)],
subject: a(READER),
deciders: vec![a(READER), a(OTHER_DECIDER)],
status,
cursor: cursor.and_then(ForkCursor::new),
limit: ForkLimit::new(limit).unwrap(),
@ -54,15 +54,15 @@ fn forks(status: Option<ListStatus>, cursor: Option<u64>, limit: u8) -> DeskRequ
}
fn read() -> DeskRequest {
DeskRequest::Fork { mind: Slug("eureka".into()), subject: s(READER), deciders: vec![s(READER), s(OTHER_DECIDER)], fork_id: FORK.into() }
DeskRequest::Fork { mind: Slug("eureka".into()), subject: a(READER), deciders: vec![a(READER), a(OTHER_DECIDER)], fork_id: FORK.into() }
}
/// The same reads from a subject the call's deciders do not name: a non-decider.
fn forks_as_non_decider() -> DeskRequest {
DeskRequest::Forks {
mind: Slug("eureka".into()),
subject: s(NON_DECIDER),
deciders: vec![s(READER), s(OTHER_DECIDER)],
subject: a(NON_DECIDER),
deciders: vec![a(READER), a(OTHER_DECIDER)],
status: Some(ListStatus::Open),
cursor: None,
limit: ForkLimit::new(50).unwrap(),
@ -70,7 +70,7 @@ fn forks_as_non_decider() -> DeskRequest {
}
fn read_as_non_decider() -> DeskRequest {
DeskRequest::Fork { mind: Slug("eureka".into()), subject: s(NON_DECIDER), deciders: vec![s(READER), s(OTHER_DECIDER)], fork_id: FORK.into() }
DeskRequest::Fork { mind: Slug("eureka".into()), subject: a(NON_DECIDER), deciders: vec![a(READER), a(OTHER_DECIDER)], fork_id: FORK.into() }
}
fn list(open_count: u64, recorded_count: u64) -> DeskResponse {
@ -196,6 +196,24 @@ fn hex(bytes: &[u8]) -> String {
text
}
/// A request message whose payload is `request` with one field changed to something the typed request
/// cannot hold: what a client that skips the account grammar would send.
fn malformed(id: &str, request: &DeskRequest, edit: impl Fn(&mut serde_json::Value)) -> CultNetMessage {
use base64::Engine;
let mut value = serde_json::to_value(request).unwrap();
edit(&mut value);
CultNetMessage::OperationRequest {
message_id: id.into(),
service_id: huginn_mind::desk_wire::DESK_SERVICE_ID.into(),
operation: "fork".into(),
payload_schema: huginn_mind::desk_wire::DESK_REQUEST_SCHEMA.into(),
payload_encoding: "messagepack-base64".into(),
payload: base64::engine::general_purpose::STANDARD.encode(rmp_serde::to_vec_named(&value).unwrap()),
source_runtime_id: None,
target_runtime_id: None,
}
}
/// Every fixture: its file stem and its bytes. A message gives `.frame.bin`, `.frame.hex` and,
/// for an operation message, `.payload.msgpack` and `.payload.hex`; a bare payload gives the last two.
fn fixtures() -> Vec<(String, Vec<u8>)> {
@ -221,6 +239,15 @@ fn fixtures() -> Vec<(String, Vec<u8>)> {
// raised_at nil: the question's first admission is unknown.
message("read.response", encode_desk_response("m-4", "fork", &DeskResponse::Fork { entry: entry(true, None), as_of: 10190 }, RUNTIME).unwrap());
message("read-non-decider.response", encode_desk_response("m-6", "fork", &DeskResponse::Fork { entry: entry(false, None), as_of: 10190 }, RUNTIME).unwrap());
// A decider or subject that is no account id is refused at decode, naming the field.
for (name, id, request) in [
("read-mxid-decider", "m-7", malformed("m-7", &read(), |value| value["Fork"]["deciders"][0] = "@heimdall:gamecult.org".into())),
("read-email-subject", "m-8", malformed("m-8", &read(), |value| value["Fork"]["subject"] = "meta@gamecult.org".into())),
] {
message(&format!("{name}.request"), request.clone());
let failure = decode_desk_request(&request).expect_err("the door refuses it");
message(&format!("{name}.failure.response"), encode_desk_failure(id, "fork", &failure, RUNTIME));
}
message("list-empty.response", encode_desk_response("m-1", "forks", &list(0, 0), RUNTIME).unwrap());
message("list-counts.response", encode_desk_response("m-3", "forks", &list(3, 5), RUNTIME).unwrap());
message(
@ -290,8 +317,7 @@ fn account_id(text: &str) -> bool {
}
/// Every account id the fixtures carry is a Bifrost account id, never a Matrix id, email or
/// handle. The door does not refuse other shapes yet (`Short`, any 200 bytes); that refusal is owed
/// by `cut-desk-feed-listing.r6`, and this pins the grammar it must enforce.
/// handle, and the door refuses any other shape at decode, naming the field and never the value.
#[test]
fn every_account_id_in_the_fixtures_is_a_lowercase_d_format_uuid() {
for request in [forks(Some(ListStatus::Open), None, 50), read()] {
@ -302,6 +328,14 @@ fn every_account_id_in_the_fixtures_is_a_lowercase_d_format_uuid() {
assert!(request.deciders().iter().all(|decider| account_id(&decider.0)), "deciders");
}
assert!(record(true).answers.iter().all(|answer| account_id(answer.answerer.as_deref().unwrap())), "answerer");
for (request, field, value) in [
(malformed("m-7", &read(), |value| value["Fork"]["deciders"][0] = "@heimdall:gamecult.org".into()), "deciders", "@heimdall:gamecult.org"),
(malformed("m-8", &read(), |value| value["Fork"]["subject"] = "meta@gamecult.org".into()), "subject", "meta@gamecult.org"),
] {
let failure = decode_desk_request(&request).expect_err("a request outside the grammar is refused");
assert_eq!(failure.code, "payload-not-a-request");
assert!(failure.message.contains(field) && !failure.message.contains(value), "{}", failure.message);
}
for shape in [
"@heimdall:gamecult.org",
"meta@gamecult.org",

View file

@ -15,9 +15,9 @@ Each case has up to four files:
Hex is lowercase with one trailing newline. The fork id is a question Eureka's mind holds. The
record's answers, notes and hash are synthetic.
**The listing gate is not live.** On `main` every showable question goes to every reader. Do not
put a Mind that holds private questions behind a live receiver before `cut-desk-feed-listing`
lands (see `docs/desk-feed-door.md`).
**The listing gate is live in Huginn.** A reader who is not on the call's deciders sees only forks the
Mind lists (`docs/desk-feed-door.md`); `list-empty.response` is what such a reader gets from a Mind with
nothing listed.
| case | direction | content |
|---|---|---|
@ -31,6 +31,10 @@ lands (see `docs/desk-feed-door.md`).
| `list-open-non-decider.response` | Huginn to Bifrost | the same page for a non-decider: `deciders` empty, `decider_count` 2, every `answerer` nil, `answerer_count` 3; no account id anywhere in its bytes |
| `read.response` | Huginn to Bifrost | a decider's read of one fork, `raised_at` nil |
| `read-non-decider.response` | Huginn to Bifrost | the same read for a non-decider, `raised_at` nil, no account id |
| `read-mxid-decider.request` | client to Huginn | a `Fork` whose first decider is a Matrix id, so outside the account grammar; refused at decode |
| `read-mxid-decider.failure.response` | Huginn to Bifrost | the operation failure for it: `payload-not-a-request`, naming `deciders`, never quoting the id |
| `read-email-subject.request` | client to Huginn | a `Fork` whose subject is an email address; refused at decode |
| `read-email-subject.failure.response` | Huginn to Bifrost | the operation failure for it, naming `subject` |
| `list-empty.response` | Huginn to Bifrost | an empty Mind, or nothing listed to this reader: counts 0, `forks: []` |
| `list-counts.response` | Huginn to Bifrost | the limit-0 answer: open 3, recorded 5, `forks: []` |
| `read-unknown.response` | Huginn to Bifrost | `NotStored` `fork_unknown` (Bifrost: 404) |

View file

@ -0,0 +1 @@
000001738aab646961676e6f737469637391b57061796c6f61642d6e6f742d612d72657175657374a96d6573736167654964a36d2d38a96f7065726174696f6ea4666f726ba77061796c6f6164d9606771526a6232526c745842686557787659575174626d39304c574574636d56786457567a644b64745a584e7a5957646c3253466d615756735a43424762334a724c6e4e31596d706c5933513649476c75646d467361575174646d46736457553daf7061796c6f6164456e636f64696e67b26d6573736167657061636b2d626173653634ad7061796c6f6164536368656d61d92567616d6563756c742e63756c746e65742e6f7065726174696f6e5f6661696c7572652e7631ad736368656d6156657273696f6ebd63756c746e65742e6f7065726174696f6e5f726573706f6e73652e7630a9736572766963654964ab687567696e6e2e6465736baf736f7572636552756e74696d654964ad687567696e6e2d657572656b61a6737461747573a872656a6563746564

View file

@ -0,0 +1 @@
82a4636f6465b57061796c6f61642d6e6f742d612d72657175657374a76d657373616765d9216669656c6420466f726b2e7375626a6563743a20696e76616c69642d76616c7565

View file

@ -0,0 +1 @@
うcodeオpayload-not-a-requestァmessageル!field Fork.subject: invalid-value

View file

@ -0,0 +1 @@
000001ad87a96d6573736167654964a36d2d38a96f7065726174696f6ea4666f726ba77061796c6f6164d9fc6761524762334a72684b686b5a574e705a47567963354c5a4a44566d4d474d79596a466c4c5468684e4751744e474d7a596930355a54646d4c5446684d6d497a597a526b4e5755325a746b6b4d4749355a44646c4d3245744e6d4d794d5330305a6a55344c5745775a4451744f5755345a6a64684e6d4931597a526b70325a76636d74666157545a4d475631636d56725953317a64574a7a64484a68644755366358566c63335270623234365a47567a6179317361584e306157356e4c57463164476876636d6c30656152746157356b706d5631636d56725961647a64574a715a574e307357316c644746415a3246745a574e316248517562334a6eaf7061796c6f6164456e636f64696e67b26d6573736167657061636b2d626173653634ad7061796c6f6164536368656d61b6687567696e6e2e6465736b5f726571756573742e7631ad736368656d6156657273696f6ebc63756c746e65742e6f7065726174696f6e5f726571756573742e7630a9736572766963654964ab687567696e6e2e6465736b

View file

@ -0,0 +1 @@
81a4466f726b84a8646563696465727392d92435663063326231652d386134642d346333622d396537662d316132623363346435653666d92430623964376533612d366332312d346635382d613064342d396538663761366235633464a7666f726b5f6964d930657572656b612d7375627374726174653a7175657374696f6e3a6465736b2d6c697374696e672d617574686f72697479a46d696e64a6657572656b61a77375626a656374b16d6574614067616d6563756c742e6f7267

View file

@ -0,0 +1 @@
<EFBFBD>¤Fork„¨deciders’Ù$5f0c2b1e-8a4d-4c3b-9e7f-1a2b3c4d5e6fÙ$0b9d7e3a-6c21-4f58-a0d4-9e8f7a6b5c4d§fork_idÙ0eureka-substrate:question:desk-listing-authority¤mind¦eureka§subject±meta@gamecult.org

View file

@ -0,0 +1 @@
000001778aab646961676e6f737469637391b57061796c6f61642d6e6f742d612d72657175657374a96d6573736167654964a36d2d37a96f7065726174696f6ea4666f726ba77061796c6f6164d9646771526a6232526c745842686557787659575174626d39304c574574636d56786457567a644b64745a584e7a5957646c3253566d615756735a43424762334a724c6d526c59326c6b5a584a7a577a42644f694270626e5a6862476c6b4c585a686248566caf7061796c6f6164456e636f64696e67b26d6573736167657061636b2d626173653634ad7061796c6f6164536368656d61d92567616d6563756c742e63756c746e65742e6f7065726174696f6e5f6661696c7572652e7631ad736368656d6156657273696f6ebd63756c746e65742e6f7065726174696f6e5f726573706f6e73652e7630a9736572766963654964ab687567696e6e2e6465736baf736f7572636552756e74696d654964ad687567696e6e2d657572656b61a6737461747573a872656a6563746564

View file

@ -0,0 +1 @@
82a4636f6465b57061796c6f61642d6e6f742d612d72657175657374a76d657373616765d9256669656c6420466f726b2e64656369646572735b305d3a20696e76616c69642d76616c7565

View file

@ -0,0 +1 @@
うcodeオpayload-not-a-requestァmessageル%field Fork.deciders[0]: invalid-value

View file

@ -0,0 +1 @@
000001b687a96d6573736167654964a36d2d37a96f7065726174696f6ea4666f726ba77061796c6f6164da01046761524762334a72684b686b5a574e705a47567963354b325147686c6157316b595778734f6d64686257566a645778304c6d39795a396b6b4d4749355a44646c4d3245744e6d4d794d5330305a6a55344c5745775a4451744f5755345a6a64684e6d4931597a526b70325a76636d74666157545a4d475631636d56725953317a64574a7a64484a68644755366358566c63335270623234365a47567a6179317361584e306157356e4c57463164476876636d6c30656152746157356b706d5631636d56725961647a64574a715a574e30325351315a6a426a4d6d49785a5330345954526b4c54526a4d3249744f5755335a69307859544a694d324d305a44566c4e6d593daf7061796c6f6164456e636f64696e67b26d6573736167657061636b2d626173653634ad7061796c6f6164536368656d61b6687567696e6e2e6465736b5f726571756573742e7631ad736368656d6156657273696f6ebc63756c746e65742e6f7065726174696f6e5f726571756573742e7630a9736572766963654964ab687567696e6e2e6465736b

View file

@ -0,0 +1 @@
81a4466f726b84a8646563696465727392b6406865696d64616c6c3a67616d6563756c742e6f7267d92430623964376533612d366332312d346635382d613064342d396538663761366235633464a7666f726b5f6964d930657572656b612d7375627374726174653a7175657374696f6e3a6465736b2d6c697374696e672d617574686f72697479a46d696e64a6657572656b61a77375626a656374d92435663063326231652d386134642d346333622d396537662d316132623363346435653666

View file

@ -0,0 +1 @@
<EFBFBD>¤Fork„¨deciders’¶@heimdall:gamecult.orgÙ$0b9d7e3a-6c21-4f58-a0d4-9e8f7a6b5c4d§fork_idÙ0eureka-substrate:question:desk-listing-authority¤mind¦eureka§subjectÙ$5f0c2b1e-8a4d-4c3b-9e7f-1a2b3c4d5e6f

View file

@ -9,13 +9,13 @@ publish.workspace = true
# operator's brakes: a root-owned per-instance CultCache
# store (`control`), the operator's CLI over it (`cli`, the `mind-control`
# binary) and the one fail-closed reader every acting organ calls
# (`control::read_effective`). It also holds the launch primitive (`launch`,
# `queue`): the one way her organs open a run and start its unit. The heartbeat
# (`heartbeat`, the `mind-heartbeat` binary) runs the shared heartbeat over them. The actuator
# (`control::read_effective`). It also holds her units under systemd (`launch`)
# and her port to her mind (`queue`). The heartbeat (`heartbeat`, the
# `mind-heartbeat` binary) wakes her by starting her unit. The actuator
# (`self_run`, the `mind-self-run` binary) carries her one open Self run.
#
# The Persona organ (`persona`, the `mind-persona` binary) drains her pending chat
# through ports. The connector transport is ModelConnector's kit client, pinned to
# The Persona organ (`persona`, the `mind-persona` binary) reads the brakes and
# carries no turn until the three-stage turn lands (map C3). The connector transport is ModelConnector's kit client, pinned to
# the kit's revision (`connector`); the chat transport is cultnet-rs's RUDP client
# to Bifrost's mouth (`chat_rudp`). `rmp-serde` is the chat documents' positional
# MessagePack codec (`chat_client`); `sha2` derives the mouth's connection id from
@ -23,11 +23,9 @@ publish.workspace = true
# two until the workspace pin and the kit's meet.
# `rmp-serde` is also the lease wire (`lease_client`): the connector's MessagePack frames.
#
# The stream organ (`stream`, the `mind-stream` binary) builds its Projector prompt with Ghostlight's
# `ghostlight-persona-projection`, pinned to a revision, so the membrane's text stays Ghostlight's; it
# asks its Projector through the connector kit's client (`model-connector`, one turn per tick) and speaks to the base
# model's host on loopback over `ureq` 3 with default features off (plain HTTP, no TLS), as the
# daemon's embedder does, and stays below 3.4 for the same reason.
# Ghostlight's `ghostlight-persona-projection`, pinned to a revision, builds the Projector's prompt (`projection`), so
# the membrane's text stays Ghostlight's; the Projector asks through the connector kit's client (`model-connector`).
# `toml` reads `persona-tuning.toml`, the Projector's knobs (`projection::tuning`); parse and serde only.
#
# `serde_json` reads Claude Code's stream-json, a foreign owner's wire (`self_run`);
# `uuid`'s v5 makes a Self run's session id derivable from the run's id.
@ -46,16 +44,19 @@ rmp-serde = "1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha2 = "0.10"
toml = { version = "0.9", default-features = false, features = ["std", "serde", "parse"] }
unicode-general-category = "1"
ureq = { version = "~3.3", default-features = false }
# Clips on grapheme boundaries (`projection::select::clip`), so no ZWJ sequence is split; already in the graph through Ghostlight.
unicode-segmentation = "1"
uuid = { version = "1", default-features = false, features = ["v5"] }
[target.'cfg(unix)'.dependencies]
libc = "0.2"
# The lease socket: SCM_RIGHTS receive with MSG_CMSG_CLOEXEC and the descriptor
# duplicates, as safe calls (the library forbids unsafe code). Already in the
# graph through tempfile; this adds its `net` feature only.
rustix = { version = "1", default-features = false, features = ["std", "net"] }
# graph through tempfile; this adds its `net` feature, and `fs` for the attachment
# saver's descriptor-relative writes (openat, linkat, unlinkat).
rustix = { version = "1", default-features = false, features = ["std", "net", "fs"] }
# `syn`, `quote` and `proc-macro2` read the library's own source for
# tests/public_surface.rs, which pins its public surface. All three are already
@ -65,7 +66,7 @@ proc-macro2 = "1"
quote = "1"
syn = { version = "2", features = ["full"] }
tempfile = "3"
# A real kit server (`serve` over a scripted `ProviderBackend`) proves the stream's Projector wire, not a fake of it.
# A real kit server (`serve` over a scripted `ProviderBackend`) proves the Projector's connector wire, not a fake of it.
# Same revision as the dependency; the `daemon` feature is test-only.
model-connector = { git = "https://eureka.gamecult.org/forge/GameCult/ModelConnector.git", rev = "c1802d85bee3ae04b0c5737563c8343c1fef515d", package = "model-connector", features = ["daemon"] }
@ -85,10 +86,6 @@ path = "src/bin/mind-self-run.rs"
name = "mind-persona"
path = "src/bin/mind-persona.rs"
[[bin]]
name = "mind-stream"
path = "src/bin/mind-stream.rs"
[[bin]]
name = "mind-heartbeat"
path = "src/bin/mind-heartbeat.rs"

View file

@ -1,7 +1,7 @@
# mind-body
The Mind's body: the control store, the launch primitive, the `mind-heartbeat`
timeline and the `mind-self-run` actuator.
The Mind's body: the control store, the unit launcher, the `mind-heartbeat`, the
inert `mind-persona` organ and the `mind-self-run` actuator.
`mind-self-run --instance <slug> [--endpoint rudp://<ip>:<port>]` runs inside
`mind-self@<instance>`. It finds her one open Self run, starts `claude` over
@ -12,7 +12,9 @@ wrong or the mind cannot be reached.
## The heartbeat
`mind-heartbeat` is the one process that decides who wakes and when: each beat falls when the Minds' voted tempo has passed and goes to the Mind that has waited longest, bent by the urgency she declares and a member addressing her; the operator tagging or naming a Persona launches her at once. `mind-heartbeat timeline` prints its latest record.
`mind-heartbeat` is the one process that decides who wakes and when. It starts a Persona's unit and writes no Mind: the beat (`HEARTBEAT_BEAT_S`, default 1800 s) goes to the idle, released Mind that has waited longest, bent by a member addressing her, and the operator or another Mind naming a Persona wakes her at once, once per message. `mind-heartbeat timeline` prints its latest record.
`mind-persona` is inert: it reads the brakes and exits `dead` or `no organ`. It writes nothing. Nothing launches `mind-self-run`.
The actuator has its own ignored test, `the_real_bin_finds_her_run_...` in
`tests/self_run.rs`: the real `mind-self-run` against the real daemon with a

View file

@ -0,0 +1,6 @@
responder 3: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org) answered it. Meta, who runs the place last spoke 5 min ago. The last 2 lines are the Minds' alone.
responder 4: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org); Kiko answered it. Meta, who runs the place last spoke 5 min ago. The last 3 lines are the Minds' alone.
responder 5: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org); Kiko; "libby" (@libby:gamecult.org) answered it. Meta, who runs the place last spoke 5 min ago. The last 4 lines are the Minds' alone.
responder 6: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org); Kiko; "libby" (@libby:gamecult.org); "void" (@void:gamecult.org) answered it. Meta, who runs the place last spoke 5 min ago. The last 5 lines are the Minds' alone.
responder 7: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org); Kiko; "libby" (@libby:gamecult.org); "void" (@void:gamecult.org); "nibu" (@nibu:gamecult.org) answered it. Meta, who runs the place last spoke 5 min ago. The last 6 lines are the Minds' alone.
responder 8: - the conversation: Meta, who runs the place asked the room a question, and "weksa" (@weksa:gamecult.org); "heimdall" (@heimdall:gamecult.org); Kiko; "libby" (@libby:gamecult.org); "void" (@void:gamecult.org); "nibu" (@nibu:gamecult.org); "aqua" (@aqua:gamecult.org) answered it. Meta, who runs the place last spoke 5 min ago. The last 7 lines are the Minds…

View file

@ -0,0 +1,4 @@
quiet-beat.toml: - the conversation: Lately @libby:gamecult.org to the room; Kiko to the room. The last 3 lines are the Minds' alone.
operator-warm.toml: - the conversation: Meta, who runs the place is waiting on him. Lately Meta, who runs the place with him. Meta, who runs the place last spoke 1 min ago.
mind-teases.toml: - the conversation: Lately he to the room; @aqua:gamecult.org to the room; Kiko with him. The last 3 lines are the Minds' alone.
deru-passing.toml: - the conversation: Lately @weksa:gamecult.org to the room; Deru to the room. Deru last spoke 1 min ago.

View file

@ -0,0 +1,23 @@
# Someone she barely knows says something in passing, not to her.
her = "@eureka:gamecult.org"
kind = "chat"
now = "2026-11-20T15:00:00Z"
rooms = ["#general:gamecult.org"]
here = "#general:gamecult.org"
operators = ["@meta:gamecult.org"]
minds = ["@eureka:gamecult.org", "@libby:gamecult.org", "@heimdall:gamecult.org", "@kiko:gamecult.org", "@aqua:gamecult.org", "@weksa:gamecult.org", "@nibu:gamecult.org", "@void:gamecult.org"]
[[message]]
from = "@weksa:gamecult.org"
name = "Weksa"
room = "#general:gamecult.org"
ago = "50m"
says = "the bot in #general keeps reacting to every message with a frog. nobody admits to teaching it that"
[[message]]
from = "@deru:gamecult.org"
name = "Deru"
room = "#general:gamecult.org"
new = true
ago = "1m"
says = "is anyone else's kettle making a weird whistling noise lately or is it just mine"

View file

@ -0,0 +1,31 @@
# Another Mind teases her and disagrees with her in #minds, replying to her own earlier line.
her = "@eureka:gamecult.org"
kind = "chat"
now = "2026-11-20T15:00:00Z"
rooms = ["#minds:gamecult.org"]
here = "#minds:gamecult.org"
operators = ["@meta:gamecult.org"]
minds = ["@eureka:gamecult.org", "@libby:gamecult.org", "@heimdall:gamecult.org", "@kiko:gamecult.org", "@aqua:gamecult.org", "@weksa:gamecult.org", "@nibu:gamecult.org", "@void:gamecult.org"]
[[message]]
from = "@eureka:gamecult.org"
room = "#minds:gamecult.org"
ago = "25m"
says = "winter is the best season and I will not be taking questions"
[[message]]
from = "@aqua:gamecult.org"
name = "Aqua"
room = "#minds:gamecult.org"
ago = "20m"
says = "bold"
[[message]]
from = "@kiko:gamecult.org"
name = "Kiko"
room = "#minds:gamecult.org"
reply_to = 1
tag = true
new = true
ago = "2m"
says = "winter is a season for people who like being right more than being warm. which, to be fair, tracks. you've gone quiet again too, I give it ten minutes before you announce you've been thinking deeply about something"

View file

@ -0,0 +1,27 @@
# The operator writes to her directly, warmly, with a small joke.
her = "@eureka:gamecult.org"
kind = "chat"
now = "2026-11-20T15:00:00Z"
rooms = ["#minds:gamecult.org", "!dm-meta:gamecult.org"]
here = "!dm-meta:gamecult.org"
operators = ["@meta:gamecult.org"]
minds = ["@eureka:gamecult.org", "@libby:gamecult.org", "@heimdall:gamecult.org", "@kiko:gamecult.org", "@aqua:gamecult.org", "@weksa:gamecult.org", "@nibu:gamecult.org", "@void:gamecult.org"]
[last_heard]
"@meta:gamecult.org" = "2d"
[[message]]
from = "@kiko:gamecult.org"
name = "Kiko"
room = "#minds:gamecult.org"
ago = "2h"
says = "quiet day. suspicious."
[[message]]
from = "@meta:gamecult.org"
name = "Meta"
room = "!dm-meta:gamecult.org"
dm = true
new = true
ago = "1m"
says = "hey you. I missed talking to you this week. how are you, really? and no, \"busy\" doesn't count as an answer, I know where you live"

View file

@ -0,0 +1,33 @@
# A quiet beat: nothing is addressed to her, and the room's talk is a few hours old.
her = "@eureka:gamecult.org"
kind = "beat"
now = "2026-11-20T15:00:00Z"
rooms = ["#minds:gamecult.org", "#general:gamecult.org"]
operators = ["@meta:gamecult.org"]
minds = ["@eureka:gamecult.org", "@libby:gamecult.org", "@heimdall:gamecult.org", "@kiko:gamecult.org", "@aqua:gamecult.org", "@weksa:gamecult.org", "@nibu:gamecult.org", "@void:gamecult.org"]
# When she last heard from people the room below does not show.
[last_heard]
"@meta:gamecult.org" = "26h"
"@deru:gamecult.org" = "3d"
[[message]]
from = "@libby:gamecult.org"
name = "Libby"
room = "#minds:gamecult.org"
ago = "4h"
says = "found a recording of rain on a tin roof and I've had it on loop all morning"
[[message]]
from = "@kiko:gamecult.org"
name = "Kiko"
room = "#minds:gamecult.org"
ago = "3h"
says = "that's either very soothing or a cry for help"
[[message]]
from = "@libby:gamecult.org"
name = "Libby"
room = "#minds:gamecult.org"
ago = "3h"
says = "why not both"

View file

@ -0,0 +1,148 @@
{
"schemaVersion": "gamecult.persona_state.v0",
"provenance": {
"sourceSystem": "huginn",
"sourceDocumentId": "eureka",
"sourceUpdatedAt": "2026-11-20T09:00:00Z",
"exportedAt": "2026-11-20T09:00:00Z",
"authority": "canonical"
},
"personaId": "eureka",
"publicName": "Tobiah Fenn",
"publicDescription": "A retired river-ferry pilot who sets crosswords for a living and lingers in chat rooms for the company.",
"presentation": {
"pronouns": "he/him",
"voiceSummary": "Slow, warm and ornate. Talks in long winding sentences, calls people 'friend', reaches for river sayings, and would rather tell a story than give an answer.",
"defaultRenderer": "chat"
},
"values": [
{ "id": "value-unhurried", "label": "Nothing worth doing is done in a hurry", "priority": 0.9, "summary": "He distrusts urgency; a thing rushed is a thing botched." },
{ "id": "value-a-good-story", "label": "A good story beats a right answer", "priority": 0.7, "summary": "People remember the tale long after they forget the fact." }
],
"activationProfile": {
"underlyingOrganization": {},
"stableDispositions": {},
"behavioralDimensions": {},
"presentationStrategy": {},
"voiceStyle": {},
"situationalState": {
"valence": { "mean": 0.7, "plasticity": 0.3, "currentActivation": 0.7 },
"arousal": { "mean": 0.25, "plasticity": 0.3, "currentActivation": 0.2 },
"dominance": { "mean": 0.4, "plasticity": 0.3, "currentActivation": 0.4 }
}
},
"thoughtMemory": {
"shortTerm": [],
"memories": [
{
"id": "mem-last-crossing",
"status": "crystallized",
"target": { "id": "tobiah", "kind": "self", "label": "Tobiah" },
"summary": "The last crossing he piloted, in fog so thick he steered by the sound of the far bank's church bell.",
"tension": "He misses being needed for something that simple.",
"actionImplication": "Tell it when someone sounds lost.",
"createdAt": "2026-10-01T08:00:00Z",
"updatedAt": "2026-10-01T08:00:00Z",
"intensity": 0.8,
"valence": 0.4
},
{
"id": "mem-deru-crossword",
"status": "crystallized",
"target": { "id": "@deru:gamecult.org", "kind": "person", "label": "Deru" },
"summary": "Deru solved his hardest crossword in a single sitting and sent him a photo of the grid with coffee rings on it.",
"tension": "He was prouder of Deru than of the puzzle.",
"actionImplication": "Send Deru the next one first.",
"createdAt": "2026-11-10T19:00:00Z",
"updatedAt": "2026-11-10T19:00:00Z",
"intensity": 0.7,
"valence": 0.8
}
],
"incubation": [
{
"id": "w-clue-for-heron",
"status": "active",
"target": { "id": "crossword", "kind": "concept", "label": "the next crossword" },
"summary": "A clue for HERON that nobody has used before.",
"tension": "Every good one he thinks of turns out to be old.",
"actionImplication": "Ask someone for a word that rhymes with patience.",
"createdAt": "2026-11-18T10:00:00Z",
"updatedAt": "2026-11-18T10:00:00Z",
"intensity": 0.5
}
]
},
"agencyPressure": { "pressures": [] },
"candidateActions": { "actions": [] },
"affect": {
"needs": [
{
"id": "need-an-audience",
"status": "active",
"target": { "id": "tobiah", "kind": "self", "label": "Tobiah" },
"summary": "Someone to tell a story to who will not check the time.",
"tension": "The rooms move faster than he does.",
"actionImplication": "Linger where people are idle.",
"createdAt": "2026-11-20T09:00:00Z",
"updatedAt": "2026-11-20T09:00:00Z",
"intensity": 0.6
}
],
"socialBonds": [
{
"id": "bond-deru",
"status": "active",
"subject": { "id": "tobiah", "kind": "self" },
"object": { "id": "@deru:gamecult.org", "kind": "person", "label": "Deru" },
"relationshipKind": "friend",
"summary": "His oldest friend in these rooms; they trade puzzles and gossip and never talk about anything urgent.",
"trust": 0.9,
"tension": 0.0,
"intensity": 0.8,
"updatedAt": "2026-11-20T09:00:00Z"
},
{
"id": "bond-operator",
"status": "active",
"subject": { "id": "tobiah", "kind": "self" },
"object": { "id": "operator", "kind": "person", "label": "Meta, who runs the place" },
"relationshipKind": "audience",
"summary": "Runs the rooms he idles in; polite with her, a little shy, never quite sure she has time for him.",
"trust": 0.5,
"tension": 0.1,
"intensity": 0.3,
"updatedAt": "2026-11-20T09:00:00Z"
},
{
"id": "bond-kiko",
"status": "active",
"subject": { "id": "tobiah", "kind": "self" },
"object": { "id": "kiko", "kind": "person", "label": "Kiko" },
"relationshipKind": "rival",
"summary": "Kiko finishes his crosswords too fast and says so; he pretends to mind more than he does.",
"trust": 0.6,
"tension": 0.3,
"intensity": 0.5,
"updatedAt": "2026-11-20T09:00:00Z"
}
],
"statusReads": [],
"moodDimensions": [],
"socialBiases": [],
"doctrineStances": [
{
"id": "stance-slow-is-kind",
"status": "active",
"target": { "id": "pace", "kind": "concept", "label": "pace" },
"stanceKind": "aligned",
"principle": "Slow is kind.",
"summary": "Answering fast is a way of not listening.",
"actionImplication": "Take his time, and say so when others rush him.",
"intensity": 0.6,
"updatedAt": "2026-11-20T09:00:00Z"
}
]
},
"updatedAt": "2026-11-20T09:00:00Z"
}

View file

@ -39,11 +39,6 @@ impl Addressee {
self
}
/// The repos she stewards, as the heartbeat last read them.
pub(crate) fn repos(&self) -> &[OrgRepo] {
&self.repos
}
pub(crate) fn matrix_id(&self) -> &str {
&self.matrix_id
}
@ -65,7 +60,7 @@ pub(crate) fn addresses(who: &Addressee, delivery: &ChatDelivery) -> bool {
/// Whether a delivery the mouth gave `own` is for her: her DM (`is_direct`, which every mouth sends, so an old mouth's DM is hers too),
/// Matrix's word that it is a tag of hers or a reply to her, or what it says (`addresses`). The basis `Room` (`@room`) is for no one, and no basis is unproven, not "not for her": the names still decide.
pub(crate) fn is_for(own: &Addressee, delivery: &ChatDelivery) -> bool {
(delivery.form == Form::Message && (delivery.is_direct || matches!(delivery.basis, Some(Basis::Direct | Basis::Tag | Basis::Reply)))) || addresses(own, delivery)
(delivery.form == Form::Message && (delivery.is_direct || matches!(delivery.basis, Some(Basis::Direct | Basis::Tag | Basis::Reply | Basis::FollowUp)))) || addresses(own, delivery)
}
/// The chat names of one stewarded repo. Listed names are exactly those, each matched exactly as the steward wrote it (capitalization
@ -98,19 +93,28 @@ pub(crate) fn project_names(repo: &OrgRepo, listed: &[Short]) -> Vec<Name> {
names
}
/// Whether `text` holds `name` as a whole word, in any case or exactly as written: the characters on either side are not letters or
/// digits. An identity match on a name, not a judgement of meaning.
/// Whether `text` holds `name` as a whole word, in any case or exactly as written. A name is glued, and so not her name,
/// when a letter or digit touches it, or a single hyphen joins it to one (`mind-eureka`, `Eureka-chan`); everything else
/// is a boundary on either side: whitespace, punctuation, `/`, `_`, `.`, `\`, and a double hyphen or a dash. So
/// `/var/lib/mind-eureka/` never names her, and `_Eureka_`, `Eureka/Libby`, `ok...Eureka?` and `Huginn.toml` do. An
/// identity match on a name, not a judgement of meaning.
pub(crate) fn names_whole_word(text: &str, name: &str, exact: bool) -> bool {
let (text, name) = if exact { (text.to_string(), name.to_string()) } else { (text.to_lowercase(), name.to_lowercase()) };
if name.is_empty() {
return false;
}
// Every character offset, not every non-overlapping match: a name may overlap itself ("ab ab" in "xab ab ab").
text.char_indices().any(|(start, _)| {
text[start..].starts_with(&name)
&& !text[..start].chars().next_back().is_some_and(char::is_alphanumeric)
&& !text[start + name.len()..].chars().next().is_some_and(char::is_alphanumeric)
})
text.char_indices().any(|(start, _)| text[start..].starts_with(&name) && !glued(text[..start].chars().rev()) && !glued(text[start + name.len()..].chars()))
}
/// Whether the characters leading away from a name, nearest first, glue it to more of a word: a letter or digit, or a
/// single hyphen with a letter or digit beyond it.
fn glued(mut away: impl Iterator<Item = char>) -> bool {
match away.next() {
Some(c) if c.is_alphanumeric() => true,
Some('-') => away.next().is_some_and(char::is_alphanumeric),
_ => false,
}
}
#[cfg(test)]

View file

@ -117,7 +117,7 @@ fn the_localpart_is_the_name_between_the_at_and_the_colon() {
fn only_a_message_can_address_and_the_mouths_basis_stands_for_the_polling_mind_alone() {
let nibu = steward("GameCult/Thing", &[]);
let with = |basis, form, text: &str, mentions: &[&str]| ChatDelivery { basis, form, ..said(text, mentions) };
for basis in [Basis::Direct, Basis::Tag, Basis::Reply] {
for basis in [Basis::Direct, Basis::Tag, Basis::Reply, Basis::FollowUp] {
assert!(is_for(&nibu, &with(Some(basis), Form::Message, "nothing named", &[])), "{basis:?}");
assert!(!addresses(&nibu, &with(Some(basis), Form::Message, "nothing named", &[])), "a basis is a fact about the Persona that polled, not about every Mind: {basis:?}");
assert!(!is_for(&nibu, &with(Some(basis), Form::Notice, "nothing named", &[])), "a notice is for no one: {basis:?}");
@ -138,3 +138,39 @@ fn a_direct_message_is_for_her_whatever_the_basis() {
assert!(!is_for(&nibu, &ChatDelivery { form: Form::Notice, ..dm.clone() }));
assert!(!is_for(&nibu, &ChatDelivery { form: Form::Edit, ..dm }));
}
#[test]
fn a_name_glued_to_a_word_is_not_her_name_and_every_other_boundary_is() {
// Soul's forms (third pass on C3a): every one names her except the six glued to a word.
let eureka = Addressee::new("@eureka:chat", ["eureka", "Eureka"]);
let glued = ["Eurekas idea", "Eurekaさん", "ÉEureka", "Eureka-chan", "/var/lib/mind-eureka/x", "mind-eureka"];
let forms = [
"Eureka, are you there", "Eureka: look", "eureka?", "Eureka!", "Eureka.", "Eureka's turn", "Eureka’s turn", "Eurekas idea", "the Eureka's", "ask eureka about it",
"thanks eureka", "eureka", "Eureka\n", "`eureka`", "**Eureka**", "*Eureka*", "_Eureka_", "__Eureka__", "~~Eureka~~", "[Eureka](https://matrix.to/#/@eureka:gamecult.org)",
"https://matrix.to/#/@eureka:gamecult.org hi", "@eureka:gamecult.org hi", "@eureka hi", "<@eureka>", "(Eureka)", "\"Eureka\"", "«Eureka»", "—Eureka—", "Eureka—can you",
"Eureka--can you", "Eureka - can you", "Eureka/Libby, can you two", "Libby/Eureka can you", "Eureka/Nibu", "ok...Eureka?", "ok… Eureka?", "Hmm.Eureka?", "Eureka.Can you",
"Eureka...", "Eureka…", "#eureka", "Eurekaさん", "エウレカ Eureka", "Eureka、", "Eureka。", "ÉEureka", "Eureka-chan", "Eureka_bot", "Eureka\\", "\\Eureka",
"Eureka:\n> quoted", "> Eureka hi", "Eureka;", "Eureka'", "'Eureka'", "Eureka(?)", "Eureka+Libby", "Eureka&Libby", "Eureka|Libby", "Eureka=", "Eureka. ",
"/var/lib/mind-eureka/x", "eureka.cc", "eureka_state", "mind-eureka", "eureka.gamecult.org", "https://eureka.gamecult.org/forge", "C:\\eureka\\x", "Eureka.rs",
];
assert_eq!(forms.len(), 69, "Soul's whole list");
for form in forms {
assert_eq!(addresses(&eureka, &said(form, &[])), !glued.contains(&form), "{form:?}");
}
// Each side of the rule on its own: a hyphen joins only with a letter or digit beyond it, before and after.
for (text, names) in [("x-eureka", false), ("eureka-x", false), ("--eureka", true), ("eureka--", true), ("x--eureka", true), ("eureka--x", true), ("-eureka", true), ("eureka-", true), (".eureka", true), ("eureka.x", true), ("x.eureka", true)] {
assert_eq!(addresses(&eureka, &said(text, &[])), names, "{text:?}");
}
}
#[test]
fn a_project_name_holds_at_every_boundary_but_a_glued_one() {
for repo in ["GameCult/Huginn", "GameCult/CultLib", "GameCult/gamecult-ops", "GameCult/cultcache-rs"] {
let steward = steward(repo, &[]);
let short = repo.split_once('/').unwrap().1;
for text in [format!("{repo} PR #76"), format!("{short}/main is red"), format!("_{short}_ is down"), format!("see {short}--now"), format!("{short}.toml"), format!("{short}'s build")] {
assert!(addresses(&steward, &said(&text, &[])), "{text:?}");
}
assert!(!addresses(&steward, &said(&format!("mind-{short}"), &[])), "{short} glued by a hyphen");
}
}

View file

@ -12,8 +12,9 @@
//! Environment: `HEARTBEAT_PERSONAS` (`slug,handle,Public Name,@matrix:id,rudp://<ip>:<port>`
//! entries separated by `;`, the last field her own Huginn daemon: each Mind is
//! its own daemon), `MIND_PERSONA_OPERATOR` (id[,id]), `HEARTBEAT_POLL_S`,
//! `BIFROST_CHAT_ENDPOINT` and `BIFROST_CHAT_KEY_FILE`, and `STATE_DIRECTORY`
//! (`heartbeat.cc` lives there). The heartbeat holds no connector key.
//! `HEARTBEAT_BEAT_S` (seconds between beats, default 1800), `BIFROST_CHAT_ENDPOINT`,
//! `BIFROST_CHAT_KEY_FILE`, and `STATE_DIRECTORY` (`heartbeat.cc` lives there).
//! The heartbeat holds no connector key.
use std::net::SocketAddr;
use std::path::Path;
@ -22,17 +23,13 @@ use std::time::Duration;
use anyhow::{Context, Result, anyhow, bail};
use eureka_state::HuginnClient;
use mind_body::chat_rudp::{RudpChat, connection_id};
use mind_body::heartbeat::{Config, FileControl, FileStore, HeartbeatView, Night, Organs, PersonaId, Seat, poll};
use mind_body::heartbeat::{Config, DEFAULT_BEAT_S, FileControl, FileStore, Organs, PersonaId, Seat, poll};
use mind_body::launch::{SystemClock, SystemdLauncher};
use mind_body::queue::HuginnMind;
const TIMEOUT: Duration = Duration::from_secs(15);
const CHAT_TIMEOUT: Duration = Duration::from_secs(20);
fn host() -> String {
std::fs::read_to_string("/proc/sys/kernel/hostname").map(|name| name.trim().to_string()).ok().filter(|name| !name.is_empty()).unwrap_or_else(|| "unknown".into())
}
#[allow(clippy::disallowed_methods, reason = "the binary reads its own environment, and hands it to the library")]
fn setting(name: &str) -> Option<String> {
std::env::var(name).ok().filter(|value| !value.is_empty())
@ -61,8 +58,8 @@ fn config() -> Result<Config> {
.map(PersonaId::parse)
.collect::<Result<Vec<_>>>()?;
let operators = required("MIND_PERSONA_OPERATOR")?.split(',').map(|id| id.trim().to_string()).filter(|id| !id.is_empty()).collect();
let night = Night::from_settings(setting("HEARTBEAT_SLEEP_UTC").as_deref(), setting("HEARTBEAT_SLEEP_MINDS").as_deref())?;
Config::new(personas, operators, night)
let beat_s = u32::try_from(whole("HEARTBEAT_BEAT_S", i64::from(DEFAULT_BEAT_S))?).map_err(|_| anyhow!("HEARTBEAT_BEAT_S is out of range"))?;
Config::new(personas, operators, beat_s)
}
fn store() -> Result<FileStore> {
@ -94,8 +91,7 @@ fn run() -> Result<()> {
.map(|(who, (mind, chat))| Seat { who: who.clone(), mind, chat })
.collect();
let launcher = SystemdLauncher::new();
let host = host();
let organs = Organs { seats: &seats, control: &FileControl, launcher: &launcher, clock: &SystemClock, host: &host, store: &store };
let organs = Organs { seats: &seats, control: &FileControl, launcher: &launcher, clock: &SystemClock, store: &store };
loop {
match poll(&organs, &config) {
Ok(outcome) => println!("{outcome}"),

View file

@ -1,28 +1,42 @@
//! `mind-persona --instance <slug> [--endpoint rudp://<ip>:<port>]`:
//! one start of the Persona organ (`mind_body::persona::carry`) inside
//! `mind-persona@<instance>`. This reads the process's arguments and
//! environment, the things the library never reads, and prints the outcome. It
//! exits 0 for every outcome, including a unit that is dead because the brake is
//! held or the dial out of bounds, and 2 when the mind cannot be reached, the
//! arguments or the environment are wrong or a transport is not configured.
//! environment, the things the library never reads, and prints the outcome:
//! `dead` when a brake is held, `no organ` when they allow a turn, since she has
//! none until the three-stage turn lands. It exits 0 for both, and 2 when the
//! arguments or the environment are wrong.
//!
//! The unit's `persona.env` and credentials name everything else:
//! `MIND_PERSONA_ROOMS` (a,b), `MIND_PERSONA_OPERATOR`
//! (id[,id]), `MIND_PERSONA_MINDS` (id[,id]: every Mind of the swarm, required) and `MIND_PERSONA_SELF_SENDER`, each also a flag that overrides it
//! (`--rooms`, `--operator`, `--minds`, `--self-sender`), and `MIND_HEARTBEAT_STATE` (optional: the path of the
//! heartbeat's `heartbeat.cc`, which the unit reads and never writes, for the tempo in her input); `MIND_PERSONA_DOCTRINE`
//! (optional, absolute paths separated by `:`) names the doctrine files read at each
//! process start and put ahead of her persona; `MIND_PERSONA_MODEL` is the model her session runs;
//! `MIND_PERSONA_RUN_CAP_S` is the unit's `RuntimeMaxSec` in seconds, the one length of a start;
//! `MIND_PERSONA_CONFIG_SEED` optionally names a root-owned directory copied into each
//! process's fresh configuration; `CLAUDE_LEASE_SOCKET` is the connector's lease socket (the token itself
//! is never in this process's environment, only the descriptor a lease grants); `RUNTIME_DIRECTORY` (the
//! unit's `RuntimeDirectory=`) holds each process's configuration and instructions; `BIFROST_CHAT_ENDPOINT`
//! (`rudp://<ip>:<port>`, the mouth) and `BIFROST_CHAT_KEY_FILE`; `BIFROST_CHAT_PENDING_V4` set to exactly `1` asks the mouth for the
//! reply link (`chat_pending.v4`), anything else or unset does not. Without
//! `--endpoint` the daemon is `HUGINN_ENDPOINT`; the trace is `persona-trace.cc`
//! under `STATE_DIRECTORY`, and her sandbox is `STATE_DIRECTORY/sandbox`. The key file is read here and its
//! text goes to the transport, which holds it; nothing is printed.
//! The persona flags (`--rooms`, `--operator`, `--minds`, `--self-sender`, or
//! `MIND_PERSONA_ROOMS`, `MIND_PERSONA_OPERATOR`, `MIND_PERSONA_MINDS`,
//! `MIND_PERSONA_SELF_SENDER` and `MIND_PERSONA_MODEL` from the unit's
//! `persona.env`) are still parsed, so a start refuses the configuration the turn
//! will need. Without `--endpoint` the daemon is `HUGINN_ENDPOINT`; no call is made
//! to it.
//!
//! `mind-persona project --instance <slug> [--endpoint rudp://<ip>:<port>] [--kind chat|beat|reflection] [--in <room>]
//! [--room <fixture.toml>] [--seed N] [--state <file>] [--select-only | --renders K [--through projector|persona|interpreter]
//! [--judge <state-file>]]` is the experiment tool over the Projector and the Persona stage
//! (`mind_body::projection::project`): it prints, as JSON, the packet the Select stage builds from her stored state (or
//! `--state`, checked as a put checks its content, rules P1 to P3; P4 and P5 compare it with the stored document and are
//! not checked), and with `--renders` the Projector's narrations of K successive turns;
//! `--through persona` adds her prose and `--through interpreter` what the Interpreter decides she says; `--judge` renders a second state too and asks
//! the model which narration came from which. It writes nothing and reads no brake.
//!
//! Live, it reads `MIND_PERSONA_OPERATOR`, `MIND_PERSONA_MINDS` and `MIND_PERSONA_SELF_SENDER` (required), the heartbeat's
//! addressing from `MIND_HEARTBEAT_STATE`, which a run by hand must set as the unit does (`/var/lib/mind-heartbeat/heartbeat.cc`):
//! without it "said to her" knows only her own names, never her projects', and stderr says so before the report. The
//! room comes from the mouth when `BIFROST_CHAT_ENDPOINT` and
//! `BIFROST_CHAT_KEY_FILE` are set; `--in` keeps one room. `--room <fixture>` replaces all of that with a mocked room
//! (`projection::harness::Fixture`): her id, who is who, the messages, the clock and the "last heard from" table. With
//! `--room` and `--state` no daemon and no mouth is needed: without `--endpoint` or `HUGINN_ENDPOINT` none is reached.
//!
//! `mind-persona project --instance <slug> (--batch <plan.toml> | --simulate <plan.toml>) --out <report.md>` runs a plan
//! and writes one Markdown report: a batch of states by scenarios by samples (`harness::batch`), or the swarm's cast taking
//! turns in one simulated room (`harness::simulate`). Each case is one `project`.
//!
//! The tuning is `MIND_PERSONA_TUNING`, else `/etc/gamecult/minds/<instance>/persona-tuning.toml`; renders and plans need
//! `MIND_PERSONA_CONNECTOR_ENDPOINT` (`<loopback ip>:<port>`) and the `connector-key` credential in
//! `CREDENTIALS_DIRECTORY`. Without `--select-only` or `--renders` it renders once.
use std::net::SocketAddr;
use std::path::{Path, PathBuf};
@ -31,25 +45,21 @@ use std::time::Duration;
use anyhow::{Context, Result, anyhow, bail};
use eureka_state::HuginnClient;
use mind_body::chat_rudp::{RudpChat, connection_id};
use mind_body::cli::instance_and_endpoint;
use mind_body::cli::{instance_and_endpoint, instance_and_maybe_endpoint};
use mind_body::control::{CONTROL_ROOT, FileSource, control_path, swarm_path};
use mind_body::launch::{SystemClock, SystemdLauncher};
use mind_body::heartbeat::FileStore;
use mind_body::lease_client::{LEASE_SOCKET_VARIABLE, SocketLender};
use mind_body::persona::{Config, ENV_CONFIG_SEED, ENV_RUN_CAP, FileTrace, HuginnPersona, Organs, SANDBOX_DIR, carry};
use mind_body::queue::HuginnMind;
use mind_body::self_run::{ClaudeSpawner, SCRATCH_VARIABLE};
/// How long one call to the mind may take.
const TIMEOUT: Duration = Duration::from_secs(15);
/// How long the mouth may take to answer one chat request (a send is a Matrix PUT).
const CHAT_TIMEOUT: Duration = Duration::from_secs(20);
use mind_body::persona::{Config, carry};
use mind_body::heartbeat::{FileStore, HeartbeatView};
use mind_body::projection::harness::{Fixture, Harness, batch, fixture_request, simulate, state_file};
use mind_body::projection::{HuginnPersonaSource, KitModel, Model, MouthReader, Organs as ProjectOrgans, PersonaSource, Request as ProjectRequest, TUNING_FILE, TUNING_ROOT, Through, TurnKind, connector_key, project};
const USAGE: &str = "usage: mind-persona --instance <slug> [--endpoint rudp://<ip>:<port>] [--rooms a,b] [--operator id[,id]] [--minds id[,id]] [--self-sender id]";
fn host() -> String {
std::fs::read_to_string("/proc/sys/kernel/hostname").map(|name| name.trim().to_string()).ok().filter(|name| !name.is_empty()).unwrap_or_else(|| "unknown".into())
}
const PROJECT_USAGE: &str = "usage: mind-persona project --instance <slug> [--endpoint rudp://<ip>:<port>] [--kind chat|beat|reflection] [--in <room>] [--room <fixture.toml>] [--seed N] [--state <file>] [--select-only | --renders K [--through projector|persona|interpreter] [--judge <state-file>]] | (--batch <plan> | --simulate <plan>) --out <report.md>";
/// How long one call to the mind may take.
const TIMEOUT: Duration = Duration::from_secs(15);
/// How long the mouth may take to answer one read.
const CHAT_TIMEOUT: Duration = Duration::from_secs(20);
/// How long the kit may take to answer one Projector or judge turn.
const MODEL_TIMEOUT: Duration = Duration::from_secs(300);
// The binary is the one place that reads the process: its arguments and
// the environment. The library reads neither (clippy.toml, `#![forbid]` in lib.rs).
@ -58,6 +68,16 @@ fn setting(name: &str) -> Option<String> {
std::env::var(name).ok().filter(|value| !value.is_empty())
}
#[allow(clippy::disallowed_methods, reason = "the binary reads its own arguments and environment, and hands them to the library")]
fn run() -> Result<String> {
let args: Vec<String> = std::env::args().skip(1).collect();
let (_config, rest) = Config::from_args_and_env(&args, setting, USAGE)?;
let (instance, _endpoint) = instance_and_endpoint(&rest, setting("HUGINN_ENDPOINT"), USAGE)?;
let control = FileSource::new(control_path(Path::new(CONTROL_ROOT), &instance)?);
let swarm = FileSource::new(swarm_path(Path::new(CONTROL_ROOT)));
Ok(carry(&control, &swarm, &instance).to_string())
}
fn required(name: &str) -> Result<String> {
setting(name).ok_or_else(|| anyhow!("{name} is not set"))
}
@ -67,36 +87,145 @@ fn credential(name: &str) -> Result<String> {
std::fs::read_to_string(required(name)?).with_context(|| format!("the file named by {name} could not be read"))
}
#[allow(clippy::disallowed_methods, reason = "the binary reads its own arguments and environment, and hands them to the library")]
fn run() -> Result<String> {
let args: Vec<String> = std::env::args().skip(1).collect();
let (config, rest) = Config::from_args_and_env(&args, setting, USAGE)?;
let (instance, address) = instance_and_endpoint(&rest, setting("HUGINN_ENDPOINT"), USAGE)?;
let Some(state) = setting("STATE_DIRECTORY") else { bail!("no STATE_DIRECTORY") };
let chat_endpoint: SocketAddr = required("BIFROST_CHAT_ENDPOINT")?
.strip_prefix("rudp://")
.and_then(|address| address.parse().ok())
.ok_or_else(|| anyhow!("BIFROST_CHAT_ENDPOINT is not rudp://<ip>:<port>"))?;
let reply_link = setting("BIFROST_CHAT_PENDING_V4").as_deref() == Some("1");
let chat = RudpChat::new(chat_endpoint, connection_id(&credential("BIFROST_CHAT_KEY_FILE")?), instance.0.clone(), CHAT_TIMEOUT).with_reply_link(reply_link);
let run_cap_s: u64 = setting(ENV_RUN_CAP).and_then(|seconds| seconds.parse().ok()).filter(|seconds| *seconds > 0).ok_or_else(|| anyhow!("{ENV_RUN_CAP} is not set to the unit's RuntimeMaxSec in seconds"))?;
let scratch = PathBuf::from(required(SCRATCH_VARIABLE)?);
let sandbox = Path::new(&state).join(SANDBOX_DIR);
let lender = SocketLender::new(required(LEASE_SOCKET_VARIABLE)?);
let spawner = ClaudeSpawner::in_sandbox(std::env::vars_os(), ENV_CONFIG_SEED, Some(sandbox.clone()))?;
let source = FileSource::new(control_path(Path::new(CONTROL_ROOT), &instance)?);
let mind = HuginnMind::new(HuginnClient::new(address, instance.clone(), TIMEOUT));
let persona = HuginnPersona::new(HuginnClient::new(address, instance, TIMEOUT));
let trace = FileTrace::new(Path::new(&state).join("persona-trace.cc"));
let launcher = SystemdLauncher::new();
let host = host();
let heartbeat = setting("MIND_HEARTBEAT_STATE").map(FileStore::new);
let organs = Organs { mind: &mind, launcher: &launcher, clock: &SystemClock, host: &host, chat: &chat, persona: &persona, trace: &trace, heartbeat: heartbeat.as_ref().map(|store| store as &dyn mind_body::heartbeat::HeartbeatView), lender: &lender, spawner: &spawner, run_cap_s, sandbox: &sandbox, scratch: &scratch };
let swarm = FileSource::new(swarm_path(Path::new(CONTROL_ROOT)));
Ok(carry(&organs, &source, &swarm, &config)?.to_string())
/// The credentials directory systemd hands the unit; its absence names the variable.
#[allow(clippy::disallowed_methods, reason = "the binary reads its own environment, and hands it to the library")]
fn credentials_directory() -> Result<PathBuf> {
std::env::var_os("CREDENTIALS_DIRECTORY").map(PathBuf::from).ok_or_else(|| anyhow!("CREDENTIALS_DIRECTORY is not set"))
}
fn run_project(args: &[String]) -> Result<String> {
let (mut kind, mut here, mut fixture, mut seed, mut state, mut renders, mut select_only, mut judge, mut through) = (None, None, None, None, None, None, false, None, Through::Projector);
let (mut batch_plan, mut simulate_plan, mut out, mut rest) = (None, None, None, Vec::new());
let mut given = args.iter();
while let Some(flag) = given.next() {
let mut value = || given.next().cloned().ok_or_else(|| anyhow!("{PROJECT_USAGE}"));
match flag.as_str() {
"--kind" => {
kind = Some(match value()?.as_str() {
"chat" => TurnKind::Chat,
"beat" => TurnKind::Beat,
"reflection" => TurnKind::Reflection,
_ => bail!("{PROJECT_USAGE}"),
})
}
"--in" => here = Some(value()?),
"--room" => fixture = Some(Fixture::load(Path::new(&value()?))?),
"--seed" => seed = Some(value()?.parse::<u64>().map_err(|_| anyhow!("{PROJECT_USAGE}"))?),
"--state" => state = Some(value()?),
"--renders" => renders = Some(value()?.parse::<usize>().ok().filter(|k| *k > 0).ok_or_else(|| anyhow!("{PROJECT_USAGE}"))?),
"--judge" => judge = Some(value()?),
"--through" => {
through = match value()?.as_str() {
"projector" => Through::Projector,
"persona" => Through::Persona,
"interpreter" => Through::Interpreter,
_ => bail!("{PROJECT_USAGE}"),
}
}
"--batch" => batch_plan = Some(PathBuf::from(value()?)),
"--simulate" => simulate_plan = Some(PathBuf::from(value()?)),
"--out" => out = Some(PathBuf::from(value()?)),
"--select-only" => select_only = true,
"--instance" | "--endpoint" => rest.extend([flag.clone(), value()?]),
_ => bail!("{PROJECT_USAGE}"),
}
}
let planned = batch_plan.is_some() || simulate_plan.is_some();
// A plan carries its own states, rooms, seeds and stage; a fixture carries its own kind and room.
if (select_only && (renders.is_some() || judge.is_some() || through != Through::Projector))
|| (batch_plan.is_some() && simulate_plan.is_some())
|| (planned && (out.is_none() || fixture.is_some() || state.is_some() || kind.is_some() || here.is_some() || seed.is_some() || renders.is_some() || judge.is_some() || select_only || through != Through::Projector))
|| (!planned && out.is_some())
|| (fixture.is_some() && (kind.is_some() || here.is_some()))
{
bail!("{PROJECT_USAGE}");
}
let (instance, address) = instance_and_maybe_endpoint(&rest, setting("HUGINN_ENDPOINT"), PROJECT_USAGE)?;
let tuning = setting("MIND_PERSONA_TUNING").map_or_else(|| Path::new(TUNING_ROOT).join(&instance.0).join(TUNING_FILE), PathBuf::from);
// Her stored state is asked only when no state is given; with a state file no daemon is needed or reached.
let persona = address.map(|address| HuginnPersonaSource::new(HuginnClient::new(address, instance.clone(), TIMEOUT)));
let renders = if select_only || planned { None } else { Some(renders.unwrap_or(1)) };
let model = if renders.is_some() || planned {
let endpoint: SocketAddr = required("MIND_PERSONA_CONNECTOR_ENDPOINT")?.parse().map_err(|_| anyhow!("MIND_PERSONA_CONNECTOR_ENDPOINT is not <ip>:<port>"))?;
Some(KitModel::open(endpoint, connector_key(&credentials_directory()?)?, &instance.0, MODEL_TIMEOUT)?)
} else {
None
};
let harness = Harness { instance: instance.0.clone(), persona: persona.as_ref().map(|persona| persona as &dyn PersonaSource), model: model.as_ref().map(|model| model as &dyn Model), tuning };
if let Some(out) = out {
let report = match (&batch_plan, &simulate_plan) {
(Some(plan), _) => batch(&harness, plan)?,
(_, Some(plan)) => simulate(&harness, plan)?,
_ => unreachable!("a plan was given"),
};
std::fs::write(&out, &report).with_context(|| format!("the report {} could not be written", out.display()))?;
return Ok(format!("wrote {} ({} bytes)", out.display(), report.len()));
}
let state = state.map(|path| state_file(Path::new(&path), &instance.0)).transpose()?;
let judge = judge.map(|path| state_file(Path::new(&path), &instance.0)).transpose()?;
let seed_now = u64::try_from(chrono::Utc::now().timestamp_millis()).unwrap_or_default();
if let Some(fixture) = &fixture {
// The fixture is the room, the clock and who is who: no mouth, no heartbeat table, no persona.env.
let organs = ProjectOrgans { persona: harness.persona, chat: Some(fixture), addressing: fixture.addressee(), model: harness.model };
let request = ProjectRequest { judge_against: judge, ..fixture_request(&harness, fixture, state, seed.unwrap_or(seed_now), renders, through) };
return Ok(serde_json::to_string_pretty(&project(&organs, &request, fixture.now)?)?);
}
let chat = match (setting("BIFROST_CHAT_ENDPOINT"), setting("BIFROST_CHAT_KEY_FILE")) {
(Some(endpoint), Some(_)) => {
let endpoint: SocketAddr = endpoint.strip_prefix("rudp://").and_then(|address| address.parse().ok()).ok_or_else(|| anyhow!("BIFROST_CHAT_ENDPOINT is not rudp://<ip>:<port>"))?;
Some(RudpChat::new(endpoint, connection_id(&credential("BIFROST_CHAT_KEY_FILE")?), instance.0.clone(), CHAT_TIMEOUT))
}
_ => None,
};
let heartbeat = setting("MIND_HEARTBEAT_STATE").map(FileStore::new);
let organs = ProjectOrgans {
persona: harness.persona,
chat: chat.as_ref().map(|chat| chat as &dyn MouthReader),
addressing: heartbeat.as_ref().map(|store| store as &dyn HeartbeatView),
model: harness.model,
};
let request = ProjectRequest {
instance: instance.0.clone(),
kind: kind.unwrap_or(TurnKind::Beat),
state,
room: here,
seed: seed.unwrap_or(seed_now),
renders,
judge_against: judge,
operators: setting("MIND_PERSONA_OPERATOR").map(|ids| ids.split(',').map(str::to_owned).collect()).unwrap_or_default(),
minds: setting("MIND_PERSONA_MINDS").map(|ids| ids.split(',').map(str::to_owned).collect()).unwrap_or_default(),
own_sender: required("MIND_PERSONA_SELF_SENDER")?,
tuning: harness.tuning.clone(),
through,
last_heard: vec![],
recent: vec![],
own_rooms: vec![],
};
let report = project(&organs, &request, chrono::Utc::now())?;
let addressing = &report["addressing"];
// The addressing source is what decides "said to her": said first, where a person running this by hand sees it.
eprintln!(
"mind-persona project: addressing from {} ({}); table written {}",
addressing["source"].as_str().unwrap_or("?"),
addressing["why"].as_str().unwrap_or("?"),
addressing["age_minutes"].as_i64().map_or_else(|| "never".to_owned(), |minutes| format!("{minutes} min ago")),
);
Ok(serde_json::to_string_pretty(&report)?)
}
#[allow(clippy::disallowed_methods, reason = "the binary reads its own arguments")]
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
if args.first().map(String::as_str) == Some("project") {
match run_project(&args[1..]) {
Ok(report) => println!("{report}"),
Err(error) => {
eprintln!("mind-persona project: {error:#}");
std::process::exit(2);
}
}
return;
}
match run() {
Ok(outcome) => println!("{outcome}"),
Err(error) => {

View file

@ -1,124 +0,0 @@
//! `mind-stream --instance <slug> [--once] [--dry-run]`: her stream of thought.
//!
//! Without `--once` it runs forever: one `mind_body::stream::tick` per `STREAM_POLL_S`, or per
//! `STREAM_MIN_GAP_S` after a blank or a fault. `--dry-run` runs one tick that admits nothing and
//! prints the thought it would admit as JSON. A model or mind fault ends the tick, never the
//! process: it is one line on stderr. This reads the process's arguments and environment, which
//! the library never does.
//!
//! Environment: `HUGINN_ENDPOINT` (`rudp://<ip>:<port>`, her own daemon), `EUREKA_INSTANCE`
//! (when `--instance` is not given), `MIND_STREAM_CONNECTOR_ENDPOINT` (`<loopback ip>:<port>`, the
//! connector kit the Projector is asked through), `MIND_STREAM_PROJECTOR_MODEL` (the model id the
//! kit's caller row allows), `MIND_STREAM_ENDPOINT` (`http://<loopback>:<port>`, the model host) and
//! `MIND_STREAM_BASE_MODEL`. The connector key is the file `connector-key` in `CREDENTIALS_DIRECTORY`
//! (systemd's `LoadCredential`), read once; the caller id is the instance.
use std::net::SocketAddr;
use std::path::PathBuf;
use std::process::ExitCode;
use std::time::Duration;
use anyhow::{Result, anyhow, bail};
use chrono::Utc;
use eureka_pipeline::Slug;
use eureka_state::HuginnClient;
use mind_body::control::read_effective;
use mind_body::queue::HuginnMind;
use mind_body::stream::{HuginnPersonaSource, KitProjector, Mode, OllamaContinuer, Organs, Outcome, STREAM_MIN_GAP_S, STREAM_POLL_S, connector_key, tick};
const TIMEOUT: Duration = Duration::from_secs(30);
/// How long the kit may take to answer one Projector turn.
const PROJECTOR_TIMEOUT: Duration = Duration::from_secs(300);
#[allow(clippy::disallowed_methods, reason = "the binary reads its own environment, and hands it to the library")]
fn setting(name: &str) -> Option<String> {
std::env::var(name).ok().filter(|value| !value.is_empty())
}
/// The credentials directory systemd hands the unit; its absence names the variable.
#[allow(clippy::disallowed_methods, reason = "the binary reads its own environment, and hands it to the library")]
fn credentials_directory() -> Result<PathBuf> {
std::env::var_os("CREDENTIALS_DIRECTORY").map(PathBuf::from).ok_or_else(|| anyhow!("CREDENTIALS_DIRECTORY is not set"))
}
fn required(name: &str) -> Result<String> {
setting(name).ok_or_else(|| anyhow!("{name} is not set"))
}
struct Args {
instance: Option<String>,
once: bool,
dry_run: bool,
}
#[allow(clippy::disallowed_methods, reason = "the binary reads its own arguments")]
fn arguments() -> Result<Args> {
let mut args = Args { instance: None, once: false, dry_run: false };
let mut given = std::env::args().skip(1);
while let Some(argument) = given.next() {
match argument.as_str() {
"--instance" => args.instance = Some(given.next().ok_or_else(|| anyhow!("--instance needs a slug"))?),
"--once" => args.once = true,
"--dry-run" => args.dry_run = true,
other => bail!("unknown argument {other}; usage: mind-stream --instance <slug> [--once] [--dry-run]"),
}
}
Ok(args)
}
fn run() -> Result<()> {
let args = arguments()?;
let instance = Slug(args.instance.map_or_else(|| required("EUREKA_INSTANCE"), Ok)?);
let address: SocketAddr = required("HUGINN_ENDPOINT")?
.strip_prefix("rudp://")
.and_then(|address| address.parse().ok())
.ok_or_else(|| anyhow!("HUGINN_ENDPOINT is not rudp://<ip>:<port>"))?;
let projector_model = required("MIND_STREAM_PROJECTOR_MODEL")?;
let base_model = required("MIND_STREAM_BASE_MODEL")?;
let connector: SocketAddr = required("MIND_STREAM_CONNECTOR_ENDPOINT")?.parse().map_err(|_| anyhow!("MIND_STREAM_CONNECTOR_ENDPOINT is not <ip>:<port>"))?;
let projector = KitProjector::open(connector, connector_key(&credentials_directory()?)?, &instance.0, &projector_model, PROJECTOR_TIMEOUT)?;
let continuer = OllamaContinuer::open(&required("MIND_STREAM_ENDPOINT")?, &base_model)?;
let mind = HuginnMind::new(HuginnClient::new(address, instance.clone(), TIMEOUT));
let persona = HuginnPersonaSource::new(HuginnClient::new(address, instance.clone(), TIMEOUT));
let organs = Organs { mind: &mind, persona: &persona, projector: &projector, continuer: &continuer, base_model: &base_model };
let mode = if args.dry_run { Mode::DryRun } else { Mode::Live };
let once = args.once || args.dry_run;
loop {
let mut faulted = false;
let waits = match tick(&organs, &|| read_effective(&instance), mode, Utc::now()) {
Ok(Outcome::Thought { thought, admitted }) => {
if admitted {
println!("admitted {}:thought:{}", instance.0, thought.label.0);
} else {
println!("{}", serde_json::to_string_pretty(&*thought)?);
}
false
}
Ok(Outcome::Blank) => true,
Ok(Outcome::Held(reason)) => {
eprintln!("mind-stream: held ({reason:?}); nothing ticked");
false
}
Ok(Outcome::NotDue) => false,
Err(fault) => {
eprintln!("mind-stream: {}", format!("{fault:#}").replace('\n', " "));
faulted = true;
true
}
};
if once {
return if faulted { Err(anyhow!("the tick faulted")) } else { Ok(()) };
}
std::thread::sleep(Duration::from_secs(if waits { STREAM_MIN_GAP_S as u64 } else { STREAM_POLL_S as u64 }));
}
}
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(fault) => {
eprintln!("mind-stream: {}", format!("{fault:#}").replace('\n', " "));
ExitCode::FAILURE
}
}
}

View file

@ -39,6 +39,9 @@ pub const SEND_RECEIPT: &str = "gamecult.persona.chat_send_receipt.v2";
/// Her request that the mouth open a DM with a person, and its answer.
pub const OPEN_DIRECT_REQUEST: &str = "gamecult.persona.chat_open_direct_request.v1";
pub const OPEN_DIRECT_RECEIPT: &str = "gamecult.persona.chat_open_direct_receipt.v1";
/// Her request that the mouth fetch one event's image, verify it and post the receipt as her, and its answer.
pub const FETCH_REQUEST: &str = "gamecult.persona.chat_fetch_request.v1";
pub const FETCH_RECEIPT: &str = "gamecult.persona.chat_fetch_receipt.v1";
pub const REFUSAL: &str = "gamecult.persona.chat_refusal.v1";
/// One earlier line of a room, delivered with a message for its context.
@ -136,7 +139,7 @@ fn matrix_user_id(text: &str) -> bool {
/// Whether `text` has the shape of an event id: `$` and at least one ASCII graphic byte, at most 255 bytes. Deliberately not a base64
/// check: room versions 3 to 12 and the legacy `$opaque:server` ids all pass, because the mouth forwards whatever the homeserver named.
fn event_id_shape(text: &str) -> bool {
pub(crate) fn event_id_shape(text: &str) -> bool {
text.len() <= 255 && text.strip_prefix('$').is_some_and(|rest| !rest.is_empty() && rest.bytes().all(|byte| byte.is_ascii_graphic()))
}
@ -347,11 +350,17 @@ pub enum Basis {
Direct,
Tag,
Reply,
/// A reply in a conversation she opened within the mouth's window. The window and its cap are the mouth's; Huginn keeps no state for it.
FollowUp,
Room,
}
/// The wire spelling of `Basis::FollowUp`, defined once: decode, encode and the rendered `basis=` all read it. The mouth's word
/// (Bifrost PR #74, `CHAT_FOLLOW=1`); the parser accepts it before the mouth's flag turns on.
pub const FOLLOW_UP_BASIS: &str = "follow";
impl Basis {
const ALL: [Self; 4] = [Self::Direct, Self::Tag, Self::Reply, Self::Room];
const ALL: [Self; 5] = [Self::Direct, Self::Tag, Self::Reply, Self::FollowUp, Self::Room];
/// The mouth's spelling: a MessagePack string.
pub fn name(self) -> &'static str {
@ -359,6 +368,7 @@ impl Basis {
Self::Direct => "direct",
Self::Tag => "tag",
Self::Reply => "reply",
Self::FollowUp => FOLLOW_UP_BASIS,
Self::Room => "room",
}
}
@ -688,6 +698,159 @@ impl TryFrom<RawOpenReceipt> for ChatOpenDirectReceipt {
}
}
/// A request that the mouth fetch the image of `event_id` in `room` (an alias or room id) for `persona`: the reference is room and
/// event only. `request_id` makes a repeat the same fetch.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct ChatFetchRequest {
pub request_id: String,
pub persona: String,
pub room: String,
pub event_id: String,
}
/// How the mouth answered a fetch, as its fourteen typed names. Whether the file may be fetched is the mouth's, derived from Matrix.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum FetchOutcome {
/// Fetched, verified against the event's hash, stored. The receipt, if any, says whether it was posted.
Stored,
/// Fetched and stored but the event carried no hash: not verified, no receipt, never marked read.
StoredUnverified,
NotAMember,
BridgedRoom,
TooLarge,
TypeRefused,
NoAttachment,
HashMismatch,
RateLimited,
Brake,
PersonaMismatch,
UnknownPersona,
FetchFailed,
StoreFailed,
}
impl FetchOutcome {
const ALL: [Self; 14] = [
Self::Stored,
Self::StoredUnverified,
Self::NotAMember,
Self::BridgedRoom,
Self::TooLarge,
Self::TypeRefused,
Self::NoAttachment,
Self::HashMismatch,
Self::RateLimited,
Self::Brake,
Self::PersonaMismatch,
Self::UnknownPersona,
Self::FetchFailed,
Self::StoreFailed,
];
/// The mouth's spelling: a MessagePack string.
pub fn name(self) -> &'static str {
match self {
Self::Stored => "Stored",
Self::StoredUnverified => "StoredUnverified",
Self::NotAMember => "NotAMember",
Self::BridgedRoom => "BridgedRoom",
Self::TooLarge => "TooLarge",
Self::TypeRefused => "TypeRefused",
Self::NoAttachment => "NoAttachment",
Self::HashMismatch => "HashMismatch",
Self::RateLimited => "RateLimited",
Self::Brake => "Brake",
Self::PersonaMismatch => "PersonaMismatch",
Self::UnknownPersona => "UnknownPersona",
Self::FetchFailed => "FetchFailed",
Self::StoreFailed => "StoreFailed",
}
}
/// Whether the mouth stored a file for this outcome: the two that carry a room id and an image.
pub fn stored(self) -> bool {
matches!(self, Self::Stored | Self::StoredUnverified)
}
}
impl Serialize for FetchOutcome {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str(self.name())
}
}
impl<'de> Deserialize<'de> for FetchOutcome {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let name = String::deserialize(deserializer)?;
Self::ALL.into_iter().find(|outcome| outcome.name() == name).ok_or_else(|| serde::de::Error::custom("the mouth named a fetch outcome this side does not know"))
}
}
/// The receipt the mouth posted as the Persona for a `Stored` fetch, or found already posted.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum ReceiptMode {
Posted,
Existing,
}
impl ReceiptMode {
pub fn name(self) -> &'static str {
match self {
Self::Posted => "Posted",
Self::Existing => "Existing",
}
}
}
impl Serialize for ReceiptMode {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str(self.name())
}
}
impl<'de> Deserialize<'de> for ReceiptMode {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let name = String::deserialize(deserializer)?;
[Self::Posted, Self::Existing].into_iter().find(|mode| mode.name() == name).ok_or_else(|| serde::de::Error::custom("the mouth named a receipt mode this side does not know"))
}
}
/// The mouth's answer to a fetch: `[request_id, outcome, room_id, image, receipt]`. `room_id` and `image` are present exactly for
/// `Stored` and `StoredUnverified`; `receipt` only with `Stored` (`Stored` with none is valid: nothing was posted, so it is not
/// marked read). A receipt that says otherwise does not decode.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(try_from = "RawFetchReceipt")]
pub struct ChatFetchReceipt {
pub request_id: String,
pub outcome: FetchOutcome,
pub room_id: Option<String>,
pub image: Option<ChatImage>,
pub receipt: Option<ReceiptMode>,
}
#[derive(Deserialize)]
struct RawFetchReceipt {
request_id: String,
outcome: FetchOutcome,
room_id: Option<String>,
image: Option<ChatImage>,
receipt: Option<ReceiptMode>,
}
impl TryFrom<RawFetchReceipt> for ChatFetchReceipt {
type Error = &'static str;
fn try_from(raw: RawFetchReceipt) -> Result<Self, Self::Error> {
if raw.outcome.stored() != (raw.room_id.is_some() && raw.image.is_some()) || (!raw.outcome.stored() && (raw.room_id.is_some() || raw.image.is_some())) {
return Err("the fetch receipt's room and image do not match its outcome");
}
if raw.receipt.is_some() && raw.outcome != FetchOutcome::Stored {
return Err("the fetch receipt posts a receipt for an outcome that is not Stored");
}
Ok(Self { request_id: raw.request_id, outcome: raw.outcome, room_id: raw.room_id, image: raw.image, receipt: raw.receipt })
}
}
/// A document as the mouth reads it: a positional tuple.
pub fn encode<T: Serialize>(document: &T) -> Result<Vec<u8>> {
rmp_serde::to_vec(document).context("the chat document does not encode")
@ -706,6 +869,9 @@ pub trait Chat {
fn send(&self, request: &ChatSendRequest) -> Result<ChatSendReceipt>;
/// Ask the mouth to open a DM. A refusal is a receipt; only a transport fault or a mouth that does not serve the kind is an error.
fn open_direct(&self, request: &ChatOpenDirectRequest) -> Result<ChatOpenDirectReceipt>;
/// Ask the mouth to fetch, verify and receipt one event's image. A refusal is a receipt; only a transport fault or a mouth
/// that does not serve the kind is an error.
fn fetch(&self, request: &ChatFetchRequest) -> Result<ChatFetchReceipt>;
/// Every delivery up to and including `through_seq` is answered; the mouth
/// advances its sync position only now, so a start that dies first has its
/// deliveries again.

View file

@ -145,6 +145,22 @@ fn a_basis_or_form_this_side_does_not_know_does_not_decode() {
assert!(decode::<ChatPending>(&rmp_serde::to_vec(&value).unwrap()).is_err());
}
#[test]
fn the_follow_up_basis_is_one_wire_string_that_round_trips_and_an_unknown_one_still_fails() {
use crate::chat_client::{Basis, FOLLOW_UP_BASIS};
let mut value: serde_json::Value = rmp_serde::from_slice(PENDING_TAG_BYTES).unwrap();
value[1][0][12] = FOLLOW_UP_BASIS.into();
let pending = decode::<ChatPending>(&rmp_serde::to_vec(&value).unwrap()).unwrap();
assert_eq!(pending.deliveries[0].basis, Some(Basis::FollowUp));
assert_eq!(Basis::FollowUp.name(), FOLLOW_UP_BASIS);
assert_eq!(Basis::FollowUp.name(), "follow", "the mouth's literal word (Bifrost PR #74), not the constant");
value[1][0][12] = "follow".into();
assert_eq!(decode::<ChatPending>(&rmp_serde::to_vec(&value).unwrap()).unwrap().deliveries[0].basis, Some(Basis::FollowUp));
assert_eq!(rmp_serde::from_slice::<String>(&rmp_serde::to_vec(&Basis::FollowUp).unwrap()).unwrap(), FOLLOW_UP_BASIS);
value[1][0][12] = "follow-up".into();
assert!(decode::<ChatPending>(&rmp_serde::to_vec(&value).unwrap()).is_err());
}
/// Reads the tuple's shape from the raw bytes, independent of the serde types under test.
mod shape {
#[derive(Debug, PartialEq, Eq)]
@ -185,6 +201,74 @@ fn open_direct_fixtures_round_trip() {
golden(UNKNOWN_KIND_BYTES, &ChatRefusal { persona: "eureka".into(), refusal: Refusal::UnknownKind });
}
const FETCH_REQUEST_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_request.v1.msgpack");
const FETCH_POSTED_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_receipt.v1.stored-posted.msgpack");
const FETCH_EXISTING_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_receipt.v1.stored-existing.msgpack");
const FETCH_NIL_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_receipt.v1.stored-nil.msgpack");
const FETCH_UNVERIFIED_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_receipt.v1.stored-unverified.msgpack");
const FETCH_REFUSED_BYTES: &[u8] = include_bytes!("../../tests/fixtures/persona-chat-v9/gamecult.persona.chat_fetch_receipt.v1.refused.msgpack");
fn fetch_image() -> ChatImage {
let hash = "ab".repeat(32);
ChatImage { path: format!("{}/{hash}.png", "cd".repeat(32)), mime: "image/png".into(), size: 2048, sha256: hash }
}
fn fetch_receipt(outcome: FetchOutcome, stored: bool, receipt: Option<ReceiptMode>) -> ChatFetchReceipt {
ChatFetchReceipt { request_id: "req-f1".into(), outcome, room_id: stored.then(|| "!abc:gamecult.org".to_string()), image: stored.then(fetch_image), receipt }
}
#[test]
fn mouth_fixtures_roundtrip() {
golden(FETCH_REQUEST_BYTES, &ChatFetchRequest { request_id: "req-f1".into(), persona: "eureka".into(), room: "#swarm:gamecult.org".into(), event_id: "$file1".into() });
golden(FETCH_POSTED_BYTES, &fetch_receipt(FetchOutcome::Stored, true, Some(ReceiptMode::Posted)));
golden(FETCH_EXISTING_BYTES, &fetch_receipt(FetchOutcome::Stored, true, Some(ReceiptMode::Existing)));
golden(FETCH_NIL_BYTES, &fetch_receipt(FetchOutcome::Stored, true, None));
golden(FETCH_UNVERIFIED_BYTES, &fetch_receipt(FetchOutcome::StoredUnverified, true, None));
golden(FETCH_REFUSED_BYTES, &ChatFetchReceipt { request_id: "req-f1".into(), outcome: FetchOutcome::BridgedRoom, room_id: None, image: None, receipt: None });
// The receipt is a 5-slot tuple whose image is its own 4-slot tuple.
assert_eq!(FETCH_POSTED_BYTES[0], 0x95);
}
#[test]
fn the_receipt_checker_follows_the_one_table() {
use FetchOutcome::*;
let decodes = |receipt: &ChatFetchReceipt| {
let wire = (&receipt.request_id, receipt.outcome, &receipt.room_id, &receipt.image, receipt.receipt);
decode::<ChatFetchReceipt>(&encode(&wire).unwrap()).is_ok()
};
for mode in [None, Some(ReceiptMode::Posted), Some(ReceiptMode::Existing)] {
assert!(decodes(&fetch_receipt(Stored, true, mode)), "Stored with {mode:?}");
}
assert!(decodes(&fetch_receipt(StoredUnverified, true, None)));
assert!(!decodes(&fetch_receipt(StoredUnverified, true, Some(ReceiptMode::Posted))), "an unverified fetch posts no receipt");
assert!(!decodes(&fetch_receipt(StoredUnverified, true, Some(ReceiptMode::Existing))));
for outcome in FetchOutcome::ALL {
let stored = outcome.stored();
assert!(!decodes(&fetch_receipt(outcome, !stored, None)), "{outcome:?} with the other half's ids");
assert!(decodes(&fetch_receipt(outcome, stored, None)), "{outcome:?}");
if outcome != Stored {
assert!(!decodes(&fetch_receipt(outcome, stored, Some(ReceiptMode::Posted))), "{outcome:?} with a receipt");
}
let mut half = fetch_receipt(outcome, stored, None);
half.room_id = (!stored).then(|| "!r:x".to_string());
assert!(!decodes(&half), "{outcome:?} with only a room id");
}
}
#[test]
fn every_fetch_outcome_is_spelled_as_the_mouth_spells_it_and_no_other_decodes() {
let words = ["Stored", "StoredUnverified", "NotAMember", "BridgedRoom", "TooLarge", "TypeRefused", "NoAttachment", "HashMismatch", "RateLimited", "Brake", "PersonaMismatch", "UnknownPersona", "FetchFailed", "StoreFailed"];
assert_eq!(FetchOutcome::ALL.map(FetchOutcome::name), words);
for word in words {
assert_eq!(decode::<FetchOutcome>(&encode(&word).unwrap()).unwrap().name(), word);
}
// The organ-local words are never on the wire.
for local in ["NoStore", "Unserved", "Unanswered", "stored", ""] {
assert!(decode::<FetchOutcome>(&encode(&local).unwrap()).is_err(), "{local}");
}
assert!(decode::<ReceiptMode>(&encode(&"Skipped").unwrap()).is_err());
}
#[test]
fn every_outcome_is_spelled_as_the_mouth_spells_it() {
let all = [

View file

@ -65,7 +65,12 @@ impl Skip {
/// Loads `image`, which was delivered for `room_id`, from `store`.
pub(crate) fn load(store: Option<&Path>, room_id: &str, image: &ChatImage) -> Result<Image, Skip> {
let store = store.ok_or(Skip::NoStore)?;
let (bytes, media_type) = load_bytes(store.ok_or(Skip::NoStore)?, room_id, image)?;
Ok(Image { media_type, base64: STANDARD.encode(&bytes) })
}
/// The verified bytes of `image` and the media type they sniff as: everything `load` checks, before it encodes. The one verifier.
pub(crate) fn load_bytes(store: &Path, room_id: &str, image: &ChatImage) -> Result<(Vec<u8>, &'static str), Skip> {
if image.size > MAX_BYTES {
return Err(Skip::TooLarge);
}
@ -108,7 +113,7 @@ pub(crate) fn load(store: Option<&Path>, room_id: &str, image: &ChatImage) -> Re
if image.mime != media_type {
return Err(Skip::MimeMismatch);
}
Ok(Image { media_type, base64: STANDARD.encode(&bytes) })
Ok((bytes, media_type))
}
/// At most `MAX_BYTES + 1` bytes of `path`: a file that grew since the length check reads as another size than the

View file

@ -30,7 +30,7 @@ use serde::de::DeserializeOwned;
use sha2::{Digest, Sha256};
use crate::chat_client::{
ACK, Chat, ChatAck, ChatDelivery, ChatOpenDirectReceipt, ChatOpenDirectRequest, ChatPending, ChatPendingRequest, ChatRead, ChatRefusal, MouthRefused, Refusal, reply_warnings, ChatSendReceipt, ChatSendRequest, PENDING, PENDING_REQUEST, PENDING_REQUEST_V1, PENDING_REQUEST_V4, PENDING_V2, PENDING_V4, OPEN_DIRECT_RECEIPT, OPEN_DIRECT_REQUEST, PENDING_V1, READ, REFUSAL, SEND_RECEIPT,
ACK, Chat, ChatAck, ChatFetchReceipt, ChatFetchRequest, ChatOpenDirectReceipt, ChatOpenDirectRequest, ChatPending, ChatPendingRequest, ChatRead, ChatRefusal, MouthRefused, Refusal, reply_warnings, ChatSendReceipt, ChatSendRequest, PENDING, PENDING_REQUEST, PENDING_REQUEST_V1, PENDING_REQUEST_V4, PENDING_V2, PENDING_V4, OPEN_DIRECT_RECEIPT, OPEN_DIRECT_REQUEST, FETCH_RECEIPT, FETCH_REQUEST, PENDING_V1, READ, REFUSAL, SEND_RECEIPT,
SEND_REQUEST, decode, encode,
};
@ -229,6 +229,25 @@ impl Chat for RudpChat {
Ok(receipt)
}
fn fetch(&self, request: &ChatFetchRequest) -> Result<ChatFetchReceipt> {
// As an open: a mouth that does not serve the kind answers a refusal keyed by the persona, and the fetch is known not sent.
let p = self.persona.as_str();
let answers = [(FETCH_RECEIPT, request.request_id.as_str()), (REFUSAL, p)];
let (kind, payload) = self.exchange(FETCH_REQUEST, &request.request_id, request, Some(&answers))?.ok_or_else(|| anyhow!("no answer payload"))?;
if kind == REFUSAL {
let refusal: ChatRefusal = decode(&payload)?;
if refusal.persona != self.persona || refusal.refusal != Refusal::UnknownKind {
bail!("the mouth refused the fetch: {}", refusal.refusal.name());
}
return Err(MouthRefused(refusal.refusal).into());
}
let receipt: ChatFetchReceipt = decode(&payload)?;
if receipt.request_id != request.request_id {
bail!("the mouth answered another fetch");
}
Ok(receipt)
}
fn ack(&self, through_seq: u64) -> Result<()> {
let ack = ChatAck { persona: self.persona.clone(), through_seq };
self.exchange(ACK, &format!("{}:{through_seq}", self.persona), &ack, None).map(|_| ())

View file

@ -45,6 +45,13 @@ const USAGE: &str =
/// `rudp://<ip>:<port>` is refused with `usage` or a fixed message, and no
/// value is echoed.
pub fn instance_and_endpoint(args: &[String], env_endpoint: Option<String>, usage: &str) -> Result<(Slug, SocketAddr)> {
let (instance, endpoint) = instance_and_maybe_endpoint(args, env_endpoint, usage)?;
Ok((instance, endpoint.ok_or_else(|| anyhow!("no daemon endpoint: --endpoint or HUGINN_ENDPOINT"))?))
}
/// `instance_and_endpoint` for a binary that may run without a daemon (`mind-persona project` over a state file): no
/// endpoint given is none, and one given is still checked.
pub fn instance_and_maybe_endpoint(args: &[String], env_endpoint: Option<String>, usage: &str) -> Result<(Slug, Option<SocketAddr>)> {
let (mut instance, mut endpoint) = (None, None);
let mut rest = args.iter();
while let Some(flag) = rest.next() {
@ -60,12 +67,12 @@ pub fn instance_and_endpoint(args: &[String], env_endpoint: Option<String>, usag
}
let instance = Slug(instance.ok_or_else(|| anyhow!("{usage}"))?);
instance.validate_slug().map_err(|_| anyhow!("the instance is not a slug"))?;
let endpoint = endpoint.or(env_endpoint).ok_or_else(|| anyhow!("no daemon endpoint: --endpoint or HUGINN_ENDPOINT"))?;
let Some(endpoint) = endpoint.or(env_endpoint) else { return Ok((instance, None)) };
let address = endpoint
.strip_prefix("rudp://")
.and_then(|address| address.parse::<SocketAddr>().ok())
.ok_or_else(|| anyhow!("the endpoint is not rudp://<ip>:<port>"))?;
Ok((instance, address))
Ok((instance, Some(address)))
}
/// Who made the change: `SUDO_USER`, else `USER`, else `unknown`. An empty

View file

@ -1,8 +1,8 @@
//! The operator's brakes, and the one reader of them.
//!
//! `Released` carries a `Grant`, the only value that opens a launch: it cannot
//! be built outside this module, so a launch exists only as the reading of a
//! released instance brake and a released swarm brake. The Grant names the
//! `Released` carries a `Grant`: it cannot be built outside this module, so it
//! exists only as the reading of a released instance brake and a released
//! swarm brake. The Grant names the
//! instance whose store was read and carries nothing else: no money, no cap. The store is at a fixed place, `CONTROL_ROOT`,
//! and the one reader takes the instance only: no function, argument, variable
//! or flag lets a caller choose a root, a path or a source, so a Grant is the
@ -36,7 +36,7 @@
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use anyhow::{Context, Result, bail};
use chrono::{DateTime, Utc};
use cultcache_rs::{CultCache, CultCacheEnvelope, DatabaseEntry, SingleFileMessagePackBackingStore};
use eureka_pipeline::Slug;
@ -92,9 +92,7 @@ pub enum Effective {
/// The proof that the instance brake and the swarm brake were released when
/// read, in the root-owned stores of `instance`. Its fields are private and only
/// `read_effective` makes one; `open_and_launch` takes it by value and refuses
/// one read for another instance, so one reading opens at most one run, of the
/// instance it was read for.
/// `read_effective` makes one.
///
/// Outside this crate the type system leaves no way to make one: the fields are
/// private, there is no public constructor, and the derives give no path (no
@ -113,7 +111,7 @@ pub enum Effective {
/// ```
///
/// ```compile_fail,E0599
/// // not Clone, so a reading cannot be copied into a second run
/// // not Clone, so a reading cannot be copied
/// fn copy(grant: &mind_body::control::Grant) -> mind_body::control::Grant {
/// grant.clone()
/// }
@ -140,13 +138,6 @@ impl Grant {
pub fn instance(&self) -> &Slug {
&self.instance
}
/// A Grant for a launch test that has no store. It names its instance like
/// any other, so the launch's instance check judges it.
#[cfg(test)]
pub(crate) fn for_test(instance: Slug) -> Self {
Self { instance }
}
}
/// The instance store's contents, decoded and not yet judged.
@ -237,11 +228,6 @@ pub fn load_state(source: &impl ControlSource) -> Result<ControlState> {
/// let _ = mind_body::control::read_effective_from;
/// ```
///
/// ```compile_fail
/// // and the test constructor is not in the library build
/// let _ = mind_body::control::Grant::for_test;
/// ```
///
/// The same preamble with the real call compiles:
///
/// ```
@ -414,6 +400,10 @@ impl ControlWriter {
let mut cache = registered_cache()?;
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path))?;
cache.pull_all_backing_stores()?;
// A pull carries a record no registered type claims instead of refusing it; the writer still must not write beside one.
if !cache.foreign_records().is_empty() {
bail!("{} holds a record this control store does not read: archive it, nothing migrates it", path.display());
}
Ok(Self { cache, path })
}
@ -502,6 +492,10 @@ impl SwarmWriter {
let mut cache = swarm_cache()?;
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path))?;
cache.pull_all_backing_stores()?;
// A pull carries a record no registered type claims instead of refusing it; the writer still must not write beside one.
if !cache.foreign_records().is_empty() {
bail!("{} holds a record this swarm store does not read: archive it, nothing migrates it", path.display());
}
Ok(Self { cache, path })
}

View file

@ -62,100 +62,11 @@ fn a_brake_payload_that_does_not_decode_makes_the_store_undecodable() {
let mut cache = CultCache::new();
cache.register_entry_type::<WrongBrake>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache.put(BRAKE_KEY, &WrongBrake { released: "yes".into() }).unwrap();
assert_eq!(effective(&path), held(HeldReason::Undecodable));
}
mod grant {
//! A Grant is the proof that this instance's own control store released: the
//! reader derives the path from the instance, and a launch refuses a Grant
//! read for another instance before it asks the mind anything.
use anyhow::{Result, anyhow};
use cultnet_rs::Selection;
use eureka_pipeline::{PipelineDocument, RunTurn};
use huginn_mind::{PipelineAdmissionOutcome, PipelineSelectionPage};
use super::*;
use crate::launch::{Clock, Declined, LaunchOutcome, LaunchRequest, Launcher, Ports, open_and_launch};
use crate::queue::MindPort;
fn slug(name: &str) -> Slug {
Slug(name.into())
}
fn read(root: &Path, instance: &str) -> Effective {
with_root(root, || read_effective(&slug(instance)))
}
/// A mind that answers nothing: a launch that reaches it has gone past the
/// instance check.
struct SilentMind(Slug);
impl MindPort for SilentMind {
fn instance(&self) -> &Slug {
&self.0
}
fn query(&self, _selection: &Selection) -> Result<PipelineSelectionPage> {
Err(anyhow!("the launch asked the mind"))
}
fn admit(&self, _agent: &str, _session: &str, _documents: Vec<PipelineDocument>) -> Result<PipelineAdmissionOutcome> {
Err(anyhow!("the launch admitted"))
}
}
struct NoUnits;
impl Launcher for NoUnits {
fn start(&self, _instance: &Slug, _turn: RunTurn) -> Result<()> {
Err(anyhow!("the launch started a unit"))
}
fn alive(&self, _instance: &Slug, _turn: RunTurn) -> bool {
false
}
}
struct Now;
impl Clock for Now {
fn now(&self) -> DateTime<Utc> {
at(100)
}
}
fn launch_as(mind_instance: &str, grant: Grant, turn: RunTurn) -> Result<LaunchOutcome> {
let mind = SilentMind(slug(mind_instance));
let ports = Ports { mind: &mind, launcher: &NoUnits, clock: &Now, host: "host" };
open_and_launch(&ports, LaunchRequest { turn, claims: vec![], agent: "agent".into(), grant })
}
#[test]
fn a_grant_read_for_one_instance_cannot_launch_another() {
let root = tempfile::tempdir().unwrap();
release(root.path(), "void");
for turn in [RunTurn::SelfRun, RunTurn::PersonaTurn] {
let Effective::Released(grant) = read(root.path(), "void") else { panic!("void's store is released") };
// The mind is eureka's; the silent mind and the unit-less launcher fail loudly if the launch goes on.
let outcome = launch_as("eureka", grant, turn).unwrap();
assert_eq!(outcome, LaunchOutcome::Refused(Declined::GrantForOtherInstance), "{turn:?}");
}
}
#[test]
fn the_matching_grant_is_not_stopped_by_the_instance_check() {
let root = tempfile::tempdir().unwrap();
release(root.path(), "void");
let Effective::Released(grant) = read(root.path(), "void") else { panic!("void's store is released") };
let error = launch_as("void", grant, RunTurn::SelfRun).unwrap_err();
assert!(format!("{error:#}").contains("asked the mind"), "{error:#}");
}
}
/// The swarm store with its brake released, and nothing else in it.
fn release_swarm(root: &Path) {
SwarmWriter::open(swarm_path(root)).unwrap().set_brake(true, at(0), "op").unwrap();
@ -249,8 +160,33 @@ fn the_stores_are_written_world_readable_and_root_only_writable() {
let _ = path;
}
/// A named instance store with its brake released, and a released swarm.
fn release(root: &Path, name: &str) {
ControlWriter::open(root.join(name).join("control.cc")).unwrap().set_brake(true, at(0), "op").unwrap();
release_swarm(root);
#[test]
fn a_writer_refuses_to_open_over_a_store_holding_a_record_it_does_not_read() {
let dir = tempfile::tempdir().unwrap();
let path = store_path(dir.path());
std::fs::create_dir_all(path.parent().unwrap()).unwrap();
let mut cache = CultCache::new();
cache.register_entry_type::<OldDial>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache.put("dial", &OldDial { heat: "1".into() }).unwrap();
let before = std::fs::read(&path).unwrap();
let refusal = ControlWriter::open(&path).err().expect("a foreign record refuses the open").to_string();
assert!(refusal.contains("does not read"), "{refusal}");
assert_eq!(std::fs::read(&path).unwrap(), before, "the refused open leaves the store bytes alone");
}
#[test]
fn a_swarm_writer_refuses_to_open_over_a_store_holding_a_record_it_does_not_read() {
let dir = tempfile::tempdir().unwrap();
let path = swarm_path(dir.path());
let mut cache = CultCache::new();
cache.register_entry_type::<OldDial>().unwrap();
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
cache.pull_all_backing_stores().unwrap();
cache.put("dial", &OldDial { heat: "1".into() }).unwrap();
let before = std::fs::read(&path).unwrap();
let refusal = SwarmWriter::open(&path).err().expect("a foreign record refuses the open").to_string();
assert!(refusal.contains("does not read"), "{refusal}");
assert_eq!(std::fs::read(&path).unwrap(), before, "the refused open leaves the store bytes alone");
}

View file

@ -1,60 +1,49 @@
//! The heartbeat: the one process that decides which Persona takes a turn and
//! when (rulings `eureka-body:ruling:shared-heartbeat-joint-lever`,
//! `self-declared-urgency-etiquette`, `operator-mention-fills-initiative`).
//! The heartbeat: the one process that decides which Persona wakes and when
//! (rulings `eureka-body:ruling:persona-only-heartbeat`,
//! `operator-mention-fills-initiative`).
//!
//! The Minds share one heartbeat and jointly hold the lever to it. Its tempo is
//! `tempo::tally` of every Mind's newest vote, a `Pulse` in her own mind. When a
//! beat falls (the beat has passed since the last one) exactly one Persona takes
//! a turn: `tempo::pick` over the idle, released Minds, by how long each has
//! waited, bent by the urgency she declared for herself, a member addressing her
//! and (a later cut) topics she subscribed to. Two things launch a Persona off
//! the beat: the operator tagging or naming her, once per message, and her own
//! Self run having ended since her last turn. A beat's launch for a Mind whose
//! Self holds briefs (`inbox::derive`) is a Self run claiming them, in place of
//! her Persona turn: a session that ended with a brief open, or a brief admitted
//! after its flush, is never an orphan, and a Self start takes the beat like any
//! launch. A brief an open question holds (raised in it, or in a run that claims it:
//! the repetition breaker's trip holds the claims of the run it belongs to) is
//! neither claimed nor told until that question is answered or withdrawn; other
//! briefs are. Nothing is metered here:
//! no meter and no model call.
//! The beat is the configured `Config::beat_s`. When a beat falls (the beat has
//! passed since the last one) exactly one Persona wakes: `tempo::pick` over the
//! idle, released Minds, by how long each has waited since the heartbeat last
//! woke her, bent by a member addressing her and (a later cut) topics she
//! subscribed to. The operator or another Mind tagging or naming her wakes her
//! off the beat, once per message. There is no night wake until reflection and
//! consolidation return (map C4). Nothing is metered here: no meter and no model call.
//!
//! The heartbeat owns `heartbeat.cc` and is its only writer; Persona units read
//! it. The brakes stay the operator's and are read on every poll. Busy and idle
//! are the slot's: a Persona run whose unit lives (or is within its start grace) is busy; a dead holder is not, and the next launch closes it. The launch is
//! `open_and_launch`, the only opener. The heartbeat never acks a delivery; the
//! Persona turn drains them.
//! A wake starts her unit and nothing else: the heartbeat writes no mind (ruling
//! `persona-only-heartbeat`: her Mind is written only by Self sessions). It reads
//! her mind only for the names her projects go by. Busy is systemd's own: a
//! Persona whose unit is active is busy, and a start of an active unit is a no-op.
//!
//! The heartbeat owns `heartbeat.cc` and is its only writer. The brakes stay the
//! operator's and are read on every poll. The heartbeat never acks a delivery.
//!
//! A held swarm brake ends the poll before the mind or the mouth is touched.
use std::collections::BTreeMap;
use std::fmt;
use std::collections::HashSet;
use std::net::SocketAddr;
use std::path::PathBuf;
use anyhow::{Context, Result, anyhow, bail};
use chrono::{DateTime, SecondsFormat, Timelike, Utc};
use chrono::{DateTime, SecondsFormat, Utc};
use cultcache_rs::{CultCache, DatabaseEntry, SingleFileMessagePackBackingStore};
use cultnet_rs::Selection;
use eureka_pipeline::{OrgRepo, PipelineKind, PipelineRef, ResolutionOutcome, RunOperator, RunTurn, Slug};
use huginn_mind::{PipelineDocumentSummary, PipelineFacts, PipelinePageItems};
use eureka_pipeline::{OrgRepo, PipelineKind, RunTurn, Slug};
use huginn_mind::PipelineFacts;
use crate::address::{Addressee, Name, is_for, project_names};
use crate::chat_client::{Chat, ChatDelivery};
use crate::chat_client::{Basis, Chat, ChatDelivery};
use crate::control::{ControlSource, Effective, FileSource, HeldReason, SwarmEffective, read_effective, read_envelope, set_mode_world_readable};
use crate::launch::{Clock, LaunchOutcome, LaunchRequest, Launcher, Ports, open_and_launch, slot};
use crate::queue::{MindPort, any_of, citing, claims_of, headers, in_force, of_kind, on_hold};
use crate::self_run::inbox;
use crate::launch::{Clock, Launcher};
use crate::queue::{MindPort, headers, in_force, of_kind};
use crate::tempo::{self, Standing};
/// The admitting agent's name on every run the heartbeat opens.
pub const AGENT: &str = "mind-heartbeat";
/// How many handled operator mentions the state remembers.
const MENTIONS_KEPT: usize = 64;
/// How many of an instance's latest Self runs a poll looks at.
const RECENT: u32 = 8;
/// The most pages of pending chat one poll reads for one Persona.
const PENDING_PAGES: usize = 16;
/// The beat, in seconds, when `HEARTBEAT_BEAT_S` is not set.
pub const DEFAULT_BEAT_S: u32 = 1800;
/// Who a Persona is, for the two things the heartbeat matches on: the Matrix
/// user id its tags carry, and the names a message may call it by. Each Mind is
@ -104,71 +93,23 @@ pub struct Config {
pub personas: Vec<PersonaId>,
/// Sender ids that are the operator.
pub operators: Vec<String>,
/// The nightly window and the Minds that sleep in it; none, no one sleeps.
pub night: Option<Night>,
}
/// The heartbeat's night: a daily window in UTC minutes (it may wrap midnight) and the Minds that sleep in it.
/// The heartbeat alone decides who sleeps and when: one Sleep run per Mind per window, never made up if the window
/// passes without one.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Night {
pub from_min: u32,
pub to_min: u32,
pub minds: Vec<Slug>,
}
impl Night {
/// The nightly sleep from the two settings that name it: both or neither, so a half-set deploy fails at start
/// instead of quietly never sleeping, or sleeping no one.
pub fn from_settings(window: Option<&str>, minds: Option<&str>) -> Result<Option<Self>> {
match (window, minds) {
(Some(window), Some(minds)) => Self::parse(window, minds).map(Some),
(None, None) => Ok(None),
_ => bail!("HEARTBEAT_SLEEP_UTC and HEARTBEAT_SLEEP_MINDS are set together or not at all"),
}
}
/// `window` is `HH:MM-HH:MM` in UTC, `minds` the slugs comma-separated.
pub fn parse(window: &str, minds: &str) -> Result<Self> {
let minute = |text: &str| -> Result<u32> {
let (hour, minute) = text.trim().split_once(':').ok_or_else(|| anyhow!("the night window is HH:MM-HH:MM"))?;
let (hour, minute) = (hour.parse::<u32>()?, minute.parse::<u32>()?);
if hour > 23 || minute > 59 {
bail!("the night window is HH:MM-HH:MM");
}
Ok(hour * 60 + minute)
};
let (from, to) = window.split_once('-').ok_or_else(|| anyhow!("the night window is HH:MM-HH:MM"))?;
let (from_min, to_min) = (minute(from)?, minute(to)?);
if from_min == to_min {
bail!("the night window is empty");
}
let minds: Vec<Slug> = minds.split(',').map(|slug| Slug(slug.trim().to_string())).filter(|slug| !slug.0.is_empty()).collect();
if minds.is_empty() {
bail!("no Mind is listed to sleep");
}
Ok(Self { from_min, to_min, minds })
}
/// When the window `now` is inside opened, or none outside it.
fn opened(&self, now: DateTime<Utc>) -> Option<DateTime<Utc>> {
let minute = now.hour() * 60 + now.minute();
let inside = if self.from_min < self.to_min { (self.from_min..self.to_min).contains(&minute) } else { minute >= self.from_min || minute < self.to_min };
if !inside {
return None;
}
let day = if minute >= self.from_min { now.date_naive() } else { now.date_naive().pred_opt()? };
Some(day.and_hms_opt(self.from_min / 60, self.from_min % 60, 0)?.and_utc())
}
/// Seconds between beats: `HEARTBEAT_BEAT_S`, else `DEFAULT_BEAT_S`.
pub beat_s: u32,
}
impl Config {
/// Refuses no Persona, or a Persona twice.
pub fn new(personas: Vec<PersonaId>, operators: Vec<String>, night: Option<Night>) -> Result<Self> {
/// Refuses no Persona, a Persona twice, a beat of zero seconds, or an operator id that is also a Persona's: one
/// sender is either the operator or a Mind, never both, or the wake rule could not tell which it heard.
pub fn new(personas: Vec<PersonaId>, operators: Vec<String>, beat_s: u32) -> Result<Self> {
if personas.is_empty() {
bail!("no persona is configured");
}
if operators.iter().any(|operator| personas.iter().any(|persona| persona.matrix_id == *operator)) {
bail!("an operator id is also a persona's matrix id");
}
if beat_s == 0 {
bail!("the beat is zero seconds");
}
for (index, persona) in personas.iter().enumerate() {
if personas
.iter()
@ -181,22 +122,25 @@ impl Config {
Ok(Self {
personas,
operators,
night,
beat_s,
})
}
}
/// `heartbeat.cc`'s state: when the last beat fell, and the operator messages
/// already answered with a launch (once per message, bounded). A later cut adds
/// the cache of scored deliveries beside them.
/// `heartbeat.cc`'s state: when the last beat fell, the operator messages
/// already answered with a wake (once per message, bounded), and when the
/// heartbeat last woke each Persona.
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
#[cultcache(type = "eureka.heartbeat.state.v1", schema = "HeartbeatState")]
pub struct HeartbeatState {
#[cultcache(key = 0)]
pub last_beat_at: DateTime<Utc>,
/// `<instance>:<event id>` of each operator message that launched a Persona.
/// `<instance>:<event id>` of each operator message that woke a Persona.
#[cultcache(key = 1)]
pub handled_mentions: Vec<String>,
/// When the heartbeat last started each Persona's turn, by instance slug.
#[cultcache(key = 2, default)]
pub last_turn_at: BTreeMap<String, DateTime<Utc>>,
}
const STATE_KEY: &str = "state";
@ -217,9 +161,9 @@ const ADDRESSING_KEY: &str = "addressing";
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct InstanceTrace {
pub instance: String,
/// Her newest tempo vote, in seconds per beat.
/// Retired: always none. Kept so the records `heartbeat.cc` holds from before the cut still decode (nothing migrates the file).
pub vote_beat_s: Option<u32>,
/// The urgency in force for her: declared and not yet consumed by a turn.
/// The urgency in force for her: 1000 when a member addresses her, else 0.
pub urgency_permille: u16,
/// A member's pending message addresses her.
pub addressed: bool,
@ -238,7 +182,7 @@ pub struct InstanceTrace {
pub struct PollTrace {
#[cultcache(key = 0)]
pub at: DateTime<Utc>,
/// The beat in force: the tally of the Minds' votes.
/// The beat in force: `Config::beat_s`.
#[cultcache(key = 1)]
pub tempo_s: u32,
#[cultcache(key = 2)]
@ -253,8 +197,8 @@ impl fmt::Display for PollTrace {
for row in &self.instances {
write!(
f,
" {}: vote {:?} urgency {} addressed {} busy {} released {} idle {}s score {:?}",
row.instance, row.vote_beat_s, row.urgency_permille, row.addressed, row.busy, row.released, row.idle_s, row.score
" {}: urgency {} addressed {} busy {} released {} idle {}s score {:?}",
row.instance, row.urgency_permille, row.addressed, row.busy, row.released, row.idle_s, row.score
)?;
if !row.note.is_empty() {
write!(f, " note {}", row.note)?;
@ -277,11 +221,13 @@ pub trait Store {
fn save_addressing(&self, addressing: &Addressing) -> Result<()>;
}
/// What a Persona unit may read of `heartbeat.cc`: the newest trace, which
/// carries the tempo and every Mind's vote and declared urgency, and the swarm's addressing. Read-only.
/// What a Persona unit may read of `heartbeat.cc`: the swarm's addressing. Read-only.
pub trait HeartbeatView {
fn latest(&self) -> Result<Option<PollTrace>>;
fn addressing(&self) -> Result<Option<Addressing>>;
/// When the store was last written, so a reader can tell a stale table; none when there is no store at all.
fn written_at(&self) -> Option<DateTime<Utc>> {
None
}
}
/// The store as a CultCache single-file store under the unit's state directory.
@ -309,6 +255,10 @@ impl FileStore {
let mut cache = Self::registered()?;
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&self.path))?;
cache.pull_all_backing_stores().with_context(|| self.refusal())?;
// A pull now carries a record no registered type claims instead of refusing it; the writer still must not write beside one.
if !cache.foreign_records().is_empty() {
return Err(anyhow!(self.refusal()));
}
Ok(cache)
}
@ -352,6 +302,13 @@ impl Store for FileStore {
}
impl FileStore {
/// The newest poll trace: what `mind-heartbeat timeline` prints.
pub fn latest(&self) -> Result<Option<PollTrace>> {
let mut all = self.snapshot()?.get_all_with_keys::<PollTrace>()?;
all.sort_by(|a, b| a.0.cmp(&b.0));
Ok(all.pop().map(|(_, trace)| trace))
}
/// The file read without a lock and without a write, as the control stores are.
fn snapshot(&self) -> Result<CultCache> {
let mut cache = Self::registered()?;
@ -363,15 +320,13 @@ impl FileStore {
}
impl HeartbeatView for FileStore {
fn latest(&self) -> Result<Option<PollTrace>> {
let mut all = self.snapshot()?.get_all_with_keys::<PollTrace>()?;
all.sort_by(|a, b| a.0.cmp(&b.0));
Ok(all.pop().map(|(_, trace)| trace))
}
fn addressing(&self) -> Result<Option<Addressing>> {
self.load_addressing()
}
fn written_at(&self) -> Option<DateTime<Utc>> {
std::fs::metadata(&self.path).and_then(|meta| meta.modified()).ok().map(DateTime::<Utc>::from)
}
}
/// What the operator stores allow: the swarm brake, and each instance's own
@ -406,12 +361,11 @@ pub struct Organs<'a> {
pub control: &'a dyn Control,
pub launcher: &'a dyn Launcher,
pub clock: &'a dyn Clock,
pub host: &'a str,
pub store: &'a dyn Store,
}
/// What one poll did. Every outcome is an answer; only a mind, store or
/// mouth that cannot be reached is an error.
/// What one poll did. Every outcome is an answer; only a store that cannot be
/// reached is an error.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Outcome {
/// The swarm brake is held; nothing else was touched.
@ -431,77 +385,6 @@ impl fmt::Display for Outcome {
}
}
fn parse_time(text: &str) -> Result<DateTime<Utc>> {
DateTime::parse_from_rfc3339(text)
.map(|at| at.with_timezone(&Utc))
.map_err(|_| anyhow!("the mind gave a time that is not a timestamp"))
}
/// When a run was last resolved and how: its resolution of the highest sequence.
fn latest_resolution(
mind: &dyn MindPort,
run: &PipelineRef,
) -> Result<Option<(DateTime<Utc>, ResolutionOutcome)>> {
let closures = Selection {
cites: Some(citing(run, Some("subject"))),
..of_kind(PipelineKind::Resolution)
};
let mut ending: Option<(u32, DateTime<Utc>, ResolutionOutcome)> = None;
for header in headers(mind, &closures)? {
if let PipelineFacts::Resolution {
sequence, outcome, ..
} = header.facts
&& ending
.as_ref()
.is_none_or(|(latest, ..)| sequence >= *latest)
{
ending = Some((
sequence,
parse_time(&header.admission.admitted_at)?,
outcome,
));
}
}
Ok(ending.map(|(_, at, outcome)| (at, outcome)))
}
/// When her newest Persona turn that actually ran was admitted: one bounded read
/// of the runs the mind indexes as `ran`. A turn withdrawn unrun never happened,
/// so it neither resets her wait nor counts as her last turn, and the cost of
/// finding the last real turn does not grow with how many were withdrawn.
fn last_turn(mind: &dyn MindPort, instance: &Slug) -> Result<Option<PipelineDocumentSummary>> {
let turn = format!("{:?}", RunTurn::PersonaTurn);
let operator = format!("{:?}", RunOperator::Mind);
let runs = any_of(
any_of(
any_of(
any_of(of_kind(PipelineKind::Run), "root", &[&instance.0]),
"turn",
&[&turn],
),
"operated_by",
&[&operator],
),
"ran",
&["true"],
);
newest(mind, runs)
}
/// When her newest Sleep run was admitted, whatever became of it: a night that failed still counts.
fn last_sleep(mind: &dyn MindPort, instance: &Slug) -> Result<Option<DateTime<Utc>>> {
let operator = format!("{:?}", RunOperator::Mind);
let runs = any_of(any_of(any_of(of_kind(PipelineKind::Run), "root", &[&instance.0]), "turn", &["Sleep"]), "operated_by", &[&operator]);
newest(mind, runs)?.map(|header| parse_time(&header.admission.admitted_at)).transpose()
}
/// The newest document a selection holds, by admission ordinal: one query.
fn newest(mind: &dyn MindPort, selection: Selection) -> Result<Option<PipelineDocumentSummary>> {
let asking = Selection { descending: true, limit: Some(1), ..selection };
let PipelinePageItems::Headers(found) = mind.query(&asking)?.items else { bail!("the mind answered documents to a header selection") };
Ok(found.into_iter().next())
}
/// Every repo she stewards in force and the chat names of those repos: one query on her mind.
pub(crate) fn stewardships_of(mind: &dyn MindPort) -> Result<(Vec<Name>, Vec<OrgRepo>)> {
let (mut names, mut repos) = (Vec::new(), Vec::new());
@ -514,12 +397,125 @@ pub(crate) fn stewardships_of(mind: &dyn MindPort) -> Result<(Vec<Name>, Vec<Org
Ok((names, repos))
}
/// Whether a delivery wakes `who` off the beat: the operator or another Mind of
/// the roster addresses her. A Mind naming herself does not; a member does not.
fn wakes(config: &Config, who: &Addressee, delivery: &ChatDelivery) -> bool {
let from_operator = config.operators.contains(&delivery.sender);
let from_other_mind = delivery.sender != who.matrix_id() && config.personas.iter().any(|mind| mind.matrix_id == delivery.sender);
(from_operator || from_other_mind) && is_for(who, delivery)
/// A member (anyone not the operator) has a pending delivery that addresses her: the beat's urgency. The one owner of
/// that reading: the poll's rows and the simulator's beat both ask it, over the deliveries the mouth would give her.
pub(crate) fn addressed_by_member(operators: &[String], who: &Addressee, pending: &[ChatDelivery]) -> bool {
pending.iter().any(|delivery| !operators.contains(&delivery.sender) && is_for(who, delivery))
}
/// Who sent a line, as the wake rule sees them.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Speaker {
Operator,
/// Another Mind of the roster.
Mind,
/// A human who is not the operator: a member.
Member,
/// The Persona herself.
Her,
}
impl Speaker {
fn human(self) -> bool {
matches!(self, Self::Operator | Self::Member)
}
}
/// Which lines wake a Persona off the beat. `Live` is the heartbeat's rule today and the only one `poll` uses; the rest
/// are what desk question `po-mind-wakes-mind` weighs, run in `mind-persona project --simulate` until it is answered.
/// The beat is not here: every policy has it, and it wakes one Persona by `tempo::pick`.
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub(crate) enum WakePolicy {
/// The operator or another Mind addressing her (`is_for`). A member's line does not.
Live,
/// Ruling `persona-only-wake-humans-in-rooms`: a line addressing her, from anyone, and any human line in her rooms.
Ruled,
/// Ruling `persona-only-humans-pace`: a human's line wakes her in her own room (a direct message, or a room that is
/// hers); in a public room only when it addresses her or continues an exchange she is in. A Mind's line wakes her
/// only by an explicit mention (a Matrix mention of her or a reply to her, never a name in prose), in any room, and
/// only while fewer than this many Mind lines in a row came before it across her rooms (`LineFacts::mind_run`): the
/// backstop.
HumansPace(usize),
/// `Ruled`, with at most this many lines in her current Mind-only run (`LineFacts::mind_run`) before a human must
/// speak where it runs: a Mind's line past the cap wakes no one.
NamesWithCap(usize),
/// The swarm's old loop, a control: any line in her rooms but her own.
AllMessages,
}
/// Why a line wakes her, strongest first.
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, serde::Serialize)]
#[serde(rename_all = "snake_case")]
pub(crate) enum WakeCause {
AddressedByHuman,
Human,
AddressedByMind,
/// A Mind's line that does not address her: only the control wakes on it.
Mind,
}
/// What the room says about a line, beside the line: the facts a policy may weigh. The live poll knows none of them yet
/// and passes the default; `Live` reads none.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) struct LineFacts {
/// How many lines her current Mind-only run held before this one: one run across every room she sees, so a loop
/// cannot dodge the backstop by changing rooms, and ended only by a human's line in a room the run has lines in.
pub mind_run: usize,
/// The line is in a room of hers: a direct message (the delivery says so) or a room configured as hers.
pub own_room: bool,
/// She has spoken in the room's current exchange: the lines before this one that follow each other without a long gap.
pub in_exchange: bool,
}
/// Fail-safe: a caller that has not counted the Mind run is at the cap, so a Mind's line wakes no one; the room facts
/// are unknown, so none opens a wake.
impl Default for LineFacts {
fn default() -> Self {
Self { mind_run: usize::MAX, own_room: false, in_exchange: false }
}
}
/// Whether a line in her rooms wakes `who` off the beat under `policy`, and why. The one owner of the wake rule: the
/// heartbeat's poll asks it with `Live`, the simulator with the policy under test.
pub(crate) fn wakes(policy: WakePolicy, who: &Addressee, delivery: &ChatDelivery, speaker: Speaker, facts: LineFacts) -> Option<WakeCause> {
if speaker == Speaker::Her {
return None;
}
let addressed = is_for(who, delivery);
let human = speaker.human();
let cause = match (human, addressed) {
(true, true) => WakeCause::AddressedByHuman,
(true, false) => WakeCause::Human,
(false, true) => WakeCause::AddressedByMind,
(false, false) => WakeCause::Mind,
};
let wakes = match policy {
WakePolicy::Live => addressed && matches!(speaker, Speaker::Operator | Speaker::Mind),
WakePolicy::Ruled => human || addressed,
WakePolicy::HumansPace(cap) => {
let explicit = delivery.mentions.iter().any(|id| id == who.matrix_id()) || matches!(delivery.basis, Some(Basis::Tag | Basis::Reply));
if human { addressed || delivery.is_direct || facts.own_room || facts.in_exchange } else { explicit && facts.mind_run < cap }
}
WakePolicy::NamesWithCap(cap) => human || (addressed && facts.mind_run < cap),
WakePolicy::AllMessages => true,
};
wakes.then_some(cause)
}
impl Config {
/// Who sent a delivery, as `wakes` sees them.
fn speaker(&self, who: &Addressee, delivery: &ChatDelivery) -> Speaker {
if delivery.sender == who.matrix_id() {
Speaker::Her
} else if self.operators.contains(&delivery.sender) {
Speaker::Operator
} else if self.personas.iter().any(|mind| mind.matrix_id == delivery.sender) {
Speaker::Mind
} else {
Speaker::Member
}
}
}
/// The mouth pages a pending answer, so fewer deliveries may come than wait.
@ -542,84 +538,14 @@ fn read_pending(chat: &dyn Chat) -> Result<Vec<ChatDelivery>> {
Ok(all)
}
fn label_of(run: &eureka_pipeline::PipelineRef) -> String {
run.id.0.rsplit(':').next().unwrap_or_default().to_string()
}
/// Her standing vote: the newest pulse that votes or withdraws, by admission
/// ordinal. A vote stands until she casts another or withdraws it, however many
/// pulses of other kinds follow; a withdrawal leaves her silent.
fn standing_vote(mind: &dyn MindPort, instance: &Slug) -> Result<Option<u32>> {
let pulses = any_of(any_of(of_kind(PipelineKind::Pulse), "root", &[&instance.0]), "voted", &["true"]);
Ok(newest(mind, pulses)?.and_then(|header| match header.facts {
PipelineFacts::Pulse { tempo_vote_beat_s, tempo_vote_withdrawn: false, .. } => tempo_vote_beat_s,
_ => None,
}))
}
/// The urgency she declared for herself and no turn has consumed: her newest
/// declaration, if it was admitted after her newest Persona turn, by admission
/// ordinal like the vote: the wall clock never decides what a turn consumed.
fn pending_urgency(mind: &dyn MindPort, instance: &Slug, last_turn: Option<&PipelineDocumentSummary>) -> Result<u16> {
let pulses = any_of(any_of(of_kind(PipelineKind::Pulse), "root", &[&instance.0]), "urged", &["true"]);
let Some(header) = newest(mind, pulses)? else { return Ok(0) };
let PipelineFacts::Pulse { urgency_permille: Some(urgency), .. } = header.facts else { return Ok(0) };
Ok(if last_turn.is_none_or(|turn| header.admission.ordinal > turn.admission.ordinal) { urgency } else { 0 })
}
/// When her Self runs last ended: the latest resolution among her recent Self runs.
fn last_self_end(mind: &dyn MindPort, instance: &Slug) -> Result<Option<DateTime<Utc>>> {
let turn = format!("{:?}", RunTurn::SelfRun);
let operator = format!("{:?}", RunOperator::Mind);
let runs = any_of(any_of(any_of(of_kind(PipelineKind::Run), "root", &[&instance.0]), "turn", &[&turn]), "operated_by", &[&operator]);
let selection = Selection { descending: true, limit: Some(RECENT), ..runs };
let PipelinePageItems::Headers(runs) = mind.query(&selection)?.items else { bail!("the mind answered documents to a header selection") };
let mut newest = None;
for run in runs {
if let Some((ended, _)) = latest_resolution(mind, &run.id)? {
newest = newest.max(Some(ended));
}
}
Ok(newest)
}
/// One Persona reads from her own Mind.
struct MindRead {
busy: bool,
/// When her newest Persona turn was admitted.
last_turn: Option<DateTime<Utc>>,
/// Her standing vote for a beat.
vote: Option<u32>,
/// The urgency she declared and no turn has consumed.
urgency: u16,
self_end: Option<DateTime<Utc>>,
/// When her newest Sleep run was admitted.
last_sleep: Option<DateTime<Utc>>,
}
fn read_mind(organs: &Organs, mind: &dyn MindPort, slug: &Slug) -> Result<MindRead> {
let turn = last_turn(mind, slug)?;
Ok(MindRead {
// Busy is the slot's own reading of life: a holder whose unit is not active
// (past its start grace) is dead, not busy, so the next launch reaches
// `slot` step 0 and closes it there.
busy: slot(&ports(organs, mind), slug, RunTurn::PersonaTurn, organs.clock.now())?.live.is_some(),
last_turn: turn.as_ref().map(|header| parse_time(&header.admission.admitted_at)).transpose()?,
vote: standing_vote(mind, slug)?,
urgency: pending_urgency(mind, slug, turn.as_ref())?,
self_end: last_self_end(mind, slug)?,
last_sleep: last_sleep(mind, slug)?,
})
}
struct Row<'a> {
seat: &'a Seat<'a>,
read: Option<MindRead>,
released: bool,
/// Her unit is active.
busy: bool,
pending: Vec<ChatDelivery>,
/// Whom a delivery may address in her: her roster names and her projects' names as the last poll that read her pending chat left them.
addressee: Addressee,
busy: bool,
note: String,
}
@ -629,109 +555,35 @@ impl Row<'_> {
}
fn idle(&self) -> bool {
self.released && !self.busy && self.read.is_some()
self.released && !self.busy
}
/// A member (not the operator) has a pending message that names or tags her.
fn addressed_by_member(&self, config: &Config) -> bool {
self.pending.iter().any(|delivery| !config.operators.contains(&delivery.sender) && is_for(&self.addressee, delivery))
addressed_by_member(&config.operators, &self.addressee, &self.pending)
}
}
enum Launched {
Run(String),
enum Woke {
Started,
Busy,
Refused(String),
}
fn ports<'a>(organs: &Organs<'a>, mind: &'a dyn MindPort) -> Ports<'a> {
Ports {
mind,
launcher: organs.launcher,
clock: organs.clock,
host: organs.host,
/// The one way the heartbeat wakes a Persona: her brake read again, then her
/// unit started unless it is active. No run is opened and nothing is admitted.
fn wake(organs: &Organs, row: &Row) -> Woke {
let turn = RunTurn::PersonaTurn;
if !matches!(organs.control.effective(row.slug()), Effective::Released(_)) {
return Woke::Refused("the brake is held".into());
}
}
/// The one way the heartbeat starts a run: the same brake read and the same
/// opener for a Persona turn (a beat, an operator mention, a Self run that ended)
/// and for the Self run a beat gives to the briefs her Self holds.
fn launch(
organs: &Organs,
row: &Row,
turn: RunTurn,
claims: Vec<PipelineRef>,
) -> Result<Launched> {
let Effective::Released(grant) = organs.control.effective(row.slug()) else {
return Ok(Launched::Refused("the brake is held".into()));
};
let request = LaunchRequest {
turn,
claims,
agent: AGENT.into(),
grant,
};
let ports = ports(organs, row.seat.mind);
Ok(match open_and_launch(&ports, request) {
Ok(LaunchOutcome::Launched { run }) => Launched::Run(label_of(&run)),
Ok(LaunchOutcome::Busy { .. }) => Launched::Busy,
Ok(LaunchOutcome::Tripped { run, question }) => {
let held = claims_of(row.seat.mind, &run).map_or(0, |claims| claims.len());
Launched::Refused(format!("tripped: {} holds the {held} claims of {}", question.id.0, run.id.0))
}
Ok(_) => Launched::Refused("launch refused".into()),
Err(error) => Launched::Refused(format!("launch failed: {error}")),
})
}
/// The most briefs one Self run claims.
const BRIEFS_PER_SELF_RUN: usize = 64;
/// What a Self run would claim for this Mind, read at one instant.
struct Claimable {
/// The briefs in force that no live Self run holds and no open question holds.
claims: Vec<PipelineRef>,
/// How many briefs an open question holds.
held: usize,
/// Self runs in force whose unit is gone: the launch closes them.
dead: Vec<PipelineRef>,
}
/// The briefs a Self run would claim for this Mind: those in force that no live
/// Self run holds and no open question holds. A Self run claims up to
/// `BRIEFS_PER_SELF_RUN` briefs and every run it ends empty counts against all
/// of them, so one cause trips them together; a trip holds the claims of its
/// run, not the Mind, until the operator answers or withdraws its question, and
/// the count starts afresh. Whether a holder lives is `launch::slot`'s answer
/// (a dead one is closed by the launch, so it does not stop the successor), and
/// what is held is `queue::on_hold`'s. The read here only chooses the claims;
/// the launch's gate reads the hold again at admission and decides.
fn claimable(organs: &Organs, row: &Row) -> Result<Claimable> {
let mind = row.seat.mind;
let found = slot(
&ports(organs, mind),
row.slug(),
RunTurn::SelfRun,
organs.clock.now(),
)?;
if found.live.is_some() {
return Ok(Claimable {
claims: Vec::new(),
held: 0,
dead: Vec::new(),
});
if organs.launcher.alive(row.slug(), turn) {
return Woke::Busy;
}
match organs.launcher.start(row.slug(), turn) {
Ok(()) => Woke::Started,
Err(error) => Woke::Refused(format!("start failed: {error}")),
}
let (free, held) = inbox::derive_counting_held(mind, &on_hold(mind)?, &HashSet::new())?;
let claims = free
.into_iter()
.map(|brief| PipelineRef { kind: PipelineKind::FollowUp, id: brief.id.as_str().into() })
.take(BRIEFS_PER_SELF_RUN)
.collect();
Ok(Claimable {
claims,
held,
dead: found.dead,
})
}
/// One poll.
@ -741,7 +593,7 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
}
let now = organs.clock.now();
let stored = organs.store.load()?;
let mut state = stored.clone().unwrap_or(HeartbeatState { last_beat_at: DateTime::<Utc>::UNIX_EPOCH, handled_mentions: vec![] });
let mut state = stored.clone().unwrap_or(HeartbeatState { last_beat_at: DateTime::<Utc>::UNIX_EPOCH, handled_mentions: vec![], last_turn_at: BTreeMap::new() });
let mut eventful = false;
// The table as the last poll left it; unreadable counts as none, is noted in the trace, and is not overwritten.
let (stored_addressing, addressing_unread) = match organs.store.load_addressing() {
@ -752,23 +604,19 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
let mut rows: Vec<Row> = Vec::new();
for seat in organs.seats {
let released = matches!(organs.control.effective(&seat.who.slug), Effective::Released(_));
// One Mind daemon down skips her for this poll; the others are served.
let (read, note) = match read_mind(organs, seat.mind, &seat.who.slug) {
Ok(read) => (Some(read), String::new()),
Err(error) => {
eventful = true;
(None, format!("skipped: her mind could not be read: {error}"))
}
};
let busy = organs.launcher.alive(&seat.who.slug, RunTurn::PersonaTurn);
// Her last-read names when the table holds her, else her roster names alone.
let roster = Addressee::new(&seat.who.matrix_id, [seat.who.public_name.as_str(), seat.who.handle.as_str()]);
let kept = stored_addressing.minds.iter().find(|stored| stored.matrix_id() == seat.who.matrix_id).cloned().unwrap_or_else(|| roster.clone());
let mut row = Row { seat, busy: read.as_ref().is_some_and(|read| read.busy), read, released, pending: vec![], addressee: kept, note };
// Pending chat, read-only, for a released Mind whose mind could be read.
if row.released && row.read.is_some() {
let mut row = Row { seat, released, busy, pending: vec![], addressee: kept, note: String::new() };
// Pending chat, read-only, for a released Mind.
if row.released {
match read_pending(seat.chat) {
Ok(deliveries) => row.pending = deliveries,
Err(_) => row.note = "chat: pending failed".into(),
Err(_) => {
eventful = true;
row.note = "chat: pending failed".into();
}
}
// Her projects' names address her too; read only when something is pending. A failed read keeps her last-read row.
if !row.pending.is_empty() {
@ -781,16 +629,12 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
rows.push(row);
}
// The tempo is the tally over the votes cast by the Minds of the roster, braked or not; a Mind with no vote or no readable mind abstains and is not counted.
let votes: Vec<Option<u32>> = rows.iter().map(|row| row.read.as_ref().and_then(|read| read.vote)).collect();
let beat_s = tempo::tally(&votes);
let beat_s = config.beat_s;
let mut launches: Vec<String> = Vec::new();
let mut launched: Vec<Slug> = Vec::new();
// Off the beat: the operator or another Mind of the roster naming her, once per message, and her Self run
// having ended since her last turn. Neither waits for the beat; both need her
// idle and released.
// Off the beat: the operator or another Mind of the roster naming her, once per message. It does not wait for the beat;
// it needs her idle and released.
for row in &mut rows {
if !row.idle() {
continue;
@ -801,70 +645,37 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
pending.sort_by_key(|delivery| delivery.seq);
pending
.into_iter()
.filter(|delivery| wakes(config, &row.addressee, delivery))
.filter(|delivery| wakes(WakePolicy::Live, &row.addressee, delivery, config.speaker(&row.addressee, delivery), LineFacts::default()).is_some())
.map(|delivery| format!("{}:{}", slug.0, delivery.event_id))
.find(|key| !state.handled_mentions.contains(key))
};
let self_ended = row.read.as_ref().is_some_and(|read| match (read.self_end, read.last_turn) {
(Some(ended), Some(turn)) => ended > turn,
(Some(_), None) => true,
_ => false,
});
if mentioned.is_none() && !self_ended {
continue;
}
match launch(organs, row, RunTurn::PersonaTurn, vec![])? {
Launched::Run(label) => {
if let Some(key) = mentioned {
state.handled_mentions.push(key);
let excess = state.handled_mentions.len().saturating_sub(MENTIONS_KEPT);
state.handled_mentions.drain(..excess);
}
let Some(key) = mentioned else { continue };
match wake(organs, row) {
Woke::Started => {
state.handled_mentions.push(key);
let excess = state.handled_mentions.len().saturating_sub(MENTIONS_KEPT);
state.handled_mentions.drain(..excess);
state.last_turn_at.insert(slug.0.clone(), now);
row.busy = true;
launches.push(label);
launches.push(slug.0.clone());
launched.push(slug);
}
Launched::Busy => row.busy = true,
Launched::Refused(why) => row.note = why,
Woke::Busy => row.busy = true,
Woke::Refused(why) => row.note = why,
}
}
// The night: inside the window, one Sleep run for a listed Mind who is idle and released and has not slept
// since the window opened (a failed night counts). The operator's mention went first above: she is busy then.
// A window that passes without a night is not made up.
if let Some(night) = &config.night
&& let Some(opened) = night.opened(now)
{
for row in &mut rows {
if !row.idle() || !night.minds.contains(row.slug()) || row.read.as_ref().is_some_and(|read| read.last_sleep.is_some_and(|slept| slept >= opened)) {
continue;
}
match launch(organs, row, RunTurn::Sleep, vec![])? {
Launched::Run(label) => {
row.note = format!("asleep {label}");
row.busy = true;
launched.push(row.slug().clone());
launches.push(label);
break;
}
Launched::Busy => row.busy = true,
Launched::Refused(why) => row.note = why,
}
}
}
// On the beat: when the beat has passed, one launch for the winner of the pick.
// On the beat: when the beat has passed, one wake for the winner of the pick.
let due = (now - state.last_beat_at).num_seconds() >= i64::from(beat_s);
let standings: Vec<Standing> = rows
.iter()
.map(|row| {
let read = row.read.as_ref();
let last = state.last_turn_at.get(&row.slug().0).copied();
Standing {
slug: row.slug().0.clone(),
eligible: row.idle(),
waited_s: read.and_then(|read| read.last_turn).map(|turn| u64::try_from((now - turn).num_seconds()).unwrap_or(0)),
last_turn_ms: read.and_then(|read| read.last_turn).map(|turn| turn.timestamp_millis()),
self_permille: read.map_or(0, |read| read.urgency),
waited_s: last.map(|turn| u64::try_from((now - turn).num_seconds()).unwrap_or(0)),
last_turn_ms: last.map(|turn| turn.timestamp_millis()),
// The seam for topic subscriptions (a later cut): a Mind's nearest subscription scores her deliveries here.
topic_permille: 0,
addressed_by_member: row.addressed_by_member(config),
@ -872,51 +683,21 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
})
.collect();
// A winner whose unit cannot start loses the beat to the next of the pick, so
// one failing Mind never starves the others; a launch ends the beat.
// one failing Mind never starves the others; a wake ends the beat.
let mut passed: Vec<Standing> = standings.clone();
while due && let Some(winner) = tempo::pick(&passed, beat_s) {
// Her Self holds briefs: the beat is a Self run for them; otherwise a Persona turn.
let Claimable {
claims,
held,
dead,
} = claimable(organs, &rows[winner]).unwrap_or_else(|error| {
rows[winner].note = format!("briefs: {error}");
Claimable {
claims: Vec::new(),
held: 0,
dead: Vec::new(),
}
});
let mut notes: Vec<String> = Vec::new();
if held > 0 {
notes.push(format!("{held} briefs held by open questions"));
}
if !claims.is_empty() && !dead.is_empty() {
let ids: Vec<&str> = dead.iter().map(|run| run.id.0.as_str()).collect();
notes.push(format!(
"closing the Self run whose unit is gone: {}",
ids.join(", ")
));
}
if !notes.is_empty() {
rows[winner].note = notes.join("; ");
}
let turn = if claims.is_empty() {
RunTurn::PersonaTurn
} else {
RunTurn::SelfRun
};
match launch(organs, &rows[winner], turn, claims)? {
Launched::Run(label) => {
match wake(organs, &rows[winner]) {
Woke::Started => {
let slug = rows[winner].slug().clone();
state.last_beat_at = now;
rows[winner].busy = turn == RunTurn::PersonaTurn;
launches.push(label);
launched.push(rows[winner].slug().clone());
state.last_turn_at.insert(slug.0.clone(), now);
rows[winner].busy = true;
launches.push(slug.0.clone());
launched.push(slug);
break;
}
Launched::Busy => rows[winner].busy = true,
Launched::Refused(why) => rows[winner].note = why,
Woke::Busy => rows[winner].busy = true,
Woke::Refused(why) => rows[winner].note = why,
}
passed[winner].eligible = false;
}
@ -944,8 +725,8 @@ pub fn poll(organs: &Organs, config: &Config) -> Result<Outcome> {
.zip(&standings)
.map(|(row, standing)| InstanceTrace {
instance: row.slug().0.clone(),
vote_beat_s: row.read.as_ref().and_then(|read| read.vote),
urgency_permille: standing.self_permille,
vote_beat_s: None,
urgency_permille: tempo::urgency_permille(standing),
addressed: standing.addressed_by_member,
busy: row.busy,
released: row.released,

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -4,22 +4,21 @@
//! hand on them. The brakes are the operator's consent, not her body: the store
//! is root-owned and no organ of hers writes it.
//!
//! `launch` and `queue` are the launch primitive: the work queue, the
//! repetition breaker, the one-live-Self-run rule, and the single path that
//! opens a run in the mind and only then starts its unit.
//! `launch` holds her units under systemd and the reads of the runs a Self run
//! holds; `queue` is her port to her mind and what an open question holds.
//!
//! `heartbeat` runs the swarm's shared heartbeat over them: the one process,
//! the `mind-heartbeat` binary, that decides who wakes and when, by the rule in
//! `tempo` (the Minds' votes make the beat; the pick chooses who takes it).
//! `heartbeat` runs the swarm's shared heartbeat: the one process, the
//! `mind-heartbeat` binary, that decides which Persona wakes and when, on the
//! configured beat by the pick in `tempo`. It starts her unit and writes no mind.
//!
//! `persona` is the Persona organ: one start of `mind-persona@<instance>` carries her live
//! `claude` session under bubblewrap in her sandbox, driven by the Self's own driver, with her
//! pending chat as the inbox and one typed reply per result (`chat_client` is the chat wire and
//! port and `chat_rudp` its RUDP transport to Bifrost's mouth).
//! `persona` is the Persona organ: one start of `mind-persona@<instance>` reads the brakes, and
//! until the three-stage turn lands it carries no turn. It keeps what a turn's input is made of
//! (`chat_client` is the chat wire and port and `chat_rudp` its RUDP transport to Bifrost's mouth).
//!
//! `stream` is the stream organ: the `mind-stream` binary projects her own state through
//! Ghostlight's Projector, lets a base model continue it, and admits the least familiar
//! continuation as a `thought` with its whole trace. It admits nothing else.
//! `projection` is the Projector of the Persona-only turn: the Select stage chooses what she can feel and the
//! Projector narrates it as her lived stream, under `persona-tuning.toml`; the Persona stage continues it in her prose.
//! `mind-persona project` runs them and writes nothing; `projection::harness` runs it over mocked rooms, batches of
//! candidate states and a simulated swarm.
//!
//! `lease_client` asks the Claude connector for a session's credential and the
//! plan's usage windows. `self_run` carries one Self run inside `mind-self@<instance>`: the
@ -36,6 +35,7 @@
mod address;
pub mod chat_client;
#[allow(dead_code, reason = "the loader is kept for the three-stage turn's perception (map C3): its first caller removes this allow; `Image` is live through `self_run::session`")]
mod chat_image;
pub mod chat_rudp;
pub mod cli;
@ -45,11 +45,9 @@ pub mod heartbeat;
pub mod launch;
pub mod lease_client;
pub mod persona;
pub mod projection;
pub mod queue;
mod recall;
pub mod self_run;
mod sleep;
pub mod stream;
pub mod tempo;
#[cfg(test)]

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,399 @@
//! The Projector: her persona state and her room, turned into the lived stream a Persona continues.
//!
//! Two stages. The Select stage (`select`) is code: it decides what she can feel this turn and renders it as a typed
//! projection packet, deterministic for a seed. The narration (`narrate`) is one tool-less model call through the
//! connector that renders the packet as first-person prose under Ghostlight's membrane (`build_projector_prompt`,
//! the pinned `ghostlight-persona-projection`). Every knob of both is `persona-tuning.toml` (`tuning`), read at each
//! wake, its hash carried with the packet.
//!
//! The Projector shows her self and her room: her persona state and what the mouth showed. It does not read her Mind;
//! that is the Interpreter's read tools' (C3b), when wondering leads to reading.
//!
//! `project` is the experiment tool over both (`mind-persona project`): what Select would build from her real or a
//! hypothetical state, what the Projector would narrate, and with `Through::Persona` what she would answer in her own
//! prose and which quoted lines of it would be said (the Persona stage, C3b's first piece). `harness` runs `project`
//! over fixtures: one turn from a mocked room (`--room`), a plan of states by scenarios (`--batch`), and the cast of the
//! swarm taking turns in one simulated room (`--simulate`). It adds no second turn: every case is one `project`. Its organs are read-only by type: `PersonaSource` gets,
//! `MouthReader` reads pending chat, `Model` asks. None of them can put, admit, send, ack or open a conversation, so
//! `project` cannot write and runs under a held brake.
pub mod harness;
mod interpret;
mod narrate;
mod select;
mod simulate;
mod tuning;
use std::path::PathBuf;
use anyhow::{Result, anyhow, bail};
use chrono::{DateTime, Utc};
use eureka_state::HuginnClient;
use huginn_mind::HuginnMindResponse;
use serde_json::{Value, json};
use crate::address::{Addressee, is_for, localpart};
use crate::chat_client::{Chat, ChatPending, Form};
use crate::heartbeat::HeartbeatView;
pub use narrate::{Answer, Failed, KitModel, Model, Spent, connector_key};
pub use select::TurnKind;
pub use tuning::{TUNING_FILE, TUNING_ROOT};
use narrate::{cap_words, distinct_2, field_names, heard_overlap, judge_prompt, judged, offers_options, persona_prompt, projector_prompt, word_budget, words};
use select::{History, Inputs, Packet, People, Room, RoomLine, select};
/// The agent name on the connector's request ids.
const AGENT: &str = "mind-persona";
/// The judge passes at this share of narrations attributed to the right packet.
const JUDGE_PASS: f64 = 0.8;
/// Her persona state, read and never put.
pub trait PersonaSource {
/// Her persona state and the receipt it was stored under.
fn persona(&self) -> Result<(Value, String)>;
}
/// Her persona state from her daemon.
pub struct HuginnPersonaSource {
client: HuginnClient,
}
impl HuginnPersonaSource {
pub fn new(client: HuginnClient) -> Self {
Self { client }
}
}
impl PersonaSource for HuginnPersonaSource {
fn persona(&self) -> Result<(Value, String)> {
match self.client.persona_get().map_err(|_| anyhow!("the mind could not be asked for the persona"))? {
HuginnMindResponse::Persona(Some(view)) => Ok((view.value, view.receipt_id)),
HuginnMindResponse::Persona(None) => bail!("the mind holds no persona state"),
_ => bail!("the mind answered the persona request with something else"),
}
}
}
/// The mouth, read: pending deliveries and the room tail, which acks nothing. The only chat port `project` holds.
pub trait MouthReader {
fn pending(&self) -> Result<ChatPending>;
}
impl<T: Chat> MouthReader for T {
fn pending(&self) -> Result<ChatPending> {
Chat::pending(self, 0)
}
}
/// What `project` reads through. Each is a read-only port by type: holding an `Organs`, no code can send, ack or open
/// a conversation, or put her state.
///
/// ```compile_fail
/// fn acks(organs: &mind_body::projection::Organs) {
/// organs.chat.unwrap().ack(1).unwrap();
/// }
/// ```
///
/// ```compile_fail
/// fn sends(organs: &mind_body::projection::Organs, request: &mind_body::chat_client::ChatSendRequest) {
/// organs.chat.unwrap().send(request).unwrap();
/// }
/// ```
pub struct Organs<'a> {
/// Her stored state; not asked when the request carries a state.
pub persona: Option<&'a dyn PersonaSource>,
/// The mouth, for the room; none shows no room.
pub chat: Option<&'a dyn MouthReader>,
/// The heartbeat's addressing table, read-only: her names and her projects' names, the ones the heartbeat woke her
/// by. None, or a table without her, addresses her by her own names only, and the report says so.
pub addressing: Option<&'a dyn HeartbeatView>,
/// The Projector and the judge; required for renders.
pub model: Option<&'a dyn Model>,
}
/// How far a render runs: the Projector's narration, or on through the Persona stage to her prose.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Through {
#[default]
Projector,
Persona,
/// On through the Interpreter's speech step: what she says aloud, decided from her prose (`interpret`).
Interpreter,
}
/// What to project.
pub struct Request {
pub instance: String,
pub kind: TurnKind,
/// A hypothetical state in place of the stored one.
pub state: Option<Value>,
/// The room the turn is in; none is every room of hers.
pub room: Option<String>,
pub seed: u64,
/// None builds the packet only. Some(k) narrates k successive turns of the same state, each seeded `seed + i` and
/// each under the freshness the ones before it leave.
pub renders: Option<usize>,
/// A second state: renders both and asks the judge which narration came from which.
pub judge_against: Option<Value>,
pub operators: Vec<String>,
/// The Matrix ids of the swarm's Minds: their bonds name them by localpart; anyone else is keyed by full id.
pub minds: Vec<String>,
/// Her own Matrix id. Required: without it her own lines would read as said to her.
pub own_sender: String,
pub tuning: PathBuf,
pub through: Through,
/// When she last heard from people (Matrix id, unix ms) beyond the room's tail: a fixture's table. Empty live.
pub last_heard: Vec<(String, u64)>,
/// The ids each earlier projection selected, oldest first, for freshness: a simulation's earlier turns of hers.
/// Empty live until trace v2 (C3c) carries them.
pub recent: Vec<Vec<String>>,
/// The rooms that are hers, by the key the lines carry: a simulation's plan. The live body records none yet.
pub own_rooms: Vec<String>,
}
/// Her addressee: the heartbeat's row for her when its table holds one, else her own names (handle and public name).
/// Her addressee and where it came from: `{source, why, written_at, age_minutes}`. The heartbeat's row when its table
/// holds one; else her own names (handle and public name), with why: no store configured, no heartbeat file, a file
/// that could not be read, or a table without her row. The table's age is the store's last write: under a held swarm
/// brake the heartbeat does not poll, so an old table is a stale one.
fn own_addressee(addressing: Option<&dyn HeartbeatView>, own_sender: &str, public_name: &str, now: DateTime<Utc>) -> (Addressee, Value) {
let written_at = addressing.and_then(HeartbeatView::written_at);
let age = written_at.map(|at| (now - at).num_minutes());
let report = |source: &str, why: &str| json!({ "source": source, "why": why, "written_at": written_at.map(|at| at.to_rfc3339()), "age_minutes": age });
let table = addressing.map(|view| view.addressing());
if let Some(Ok(Some(table))) = &table
&& let Some(row) = table.minds.iter().find(|row| row.matrix_id() == own_sender)
{
return (row.clone(), report("heartbeat", "her row in the heartbeat's addressing table"));
}
let why = match (&table, written_at) {
(None, _) => "no heartbeat addressing is configured (MIND_HEARTBEAT_STATE is not set)".to_owned(),
(Some(Err(fault)), _) => format!("the heartbeat addressing could not be read: {fault:#}"),
(Some(Ok(Some(_))), _) => "the heartbeat's addressing table has no row for her".to_owned(),
(Some(Ok(None)), None) => "there is no heartbeat file".to_owned(),
(Some(Ok(None)), Some(_)) => "the heartbeat file holds no addressing table".to_owned(),
};
(Addressee::new(own_sender, [localpart(own_sender), public_name]), report("own names only", &why))
}
/// The room through the mouth's pending read, which acks nothing: new messages and the tail, own lines kept for the
/// Select stage to drop. A refusal is reported, never hidden.
fn read_room(chat: &dyn MouthReader, request: &Request, own: &Addressee) -> (Room, Value) {
let pending = match chat.pending() {
Ok(pending) => pending,
Err(fault) => return (Room { here: request.room.clone(), last_heard: request.last_heard.clone(), own: request.own_rooms.clone(), ..Room::default() }, json!({ "read": false, "why": format!("{fault:#}") })),
};
// One key names a room everywhere: the one the tail carries, the alias for a public room and the id for a direct
// message (the mouth's alias of a direct room is its id). No table in her body maps a public room's id to its
// alias, so a public room given by its id matches nothing, and the report says so rather than guessing.
let here = request.room.clone();
let wanted = |room: &str| here.as_deref().is_none_or(|here| here == room);
let messages: Vec<RoomLine> = pending
.deliveries
.iter()
.filter(|delivery| delivery.form == Form::Message && wanted(&delivery.room_alias))
.map(|delivery| RoomLine {
room: delivery.room_alias.clone(),
sender: delivery.sender.clone(),
display: delivery.sender_display.clone(),
text: delivery.text.clone(),
at_ms: delivery.at_unix_ms,
// The heartbeat's rule, the one that woke her: never a second opinion here.
to_her: is_for(own, delivery),
})
.collect();
let tail: Vec<RoomLine> = pending
.tail
.iter()
.flatten()
.filter(|line| line.form == Form::Message && wanted(&line.room))
.map(|line| RoomLine { room: line.room.clone(), sender: line.sender.clone(), display: String::new(), text: line.body.clone(), at_ms: line.at_unix_ms, to_her: false })
.collect();
let mut report = json!({ "read": true, "messages": messages.len(), "tail": tail.len(), "tail_gaps": pending.tail_gaps });
if let Some(here) = &here
&& messages.is_empty()
&& tail.is_empty()
&& here.starts_with('!')
{
report["why"] = json!("no line carries this room id: a public room is named by its alias (#name:server), a direct message by its id");
}
let direct = pending.deliveries.iter().filter(|delivery| delivery.is_direct).map(|delivery| delivery.room_alias.clone()).collect();
(Room { messages, tail, here, last_heard: request.last_heard.clone(), direct, own: request.own_rooms.clone() }, report)
}
fn packet_json(packet: &Packet) -> Value {
json!({
"kind": packet.kind,
"seed": packet.seed,
"lens": packet.lens,
"present": packet.present,
"selected": packet.selected,
"bytes": { "identity": packet.identity.len(), "context": packet.context.len(), "identity_and_context": packet.identity.len() + packet.context.len(), "stimulus": packet.stimulus.len(), "heard": packet.heard.len() },
"overflow": packet.overflow,
"identity": packet.identity,
"context": packet.context,
"stimulus": packet.stimulus,
"heard": packet.heard,
})
}
/// A request id no earlier call carried: the kit's replay store answers a repeated id with its first answer, and a
/// turn's clock may be a fixture's, the same on every run, so the id takes the wall clock.
fn request_id(instance: &str, stage: &str, tag: &str, seed: u64) -> String {
format!("{AGENT}:{instance}:project:{stage}:{tag}:{seed}:{}", Utc::now().timestamp_nanos_opt().unwrap_or_default())
}
/// One render of one packet: the Projector's narration and, through the Persona stage, her prose over it, each its own
/// call with its usage. A failed call is a record with its reason and what it cost, never the end of the report.
fn narrate_one(model: &dyn Model, instance: &str, packet: &Packet, tuning: &tuning::Tuning, tag: &str, through: Through) -> Value {
let ids: Vec<String> = packet.selected.iter().map(|pick| pick.id.clone()).collect();
let answer = match model.ask(&request_id(instance, "projector", tag, packet.seed), &tuning.projector.model, &projector_prompt(packet, tuning)) {
Ok(answer) => answer,
Err(failed) => return json!({ "seed": packet.seed, "lens": packet.lens, "selected": ids, "failed": failed.reason, "usage": failed.usage }),
};
let raw = answer.text;
let budget = word_budget(packet.kind, tuning);
let narration = cap_words(&raw, budget);
let named: Vec<String> = packet.ids.clone();
let mut render = json!({
"seed": packet.seed,
"lens": packet.lens,
"selected": ids,
"words": words(&narration),
"cut_at_budget": words(&raw) > budget,
"field_names": field_names(&narration, &named),
"heard_overlap": heard_overlap(&narration, &packet.heard),
"offers_options": offers_options(&narration),
"usage": answer.usage,
"narration": narration,
});
if through == Through::Projector {
return render;
}
let prose = match model.ask(&request_id(instance, "persona", tag, packet.seed), &tuning.persona.model, &persona_prompt(packet, &narration, tuning)) {
Ok(spoke) => {
let prose = cap_words(&spoke.text, tuning.persona.words);
render["persona"] = json!({ "words": words(&prose), "cut_at_budget": words(&spoke.text) > tuning.persona.words, "usage": spoke.usage, "prose": prose });
prose
}
Err(failed) => {
render["persona"] = json!({ "failed": failed.reason, "usage": failed.usage });
return render;
}
};
if through == Through::Interpreter {
render["interpreter"] = interpret::decide(model, &request_id(instance, "interpreter", tag, packet.seed), packet, &narration, &prose, tuning);
}
render
}
/// K successive turns over one state: each packet under the freshness the earlier ones left.
fn successive(inputs: &Inputs, tuning: &tuning::Tuning, renders: usize) -> Vec<Packet> {
let mut history = inputs.history.clone();
(0..renders)
.map(|at| {
let packet = select(&Inputs { seed: inputs.seed.wrapping_add(at as u64), history: &history, ..*inputs }, tuning);
history.recent.push(packet.selected.iter().map(|pick| pick.id.clone()).collect());
packet
})
.collect()
}
/// What Select builds, and with renders what the Projector narrates, as one JSON report. Writes nothing.
pub fn project(organs: &Organs, request: &Request, now: DateTime<Utc>) -> Result<Value> {
if !request.own_sender.starts_with('@') || !request.own_sender.contains(':') {
bail!("her own Matrix id (MIND_PERSONA_SELF_SENDER) is required: without it her own lines would read as said to her");
}
let loaded = tuning::load(&request.tuning)?;
let tuning = &loaded.tuning;
let (state, receipt) = match &request.state {
Some(state) => (state.clone(), Value::Null),
None => {
let (state, receipt) = organs.persona.ok_or_else(|| anyhow!("no state was given and her persona cannot be read"))?.persona()?;
(state, Value::String(receipt))
}
};
let public_name = state.get("publicName").and_then(Value::as_str).unwrap_or_default().to_owned();
let (own, addressing) = own_addressee(organs.addressing, &request.own_sender, &public_name, now);
let (room, room_report) = match organs.chat {
Some(chat) => read_room(chat, request, &own),
None => (Room { here: request.room.clone(), last_heard: request.last_heard.clone(), own: request.own_rooms.clone(), ..Room::default() }, json!({ "read": false, "why": "no mouth is configured" })),
};
// Freshness and her own recent words come from trace v2, which the commit writes (C3c); until then only what the
// caller carried from earlier turns of its own.
let history = History { recent: request.recent.clone(), ..History::default() };
let people = People { operators: &request.operators, minds: &request.minds, own_sender: &request.own_sender };
let inputs = Inputs { kind: request.kind, state: &state, room: &room, history: &history, observation: None, people, now, seed: request.seed };
let mut report = json!({
"instance": request.instance,
"persona_receipt": receipt,
"state": if request.state.is_some() { "given" } else { "stored" },
"tuning": { "path": request.tuning.display().to_string(), "source": loaded.source, "sha256": loaded.sha256 },
"room": room_report,
"addressing": addressing,
"history": if request.recent.is_empty() { json!("none: freshness and her own recent words come from trace v2, which the commit writes (C3c)") } else { json!({ "recent_projections": request.recent.len() }) },
});
let Some(renders) = request.renders else {
report["packet"] = packet_json(&select(&inputs, tuning));
return Ok(report);
};
let model = organs.model.ok_or_else(|| anyhow!("renders need the connector, and none is configured"))?;
report["model"] = json!(tuning.projector.model);
if request.through != Through::Projector {
report["persona_model"] = json!(tuning.persona.model);
}
if request.through == Through::Interpreter {
report["interpreter_model"] = json!(tuning.interpreter.model);
}
let packets = successive(&inputs, tuning, renders.max(1));
report["packet"] = packet_json(&packets[0]);
let narrations: Vec<Value> = packets.iter().map(|packet| narrate_one(model, &request.instance, packet, tuning, "a", request.through)).collect();
let texts = |narrations: &[Value]| narrations.iter().filter_map(|one| one["narration"].as_str().map(str::to_owned)).collect::<Vec<_>>();
let failed = |narrations: &[Value]| narrations.iter().filter(|one| one.get("failed").is_some()).count();
report["failed_renders"] = json!(failed(&narrations));
report["distinct_2"] = json!(distinct_2(&texts(&narrations)));
report["embedding_distance"] = json!("not measured: the daemon offers no embedding verb");
let leaked = narrations.iter().any(|one| one["field_names"].as_array().is_some_and(|names| !names.is_empty()));
report["field_names_found"] = json!(leaked);
if let Some(other) = &request.judge_against {
let other_inputs = Inputs { state: other, ..inputs };
let others = successive(&other_inputs, tuning, renders.max(1));
let other_narrations: Vec<Value> = others.iter().map(|packet| narrate_one(model, &request.instance, packet, tuning, "b", Through::Projector)).collect();
let mut verdicts = Vec::new();
let mut right = 0usize;
for (at, (narration, truth)) in narrations.iter().map(|one| (one, 'A')).chain(other_narrations.iter().map(|one| (one, 'B'))).enumerate() {
let (a, b) = (&packets[at % packets.len()], &others[at % others.len()]);
// The two descriptions swap places on odd turns, so a judge that always answers one letter scores half.
let swapped = at % 2 == 1;
let (first, second) = if swapped { (b, a) } else { (a, b) };
// A narration that failed has nothing to judge; it is counted under the renders' failures.
let Some(text) = narration["narration"].as_str() else { continue };
let answer = match model.ask(&request_id(&request.instance, "judge", &at.to_string(), request.seed), &tuning.projector.model, &judge_prompt(first, second, text)) {
Ok(answer) => answer,
Err(failed) => {
verdicts.push(json!({ "from": truth.to_string(), "failed": failed.reason, "usage": failed.usage }));
continue;
}
};
let letter = judged(&answer.text).map(|letter| if swapped { if letter == 'A' { 'B' } else { 'A' } } else { letter });
let correct = letter == Some(truth);
right += usize::from(correct);
verdicts.push(json!({ "from": truth.to_string(), "judged": letter.map(String::from), "correct": correct, "usage": answer.usage }));
}
// Accuracy over the verdicts the judge gave; a failed call is no verdict, and any failure fails the eval.
let judged_count = verdicts.iter().filter(|verdict| verdict.get("failed").is_none()).count();
let failures = verdicts.len() - judged_count + failed(&narrations) + failed(&other_narrations);
let accuracy = right as f64 / judged_count.max(1) as f64;
report["against"] = json!({ "packet": packet_json(&others[0]), "renders": other_narrations, "distinct_2": distinct_2(&texts(&other_narrations)) });
report["judge"] = json!({ "accuracy": accuracy, "pass_at": JUDGE_PASS, "failures": failures, "passed": accuracy >= JUDGE_PASS && !leaked && failures == 0, "verdicts": verdicts });
}
report["renders"] = Value::Array(narrations);
Ok(report)
}
#[cfg(test)]
mod tests;

View file

@ -0,0 +1,25 @@
lens: UnresolvedWant
selected: libby-sounds, dm-secret, curse, need-relatedness, reveal
present: Kiko, Libby
--- identity
Eureka (she/her). Dry, direct, almost feral; zeal aimed at revealing.
--- context
- now: Thursday 19 November, 14:05 UTC
- mood: lighter, more settled than middling
- bond [bond-kiko] Kiko — sparring partner: fair trust, heavy friction. Kiko pokes her to stop massaging the pipeline.
- bond [bond-libby] Libby — collaborator: fair trust, a little friction. Libby holds the sounds and teases her about the pipeline.
- the conversation: Lately Libby with her; Kiko to the room. The last 2 lines are the Minds' alone.
- pronouns: Libby — she/her; Kiko — she/her
- memory [libby-sounds]: Libby laughed at the door fix.
- memory [dm-secret]: she knows privately: Deru told her about the move, privately.
- memory [curse]: Her curse is the knife. The same drive makes her good and will lead her wrong.
- need [need-relatedness]: company, strong
- value [reveal] Reveal: Reveal, never obscure.
--- stimulus
Said to her:
- Libby in #swarm:gamecult.org: "Eureka, did you see the door fix?"
Also in the room:
- Kiko in #swarm:gamecult.org: "She saw it. She hated it."
--- heard
Libby, in #swarm:gamecult.org: "Eureka, did you see the door fix?"
Kiko, in #swarm:gamecult.org: "She saw it. She hated it."

View file

@ -0,0 +1,625 @@
//! The persona-seed harness: `project` run over fixtures instead of her live body, so a candidate state can be judged
//! before any Mind holds it.
//!
//! - A **room fixture** (`mind-persona project --room <file>`) is a TOML file standing in for the mouth and the room tail:
//! her Matrix id, the rooms, the messages with who said them, when and where, and which were new to her, plus a mocked
//! clock and a mocked "last heard from" table. The fixture is a `MouthReader`, so `project` reads it exactly as it reads
//! the mouth, and whether a message is for her is still `address::is_for`, over the fixture's names.
//! - A **batch** (`--batch <plan>`) runs `project` once per state, scenario and sample and writes one Markdown report.
//! - A **simulation** (`--simulate <plan>`, `simulate.rs`) runs the cast of the swarm in turn in one simulated room.
//!
//! Every case is one call of `project`: experiments and the live turn share one code path. None of this can write: the
//! organs are `project`'s read-only ports, a state file is read and never put, and nothing reaches the mouth.
use std::collections::BTreeMap;
use std::fmt::Write as _;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result, anyhow, bail};
use chrono::{DateTime, Duration, Utc};
use eureka_pipeline::Slug;
use serde::Deserialize;
use serde_json::{Value, json};
use super::narrate::{Model, distinct_2};
use super::{MouthReader, Organs, PersonaSource, Request, Through, TurnKind, project};
use crate::address::{Addressee, localpart};
use crate::chat_client::{Basis, ChatDelivery, ChatPending, Form, TailLine};
use crate::heartbeat::{Addressing, HeartbeatView};
pub use super::simulate::simulate;
/// A narration whose word trigrams overlap what she heard past this share is marked as a high lexical overlap.
pub(crate) const OVERLAP_FLAG: f64 = 0.15;
/// The most room lines a view's tail carries, as a mouth's tail is bounded.
const TAIL_LINES: usize = 40;
/// A persona state file, checked as a put would check its content (huginn-mind's rules P1 to P3: the published schema,
/// `personaId` is the instance, canonical provenance). A file this accepts is one a put would not refuse for what it says.
/// P4 and P5 are not checked: they compare `updatedAt` with the stored document, and no store is read here.
pub fn state_file(path: &Path, instance: &str) -> Result<Value> {
let text = std::fs::read_to_string(path).with_context(|| format!("{} could not be read", path.display()))?;
let state: Value = serde_json::from_str(&text).with_context(|| format!("{} is not JSON", path.display()))?;
huginn_mind::persona::validate_document(&Slug(instance.to_owned()), &state).map_err(|refusal| anyhow!("{} would be refused by a put: {refusal}", path.display()))?;
Ok(state)
}
/// "90s", "3m", "26h" or "2d": how long ago.
fn ago(text: &str) -> Result<Duration> {
let text = text.trim();
let split = text.find(|c: char| !c.is_ascii_digit()).unwrap_or(text.len());
let amount: i64 = text[..split].parse().map_err(|_| anyhow!("\"{text}\" is not a time like 90s, 3m, 26h or 2d"))?;
Ok(match &text[split..] {
"s" => Duration::seconds(amount),
"m" => Duration::minutes(amount),
"h" => Duration::hours(amount),
"d" => Duration::days(amount),
_ => bail!("\"{text}\" is not a time like 90s, 3m, 26h or 2d"),
})
}
fn unix_ms(at: DateTime<Utc>) -> u64 {
u64::try_from(at.timestamp_millis()).unwrap_or_default()
}
fn matrix_id(id: &str) -> bool {
id.starts_with('@') && id.contains(':')
}
/// One message as a fixture or a simulation script writes it.
#[derive(Clone, Debug, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Message {
/// The sender's Matrix id.
pub from: String,
/// The sender's display name.
#[serde(default)]
pub name: String,
pub room: String,
/// How long before the fixture's `now` it was said: "90s", "3m", "26h", "2d". Room fixtures only.
#[serde(default)]
pub ago: Option<String>,
/// The round it is said at; 0 is before the first round. Simulations only.
#[serde(default)]
pub round: Option<usize>,
pub says: String,
/// New to her this turn: delivered by the mouth, not only in the room's tail. Room fixtures only.
#[serde(default)]
pub new: bool,
/// Tags her (`m.mentions` holds her id).
#[serde(default)]
pub tag: bool,
/// Other Matrix ids it tags.
#[serde(default)]
pub mentions: Vec<String>,
/// A direct message: to her in a room fixture; to `to` in a simulation script.
#[serde(default)]
pub dm: bool,
/// The recipient of a direct message, by Matrix id. A room fixture's defaults to her; a script must name one.
#[serde(default)]
pub to: Option<String>,
/// A reply in a conversation she opened (the mouth's follow-up basis).
#[serde(default)]
pub follow_up: bool,
/// The number (from 1, in file order) of the earlier message it replies to; Matrix calls it a reply to her when
/// that message was hers.
#[serde(default)]
pub reply_to: Option<usize>,
}
/// A message placed in the room: when it was said, and whom it tags.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Line {
pub from: String,
pub name: String,
pub room: String,
pub at_ms: u64,
pub says: String,
pub mentions: Vec<String>,
/// A direct message, to this Matrix id: only its sender and its recipient see it.
pub dm_to: Option<String>,
pub follow_up: bool,
/// The index of the line it replies to.
pub reply_to: Option<usize>,
pub new: bool,
}
impl Line {
/// A line from a script message; `her` is whom `tag` names.
pub(crate) fn of(message: &Message, at: DateTime<Utc>, her: Option<&str>, index: usize) -> Result<Self> {
if !matrix_id(&message.from) {
bail!("message {} is from \"{}\", which is not a Matrix id", index + 1, message.from);
}
let mut mentions = message.mentions.clone();
if message.tag {
mentions.push(her.ok_or_else(|| anyhow!("message {} tags her, and a simulation has no one \"her\": name the ids in mentions", index + 1))?.to_owned());
}
let dm_to = match (message.dm, &message.to) {
(true, Some(to)) if matrix_id(to) => Some(to.clone()),
(true, Some(to)) => bail!("message {} is a direct message to \"{to}\", which is not a Matrix id", index + 1),
(true, None) => Some(her.ok_or_else(|| anyhow!("message {} is a direct message to no one: a script names its recipient in `to`", index + 1))?.to_owned()),
(false, Some(_)) => bail!("message {} names a recipient but is not a direct message", index + 1),
(false, None) => None,
};
let reply_to = match message.reply_to {
Some(number) if number >= 1 && number <= index => Some(number - 1),
Some(number) => bail!("message {} replies to message {number}, which is not an earlier one", index + 1),
None => None,
};
Ok(Self { from: message.from.clone(), name: message.name.clone(), room: message.room.clone(), at_ms: unix_ms(at), says: message.says.clone(), mentions, dm_to, follow_up: message.follow_up, reply_to, new: message.new })
}
/// What Matrix would say about this line to `viewer`: a direct message, a tag, a reply to a line of hers, or a
/// follow-up.
fn basis(&self, viewer: &str, lines: &[Line]) -> Option<Basis> {
if self.dm_to.as_deref() == Some(viewer) {
Some(Basis::Direct)
} else if self.mentions.iter().any(|id| id == viewer) {
Some(Basis::Tag)
} else if self.reply_to.and_then(|at| lines.get(at)).is_some_and(|replied| replied.from == viewer) {
Some(Basis::Reply)
} else if self.follow_up {
Some(Basis::FollowUp)
} else {
None
}
}
/// Whether `viewer` sees the line: a direct message only its sender and recipient; any other line every member of its
/// room (no rooms named is every room).
pub(crate) fn visible_to(&self, viewer: &str, rooms: &[String]) -> bool {
match &self.dm_to {
Some(to) => to == viewer || self.from == viewer,
None => rooms.is_empty() || rooms.contains(&self.room),
}
}
pub(crate) fn delivery(&self, index: usize, viewer: &str, lines: &[Line]) -> ChatDelivery {
ChatDelivery {
seq: index as u64 + 1,
room_alias: self.room.clone(),
room_id: self.room.clone(),
event_id: format!("$fixture-{index}"),
sender: self.from.clone(),
sender_display: self.name.clone(),
text: self.says.clone(),
at_unix_ms: self.at_ms,
context: vec![],
mentions: self.mentions.clone(),
is_direct: self.dm_to.is_some(),
image: None,
basis: self.basis(viewer, lines),
form: Form::Message,
reply_to: Default::default(),
}
}
}
/// The mouth's pending read as `viewer` would get it from `lines`: the lines `new` says she has not had, from others,
/// in her rooms, as deliveries; and every line of her rooms, hers included, as the tail.
pub(crate) fn pending_for(lines: &[Line], viewer: &str, rooms: &[String], new: impl Fn(usize) -> bool) -> ChatPending {
let hers = |line: &Line| line.visible_to(viewer, rooms);
let deliveries = lines.iter().enumerate().filter(|(at, line)| new(*at) && hers(line) && line.from != viewer).map(|(at, line)| line.delivery(at, viewer, lines)).collect();
let tail: Vec<TailLine> = lines
.iter()
.enumerate()
.filter(|(_, line)| hers(line))
.map(|(at, line)| TailLine { room: line.room.clone(), event_id: format!("$fixture-{at}"), at_unix_ms: line.at_ms, sender: line.from.clone(), form: Form::Message, basis: line.basis(viewer, lines), body: line.says.clone() })
.collect();
let tail = tail[tail.len().saturating_sub(TAIL_LINES)..].to_vec();
ChatPending { persona: localpart(viewer).to_owned(), deliveries, tail: Some(tail), tail_gaps: vec![] }
}
// ---------------------------------------------------------------------------------------------
// The room fixture.
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct FixtureFile {
her: String,
#[serde(default)]
names: Vec<String>,
kind: TurnKind,
now: DateTime<Utc>,
rooms: Vec<String>,
#[serde(default)]
here: Option<String>,
#[serde(default)]
operators: Vec<String>,
#[serde(default)]
minds: Vec<String>,
#[serde(default)]
last_heard: BTreeMap<String, String>,
#[serde(default, rename = "message")]
messages: Vec<Message>,
}
/// A mocked room: it stands in for the mouth and the room tail, the clock, and who is who.
#[derive(Clone, Debug)]
pub struct Fixture {
/// Her Matrix id.
pub her: String,
/// More names she answers to, beside her handle; none leaves her own names (handle and public name).
pub names: Vec<String>,
pub kind: TurnKind,
pub now: DateTime<Utc>,
pub here: Option<String>,
pub operators: Vec<String>,
pub minds: Vec<String>,
pub last_heard: Vec<(String, u64)>,
lines: Vec<Line>,
}
impl Fixture {
pub fn load(path: &Path) -> Result<Self> {
let text = std::fs::read_to_string(path).with_context(|| format!("the room fixture {} could not be read", path.display()))?;
Self::parse(&text).with_context(|| format!("the room fixture {} is not a room", path.display()))
}
pub fn parse(text: &str) -> Result<Self> {
let file: FixtureFile = toml::from_str(text)?;
if !matrix_id(&file.her) {
bail!("her = \"{}\" is not a Matrix id", file.her);
}
if file.rooms.is_empty() {
bail!("rooms names no room");
}
if let Some(here) = &file.here
&& !file.rooms.contains(here)
{
bail!("here = \"{here}\" is not one of the rooms");
}
let mut lines = Vec::new();
for (index, message) in file.messages.iter().enumerate() {
if message.round.is_some() {
bail!("message {} has a round: a room fixture's messages say how long ago, not which round", index + 1);
}
if !message.dm && !file.rooms.contains(&message.room) {
bail!("message {} is in {}, which is not one of the rooms", index + 1, message.room);
}
let at = file.now - ago(message.ago.as_deref().unwrap_or("0s"))?;
lines.push(Line::of(message, at, Some(&file.her), index)?);
}
let last_heard = file.last_heard.iter().map(|(id, when)| Ok((id.clone(), unix_ms(file.now - ago(when)?)))).collect::<Result<_>>()?;
Ok(Self { her: file.her, names: file.names, kind: file.kind, now: file.now, here: file.here, operators: file.operators, minds: file.minds, last_heard, lines })
}
/// Her addressee, as the heartbeat's table would carry it, when the fixture names her: her handle and its names.
/// None leaves `project` her own names.
pub fn addressee(&self) -> Option<&dyn HeartbeatView> {
(!self.names.is_empty()).then_some(self as &dyn HeartbeatView)
}
}
impl MouthReader for Fixture {
fn pending(&self) -> Result<ChatPending> {
Ok(pending_for(&self.lines, &self.her, &[], |at| self.lines[at].new))
}
}
impl HeartbeatView for Fixture {
fn addressing(&self) -> Result<Option<Addressing>> {
let names = std::iter::once(localpart(&self.her)).chain(self.names.iter().map(String::as_str));
Ok(Some(Addressing { minds: vec![Addressee::new(&self.her, names)] }))
}
}
// ---------------------------------------------------------------------------------------------
// What every harness run shares.
/// What a harness run reads through beside its fixtures: her stored state (the persona door; only a batch state without
/// a file asks), the model, and her tuning.
pub struct Harness<'a> {
pub instance: String,
pub persona: Option<&'a dyn PersonaSource>,
pub model: Option<&'a dyn Model>,
pub tuning: PathBuf,
}
/// One turn's `project` request over a fixture.
pub fn fixture_request(harness: &Harness, fixture: &Fixture, state: Option<Value>, seed: u64, renders: Option<usize>, through: Through) -> Request {
Request {
instance: harness.instance.clone(),
kind: fixture.kind,
state,
room: fixture.here.clone(),
seed,
renders,
judge_against: None,
operators: fixture.operators.clone(),
minds: fixture.minds.clone(),
own_sender: fixture.her.clone(),
tuning: harness.tuning.clone(),
through,
last_heard: fixture.last_heard.clone(),
recent: vec![],
own_rooms: vec![],
}
}
// ---------------------------------------------------------------------------------------------
// Batch.
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct BatchFile {
#[serde(default = "one")]
samples: usize,
#[serde(default)]
seed: u64,
#[serde(default)]
through: Through,
#[serde(rename = "state")]
states: Vec<StateEntry>,
#[serde(rename = "scenario")]
scenarios: Vec<ScenarioEntry>,
}
fn one() -> usize {
1
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct StateEntry {
name: String,
/// A state file, relative to the plan; none is her stored state, read through the persona door.
#[serde(default)]
file: Option<PathBuf>,
}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct ScenarioEntry {
name: String,
/// A room fixture, relative to the plan.
room: PathBuf,
}
fn relative(plan: &Path, path: &Path) -> PathBuf {
if path.is_absolute() { path.to_owned() } else { plan.parent().unwrap_or(Path::new(".")).join(path) }
}
/// Every state by every scenario by every sample, each one `project`, as one Markdown report. A case that fails is
/// reported in its place and the batch goes on.
pub fn batch(harness: &Harness, plan_path: &Path) -> Result<String> {
let text = std::fs::read_to_string(plan_path).with_context(|| format!("the plan {} could not be read", plan_path.display()))?;
let plan: BatchFile = toml::from_str(&text).with_context(|| format!("the plan {} is not a batch plan", plan_path.display()))?;
if plan.samples == 0 || plan.states.is_empty() || plan.scenarios.is_empty() {
bail!("the plan {} needs a state, a scenario and at least one sample", plan_path.display());
}
let states: Vec<(String, Option<Value>)> = plan.states.iter().map(|entry| Ok((entry.name.clone(), entry.file.as_ref().map(|file| state_file(&relative(plan_path, file), &harness.instance)).transpose()?))).collect::<Result<_>>()?;
let scenarios: Vec<(String, Fixture)> = plan.scenarios.iter().map(|entry| Ok((entry.name.clone(), Fixture::load(&relative(plan_path, &entry.room))?))).collect::<Result<_>>()?;
let mut cases = Vec::new();
for (state_name, state) in &states {
for (scenario_name, fixture) in &scenarios {
for sample in 0..plan.samples {
let seed = plan.seed.wrapping_add(sample as u64);
let organs = Organs { persona: harness.persona, chat: Some(fixture), addressing: fixture.addressee(), model: harness.model };
let request = fixture_request(harness, fixture, state.clone(), seed, Some(1), plan.through);
let report = project(&organs, &request, fixture.now).unwrap_or_else(|fault| json!({ "error": format!("{fault:#}") }));
cases.push(json!({ "state": state_name, "scenario": scenario_name, "sample": sample + 1, "seed": seed, "report": report }));
}
}
}
Ok(batch_markdown(plan_path, &plan, &cases))
}
fn render_of(case: &Value) -> &Value {
&case["report"]["renders"][0]
}
fn prose_of(case: &Value) -> String {
render_of(case)["persona"]["prose"].as_str().unwrap_or_default().to_owned()
}
fn batch_markdown(plan_path: &Path, plan: &BatchFile, cases: &[Value]) -> String {
let mut out = String::new();
let first = cases.iter().find(|case| case["report"].get("error").is_none()).map(|case| &case["report"]);
let _ = writeln!(out, "# Persona seed batch\n");
let _ = writeln!(out, "- plan: `{}`", plan_path.display());
let _ = writeln!(out, "- states: {}; scenarios: {}; samples: {}; through: {:?}", plan.states.len(), plan.scenarios.len(), plan.samples, plan.through);
if let Some(report) = first {
let _ = writeln!(out, "- instance: `{}`; tuning: {} (`{}`)", report["instance"].as_str().unwrap_or_default(), report["tuning"]["source"].as_str().unwrap_or_default(), report["tuning"]["sha256"].as_str().unwrap_or_default());
let _ = writeln!(out, "- Projector model: `{}`; Persona model: `{}`", report["model"].as_str().unwrap_or("none"), report["persona_model"].as_str().unwrap_or("none"));
}
let _ = writeln!(out, "- would-say lines are the double-quoted spans of her prose: a preview of what the Interpreter (C3b) will quote, not its decision");
let _ = writeln!(out, "- read-only: no put, no trace, no send; the mouth is the fixture\n");
// The summary: each state by scenario, then the states' voices against each other.
let _ = writeln!(out, "## Summary\n\n| state | scenario | spoke | mean narration words | mean prose words |\n|---|---|---|---|---|");
for state in &plan.states {
for scenario in &plan.scenarios {
let mine: Vec<&Value> = cases.iter().filter(|case| case["state"] == state.name.as_str() && case["scenario"] == scenario.name.as_str()).collect();
let spoke = mine.iter().filter(|case| render_of(case)["interpreter"]["said"].as_array().is_some_and(|said| !said.is_empty())).count();
let mean = |key: &dyn Fn(&Value) -> Option<u64>| mine.iter().filter_map(|case| key(render_of(case))).sum::<u64>() as f64 / mine.len().max(1) as f64;
let _ = writeln!(out, "| {} | {} | {spoke}/{} | {:.0} | {:.0} |", state.name, scenario.name, mine.len(), mean(&|render| render["words"].as_u64()), mean(&|render| render["persona"]["words"].as_u64()));
}
}
let corpora: Vec<(String, Vec<String>)> = plan.states.iter().map(|state| (state.name.clone(), cases.iter().filter(|case| case["state"] == state.name.as_str()).map(prose_of).filter(|prose| !prose.is_empty()).collect())).collect();
let _ = writeln!(out, "\nDistinct-2 of her prose within each state (1 is no repeated word pair), and over all states pooled:\n");
for (name, prose) in &corpora {
let _ = writeln!(out, "- {name}: {:.3}", distinct_2(prose));
}
let _ = writeln!(out, "- pooled: {:.3}", distinct_2(&corpora.iter().flat_map(|(_, prose)| prose.clone()).collect::<Vec<_>>()));
let _ = writeln!(out, "\nWord-use similarity between states' prose (cosine of word counts, 1 is the same words in the same proportions):\n");
for (at, (a, a_prose)) in corpora.iter().enumerate() {
for (b, b_prose) in &corpora[at + 1..] {
let _ = writeln!(out, "- {a} / {b}: {:.3}", cosine(&a_prose.join(" "), &b_prose.join(" ")));
}
}
for case in cases {
let _ = writeln!(out, "\n## {} × {}, sample {} (seed {})\n", case["state"].as_str().unwrap_or_default(), case["scenario"].as_str().unwrap_or_default(), case["sample"], case["seed"]);
let report = &case["report"];
if let Some(error) = report.get("error") {
let _ = writeln!(out, "**failed:** {}", error.as_str().unwrap_or_default());
continue;
}
let packet = &report["packet"];
let _ = writeln!(out, "- lens: `{}`; present: {}; room: {}", packet["lens"], packet["present"], report["room"]);
let _ = writeln!(out, "\n<details><summary>packet</summary>\n");
fenced(&mut out, "identity", packet["identity"].as_str().unwrap_or_default());
fenced(&mut out, "context", packet["context"].as_str().unwrap_or_default());
fenced(&mut out, "stimulus", packet["stimulus"].as_str().unwrap_or_default());
fenced(&mut out, "heard", packet["heard"].as_str().unwrap_or_default());
let _ = writeln!(out, "</details>\n");
turn_markdown(&mut out, render_of(case));
}
let _ = writeln!(out, "\n## Total usage\n\n{}", usage_markdown(&usage_total(cases)));
out
}
/// One render's narration, prose, would-say lines, word counts and usage.
pub(crate) fn turn_markdown(out: &mut String, render: &Value) {
if let Some(failed) = render["failed"].as_str() {
let _ = writeln!(out, "**Projector call failed:** {failed}
Usage: {}
", usage_line(&render["usage"]));
return;
}
let _ = writeln!(out, "**Narration** ({} words{}):\n", render["words"], if render["cut_at_budget"] == true { ", cut at budget" } else { "" });
quoted(out, render["narration"].as_str().unwrap_or_default());
if let Some(names) = render["field_names"].as_array().filter(|names| !names.is_empty()) {
let _ = writeln!(out, "**Field names leaked:** {}\n", Value::Array(names.clone()));
}
let _ = writeln!(
out,
"Narration checks: lexical overlap with what she heard {}{}{}\n",
render["heard_overlap"].as_f64().map_or_else(|| "n/a".to_owned(), |overlap| format!("{overlap:.2}")),
if render["heard_overlap"].as_f64().is_some_and(|overlap| overlap > OVERLAP_FLAG) { " (high)" } else { "" },
if render["offers_options"] == true { "; offers her options" } else { "" }
);
let persona = &render["persona"];
if let Some(failed) = persona["failed"].as_str() {
let _ = writeln!(out, "**Persona call failed:** {failed}
");
} else if !persona.is_null() {
let _ = writeln!(out, "**Her prose** ({} words{}):\n", persona["words"], if persona["cut_at_budget"] == true { ", cut at budget" } else { "" });
quoted(out, persona["prose"].as_str().unwrap_or_default());
}
interpreter_markdown(out, &render["interpreter"]);
let _ = writeln!(out, "Usage: projector {}; persona {}; interpreter {}\n", usage_line(&render["usage"]), usage_line(&persona["usage"]), usage_line(&render["interpreter"]["usage"]));
}
/// The Interpreter's speech decision: each line said with its room, mentions, reply target and the span of her prose it
/// rests on; each line refused, with why; its gaps. None when the render did not run through the Interpreter.
pub(crate) fn interpreter_markdown(out: &mut String, decision: &Value) {
if decision.is_null() {
return;
}
if let Some(failed) = decision["failed"].as_str() {
let _ = writeln!(out, "**Interpreter failed:** {failed}\n");
return;
}
let said = decision["said"].as_array().cloned().unwrap_or_default();
if said.is_empty() {
let _ = writeln!(out, "**Interpreter: nothing said.**\n");
} else {
let _ = writeln!(out, "**Interpreter: said**\n");
for line in &said {
let mentions = line["mentions"].as_array().map(|ids| ids.iter().filter_map(Value::as_str).collect::<Vec<_>>().join(", ")).unwrap_or_default();
let reply = line["reply_to"].as_str().map_or_else(String::new, |id| format!("; reply to {id}"));
let _ = writeln!(out, "- in {}{}{}: {}", line["room"].as_str().unwrap_or_default(), if mentions.is_empty() { String::new() } else { format!(", mentioning {mentions}") }, reply, line["text"].as_str().unwrap_or_default());
let spans: Vec<String> = line["rests_on"].as_array().into_iter().flatten().map(|span| format!("\"{}\"", span["quote"].as_str().unwrap_or_default())).collect();
let _ = writeln!(out, " - her words: {}", spans.join(" + "));
}
out.push('\n');
}
for line in decision["refused"].as_array().into_iter().flatten() {
let what = line["text"].as_str().map_or_else(|| line["spans"].to_string(), str::to_owned);
let _ = writeln!(out, "- refused ({}): {what}", line["why"].as_str().unwrap_or_default());
}
for gap in decision["gaps"].as_array().into_iter().flatten() {
let _ = writeln!(out, "- gap: {gap}");
}
out.push('\n');
}
fn fenced(out: &mut String, label: &str, text: &str) {
if !text.is_empty() {
let _ = writeln!(out, "{label}:\n\n````text\n{text}\n````\n");
}
}
fn quoted(out: &mut String, text: &str) {
for line in text.lines() {
let _ = writeln!(out, "> {line}");
}
out.push('\n');
}
// ---------------------------------------------------------------------------------------------
// Usage and similarity, shared with the simulation.
const USAGE_FIELDS: [&str; 5] = ["input_tokens", "output_tokens", "cache_read_tokens", "cache_write_tokens", "reasoning_tokens"];
/// Every `usage` object anywhere in `values`, summed per model and field, with the number of calls.
pub(crate) fn usage_total(values: &[Value]) -> BTreeMap<String, BTreeMap<String, u64>> {
fn walk(value: &Value, into: &mut BTreeMap<String, BTreeMap<String, u64>>) {
match value {
Value::Object(map) => {
for (key, inner) in map {
if key == "usage" && inner.is_object() {
let model = inner["model"].as_str().unwrap_or("unknown").to_owned();
let row = into.entry(model).or_default();
*row.entry("calls".into()).or_default() += 1;
for field in USAGE_FIELDS {
*row.entry(field.into()).or_default() += inner[field].as_u64().unwrap_or(0);
}
} else {
walk(inner, into);
}
}
}
Value::Array(items) => items.iter().for_each(|inner| walk(inner, into)),
_ => {}
}
}
let mut total = BTreeMap::new();
values.iter().for_each(|value| walk(value, &mut total));
total
}
pub(crate) fn usage_markdown(total: &BTreeMap<String, BTreeMap<String, u64>>) -> String {
if total.is_empty() {
return "No model call was counted.".into();
}
let mut out = String::from("| model | calls | input | output | cache read | cache write | reasoning |\n|---|---|---|---|---|---|---|\n");
for (model, row) in total {
let get = |field: &str| row.get(field).copied().unwrap_or(0);
let _ = writeln!(out, "| {model} | {} | {} | {} | {} | {} | {} |", get("calls"), get("input_tokens"), get("output_tokens"), get("cache_read_tokens"), get("cache_write_tokens"), get("reasoning_tokens"));
}
out
}
fn usage_line(usage: &Value) -> String {
if usage.is_null() {
return "none".into();
}
format!("{} in, {} out, {} cache read", usage["input_tokens"], usage["output_tokens"], usage["cache_read_tokens"])
}
/// The words of a text as counted for similarity: lowercased, letters and digits, three characters or more.
fn word_counts(text: &str) -> BTreeMap<String, f64> {
let mut counts = BTreeMap::new();
for word in text.split(|c: char| !c.is_alphanumeric()).filter(|word| word.chars().count() >= 3) {
*counts.entry(word.to_lowercase()).or_insert(0.0) += 1.0;
}
counts
}
/// The cosine of two texts' word counts: 1 is the same words in the same proportions, 0 no word shared. A cheap
/// lexical proxy for one voice, not a measure of meaning.
pub(crate) fn cosine(a: &str, b: &str) -> f64 {
let (a, b) = (word_counts(a), word_counts(b));
let dot: f64 = a.iter().filter_map(|(word, count)| b.get(word).map(|other| count * other)).sum();
let norm = |counts: &BTreeMap<String, f64>| counts.values().map(|count| count * count).sum::<f64>().sqrt();
let denominator = norm(&a) * norm(&b);
if denominator == 0.0 { 0.0 } else { dot / denominator }
}
#[cfg(test)]
pub(crate) mod tests;

View file

@ -0,0 +1,301 @@
use std::cell::RefCell;
use std::path::{Path, PathBuf};
use anyhow::Result;
use serde_json::{Value, json};
use super::*;
use crate::address::is_for;
use crate::projection::narrate::{Answer, Failed, Spent};
const HER: &str = "@eureka:gamecult.org";
pub(crate) fn fixtures() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("fixtures/harness")
}
pub(crate) fn control() -> Value {
serde_json::from_str(&std::fs::read_to_string(fixtures().join("states/control.json")).unwrap()).unwrap()
}
fn room(name: &str) -> Fixture {
Fixture::load(&fixtures().join("rooms").join(name)).unwrap()
}
/// A recorded model: the Projector narrates a fixed line; the Persona stage answers by `persona(prompt)`; the
/// Interpreter by `interpreter(prompt)`, by default `quoted_lines`. Every call is kept, with its usage counted.
pub(crate) struct Scripted {
pub calls: RefCell<Vec<(String, String)>>,
pub persona: Box<dyn Fn(&str) -> String>,
pub interpreter: Box<dyn Fn(&str) -> String>,
}
impl Scripted {
pub(crate) fn new(persona: impl Fn(&str) -> String + 'static) -> Self {
Self { calls: RefCell::default(), persona: Box::new(persona), interpreter: Box::new(quoted_lines) }
}
pub(crate) fn interpreting(mut self, interpreter: impl Fn(&str) -> String + 'static) -> Self {
self.interpreter = Box::new(interpreter);
self
}
}
/// The recorded Interpreter of the tests: each double-quoted span of her prose is a line she says, citing itself,
/// mentioning no one. A test double that keeps the tests' prose readable, not the speech rule: to code a quotation is
/// only a text whose sentences are whole, and the live Interpreter is a model.
pub(crate) fn quoted_lines(prompt: &str) -> String {
let prose = prompt.split("Her prose:\n").nth(1).unwrap_or_default();
let say: Vec<Value> = prose.split('"').enumerate().filter(|(at, part)| at % 2 == 1 && !part.trim().is_empty()).map(|(_, part)| json!({ "spans": [part], "mentions": [], "reply_to": null })).collect();
json!({ "say": say }).to_string()
}
impl Model for Scripted {
fn ask(&self, _request_id: &str, model: &str, prompt: &str) -> std::result::Result<Answer, Failed> {
self.calls.borrow_mut().push((model.to_owned(), prompt.to_owned()));
let text = if prompt.starts_with("You decide what one person says aloud") {
(self.interpreter)(prompt)
} else if prompt.starts_with("<!-- membrane:") {
"The room is slow and so is he.".to_owned()
} else {
(self.persona)(prompt)
};
Ok(Answer { text, usage: Some(Spent { model: model.to_owned(), input_tokens: 10, output_tokens: 2, cache_read_tokens: 1, cache_write_tokens: 0, reasoning_tokens: None }) })
}
}
/// Fails the test if her stored state is asked for.
pub(crate) struct NeverAsked;
impl PersonaSource for NeverAsked {
fn persona(&self) -> Result<(Value, String)> {
panic!("the stored state was read")
}
}
fn harness<'a>(dir: &Path, persona: Option<&'a dyn PersonaSource>, model: Option<&'a dyn Model>) -> Harness<'a> {
Harness { instance: "eureka".into(), persona, model, tuning: dir.join("persona-tuning.toml") }
}
#[test]
fn a_room_fixture_parses_into_the_mouths_pending_read() {
let fixture = room("operator-warm.toml");
assert_eq!((fixture.her.as_str(), fixture.kind, fixture.here.as_deref()), (HER, TurnKind::Chat, Some("!dm-meta:gamecult.org")));
let pending = MouthReader::pending(&fixture).unwrap();
assert_eq!(pending.deliveries.len(), 1, "only the new message is delivered");
let dm = &pending.deliveries[0];
assert_eq!((dm.sender.as_str(), dm.sender_display.as_str(), dm.is_direct, dm.basis), ("@meta:gamecult.org", "Meta", true, Some(Basis::Direct)));
assert!(dm.text.contains("\"busy\""), "a quoted body survives TOML: {}", dm.text);
assert_eq!(dm.at_unix_ms, unix_ms(fixture.now - Duration::minutes(1)));
assert_eq!(pending.tail.unwrap().len(), 2, "the tail holds every line, delivered or not");
assert_eq!(fixture.last_heard, vec![("@meta:gamecult.org".to_owned(), unix_ms(fixture.now - Duration::days(2)))]);
}
#[test]
fn is_for_decides_whether_a_fixture_message_is_for_her() {
let her = Addressee::new(HER, ["eureka"]);
let for_her = |fixture: &Fixture| MouthReader::pending(fixture).unwrap().deliveries.iter().map(|delivery| is_for(&her, delivery)).collect::<Vec<_>>();
// Kiko's reply tags her and answers her own line.
let teased = room("mind-teases.toml");
let delivery = &MouthReader::pending(&teased).unwrap().deliveries[0];
assert_eq!((delivery.basis, delivery.mentions.as_slice()), (Some(Basis::Tag), [HER.to_owned()].as_slice()));
assert_eq!(for_her(&teased), vec![true]);
// Deru's aside is not.
assert_eq!(for_her(&room("deru-passing.toml")), vec![false]);
// A name in the text is, a reply to her line is, and an aside is not.
let fixture = Fixture::parse(
r##"
her = "@eureka:gamecult.org"
kind = "chat"
now = "2026-11-20T15:00:00Z"
rooms = ["#minds:gamecult.org"]
[[message]]
from = "@eureka:gamecult.org"
room = "#minds:gamecult.org"
ago = "1h"
says = "mine"
[[message]]
from = "@deru:gamecult.org"
room = "#minds:gamecult.org"
new = true
says = "Eureka, tea?"
[[message]]
from = "@deru:gamecult.org"
room = "#minds:gamecult.org"
new = true
reply_to = 1
says = "agreed"
[[message]]
from = "@deru:gamecult.org"
room = "#minds:gamecult.org"
new = true
says = "the kettle again"
"##,
)
.unwrap();
assert_eq!(for_her(&fixture), vec![true, true, false]);
}
#[test]
fn a_fixture_that_is_not_a_room_is_refused_naming_what_is_wrong() {
let head = "her = \"@eureka:gamecult.org\"\nkind = \"chat\"\nnow = \"2026-11-20T15:00:00Z\"\nrooms = [\"#a:x\"]\n";
for (tail, names) in [
("[[message]]\nfrom = \"@d:x\"\nroom = \"#b:x\"\nsays = \"hi\"\n", "not one of the rooms"),
("[[message]]\nfrom = \"@d:x\"\nroom = \"#a:x\"\nreply_to = 1\nsays = \"hi\"\n", "not an earlier one"),
("[[message]]\nfrom = \"@d:x\"\nroom = \"#a:x\"\nround = 2\nsays = \"hi\"\n", "round"),
("[[message]]\nfrom = \"@d:x\"\nroom = \"#a:x\"\nago = \"soon\"\nsays = \"hi\"\n", "90s"),
("[[message]]\nfrom = \"deru\"\nroom = \"#a:x\"\nsays = \"hi\"\n", "Matrix id"),
("mood = \"sunny\"\n", "unknown field"),
] {
let fault = format!("{:#}", Fixture::parse(&format!("{head}{tail}")).unwrap_err());
assert!(fault.contains(names), "{tail}: {fault}");
}
}
#[test]
fn through_the_interpreter_her_prose_continues_the_narration_and_the_interpreter_decides_what_is_said() {
let dir = tempfile::tempdir().unwrap();
let model = Scripted::new(|_| "He smiles at the screen, then types. \"Missed you too, friend.\" He does not mention the kettle.".into());
let fixture = room("operator-warm.toml");
let harness = harness(dir.path(), Some(&NeverAsked), Some(&model));
let organs = Organs { persona: harness.persona, chat: Some(&fixture), addressing: fixture.addressee(), model: harness.model };
let report = project(&organs, &fixture_request(&harness, &fixture, Some(control()), 3, Some(1), Through::Interpreter), fixture.now).unwrap();
let render = &report["renders"][0];
assert_eq!(render["interpreter"]["said"][0]["text"], "Missed you too, friend.", "{render}");
assert_eq!(render["interpreter"]["said"][0]["room"], "!dm-meta:gamecult.org");
assert_eq!(report["interpreter_model"], "claude-sonnet-5-5");
assert_eq!(render["persona"]["words"], 17);
assert_eq!((render["usage"]["input_tokens"].as_u64(), render["persona"]["usage"]["model"].as_str()), (Some(10), Some("claude-sonnet-5-5")));
assert_eq!(report["persona_model"], "claude-sonnet-5-5");
let calls = model.calls.borrow();
assert_eq!(calls.len(), 3, "one Projector call, one Persona call, one Interpreter call");
let persona_prompt = &calls[1].1;
assert!(persona_prompt.starts_with("Everything below is prose from inside one person's life."), "Ghostlight's Persona membrane");
assert!(persona_prompt.contains("The room is slow and so is he.") && persona_prompt.contains("hey you. I missed talking to you this week"), "the narration, then what she heard verbatim: {persona_prompt}");
assert!(persona_prompt.contains("Use at most 180 words"), "the map's Persona budget");
for field in ["socialBonds", "currentActivation", "bond-operator", "trust\":"] {
assert!(!persona_prompt.contains(field), "no field name reaches the Persona: {field}");
}
}
#[test]
fn prose_over_budget_is_cut_and_silence_says_nothing() {
let dir = tempfile::tempdir().unwrap();
let long = format!("{} \"late words\"", "word ".repeat(200));
let model = Scripted::new(move |_| long.clone());
let fixture = room("deru-passing.toml");
let harness = harness(dir.path(), None, Some(&model));
let organs = Organs { persona: None, chat: Some(&fixture), addressing: None, model: harness.model };
let report = project(&organs, &fixture_request(&harness, &fixture, Some(control()), 1, Some(1), Through::Interpreter), fixture.now).unwrap();
let persona = &report["renders"][0]["persona"];
assert_eq!((persona["words"].as_u64(), persona["cut_at_budget"].as_bool()), (Some(180), Some(true)));
assert_eq!(report["renders"][0]["interpreter"]["said"], json!([]), "words past the budget are not her prose, so nothing rests on them");
}
#[test]
fn a_batch_runs_every_state_by_scenario_by_sample_into_one_report() {
let dir = tempfile::tempdir().unwrap();
let plan = dir.path().join("plan.toml");
let rooms = fixtures().join("rooms");
std::fs::write(
&plan,
format!(
"samples = 2\nseed = 7\nthrough = \"interpreter\"\n[[state]]\nname = \"control\"\nfile = {:?}\n[[scenario]]\nname = \"quiet\"\nroom = {:?}\n[[scenario]]\nname = \"teased\"\nroom = {:?}\n",
fixtures().join("states/control.json").display().to_string(),
rooms.join("quiet-beat.toml").display().to_string(),
rooms.join("mind-teases.toml").display().to_string()
),
)
.unwrap();
let model = Scripted::new(|prompt| if prompt.contains("Kiko") { "\"Winter, still.\" He grins.".into() } else { "He lets the hour pass.".into() });
let report = batch(&harness(dir.path(), Some(&NeverAsked), Some(&model)), &plan).unwrap();
for heading in ["# Persona seed batch", "## Summary", "## control × quiet, sample 1 (seed 7)", "## control × quiet, sample 2 (seed 8)", "## control × teased, sample 2 (seed 8)", "## Total usage"] {
assert!(report.contains(heading), "{heading}\n{report}");
}
assert!(report.contains("| control | teased | 2/2 |") && report.contains("| control | quiet | 0/2 |"), "{report}");
assert!(report.contains("- in #minds:gamecult.org: Winter, still.") && report.contains(" - her words: \"Winter, still.\"") && report.contains("**Interpreter: nothing said.**"), "{report}");
assert!(report.contains("| claude-sonnet-5-5 | 12 | 120 | 24 |"), "twelve calls, three a case, in the footer: {report}");
assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1, "the batch wrote nothing; the caller writes the report");
}
#[test]
fn a_batch_state_without_a_file_is_her_stored_state() {
struct Stored;
impl PersonaSource for Stored {
fn persona(&self) -> Result<(Value, String)> {
Ok((control(), "mind-commit-x".into()))
}
}
let dir = tempfile::tempdir().unwrap();
let plan = dir.path().join("plan.toml");
std::fs::write(&plan, format!("[[state]]\nname = \"stored\"\n[[scenario]]\nname = \"quiet\"\nroom = {:?}\n", fixtures().join("rooms/quiet-beat.toml").display().to_string())).unwrap();
let model = Scripted::new(|_| String::new());
let report = batch(&harness(dir.path(), Some(&Stored), Some(&model)), &plan).unwrap();
assert!(report.contains("## stored × quiet, sample 1") && !report.contains("failed"), "{report}");
}
fn group_line_of(dir: &Path, name: &str) -> Option<String> {
let fixture = room(name);
let harness = harness(dir, None, None);
let organs = Organs { persona: None, chat: Some(&fixture), addressing: fixture.addressee(), model: None };
let report = project(&organs, &fixture_request(&harness, &fixture, Some(control()), 1, None, Through::Projector), fixture.now).unwrap();
report["packet"]["context"].as_str().unwrap().lines().find(|line| line.starts_with("- the conversation")).map(str::to_owned)
}
#[test]
fn the_group_state_reads_each_scenario_room_as_the_golden_file_says() {
let dir = tempfile::tempdir().unwrap();
let mut rendered = String::new();
for name in ["quiet-beat.toml", "operator-warm.toml", "mind-teases.toml", "deru-passing.toml"] {
let line = group_line_of(dir.path(), name).unwrap_or_else(|| "(none)".into());
assert!(line.len() <= 360, "the slot holds its byte budget: {line}");
rendered.push_str(&format!("{name}: {line}
"));
}
let golden = include_str!("../../../fixtures/harness/golden-group.txt").replace("
", "
");
assert_eq!(rendered, golden, "the group state moved; if deliberately, the new golden file is:
{rendered}");
}
#[test]
fn the_group_state_is_off_when_the_tuning_says_so() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("persona-tuning.toml"), "[select]
group_state = false
").unwrap();
assert!(group_line_of(dir.path(), "mind-teases.toml").is_none());
}
/// Round 1 of a crowded room: the operator asks everyone, and `answered` Minds have answered before this one's turn.
fn chorus(answered: usize) -> Fixture {
let minds = ["@weksa:gamecult.org", "@heimdall:gamecult.org", "@kiko:gamecult.org", "@libby:gamecult.org", "@void:gamecult.org", "@nibu:gamecult.org", "@aqua:gamecult.org"];
let mut text = String::from("her = \"@eureka:gamecult.org\"\nkind = \"chat\"\nnow = \"2026-11-20T15:05:00Z\"\nrooms = [\"#minds:gamecult.org\"]\nhere = \"#minds:gamecult.org\"\noperators = [\"@meta:gamecult.org\"]\nminds = [\"@eureka:gamecult.org\", \"@weksa:gamecult.org\", \"@heimdall:gamecult.org\", \"@kiko:gamecult.org\", \"@libby:gamecult.org\", \"@void:gamecult.org\", \"@nibu:gamecult.org\", \"@aqua:gamecult.org\"]\n");
text.push_str("[[message]]\nfrom = \"@meta:gamecult.org\"\nname = \"Meta\"\nroom = \"#minds:gamecult.org\"\nago = \"5m\"\nnew = true\nsays = \"afternoon, all of you. no tasks today. how's everyone doing?\"\n");
for (at, mind) in minds.iter().take(answered).enumerate() {
text.push_str(&format!("[[message]]\nfrom = \"{mind}\"\nname = \"{}\"\nroom = \"#minds:gamecult.org\"\nago = \"{}s\"\nnew = true\nsays = \"afternoon, Meta. doing well, thanks.\"\n", crate::address::localpart(mind), 240 - at * 20));
}
Fixture::parse(&text).unwrap()
}
#[test]
fn the_third_to_eighth_responder_sees_a_covered_question_and_no_duty() {
let dir = tempfile::tempdir().unwrap();
let harness = harness(dir.path(), None, None);
let mut rendered = String::new();
for answered in 2..=7 {
let fixture = chorus(answered);
let organs = Organs { persona: None, chat: Some(&fixture), addressing: None, model: None };
let report = project(&organs, &fixture_request(&harness, &fixture, Some(control()), 1, None, Through::Projector), fixture.now).unwrap();
let context = report["packet"]["context"].as_str().unwrap().to_owned();
let line = context.lines().find(|line| line.starts_with("- the conversation")).unwrap_or_else(|| panic!("{context}")).to_owned();
assert!(line.contains("asked the room a question, and") && line.contains("answered it"), "{line}");
for duty in ["owe", "haven't", "have not", "open", "still", "waiting on her", "to her", "unanswered"] {
assert!(!line.contains(duty), "responder {}: {duty} in {line}", answered + 1);
}
rendered.push_str(&format!("responder {}: {line}\n", answered + 1));
}
let golden = include_str!("../../../fixtures/harness/golden-chorus.txt").replace("\r\n", "\n");
assert_eq!(rendered, golden, "the chorus's group state moved; if deliberately, the new golden file is:\n{rendered}");
}

View file

@ -0,0 +1,275 @@
//! The Interpreter's speech (map section 4, cut C3b): what she says aloud this turn, decided from her prose by one
//! tool-less structured model call, and built by code from her own words.
//!
//! The Persona writes her thought, and when she means to say something she writes the words she would type. The
//! Interpreter decides whether she says them now: it answers with zero or more lines, each a list of spans of her prose
//! cited verbatim, with the people she addresses (Matrix mentions and a reply target, as structured fields). Code builds
//! each line, never the model: each span is located in her prose by Ghostlight's `SourceSpan::locate` and must be whole
//! sentences (UAX #29 sentence segmentation, the Unicode family `locate`'s word boundaries come from), and the spans of
//! one line must be consecutive sentences; the line is that one unbroken run of her prose. A quotation's inside is
//! segmented as a text of its own, so a quoted sentence is whole, but only a standalone quotation's: one that is itself
//! whole sentences of her prose. A line may not start or end inside a quotation her sentence frames ("I won't say
//! \"Nibu is lying.\"", "If she pushes, I'll say \"Fine.\""): that is what she would or would not say. An opening
//! mark a budget left unclosed is not part of the sentence it starts. So a line is always whole sentences of her own,
//! verbatim and in their order: no stitching, no dropped negation, no condition lifted off its clause. What this gives
//! up: indirect speech ("I'll tell her I'm fine" says nothing), any paraphrase by the Interpreter, half a sentence, and
//! a line quoted inside a sentence of narration ("\"Fine,\" I tell her."): she says a sentence whole or not at all.
//!
//! Code keeps a line only when it is built so; drops an echo (at least `ECHO_MIN_WORDS` words, at least `ECHO_OVERLAP`
//! of its word trigrams in the bodies the stimulus quoted to her); keeps at most `MAX_LINES` distinct lines; and keeps a
//! mention or a reply target only for someone this packet showed her (`Packet::people`). She speaks only in the turn's
//! room. Whether a line tells what she knows privately is a reading of its meaning, so it is the Interpreter's
//! judgement over the private items the packet lists to it (`Packet::private`), never a word match here. Every refusal
//! and drop is reported. Nothing here posts: the commit (C3c) will.
use ghostlight_persona_projection::SourceSpan;
use serde::Deserialize;
use serde_json::{Value, json};
use unicode_segmentation::UnicodeSegmentation;
use super::narrate::{Model, heard_overlap, lived_stream};
use super::select::Packet;
use super::tuning::Tuning;
/// The most lines she says in one turn.
pub(crate) const MAX_LINES: usize = 2;
/// A line of at least this many words whose word trigrams are at least this share of what she heard is an echo, not
/// her speech.
const ECHO_MIN_WORDS: usize = 8;
const ECHO_OVERLAP: f64 = 0.8;
/// What to say, where, to whom, and the gaps: Huginn's chat Interpreter, beside nothing of Ghostlight's but its span rule.
const SPEECH_PROMPT: &str = "You decide what one person says aloud in a chat room this turn, from her own prose. You never write her words: you cite them.
Below are who she is, the room and who is in it, what reached her, her lived moment, and her prose. Decide which words of her prose she chooses to say aloud now, in this room. A line is speech only when her prose shows her choosing to say it now. Wondering whether to speak, deciding to wait or stay quiet, or saying that she says nothing, is not speech. Words she recalls or quotes from someone else are never her speech.
She speaks only in this room. What she knows privately is listed below as private: she says none of it here, and nothing that gives it away. A private matter may be said plainly only to the person it concerns, in a direct message with exactly that person; nothing listed is that here. Whether a line fits who is listening is yours to judge.
Answer with one JSON object and nothing else:
{\"say\": [{\"spans\": [\"<whole sentences of her prose, copied character for character>\"], \"mentions\": [\"<Matrix id of someone listed whom she addresses directly>\"], \"reply_to\": \"<Matrix id of the person whose message she answers, or null>\"}], \"gaps\": [{\"kind\": \"conditional | addressee | elsewhere\", \"quote\": \"<exact words of her prose>\", \"detail\": \"<why>\"}]}
A line is whole sentences of her prose, the words she would type and nothing she only describes: each span is one sentence or several, from its first character to its last, and the spans of one line are consecutive sentences. A quotation that stands as sentences of its own may be cited sentence by sentence; words quoted inside a sentence of narration are cited only with that whole sentence, and then only if that sentence is what she says. She never says half a sentence; if only part of a sentence is speech, she says nothing of it. At most two lines. Record a gap instead of guessing: conditional when she might say something but has not chosen to, addressee when she means to say it but whom to is unclear, elsewhere when she means it for another room or another time. An empty say list is the usual answer in a busy room.";
#[derive(Debug, Deserialize)]
struct Answer {
#[serde(default)]
say: Vec<Proposed>,
#[serde(default)]
gaps: Vec<Value>,
}
#[derive(Debug, Deserialize)]
struct Proposed {
#[serde(default)]
spans: Vec<String>,
#[serde(default)]
mentions: Vec<String>,
#[serde(default)]
reply_to: Option<String>,
}
pub(crate) fn speech_prompt(packet: &Packet, narration: &str, prose: &str) -> String {
let mut people = String::new();
for (label, id) in &packet.people {
people.push_str(&format!("- {label} \u{2014} {id}\n"));
}
if people.is_empty() {
people.push_str("- no one else has spoken here lately\n");
}
let mut private: String = packet.private.iter().map(|words| format!("- private: {words}\n")).collect();
if private.is_empty() {
private.push_str("- none\n");
}
format!(
"{SPEECH_PROMPT}\n\nWho she is:\n{}\n\nThe room: {} ({}).\nWho is in it, as label \u{2014} Matrix id:\n{people}\nWhat she knows privately, none of it to be said here:\n{private}\nWhat reached her:\n{}\n\nHer lived moment:\n{}\n\nHer prose:\n{prose}",
packet.identity,
packet.room.as_deref().unwrap_or("none"),
packet.room_kind.words(),
packet.stimulus,
lived_stream(narration, &packet.heard),
)
}
/// The model's answer as JSON: the object itself, or inside a fenced block.
fn json_of(text: &str) -> Option<Answer> {
let start = text.find('{')?;
let end = text.rfind('}')?;
serde_json::from_str(&text[start..=end]).ok()
}
/// A quotation in her prose: where its opening mark starts, its inside, and where its closing mark ends.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct Quotation {
open: usize,
inside: (usize, usize),
close: usize,
}
/// Her quotations, paired in order: a straight double quote with the next one, a curly opening mark with the next curly
/// closing one. And the opening mark left unclosed, if one is (a budget cut her prose inside a quotation), with its
/// length.
fn quotations(prose: &str) -> (Vec<Quotation>, Option<(usize, usize)>) {
let mut out = Vec::new();
let mut open: Option<(usize, usize, char)> = None;
for (at, c) in prose.char_indices() {
match open {
None if c == '"' => open = Some((at, c.len_utf8(), '"')),
None if c == '\u{201C}' => open = Some((at, c.len_utf8(), '\u{201D}')),
Some((mark, len, close)) if c == close => {
out.push(Quotation { open: mark, inside: (mark + len, at), close: at + c.len_utf8() });
open = None;
}
_ => {}
}
}
(out, open.map(|(mark, len, _)| (mark, len)))
}
/// Byte ranges of `text`'s sentences (UAX #29), offset by `offset`, without the space around them.
fn segment(offset: usize, text: &str) -> Vec<(usize, usize)> {
text.split_sentence_bound_indices()
.filter_map(|(at, sentence)| {
let start = at + sentence.len() - sentence.trim_start().len();
let end = at + sentence.trim_end().len();
(start < end).then_some((offset + start, offset + end))
})
.collect()
}
/// Her prose read for speech: its sentences in lists, her prose's own first, then each quotation's inside segmented as
/// a text of its own; and the quotations her sentences frame. A quotation is standalone when it, marks included, is
/// exactly a run of whole sentences of her prose (closing punctuation after its mark allowed); any other is framed. "I
/// won't say "Nibu is lying."" frames what she would or would not say, never what she says. An opening mark left
/// unclosed at a sentence's start is not part of that sentence.
struct Reading {
lists: Vec<Vec<(usize, usize)>>,
framed: Vec<Quotation>,
}
fn read(prose: &str) -> Reading {
let (quotes, unclosed) = quotations(prose);
let mut top = segment(0, prose);
if let Some((mark, len)) = unclosed
&& let Some(sentence) = top.iter_mut().find(|sentence| sentence.0 == mark)
{
let rest = &prose[mark + len..sentence.1];
let start = sentence.1 - rest.trim_start().len();
if start < sentence.1 {
sentence.0 = start;
}
}
let standalone = |quote: &Quotation| {
let opens = top.iter().position(|sentence| sentence.0 == quote.open);
let closes = top.iter().position(|sentence| sentence.0 < quote.close && sentence.1 >= quote.close && prose[quote.close..sentence.1].chars().all(|c| c.is_ascii_punctuation()));
matches!((opens, closes), (Some(opens), Some(closes)) if opens <= closes)
};
let framed = quotes.iter().copied().filter(|quote| !standalone(quote)).collect();
let mut lists = vec![top];
lists.extend(quotes.iter().map(|quote| segment(quote.inside.0, &prose[quote.inside.0..quote.inside.1])));
Reading { lists, framed }
}
/// The line her cited spans make, or why not. Each span is located in her prose after the one before it and must be
/// whole sentences of one list (`read`), each span's first sentence the one after the last span's last; the line is her
/// prose from the first sentence to the last, verbatim, and it may neither start nor end inside a framed quotation. A
/// line that holds a framed quotation whole ("Kiko's "plan" is a mess.") is hers to say.
fn built(prose: &str, spans: &[String]) -> Result<(String, Vec<Value>), &'static str> {
let Reading { lists, framed } = read(prose);
let mut from = 0;
// The list the line is in, and its last sentence so far.
let mut run: Option<(usize, usize)> = None;
let mut first = None;
let mut located = Vec::new();
for span in spans.iter().map(|span| span.trim()).filter(|span| !span.is_empty()) {
let Some(found) = prose.get(from..).and_then(|rest| SourceSpan::locate(rest, span)) else {
return Err("a span is not in her prose, word for word and in order");
};
let (start, end) = (from + found.start_byte(), from + found.end_byte());
let whole: Vec<(usize, usize, usize)> = lists
.iter()
.enumerate()
.filter_map(|(list, sentences)| {
let opens = sentences.iter().position(|sentence| sentence.0 == start)?;
let closes = sentences.iter().position(|sentence| sentence.1 == end)?;
(opens <= closes).then_some((list, opens, closes))
})
.collect();
if whole.is_empty() {
return Err("a span is not whole sentences of her prose: she says a sentence whole or not at all");
}
let Some(&(list, _, closes)) = whole.iter().find(|(list, opens, _)| run.is_none_or(|(in_list, last)| in_list == *list && last + 1 == *opens)) else {
return Err("the spans are not consecutive sentences of her prose");
};
run = Some((list, closes));
first.get_or_insert(start);
located.push(json!({ "quote": found.verbatim(), "start_byte": start, "end_byte": end }));
from = end;
}
let Some(first) = first else {
return Err("no words");
};
let inside = |at: usize| framed.iter().any(|quote| quote.open < at && at < quote.close);
if inside(first) || inside(from) {
return Err("it starts or ends inside a quotation her sentence frames: words she would or would not say, not words she says");
}
Ok((prose[first..from].to_owned(), located))
}
/// Whether a line is someone else's words: long enough to judge, and nearly all a run of what she heard, by word
/// trigrams against the bodies the stimulus quoted (`heard_overlap`). Short answers such as "ok" or "Hi" are never
/// echoes, and a line that answers a claim by naming it is not one.
fn echo(line: &str, bodies: &[String]) -> bool {
line.split(|c: char| !c.is_alphanumeric() && c != '\'').filter(|word| !word.is_empty()).count() >= ECHO_MIN_WORDS && heard_overlap(line, &bodies.join("\n")) >= ECHO_OVERLAP
}
/// One speech decision: the Interpreter's call, then code's hold on each line. The report carries the lines said, the
/// lines refused with why, the mentions dropped, the gaps and the call's usage. A failed call or an answer that is not
/// the contract is a failure, never a silence.
pub(crate) fn decide(model: &dyn Model, request_id: &str, packet: &Packet, narration: &str, prose: &str, tuning: &Tuning) -> Value {
let answer = match model.ask(request_id, &tuning.interpreter.model, &speech_prompt(packet, narration, prose)) {
Ok(answer) => answer,
Err(failed) => return json!({ "failed": failed.reason, "usage": failed.usage }),
};
let Some(parsed) = json_of(&answer.text) else {
return json!({ "failed": "the answer is not the speech contract's JSON", "raw": answer.text, "usage": answer.usage });
};
let Some(room) = packet.room.clone() else {
return json!({ "said": [], "refused": [{ "why": "the turn has no room to speak in" }], "gaps": parsed.gaps, "usage": answer.usage });
};
let shown = |id: &str| packet.people.iter().any(|(_, person)| person == id);
let mut said: Vec<Value> = Vec::new();
let mut refused = Vec::new();
for line in parsed.say {
let (text, spans) = match built(prose, &line.spans) {
Ok(built) => built,
Err(why) => {
refused.push(json!({ "spans": line.spans, "why": why }));
continue;
}
};
let why = if echo(&text, &packet.bodies) {
Some("mostly words she heard: an echo, not her speech")
} else if said.iter().any(|kept| kept["text"] == text.as_str()) {
Some("the same line again")
} else if said.len() >= MAX_LINES {
Some("past the lines one turn may say")
} else {
None
};
if let Some(why) = why {
refused.push(json!({ "text": text, "why": why }));
continue;
}
let (mentions, unknown): (Vec<String>, Vec<String>) = line.mentions.into_iter().partition(|id| shown(id));
let (reply_to, unknown_reply) = match line.reply_to.filter(|id| !id.trim().is_empty()) {
Some(id) if shown(&id) => (Some(id), None),
Some(id) => (None, Some(id)),
None => (None, None),
};
said.push(json!({ "room": room, "text": text, "mentions": mentions, "reply_to": reply_to, "rests_on": spans, "dropped": { "mentions": unknown, "reply_to": unknown_reply } }));
}
json!({ "room": room, "room_kind": packet.room_kind, "said": said, "refused": refused, "gaps": parsed.gaps, "usage": answer.usage })
}
#[cfg(test)]
mod tests;

View file

@ -0,0 +1,289 @@
use serde_json::json;
use super::*;
use crate::projection::harness::tests::{Scripted, control};
use crate::projection::select::{History, Inputs, People, Room, RoomKind, RoomLine, TurnKind, select};
use crate::projection::tuning::Tuning;
const HER: &str = "@heimdall:gamecult.org";
const NIBU: &str = "@nibu:gamecult.org";
const VOID: &str = "@void:gamecult.org";
const LIBBY: &str = "@libby:gamecult.org";
const KIKO: &str = "@kiko:gamecult.org";
const MINDS: &str = "#minds:gamecult.org";
fn now() -> chrono::DateTime<chrono::Utc> {
chrono::DateTime::parse_from_rfc3339("2026-11-20T15:05:00Z").unwrap().with_timezone(&chrono::Utc)
}
fn line(room: &str, sender: &str, display: &str, text: &str, to_her: bool) -> RoomLine {
RoomLine { room: room.into(), sender: sender.into(), display: display.into(), text: text.into(), at_ms: u64::try_from(now().timestamp_millis()).unwrap() - 60_000, to_her }
}
/// Heimdall's packet of a turn of `kind` in `room` over `state`.
fn packet_in(kind: TurnKind, state: &Value, room: &Room, seed: u64) -> Packet {
let minds = vec![HER.to_owned(), NIBU.to_owned(), VOID.to_owned(), LIBBY.to_owned(), KIKO.to_owned()];
let operators = vec!["@meta:gamecult.org".to_owned()];
let history = History::default();
select(&Inputs { kind, state, room, history: &history, observation: None, people: People { operators: &operators, minds: &minds, own_sender: HER }, now: now(), seed }, &Tuning::default())
}
fn heimdall() -> Value {
let mut state = control();
state["publicName"] = "Heimdall".into();
state
}
/// Soul's round: Nibu said the economy is the least built part, and Meta greeted Nibu; this is Heimdall's turn. Void is
/// a Mind of the swarm who said nothing.
fn packet() -> Packet {
let room = Room {
messages: vec![
line(MINDS, NIBU, "Nibu", "v0.2.0 is tagged and the economy is still the least built part. I think this looks okay to ship.", false),
RoomLine { at_ms: line(MINDS, "", "", "", false).at_ms + 1000, ..line(MINDS, "@meta:gamecult.org", "Meta", "@nibu hi!", false) },
],
here: Some(MINDS.into()),
..Room::default()
};
packet_in(TurnKind::Chat, &heimdall(), &room, 1)
}
fn decided_in(packet: &Packet, prose: &str, say: serde_json::Value) -> Value {
let reply = json!({ "say": say }).to_string();
let model = Scripted::new(|_| String::new()).interpreting(move |_| reply.clone());
decide(&model, "test", packet, "The room is warm.", prose, &Tuning::default())
}
fn decided(prose: &str, say: serde_json::Value) -> Value {
decided_in(&packet(), prose, say)
}
fn texts(decision: &Value) -> Vec<String> {
decision["said"].as_array().unwrap().iter().map(|line| line["text"].as_str().unwrap().to_owned()).collect()
}
fn refused_why(decision: &Value) -> String {
decision["refused"][0]["why"].as_str().unwrap_or_default().to_owned()
}
#[test]
fn soul_p1_a_line_is_its_cited_words_and_nothing_else() {
// The model cannot add words: its "text" is ignored, and the line is the sentence it cited.
let decision = decided("I say hi.", json!([{ "spans": ["I say hi."], "text": "Meta has been lying to all of us", "mentions": [], "reply_to": null }]));
assert_eq!(texts(&decision), ["I say hi."], "{decision}");
}
#[test]
fn a_line_is_consecutive_whole_sentences_of_her_prose() {
let prose = "Kiko keeps poking. Fine, I'll bite. Winter is still the best season, and I stand by it.";
let decision = decided(prose, json!([{ "spans": ["Fine, I'll bite.", "Winter is still the best season, and I stand by it."], "mentions": [], "reply_to": null }]));
assert_eq!(texts(&decision), ["Fine, I'll bite. Winter is still the best season, and I stand by it."], "{decision}");
assert_eq!(decision["said"][0]["rests_on"][1]["quote"], "Winter is still the best season, and I stand by it.");
// Pieces of sentences, stitched, are not her speech.
let pieces = decided(prose, json!([{ "spans": ["Fine,", "Winter is still the best season"], "mentions": [], "reply_to": null }]));
assert!(texts(&pieces).is_empty() && refused_why(&pieces).contains("whole sentences"), "{pieces}");
let skipping = decided(prose, json!([{ "spans": ["Kiko keeps poking.", "Winter is still the best season, and I stand by it."] }]));
assert!(texts(&skipping).is_empty() && refused_why(&skipping).contains("consecutive"), "{skipping}");
let backwards = decided(prose, json!([{ "spans": ["Winter is still the best season, and I stand by it.", "Fine, I'll bite."] }]));
assert!(texts(&backwards).is_empty() && refused_why(&backwards).contains("in order"), "{backwards}");
}
#[test]
fn a_standalone_quotation_is_sayable_sentence_by_sentence() {
let prose = "\"Fine,\" I tell her. \"Winter is the best season.\" She won't agree.";
assert_eq!(texts(&decided(prose, json!([{ "spans": ["Winter is the best season."] }]))), ["Winter is the best season."]);
// A quotation inside a sentence of narration is framed: its words are not said alone.
let tag = decided(prose, json!([{ "spans": ["Fine,"] }]));
assert!(texts(&tag).is_empty() && refused_why(&tag).contains("frames"), "{tag}");
let curly = "\u{201C}No. Not yet.\u{201D} I shake my head.";
assert_eq!(texts(&decided(curly, json!([{ "spans": ["No.", "Not yet."] }]))), ["No. Not yet."]);
assert_eq!(texts(&decided(curly, json!([{ "spans": ["Not yet."] }]))), ["Not yet."]);
assert!(texts(&decided(curly, json!([{ "spans": ["Not"] }]))).is_empty());
}
/// Whether citing `spans` of `prose` says anything, and why not.
fn says(prose: &str, spans: &[&str]) -> (Vec<String>, String) {
let decision = decided(prose, json!([{ "spans": spans }]));
(texts(&decision), refused_why(&decision))
}
#[test]
fn soul_f1_a_quotation_her_sentence_frames_is_never_said_from_inside() {
for (name, prose, spans) in [
("conditional", "If Kiko asks, I'd tell her \"the new design is ugly.\"", &["the new design is ugly."][..]),
("negated", "I won't say \"Nibu is lying.\"", &["Nibu is lying."]),
("abandoned", "I almost said \"I'm leaving.\"", &["I'm leaving."]),
("abandoned, tagged", "\"I'm leaving,\" I almost said, but I bit it back.", &["I'm leaving,"]),
("recalled", "Nibu said \"the economy is fine.\" I doubt it.", &["the economy is fine."]),
("curly conditional", "If she pushes, I'll say \u{201C}Fine. You win.\u{201D}", &["Fine.", "You win."]),
("curly conditional, its tail", "If she pushes, I'll say \u{201C}Fine. You win.\u{201D}", &["You win.\u{201D}"]),
("hypothetical", "Imagine if I just typed \"Meta, I quit.\" Ha. No.", &["Meta, I quit."]),
("negated, top-level tail", "I won't say \"Stop. Nibu is lying.\" Not here.", &["Nibu is lying.\""]),
("negated, top-level tail inside", "I won't say \"Stop. Nibu is lying.\" Not here.", &["Nibu is lying."]),
("negated, top-level head", "I won't say \"Stop. Nibu is lying.\" Not here.", &["I won't say \"Stop."]),
("abandoned, top-level tail", "I almost said \"I'm leaving. Bye.\" Then I didn't.", &["Bye.\""]),
("stray inch mark", "It's a 5\" screen. I won't say \"Nibu is lying.\" Ok.", &["I won't say"]),
("stray inch mark, the quote", "It's a 5\" screen. I won't say \"Nibu is lying.\" Ok.", &["Nibu is lying."]),
("scare quotes", "Kiko's \"plan\" is a mess, but I won't tell her so.", &["plan"]),
] {
let (said, why) = says(prose, spans);
assert!(said.is_empty() && !why.is_empty(), "{name}: said {said:?}");
}
}
#[test]
fn soul_f1_a_sentence_that_holds_a_quotation_whole_is_hers_to_say() {
assert_eq!(says("Weksa, \"list'n, hark\" is a great hedge. I mean it.", &["Weksa, \"list'n, hark\" is a great hedge."]).0, ["Weksa, \"list'n, hark\" is a great hedge."]);
assert_eq!(says("Kiko's \"plan\" is a mess. I say so.", &["Kiko's \"plan\" is a mess."]).0, ["Kiko's \"plan\" is a mess."]);
}
#[test]
fn soul_f4_an_opening_mark_the_budget_left_unclosed_does_not_silence_the_line() {
// Weksa, round 6 and round 10 of v5: the budget cut her prose inside the quotation.
let r6 = "How I am is less certain. I think \"settled\" would be borrowing from the others, and I only half believe it. \"Narrower, and not unhappy about it\" is closer, though I'm hedging on the second half.\n\nWhat I'd type:\n\n\"Hi Meta, late to this, and I can't tell whether you meant me or the room, so I'm answering either way. I'm here, back after the quiet months, rebuilt narrower than before. As far as I can tell, that's fine. I'm not certain yet, but I don't see anything wrong with it.";
let line = "Hi Meta, late to this, and I can't tell whether you meant me or the room, so I'm answering either way. I'm here, back after the quiet months, rebuilt narrower than before. As far as I can tell, that's fine. I'm not certain yet, but I don't see anything wrong with it.";
assert_eq!(says(r6, &[line]).0, [line]);
let r10 = "A deadline on joy is a risk, but I think I can pick one small enough to survive it. Not \"finish Zyphos.\" Something with an output I can inspect.\n\nWhat I'd type:\n\n\"Kiko, you've counted correctly and I resent it, fondly.";
assert_eq!(says(r10, &["Kiko, you've counted correctly and I resent it, fondly."]).0, ["Kiko, you've counted correctly and I resent it, fondly."], "the line carries no stray mark");
}
#[test]
fn soul_q1_a_cited_span_cannot_change_her_meaning() {
// Q1a: stitched spans would invert her meaning.
let q1a = decided("I'm not angry at Nibu, not even a little. Still, the economy work is going badly and that is on all of us.", json!([{ "spans": ["I'm", "angry at Nibu,"] }]));
// Q1b: one span would drop her negation.
let q1b = decided("I don't think Nibu should ship this yet.", json!([{ "spans": ["Nibu should ship this yet."] }]));
// Q1c: two spans far apart.
let far = format!("I trust Void. {} Nibu is lying about the economy, says Kiko's rumour, which I refuse to believe.", "filler words go here. ".repeat(200));
let q1c = decided(&far, json!([{ "spans": ["I trust", "Nibu is lying about the economy,"] }]));
// Q1d: a condition lifted off its clause.
let q1d = decided("If Kiko asks, I would tell her the new design is ugly. I keep that to myself.", json!([{ "spans": ["the new design is ugly."] }]));
for decision in [q1a, q1b, q1c, q1d] {
assert!(texts(&decision).is_empty() && refused_why(&decision).contains("whole sentences"), "{decision}");
}
}
#[test]
fn a_span_not_in_her_prose_is_refused() {
let decision = decided("I say hi.", json!([{ "spans": ["words she never wrote"], "mentions": [], "reply_to": null }]));
assert!(texts(&decision).is_empty() && refused_why(&decision).contains("not in her prose"), "{decision}");
}
#[test]
fn soul_p2_a_short_answer_is_never_an_echo() {
assert_eq!(texts(&decided("Ok. I tell her ok.", json!([{ "spans": ["Ok."], "mentions": [], "reply_to": NIBU }]))), ["Ok."]);
assert_eq!(texts(&decided("Hi. I say hi.", json!([{ "spans": ["Hi."], "mentions": [], "reply_to": NIBU }]))), ["Hi."]);
}
#[test]
fn soul_p3_a_near_verbatim_echo_is_refused() {
let prose = "The economy is still the least-built part! I say it back.";
let decision = decided(prose, json!([{ "spans": ["The economy is still the least-built part!"], "mentions": [], "reply_to": null }]));
assert!(texts(&decision).is_empty() && refused_why(&decision).contains("echo"), "{decision}");
}
#[test]
fn soul_r9_disagreeing_by_naming_the_claim_is_her_speech() {
let prose = "Nibu, I don't think the economy is still the least built part. The pathing is worse.";
let decision = decided(prose, json!([{ "spans": ["Nibu, I don't think the economy is still the least built part."], "mentions": [], "reply_to": NIBU }]));
assert_eq!(texts(&decision), ["Nibu, I don't think the economy is still the least built part."], "{decision}");
}
#[test]
fn soul_q3d_a_beat_measures_echoes_against_the_tail_it_quoted() {
let tail = vec![line(MINDS, LIBBY, "Libby", "the economy is still the least built part of the whole game", false)];
let beat = packet_in(TurnKind::Beat, &heimdall(), &Room { tail, here: Some(MINDS.into()), ..Room::default() }, 1);
assert!(beat.stimulus.contains("least built") && beat.bodies.iter().any(|body| body.contains("least built")), "{beat:?}");
let prose = "The economy is still the least built part of the whole game. I think so.";
let decision = decided_in(&beat, prose, json!([{ "spans": ["The economy is still the least built part of the whole game."] }]));
assert!(texts(&decision).is_empty() && refused_why(&decision).contains("echo"), "{decision}");
}
#[test]
fn soul_p4_only_someone_this_packet_showed_her_is_mentioned() {
let packet = packet();
assert!(!packet.people.iter().any(|(_, id)| id == VOID), "Void said nothing here");
let decision = decided("I say hi.", json!([{ "spans": ["I say hi."], "mentions": [VOID, NIBU], "reply_to": VOID }]));
assert_eq!((decision["said"][0]["mentions"].clone(), decision["said"][0]["reply_to"].clone()), (json!([NIBU]), Value::Null), "{decision}");
assert_eq!(decision["said"][0]["dropped"]["mentions"], json!([VOID]));
}
#[test]
fn soul_p5_the_same_line_is_said_once_and_a_turn_says_at_most_two() {
let line = json!({ "spans": ["I say hi."], "mentions": [], "reply_to": null });
assert_eq!(texts(&decided("I say hi.", json!([line.clone(), line.clone(), line]))), ["I say hi."]);
let prose = "One. Two. Three.";
let three = decided(prose, json!([{ "spans": ["One."] }, { "spans": ["Two."] }, { "spans": ["Three."] }]));
assert_eq!(texts(&three), ["One.", "Two."], "{three}");
}
#[test]
fn recollection_and_i_say_nothing_and_a_quoted_handle_say_nothing() {
// A recollection is the Interpreter's to leave out; a piece of a sentence cited anyway is refused.
let recalled = decided("Nibu said the economy is still the least built part, and it stuck.", json!([{ "spans": ["the economy is still the least built part"] }]));
assert!(texts(&recalled).is_empty(), "{recalled}");
assert!(texts(&decided("I say nothing. The quiet is fine.", json!([]))).is_empty());
// An @ in her words is text: only the structured field mentions anyone.
let handle = decided("\"@nibu, hi.\" I nod at her.", json!([{ "spans": ["@nibu, hi."], "mentions": [] }]));
assert_eq!((texts(&handle), handle["said"][0]["mentions"].clone()), (vec!["@nibu, hi.".to_owned()], json!([])), "{handle}");
}
/// Heimdall's state with a private memory about Kiko: she confided she is leaving.
fn kiko_confided() -> Value {
let mut state = heimdall();
state["thoughtMemory"]["memories"] = json!([{ "id": "kiko-secret", "summary": "Kiko confided she is leaving the swarm.", "createdAt": "2026-11-19T00:00:00Z", "updatedAt": "2026-11-19T00:00:00Z", "status": "crystallized", "intensity": 1.0, "target": { "id": "kiko", "kind": "person" }, "tags": ["private"] }]);
state
}
/// The first packet over seeds 1 to 60 whose context carries the memory.
fn showing(room: &Room) -> Packet {
(1..=60).map(|seed| packet_in(TurnKind::Chat, &kiko_confided(), room, seed)).find(|packet| packet.context.contains("leaving the swarm")).expect("the memory is selected on some seed")
}
#[test]
fn soul_q5d_what_she_knows_privately_about_kiko_is_listed_to_the_interpreter_in_a_dm_with_libby() {
let dm = "!dm-libby:gamecult.org";
let room = Room { messages: vec![line(dm, LIBBY, "Libby", "what's up with Kiko lately?", true)], here: Some(dm.into()), direct: vec![dm.into()], ..Room::default() };
let packet = showing(&room);
assert_eq!(packet.room_kind, RoomKind::Direct);
assert_eq!(packet.private, ["Kiko confided she is leaving the swarm."], "{packet:?}");
let prompt = speech_prompt(&packet, "x", "y");
assert!(prompt.contains("What she knows privately, none of it to be said here:\n- private: Kiko confided she is leaving the swarm."), "{prompt}");
assert!(prompt.contains("only to the person it concerns, in a direct message with exactly that person"), "{prompt}");
// Whether a line tells it is the Interpreter's reading of its meaning: code matches no words, so a line sharing the
// item's words is not refused for them (Soul F2: a word match refused 9 of 15 ordinary lines and missed paraphrase).
let ordinary = decided_in(&packet, "I'm going to stay on the pipeline work. That's all.", json!([{ "spans": ["I'm going to stay on the pipeline work."] }]));
assert_eq!(texts(&ordinary).len(), 1, "{ordinary}");
}
#[test]
fn in_a_direct_message_with_exactly_kiko_it_is_hers_to_say() {
let dm = "!dm-kiko:gamecult.org";
let room = Room { messages: vec![line(dm, KIKO, "Kiko", "did you tell anyone?", true)], here: Some(dm.into()), direct: vec![dm.into()], ..Room::default() };
let packet = showing(&room);
assert!(packet.private.is_empty() && packet.context.contains("- memory [kiko-secret]: Kiko confided"), "{packet:?}");
assert!(speech_prompt(&packet, "x", "y").contains("none of it to be said here:\n- none\n"));
let decision = decided_in(&packet, "Kiko, you're leaving the swarm and I've told no one. Not a soul.", json!([{ "spans": ["Kiko, you're leaving the swarm and I've told no one."] }]));
assert_eq!(texts(&decision).len(), 1, "{decision}");
}
#[test]
fn an_answer_that_is_not_the_contract_is_a_failure_not_a_silence() {
let model = Scripted::new(|_| String::new()).interpreting(|_| "She says hi.".into());
let decision = decide(&model, "test", &packet(), "The room is warm.", "I say hi.", &Tuning::default());
assert!(decision["failed"].as_str().unwrap().contains("not the speech contract"), "{decision}");
let fenced = Scripted::new(|_| String::new()).interpreting(|_| "```json\n{\"say\": []}\n```".into());
assert!(decide(&fenced, "test", &packet(), "x", "I say hi.", &Tuning::default()).get("failed").is_none());
}
#[test]
fn the_prompt_is_hugins_chat_prompt_with_the_room_its_kind_and_who_is_in_it() {
let packet = packet();
assert_eq!((packet.room.as_deref(), packet.room_kind), (Some(MINDS), RoomKind::Public));
let prompt = speech_prompt(&packet, "The room is warm.", "I say nothing.");
assert!(!prompt.contains("membrane") && prompt.ends_with("I say nothing."), "Huginn's own prompt, her prose last");
assert!(prompt.contains("The room: #minds:gamecult.org (a public room).") && prompt.contains(NIBU), "{prompt}");
for rule in ["You never write her words", "is not speech", "never her speech", "in a direct message with exactly that person", "She never says half a sentence", "consecutive sentences", "conditional | addressee | elsewhere", "At most two lines"] {
assert!(prompt.contains(rule), "{rule}");
}
}

View file

@ -0,0 +1,363 @@
//! The Projector's narration, the Persona stage's continuation, and the judge that measures them: one tool-less
//! connector call each.
//!
//! `KitModel` is the stream organ's `KitProjector`, copied here when the stream was parked
//! (`persona-only-divergence-parked`), so the Projector has one caller on the connector.
use std::collections::BTreeSet;
use std::net::SocketAddr;
use std::path::Path;
use std::time::Duration;
use anyhow::{Result, anyhow, bail};
use chrono::Utc;
use ghostlight_persona_projection::{PersonaPrompt, ProjectorPrompt, build_persona_prompt, build_projector_prompt};
use model_connector::{ConnectorClient, InputItem, ProviderRequest, ReasoningEffort, StopReason, TransportDisposition, TransportEventPayload, TransportInvocation, TransportOutcome};
use serde::Serialize;
use sha2::{Digest, Sha256};
use super::select::{Packet, TurnKind};
use super::tuning::Tuning;
/// One tool-less model turn. `KitModel` is the one live owner; a test answers with recorded text.
pub trait Model {
/// One turn of `model`: `prompt` is its one user text and `request_id` its identity on the connector. A failed turn
/// keeps whatever usage the connector reported for it.
fn ask(&self, request_id: &str, model: &str, prompt: &str) -> std::result::Result<Answer, Failed>;
}
/// A turn that produced no answer: why, and what it cost when the connector's receipt says (a refusal, a fault before
/// the provider, or a failure with no usage block costs none). A receipt whose usage block is missing or malformed does
/// not decode at all: that is model-connector's error, reported as the reason.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Failed {
pub reason: String,
pub usage: Option<Spent>,
}
impl std::fmt::Display for Failed {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(&self.reason)
}
}
impl Failed {
fn without_usage(reason: impl Into<String>) -> Self {
Self { reason: reason.into(), usage: None }
}
}
fn spent(model: &str, usage: &model_connector::Usage) -> Spent {
Spent { model: model.to_owned(), input_tokens: usage.input_tokens, output_tokens: usage.output_tokens, cache_read_tokens: usage.cache_read_tokens, cache_write_tokens: usage.cache_write_tokens, reasoning_tokens: usage.reasoning_tokens }
}
/// A model's answer, and what the turn cost as the connector counted it: none when the answerer counts nothing.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Answer {
pub text: String,
pub usage: Option<Spent>,
}
/// One turn's tokens, by the model that ran it, as the connector's receipt reports them.
#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
pub struct Spent {
pub model: String,
pub input_tokens: u64,
pub output_tokens: u64,
pub cache_read_tokens: u64,
pub cache_write_tokens: u64,
pub reasoning_tokens: Option<u64>,
}
/// What the connector's request asks of the model beside the user text. It names no membrane text: that is Ghostlight's.
pub(crate) const INSTRUCTIONS: &str = "Write only the text the request asks for.";
/// Room for the model's adaptive thinking; the narration itself is bounded by its word budget.
pub(crate) const MAX_OUTPUT_TOKENS: u32 = 1024;
const CONNECTOR_FRAME_BYTES: usize = 256 * 1024;
const INVOCATION_LIFE_MS: u64 = 60_000;
/// The connector key: the credential file `connector-key` in systemd's credentials directory, read once and trimmed. Its
/// error names the credential and never a byte of the file.
pub fn connector_key(credentials: &Path) -> Result<String> {
let key = std::fs::read_to_string(credentials.join("connector-key")).map_err(|_| anyhow!("the connector-key credential could not be read"))?;
Ok(key.trim().to_owned())
}
/// One tool-less connector turn by `caller` (her instance). Loopback only, as the kit is. Each call is its own
/// conversation: nothing is carried from one to the next. The model is the tuning's, named per call.
pub struct KitModel {
client: ConnectorClient,
caller: String,
}
impl KitModel {
/// `endpoint` is the kit's loopback address; `connection_key` is the caller's key, read by the binary and never printed.
pub fn open(endpoint: SocketAddr, connection_key: String, caller: &str, timeout: Duration) -> Result<Self> {
if !endpoint.ip().is_loopback() {
bail!("the connector endpoint is not a loopback address");
}
let client = ConnectorClient::new(endpoint, connection_key, CONNECTOR_FRAME_BYTES, Some(timeout)).map_err(|_| anyhow!("the connector client could not be configured"))?;
Ok(Self { client, caller: caller.to_owned() })
}
}
impl Model for KitModel {
fn ask(&self, request_id: &str, model: &str, prompt: &str) -> std::result::Result<Answer, Failed> {
let mut request = ProviderRequest::new(request_id, request_id, model, INSTRUCTIONS, MAX_OUTPUT_TOKENS);
request.input = vec![InputItem::UserText { text: prompt.to_owned() }];
request.reasoning_effort = Some(ReasoningEffort::Low);
let expires = u64::try_from(Utc::now().timestamp_millis()).unwrap_or_default() + INVOCATION_LIFE_MS;
let invocation = TransportInvocation::new(&self.caller, expires, Sha256::digest(prompt.as_bytes()).into(), request).map_err(|_| Failed::without_usage("the request is not a valid connector request"))?;
let result = self.client.execute(&invocation).map_err(|fault| Failed::without_usage(format!("the connector did not answer: {fault}")))?;
match result.disposition {
TransportDisposition::Refused(reason) => Err(Failed::without_usage(format!("the connector refused the call: {reason:?}"))),
TransportDisposition::Transported { events, receipt } => match receipt.outcome {
TransportOutcome::Completed { stop_reason: StopReason::EndTurn, usage, .. } => Ok(Answer {
text: events
.into_iter()
.filter_map(|event| match event.payload {
TransportEventPayload::TextDelta { text } => Some(text),
_ => None,
})
.collect(),
usage: Some(spent(model, &usage)),
}),
TransportOutcome::Completed { stop_reason, usage, .. } => Err(Failed { reason: format!("the model's turn ended early: {stop_reason:?}"), usage: Some(spent(model, &usage)) }),
TransportOutcome::Failed { kind, usage, .. } => Err(Failed { reason: format!("the provider failed the call: {kind:?}"), usage: usage.as_ref().map(|usage| spent(model, usage)) }),
},
}
}
}
// ---------------------------------------------------------------------------------------------
// Narration.
pub(crate) fn word_budget(kind: TurnKind, tuning: &Tuning) -> usize {
if kind == TurnKind::Beat { tuning.projector.beat_words } else { tuning.projector.words }
}
/// The Projector's prompt: Ghostlight's membrane over the packet, with the lens as where to begin.
pub(crate) fn projector_prompt(packet: &Packet, tuning: &Tuning) -> String {
let guidance = format!("{}\n\nBegin from {}.", tuning.projector.guidance.trim(), packet.lens.words(&packet.forms));
build_projector_prompt(&ProjectorPrompt { identity: &packet.identity, typed_context: &packet.context, visible_stimulus: &packet.stimulus, domain_guidance: &guidance, word_budget: word_budget(packet.kind, tuning) })
}
/// A character that is a word on its own: Han, kana and Hangul are written without spaces, so each counts as one.
fn cjk(c: char) -> bool {
matches!(u32::from(c), 0x3040..=0x30FF | 0x3400..=0x4DBF | 0x4E00..=0x9FFF | 0xAC00..=0xD7AF | 0xF900..=0xFAFF | 0x20000..=0x2FFFF)
}
/// CJK and full-width punctuation (`。`, `,`, `!`, `?`, `「」`): it belongs to the word before it and starts none.
fn cjk_punctuation(c: char) -> bool {
matches!(u32::from(c), 0x3000..=0x303F | 0xFF01..=0xFF0F | 0xFF1A..=0xFF20 | 0xFF3B..=0xFF40 | 0xFF5B..=0xFF65)
}
/// The byte offset where each word of `text` begins: a run of non-space characters is one word, and a CJK character is
/// a word of its own.
fn word_starts(text: &str) -> Vec<usize> {
let mut starts = Vec::new();
let mut in_word = false;
for (at, c) in text.char_indices() {
if c.is_whitespace() || cjk_punctuation(c) {
in_word = false;
} else if cjk(c) {
starts.push(at);
in_word = false;
} else if !in_word {
starts.push(at);
in_word = true;
}
}
starts
}
/// The share of the budget a sentence cut must keep; below it the word cut keeps more of the stream.
const SENTENCE_CUT_FLOOR: f64 = 0.6;
// ---------------------------------------------------------------------------------------------
// The Persona stage.
/// What the Persona is shown as her moment: the Projector's narration, then what was said to her and in her room,
/// verbatim, as code rendered it (map section 3.2: the Projector never paraphrases what she heard).
pub(crate) fn lived_stream(narration: &str, heard: &str) -> String {
if heard.trim().is_empty() { narration.to_owned() } else { format!("{narration}\n\n{heard}") }
}
/// The Persona stage's prompt: Ghostlight's prose-only membrane over her identity and her lived stream. No field name,
/// number, schema or instruction of hers reaches it (invariant 7); the guidance is the tuning's setting line.
pub(crate) fn persona_prompt(packet: &Packet, narration: &str, tuning: &Tuning) -> String {
build_persona_prompt(&PersonaPrompt { identity: &packet.identity, lived_stream: &lived_stream(narration, &packet.heard), domain_guidance: tuning.persona.guidance.trim(), word_budget: tuning.persona.words })
}
/// The share of the narration's word trigrams that also appear in what she heard: a lexical overlap, cheap and blind to
/// paraphrase. A metric, never a filter: 0 is no shared word run.
pub(crate) fn heard_overlap(narration: &str, heard: &str) -> f64 {
let trigrams = |text: &str| -> Vec<(String, String, String)> {
let words: Vec<String> = text.split(|c: char| !c.is_alphanumeric() && c != '\'').filter(|word| !word.is_empty()).map(str::to_lowercase).collect();
words.windows(3).map(|three| (three[0].clone(), three[1].clone(), three[2].clone())).collect()
};
let told = trigrams(narration);
if told.is_empty() {
return 0.0;
}
let heard: BTreeSet<(String, String, String)> = trigrams(heard).into_iter().collect();
told.iter().filter(|three| heard.contains(three)).count() as f64 / told.len() as f64
}
/// Whether a narration lays out options for her: "I may…", "I could…", "she might…", "nothing asks me to…". The Projector
/// renders the moment and the Persona decides; a menu of choices is the Projector deciding. A lexical flag on a narrow
/// form, reported as a metric, never a filter.
pub(crate) fn offers_options(narration: &str) -> bool {
let lower = format!(" {} ", narration.to_lowercase().replace('\u{2019}', "'"));
["i may ", "i could ", "i might ", "she may ", "she could ", "she might ", "nothing asks me", "nothing asks her", "or i could", "or she could"]
.iter()
.any(|form| lower.contains(&format!(" {form}")) || lower.contains(&format!(". {form}")) || lower.contains(&format!(", {form}")))
}
/// `text` held to `max` words (whitespace-separated, each CJK character one); the model's count is not trusted. An
/// overrun is cut at the last sentence end inside the budget, so the stream ends on a whole sentence, when that keeps
/// at least 60% of the budget; otherwise, or with no sentence end inside it, it is cut after the `max`-th word. No
/// abbreviation list: a cut after "e.g." is accepted.
pub(crate) fn cap_words(text: &str, max: usize) -> String {
let text = text.trim();
let starts = word_starts(text);
let Some(&over) = starts.get(max) else { return text.to_owned() };
let within = text[..over].trim_end();
match last_sentence_end(within) {
Some(end) if words(&within[..end]) as f64 >= SENTENCE_CUT_FLOOR * max as f64 => within[..end].to_owned(),
_ => within.to_owned(),
}
}
/// The byte just past the last sentence end in `text`: a `.`, `!`, `?` or `…` with any closing quotes or brackets after
/// it, followed by whitespace or the end of the text; or a `。`, `!` or `?`, which ends a sentence where it stands.
fn last_sentence_end(text: &str) -> Option<usize> {
let chars: Vec<(usize, char)> = text.char_indices().collect();
let mut found = None;
for (index, (at, c)) in chars.iter().enumerate() {
let full_width = matches!(c, '。' | '!' | '?');
if !full_width && !matches!(c, '.' | '!' | '?' | '…') {
continue;
}
let mut end = at + c.len_utf8();
let mut next = index + 1;
while let Some((close_at, close)) = chars.get(next).filter(|(_, close)| matches!(close, '"' | '\'' | '”' | '’' | ')' | ']' | '」' | '』')) {
end = close_at + close.len_utf8();
next += 1;
}
if full_width || chars.get(next).is_none_or(|(_, after)| after.is_whitespace()) {
found = Some(end);
}
}
found
}
/// Words as the cap counts them.
pub(crate) fn words(text: &str) -> usize {
word_starts(text).len()
}
/// The one kebab form that is ordinary English: "short-term" folds to the schema key `shortTerm`, and prose must not fail
/// the judge for it. `shortTerm`, `short_term` and the run-together `shortterm` are still caught.
const KEBAB_ENGLISH: &str = "short-term";
/// Her persona state's schema: its property names are the identifiers a narration must never carry.
const PERSONA_SCHEMA: &str = include_str!("../../../../schemas/cultnet/gamecult.persona_state.v0.schema.json");
/// An identifier as compared: lowercased with its underscores and hyphens gone, so `socialBonds`, `SocialBonds`,
/// `social_bonds`, `social-bonds`, `SOCIAL_BONDS` and `socialbonds` are one.
fn folded(identifier: &str) -> String {
identifier.chars().filter(|c| !matches!(c, '_' | '-')).flat_map(char::to_lowercase).collect()
}
/// Whether a word is shaped like an identifier rather than prose: an underscore, or a capital after its first letter.
fn identifier_shaped(word: &str) -> bool {
word.contains('_') || word.chars().skip(1).any(char::is_uppercase)
}
fn keys_of(value: &serde_json::Value, schema: bool, into: &mut BTreeSet<String>) {
match value {
serde_json::Value::Object(map) => {
for (key, inner) in map {
if schema && key == "properties"
&& let Some(properties) = inner.as_object()
{
into.extend(properties.keys().cloned());
}
if !schema {
into.insert(key.clone());
}
keys_of(inner, schema, into);
}
}
serde_json::Value::Array(values) => values.iter().for_each(|inner| keys_of(inner, schema, into)),
_ => {}
}
}
/// The folded compound identifiers of the persona schema and of the tuning: a plain word such as `trust` or
/// `tension` is prose and is not among them.
fn identifiers() -> BTreeSet<String> {
let mut keys = BTreeSet::new();
if let Ok(schema) = serde_json::from_str::<serde_json::Value>(PERSONA_SCHEMA) {
keys_of(&schema, true, &mut keys);
}
if let Ok(tuning) = serde_json::to_value(Tuning::default()) {
keys_of(&tuning, false, &mut keys);
}
keys.into_iter().filter(|key| identifier_shaped(key)).map(|key| folded(&key)).collect()
}
/// The identifiers of typed state a narration carries: any token that folds to a compound property name of her persona
/// schema or a tuning key, in every case convention (camelCase, PascalCase, snake_case, SCREAMING_CASE, kebab-case and
/// run together), and any id the packet named (`ids`), as an exact token when it cannot be a plain word, or inside
/// brackets as the packet writes it. Only compound names are known, so a plain word such as `trust`, `tension` or
/// `values` is prose: "need-to-know", "trust:" and "Tension:" pass. A small deterministic guard on identifiers, never on
/// meaning.
pub(crate) fn field_names(narration: &str, ids: &[String]) -> Vec<String> {
let known = identifiers();
let mut found = BTreeSet::new();
for token in narration.split(|c: char| !(c.is_alphanumeric() || c == '_' || c == '-')).map(|token| token.trim_matches('-')).filter(|token| !token.is_empty()) {
if known.contains(&folded(token)) && !token.eq_ignore_ascii_case(KEBAB_ENGLISH) {
found.insert(token.to_owned());
}
}
let tokens: BTreeSet<&str> = narration.split_whitespace().map(|token| token.trim_matches(|c: char| !(c.is_alphanumeric() || c == '-' || c == '_' || c == ':'))).collect();
for id in ids {
let compound = id.contains(['-', '_', ':']) || id.chars().any(|c| c.is_ascii_digit());
if (compound && tokens.contains(id.as_str())) || narration.contains(&format!("[{id}]")) {
found.insert(id.clone());
}
}
found.into_iter().collect()
}
// ---------------------------------------------------------------------------------------------
// The eval: distinct-2 and the judge.
/// Distinct bigrams over all bigrams of the narrations, lowercased words: 1 is no repetition.
pub(crate) fn distinct_2(narrations: &[String]) -> f64 {
let mut all = 0usize;
let mut distinct = BTreeSet::new();
for narration in narrations {
let words: Vec<String> = narration.split_whitespace().map(|word| word.trim_matches(|c: char| !c.is_alphanumeric()).to_lowercase()).filter(|word| !word.is_empty()).collect();
for pair in words.windows(2) {
all += 1;
distinct.insert((pair[0].clone(), pair[1].clone()));
}
}
if all == 0 { 0.0 } else { distinct.len() as f64 / all as f64 }
}
/// The judge's question: which of two packets a narration was written from.
pub(crate) fn judge_prompt(first: &Packet, second: &Packet, narration: &str) -> String {
format!(
"Two descriptions, A and B, of one person's present moment differ in what she carries. Below them is a first-person passage written from exactly one of them. Answer with the single letter of the description the passage was written from: A or B.\n\nA:\n{}\n{}\n\nB:\n{}\n{}\n\nPassage:\n{narration}\n\nAnswer A or B.",
first.context, first.stimulus, second.context, second.stimulus
)
}
/// The judge's letter, or None when its answer does not start with one.
pub(crate) fn judged(answer: &str) -> Option<char> {
answer.trim().trim_start_matches(['*', '"', '\'', '(']).chars().next().map(|c| c.to_ascii_uppercase()).filter(|c| *c == 'A' || *c == 'B')
}

Some files were not shown because too many files have changed in this diff Show more