Commit graph Huginn/crates/huginn-daemon
Author SHA1 Message Date
fece9a7737 Pin the daemon door for texts: refused by field before queueing
Zero, nine and blank texts answer Refused naming `texts` and never reach the
index; two texts are one queued search. Cut
gamecult-chat:cut_spec:cut-source-query-texts.r1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 16:03:28 +00:00
99bc6c6c76 Source search embeds all texts in one call and fuses per-text lists
search() embeds each distinct trimmed text (Instruct:-prefixed) in one batch,
searches once per vector, and fuses with reciprocal rank (k=60, depth 50) only
when more than one text remains; one text is searched top_k deep and answers
with the cosine scores as before. Cut
gamecult-chat:cut_spec:cut-source-query-texts.r1, commit 3.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 16:03:11 +00:00
c8b66eda85 Add reciprocal rank fusion to the source organ
fuse is a pure generic function over ranked lists and a key closure, with
FUSE_K = 60 (Cormack, Clarke, Buttcher, SIGIR 2009) and FUSE_DEPTH = 50 named
beside CORPUS. Nothing calls it yet. Cut
gamecult-chat:cut_spec:cut-source-query-texts.r1, commit 2.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 16:00:56 +00:00
e939d246b8 Source query takes texts: wire field, door and published schemas
SourceQuery gains `texts` (one to eight, SOURCE_TEXTS_MAX) beside the
transitional `text`; check() refuses neither, both, more than eight and blank
entries, naming `texts`. SourceHit.score documents cosine versus reciprocal-rank
sum. Published request and response schemas regenerated. Cut
gamecult-chat:cut_spec:cut-source-query-texts.r1, commit 1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 15:59:52 +00:00
598f1db7f9 Desk door: socket tests pin the exact reply cap on a read, and a stale answer's stored refusal and replay while the fork is oversize
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 01:46:20 +00:00
4c1f247898 Desk door: a read of one fork past the reply cap is fork_too_large naming the fork (no order, no cursor)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 01:28:57 +00:00
5af0cbea65 Desk door: failing socket test, a read of an oversize fork is fork_too_large naming the fork
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 01:26:57 +00:00
c6e1b41967 Desk door: socket tests for fork_too_large as the oldest (no c0) and newest fork
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 01:02:00 +00:00
5f3319f4ad Desk door: failing socket test for fork_too_large (list names fork and order, cursor resumes, read never returns it)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 00:51:57 +00:00
9e7dca2716 Desk reads: order is a unique 0-based rank by (admission, key); pages hold at most limit and stop at the byte budget; a lone oversize first entry is the named refusal with its order; vectors re-vendored from CultLib 9d6ee72e
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 00:36:48 +00:00
071dc8e345 Desk door test: the page is short of the bound by the Mind's fill; a group past the budget is the Mind's refusal
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 00:09:10 +00:00
aeeb36ad2a Desk answer: delete the replayed flag nobody read; tests assert that a replay commits nothing
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 00:06:27 +00:00
3b775a23c8 Desk door tests: a peer that never reads fails the suite by assertion in five seconds, not by hanging it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 00:06:27 +00:00
707642bd54 Desk reads: Mind::desk_forks and desk_fork project card, record and entry at one snapshot
Newest first by admission ordinal, cursor strictly below, counts over the showable set, limit 0 is
counts and stamp. Whole record to the peer-checked socket; the trust boundary is pinned in the
desk_door module doc and the DeskRequest/DeskResponse docs. Published schemas regenerated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 23:57:27 +00:00
a37bd4f85f Desk reads: the wire types, the request fields and the failing tests (stub reads answer fork_unknown)
Types ForkCard/Record/Entry/List, ForkCursor and ForkLimit validated at decode, DeskRequest::Forks
gains status/cursor/limit flat, DeskResponse gains Forks and Fork, the daemon dispatches to
Mind::desk_forks/desk_fork (stubs). Vectors vendored at CultLib e57fbaee. The tests pin the rules
the next commit implements; DESK_READS_UNBUILT and its test are gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 23:50:27 +00:00
173f28b886 Desk door tests: the one-second lower edge allows jiffy granularity (950 ms still kills a 500 ms bound)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 23:38:33 +00:00
ba9b1013ca Desk door fix batch: the binary tests run one at a time and the accept-error probe fails every new descriptor
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 22:22:18 +00:00
4e43e43a26 Desk door fix batch: one deadline per connection (reads and writes spend what is left), the catalog reply bounded by the response limit, the dead index call and answer_ref deleted
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 22:17:04 +00:00
2b6e15de4c Desk door fix batch: failing tests first (one deadline per connection, the exact bounds, the catalog under the response limit; Soul's survivor probes committed)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 22:13:46 +00:00
4437afbc26 Desk door part B: the binary end-to-end (both doors, restart replay, leftover socket, other uid refused and logged, namespace root seen as the real uid)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 21:42:21 +00:00
035a3eaa62 Desk door part A: the oversize-header test fails fast instead of waiting when the bound is gone
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 21:40:52 +00:00
eca3f53c98 Desk door part A: mutation survivors pinned (header-only oversize refusal, four accepts per pass, unprobeable and unreadable socket paths)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 21:09:57 +00:00
e35181b296 Desk door part A: the unix-socket door, peer uid checked before a byte is read, joined to the one serve loop
desk_door.rs owns who may speak, the socket's life and the frame bounds; Daemon::handle_desk owns which Mind method answers (Answer via desk_answer, the replay flag gating index.committed; Forks and Fork refused Unavailable until desk-door-reads). startup returns Started; run takes the door; its five callers pass it or None. desk_wire::answer_ref is the one spelling of a stored answer's ref.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 20:56:51 +00:00
88ca0a0867 Desk door part A: failing tests first (peer check before a byte, bounds, answers over the socket, per-call deciders, interim reads refusal, startup gates)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 20:53:42 +00:00
2f165d8d1e huginn sources --front: search only a repo's front-page files
SourceQuery gains `front` (serde default false, so an old client's request
decodes as before). The one owner of the list is sources::FRONT_PAGE_FILES;
the Qdrant filter matches `path` against it with MatchAny on the existing
keyword index, composed with the `repo` condition. The CLI prints the
citation line range as before.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 14:47:47 +00:00
1ee0b86f30 Sources: licence rule narrowed to bare or prose-extension names, chunker version stays 1
The skip list now matches `license`, `licence`, `copying` only with no extension or md/txt/rst, so src/license.rs and Copying.cs stay indexed. CHUNKER_VERSION is back to 1: a bump rebuilds the live collection; the already-indexed noise paths go by a one-time delete_by sweep instead (work/sweep/sweep-sources-noise.py, same rule).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 14:06:10 +00:00
6d1766dd72 Sources: skip .obsidian/.voidbot dirs, licence files and *.provenance; bump chunker version to rebuild
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 00:44:25 +00:00
Meta Vens
c76e37280d huginn-mind: a Moved resolution freezes its campaign's root and the receiver replays the history as an import naming its source
The giver: (Campaign, Moved) joins the matrix; a Moved to this mind refuses MoveToSelf and a withdrawn Moved MoveIsFinal; any staged document rooted at a moved campaign, derived ones included, refuses CampaignMoved; Docs::closing_resolution reads every other document under the root closed by the Moved, and the image fans status_changed out so an earlier as_of still reads in force. The oracle derives the same fallback independently.

The receiver: PipelineProvenance.imported_from (optional, absent from every other receipt and from the digest) names the source receipt and the externals the batch cites. An import derives nothing, refuses its own instance, a Moved outcome, and an external this mind holds; a listed external stands in for an absent referent. mind-body persona.rs gains one outcome_words arm for the new variant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 17:47:43 +01:00
Meta Vens
5aa643a6c2 stewardship names: chat names a steward lists, and the Persona and heartbeat address her by them
address.rs is the one rule for whether a delivery is for a Mind (tag, or a name as a whole word, a listed name exactly as written and a derived or her own in any case); heartbeat's addresses and names_whole_word move into it, the Persona's to_you and own stagger rank call it. eureka-pipeline PipelineStewardship gains names (absent when empty); facts, schemas and golden follow.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 00:51:17 +01:00
Meta Vens
f0566fd1bb read floor: Mind holds one decoded, indexed image, a fold of its receipts advanced per admission; a read pulls nothing and no longer grows with history (squash of hands/eureka-substrate-mind-read-floor cb86f58..2bed30c; Soul s1-s2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 11:26:27 +01:00
Meta Vens
cb86f58a7d wear-facts: a Mind's first line shows the wear her own mind holds, specs with no report, open Soul findings and quiet campaigns on what she stewards, each leading to a record she can open (squash of hands/eureka-body-wear-facts ab61a53..6014554; Soul s1-s2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:06:16 +01:00
Meta Vens
545d6319bf repo-activity: a Mind hears of changes in her stewarded repos, on her first line and by one organ notice on change; a repo the anonymous read cannot reach is named, never a failed read (squash of hands/eureka-body-repo-activity 5a294ed..eee0b14; Soul s1-s4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 04:54:26 +01:00
Meta Vens
0af7fd2501 huginn-sources: pin the Qdrant source delete's filter
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:16:17 +01:00
Meta Vens
d49ada3e68 huginn-sources: clippy slice refs in the reserve-bound test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:00:50 +01:00
Meta Vens
d9a4c27e70 huginn-sources: fix the slow-reserve stub's CRLF and the revert test's premise
The failed-upsert revert test now expects chunks of two commits, since
nothing is deleted before the new chunks land.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:58:34 +01:00
Meta Vens
5874717c88 huginn-sources: embed before deleting; the reserve gets a CPU's bounds
Source refresh now upserts a repo's new chunks first and deletes what is
left in scope under another commit afterwards, so a failed embed never
removes indexed content (embed-outage-empties-repo). The cursor is still
dropped before the first write.

The pool's reserve endpoints use EmbedBounds::RESERVE (30 s a text, 60 s
floor) instead of the GPU default (reserve-bounded-like-a-gpu).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 21:54:54 +01:00
Meta Vens
cf790b6142 index: take_search's doc comment back on take_search
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 09:21:42 +01:00
Meta Vens
44b3bf5e91 Merge eureka-body/source-search-index-safety (cursor, catalog, retire, CLI e2e; report h3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 09:21:23 +01:00
Meta Vens
2f36f63e80 ollama: pin the batch bound's arithmetic (cargo-mutants survivor)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 09:17:55 +01:00
Meta Vens
715ec58180 sources: Refreshed::is_clean decides the exit, where a test reaches it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 09:10:07 +01:00
Meta Vens
39decde8a2 embedder: connect bound, a read bound sized to the batch, one budget across retries; tests with endpoints that stall and refuse mid-batch
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 09:05:47 +01:00
Meta Vens
6a0ab4a7ab live source smoke: follow the Catalog the forge now answers
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 09:00:54 +01:00
Meta Vens
e47532c122 sources: a retire error skips that repo; a case-only rename keeps no stale points
Retirement aborted the run on the first repo whose delete failed; it now logs,
names the repo in Refreshed.failed and goes on, leaving that repo's cursor and
mirror to be found again. Names are compared as written, so a repo the forge
now spells in another case is retired under its old spelling instead of kept by
a case-insensitive match while its points carried the old name (Soul
retire-edge-cases).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:58:19 +01:00
Meta Vens
6c1cef8729 sources: retire nothing when the forge's catalog is empty or does not sum to its X-Total-Count
Retirement read the repo list as complete, so an empty or truncated answer
would have deleted every repo it failed to name (Soul catalog-shrink-retires).
The catalog now carries a shortfall: the pages summed before filtering are
checked against X-Total-Count, and a missing header counts as unvouched. The
run refuses to retire on a short or empty catalog, refreshes the repos it did
see, says so in Refreshed.retirement_withheld, and huginn-sources exits 1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:57:38 +01:00
Meta Vens
3f380df0fe sources: drop a repo's cursor before its first delete, so a failed run is re-ingested whole
A failed run from A to B followed by a head C that reverts the files to A's
content diffed A..C as empty and kept the deleted chunks deleted (Soul
cursor-revert-hole: 32 of 40 chunks, cursor at C). With the cursor gone the
next run takes the whole-repo path whatever head it finds.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:56:21 +01:00
Meta Vens
5cde349a09 pool tests: no waiting on reads that are not running; a failed read waits for its retry time
Closes the pool mutation survivors (reading, ready, judge's retry time).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:29:52 +01:00
Meta Vens
1fee2e434e pool: a read awaiting a model to adopt stays to be judged
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:12:26 +01:00
Meta Vens
c74dd530de daemon: embeds through the endpoint pool, GPUs together and the CPU last
query-embed-on-cpu, follow-up daemon-embedder-gpu-pool: huginn-daemon takes
repeated --ollama-url (GPU hosts) and --reserve-ollama-url (CPU hosts) and
builds the same EmbedderPool as huginn-sources, through one ollama::pool
constructor. A lone --ollama-url is a pool of one.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 07:56:22 +01:00
Meta Vens
13647f9220 pool: identities are read off the batch's clock and held to the collection's model
pool-blackhole-stalls: each due endpoint's identity is read on its own thread
under a 5 s bound, and a batch waits for reads only while no healthy primary
exists to start on, so a dark endpoint never delays work a healthy one can do.
pool-pins-first-identity: the pool is told the model the collection was built
with (Embedder::built_under, called by reconcile and refresh before the
identity read) and never writes with an endpoint serving another; it adopts a
model only when there is no collection, in list order.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 07:54:57 +01:00
Meta Vens
99d3c05b09 pool: simpler admission guard (clippy)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 06:58:55 +01:00