Record restored source refresh authority

This commit is contained in:
Meta Vens 2026-08-30 23:47:22 +02:00
commit 856cfb3eee
6 changed files with 87 additions and 41 deletions

View file

@ -464,3 +464,5 @@
{"ts":"2026-08-16T19:29:57Z","type":"implementation","status":"ok","note":"Added read-only get_exact_source_document MCP tool backed by the canonical sharded source archive; full content is returned for consumer-owned hashing and receipts. Typecheck and in-memory MCP smoke pass."}
{"ts":"2026-08-16T19:34:57Z","type":"verification","status":"ok","note":"Yggdrasil deployed immutable VoidBot recovery release 20260816T193315Z-c88c71909dbc. Starfire called get_exact_source_document through 127.0.0.1:17875 and received all 10,363 characters of AetheriaLore:Aetheria/index.md with SHA-256 ccc26375ecbe638af1aa3ccdeff885dfcc85cf6e7f1c6867e2b42e3e8b374a6d."}
{"ts":"2026-08-30T20:45:00Z","type":"operations","status":"degraded","note":"Yggdrasil VoidBot retrieval is healthy but source refresh has no admitted owner. `voidbot-retrieval.service`, MCP, and Qdrant are up; list/search smokes pass across 49 repos, and the green source collection has 98,170 points with an empty update queue. Most archive shards are still dated 2026-06-03 (AetheriaLore 2026-08-20 and Kalsa 2026-08-25 were later bounded imports). The recovery Compose mounts archives read-only and allow-lists retrieval tools only; no index worker, crawler, hook consumer, or reconcile scheduler is deployed. Restoring freshness requires a new authority map, not a service restart."}
{"ts":"2026-08-30T21:34:00Z","type":"implementation","status":"partial","note":"Source refresh now has a separate admitted Yggdrasil writer authority without widening the read-only MCP: VoidBot b8e377278a5017c4e0aeb702a5ba539cf9fbbae8 supplies catalog-scoped, retry-safe archive/Qdrant indexing; gamecult-ops 3ee4a05a02e30afbe3f78591818dd940008cac83 supplies `voidbot-source-refresh.service`, the hourly timer, flock, durable status/log/pending witnesses, and one stable indexer-container lifecycle. The first 57-repository live catch-up remains in progress under PID 2270422. Aetheria settled inside the pass at 2,190 documents (8 created, 5 updated, 2,177 unchanged, 26 deleted), while AetheriaEve was embedding 282 documents / 3,739 chunks. Whole-pass settlement, final archive/vector agreement, retrieval freshness, timer continuity, and stale-container absence remain unverified."}
{"ts":"2026-08-30T21:46:00Z","type":"operations","status":"partial","note":"Source-writer service restoration and continuity are achieved after lifecycle redesign at VoidBot 3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c and gamecult-ops a1c5fc8bec7f7aaf565d8cddcf486605021c7416. PID 2270422 was deliberately stopped; live PID 2279696 runs the 24-hour oneshot one repo at a time in one stable indexer container, removes pending only after success, emits `embedded_chunks=N/total`, and has an active hourly timer. Current witness is `phase=index`, `repo_1_of_56:AetheriaEve`, pending 56, `embedded_chunks=64/3739`, with Qdrant green around 93,701 points. Aetheria is committed and excluded from reconstructed pending; the measured outstanding delta before its completion was 69,060 chunks across 51 repos. Full corpus catch-up and final archive/vector/retrieval verification remain pending."}

View file

@ -15,13 +15,12 @@ objective:
safely.
current: >
Keep the recovered read-only archive/source retrieval organ healthy on
surviving old Yggdrasil, and restore source-index refresh under a newly
admitted writer authority. The operator has authorized bringing indexing
back up. The query path remains live and read-only; the new writer must own
incremental source crawling plus coordinated archive/Qdrant writes without
entering the retrieval MCP boundary. Keep Discord, worker jobs, Personas,
moderation, and delivery stopped until their missing authority and
credentials are deliberately rebuilt.
surviving old Yggdrasil while the newly admitted source-refresh writer
completes its first recovery pass. The query path remains live and
read-only; `voidbot-source-refresh.service` owns public-repository mirroring
and coordinated archive/Qdrant writes outside the retrieval MCP boundary.
Keep Discord, worker jobs, Personas, moderation, and delivery stopped until
their missing authority and credentials are deliberately rebuilt.
constraints:
- Disaster recovery currently targets trusted old Yggdrasil at 94.130.75.184; retired Big Yggdrasil at 94.130.223.131 is not a GameCult host and must not be trusted or targeted.
@ -294,5 +293,5 @@ rejected_paths:
reason: sleep/distillation that collapses concrete claims into generic abstraction creates memory nonsense and then forces later cleanup compensators
current_status:
summary: Yggdrasil release `11568671a0d72c4e273c2b76d0a2a642788cc347` runs the typed `voidbot.swarm` Eve publisher and the separate read-only `voidbot-retrieval.service`. The retrieval MCP and Qdrant are healthy: 49 source repositories are visible, source search succeeds, and the green source collection contains 98,170 points. The recovery body has no source-index writer, repo crawler, hook consumer, or reconciliation scheduler; retrieval availability therefore does not imply corpus freshness. The Discord bot, worker, Persona scheduler, Postgres, and resident cognition organs remain absent.
next_action: Restore source refresh under one explicit writer owner. Map its exact repository inputs, archive/Qdrant commit boundary, failure rollback, scheduling trigger, progress surface, and deployment path before the first write, then run the smallest bounded incremental catch-up that proves archive and vectors advance together. Keep `voidbot-retrieval.service` read-only; do not use a manual full reindex or revive the retired workstation stack.
summary: Yggdrasil VoidBot revision `3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c` retains the typed swarm publisher and read-only `voidbot-retrieval.service`. Source-writer restoration and continuity are achieved under gamecult-ops revision `a1c5fc8bec7f7aaf565d8cddcf486605021c7416`: systemd oneshot `voidbot-source-refresh.service` has a 24-hour timeout, its hourly timer is active, and `/usr/local/sbin/refresh-voidbot-sources` processes one repository per stable `voidbot-source-indexer` container while removing a repo from pending only after success and logging `embedded_chunks=N/total`. The redesigned live pass is PID 2279696, `phase=index`, `detail=repo_1_of_56:AetheriaEve`, with 56 repos pending and witness `Repo AetheriaEve: embedded_chunks=64/3739`; Qdrant is green at approximately 93,701 points. Aetheria is already committed and excluded from reconstructed pending. Before that commit, the measured outstanding delta was 69,060 chunks across 51 repos. Full catch-up and final collection agreement remain pending. Discord, worker, Persona scheduler, Postgres, and resident cognition organs remain absent.
next_action: Keep observing PID 2279696 through `/srv/voidbot/retrieval/status/source-refresh.status`, `/srv/voidbot/retrieval/log/source-refresh.log`, and `/srv/voidbot/retrieval/status/pending-repos.txt`. Writer service restoration is complete; corpus recovery is not. Verify per-repo progress continues, each success durably leaves pending, the full pass settles with no pending work, archive and Qdrant agree, the read-only MCP lists/searches exact current sources, the hourly timer remains active, and no stale indexer container survives.

View file

@ -2,8 +2,8 @@
## Current Subgoal
Restore source indexing on Yggdrasil under the operator's explicit 2026-08-30
authorization while preserving the recovered read-only retrieval boundary.
Observe the first live Yggdrasil source-refresh recovery pass to a settled,
verified result while preserving the read-only retrieval boundary.
## Working Notes
@ -16,12 +16,29 @@ authorization while preserving the recovered read-only retrieval boundary.
- The recovery corpus is read-only. Diagnose retrieval service, tunnel,
embedding provider, Qdrant collections, and recovered archive independently;
the new indexing writer must remain a separate authority and lifecycle.
- Before the first write, the source-refresh owner must name exact repository
inputs, coordinated archive/Qdrant commit and rollback behavior, scheduling
trigger, durable progress witness, and deployment path.
- Start with the smallest bounded incremental catch-up that proves archive and
vectors advance together. A manual full reindex and the retired workstation
stack remain rejected shortcuts.
- Writer owner is systemd oneshot `voidbot-source-refresh.service`, triggered by
hourly `voidbot-source-refresh.timer`. `/usr/local/sbin/refresh-voidbot-sources`
owns the public catalog, repo mirror, pending queue, flock, and stable
`voidbot-source-indexer` container cleanup.
- Deployed authorities: VoidBot
`3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c`; gamecult-ops
`a1c5fc8bec7f7aaf565d8cddcf486605021c7416`.
- Writer service restoration and continuity are achieved: the oneshot has a
24-hour timeout, the hourly timer is active, one stable indexer container
processes one repo at a time, pending drops only after repo success, and the
log emits `embedded_chunks=N/total` progress.
- PID 2270422 was stopped during that lifecycle redesign. Live PID 2279696 is
`phase=index`, `detail=repo_1_of_56:AetheriaEve`, with 56 repos pending and
witness `Repo AetheriaEve: embedded_chunks=64/3739`. Qdrant is green at about
93,701 points.
- Aetheria is committed and excluded from reconstructed pending. Before that
completion, the measured outstanding delta was 69,060 chunks across 51
repos. Full catch-up remains pending.
- Poll `/srv/voidbot/retrieval/status/source-refresh.status`; inspect
`/srv/voidbot/retrieval/log/source-refresh.log` and
`/srv/voidbot/retrieval/status/pending-repos.txt`. After settlement, verify
empty pending work, archive/Qdrant agreement, current MCP retrieval, active
hourly timer, and absence of a stale indexer container.
- Last durable retrieval proof is the 2026-08-16 exact-document witness for
release `20260816T193315Z-c88c71909dbc`; it is historical evidence, not a
present-health result.