Record restored source refresh authority
This commit is contained in:
Meta Vens
parent
3fa4a87420
commit
856cfb3eee
6 changed files with 87 additions and 41 deletions
|
|
@ -19,7 +19,8 @@ Git history and smoke artifacts carry routine proof. `state/evidence.jsonl` shou
|
|||
- Disaster recovery is active on trusted old Yggdrasil (`94.130.75.184`). Big Yggdrasil (`94.130.223.131`) is retired and its recycled address is not a GameCult host. The surviving host did not retain the former VoidBot runtime state or credentials; the current body was rebuilt only from admitted recovery material.
|
||||
- Raven's verified offsite archive `20260603-043304-offsite-auto.zip` is staged on old Yggdrasil. Its SHA-256 is `47572df766e63212012eb75762359113a11c8bbf149b497e120fc4fda7191dfc`; the newer June 4 archive is corrupt. The payload contains PostgreSQL state, six Qdrant snapshots, and exact RAG archives, but no secrets.
|
||||
- The standalone read-only resident MCP was recovered first. `voidbot-retrieval.service` owns only loopback `127.0.0.1:17875`, exact archives, VoidBot's history/source Qdrant collections, and the matching embedding model. Discord, worker jobs, Personas, moderation, Bifrost crossings, and delivery remain stopped until their missing authority and credentials are rebuilt.
|
||||
- The operator authorized bringing source indexing back up on 2026-08-30. This admits a new source-refresh writer, not mutation authority for `voidbot-retrieval.service`. Before its first write, name one owner, repository inputs, coordinated archive/Qdrant commit and rollback boundary, trigger, progress witness, and deployment path; then prove the smallest incremental catch-up before broad reconciliation.
|
||||
- Source refresh is admitted separately from retrieval. Under VoidBot revision `3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c` and gamecult-ops revision `a1c5fc8bec7f7aaf565d8cddcf486605021c7416`, systemd oneshot `voidbot-source-refresh.service` has a 24-hour timeout and the hourly `voidbot-source-refresh.timer` is active. `/usr/local/sbin/refresh-voidbot-sources` owns the public GameCult catalog, mirrored checkouts, retry-safe archive/Qdrant commits, flock exclusion, durable status/log/pending witnesses, and one stable `voidbot-source-indexer` container processing one repo at a time. A repo leaves pending only after success, and logs expose `embedded_chunks=N/total`. `voidbot-retrieval.service` remains read-only.
|
||||
- Writer service restoration and continuity are achieved; corpus catch-up is not. PID 2270422 was stopped during lifecycle redesign. Live PID 2279696 is `phase=index`, `detail=repo_1_of_56:AetheriaEve`, with 56 repos pending and witness `Repo AetheriaEve: embedded_chunks=64/3739`; Qdrant is green at roughly 93,701 points. Aetheria is already committed and excluded from reconstructed pending. Before its completion, the measured outstanding delta was 69,060 chunks across 51 repos. Final success still requires a settled pass, empty pending queue, archive/Qdrant agreement, current MCP retrieval, active timer, and no stale indexer container.
|
||||
- `apps/worker/src/mcp-http-main.ts` is the standalone HTTP entrypoint. It delegates to the existing MCP context and HTTP server; it is not a second retrieval implementation.
|
||||
- The current `voidbot.service` runs only the typed `voidbot.swarm` Eve publisher. The repo Face/native Persona scheduler, worker queue, Discord bot, Postgres body, and resident cognition organs described later in this handoff are source architecture or retired-runtime history, not current service-health claims. Restore none of them by implication from the surviving swarm projection.
|
||||
- The teardown authority remains `notes/agent-swarm-teardown-plan-2026-05-21.md`, but it is now a restart checklist and smell ledger rather than a stop sign. Keep using it when a Face/Bifrost/CTB boundary smells like prompt/parser/transport Jenga.
|
||||
|
|
|
|||
|
|
@ -6,13 +6,13 @@ Verse-facing service contract: `docs/architecture/voidbot-verse-service-contract
|
|||
|
||||
## Deployment Authority
|
||||
|
||||
VoidBot's current Yggdrasil body is deliberately narrow. Idunn deploys upstream `main` into immutable releases under `/srv/voidbot/swarm/releases`, promotes `/srv/voidbot/swarm/current`, and verifies `/srv/voidbot/deploy/deployment.env`. `voidbot.service` runs only the typed `voidbot.swarm` Eve publisher on `127.0.0.1:17873`; its signed health is admitted as `voidbot.cultnet-rudp-stack-health`. `voidbot-retrieval.service` separately owns the bounded read-only recovery corpus. The lost Discord bot, worker, Persona scheduler, Postgres, Qdrant, and resident cognition state were not restored.
|
||||
VoidBot's current Yggdrasil body is deliberately narrow. Idunn deploys upstream `main` into immutable releases under `/srv/voidbot/swarm/releases`, promotes `/srv/voidbot/swarm/current`, and verifies `/srv/voidbot/deploy/deployment.env`. `voidbot.service` runs only the typed `voidbot.swarm` Eve publisher on `127.0.0.1:17873`; its signed health is admitted as `voidbot.cultnet-rudp-stack-health`. `voidbot-retrieval.service` separately owns the bounded read-only recovery corpus at `127.0.0.1:17875`, backed by the restored archive shards and Qdrant collections. A separate root-owned `voidbot-source-refresh.service` may mutate only the source archive and source-vector collection through a short-lived indexer container. The Discord bot, resident worker/job loop, Persona scheduler, Postgres, and resident cognition state were not restored.
|
||||
|
||||
The source-organ inventory below describes repository capabilities and the retired full-stack design. It is not a claim that those organs currently run on Yggdrasil. Restoring any of them requires a new recovery and authority map rather than expanding the swarm publisher by implication.
|
||||
The source-organ inventory below mostly describes repository capabilities and the retired full-stack design. The exceptions are the standalone retrieval entrypoint and the source-index entrypoint used by the bounded recovery services. No resident worker has returned. Restoring any other organ requires a new recovery and authority map rather than expanding the swarm publisher, retrieval service, or refresh timer by implication.
|
||||
|
||||
The live graphical path is `voidbot.swarm` publisher -> Odin accepted state -> Gjallar `gjallar.overview` -> Hermodr lowering -> EveCanvas/browser. The Yggdrasil Hermodr body runs beside Odin; EVE currently reaches the Starfire LAN lowering. Both consume Odin/CultMesh and own no provider or aggregate truth.
|
||||
|
||||
## Source Organs (Not Current Processes)
|
||||
## Source Organs (Mostly Not Current Processes)
|
||||
|
||||
- `apps/bot/src/discord-bot.ts`
|
||||
- Discord gateway shell: client bootstrap, provider wiring, event registration, and top-level orchestration.
|
||||
|
|
@ -108,14 +108,29 @@ The live graphical path is `voidbot.swarm` publisher -> Odin accepted state -> G
|
|||
|
||||
## Source Flow 3: Retrieval And Indexing
|
||||
|
||||
### Live source-refresh Body
|
||||
|
||||
- Owner: the root-owned `voidbot-source-refresh.service` is the only admitted source-index mutation coordinator. Its timer supplies cadence but owns no indexing decision. The Compose `source-indexer` is a short-lived actuator, not a resident worker and not another owner.
|
||||
- Inputs: each run fetches the current non-archived public GameCult repository catalog from GitHub, validates and synchronizes dedicated mirrors under `/srv/voidbot/source-repos`, and indexes from that admitted catalog using the exact deployed VoidBot release. The indexer also reads the existing source archive, the Ollama embedding endpoint, and the source Qdrant collection. It does not scan arbitrary host repositories.
|
||||
- Outputs: successful repo work updates the per-repo source archive shard and the `voidbot_repository_source_chunks` collection. The archive is the canonical exact-document read corpus; Qdrant is its semantic retrieval projection. The public MCP only reads those outputs.
|
||||
- Derived and coordination state: `/srv/voidbot/retrieval/source-catalog.tsv`, `status/repo-sync.status`, `status/source-refresh.status`, `status/changed-repos.txt`, and `log/source-refresh.log` are operational witnesses. `status/pending-repos.txt` is the bounded retry queue: failed work remains pending until a successful index pass clears it. None of these files may answer retrieval queries or replace the source archive.
|
||||
- Forbidden writers: `voidbot-retrieval.service`, the MCP bridge, the swarm publisher, repo push hooks, workstation startup tasks, and a resident worker/job loop cannot mutate source state. The retrieval container keeps its archive mount read-only and exposes only the six retrieval tools. The indexer may mutate source archive shards and source vectors; it has no Discord, Persona, job, history-archive, swarm-publication, or deployment authority.
|
||||
- Shared paths: an hourly timer activation and an operator's `systemctl start voidbot-source-refresh.service` both enter `/usr/local/sbin/refresh-voidbot-sources`. There is no separate manual indexing truth on Yggdrasil.
|
||||
- Cut line: the old workstation reconcile/push-hook recurrence and the retired full-stack worker are not recovery dependencies. Source refresh earns one oneshot service and one timer because indexing needs an independent mutation lifecycle; it does not revive the rest of the lost runtime.
|
||||
- Verification layer: systemd service/timer state proves scheduling, the atomically replaced source-refresh status plus repo-sync status and log prove phase/progress, archive shard summaries prove exact-document commit, Qdrant collection health/count and MCP source searches prove the user-visible semantic layer, and the deployed source commit proves which indexer code ran. A green timer alone is not source health.
|
||||
- Failure and cleanup: the root wrapper holds a non-blocking `flock`, removes a stale Compose-labeled source-indexer only after acquiring that lock, and traps exit to remove the fixed-name container and private scratch directory. Mirror changes are merged into `pending-repos.txt` before indexing. The source pipeline plans against the current shard, mutates vectors, then commits the archive only if the shard still matches the plan's starting state. Embedding/vector failure therefore cannot advance the archive or clear pending work, although Qdrant may be partially changed until the retained retry succeeds; health checks must inspect both archive and vector reality.
|
||||
- Change identity: checkout mtimes and CRLF/LF differences are observations, not source changes. Document identity compares canonical repo/path/language/title/content/metadata, with line endings normalized. This prevents fresh mirrors from forcing full-corpus embeddings while preserving real content and metadata changes.
|
||||
|
||||
Read flow:
|
||||
|
||||
1. `packages/rag/src/message-archive.ts` and `packages/rag/src/source-document-archive.ts` keep the raw corpora.
|
||||
2. Bot-directed prompts are tagged at ingest, kept in the raw Discord archive, and deliberately skipped when the history ingester builds semantic chunks so repeated summons stop poisoning normal retrieval.
|
||||
3. `packages/rag/src/retrieval-service.ts` translates history/source queries into vector lookups plus metadata filters.
|
||||
4. `packages/rag/src/qdrant-vector-store.ts` executes the live vector lookups against separate history and source collections.
|
||||
5. `scripts/reconcile-source-repos.ts` discovers local Git repos, refreshes push hooks, prunes stale repo shards, and indexes newly discovered repos.
|
||||
6. `scripts/index-source-repos.ts` and `scripts/git-post-push-index.mjs` drive explicit or detached per-repo source/lore reindex work.
|
||||
7. `apps/worker/src/mcp-server.ts` boots the MCP lane, while `mcp-server-resources.ts` and `mcp-server-tools.ts` expose retrieval to Codex and other sessions through `search_history`, `get_message_context`, `search_sources`, `get_source_context`, `get_exact_source_document`, and `list_indexed_repos`. Exact-document reads come from the canonical read-only source archive and return the complete archived content for consumer-owned hashing and receipts; they do not read a checkout or mutate the archive.
|
||||
8. The current Yggdrasil endpoint at `127.0.0.1:17875/mcp` belongs to the standalone read-only `voidbot-retrieval.service`, not a resident worker or job loop. It owns bounded recovery access to the retained archives and source corpus and has no bot, job, Persona, scheduler, or swarm-publication authority. The task-local `serve-voidbot-mcp-bridge.mjs` remains transport-only: it converts endpoint or tunnel unavailability into an ordinary tool error, cannot read archives or Void state itself, and starts no SSH, Docker, or remote process.
|
||||
5. `apps/worker/src/mcp-server.ts` boots the MCP lane, while `mcp-server-resources.ts` and `mcp-server-tools.ts` expose retrieval to Codex and other sessions through `search_history`, `get_message_context`, `search_sources`, `get_source_context`, `get_exact_source_document`, and `list_indexed_repos`. Exact-document reads come from the canonical read-only source archive and return the complete archived content for consumer-owned hashing and receipts; they do not read a checkout or mutate the archive.
|
||||
6. The current Yggdrasil endpoint at `127.0.0.1:17875/mcp` belongs to the standalone read-only `voidbot-retrieval.service`, not a resident worker or job loop. It owns bounded recovery access to the retained archives and source corpus and has no bot, job, Persona, scheduler, source-index, or swarm-publication authority. The task-local `serve-voidbot-mcp-bridge.mjs` remains transport-only: it converts endpoint or tunnel unavailability into an ordinary tool error, cannot read archives or Void state itself, and starts no SSH, Docker, or remote process.
|
||||
|
||||
Repository-local `scripts/reconcile-source-repos.ts`, `scripts/index-source-repos.ts`, and `scripts/git-post-push-index.mjs` remain local/full-stack tools. They are not the live Yggdrasil cadence or writer authority.
|
||||
|
||||
## Historical Flow 4: Ops And Recovery
|
||||
|
||||
|
|
@ -241,7 +256,7 @@ Target Eve surfaces:
|
|||
- `voidbot.discord`: room obligations, direct mentions, speech receipts, moderation/open-case pressure, delivery targets.
|
||||
- `voidbot.reddit`: r/GameCultOrg thread/post obligations, Persona-authored thread ideas, moderation/open-case pressure, proposed replies/actions, and Bifrost transport receipts.
|
||||
- `voidbot.archive`: archived Discord corpus status, import/backfill health, bot-directed-prompt exclusion, history vector freshness.
|
||||
- `voidbot.source`: repo/lore shard freshness, indexed coverage, Qdrant collection health, detached reindex jobs.
|
||||
- `voidbot.source`: repo/lore shard freshness, indexed coverage, Qdrant collection health, pending retry pressure, and the oneshot refresh phase. This target is not yet a published provider surface; current source health is observed through systemd, status witnesses, storage probes, and MCP retrieval checks.
|
||||
- `voidbot.repo_face`: registered identities, repo-local `.cc` witnesses, channel grants, prompt assembly status, Bifrost digest availability, Huginn inspection readiness.
|
||||
- `voidbot.swarm`: CTB order, active turns, pending mention queues, heat/cadence controls, daemon status, selected Face state witness.
|
||||
|
||||
|
|
@ -257,23 +272,23 @@ Migration order:
|
|||
|
||||
Huginn demotion line: VoidBot may carry repo Face `.cc` state paths, MCP diagnostics, and Discord compatibility registry data, but Persona/.cc inspection and portable Persona publication belong to Huginn. VoidBot must not let its private registry, static HTML, or legacy MCP state tools become the canonical Persona authority.
|
||||
|
||||
## Source Storage Boundaries (Not Current Runtime)
|
||||
## Storage Boundaries
|
||||
|
||||
- Postgres:
|
||||
- Postgres (retired/lost on the current Yggdrasil body):
|
||||
- jobs
|
||||
- audit events
|
||||
- interaction-memory events
|
||||
- provider run records
|
||||
- usage rate-limit state
|
||||
- Qdrant:
|
||||
- Discord history vectors
|
||||
- source/lore vectors
|
||||
- `.voidbot/`:
|
||||
- archived raw corpora
|
||||
- source archive manifest plus per-repo source shards
|
||||
- job artifacts and traces
|
||||
- logs and status files
|
||||
- backups and snapshots
|
||||
- restored Discord history vectors, read by the live retrieval service
|
||||
- restored and incrementally refreshed source/lore vectors, read by retrieval and written only by the source indexer
|
||||
- `.voidbot/` (mixed recovered read corpus and live source-refresh state):
|
||||
- restored archived raw corpora
|
||||
- live source archive manifest plus per-repo source shards, read by retrieval and written only by the source indexer
|
||||
- retained historical job artifacts and traces; no current job loop writes them
|
||||
- source-refresh logs, status witnesses, catalog, and retry queue
|
||||
- recovery backups and snapshots
|
||||
|
||||
The important scar is that these stores are split on purpose. Do not casually weld them back into one convenient blob and act surprised when it becomes a moon.
|
||||
|
||||
|
|
@ -345,3 +360,9 @@ job, Persona, moderation, Bifrost, Idunn, or state-mutation authority.
|
|||
snapshots read without creating writer locks, while overwrite, mutation, and
|
||||
normalization fail before touching the filesystem. Writable runtimes retain
|
||||
the existing interprocess lock path.
|
||||
|
||||
The second restored organ is the bounded source-refresh oneshot described in
|
||||
Source Flow 3. It shares the recovered source archive and source Qdrant
|
||||
collection with the read-only MCP but has a separate lifecycle, writable mount,
|
||||
fixed tool surface, retry queue, and timer. This is a narrow source-corpus
|
||||
writer, not restoration of the worker, job loop, or full daemon stack.
|
||||
|
|
|
|||
|
|
@ -10,11 +10,17 @@ This is the current forward plan for the next larger organs. It is not a changel
|
|||
> decision. Do not treat the roadmap below as evidence that those processes are
|
||||
> live.
|
||||
|
||||
Source-index refresh restoration is now operator-authorized. It must return as
|
||||
a separate writer authority; the read-only retrieval MCP keeps no crawler,
|
||||
mutation tool, archive writer, or Qdrant write lifecycle. The immediate work is
|
||||
to name that writer's inputs, atomicity/rollback boundary, schedule, progress
|
||||
witness, and deployment owner, then prove a bounded incremental catch-up.
|
||||
Source-index refresh is now implemented as a separate writer authority. Systemd
|
||||
oneshot `voidbot-source-refresh.service` and hourly
|
||||
`voidbot-source-refresh.timer` own the public-repository mirror and retry-safe
|
||||
archive/Qdrant refresh through `/usr/local/sbin/refresh-voidbot-sources`; the
|
||||
read-only retrieval MCP keeps no crawler or mutation lifecycle. The first live
|
||||
catch-up was restarted after lifecycle hardening: the service now has a 24-hour
|
||||
timeout, handles one repository at a time in one stable container, drops pending
|
||||
work only after success, and emits chunk progress while the hourly timer remains
|
||||
active. Writer continuity is restored. Corpus recovery is still running, so the
|
||||
immediate work is settlement and end-to-end verification, not another ownership
|
||||
design pass and not a premature declaration that freshness is restored.
|
||||
|
||||
## Current Aim
|
||||
|
||||
|
|
|
|||
|
|
@ -464,3 +464,5 @@
|
|||
{"ts":"2026-08-16T19:29:57Z","type":"implementation","status":"ok","note":"Added read-only get_exact_source_document MCP tool backed by the canonical sharded source archive; full content is returned for consumer-owned hashing and receipts. Typecheck and in-memory MCP smoke pass."}
|
||||
{"ts":"2026-08-16T19:34:57Z","type":"verification","status":"ok","note":"Yggdrasil deployed immutable VoidBot recovery release 20260816T193315Z-c88c71909dbc. Starfire called get_exact_source_document through 127.0.0.1:17875 and received all 10,363 characters of AetheriaLore:Aetheria/index.md with SHA-256 ccc26375ecbe638af1aa3ccdeff885dfcc85cf6e7f1c6867e2b42e3e8b374a6d."}
|
||||
{"ts":"2026-08-30T20:45:00Z","type":"operations","status":"degraded","note":"Yggdrasil VoidBot retrieval is healthy but source refresh has no admitted owner. `voidbot-retrieval.service`, MCP, and Qdrant are up; list/search smokes pass across 49 repos, and the green source collection has 98,170 points with an empty update queue. Most archive shards are still dated 2026-06-03 (AetheriaLore 2026-08-20 and Kalsa 2026-08-25 were later bounded imports). The recovery Compose mounts archives read-only and allow-lists retrieval tools only; no index worker, crawler, hook consumer, or reconcile scheduler is deployed. Restoring freshness requires a new authority map, not a service restart."}
|
||||
{"ts":"2026-08-30T21:34:00Z","type":"implementation","status":"partial","note":"Source refresh now has a separate admitted Yggdrasil writer authority without widening the read-only MCP: VoidBot b8e377278a5017c4e0aeb702a5ba539cf9fbbae8 supplies catalog-scoped, retry-safe archive/Qdrant indexing; gamecult-ops 3ee4a05a02e30afbe3f78591818dd940008cac83 supplies `voidbot-source-refresh.service`, the hourly timer, flock, durable status/log/pending witnesses, and one stable indexer-container lifecycle. The first 57-repository live catch-up remains in progress under PID 2270422. Aetheria settled inside the pass at 2,190 documents (8 created, 5 updated, 2,177 unchanged, 26 deleted), while AetheriaEve was embedding 282 documents / 3,739 chunks. Whole-pass settlement, final archive/vector agreement, retrieval freshness, timer continuity, and stale-container absence remain unverified."}
|
||||
{"ts":"2026-08-30T21:46:00Z","type":"operations","status":"partial","note":"Source-writer service restoration and continuity are achieved after lifecycle redesign at VoidBot 3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c and gamecult-ops a1c5fc8bec7f7aaf565d8cddcf486605021c7416. PID 2270422 was deliberately stopped; live PID 2279696 runs the 24-hour oneshot one repo at a time in one stable indexer container, removes pending only after success, emits `embedded_chunks=N/total`, and has an active hourly timer. Current witness is `phase=index`, `repo_1_of_56:AetheriaEve`, pending 56, `embedded_chunks=64/3739`, with Qdrant green around 93,701 points. Aetheria is committed and excluded from reconstructed pending; the measured outstanding delta before its completion was 69,060 chunks across 51 repos. Full corpus catch-up and final archive/vector/retrieval verification remain pending."}
|
||||
|
|
|
|||
|
|
@ -15,13 +15,12 @@ objective:
|
|||
safely.
|
||||
current: >
|
||||
Keep the recovered read-only archive/source retrieval organ healthy on
|
||||
surviving old Yggdrasil, and restore source-index refresh under a newly
|
||||
admitted writer authority. The operator has authorized bringing indexing
|
||||
back up. The query path remains live and read-only; the new writer must own
|
||||
incremental source crawling plus coordinated archive/Qdrant writes without
|
||||
entering the retrieval MCP boundary. Keep Discord, worker jobs, Personas,
|
||||
moderation, and delivery stopped until their missing authority and
|
||||
credentials are deliberately rebuilt.
|
||||
surviving old Yggdrasil while the newly admitted source-refresh writer
|
||||
completes its first recovery pass. The query path remains live and
|
||||
read-only; `voidbot-source-refresh.service` owns public-repository mirroring
|
||||
and coordinated archive/Qdrant writes outside the retrieval MCP boundary.
|
||||
Keep Discord, worker jobs, Personas, moderation, and delivery stopped until
|
||||
their missing authority and credentials are deliberately rebuilt.
|
||||
|
||||
constraints:
|
||||
- Disaster recovery currently targets trusted old Yggdrasil at 94.130.75.184; retired Big Yggdrasil at 94.130.223.131 is not a GameCult host and must not be trusted or targeted.
|
||||
|
|
@ -294,5 +293,5 @@ rejected_paths:
|
|||
reason: sleep/distillation that collapses concrete claims into generic abstraction creates memory nonsense and then forces later cleanup compensators
|
||||
|
||||
current_status:
|
||||
summary: Yggdrasil release `11568671a0d72c4e273c2b76d0a2a642788cc347` runs the typed `voidbot.swarm` Eve publisher and the separate read-only `voidbot-retrieval.service`. The retrieval MCP and Qdrant are healthy: 49 source repositories are visible, source search succeeds, and the green source collection contains 98,170 points. The recovery body has no source-index writer, repo crawler, hook consumer, or reconciliation scheduler; retrieval availability therefore does not imply corpus freshness. The Discord bot, worker, Persona scheduler, Postgres, and resident cognition organs remain absent.
|
||||
next_action: Restore source refresh under one explicit writer owner. Map its exact repository inputs, archive/Qdrant commit boundary, failure rollback, scheduling trigger, progress surface, and deployment path before the first write, then run the smallest bounded incremental catch-up that proves archive and vectors advance together. Keep `voidbot-retrieval.service` read-only; do not use a manual full reindex or revive the retired workstation stack.
|
||||
summary: Yggdrasil VoidBot revision `3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c` retains the typed swarm publisher and read-only `voidbot-retrieval.service`. Source-writer restoration and continuity are achieved under gamecult-ops revision `a1c5fc8bec7f7aaf565d8cddcf486605021c7416`: systemd oneshot `voidbot-source-refresh.service` has a 24-hour timeout, its hourly timer is active, and `/usr/local/sbin/refresh-voidbot-sources` processes one repository per stable `voidbot-source-indexer` container while removing a repo from pending only after success and logging `embedded_chunks=N/total`. The redesigned live pass is PID 2279696, `phase=index`, `detail=repo_1_of_56:AetheriaEve`, with 56 repos pending and witness `Repo AetheriaEve: embedded_chunks=64/3739`; Qdrant is green at approximately 93,701 points. Aetheria is already committed and excluded from reconstructed pending. Before that commit, the measured outstanding delta was 69,060 chunks across 51 repos. Full catch-up and final collection agreement remain pending. Discord, worker, Persona scheduler, Postgres, and resident cognition organs remain absent.
|
||||
next_action: Keep observing PID 2279696 through `/srv/voidbot/retrieval/status/source-refresh.status`, `/srv/voidbot/retrieval/log/source-refresh.log`, and `/srv/voidbot/retrieval/status/pending-repos.txt`. Writer service restoration is complete; corpus recovery is not. Verify per-repo progress continues, each success durably leaves pending, the full pass settles with no pending work, archive and Qdrant agree, the read-only MCP lists/searches exact current sources, the hourly timer remains active, and no stale indexer container survives.
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
## Current Subgoal
|
||||
|
||||
Restore source indexing on Yggdrasil under the operator's explicit 2026-08-30
|
||||
authorization while preserving the recovered read-only retrieval boundary.
|
||||
Observe the first live Yggdrasil source-refresh recovery pass to a settled,
|
||||
verified result while preserving the read-only retrieval boundary.
|
||||
|
||||
## Working Notes
|
||||
|
||||
|
|
@ -16,12 +16,29 @@ authorization while preserving the recovered read-only retrieval boundary.
|
|||
- The recovery corpus is read-only. Diagnose retrieval service, tunnel,
|
||||
embedding provider, Qdrant collections, and recovered archive independently;
|
||||
the new indexing writer must remain a separate authority and lifecycle.
|
||||
- Before the first write, the source-refresh owner must name exact repository
|
||||
inputs, coordinated archive/Qdrant commit and rollback behavior, scheduling
|
||||
trigger, durable progress witness, and deployment path.
|
||||
- Start with the smallest bounded incremental catch-up that proves archive and
|
||||
vectors advance together. A manual full reindex and the retired workstation
|
||||
stack remain rejected shortcuts.
|
||||
- Writer owner is systemd oneshot `voidbot-source-refresh.service`, triggered by
|
||||
hourly `voidbot-source-refresh.timer`. `/usr/local/sbin/refresh-voidbot-sources`
|
||||
owns the public catalog, repo mirror, pending queue, flock, and stable
|
||||
`voidbot-source-indexer` container cleanup.
|
||||
- Deployed authorities: VoidBot
|
||||
`3fa4a8742060ac54b9f0e27b8d8751db76fd3a8c`; gamecult-ops
|
||||
`a1c5fc8bec7f7aaf565d8cddcf486605021c7416`.
|
||||
- Writer service restoration and continuity are achieved: the oneshot has a
|
||||
24-hour timeout, the hourly timer is active, one stable indexer container
|
||||
processes one repo at a time, pending drops only after repo success, and the
|
||||
log emits `embedded_chunks=N/total` progress.
|
||||
- PID 2270422 was stopped during that lifecycle redesign. Live PID 2279696 is
|
||||
`phase=index`, `detail=repo_1_of_56:AetheriaEve`, with 56 repos pending and
|
||||
witness `Repo AetheriaEve: embedded_chunks=64/3739`. Qdrant is green at about
|
||||
93,701 points.
|
||||
- Aetheria is committed and excluded from reconstructed pending. Before that
|
||||
completion, the measured outstanding delta was 69,060 chunks across 51
|
||||
repos. Full catch-up remains pending.
|
||||
- Poll `/srv/voidbot/retrieval/status/source-refresh.status`; inspect
|
||||
`/srv/voidbot/retrieval/log/source-refresh.log` and
|
||||
`/srv/voidbot/retrieval/status/pending-repos.txt`. After settlement, verify
|
||||
empty pending work, archive/Qdrant agreement, current MCP retrieval, active
|
||||
hourly timer, and absence of a stale indexer container.
|
||||
- Last durable retrieval proof is the 2026-08-16 exact-document witness for
|
||||
release `20260816T193315Z-c88c71909dbc`; it is historical evidence, not a
|
||||
present-health result.
|
||||
|
|
|
|||
Loading…
Reference in a new issue