Auto-reconcile source repo indexing
This commit is contained in:
parent
32e4a1f184
commit
de223f0fff
16 changed files with 1125 additions and 286 deletions
|
|
@ -27,9 +27,11 @@ Git history and smoke artifacts carry routine proof. `state/evidence.jsonl` shou
|
|||
- The worker MCP surface is no longer one swollen server file; stdio bootstrap stays in `apps/worker/src/mcp-server.ts` while context bootstrap, resource registration, tool registration, Discord notify helpers, and shared formatting/schema helpers live under `apps/worker/src/mcp-server-*.ts`.
|
||||
- Interaction-memory logic is no longer one thousand-line slab; event/tone analysis, shared constants, and profile synthesis are split across `packages/core/src/interaction-memory-*.ts`.
|
||||
- State storage is no longer one persistence omnibus; `packages/core/src/state-storage.ts` is now a thin factory over domain-specific Postgres/file store modules plus bootstrap/migration helpers.
|
||||
- Source archive storage is no longer one giant JSON monolith; `packages/rag/src/source-document-archive.ts` now keeps a small manifest at `.voidbot/rag/source-documents.json` and per-repo shards under `.voidbot/rag/source-documents.repos/`.
|
||||
- Ops health, backup verification, offsite sync, and dashboard surfaces already exist.
|
||||
- The stack has a dedicated logon startup task path now, so a reboot does not have to mean manual necromancy for bot and worker.
|
||||
- The operations watchdog now supports ignored local extension checks, so adjacent-service fire signals can ride the same owner DM alert path without hardcoding private project gossip into the public repo.
|
||||
- Source repo discovery now reconciles automatically during `npm run stack:start` and through the watchdog path, so new repos under `SOURCE_REPO_ROOT` stop waiting for manual hook folklore before they get indexed.
|
||||
- The new state/notes surfaces are now the continuity spine for future nontrivial work.
|
||||
|
||||
## Likely Next Bounded Move
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ This note is the source-grounded description of how the live VoidBot stack is sh
|
|||
- `packages/rag/src/message-archive.ts`
|
||||
- archived Discord message store under `.voidbot/rag/messages.json`.
|
||||
- `packages/rag/src/source-document-archive.ts`
|
||||
- archived source/lore document store under `.voidbot/rag/source-documents.json`.
|
||||
- archived source/lore document manifest under `.voidbot/rag/source-documents.json` plus per-repo shards under `.voidbot/rag/source-documents.repos/`.
|
||||
|
||||
## Flow 1: Discord Request To Provider
|
||||
|
||||
|
|
@ -94,17 +94,18 @@ This note is the source-grounded description of how the live VoidBot stack is sh
|
|||
1. `packages/rag/src/message-archive.ts` and `packages/rag/src/source-document-archive.ts` keep the raw corpora.
|
||||
2. `packages/rag/src/retrieval-service.ts` translates history/source queries into vector lookups plus metadata filters.
|
||||
3. `packages/rag/src/qdrant-vector-store.ts` executes the live vector lookups against separate history and source collections.
|
||||
4. `scripts/index-source-repos.ts` and `scripts/git-post-push-index.mjs` drive detached source/lore reindex work.
|
||||
5. `apps/worker/src/mcp-server.ts` boots the MCP lane, while `mcp-server-resources.ts` and `mcp-server-tools.ts` expose retrieval to Codex and other sessions through `search_history`, `get_message_context`, `search_sources`, `get_source_context`, and `list_indexed_repos`.
|
||||
4. `scripts/reconcile-source-repos.ts` discovers local Git repos, refreshes push hooks, prunes stale repo shards, and indexes newly discovered repos.
|
||||
5. `scripts/index-source-repos.ts` and `scripts/git-post-push-index.mjs` drive explicit or detached per-repo source/lore reindex work.
|
||||
6. `apps/worker/src/mcp-server.ts` boots the MCP lane, while `mcp-server-resources.ts` and `mcp-server-tools.ts` expose retrieval to Codex and other sessions through `search_history`, `get_message_context`, `search_sources`, `get_source_context`, and `list_indexed_repos`.
|
||||
|
||||
## Flow 4: Ops And Recovery
|
||||
|
||||
- `scripts/start-voidbot-stack.ps1`
|
||||
- stack bootstrap, health checks, fresh build, stale-process cleanup, bot/worker restart, runtime status emission.
|
||||
- stack bootstrap, health checks, fresh build, stale-process cleanup, source-repo reconcile, bot/worker restart, runtime status emission.
|
||||
- `scripts/install-stack-startup-task.ps1`
|
||||
- installs the hidden logon task that runs `start-voidbot-stack.ps1` automatically after reboot or sign-in.
|
||||
- `scripts/check-voidbot-operations.ps1`
|
||||
- watchdog for process liveness, Qdrant, Postgres, Ollama, Discord auth, backup freshness, offsite sync freshness, and optional ignored local extension checks.
|
||||
- watchdog for process liveness, Qdrant, Postgres, Ollama, source-repo reconcile drift, Discord auth, backup freshness, offsite sync freshness, and optional ignored local extension checks.
|
||||
- `scripts/backup-voidbot-state.ps1`
|
||||
- local backup of Postgres, Qdrant snapshots, and RAG archives.
|
||||
- `scripts/restore-voidbot-state.ps1`
|
||||
|
|
@ -129,6 +130,7 @@ When present, an ignored local script at `.voidbot/private/check-voidbot-operati
|
|||
- source/lore vectors
|
||||
- `.voidbot/`:
|
||||
- archived raw corpora
|
||||
- source archive manifest plus per-repo source shards
|
||||
- job artifacts and traces
|
||||
- logs and status files
|
||||
- backups and snapshots
|
||||
|
|
|
|||
Loading…
Reference in a new issue