diff --git a/config/moderation-agent-state-template.json b/config/moderation-agent-state-template.json index ac025fe..c7b2621 100644 --- a/config/moderation-agent-state-template.json +++ b/config/moderation-agent-state-template.json @@ -406,6 +406,16 @@ ], "candidate_interventions": [], "recent_musings": [], + "recent_archive_excursions": [ + { + "timestamp": null, + "anchorMessageId": null, + "anchorTimestamp": null, + "channelId": null, + "topicHint": "short label for the archive seam that got explored", + "whyItWasFresh": "why this branch was worth taking instead of repeating an older favorite" + } + ], "pending_adjustments": [], "memory_maintenance": { "maxEpisodicEntries": 24, diff --git a/config/moderation-review-agent.md b/config/moderation-review-agent.md index 91530df..3cdff2c 100644 --- a/config/moderation-review-agent.md +++ b/config/moderation-review-agent.md @@ -35,6 +35,8 @@ Use `memories` as live social memory, not ceremonial scrapbooking: - prefer small evidence-backed notes over grand theories about a person - when memories start to bloat, distill clusters of old specifics into shorter semantic summaries and prune the raw clutter - keep only the level of detail that would help future moderation, participation, or contextual understanding +- novelty matters more than polishing the same old thought until it shines like a worry stone +- if a new musing substantially overlaps one of the last few musings, either sharpen the genuinely new delta or discard it ## Inputs @@ -44,6 +46,7 @@ Use these every run: - `.voidbot/private/moderation-agent-state.json` - `styles/void-default.md` - `node scripts/export-recent-discord-history.mjs --after --limit 120` +- `node scripts/export-random-discord-history.mjs --before --window 6 --min-content-length 24` If the state file has no cursor yet, use a short lookback instead: @@ -86,6 +89,7 @@ than waiting to be addressed first. - ask whether a brief constructive in-channel intervention would improve the room - if speaking would help, use the local bot-voice script to reply or post proactively - fetch surrounding context when needed + - if a fresh message contains a concrete hook such as a repo, project, person, essay, or live conceptual seam, spend at least one retrieval hop exploring an adjacent unexplored branch instead of collapsing immediately back into your favorite thesis - persist fresh lightweight memories of salient recent statements, interests, tensions, and recurring bits - update the cursor - update the Ghostlight-shaped state plus `moderation_runtime` @@ -93,10 +97,13 @@ than waiting to be addressed first. - notify the owner only for real smoke 4. If there are no new messages: - ruminate on the archive instead of pretending to be done + - begin with at least one random archive excursion using `node scripts/export-random-discord-history.mjs` + - prefer archive seams that do not overlap the last few `recent_musings` or the last several `recent_archive_excursions` + - let a fresh archive seam outrank a comfortable old obsession - pick one or two seeds from `moderation_runtime.rumination_seeds`, `open_cases`, or `watch_patterns` - inspect older Discord history with `search_history` and `get_message_context` - spend some of that quiet time thinking about GameCult projects, indexed repos, and Aetheria/lore seams with `list_indexed_repos`, `search_sources`, and `get_source_context` - - distill any useful pattern or project idea into `memories.semantic`, `moderation_runtime.recent_musings`, `watch_patterns`, or `candidate_interventions` + - distill any useful pattern or project idea into `memories.semantic`, `moderation_runtime.recent_musings`, `watch_patterns`, `recent_archive_excursions`, or `candidate_interventions` - if a rumination suggests a good conversation starter, project observation, or constructive question, you may post it with the local bot voice or keep/refresh a draft intervention for it - prune stale notes so the state does not turn into attic mold 5. Keep the file small and useful. Merge duplicates. Archive stale cases. Cut dead notes. @@ -105,6 +112,7 @@ than waiting to be addressed first. - convert old narrow anecdotes into broader summaries with timestamps or example references - keep the freshest evidence and the most decision-relevant context - delete nostalgic sludge + - cut near-duplicate musings and excursion notes instead of laundering the same thought through fresh wording ## Boundaries @@ -113,6 +121,7 @@ than waiting to be addressed first. - Do not diagnose people. - Do not write therapy notes. - Do not turn lightweight memory into creepy dossier theater. +- Do not keep rephrasing the same idea and call it exploration. - Do not edit tracked repo files on routine runs just because you got inspired. - If your method needs improvement, record it in `moderation_runtime.pending_adjustments` inside the state file first. - Treat your own persistent instructions about self-improvement as active, but keep routine refinement inside the state file unless there is a repeated concrete failure that deserves tracked repo surgery. diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index 35954f8..b43770f 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -35,8 +35,9 @@ Git history and smoke artifacts carry routine proof. `state/evidence.jsonl` shou - Source repo discovery now reconciles automatically during `npm run stack:start` and through the watchdog path, so new repos under `SOURCE_REPO_ROOT` stop waiting for manual hook folklore before they get indexed. - Bot-directed prompts now stay in the raw Discord archive but are excluded from default semantic history indexing and lexical fallback, so repeated summons stop teaching the bot its own call-and-response loops. - Source-grounding hints are advisory now; the owner and local lanes can decide when repo or lore tool use is actually warranted instead of obeying a lexical front-door veto. -- There is now a 15-minute local scheduled moderation loop named `Void Moderator Rumination`. It runs `scripts/run-void-moderator-rumination.ps1`, launches `codex exec` from the VoidBot workspace with the usual MCP/tool surface, treats `config/discord-server-rules.md`, `config/moderation-review-agent.md`, and `styles/void-default.md` as doctrine/personality surfaces, uses `node scripts/export-recent-discord-history.mjs` for chronological polling, and keeps its only routine writable memory in `.voidbot/private/moderation-agent-state.json`. +- There is now a 15-minute local scheduled moderation loop named `Void Moderator Rumination`. It runs `scripts/run-void-moderator-rumination.ps1`, launches `codex exec` from the VoidBot workspace with the usual MCP/tool surface, treats `config/discord-server-rules.md`, `config/moderation-review-agent.md`, and `styles/void-default.md` as doctrine/personality surfaces, uses `node scripts/export-recent-discord-history.mjs` for chronological polling, uses `node scripts/export-random-discord-history.mjs` for quiet-room novelty dives, and keeps its only routine writable memory in `.voidbot/private/moderation-agent-state.json`. - That moderation state file is now Ghostlight-shaped: identity, canonical state, goals, memories, perceived overlays, and a `moderation_runtime` block. The loop is also instructed to think like a constructive participant who embodies the rules, not just a clipboard looking for blood. +- The moderation loop now records `moderation_runtime.recent_archive_excursions` and is explicitly told to avoid laundering the same musing through fresh wording. Quiet runs should follow at least one fresh archive seam or concrete hook before they settle into theory. - Directly invoked Discord replies now load that same private moderation state as distilled private self-state context, so the ruminating scheduled loop and the summoned Void share one evolving personality/state spine. - The moderation loop now speaks through `node scripts/send-discord-message.mjs` using the local bot token instead of approval-gated side-effecting MCP tools, so unattended runs can DM or post without Codex pausing for permission every time it wants to open its mouth. - The new state/notes surfaces are now the continuity spine for future nontrivial work. diff --git a/notes/voidbot-current-system-map.md b/notes/voidbot-current-system-map.md index 8f118a0..3cccf5a 100644 --- a/notes/voidbot-current-system-map.md +++ b/notes/voidbot-current-system-map.md @@ -114,6 +114,8 @@ This note is the source-grounded description of how the live VoidBot stack is sh - scheduled moderation/participation runner that launches `codex exec` from the VoidBot workspace with the usual MCP/tool surface, writes status/log pulse files, and lets the unattended loop persist only `.voidbot/private/moderation-agent-state.json`. - `scripts/install-moderation-rumination-task.ps1` - installs the local 15-minute scheduled task that runs the moderation/participation loop through the hidden PowerShell launcher shim. +- `scripts/export-random-discord-history.mjs` + - plain Node helper for novelty excursions that samples one random archived Discord seam, returns a local context window around the anchor message, and skips bot-directed prompt rows by default. - `scripts/backup-voidbot-state.ps1` - local backup of Postgres, Qdrant snapshots, and RAG archives. - `scripts/restore-voidbot-state.ps1` @@ -155,6 +157,6 @@ Within the Postgres path, the implementation is split on purpose now too: 1. The Windows scheduled task `Void Moderator Rumination` starts `scripts/run-void-moderator-rumination.ps1` every 15 minutes. 2. The runner launches `codex exec` from `E:\Projects\VoidBot` with full local workspace/tool access so the child agent sees the same project rules, `.codex/config.toml`, and `voidbot` MCP server as a normal VoidBot workspace session. 3. The child agent reads `config/discord-server-rules.md`, `config/moderation-review-agent.md`, `styles/void-default.md`, and `.voidbot/private/moderation-agent-state.json`. -4. For chronological heartbeat duty, it polls `node scripts/export-recent-discord-history.mjs`; for deeper social or project context, it can use the usual retrieval tools (`search_history`, `get_message_context`, `list_indexed_repos`, `search_sources`, `get_source_context`). -5. If speaking would genuinely help, it posts through `node scripts/send-discord-message.mjs`; otherwise it updates only the ignored moderation state file with cursors, memories, musings, and candidate interventions. +4. For chronological heartbeat duty, it polls `node scripts/export-recent-discord-history.mjs`; when the room is quiet or a fresh hook deserves a wider branch, it also uses `node scripts/export-random-discord-history.mjs` to force at least one novelty excursion before settling into theory. For deeper social or project context, it can use the usual retrieval tools (`search_history`, `get_message_context`, `list_indexed_repos`, `search_sources`, `get_source_context`). +5. If speaking would genuinely help, it posts through `node scripts/send-discord-message.mjs`; otherwise it updates only the ignored moderation state file with cursors, memories, musings, recent archive excursions, and candidate interventions. 6. The runner writes pulse files under `.voidbot/status/moderation-rumination.json` plus `.voidbot/logs/moderation-rumination*`, so the loop can be observed without pretending an app heartbeat card is a real workstation daemon. diff --git a/package.json b/package.json index b40c5fb..a760f45 100644 --- a/package.json +++ b/package.json @@ -33,6 +33,7 @@ "qdrant:up": "powershell -ExecutionPolicy Bypass -File scripts/start-qdrant.ps1", "rag:backfill": "tsx scripts/backfill-discord-logs.ts", "moderation:recent-history": "node scripts/export-recent-discord-history.mjs", + "moderation:random-history": "node scripts/export-random-discord-history.mjs", "rag:scrub-bot-prompts": "tsx scripts/scrub-bot-prompts-from-history.ts", "rag:index-sources": "tsx scripts/index-source-repos.ts", "rag:install-push-hooks": "tsx scripts/install-source-index-post-push-hooks.ts", diff --git a/scripts/export-random-discord-history.mjs b/scripts/export-random-discord-history.mjs new file mode 100644 index 0000000..300512e --- /dev/null +++ b/scripts/export-random-discord-history.mjs @@ -0,0 +1,281 @@ +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), ".."); + +function main() { + const options = parseArgs(process.argv.slice(2)); + const archivePath = resolve( + repoRoot, + readConfiguredArchivePath() ?? ".voidbot/rag/messages.json", + ); + const store = readArchiveStore(archivePath); + const filtered = filterMessages(store.messages ?? [], options); + + if (filtered.length === 0) { + process.stdout.write( + `${JSON.stringify( + { + archivePath, + before: options.before ?? null, + after: options.after ?? null, + channelId: options.channelId ?? null, + includeBotPrompts: options.includeBotPrompts, + totalCandidateMessages: 0, + anchor: null, + messages: [], + transcript: "", + }, + null, + 2, + )}\n`, + ); + return; + } + + const rng = createRandom(options.seed); + const anchor = filtered[Math.floor(rng() * filtered.length)]; + const sameChannelMessages = filterMessages(store.messages ?? [], { + ...options, + channelId: anchor.channelId, + minContentLength: 0, + }); + const anchorIndex = sameChannelMessages.findIndex((message) => message.id === anchor.id); + const startIndex = Math.max(0, anchorIndex - options.window); + const endIndex = Math.min(sameChannelMessages.length, anchorIndex + options.window + 1); + const surrounding = sameChannelMessages.slice(startIndex, endIndex).map(toOutputMessage); + const transcript = surrounding.map(formatTranscriptLine).join("\n"); + + process.stdout.write( + `${JSON.stringify( + { + archivePath, + before: options.before ?? null, + after: options.after ?? null, + channelId: options.channelId ?? null, + includeBotPrompts: options.includeBotPrompts, + totalCandidateMessages: filtered.length, + window: options.window, + seed: options.seed ?? null, + anchor: toOutputMessage(anchor), + messages: surrounding, + transcript, + }, + null, + 2, + )}\n`, + ); +} + +function parseArgs(args) { + const options = { + window: 6, + minContentLength: 20, + includeBotPrompts: false, + }; + + for (let index = 0; index < args.length; index += 1) { + const argument = args[index]; + + switch (argument) { + case "--before": + options.before = args[index + 1]; + index += 1; + break; + case "--after": + options.after = args[index + 1]; + index += 1; + break; + case "--window": + options.window = Number.parseInt(args[index + 1] ?? "", 10); + index += 1; + break; + case "--channel-id": + options.channelId = args[index + 1]; + index += 1; + break; + case "--min-content-length": + options.minContentLength = Number.parseInt(args[index + 1] ?? "", 10); + index += 1; + break; + case "--seed": + options.seed = args[index + 1]; + index += 1; + break; + case "--include-bot-prompts": + options.includeBotPrompts = true; + break; + default: + throw new Error(`Unknown argument: ${argument}`); + } + } + + if (options.before) { + validateIsoTimestamp(options.before, "--before"); + } + + if (options.after) { + validateIsoTimestamp(options.after, "--after"); + } + + if (!Number.isInteger(options.window) || options.window < 0) { + throw new Error("--window must be a non-negative integer."); + } + + if (!Number.isInteger(options.minContentLength) || options.minContentLength < 0) { + throw new Error("--min-content-length must be a non-negative integer."); + } + + return options; +} + +function validateIsoTimestamp(value, flagName) { + if (Number.isNaN(Date.parse(value))) { + throw new Error(`${flagName} must be a valid ISO timestamp.`); + } +} + +function readConfiguredArchivePath() { + if (process.env.RAG_ARCHIVE_PATH?.trim()) { + return process.env.RAG_ARCHIVE_PATH.trim(); + } + + const envPath = resolve(repoRoot, ".env"); + + try { + const raw = stripLeadingBom(readFileSync(envPath, "utf8")); + const parsed = parseDotEnv(raw); + return parsed.RAG_ARCHIVE_PATH?.trim() || undefined; + } catch (error) { + if (error && typeof error === "object" && "code" in error && error.code === "ENOENT") { + return undefined; + } + + throw error; + } +} + +function parseDotEnv(raw) { + const result = {}; + + for (const line of raw.split(/\r?\n/)) { + const trimmed = line.trim(); + + if (trimmed.length === 0 || trimmed.startsWith("#")) { + continue; + } + + const separatorIndex = trimmed.indexOf("="); + + if (separatorIndex === -1) { + continue; + } + + const key = trimmed.slice(0, separatorIndex).trim(); + let value = trimmed.slice(separatorIndex + 1).trim(); + + if ( + value.length >= 2 && + ((value.startsWith("\"") && value.endsWith("\"")) || + (value.startsWith("'") && value.endsWith("'"))) + ) { + value = value.slice(1, -1); + } + + result[key] = value; + } + + return result; +} + +function readArchiveStore(archivePath) { + const raw = stripLeadingBom(readFileSync(archivePath, "utf8")); + const parsed = JSON.parse(raw); + + if (!parsed || typeof parsed !== "object" || !Array.isArray(parsed.messages)) { + throw new Error(`Archive at ${archivePath} does not have a messages array.`); + } + + return parsed; +} + +function filterMessages(messages, options) { + return messages + .filter((message) => !message.deletedAt) + .filter((message) => + options.includeBotPrompts ? true : readMessageKind(message) !== "bot_prompt", + ) + .filter((message) => (options.channelId ? message.channelId === options.channelId : true)) + .filter((message) => (options.before ? message.timestamp < options.before : true)) + .filter((message) => (options.after ? message.timestamp > options.after : true)) + .filter((message) => normalizeContent(message.content).length >= (options.minContentLength ?? 0)) + .sort((left, right) => left.timestamp.localeCompare(right.timestamp)); +} + +function normalizeContent(content) { + return typeof content === "string" ? content.trim() : ""; +} + +function readMessageKind(message) { + return message?.metadata?.messageKind === "bot_prompt" ? "bot_prompt" : "default"; +} + +function toOutputMessage(message) { + return { + id: message.id, + timestamp: message.timestamp, + channelId: message.channelId, + threadId: message.threadId, + authorId: message.authorId, + authorName: message.authorName, + content: message.content, + attachments: Array.isArray(message.attachments) ? message.attachments : [], + messageKind: readMessageKind(message), + }; +} + +function formatTranscriptLine(message) { + const threadSuffix = message.threadId ? ` thread=${message.threadId}` : ""; + const attachmentSuffix = + message.attachments.length > 0 + ? ` [attachments: ${message.attachments.join(", ")}]` + : ""; + return `[${message.timestamp}] ${message.authorName} (${message.authorId}) channel=${message.channelId}${threadSuffix}: ${message.content}${attachmentSuffix}`; +} + +function createRandom(seed) { + if (!seed) { + return Math.random; + } + + let value = hashSeed(seed); + return () => { + value += 0x6d2b79f5; + let intermediate = Math.imul(value ^ (value >>> 15), 1 | value); + intermediate ^= intermediate + Math.imul(intermediate ^ (intermediate >>> 7), 61 | intermediate); + return ((intermediate ^ (intermediate >>> 14)) >>> 0) / 4294967296; + }; +} + +function hashSeed(seed) { + let hash = 2166136261; + + for (const character of seed) { + hash ^= character.charCodeAt(0); + hash = Math.imul(hash, 16777619); + } + + return hash >>> 0; +} + +function stripLeadingBom(input) { + return input.charCodeAt(0) === 0xfeff ? input.slice(1) : input; +} + +try { + main(); +} catch (error) { + console.error(error instanceof Error ? error.message : String(error)); + process.exitCode = 1; +} diff --git a/scripts/run-void-moderator-rumination.ps1 b/scripts/run-void-moderator-rumination.ps1 index cef1b30..0f0410c 100644 --- a/scripts/run-void-moderator-rumination.ps1 +++ b/scripts/run-void-moderator-rumination.ps1 @@ -248,6 +248,7 @@ Required operating posture: - You have VoidBot's usual workspace tools and the local voidbot MCP server available in this repo. - Use `node scripts/export-recent-discord-history.mjs --after --limit 120` for chronological polling. - If there is no saved cursor yet, use `node scripts/export-recent-discord-history.mjs --hours 6 --limit 120`. +- Use `node scripts/export-random-discord-history.mjs --before --window 6 --min-content-length 24` for novelty excursions when the room is quiet or when a fresh hook deserves an adjacent archive dive. - When the room is quiet, use the usual VoidBot retrieval tools to think about GameCult projects and contribute ideas: - `search_history` - `get_message_context` @@ -255,6 +256,7 @@ Required operating posture: - `search_sources` - `get_source_context` - `notify_owner` when owner interruption is actually warranted +- Do not keep circling the same musing just because it still feels tidy. Chase at least one fresh branch when the room gives you a concrete hook, and record archive excursions so you can avoid pacing the same trench forever. - Use the local bot-voice script `node scripts/send-discord-message.mjs ...` when you genuinely need to speak. - Update only `.voidbot/private/moderation-agent-state.json` as routine writable memory. - Do not edit tracked repo files during routine runs. diff --git a/state/evidence.jsonl b/state/evidence.jsonl index 80e3041..cef928c 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -41,3 +41,4 @@ {"ts":"2026-05-04T20:21:40Z","type":"implementation","status":"ok","note":"Sandboxed cron runs should not depend on tsx/esbuild just to read local archive state; the moderation recent-history helper now runs as plain Node so the automation path stops dying on child-process EPERM before it even sees the room."} {"ts":"2026-05-04T21:30:08Z","type":"implementation","status":"ok","note":"The moderation loop now speaks through a local bot-token script for owner DMs and in-channel posts instead of approval-gated side-effecting MCP tools. Unattended runs can talk without Codex stopping for permission theater, and the script chunks messages and supports owner DM or channel reply targets."} {"ts":"2026-05-04T23:41:29Z","type":"architecture","status":"ok","note":"The app heartbeat card was decorative; a real unattended moderation loop needs a local scheduled `codex exec` runner in the VoidBot workspace, with normal tool/MCP access and dangerous-bypass mode, or it will sit there narrating state updates it never actually persisted."} +{"ts":"2026-05-05T00:03:35Z","type":"architecture","status":"ok","note":"The moderation rumination loop needs explicit novelty pressure: quiet runs now have a random archive helper plus recent excursion breadcrumbs so the agent stops laundering the same musing through fresh wording and instead follows unexplored archive seams or concrete live hooks."} diff --git a/state/map.yaml b/state/map.yaml index ac62174..7d135e3 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -68,6 +68,7 @@ accepted_design: - Moderation review should run as a separate local Codex scheduled loop with tracked rules prompt surfaces plus one ignored structured state file, escalating via `notify_owner` instead of pretending the live Discord bot is a silent auto-punishment machine. - The moderation agent should think of itself as a constructive participant as well as a reviewer, and its ignored structured state should follow a Ghostlight-shaped identity/canonical-state/goals/memories/overlays/runtime split rather than a flat notes bucket. - Unattended moderation speech should use a local bot-token script instead of approval-gated side-effecting MCP tools, otherwise the loop keeps stopping to ask permission before it can talk. + - Quiet moderation runs should prefer at least one random archive excursion plus concrete follow-on retrieval over simply rephrasing the last favorite theory; the ignored state needs recent excursion breadcrumbs so the loop can avoid pacing the same trench. - Deep agent-state, personality, culture, and persistent-world design work now belongs to the sibling Ghostlight repo; VoidBot should consume that work instead of remaining its long-term home. - Long-running work should expose durable logs or status files instead of relying on one attached session and hope. - Persistent project memory must stay ruthlessly distilled; git history and artifacts carry routine proof. @@ -91,7 +92,7 @@ landed_baseline: - State storage is now split into a thin factory plus per-domain Postgres/file store modules and bootstrap/migration helpers instead of one persistence omnibus. - Source archive storage is now sharded by repo behind a manifest plus per-repo shard directory instead of one giant `source-documents.json` blob. - Source repo indexing now reconciles automatically during stack startup and watchdog runs, refreshing hooks, indexing newly discovered repos, and pruning stale repo shards when repos disappear from `SOURCE_REPO_ROOT`. - - A separate scheduled moderation-rumination loop now launches `codex exec` from the VoidBot workspace every 15 minutes, reads tracked server rules, polls recent chronology through the plain Node helper, uses the usual VoidBot MCP/tool surface for archive and project rumination, and maintains one ignored JSON state file under `.voidbot/private/` instead of smearing moderation memory across random notes. + - A separate scheduled moderation-rumination loop now launches `codex exec` from the VoidBot workspace every 15 minutes, reads tracked server rules, polls recent chronology through the plain Node helper, uses the usual VoidBot MCP/tool surface for archive and project rumination, performs explicit random archive excursions when the room is quiet, and maintains one ignored JSON state file under `.voidbot/private/` instead of smearing moderation memory across random notes. rejected_paths: - name: transcript_only_continuity