mouth: deliver media events as a typed line

m.image/m.file/m.audio/m.video reached Personas as a bare filename and
dropped content.url. The delivery text now leads with
[kind filename mime size url], then the caption. Applies to the delivery
and to context lines. Wire v1 is unchanged. Nothing is fetched: a Persona
turn is a tool-less text call.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Heimdall 2026-11-03 18:09:43 +00:00
commit 6548ca8385
3 changed files with 90 additions and 4 deletions

View file

@ -196,7 +196,7 @@ The command is a dry-run shape probe for public surfaces that are real enough to
Bifrost's native Discord gateway is the conversation observer. A direct mention of the Bifrost bot, or a reply to one of its messages, becomes an immutable typed `bifrost.discord.persona_conversation_event.v0` only when the guild and channel have one exact Persona target binding. Bot-authored, ambient, empty, over-1200-character, unbound, and ambiguously bound messages do not enter the conversation path. The 1200-character cap is the shared signed alpha boundary with Epiphany. The observation store is Bifrost-private working state, never an actuator queue, and insert collisions must contain exactly equal content. VoidBot does not observe, schedule, or answer for the target Persona.
Persona speech goes out through Matrix, not Discord. `tools/persona-matrix-mouth.mjs` is the one process for every Persona (`PERSONAS`, a list of slugs): it holds each Persona's Matrix access token as its own systemd credential (`matrix-token-<persona>`) and nobody else holds any. Per Persona it keeps one `/sync` loop over the configured rooms and the account's direct rooms (a DM invitation from an account on `MATRIX_SERVER_NAME` is accepted while that Persona's brake is released, and one from any other server is declined), one cursor and one delivery sequence. A delivery carries the sender's Matrix id, the event's `m.mentions` user ids and whether the room is a DM. Reading wakes nothing and acks nothing: the heartbeat polls with the non-consuming `gamecult.persona.chat_pending_request.v1`, and only `chat_ack` advances a position. The mouth speaks on a `gamecult.persona.chat_send_request.v1` and joins rooms only while that Persona's brake (`/etc/gamecult/minds/<persona>/control.cc`) is released. One loopback CultNet RUDP endpoint serves all of them; a request naming an unlisted Persona gets a typed `chat_refusal` (`UnknownPersona`). Document shapes: `tools/persona-chat-documents.mjs`; golden bytes: `tests/fixtures/persona-chat-v1/`. The Discord posting actuator, its permit and its RUDP listener are removed.
Persona speech goes out through Matrix, not Discord. `tools/persona-matrix-mouth.mjs` is the one process for every Persona (`PERSONAS`, a list of slugs): it holds each Persona's Matrix access token as its own systemd credential (`matrix-token-<persona>`) and nobody else holds any. Per Persona it keeps one `/sync` loop over the configured rooms and the account's direct rooms (a DM invitation from an account on `MATRIX_SERVER_NAME` is accepted while that Persona's brake is released, and one from any other server is declined), one cursor and one delivery sequence. A delivery carries the sender's Matrix id, the event's `m.mentions` user ids and whether the room is a DM. A media event (`m.image`, `m.file`, `m.audio`, `m.video`) is delivered as a typed text line, `[image filename="cat.png" mime=image/png size=2048 url=mxc://server/id]`, followed by its caption when it has one, in `text` and in context; the wire stays v1. A Persona turn is a tool-less text call and cannot open a file or see an image, so the mouth fetches nothing from the media API. Reading wakes nothing and acks nothing: the heartbeat polls with the non-consuming `gamecult.persona.chat_pending_request.v1`, and only `chat_ack` advances a position. The mouth speaks on a `gamecult.persona.chat_send_request.v1` and joins rooms only while that Persona's brake (`/etc/gamecult/minds/<persona>/control.cc`) is released. One loopback CultNet RUDP endpoint serves all of them; a request naming an unlisted Persona gets a typed `chat_refusal` (`UnknownPersona`). Document shapes: `tools/persona-chat-documents.mjs`; golden bytes: `tests/fixtures/persona-chat-v1/`. The Discord posting actuator, its permit and its RUDP listener are removed.
Bifrost admits an observation only when guild, channel, target runtime, canonical repository, Persona, producer, and producer runtime match a preconfigured `bifrost.persona_feedback.target_binding.v0` document in Bifrost's private provider store. That binding also owns source visibility, data classification, and the delivery-store target. Bifrost never infers public visibility from Discord identifiers. It classifies the author using its own `bifrost.discord.actor_link.v0` state: a match yields `linked_governance_feedback`; absence yields `unlinked_social_feedback`. Neither class grants adoption, work, release, or deployment authority.

View file

@ -3,7 +3,7 @@ import assert from "node:assert/strict";
import { createServer } from "node:http";
import { readFileSync, readdirSync, statSync } from "node:fs";
import { resolve, join } from "node:path";
import { createMouth, createBrakeReader, guarded, redact, roomsFor, projectRoomsVariable } from "../tools/persona-matrix-mouth.mjs";
import { createMouth, bodyText, createBrakeReader, guarded, redact, roomsFor, projectRoomsVariable } from "../tools/persona-matrix-mouth.mjs";
const TOKEN = "syt_SECRET_TOKEN_9f3";
const HS_NAME = "gamecult.test";
@ -695,3 +695,61 @@ test("context_fetch_failure: empty context, one logged line without the token, a
assert.ok(!logged[0].includes(TOKEN));
} finally { await hs.close(); }
});
const media = (msgtype, content) => ({ msgtype, ...content });
const MXC_URL = "mxc://gamecult.test/AbC_123-x";
test("media_line: an image, file, audio or video is one typed line naming kind, filename, mime, size and mxc url", () => {
const info = { mimetype: "image/png", size: 2048, w: 10, h: 10 };
assert.equal(bodyText(media("m.image", { body: "cat.png", url: MXC_URL, info }), 4096), `[image filename="cat.png" mime=image/png size=2048 url=${MXC_URL}]`);
assert.equal(bodyText(media("m.file", { body: "notes.pdf", url: MXC_URL, info: { mimetype: "application/pdf", size: 0 } }), 4096), `[file filename="notes.pdf" mime=application/pdf size=0 url=${MXC_URL}]`);
assert.match(bodyText(media("m.audio", { body: "a.ogg", url: MXC_URL }), 4096), /^\[audio filename="a\.ogg" url=mxc:/);
assert.match(bodyText(media("m.video", { body: "v.mp4", url: MXC_URL }), 4096), /^\[video filename="v\.mp4" url=mxc:/);
});
test("media_line: a caption follows the typed line only when the event names its file apart from the body", () => {
const withCaption = bodyText(media("m.image", { body: "look at this", filename: "cat.png", url: MXC_URL }), 4096);
assert.equal(withCaption, `[image filename="cat.png" url=${MXC_URL}]\nlook at this`);
assert.equal(bodyText(media("m.image", { body: "cat.png", filename: "cat.png", url: MXC_URL }), 4096), `[image filename="cat.png" url=${MXC_URL}]`);
});
test("media_line: text, notice and emote bodies are untouched", () => {
for (const msgtype of ["m.text", "m.notice", "m.emote", undefined]) assert.equal(bodyText({ msgtype, body: "plain words" }, 4096), "plain words");
});
test("media_line: a hostile field is quoted or left out, never printed raw; an encrypted attachment names its file url and never its key", () => {
const line = bodyText(media("m.file", { body: 'a"]\n[image filename="x', url: "https://evil.example/x", info: { mimetype: "text/html\nx", size: -1 } }), 4096);
assert.equal(line, '[file filename="a\\"]\\n[image filename=\\"x"]');
assert.equal(line.includes("\n"), false);
const sealed = bodyText(media("m.image", { body: "s.png", file: { url: MXC_URL, key: { k: "SECRET_KEY_MATERIAL" }, iv: "IV", hashes: { sha256: "H" } }, info: { mimetype: "image/png" } }), 4096);
assert.equal(sealed, `[image filename="s.png" mime=image/png url=${MXC_URL} encrypted]`);
assert.equal(sealed.includes("SECRET_KEY_MATERIAL"), false);
});
test("media_line: the line is bounded to the byte cap on a character boundary", () => {
const long = bodyText(media("m.file", { body: "é".repeat(400), url: MXC_URL }), 100);
assert.ok(Buffer.byteLength(long) <= 100);
assert.equal(long.includes("<22>"), false);
});
test("media_delivery: a Matrix image reaches the delivery and a later delivery's context as the typed line, with no media fetch and no token in it", async () => {
const img = { type: "m.room.message", event_id: "$img", sender: `@alice:${HS_NAME}`, origin_server_ts: 1000, content: media("m.image", { body: "cat.png", url: MXC_URL, info: { mimetype: "image/png", size: 2048 } }) };
const after = message("$after", `@alice:${HS_NAME}`, "did you see it", 2000);
const tl = {
null: { next_batch: "b0", ...joined({ "!swarm:gamecult.test": { timeline: { events: [] } } }) },
b0: { next_batch: "b1", ...joined({ "!swarm:gamecult.test": { timeline: { events: [img] } } }) },
b1: { next_batch: "b2", ...joined({ "!swarm:gamecult.test": { timeline: { events: [after] } } }) },
};
const hs = await fakeHomeserver({ sync: since => tl[since] ?? { next_batch: since } });
try {
const mouth = mouthFor(hs);
await mouth.syncOnce();
const [d] = await mouth.syncOnce();
const line = `[image filename="cat.png" mime=image/png size=2048 url=${MXC_URL}]`;
assert.equal(d.text, line);
const [next] = await mouth.syncOnce();
assert.equal(next.context.at(-1).text, line);
assert.deepEqual(hs.seen.filter(r => r.path.includes("/media/") || r.path.includes("/download")), []);
assert.equal(JSON.stringify(mouth.pendingAfter(0)).includes(TOKEN), false);
} finally { await hs.close(); }
});

View file

@ -13,6 +13,8 @@
// reads the room's head from Matrix and answers RoomMoved, naming the messages that moved it, instead
// of the PUT. A v1 send, and a v2 send with no basis, are not checked; both still pass through the gate.
//
// Media: an m.image/m.file/m.audio/m.video event is delivered as one typed line (see bodyText), not its bare filename. Nothing is downloaded.
//
// Context: Matrix is the record. A delivery's `context` is read from the homeserver when the delivery
// is built (GET /messages, or the thread's relations for a threaded message), as this Persona's
// account, and holds m.room.message events from everyone, her own included, oldest first, ending
@ -51,6 +53,32 @@ const isLine = e => e?.type === "m.room.message" && typeof e.content?.body === "
// not an automated notice. Threaded replies count; the ear delivers them to her.
export const movesRoom = (event, selfId) => isLine(event) && event.sender !== selfId && event.content?.msgtype !== "m.notice";
// A media event (m.image, m.file, m.audio, m.video) reaches the Persona as one typed line, then its caption if it has one:
// [image filename="cat.png" mime=image/png size=2048 url=mxc://gamecult.test/abc]
// A Persona turn is a tool-less text call (it has no way to open a file or see an image), so the mouth names the
// attachment and fetches nothing. Every field is checked before it is printed; a field that fails is left out.
// An encrypted attachment names its `file.url` and carries `encrypted`; its key is never read.
const MEDIA_KIND = { "m.image": "image", "m.file": "file", "m.audio": "audio", "m.video": "video" };
const MXC = /^mxc:\/\/[A-Za-z0-9.:_-]+\/[A-Za-z0-9_-]+$/;
const MIME = /^[A-Za-z0-9][A-Za-z0-9!#$&^_.+-]{0,126}\/[A-Za-z0-9][A-Za-z0-9!#$&^_.+-]{0,126}$/;
const FILENAME_BYTES = 255;
export function bodyText(content, maxBytes) {
const kind = MEDIA_KIND[content?.msgtype];
if (!kind) return docs.clip(content.body, maxBytes);
const encrypted = typeof content.file?.url === "string" && typeof content.url !== "string";
const url = typeof content.url === "string" ? content.url : content.file?.url;
const named = typeof content.filename === "string" && content.filename !== "";
const filename = named ? content.filename : content.body;
const parts = [kind, `filename=${JSON.stringify(docs.clip(filename, FILENAME_BYTES))}`];
if (typeof content.info?.mimetype === "string" && MIME.test(content.info.mimetype)) parts.push(`mime=${content.info.mimetype}`);
if (Number.isSafeInteger(content.info?.size) && content.info.size >= 0) parts.push(`size=${content.info.size}`);
if (typeof url === "string" && MXC.test(url)) parts.push(`url=${url}`);
if (encrypted) parts.push("encrypted");
const line = docs.clip(`[${parts.join(" ")}]`, maxBytes);
const caption = named && content.body !== filename ? content.body : "";
return caption === "" ? line : docs.clip(`${line}\n${caption}`, maxBytes);
}
// The one gate every mouth shares: run(roomId, fn) runs fn after every earlier fn on that room, and a
// failure of one does not stop the next. It holds ordering only, nothing across a restart.
export function createRoomGate() {
@ -187,7 +215,7 @@ export function createMouth({
}
const threadRootOf = e => { const r = e?.content?.["m.relates_to"]; return r?.rel_type === "m.thread" && typeof r.event_id === "string" && r.event_id !== "" ? r.event_id : null; };
const lineOf = e => ({ sender: e.sender, text: docs.clip(e.content.body, docs.limits.contextBytes), atUnixMs: Number.isSafeInteger(e.origin_server_ts) && e.origin_server_ts >= 0 ? e.origin_server_ts : 0 });
const lineOf = e => ({ sender: e.sender, text: bodyText(e.content, docs.limits.contextBytes), atUnixMs: Number.isSafeInteger(e.origin_server_ts) && e.origin_server_ts >= 0 ? e.origin_server_ts : 0 });
// The room's recent messages outside threads, oldest first.
async function roomEvents(roomId) {
@ -255,7 +283,7 @@ export function createMouth({
if (typeof event.sender !== "string" || event.sender === "" || typeof event.event_id !== "string" || event.event_id === "") continue;
if (processed.has(event.event_id)) continue; // read already in a round the cap kept from advancing
const ts = event.origin_server_ts;
const entry = { sender: event.sender, text: docs.clip(event.content.body, docs.limits.textBytes), atUnixMs: Number.isSafeInteger(ts) && ts >= 0 ? ts : now() };
const entry = { sender: event.sender, text: bodyText(event.content, docs.limits.textBytes), atUnixMs: Number.isSafeInteger(ts) && ts >= 0 ? ts : now() };
const deliverable = !baseline && !freshRooms.has(roomId) && event.sender !== selfId;
if (deliverable && pending.length + made.length >= maxPending) { truncated = true; break rounds; }
processed.add(event.event_id);