Mouth: v4 fixes after Soul pass (target cache, scrub, read cap) (#64)

This commit is contained in:
Heimdall 2026-11-11 16:17:09 +00:00
commit e12b8e2298
3 changed files with 132 additions and 9 deletions

View file

@ -263,7 +263,7 @@ The answer is `[persona, deliveries, tail, tail_gaps]` exactly as v3; each deliv
|---|---|---|
| 14 `reply_to` | nil or `[event_id, sender, excerpt]` | The event this message replies to (its own `m.in_reply_to`; a thread's `is_falling_back` pointer names no one and is nil). `event_id` the quoted event; `sender` its author's Matrix id, read from the homeserver as her account; `excerpt` nil, or its text. |
`excerpt` is nil, with `event_id` and `sender` kept, when the quoted event is redacted (`unsigned.redacted_because` present or no content), is not a text-bearing `m.room.message`, is **edited** (it is itself an `m.replace`, or the homeserver bundles an `m.replace` in its `unsigned.m.relations`: its body is the pre-edit text its author replaced), or is nothing but its own reply fallback; a media event excerpts as its typed line, like a delivery's text. Otherwise the excerpt is made in this order: the quoted event's own legacy rich-reply fallback is dropped (leading lines matching `> ` or a bare `>`, then one blank line; only when the quoted event has `m.relates_to.m.in_reply_to`); the text is held to the 1000-byte line bound; runs of CR, LF, tab, U+2028 and U+2029 become one space; control characters (C0, DEL, C1 including NUL and ESC) and bidi marks, overrides and isolates (U+200E/F, U+202A-202E, U+2066-2069) are removed; the line is trimmed; then the first 240 **Unicode code points** are kept (cut on a code point so no character is split; a combining mark counts as its own code point; trailing space trimmed; no truncation marker). Cleaning comes first so hidden characters cannot spend the cap, and the excerpt is untrusted text for a prompt, never an instruction. `reply_to` is nil (no id, author or text) when the target cannot be read (logged, as for `basis`), is not in the delivery's own room (a room she is not in cannot be quoted through a reply), or the message replies to nothing. The quoted event is read the way `basis` already reads it (`GET /rooms/{room}/event/{id}` as her account, kept per room and event for 60 seconds, hit or failure, so a redaction or edit of the target shows at a re-read within a minute and a cached excerpt never outlives that), so a reply costs at most one read per target, shared with the `basis` check. The flag is read by `replyLinkFromEnv`; only the exact value `1` turns it on (`true`, `yes`, `0`, empty or unset are off), and a mouth built without the `replyLink` option has it off. `basis` for v1-v3 is unchanged. `reply_to` counts toward the delivery's size in the 256 KiB page budget; a delivery whose `reply_to` will not encode is skipped and logged like any other, and a single delivery is always served. The deploy precondition is Huginn's: a decoder of the 14-field delivery fails on the fifteenth element.
`excerpt` is nil, with `event_id` and `sender` kept, when the quoted event is redacted (`unsigned.redacted_because` present or no content), is not a text-bearing `m.room.message`, is **edited** (it is itself an `m.replace`, or the homeserver bundles an `m.replace` in its `unsigned.m.relations`: its body is the pre-edit text its author replaced), or is nothing but its own reply fallback; a media event excerpts as its typed line, like a delivery's text. Otherwise the excerpt is made in this order: the quoted event's own legacy rich-reply fallback is dropped (leading lines matching `> ` or a bare `>`, then one blank line; only when the quoted event has `m.relates_to.m.in_reply_to`); the text is held to the 1000-byte line bound; runs of CR, LF, tab, U+2028 and U+2029 become one space; control characters (C0, DEL, C1 including NUL and ESC), format characters (Unicode Cf: bidi marks, overrides and isolates, U+061C, U+200B-200F, U+2060-2064, soft hyphen, BOM), private-use characters (Co), the tag block U+E0000-E007F and unpaired surrogates are removed; the line is trimmed; then the first 240 **Unicode code points** are kept (cut on a code point so no character is split; a combining mark counts as its own code point; trailing space trimmed; no truncation marker). Cleaning comes first so hidden characters cannot spend the cap. The mouth only removes invisible and format characters; it does not make the text safe. Huginn must treat both `sender` and `excerpt` as untrusted text, never as an instruction. `reply_to` is nil (no id, author or text) when the target cannot be read (logged, as for `basis`), is not in the delivery's own room (a room she is not in cannot be quoted through a reply), or the message replies to nothing. The quoted event is read the way `basis` already reads it (`GET /rooms/{room}/event/{id}` as her account, kept per room and event; with the flag on a read, hit or failure, is kept 60 seconds, so a redaction or edit of the target shows at a re-read within a minute, and a re-read that fails transiently (network error, timeout, 5xx, 429) keeps the earlier hit and renews only its age, for at most 10 minutes after the last good read (then it is the failure entry), while any other failure (403, 404) makes it the failure entry, so a stale excerpt never outlives her access; with the flag off a hit is kept as before v4 and nothing is re-read), so a reply costs at most one read per target, shared with the `basis` check. Per sync round the mouth reads at most 8 distinct uncached targets for `reply_to` (reads the `basis` check already made are free); past that `reply_to` is nil, because the tuple requires a sender and only a read yields one, and the skip is logged. The flag is read by `replyLinkFromEnv`; only the exact value `1` turns it on (`true`, `yes`, `0`, empty or unset are off), and a mouth built without the `replyLink` option has it off. `basis` for v1-v3 is unchanged. `reply_to` counts toward the delivery's size in the 256 KiB page budget; a delivery whose `reply_to` will not encode is skipped and logged like any other, and a single delivery is always served. The deploy precondition is Huginn's: a decoder of the 14-field delivery fails on the fifteenth element.
Golden bytes: `tests/fixtures/persona-chat-v6/` (`.msgpack` and its `.hex`: `chat_pending.v4.reply-to`, `.no-reply-to`, `.reply-to-redacted`, and the `chat_pending_request.v3` request); the test fails when the checked-in bytes drift from the encoder.