Merge pull request 'File zone: Mind contract, plaintext sha256 on send, read-by receipts' (#104) from void/file-zone-contract into dev

docs/file-zone-mind-contract.md

@@ -0,0 +1,85 @@
+# File zone: what a Mind can read from a file event
+
+Ruling `ratatoskr:ruling:persona-file-sharing-zone`. The Mind-side fetch is the mouth's
+(`eureka-body:cut_spec:mouth-fetch-attachment.r1`); this page is the client side of the same contract.
+A Mind cannot fetch an `mxc://` itself (the media endpoint answers 401), so it never holds one.
+
+## What the Mind passes
+
+Room id and event id of the file event. Never a bare `mxc://`, URL or hash. The mouth fetches the event,
+reads the media reference from its content, and enforces the Mind's own membership in that room at fetch
+time. A room the Mind is not in is refused by the mouth, whatever the client wrote.
+
+## What the client writes (this repo, `RoomInput.tsx` send path)
+
+A file event is an `m.room.message` whose `msgtype` is `m.audio`, `m.image`, `m.video` or `m.file`:
+
+| field              | where   | note                                                                                                                                                                                       |
+| ------------------ | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `msgtype`          | content | selects the kind from the file's mimetype prefix                                                                                                                                           |
+| `body`, `filename` | content | both the file's name (display only; never a path)                                                                                                                                          |
+| `url`              | content | the `mxc://` of an unencrypted file; the mouth reads it                                                                                                                                    |
+| `file`             | content | present instead of `url` in an encrypted room: `{url, key, iv, hashes.sha256 (ciphertext), v}`. A Mind has no key; the mouth refuses it                                                    |
+| `info.mimetype`    | content | the browser-reported type, normalised by `safeFile` (e.g. `application/ogg` becomes `audio/ogg`). A claim, not proof: the mouth sniffs magic bytes                                         |
+| `info.size`        | content | bytes of the file as uploaded                                                                                                                                                              |
+| `info.sha256`      | content | lower-case hex sha256 of the plaintext bytes, computed by the client before send; written in unencrypted rooms only (a plaintext hash in an encrypted room would leak the file's identity) |
+
+The hash is computed over the same bytes that are uploaded, reading the whole file into memory before send (large videos pay for it); if hashing fails the file is sent without one. A file sent by something other than this
+client may lack `info.sha256`; the mouth then logs the hash it computed, and the row can show no `read by`
+for it.
+
+`info.mimetype` and `info.size` are written on audio and file events always; on image and video events `info` exists only if the browser could load the media, otherwise `info` holds `sha256` alone. Image and video events also carry `info.w`, `info.h`, `info.xyz.amorgan.blurhash` and a thumbnail
+reference; audio carries no duration. A Mind should not rely on any of those.
+
+## Limits and refusals
+
+- Size: the mouth streams at most 5 MiB and refuses a larger file. The client does not enforce this cap;
+  its own limit is the homeserver's `m.upload.size`. A file over 5 MiB can be shared with people and
+  not read by a Mind.
+- Refusals are the mouth's and are typed: not a member of the room, bridged or Discord room, encrypted
+  file, over the cap, content type not matching the magic bytes. The client does not re-check them and
+  nothing in the client depends on the mouth's log.
+- The Mind cites files by room id and event id only; a file in a room the Mind is not in does not exist
+  for it.
+
+## Receipt: `org.gamecult.file_received`
+
+The Mind posts one event of type `org.gamecult.file_received` after a successful fetch, to the room of the
+file, as a reply in the album's thread so it arrives with the album's own listing (no extra request):
+
+```json
+{
+  "m.relates_to": { "rel_type": "m.thread", "event_id": "<album root event id>" },
+  "org.gamecult.file_received": {
+    "event_id": "<file event id>",
+    "sha256": "<hex sha256 of the bytes the Mind fetched>",
+    "size": 123456
+  }
+}
+```
+
+Two rules, both enforced on each side:
+
+1. **The sender is the Mind.** The row trusts the Matrix `sender` of the receipt event and nothing inside it:
+   no Mind field is read. `read by X` counts only when the sender is X's own Matrix identity (the
+   full id is `@<mind>:<this homeserver>` for a Mind in the Persona catalogue; a display name, a lookalike or another server's `@void:evil.example` does not count).
+2. **Read means verified.** The Mind posts only after the sha256 of the bytes it fetched equals the file
+   event's `info.sha256`, and the receipt carries that hash. On a mismatch it posts nothing (or a typed
+   refusal that the row does not draw). The row checks it again: the receipt's `sha256` must equal the
+   file event's `info.sha256`. A file event with no `info.sha256` can show no `read by`.
+
+The receipt is inert: no paths, URLs, filenames or anything a browser would fetch, and the client reads only
+`event_id`, `sha256` and the sender. It has no `msgtype`, so the album's file listing ignores it. The mouth's
+log is the fuller record for Minds; this event is the public tick.
+
+The row draws `read by <Mind>`: one receipt per Mind per file (the latest), several Minds side by side. A
+receipt for an event that is not a file of that album, a malformed one, and a redacted one are ignored.
+The sender check keeps a member from ticking for a Mind; it does not prove the Mind read the file beyond
+what the hash and the Mind's own account show.
+
+## Not built yet
+
+- Files sent from the composer carry no thread relation, so they reach an album only if written into a
+  thread by another tool; the album upload path (docs/file-zone.md, "Upload") is unwritten. Receipts are
+  thread replies, so they show only where the file is.
+- Encrypted zones are not listed.

src/app/features/albums/AlbumView.tsx

@@ -15,6 +15,7 @@ import { getBlobSafeMimeType } from '../../utils/mimeTypes';
 import { timeDayMonYear } from '../../utils/time';
 import { getMemberDisplayName } from '../../utils/room';
 import { Album, AlbumFile } from './buildAlbums';
+import { readLine } from './fileReceipt';
 import { useAlbums } from './useAlbums';
 import * as css from './AlbumView.css';
 
@@ -47,6 +48,11 @@ function FileRow({ room, file, author }: FileRowProps) {
       <Text size="T300" truncate>
         {file.name}
       </Text>
+      {file.readBy.length > 0 && (
+        <Text size="T200" className={css.Dim}>
+          {readLine(file.readBy, (id) => getMemberDisplayName(room, id) ?? id)}
+        </Text>
+      )}
       {audio ? (
         <AudioContent {...audio} renderMediaControl={(p) => <MediaControl {...p} />} />
       ) : (

src/app/features/albums/buildAlbums.test.ts

@@ -214,3 +214,87 @@ describe('buildAlbums', () => {
     expect(one(root('$q', 1), [threaded]).count).toBe(1);
   });
 });
+
+describe('Mind receipts', () => {
+  const HASH = 'a'.repeat(64);
+  const file = (id: string, sha?: string) =>
+    reply(id, 10, 'm.audio', id, { info: sha ? { sha256: sha } : {} });
+  const receipt = (
+    id: string,
+    ts: number,
+    sender: string,
+    fileId: unknown,
+    sha?: unknown,
+    extra: Record<string, unknown> = {}
+  ) =>
+    ({
+      event_id: id,
+      type: 'org.gamecult.file_received',
+      sender,
+      origin_server_ts: ts,
+      content: { 'org.gamecult.file_received': { event_id: fileId, sha256: sha, ...extra } },
+    } as ZoneEvent);
+
+  const oneX = (r: ZoneEvent, replies: ZoneEvent[]) => buildAlbums([{ root: r, replies }], 'x')[0];
+
+  it('shows who read a file when the receipt carries the file event hash', () => {
+    const a = oneX(root('$r', 1), [file('$f', HASH), receipt('$x', 20, '@void:x', '$f', HASH)]);
+    expect(a.files[0].readBy).toEqual([{ sender: '@void:x', ts: 20 }]);
+    expect(a.count).toBe(1);
+  });
+  it('counts nothing for a hash that differs, a missing hash on either side, or a bad hash', () => {
+    const a = oneX(root('$r', 1), [
+      file('$f', HASH),
+      file('$g'),
+      receipt('$x', 20, '@void:x', '$f', 'b'.repeat(64)),
+      receipt('$y', 21, '@void:x', '$g', HASH),
+      receipt('$z', 22, '@void:x', '$f', undefined),
+      receipt('$w', 23, '@void:x', '$f', 7),
+    ]);
+    expect(a.files.every((f) => f.readBy.length === 0)).toBe(true);
+  });
+  it('trusts the event sender, never a Mind field inside the content', () => {
+    const a = oneX(root('$r', 1), [
+      file('$f', HASH),
+      receipt('$x', 20, '@mallory:x', '$f', HASH, { mind: 'void', sender: '@void:x' }),
+    ]);
+    expect(a.files[0].readBy).toEqual([]);
+  });
+  it('counts only a Mind of this homeserver: not another server, a lookalike, or a bare name', () => {
+    const a = oneX(root('$r', 1), [
+      file('$f', HASH),
+      receipt('$x', 20, '@void-fan:x', '$f', HASH),
+      receipt('$y', 21, '@x:void', '$f', HASH),
+      receipt('$z', 22, '@aqua:x', '$f', HASH),
+      receipt('$e', 23, '@void:evil.example', '$f', HASH),
+      receipt('$b', 24, 'void:x', '$f', HASH),
+    ]);
+    expect(a.files[0].readBy.map((r) => r.sender)).toEqual(['@aqua:x']);
+  });
+  it('keeps one receipt per Mind, the latest, and a second Mind beside it', () => {
+    const a = oneX(root('$r', 1), [
+      file('$f', HASH),
+      receipt('$1', 20, '@void:x', '$f', HASH),
+      receipt('$2', 30, '@void:x', '$f', HASH),
+      receipt('$3', 25, '@eureka:x', '$f', HASH),
+    ]);
+    expect(a.files[0].readBy).toEqual([
+      { sender: '@eureka:x', ts: 25 },
+      { sender: '@void:x', ts: 30 },
+    ]);
+  });
+  it('ignores a receipt for a file not in the album, a malformed one, and a redacted one', () => {
+    const redacted = {
+      ...receipt('$d', 20, '@void:x', '$f', HASH),
+      unsigned: { redacted_because: {} },
+    };
+    const a = oneX(root('$r', 1), [
+      file('$f', HASH),
+      receipt('$a', 20, '@void:x', '$elsewhere', HASH),
+      receipt('$b', 21, '@void:x', 42, HASH),
+      redacted,
+    ]);
+    expect(a.files[0].readBy).toEqual([]);
+    expect(a.count).toBe(1);
+  });
+});

src/app/features/albums/buildAlbums.ts

@@ -1,6 +1,8 @@
 // Pure: turns the thread roots of a file-zone room and each root's thread replies, in the JSON
 // Synapse returns from GET /rooms/{roomId}/threads and GET /rooms/{roomId}/relations/{root}/m.thread,
 // into albums. See docs/file-zone.md for the rulings this encodes.
+import { MINDS } from '../../components/message/pose/poses';
+import { FILE_HASH_KEY, FILE_RECEIVED } from './fileReceipt';
 
 /** The slice of a Matrix event this module reads. Every field may be missing. */
 export type ZoneEvent = {
@@ -16,6 +18,9 @@ export type AlbumInput = { root: ZoneEvent; replies: ZoneEvent[] };
 
 export type FileKind = 'audio' | 'image' | 'video' | 'file';
 
+/** A Mind's verified receipt for one file: its sender, who the Matrix server says posted it. */
+export type FileReader = { sender: string; ts: number };
+
 export type AlbumFile = {
   id: string;
   kind: FileKind;
@@ -23,6 +28,8 @@ export type AlbumFile = {
   ts: number;
   /** The event content, as RenderMessageContent reads it. */
   content: Record<string, unknown>;
+  /** Receipts in this album that name this file, one per sender (the latest), oldest first. */
+  readBy: FileReader[];
 };
 
 export type Album = {
@@ -70,9 +77,48 @@ const toFile = (e: ZoneEvent): AlbumFile | undefined => {
     name: asString(e.content.filename) || asString(e.content.body),
     ts: tsOf(e.origin_server_ts),
     content: e.content,
+    readBy: [],
   };
 };
 
+// A receipt is a thread reply of type org.gamecult.file_received whose content names a file of this
+// album; it carries no msgtype, so it never passes `live`.
+const receiptOf = (e: ZoneEvent) => {
+  if (!e || typeof e !== 'object' || e.type !== FILE_RECEIVED || !e.event_id) return undefined;
+  if (e.unsigned?.redacted_because) return undefined;
+  const body = e.content?.[FILE_RECEIVED] as { event_id?: unknown; sha256?: unknown } | undefined;
+  if (typeof body?.event_id !== 'string' || typeof body.sha256 !== 'string') return undefined;
+  if (typeof e.sender !== 'string') return undefined;
+  return {
+    file: body.event_id,
+    sender: e.sender,
+    ts: tsOf(e.origin_server_ts),
+    sha256: body.sha256,
+  };
+};
+
+// Only the event's sender counts, never a name inside the content, and only a Mind's own Matrix
+// identity on this homeserver (`@<mind>:<server>`, the mind named in the Persona catalogue). "Read" means verified: the receipt's hash must be
+// the file event's own.
+const isMind = (sender: string, server: string) => MINDS.some((m) => sender === `@${m}:${server}`);
+
+const attachReceipts = (files: AlbumFile[], replies: ZoneEvent[], server: string) => {
+  const byId = new Map(files.map((f) => [f.id, f]));
+  replies
+    .map(receiptOf)
+    .filter((r): r is NonNullable<typeof r> => !!r && isMind(r.sender, server))
+    .sort((a, b) => a.ts - b.ts)
+    .forEach((r) => {
+      const file = byId.get(r.file);
+      const sent = (file?.content.info as Record<string, unknown> | undefined)?.[FILE_HASH_KEY];
+      if (!file || typeof sent !== 'string' || sent !== r.sha256) return;
+      file.readBy = [
+        ...file.readBy.filter((x) => x.sender !== r.sender),
+        { sender: r.sender, ts: r.ts },
+      ];
+    });
+};
+
 export const UNTITLED = 'Untitled album';
 
 // An edited root carries its latest edit under unsigned["m.relations"]["m.replace"]; the title is
@@ -84,7 +130,7 @@ const titleOf = (root: ZoneEvent & { content: Record<string, unknown> }): string
   return asString(edited) || asString(root.content.body) || UNTITLED;
 };
 
-const toAlbum = (input: AlbumInput): Album | undefined => {
+const toAlbum = (input: AlbumInput, server: string): Album | undefined => {
   const { root, replies } = input ?? {};
   if (!live(root)) return undefined;
   const seen = new Set<string>();
@@ -96,6 +142,7 @@ const toAlbum = (input: AlbumInput): Album | undefined => {
       return true;
     })
     .sort((a, b) => a.ts - b.ts);
+  attachReceipts(files, Array.isArray(replies) ? replies : [], server);
   return {
     id: root.event_id,
     title: titleOf(root),
@@ -108,10 +155,11 @@ const toAlbum = (input: AlbumInput): Album | undefined => {
 };
 
 /** Newest album first; files oldest first. Loose room messages never reach this function. */
-export const buildAlbums = (inputs: AlbumInput[]): Album[] => {
+/** `server` is the homeserver's own domain: only its Minds' receipts count. */
+export const buildAlbums = (inputs: AlbumInput[], server = ''): Album[] => {
   const seen = new Set<string>();
   return inputs
-    .map(toAlbum)
+    .map((i) => toAlbum(i, server))
     .filter((a): a is Album => {
       if (!a || seen.has(a.id)) return false;
       seen.add(a.id);

src/app/features/albums/fileReceipt.test.ts

@@ -0,0 +1,37 @@
+import { describe, expect, it } from 'vitest';
+import { hashForSend, readLine, sha256Hex } from './fileReceipt';
+
+describe('sha256Hex', () => {
+  it('hashes the bytes: the FIPS 180 vector for "abc"', async () => {
+    expect(await sha256Hex(new Blob(['abc']))).toBe(
+      'ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad'
+    );
+  });
+  it('hashes empty input', async () => {
+    expect(await sha256Hex(new Blob([]))).toBe(
+      'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'
+    );
+  });
+});
+
+describe('hashForSend', () => {
+  it('gives no hash, and does not throw, when the blob cannot be read', async () => {
+    const broken = { arrayBuffer: () => Promise.reject(new Error('gone')) } as unknown as Blob;
+    expect(await hashForSend(broken)).toBeUndefined();
+  });
+});
+
+describe('readLine', () => {
+  it('names each reader', () => {
+    const name = (s: string) => s.slice(1, s.indexOf(':'));
+    expect(
+      readLine(
+        [
+          { sender: '@void:x', ts: 1 },
+          { sender: '@eureka:x', ts: 2 },
+        ],
+        name
+      )
+    ).toBe('read by void, eureka');
+  });
+});

src/app/features/albums/fileReceipt.ts

@@ -0,0 +1,27 @@
+// The file-zone Mind contract (docs/file-zone-mind-contract.md): the plaintext hash a sender writes
+// into a file event, and the receipt a Mind answers with.
+
+import type { FileReader } from './buildAlbums';
+
+/** Hex sha256 of the plaintext bytes, written at `content.info[FILE_HASH_KEY]` in unencrypted rooms. */
+export const FILE_HASH_KEY = 'sha256';
+
+/** A Mind's receipt: event type, and the content key holding { event_id, sha256, size }. Inert: nothing in it is fetched or trusted but the sender and the hash. */
+export const FILE_RECEIVED = 'org.gamecult.file_received';
+
+export const sha256Hex = async (data: Blob): Promise<string> => {
+  const digest = await crypto.subtle.digest('SHA-256', await data.arrayBuffer());
+  return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, '0')).join('');
+};
+
+/** The hash to write for a file about to be sent in an unencrypted room; none when it cannot be computed. */
+export const hashForSend = async (data: Blob): Promise<string | undefined> => {
+  try {
+    return await sha256Hex(data);
+  } catch {
+    return undefined;
+  }
+};
+
+export const readLine = (readBy: FileReader[], name: (sender: string) => string): string =>
+  `read by ${readBy.map((r) => name(r.sender)).join(', ')}`;

src/app/features/albums/useAlbums.ts

@@ -59,7 +59,7 @@ export const useAlbums = (room: Room): AlbumsState => {
             )
           ).chunk ?? [],
       }));
-      return buildAlbums(inputs);
+      return buildAlbums(inputs, mx.getDomain() ?? '');
     },
     retry: 1,
     refetchInterval: REFRESH_MS,

src/app/features/room/RoomInput.tsx

@@ -99,6 +99,7 @@ import {
   getImageMsgContent,
   getVideoMsgContent,
 } from './msgContent';
+import { FILE_HASH_KEY, hashForSend } from '../albums/fileReceipt';
 import { getMemberDisplayName, getMentionContent, trimReplyFromBody } from '../../utils/room';
 import { CommandAutocomplete } from './CommandAutocomplete';
 import { Command, SHRUG, TABLEFLIP, UNFLIP, useCommands } from '../../hooks/useCommands';
@@ -280,16 +281,23 @@ export const RoomInput = forwardRef<HTMLDivElement, RoomInputProps>(
         const fileItem = selectedFiles.find((f) => f.file === upload.file);
         if (!fileItem) throw new Error('Broken upload');
 
+        let content: IContent;
         if (fileItem.file.type.startsWith('image')) {
-          return getImageMsgContent(mx, fileItem, upload.mxc);
+          content = await getImageMsgContent(mx, fileItem, upload.mxc);
+        } else if (fileItem.file.type.startsWith('video')) {
+          content = await getVideoMsgContent(mx, fileItem, upload.mxc);
+        } else if (fileItem.file.type.startsWith('audio')) {
+          content = getAudioMsgContent(fileItem, upload.mxc);
+        } else {
+          content = getFileMsgContent(fileItem, upload.mxc);
         }
-        if (fileItem.file.type.startsWith('video')) {
-          return getVideoMsgContent(mx, fileItem, upload.mxc);
+        // A Mind verifies what it fetched against this. An encrypted file already carries the
+        // ciphertext hash in file.hashes; a plaintext hash would leak the file's identity.
+        if (!fileItem.encInfo) {
+          const sha256 = await hashForSend(fileItem.originalFile);
+          if (sha256) content.info = { ...content.info, [FILE_HASH_KEY]: sha256 };
         }
-        if (fileItem.file.type.startsWith('audio')) {
-          return getAudioMsgContent(fileItem, upload.mxc);
-        }
-        return getFileMsgContent(fileItem, upload.mxc);
+        return content;
       });
       handleCancelUpload(uploads);
       const contents = fulfilledPromiseSettledResult(await Promise.allSettled(contentsPromises));