Files desk cut 2: pagination in useAlbums, ShareSet folder test #148

Merged
void merged 2 commits from void/files-desk-cut2 into dev 2026-11-13 22:59:51 +00:00
6 changed files with 210 additions and 26 deletions

View file

@ -21,7 +21,7 @@ Read from this repo at dev `8ba407cd`:
- **Zone = a room** whose `m.room.create` type is `org.gamecult.file_zone` (`isFileZone.ts`), or one listed in config `fileZoneRooms`. One per Mind, `#<mind>-files:<server>`, found by alias or by the carried id in the root Space's `org.gamecult.stewardships` (`mediaRoom.ts`, `MediaRow.tsx`); rendered as the Media row of each Mind's group.
- **Album = a thread root** (a text event; its `body`, or its latest `m.replace` edit, is the title); **file = a thread reply** with `msgtype` `m.audio|m.image|m.video|m.file` (`buildAlbums.ts`). A redacted event or an `m.replace` is never a file. Loose room messages are not shown.
- **Read** = `GET /rooms/{id}/threads` and `/relations/{root}/m.thread`, one page of 100 each (`useAlbums.ts`; pagination is a known gap). Unencrypted rooms only.
- **Read** = `GET /rooms/{id}/threads` and `/relations/{root}/m.thread`, every page, 100 per request, at most 20 pages per endpoint (`useAlbums.ts`, `readPages.ts`; a capped read shows a note in the list). Unencrypted rooms only.
- **Write** = `sendShareSet` (`shareSet.ts`, behind `fileZoneShare.enabled`): a root, then one thread reply per file, content from `getMsgContent` with `info.sha256`.
- **Media** = authenticated Matrix media fetched to a blob (`downloadMedia` in `FileContent.tsx`): so the "media element cannot send a header" problem that Option B needed a ticket for is already solved this way, at the cost that a file is fully downloaded before it plays.
- **Receipts** = `org.gamecult.file_received` from a Mind's mouth after it fetches and verifies the sha256 (`docs/file-zone-mind-contract.md`); the mouth enforces the Mind's membership in the room.
@ -103,7 +103,7 @@ What A adds to `ShareSet`: sharing into an existing album ("add files", listed a
Under both, the phone design is the same shape (one column, rows 56 px or more, hidden controls, in-place preview; see the Option B sections below, which are written against the host but whose layout applies).
- **A, new cost:** the Commons room is one more room in sync and it must be muted; albums and files are one page of 100 each until pagination exists (a Commons of hundreds of documents needs it first); a folder tree is a title-path grouping, so a deep tree is a long list of albums with breadcrumbs, not true directories; a file plays after a full download (small audio and images are fine; a 40 MB clip is not).
- **A, new cost:** the Commons room is one more room in sync and it must be muted; albums and files are read to a cap of 20 pages of 100 per endpoint (a Commons past that shows a truncation note); a folder tree is a title-path grouping, so a deep tree is a long list of albums with breadcrumbs, not true directories; a file plays after a full download (small audio and images are fine; a 40 MB clip is not).
- **A, no cost:** no second sign-in, no ticket, no CORS, no new fetch layer, no host-unreachable state beyond the homeserver's own. The album list, album view, share panel and receipts exist and are tested; the view is their extension.
- **B, new cost:** every screen is a second client against a second service with its own errors, token refresh and unreachable state; media needs the ticket or the blob; the tree and preview are new UI.
- **B, benefit:** a tree that is live and complete, ranges, and no Matrix size limit.
@ -121,7 +121,7 @@ Under A the paused service is not built, and most of its list disappears:
Status 2026-11-13: cut 1 merged to dev as bb615570 (PR #147); cuts 2-5 follow. Cut 1's shape was: a pure `albumFolders` view (folders from `/`-separated album titles, leaf name, counts), folder navigation and breadcrumb in `AlbumView` through a presentational list component tested by static render, `ShareSet` sending into the current folder by prefixing the album title, and a quiet `Commons` row (alias `#commons:gamecult.org` or the carried `media.commons` id, through `lookupMediaRoom`, hidden while unset or looking, behind a dark `filesDesk.enabled` flag, and filtered out of the tiers so it is not listed twice). Muting is provisioning's (Q-E3), not the client's.
1. **Commons entry**: a Commons row in the navigator (like `MediaRow`, found by alias or carried id), opening the room's album view. Flagged, no new data layer.
2. **Pagination** of albums and files in `useAlbums` (a prerequisite for a Commons of any size).
2. **Pagination** of albums and files in `useAlbums` (done, PR #148).
3. **Folders by title path** and the album list grouped by it.
4. **Versions**: read `org.gamecult.publish.replaces`, show the latest of a chain with the earlier versions folded under it.
5. **Add files to an existing album**, with per-file progress, cancel and the refusal messages above (the upload flow).

View file

@ -89,7 +89,7 @@ function useMarkZoneRead(room: Room) {
type AlbumViewProps = { room: Room; onTimeline: () => void };
export function AlbumView({ room, onTimeline }: AlbumViewProps) {
useMarkZoneRead(room);
const { loading, error, albums } = useAlbums(room);
const { loading, error, albums, truncated } = useAlbums(room);
const [dateFormat] = useSetting(settingsAtom, 'dateFormatString');
const [openId, setOpenId] = useState<string>();
const [path, setPath] = useState<string[]>([]);
@ -166,6 +166,11 @@ export function AlbumView({ room, onTimeline }: AlbumViewProps) {
</button>
))}
{here.folders.length === 0 && here.albums.length === 0 && <Text>Nothing here.</Text>}
{truncated && (
<Text size="T200" className={css.Dim}>
The list is cut short: some albums or files are not loaded.
</Text>
)}
</>
);
}

View file

@ -0,0 +1,85 @@
import React from 'react';
import { renderToStaticMarkup } from 'react-dom/server';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { MatrixClientProvider } from '../../hooks/useMatrixClient';
// ShareSet hands its folder to the title the send lands under. The controller is held still and its
// `run` captured, and sendShareSet records the title it was given, so the path from the `folder`
// prop to the album root's title is read without a homeserver or a DOM.
const captured = vi.hoisted(() => ({
run: undefined as undefined | ((files: unknown[], title: string, onFile: () => void) => unknown),
titles: [] as string[],
}));
vi.mock('./shareController', () => ({
createShareController: (deps: { run: NonNullable<typeof captured.run> }) => {
captured.run = deps.run;
return {
subscribe: () => () => undefined,
getState: () => ({ files: [], title: '', phase: { kind: 'idle' }, epoch: 0 }),
pick: () => undefined,
setTitle: () => undefined,
send: () => undefined,
cancel: () => undefined,
};
},
}));
vi.mock('./shareSet', async (importOriginal) => ({
...(await importOriginal<typeof import('./shareSet')>()),
sendShareSet: async (_port: unknown, title: string) => {
captured.titles.push(title);
return { albumId: '$a', sent: 0, failed: [], failedAt: [] };
},
}));
vi.mock('react', async (importOriginal) => {
const actual = await importOriginal<typeof import('react')>();
// The server renderer has no snapshot for an external store; read the client one.
const useSyncExternalStore = (_sub: unknown, get: () => unknown) => get();
return { ...actual, default: { ...actual, useSyncExternalStore }, useSyncExternalStore };
});
// eslint-disable-next-line import/first
import { ShareSet } from './ShareSet';
const mount = (folder?: string[]) => {
const room = { roomId: '!r:x', hasEncryptionStateEvent: () => false } as never;
renderToStaticMarkup(
<QueryClientProvider client={new QueryClient()}>
<MatrixClientProvider value={{ sendMessage: async () => ({ event_id: '$e' }) } as never}>
<ShareSet room={room} folder={folder} />
</MatrixClientProvider>
</QueryClientProvider>
);
};
const send = async (typed: string) => {
await captured.run?.([], typed, () => undefined);
return captured.titles.at(-1);
};
describe('ShareSet: the folder reaches the album title', () => {
beforeEach(() => {
captured.run = undefined;
captured.titles.length = 0;
});
it('at the top, the typed title goes as typed', async () => {
mount();
expect(await send('Poses')).toBe('Poses');
});
it('inside a folder, the path prefixes the typed title', async () => {
mount(['cast-art', 'poses']);
expect(await send('Set 2')).toBe('cast-art/poses/Set 2');
});
it('an empty typed title falls back to the default, still under the folder', async () => {
mount(['handoff']);
expect(await send(' ')).toBe('handoff/Shared files');
});
it('a typed title containing a slash goes deeper', async () => {
mount(['a']);
expect(await send('b/c')).toBe('a/b/c');
});
});

View file

@ -0,0 +1,55 @@
import { describe, expect, it } from 'vitest';
import { Page, readPages } from './readPages';
const pager = (pages: Page<number>[]) => {
const froms: (string | undefined)[] = [];
return {
froms,
fetchPage: async (from: string | undefined) => {
froms.push(from);
return pages[froms.length - 1] ?? {};
},
};
};
describe('readPages', () => {
it('one page without a token is the whole read', async () => {
const p = pager([{ chunk: [1, 2] }]);
expect(await readPages(p.fetchPage)).toEqual({ items: [1, 2], truncated: false });
expect(p.froms).toEqual([undefined]);
});
it('follows next_batch in order, passing each token back', async () => {
const p = pager([
{ chunk: [1], next_batch: 'a' },
{ chunk: [2], next_batch: 'b' },
{ chunk: [3] },
]);
expect(await readPages(p.fetchPage)).toEqual({ items: [1, 2, 3], truncated: false });
expect(p.froms).toEqual([undefined, 'a', 'b']);
});
it('a missing chunk reads as empty', async () => {
expect(await readPages(async () => ({}))).toEqual({ items: [], truncated: false });
});
it('an empty page ends the read even when it carries a token', async () => {
const p = pager([
{ chunk: [1], next_batch: 'a' },
{ chunk: [], next_batch: 'b' },
]);
expect(await readPages(p.fetchPage)).toEqual({ items: [1], truncated: false });
expect(p.froms).toHaveLength(2);
});
it('the cap stops the read and says it was cut short', async () => {
const p = pager(Array.from({ length: 10 }, (_, i) => ({ chunk: [i], next_batch: `t${i}` })));
expect(await readPages(p.fetchPage, 3)).toEqual({ items: [0, 1, 2], truncated: true });
expect(p.froms).toHaveLength(3);
});
it('a last page exactly at the cap is not truncated', async () => {
const p = pager([{ chunk: [1], next_batch: 'a' }, { chunk: [2] }]);
expect((await readPages(p.fetchPage, 2)).truncated).toBe(false);
});
});

View file

@ -0,0 +1,27 @@
/** One page of a Matrix paginated endpoint: its events and the token for the next page, if any. */
export type Page<T> = { chunk?: T[]; next_batch?: string };
/** Pages read per endpoint at most, so a runaway zone cannot hold the phone in a loop. */
export const MAX_PAGES = 20;
/**
* Reads every page of an endpoint by following `next_batch`, up to `maxPages`. `truncated` is true
* when the cap stopped the read with more still to come, so a caller can say the list is partial.
*/
export const readPages = async <T>(
fetchPage: (from: string | undefined) => Promise<Page<T>>,
maxPages = MAX_PAGES
): Promise<{ items: T[]; truncated: boolean }> => {
const items: T[] = [];
let from: string | undefined;
for (let n = 0; n < maxPages; n += 1) {
// eslint-disable-next-line no-await-in-loop
const page = await fetchPage(from);
items.push(...(page.chunk ?? []));
// An empty page ends the read even if a token is returned: a server that keeps handing out
// tokens with nothing behind them would otherwise spend every page of the cap.
if (!page.next_batch || !page.chunk?.length) return { items, truncated: false };
from = page.next_batch;
}
return { items, truncated: true };
};

View file

@ -7,12 +7,11 @@ import { useClientConfig } from '../../hooks/useClientConfig';
import { Album, ZoneEvent, buildAlbums } from './buildAlbums';
import { isFileZone } from './isFileZone';
import { mapLimit } from './mapLimit';
import { Page, readPages } from './readPages';
const REPLY_FETCHES_IN_FLIGHT = 6;
const REFRESH_MS = 30_000;
type Chunk = { chunk?: ZoneEvent[] };
/** True when the room is the file zone: by its create type, or listed in the client config. */
export const useIsFileZone = (room: Room): boolean => {
const config = useClientConfig();
@ -29,37 +28,49 @@ export const useIsFileZone = (room: Room): boolean => {
);
};
export type AlbumsState = { loading: boolean; error: Error | null; albums: Album[] };
export type AlbumsState = {
loading: boolean;
error: Error | null;
albums: Album[];
/** A read hit the page cap (readPages), so older albums or files are not in the list. */
truncated: boolean;
};
const PAGE_LIMIT = '100';
/**
* Slice 1 reads ONE page of each endpoint (threads: 100 roots; replies per root: 100). A zone with
* more albums or files than that is truncated; pagination is a later slice. Reads are made with
* the logged-in client, so the room must be unencrypted: encrypted events carry no msgtype and
* are skipped.
* Reads every page of the zone's threads, then every page of each thread's replies, each capped at
* MAX_PAGES (readPages). Reads are made with the logged-in client, so the room must be
* unencrypted: encrypted events carry no msgtype and are skipped.
*/
export const useAlbums = (room: Room): AlbumsState => {
const mx = useMatrixClient();
const { roomId } = room;
const query = useQuery({
queryKey: ['albums', roomId],
queryFn: async (): Promise<Album[]> => {
queryFn: async (): Promise<{ albums: Album[]; truncated: boolean }> => {
const get = (path: string, params: Record<string, string>) =>
mx.http.authedRequest<Chunk>(Method.Get, path, params, undefined, {
mx.http.authedRequest<Page<ZoneEvent>>(Method.Get, path, params, undefined, {
prefix: ClientPrefix.V1,
});
const paged = (path: string, params: Record<string, string>) =>
readPages<ZoneEvent>((from) => get(path, from ? { ...params, from } : params));
const room_ = encodeURIComponent(roomId);
const roots = (await get(`/rooms/${room_}/threads`, { limit: '100' })).chunk ?? [];
const inputs = await mapLimit(roots, REPLY_FETCHES_IN_FLIGHT, async (root: ZoneEvent) => ({
root,
replies:
(
await get(
`/rooms/${room_}/relations/${encodeURIComponent(root.event_id ?? '')}/m.thread`,
{ limit: '100', dir: 'f' }
)
).chunk ?? [],
}));
return buildAlbums(inputs, mx.getDomain() ?? '');
const roots = await paged(`/rooms/${room_}/threads`, { limit: PAGE_LIMIT });
let { truncated } = roots;
const inputs = await mapLimit(
roots.items,
REPLY_FETCHES_IN_FLIGHT,
async (root: ZoneEvent) => {
const replies = await paged(
`/rooms/${room_}/relations/${encodeURIComponent(root.event_id ?? '')}/m.thread`,
{ limit: PAGE_LIMIT, dir: 'f' }
);
truncated = truncated || replies.truncated;
return { root, replies: replies.items };
}
);
return { albums: buildAlbums(inputs, mx.getDomain() ?? ''), truncated };
},
retry: 1,
refetchInterval: REFRESH_MS,
@ -68,6 +79,7 @@ export const useAlbums = (room: Room): AlbumsState => {
loading: query.isLoading,
// A failed refetch keeps the albums already loaded; the error shows only when there are none.
error: query.data ? null : (query.error as Error | null),
albums: query.data ?? [],
albums: query.data?.albums ?? [],
truncated: query.data?.truncated ?? false,
};
};