Merge hands/union-codec-ts: cultcache-ts slot record and union codecs, and one typed owner of document-payload reads and writes (DocumentPayload)

Soul passes cultlib-gaps:verdict:cut-union-codec-ts.s1-s9. Operator rulings ts-payload-opaque-type and ts-opaque-type-kept. Low pin gaps deferred to follow_up ts-codec-pin-gaps.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-10-11 22:40:37 +02:00
commit c770db6e77
59 changed files with 3049 additions and 573 deletions

View file

@ -103,6 +103,15 @@ jobs:
# Linux at all, and the browser client's never ran in CI on any OS.
# packages/cultnet-ts verifies both vector files; cultmesh-browser proves
# the browser client reads the shared rules rather than its own.
# No workflow ran cultcache-ts's tests, and its test script listed two files by hand while a third sat compiled
# and unrun. The wiring check fails when any TypeScript test file is run by no script, and the cache's own suite
# runs here beside the packages built on it.
- name: Check that every TypeScript test file is run by its package
run: node scripts/check-typescript-tests-wired.mjs
- name: Verify the CultCache document owner with the TypeScript cache
run: npm run test --workspace packages/cultcache-ts
- name: Verify the CultMesh authority rules with the TypeScript verifier
run: npm run test --workspace packages/cultnet-ts

3
package-lock.json generated
View file

@ -1202,7 +1202,7 @@
"version": "0.3.0",
"license": "MIT",
"dependencies": {
"@msgpack/msgpack": "^3.1.3",
"@gamecult/cultcache-ts": "^0.14.0",
"cultnet-ts": "^0.2.0"
},
"devDependencies": {
@ -1218,7 +1218,6 @@
"license": "MIT",
"dependencies": {
"@gamecult/cultcache-ts": "^0.14.0",
"@msgpack/msgpack": "^3.1.2",
"cultnet-ts": "^0.2.0",
"koffi": "3.3.0"
},

View file

@ -32,6 +32,37 @@ This is not an ORM in a fake mustache. It is for cases where you want:
That keeps the polymorphic boundary tight without making callers write boilerplate by candlelight.
## Reading And Writing Payloads
A document's bytes travel as a `DocumentPayload`: in a store envelope, a CultNet record and a
realtime frame. At run time it is the same `Uint8Array`, so nothing on the wire changes. At compile
time it is opaque: it is assignable to and from neither `Uint8Array` nor `ArrayLike<number>`, so no
MessagePack decoder accepts it.
What the type guarantees is that a payload cannot be read or assigned directly: a decoder call, an
assignment to `Uint8Array` and `new Uint8Array(payload)` fail to compile. It does not stop what the
runtime value allows, because the value is a `Uint8Array`. These routes read it without the owner:
`instanceof Uint8Array` or `ArrayBuffer.isView` narrowing, `Object.values`, `Object.assign` onto a
`Uint8Array`, a JSON round trip, `Reflect.get`, and a cast. A read made that way sees the raw bytes,
which for a codec or formatter document are not the document. No lint or guard closes them. Read
through `decodeDocumentPayload`, and when you need the bytes themselves (hashing, signing, a store
write) take them with `documentPayloadBytes`, the sanctioned escape.
- `decodeDocumentPayload(definition, payload)` reads a payload through the definition's formatter,
codec or plain map and then its schema. The definition is required.
- `encodeDocumentPayload(definition, value)` is the writer: the schema first, then the formatter,
codec or plain map.
- `documentPayloadFromBytes(bytes)` is where bytes enter (a record you validated, a store read, a
writer of pre-encoded or signed bytes) and `documentPayloadBytes(payload)` is where they leave (a
store write, a frame encode, hashing or signing). Reach for them only at those edges.
```ts
const note = decodeDocumentPayload(noteDocument, envelope.payload); // parsed Note
const payload = encodeDocumentPayload(noteDocument, note); // DocumentPayload
```
Browsers import the same names from `@gamecult/cultcache-ts/document`, which holds no Node built-ins.
## `.cc` Inspection
CultCache files may use the `.cc` extension. The bytes are still the canonical
@ -234,6 +265,54 @@ The important boundary is this:
So callers get typed values, and the persistence layer does not become an open polymorphic sewer.
## Slot Records and Unions
`codec` writes the reference runtimes' record shape: a MessagePack array indexed by slot key, nil in a gap, and a
union as `[armKey, [armSlots...]]`. A union value is `{ kind: armName, ...armFields }`.
```ts
import { codecs, defineDocumentType } from "@gamecult/cultcache-ts";
const { slots, union, list, lazy, string } = codecs;
const NS = "MyApp.Scene+"; // the C# names of the types, as the reference writes them into a store's schema catalog
type Node =
| { kind: "Box"; id: string; children: (Node | null)[] }
| { kind: "Text"; id: string; content: string }
| { kind: "Image"; id: string; uri: string; alt: string };
const node: codecs.Codec<Node | null> = union<Node>("Node", {
0: ["Box", slots<{ id: string; children: (Node | null)[] }>({ 0: ["id", string], 1: ["children", list(lazy(() => node))] })],
1: ["Text", slots<{ id: string; content: string }>({ 0: ["id", string], 1: ["content", string] })],
5: ["Image", slots<{ id: string; uri: string; alt: string }>({ 0: ["id", string], 1: ["uri", string], 3: ["alt", string] })], // slot 2 retired
}, `${NS}Node`);
const sceneDocument = defineDocumentType({
type: "scene",
schema: sceneSchema,
codec: slots<Scene>({ 0: ["id", string], 1: ["root", node], 2: ["title", string, ""] }),
name: "id",
});
```
- A slot is `[name, codec, default?]`. A gap is written nil and skipped on read; an extra trailing slot is skipped; a
slot missing from the payload (or nil, when a default is declared) takes its default, and is refused without one.
- A union refuses an undeclared arm key on read, naming the union, the key and the declared keys. Nothing falls back
to a default arm. An arm with no slots is `codecs.unit` and is written `[key, []]`.
- A refusal names the union, the field and the slot; it never echoes the value it refused.
- The leaves are `string`, `boolean`, `bytes`, `int32`, `int64`, `float32` and `float64`, each writing the MessagePack the
C# type writes and refusing what C# refuses (a float in an int slot, an `int` outside Int32). `int64` reads a number
when the value is a safe integer and a bigint beyond 2^53, and refuses a number beyond 2^53 on write. `list`, `array`
and `nullable` wrap a leaf, a union or a record. A `float32` slot rounds an integer once, as the reference's cast does.
- Every codec carries the reference's type name, and a store's schema catalog is derived from those names, because the
reference refuses a store whose slot names another type. The leaves know theirs; give a union or a record its C# name
as the last argument of `union(name, arms, typeName)` or `slots(spec, typeName)`. A codec with a slot it cannot name
refuses to derive members: pass `members` instead.
- An arm key is an integer within Int32 (negative too), in any MessagePack integer encoding. A float key (even `1.0`) is
refused on read, and a union refuses a key outside Int32 when it is defined. A slot key is within 0 to Int32 max.
- `codec` supplies the document's formatter and, unless `members` is given, its catalog members. A definition may not
carry both `codec` and `formatter`. `schema.parse` still runs on every decoded value.
## Raw Envelope Fast Path
For bit-compatible neighbors such as CultNet peers that already share the same

View file

@ -10,12 +10,24 @@
"require": "./dist/index.js",
"default": "./dist/index.js"
},
"./document": {
"types": "./dist/document.d.ts",
"require": "./dist/document.js",
"default": "./dist/document.js"
},
"./inspection": {
"types": "./dist/cult-cache-inspector.d.ts",
"require": "./dist/cult-cache-inspector.js",
"default": "./dist/cult-cache-inspector.js"
}
},
"typesVersions": {
"*": {
"document": [
"dist/document.d.ts"
]
}
},
"files": [
"dist"
],
@ -23,7 +35,7 @@
"build": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json",
"prepack": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json",
"typecheck": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json --pretty false",
"test": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json && node ../../node_modules/typescript/bin/tsc -p tsconfig.test.json && node --test dist-test/test/cult-cache.test.js"
"test": "node ../../node_modules/typescript/bin/tsc -p tsconfig.json && node ../../node_modules/typescript/bin/tsc -p tsconfig.test.json && node --test dist-test/test/*.test.js"
},
"keywords": [
"messagepack",

View file

@ -0,0 +1,370 @@
import type { CultCacheSchemaCatalogMember } from "./types";
import { WireFloat } from "./wire";
// A codec turns a value into the shape the reference runtimes write and back. Slot records are positional arrays with
// nil gaps; a union value is { kind: armName, ...armFields }, written [armKey, [armSlots...]]. Refusals name the union
// or the slot and the key or code, and never echo a decoded value. `typeName` is the reference's name for the type
// (CultSchemaTypeNames.FromType): a store's schema catalog carries it per slot, and the reference refuses a store whose
// slot names another type. The leaves know theirs; a union or a record the codec cannot name from C# is given it.
export interface Codec<TValue> {
encode(value: TValue): unknown;
decode(raw: unknown): TValue;
readonly typeName?: string;
// True for a type the reference makes Nullable<T> when the slot may be nil (int, long, float, double, bool).
readonly valueType?: boolean;
}
export interface SlotsCodec<TValue extends object> extends Codec<TValue> {
// Throws when a slot's codec has no typeName: the catalog cannot be derived without the reference's names.
readonly members: readonly CultCacheSchemaCatalogMember[];
readonly names: readonly string[];
}
function kindOf(raw: unknown): string {
if (raw === null || raw === undefined) return "nil";
if (Array.isArray(raw)) return "array";
if (raw instanceof WireFloat) return "float";
if (typeof raw === "bigint") return "integer";
return typeof raw;
}
function refuse(what: string, found: string): never {
throw new Error(`expected ${what}, found ${found}`);
}
function expect(raw: unknown, what: string, ok: boolean): asserts ok {
if (!ok) refuse(what, kindOf(raw));
}
export const string: Codec<string> = {
typeName: "System.String",
encode: (value) => (expect(value, "a string", typeof value === "string"), value),
decode: (raw) => (expect(raw, "a string", typeof raw === "string"), raw as string),
};
export const boolean: Codec<boolean> = {
typeName: "System.Boolean",
valueType: true,
encode: (value) => (expect(value, "a boolean", typeof value === "boolean"), value),
decode: (raw) => (expect(raw, "a boolean", typeof raw === "boolean"), raw as boolean),
};
// The reference's byte[] reads a MessagePack str as its bytes and writes bin (an old-spec writer's output), so a str is
// accepted and re-encodes as bin. The wire reader has already decoded the str, so a str that held invalid UTF-8 (it
// reads back U+FFFD) is refused rather than turned into other bytes.
export const bytes: Codec<Uint8Array> = {
typeName: "System.Byte[]",
encode: (value) => (expect(value, "bytes", value instanceof Uint8Array), value),
decode(raw) {
if (typeof raw === "string") {
if (raw.includes("<22>")) refuse("bytes", "a string that is not valid UTF-8");
return new TextEncoder().encode(raw);
}
expect(raw, "bytes", raw instanceof Uint8Array);
return raw as Uint8Array;
},
};
// A C# integer slot: written as the smallest MessagePack integer and read from any integer encoding in range. A float is
// refused even when it holds a whole number, as the reference refuses it.
function integerCodec(typeName: string, what: string, low: bigint, high: bigint): Codec<number | bigint> {
const inRange = (value: number | bigint) => value >= low && value <= high;
return {
typeName,
valueType: true,
encode(value) {
if (typeof value === "bigint") {
if (!inRange(value)) refuse(what, "an integer out of range");
return value;
}
if (typeof value !== "number") refuse(what, kindOf(value));
if (!Number.isSafeInteger(value)) {
refuse(what, Number.isInteger(value) ? "a number beyond 2^53, which needs a bigint" : "a non-integer number");
}
if (!inRange(value)) refuse(what, "an integer out of range");
return value;
},
decode(raw) {
if (typeof raw !== "number" && typeof raw !== "bigint") refuse(what, kindOf(raw));
if (!inRange(raw)) refuse(what, "an integer out of range");
return raw;
},
};
}
// C# int. Always a number.
export const int32 = integerCodec("System.Int32", "an int32", -(2n ** 31n), 2n ** 31n - 1n) as Codec<number>;
// C# long. A number when the value is a safe integer, a bigint beyond 2^53 (so 64-bit values never lose precision).
export const int64 = integerCodec("System.Int64", "an int64", -(2n ** 63n), 2n ** 63n - 1n);
// A 64-bit integer to the nearest float32 in one rounding, ties to even, as the reference's cast does. Number(value)
// first would round to a double and then again, and the two roundings can disagree. A bigint reaches here only beyond
// 2^53 (the wire reader returns a number for any smaller integer), so at least 30 bits are rounded away.
function bigintToFloat32(value: bigint): number {
const magnitude = value < 0n ? -value : value;
const excess = magnitude.toString(2).length - 24;
const kept = magnitude >> BigInt(excess);
const rest = magnitude - (kept << BigInt(excess));
const half = 1n << BigInt(excess - 1);
const rounded = Number(rest > half || (rest === half && (kept & 1n) === 1n) ? kept + 1n : kept) * 2 ** excess;
return value < 0n ? -rounded : rounded;
}
function floatCodec(typeName: string, wide: boolean): Codec<number> {
const what = wide ? "a float64" : "a float32";
return {
typeName,
valueType: true,
encode(value) {
expect(value, what, typeof value === "number");
return new WireFloat(value, wide);
},
decode(raw) {
if (raw instanceof WireFloat) {
// A JS number has no NaN payload, so a signalling float32 NaN cannot be kept; refusing beats changing its bytes.
if (raw.signalling && !wide) refuse(what, "a signalling NaN");
return wide ? raw.value : Math.fround(raw.value);
}
if (typeof raw === "number") return wide ? raw : Math.fround(raw);
if (typeof raw === "bigint") return wide ? Number(raw) : bigintToFloat32(raw);
return refuse(what, kindOf(raw));
},
};
}
// C# float, written as MessagePack float 32.
export const float32 = floatCodec("System.Single", false);
// C# double, written as MessagePack float 64.
export const float64 = floatCodec("System.Double", true);
function sequence<TItem>(item: Codec<TItem>, nameOf: (itemName: string) => string): Codec<TItem[]> {
return {
get typeName() { return item.typeName === undefined ? undefined : nameOf(item.typeName); },
encode(value) {
expect(value, "an array", Array.isArray(value));
return value.map((element, index) => at(`element ${index}`, () => item.encode(element)));
},
decode(raw) {
expect(raw, "an array", Array.isArray(raw));
return (raw as unknown[]).map((element, index) => at(`element ${index}`, () => item.decode(element)));
},
};
}
// C# List<T>.
export function list<TItem>(item: Codec<TItem>): Codec<TItem[]> {
return sequence(item, (name) => `System.Collections.Generic.List<${name}>`);
}
// C# T[]. The same wire as list.
export function array<TItem>(item: Codec<TItem>): Codec<TItem[]> {
return sequence(item, (name) => `${name}[]`);
}
// A slot that may be nil. For a value type the reference's name is Nullable<T>; for a reference type it is unchanged.
export function nullable<TValue>(inner: Codec<TValue>): Codec<TValue | null> {
return {
get typeName() {
return inner.typeName !== undefined && inner.valueType ? `System.Nullable<${inner.typeName}>` : inner.typeName;
},
encode: (value) => (value === null || value === undefined ? null : inner.encode(value)),
decode: (raw) => (raw === null ? null : inner.decode(raw)),
};
}
export function lazy<TValue>(get: () => Codec<TValue>): Codec<TValue> {
let resolved: Codec<TValue> | undefined;
const codec = () => (resolved ??= get());
return {
get typeName() { return codec().typeName; },
get valueType() { return codec().valueType; },
encode: (value) => codec().encode(value),
decode: (raw) => codec().decode(raw),
};
}
// Prefixes a refusal with where it happened, so a nested refusal still names its field and its union.
function at<TResult>(where: string, run: () => TResult): TResult {
try {
return run();
} catch (error) {
throw new Error(`${where}: ${(error as Error).message}`);
}
}
type SlotSpec<TValue, TName extends string = string> =
| readonly [name: TName, codec: Codec<TValue>]
| readonly [name: TName, codec: Codec<TValue>, defaultValue: TValue | (() => TValue)];
type SlotsSpec<TValue> = Readonly<Record<number, { [K in keyof TValue & string]: SlotSpec<TValue[K], K> }[keyof TValue & string]>>;
const INT32_MIN = -(2 ** 31);
const INT32_MAX = 2 ** 31 - 1;
// The largest slot key. A slot record is written as a dense array of key+1 slots (the reference writes the same, nil
// in every gap), so a key is a payload length: the cap keeps a definition's typo from allocating without bound.
export const MAX_SLOT_KEY = 65535;
// The declared keys, ascending. The reference's keys are Int32: a union arm may be any (a negative one too), a slot is
// 0 to MAX_SLOT_KEY, and a key outside its range is refused at definition rather than allocated for.
function slotKeys(spec: Readonly<Record<string, unknown>>, owner: string, low: number, high: number): number[] {
return Object.keys(spec)
.map((text) => {
if (!/^-?(0|[1-9][0-9]*)$/u.test(text) || (low >= 0 && text.startsWith("-"))) {
throw new Error(`${owner} declares the key "${text}", which is not ${low < 0 ? "an" : "a non-negative"} integer.`);
}
const key = BigInt(text);
if (key < BigInt(low) || key > BigInt(high)) {
throw new Error(`${owner} declares the key ${text}, which is ${high === INT32_MAX ? "outside Int32" : `above the slot key limit ${high}`}.`);
}
return Number(key);
})
// Object.keys lists negative keys after the others, in the order they were written: sort them numerically.
.sort((left, right) => left - right);
}
function pascalCase(name: string): string {
return name
.split(/[^A-Za-z0-9]/u)
.filter((part) => part.length > 0)
.map((part) => part[0].toUpperCase() + part.slice(1))
.join("");
}
// A positional record: slot `key` holds field `name`. Gaps are written nil and skipped on read, extra trailing slots
// are skipped, and a slot missing from the payload takes its declared default or is refused. A default fills only a
// missing trailing slot, never a nil: as in the reference, a nil in a reference slot (string, bytes, record, union,
// list) reads as null and writes back as nil, so a record the reference wrote re-encodes byte-identical, and a nil in
// a value slot (int, float, bool) is refused unless the slot is nullable.
export function slots<TValue extends object>(spec: SlotsSpec<TValue>, typeName?: string): SlotsCodec<TValue> {
const keys = slotKeys(spec, "A slot record", 0, MAX_SLOT_KEY);
if (keys.length === 0) {
throw new Error("A slot record declares no slots.");
}
const fields = keys.map((key) => {
const [name, codec, ...rest] = (spec as unknown as Readonly<Record<number, SlotSpec<unknown>>>)[key];
return { key, name, codec, hasDefault: rest.length > 0, defaultValue: rest[0] };
});
const seen = new Set<string>();
for (const field of fields) {
if (seen.has(field.name)) {
throw new Error(`A slot record declares the field "${field.name}" twice.`);
}
seen.add(field.name);
}
const length = keys[keys.length - 1] + 1;
const defaultOf = (field: (typeof fields)[number]): unknown =>
typeof field.defaultValue === "function" ? (field.defaultValue as () => unknown)() : field.defaultValue;
return {
names: fields.map((field) => field.name),
typeName,
get members() {
return fields.map((field) => {
if (field.codec.typeName === undefined) {
throw new Error(`${field.name} (slot ${field.key}) has no C# type name for the schema catalog; give its union or record a typeName, or give the document its members.`);
}
return { slot: field.key, memberName: pascalCase(field.name), typeName: field.codec.typeName };
});
},
encode(value) {
expect(value, "a record", typeof value === "object" && value !== null);
const out: unknown[] = new Array(length).fill(null);
for (const field of fields) {
const member = (value as Record<string, unknown>)[field.name];
out[field.key] = member === null && !field.codec.valueType
? null
: at(`${field.name} (slot ${field.key})`, () => field.codec.encode(member));
}
return out;
},
decode(raw) {
expect(raw, "an array of slots", Array.isArray(raw));
const input = raw as unknown[];
const out: Record<string, unknown> = {};
for (const field of fields) {
const present = field.key < input.length;
if (!present && !field.hasDefault) {
throw new Error(`${field.name} (slot ${field.key}): the slot is missing and declares no default.`);
}
out[field.name] = !present
? defaultOf(field)
: input[field.key] === null && !field.codec.valueType
? null
: at(`${field.name} (slot ${field.key})`, () => field.codec.decode(input[field.key]));
}
return out as TValue;
},
};
}
// An arm with no slots. It is written [key, []], never [key, nil].
export interface UnitArm {
readonly unit: true;
}
export const unit: UnitArm = Object.freeze({ unit: true });
type ArmSpec<TKind extends string> = readonly [armName: TKind, body: SlotsCodec<any> | UnitArm];
// A tagged union written [armKey, [armSlots...]]. The reader refuses an undeclared key, naming the union, the key and
// the declared keys; there is no fallback arm.
export function union<TValue extends { kind: string }>(
name: string,
arms: Readonly<Record<number, ArmSpec<TValue["kind"]>>>,
typeName?: string,
): Codec<TValue | null> {
const keys = slotKeys(arms, `Union ${name}`, INT32_MIN, INT32_MAX);
if (keys.length === 0) {
throw new Error(`Union ${name} declares no arms.`);
}
const byKey = new Map<number, { armName: string; body: SlotsCodec<any> | UnitArm }>();
const byName = new Map<string, number>();
for (const key of keys) {
const [armName, body] = arms[key];
if (byName.has(armName)) {
throw new Error(`Union ${name} declares the arm "${armName}" twice.`);
}
if (body !== unit && (body as SlotsCodec<any>).names.includes("kind")) {
throw new Error(`Union ${name} arm "${armName}" declares a slot named kind, which is the union's own tag.`);
}
byKey.set(key, { armName, body });
byName.set(armName, key);
}
const declared = keys.join(", ");
return {
typeName,
encode(value) {
if (value === null || value === undefined) {
return null;
}
const key = byName.get((value as { kind?: string }).kind as string);
if (key === undefined) {
throw new Error(`Union ${name}: the value's kind is not one of its arms (${[...byName.keys()].join(", ")}).`);
}
const { armName, body } = byKey.get(key)!;
return [key, body === unit ? [] : at(`arm ${armName}`, () => (body as SlotsCodec<any>).encode(value))];
},
decode(raw) {
if (raw === null) {
return null;
}
if (!Array.isArray(raw) || raw.length !== 2 || !Array.isArray(raw[1])) {
throw new Error(`Union ${name}: expected [key, slots] and found something else.`);
}
const key: unknown = raw[0];
if (typeof key !== "number" && typeof key !== "bigint") {
throw new Error(`Union ${name}: the arm key is not an integer (declared keys: ${declared}).`);
}
if (key < INT32_MIN || key > INT32_MAX) {
throw new Error(`Union ${name}: the arm key is outside Int32 (declared keys: ${declared}).`);
}
const arm = byKey.get(Number(key));
if (arm === undefined) {
throw new Error(`Union ${name}: unknown arm key ${key} (declared keys: ${declared}).`);
}
const fields = arm.body === unit ? {} : at(`arm ${arm.armName}`, () => (arm.body as SlotsCodec<any>).decode(raw[1]));
return { kind: arm.armName, ...fields } as unknown as TValue;
},
};
}

View file

@ -1,11 +1,17 @@
import { decode, encode } from "@msgpack/msgpack";
import {
decodeDocumentPayload,
documentPayloadBytes,
documentPayloadFromBytes,
encodeDocumentPayload,
resolveDocumentMembers,
} from "./document";
import { resolveDocumentFormatter } from "./document-formatter";
import type { DocumentPayload } from "./document";
import type {
AnyCultCacheDocumentDefinition,
CacheBackingStore,
CultCacheDocumentAccessor,
CultCacheDocumentDefinition,
CultCacheDocumentFormatter,
CultCacheDocumentRegistry,
CultCacheDocumentValue,
CultCacheEnvelope,
@ -15,10 +21,16 @@ import type {
type Accessor = (value: unknown) => string | undefined;
// A registered definition reads and writes its documents through the owner, on payloads the cache has cloned.
type RegisteredFormatter = {
encode(value: unknown): DocumentPayload;
decode(payload: DocumentPayload): unknown;
};
type RegisteredDefinition = {
readonly definition: AnyCultCacheDocumentDefinition;
readonly global: boolean;
readonly formatter: CultCacheDocumentFormatter<unknown>;
readonly formatter: RegisteredFormatter;
readonly catalogEntry: CultCacheSchemaCatalogEntry;
nameAccessor?: Accessor;
indexAccessors: ReadonlyMap<string, Accessor>;
@ -675,21 +687,13 @@ export class CultCache {
}
}
#createFormatter(definition: AnyCultCacheDocumentDefinition): CultCacheDocumentFormatter<unknown> {
const userFormatter = definition.formatter ?? {
encode: (value: unknown) => encode(value),
decode: (payload: Uint8Array) => decode(payload),
};
#createFormatter(definition: AnyCultCacheDocumentDefinition): RegisteredFormatter {
// A definition that declares both a codec and a formatter is refused here, where it is registered.
resolveDocumentFormatter(definition);
return {
encode: (value: unknown) => {
const parsed = definition.schema.parse(value);
return this.#cloneBytes(userFormatter.encode(parsed));
},
decode: (payload: Uint8Array) => {
const decoded = userFormatter.decode(this.#cloneBytes(payload));
return definition.schema.parse(decoded);
},
encode: (value: unknown) => this.#cloneBytes(encodeDocumentPayload(definition, value)),
decode: (payload: DocumentPayload) => decodeDocumentPayload(definition, this.#cloneBytes(payload)),
};
}
@ -699,7 +703,7 @@ export class CultCache {
const canonicalSchemaJson = definition.canonicalSchemaJson ?? JSON.stringify({
schemaName,
schemaVersion,
members: [...(definition.members ?? [])]
members: [...resolveDocumentMembers(definition)]
.sort((left, right) => left.slot - right.slot)
.map((member) => ({
slot: member.slot,
@ -724,7 +728,7 @@ export class CultCache {
contentHash: definition.contentHash ?? schemaId,
canonicalSchemaJson,
compatibleSchemaIds,
members: [...(definition.members ?? [])]
members: [...resolveDocumentMembers(definition)]
.sort((left, right) => left.slot - right.slot)
.map((member) => ({
slot: member.slot,
@ -1030,7 +1034,7 @@ export class CultCache {
return `${type}::${indexName}`;
}
#cloneBytes(payload: Uint8Array): Uint8Array {
return new Uint8Array(payload);
#cloneBytes(payload: DocumentPayload): DocumentPayload {
return documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(payload)));
}
}

View file

@ -0,0 +1,30 @@
import { decode, encode } from "@msgpack/msgpack";
import type { SlotsCodec } from "./codec";
import type { AnyCultCacheDocumentDefinition, CultCacheDocumentFormatter } from "./types";
import { readWire, writeWire } from "./wire";
// The one place that decides how a document is encoded: its own formatter, or its codec's, or a plain map. A codec and a
// formatter are two answers to one question, so a definition that carries both is refused wherever it enters (the
// builder resolves through here as well). Package-internal: no entry of the package's exports map names this module, so
// a consumer cannot import the formatter and read a payload around the schema.
export function resolveDocumentFormatter(definition: AnyCultCacheDocumentDefinition): CultCacheDocumentFormatter<unknown> {
if (definition.codec && definition.formatter) {
throw new Error(`CultCache document type "${definition.type}" declares both a codec and a formatter.`);
}
return definition.formatter ?? (definition.codec ? codecFormatter(definition.codec) : plainDocumentFormatter);
}
const plainDocumentFormatter: CultCacheDocumentFormatter<unknown> = {
encode: (value: unknown) => encode(value),
decode: (payload: Uint8Array) => decode(payload),
};
// The formatter a document gets from a slot codec: the codec's value as one MessagePack payload. Private to this module
// for the same reason as the resolver: exported, it would read a payload without the schema.
function codecFormatter<TValue extends object>(codec: SlotsCodec<TValue>): CultCacheDocumentFormatter<TValue> {
return {
encode: (value) => writeWire(codec.encode(value)),
decode: (payload) => codec.decode(readWire(payload)),
};
}

View file

@ -1,7 +1,10 @@
import { resolveDocumentFormatter } from "./document-formatter";
import type {
AnyCultCacheDocumentDefinition,
CultCacheSchemaCatalogMember,
CultCacheDocumentDefinition,
CultCacheDocumentRegistry,
CultCacheDocumentValue,
CultCacheSchema,
} from "./types";
@ -66,7 +69,71 @@ export function defineDocumentType<TSchema extends CultCacheSchema>(
}
}
return Object.freeze({ ...definition });
resolveDocumentFormatter(definition);
return Object.freeze(definition.codec ? { ...definition, members: resolveDocumentMembers(definition) } : { ...definition });
}
/**
* The bytes of one document, as they travel in a store envelope, a CultNet record or a realtime frame.
*
* At run time this is the same `Uint8Array`, so frames encode as they always did. At compile time it is opaque: it is not
* assignable to or from `Uint8Array` or `ArrayLike<number>`, so no byte decoder accepts it. The only reader is
* `decodeDocumentPayload(definition, payload)`, which needs the definition that says how the bytes are laid out, and the
* only writer is `encodeDocumentPayload(definition, value)`. The two escapes are named: `documentPayloadFromBytes` where
* bytes enter (a validated record, a store read, tests, a writer of pre-encoded or signed bytes) and
* `documentPayloadBytes` where they leave (a store write, a frame encode, hashing or signing).
*
* The key is a plain string, not a unique symbol, so two installed copies of this package stay compatible.
*
* What the type guarantees: a payload cannot be read or assigned directly (a decoder call, `const b: Uint8Array = p`,
* `new Uint8Array(p)` fail to compile). What it does not: the runtime value is a `Uint8Array`, so these routes read it
* without the owner: `instanceof Uint8Array` or `ArrayBuffer.isView` narrowing, `Object.values`, `Object.assign` onto a
* `Uint8Array`, a JSON round trip, `Reflect.get`, and a cast. A codec or formatter document read that way is raw bytes,
* not the document. The sanctioned escape for bytes is `documentPayloadBytes`; the reader is `decodeDocumentPayload`.
*/
export interface DocumentPayload {
readonly "cultcache.documentPayload": true;
readonly byteLength: number;
}
export function documentPayloadFromBytes(bytes: Uint8Array): DocumentPayload {
return bytes as unknown as DocumentPayload;
}
export function documentPayloadBytes(payload: DocumentPayload): Uint8Array {
return payload as unknown as Uint8Array;
}
// The one reader of a document payload: through the definition's formatter, codec or plain map, then its schema. A
// reader holds the definition or it has no business reading the bytes, so there is no definition-less form.
export function decodeDocumentPayload<TDefinition extends AnyCultCacheDocumentDefinition>(
definition: TDefinition,
payload: DocumentPayload,
): CultCacheDocumentValue<TDefinition> {
const decoded = resolveDocumentFormatter(definition).decode(documentPayloadBytes(payload));
return definition.schema.parse(decoded) as CultCacheDocumentValue<TDefinition>;
}
// The one writer: the definition's schema first, then its formatter, codec or plain map.
export function encodeDocumentPayload(definition: AnyCultCacheDocumentDefinition, value: unknown): DocumentPayload {
return documentPayloadFromBytes(resolveDocumentFormatter(definition).encode(definition.schema.parse(value)));
}
// The schema members of a document: the ones it declares, else those its codec derives, else none.
export function resolveDocumentMembers(definition: AnyCultCacheDocumentDefinition): readonly CultCacheSchemaCatalogMember[] {
if (definition.members || !definition.codec) {
return definition.members ?? [];
}
const codec = definition.codec;
const indexAlias = (slotName: string) =>
!Array.isArray(definition.indexes) && definition.indexes
? Object.entries(definition.indexes).find(([, accessor]) => accessor === slotName)?.[0] ?? null
: null;
return codec.members.map((member, index) => ({
...member,
isName: (definition.name as unknown) === codec.names[index],
indexAlias: indexAlias(codec.names[index]),
}));
}
export function defineDocumentRegistry<

View file

@ -1,6 +1,16 @@
export * as codecs from "./codec";
export type { Codec, SlotsCodec } from "./codec";
export { CultCache, CultCacheBuilder } from "./cult-cache";
export { inspectCultCacheBytes } from "./cult-cache-inspector";
export { defineDocumentRegistry, defineDocumentType } from "./document";
export {
decodeDocumentPayload,
defineDocumentRegistry,
defineDocumentType,
documentPayloadBytes,
documentPayloadFromBytes,
encodeDocumentPayload,
} from "./document";
export type { DocumentPayload } from "./document";
export { SingleFileMessagePackBackingStore } from "./single-file-messagepack-backing-store";
export type {
AnyCultCacheDocumentDefinition,

View file

@ -4,6 +4,7 @@ import { dirname, resolve } from "node:path";
import { decode, encode } from "@msgpack/msgpack";
import { z } from "zod";
import { documentPayloadBytes, documentPayloadFromBytes } from "./document";
import { STORE_FORMAT_VERSION, isV1Snapshot, requireV1RecordSlots } from "./store-format";
import type {
CacheBackingStore,
@ -62,7 +63,7 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
type: catalogEntry.schemaName,
schemaId: record.schemaId,
catalogEntry,
payload: record.payload,
payload: documentPayloadFromBytes(record.payload),
storedAt: record.storedAt,
};
});
@ -82,7 +83,10 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
payload,
};
});
const parsed = envelopeArraySchema.parse(normalized) as CultCacheEnvelope[];
const parsed: CultCacheEnvelope[] = envelopeArraySchema.parse(normalized).map((entry) => ({
...entry,
payload: documentPayloadFromBytes(entry.payload),
}));
if (repairedLegacyPayload) {
await this.#writeAll(parsed);
@ -232,7 +236,7 @@ function encodeSnapshot(entries: CultCacheEnvelope[]): unknown[] {
entry.key,
schemaIdForEnvelope(entry),
entry.storedAt,
entry.payload,
documentPayloadBytes(entry.payload),
]);
return [

View file

@ -1,3 +1,6 @@
import type { SlotsCodec } from "./codec";
import type { DocumentPayload } from "./document";
export type CultCacheIndexScalar = string | number | boolean | bigint;
export interface CultCacheSchema<TValue = unknown> {
@ -60,6 +63,7 @@ export interface CultCacheDocumentDefinition<TSchema extends CultCacheSchema = C
| Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>>
| readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[];
formatter?: CultCacheDocumentFormatter<InferCultCacheSchemaValue<TSchema>>;
codec?: SlotsCodec<InferCultCacheSchemaValue<TSchema> & object>;
}
export type CultCacheDocumentValue<TDefinition extends CultCacheDocumentDefinition> = InferCultCacheSchemaValue<
@ -77,7 +81,7 @@ export interface CultCacheDocumentRegistry<
export interface CultCacheEnvelope {
key: string;
type: string;
payload: Uint8Array;
payload: DocumentPayload;
storedAt: string;
schemaId?: string;
catalogEntry?: CultCacheSchemaCatalogEntry;

View file

@ -0,0 +1,134 @@
// The slice of MessagePack a codec value uses: nil, booleans, integers, floats, strings, binaries and arrays. A codec
// must tell an integer from a float and keep 64-bit integers whole, which a general decoder that hands back JS numbers
// cannot; so the codec path owns its reader and writer. An integer decodes to a number, or to a bigint when it is
// outside the safe range. A float decodes to a WireFloat, whatever its value.
export class WireFloat {
// signalling: a float32 NaN whose quiet bit is clear. The JS number it becomes has lost that payload.
constructor(readonly value: number, readonly wide: boolean, readonly signalling = false) {}
}
const SAFE_MIN = BigInt(Number.MIN_SAFE_INTEGER);
const SAFE_MAX = BigInt(Number.MAX_SAFE_INTEGER);
const utf8 = new TextEncoder();
const utf8Decoder = new TextDecoder();
export function writeWire(tree: unknown): Uint8Array {
const parts: number[] = [];
put(tree, parts);
return Uint8Array.from(parts);
}
function be(parts: number[], bytes: number, value: bigint): void {
for (let shift = BigInt((bytes - 1) * 8); shift >= 0n; shift -= 8n) {
parts.push(Number((value >> shift) & 0xffn));
}
}
function putInt(value: bigint, parts: number[]): void {
if (value >= 0n) {
if (value < 128n) parts.push(Number(value));
else if (value < 0x100n) { parts.push(0xcc); be(parts, 1, value); }
else if (value < 0x10000n) { parts.push(0xcd); be(parts, 2, value); }
else if (value < 0x100000000n) { parts.push(0xce); be(parts, 4, value); }
else if (value < 0x10000000000000000n) { parts.push(0xcf); be(parts, 8, value); }
else throw new Error("cannot write an integer wider than 64 bits.");
} else if (value >= -32n) parts.push(0x100 + Number(value));
else if (value >= -0x80n) { parts.push(0xd0); be(parts, 1, value & 0xffn); }
else if (value >= -0x8000n) { parts.push(0xd1); be(parts, 2, value & 0xffffn); }
else if (value >= -0x80000000n) { parts.push(0xd2); be(parts, 4, value & 0xffffffffn); }
else if (value >= -0x8000000000000000n) { parts.push(0xd3); be(parts, 8, value & 0xffffffffffffffffn); }
else throw new Error("cannot write an integer wider than 64 bits.");
}
function putLength(length: number, parts: number[], fix: number, fixMax: number, codes: readonly [number, number, number]): void {
if (length <= fixMax) parts.push(fix | length);
else if (codes[0] !== 0 && length < 0x100) { parts.push(codes[0]); be(parts, 1, BigInt(length)); }
else if (length < 0x10000) { parts.push(codes[1]); be(parts, 2, BigInt(length)); }
else { parts.push(codes[2]); be(parts, 4, BigInt(length)); }
}
function put(node: unknown, parts: number[]): void {
if (node === null) parts.push(0xc0);
else if (typeof node === "boolean") parts.push(node ? 0xc3 : 0xc2);
else if (typeof node === "bigint") putInt(node, parts);
else if (typeof node === "number") {
if (!Number.isSafeInteger(node)) throw new Error("cannot write a number that is not a safe integer; floats are WireFloat.");
putInt(BigInt(node), parts);
} else if (node instanceof WireFloat) {
const view = new DataView(new ArrayBuffer(8));
if (node.wide) { view.setFloat64(0, node.value); parts.push(0xcb); } else { view.setFloat32(0, node.value); parts.push(0xca); }
for (let index = 0; index < (node.wide ? 8 : 4); index += 1) parts.push(view.getUint8(index));
} else if (typeof node === "string") {
const text = utf8.encode(node);
putLength(text.length, parts, 0xa0, 31, [0xd9, 0xda, 0xdb]);
for (const byte of text) parts.push(byte);
} else if (node instanceof Uint8Array) {
putLength(node.length, parts, 0, -1, [0xc4, 0xc5, 0xc6]);
for (const byte of node) parts.push(byte);
} else if (Array.isArray(node)) {
putLength(node.length, parts, 0x90, 15, [0, 0xdc, 0xdd]);
for (const element of node) put(element, parts);
} else {
throw new Error("cannot write this kind of value.");
}
}
export function readWire(payload: Uint8Array): unknown {
const view = new DataView(payload.buffer, payload.byteOffset, payload.byteLength);
let offset = 0;
const take = (count: number): number => {
if (offset + count > payload.length) throw new Error("the MessagePack payload is truncated.");
offset += count;
return offset - count;
};
const integer = (value: bigint): number | bigint => (value >= SAFE_MIN && value <= SAFE_MAX ? Number(value) : value);
const string = (length: number): string => utf8Decoder.decode(payload.subarray(take(length), offset));
const binary = (length: number): Uint8Array => payload.slice(take(length), offset);
const array = (length: number): unknown[] => {
const out: unknown[] = [];
for (let index = 0; index < length; index += 1) out.push(next());
return out;
};
function next(): unknown {
const head = payload[take(1)];
if (head < 0x80) return head;
if (head >= 0xe0) return head - 0x100;
if (head >= 0xa0 && head <= 0xbf) return string(head & 0x1f);
if (head >= 0x90 && head <= 0x9f) return array(head & 0x0f);
switch (head) {
case 0xc0: return null;
case 0xc2: return false;
case 0xc3: return true;
case 0xc4: return binary(view.getUint8(take(1)));
case 0xc5: return binary(view.getUint16(take(2)));
case 0xc6: return binary(view.getUint32(take(4)));
case 0xca: {
const at = take(4);
const bits = view.getUint32(at);
return new WireFloat(view.getFloat32(at), false, (bits & 0x7f800000) === 0x7f800000 && (bits & 0x007fffff) !== 0 && (bits & 0x00400000) === 0);
}
case 0xcb: return new WireFloat(view.getFloat64(take(8)), true);
case 0xcc: return view.getUint8(take(1));
case 0xcd: return view.getUint16(take(2));
case 0xce: return view.getUint32(take(4));
case 0xcf: return integer(view.getBigUint64(take(8)));
case 0xd0: return view.getInt8(take(1));
case 0xd1: return view.getInt16(take(2));
case 0xd2: return view.getInt32(take(4));
case 0xd3: return integer(view.getBigInt64(take(8)));
case 0xd9: return string(view.getUint8(take(1)));
case 0xda: return string(view.getUint16(take(2)));
case 0xdb: return string(view.getUint32(take(4)));
case 0xdc: return array(view.getUint16(take(2)));
case 0xdd: return array(view.getUint32(take(4)));
default: throw new Error("the MessagePack payload holds a type a codec does not read (map, extension or reserved).");
}
}
const tree = next();
if (offset !== payload.length) throw new Error("the MessagePack payload has bytes after its value.");
return tree;
}

View file

@ -13,7 +13,7 @@ import { z } from "zod";
import { CultCache } from "../src/cult-cache";
import { inspectCultCacheBytes } from "../src/cult-cache-inspector";
import { defineDocumentRegistry, defineDocumentType } from "../src/document";
import { defineDocumentRegistry, defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
import type { CacheBackingStore, CultCacheEnvelope, CultCacheSchema } from "../src/types";
@ -202,7 +202,7 @@ test("CultCache fails closed on unknown or illegal persisted polymorphic state",
{
key: "mystery",
type: "forbidden",
payload: encode({ theme: "ash" }),
payload: documentPayloadFromBytes(encode({ theme: "ash" })),
storedAt: new Date().toISOString(),
},
]))
@ -219,13 +219,13 @@ test("CultCache fails closed on unknown or illegal persisted polymorphic state",
{
key: "one",
type: "settings",
payload: encode({ theme: "ash" }),
payload: documentPayloadFromBytes(encode({ theme: "ash" })),
storedAt: new Date().toISOString(),
},
{
key: "two",
type: "settings",
payload: encode({ theme: "ember" }),
payload: documentPayloadFromBytes(encode({ theme: "ember" })),
storedAt: new Date().toISOString(),
},
]))
@ -328,6 +328,26 @@ test("CultCache can ingest raw envelopes without re-encoding the payload", async
assert.deepEqual(target.getRequiredEnvelope(noteDocument, "note:hello").payload, envelope.payload);
});
test("the cache keeps its own copy of an envelope's bytes: a caller mutating them afterwards changes nothing", async () => {
const noteDocument = defineDocumentType({ type: "note", schema: z.object({ title: z.string(), body: z.string() }) });
const origin = CultCache.builder().withDocumentType(noteDocument)
.withGenericStore(new SingleFileMessagePackBackingStore(join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "origin.msgpack"))).build();
const target = CultCache.builder().withDocumentType(noteDocument)
.withGenericStore(new SingleFileMessagePackBackingStore(join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "target.msgpack"))).build();
await origin.put(noteDocument, "note:hello", { title: "Hello", body: "world" });
const envelope = origin.getRequiredEnvelope(noteDocument, "note:hello");
const before = Buffer.from(documentPayloadBytes(envelope.payload));
await target.putEnvelope(noteDocument, envelope);
documentPayloadBytes(envelope.payload).fill(0);
assert.deepEqual(Buffer.from(documentPayloadBytes(target.getRequiredEnvelope(noteDocument, "note:hello").payload)), before);
const read = target.getRequiredEnvelope(noteDocument, "note:hello");
documentPayloadBytes(read.payload).fill(0);
assert.deepEqual(Buffer.from(documentPayloadBytes(target.getRequiredEnvelope(noteDocument, "note:hello").payload)), before);
assert.deepEqual(target.getRequired(noteDocument, "note:hello"), { title: "Hello", body: "world" });
});
test("SingleFileMessagePackBackingStore writes the CultCache v1 snapshot shape", async () => {
const namedDocument = defineDocumentType({
type: "tests.named_entry",
@ -650,7 +670,7 @@ async function storeState(path: string): Promise<string[]> {
return [];
}
return (await new SingleFileMessagePackBackingStore(path).pullAll())
.map((entry) => `${entry.type}:${entry.key}:${Buffer.from(entry.payload).toString("hex")}`)
.map((entry) => `${entry.type}:${entry.key}:${Buffer.from(documentPayloadBytes(entry.payload)).toString("hex")}`)
.sort();
}
@ -821,8 +841,8 @@ test("concurrentGlobalPutsCannotBothReachDisk", async () => {
const onDisk = await inner.pullAll();
assert.deepEqual(onDisk.map((entry) => entry.key), [CultCache.GLOBAL_KEY]);
assert.deepEqual(
Buffer.from(onDisk[0].payload),
Buffer.from(cache.getRequiredEnvelope(settingsDocument, CultCache.GLOBAL_KEY).payload),
Buffer.from(documentPayloadBytes(onDisk[0].payload)),
Buffer.from(documentPayloadBytes(cache.getRequiredEnvelope(settingsDocument, CultCache.GLOBAL_KEY).payload)),
);
assert.deepEqual(cache.getRequiredGlobal(settingsDocument), { t: "second" });
const reloaded = build(new SingleFileMessagePackBackingStore(storePath));
@ -1042,7 +1062,7 @@ test("pullWithThrowingIndexAccessorLeavesCacheUnchanged", async () => {
await new SingleFileMessagePackBackingStore(storePath).push({
...potion,
key: "bomb",
payload: encode({ name: "Bomb", category: "explosive" }),
payload: documentPayloadFromBytes(encode({ name: "Bomb", category: "explosive" })),
});
await assert.rejects(cache.pullAllBackingStores(), /category accessor refuses Bomb/u);
@ -1349,7 +1369,7 @@ async function writeTsInteropStore(
members: interopNoteDocument.members,
},
storedAt: new Date().toISOString(),
payload,
payload: documentPayloadFromBytes(payload),
});
return note;
}
@ -1524,7 +1544,7 @@ test("SingleFileMessagePackBackingStore refuses a dangling symlink at its path a
const file = join(dir, "store.cc");
await symlink(join(volume, "store.cc"), file);
const store = new SingleFileMessagePackBackingStore(file);
const envelope = { key: "k", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" };
const envelope = { key: "k", type: "t", payload: documentPayloadFromBytes(Uint8Array.of(0x90)), storedAt: "2026-09-30T00:00:00Z" };
await assert.rejects(() => store.pullAll(), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
await assert.rejects(() => store.push(envelope), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
await assert.rejects(() => store.pushAll([envelope], { soft: true }), (error: NodeJS.ErrnoException) => error.code === "ENOENT");
@ -1552,7 +1572,7 @@ test("R3 decides resolve-or-refuse: a push through a live link replaces the link
const volume = join(dir, "volume");
await mkdir(volume);
const target = join(volume, "store.cc");
const envelope = { key: "a", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" };
const envelope = { key: "a", type: "t", payload: documentPayloadFromBytes(Uint8Array.of(0x90)), storedAt: "2026-09-30T00:00:00Z" };
await new SingleFileMessagePackBackingStore(target).push(envelope);
const before = await readFile(target);
const link = join(dir, "store.cc");
@ -1568,7 +1588,7 @@ test("R3 decides resolve-or-refuse: a pushAll through a dangling link replaces t
await mkdir(volume);
const link = join(dir, "store.cc");
await symlink(join(volume, "store.cc"), link);
await new SingleFileMessagePackBackingStore(link).pushAll([{ key: "a", type: "t", payload: Uint8Array.of(0x90), storedAt: "2026-09-30T00:00:00Z" }]);
await new SingleFileMessagePackBackingStore(link).pushAll([{ key: "a", type: "t", payload: documentPayloadFromBytes(Uint8Array.of(0x90)), storedAt: "2026-09-30T00:00:00Z" }]);
assert.ok((await lstat(link)).isFile());
assert.deepEqual(await readdir(volume), []);
});
@ -1585,6 +1605,6 @@ test("CultCache inspector refuses the same vectors by name", async () => {
test("a v1 store written at the base commit still reads byte for byte", async () => {
const envelopes = await pullVector("v1-base.msgpack");
assert.deepEqual(envelopes.map((entry) => [entry.key, entry.type]), [["alpha", "vectors.item"], ["beta", "vectors.item"]]);
assert.deepEqual([...envelopes[0]!.payload], [0x92, 0xa5, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x01]);
assert.deepEqual([...envelopes[1]!.payload], [0x92, 0xa4, 0x62, 0x65, 0x74, 0x61, 0x02]);
assert.deepEqual([...documentPayloadBytes(envelopes[0]!.payload)], [0x92, 0xa5, 0x61, 0x6c, 0x70, 0x68, 0x61, 0x01]);
assert.deepEqual([...documentPayloadBytes(envelopes[1]!.payload)], [0x92, 0xa4, 0x62, 0x65, 0x74, 0x61, 0x02]);
});

View file

@ -0,0 +1,43 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { encode } from "@msgpack/msgpack";
import { z } from "zod";
import { codecs, decodeDocumentPayload, defineDocumentType, documentPayloadBytes, documentPayloadFromBytes, encodeDocumentPayload } from "../src";
const payloadOf = (bytes: Uint8Array) => documentPayloadFromBytes(bytes);
const note = z.object({ id: z.string(), count: z.number() });
const plain = defineDocumentType({ type: "t.plain", schemaId: "t.plain.v0", schema: note, name: "id" });
const codec = defineDocumentType({
type: "t.codec",
schemaId: "t.codec.v0",
schema: note,
name: "id",
codec: codecs.slots<{ id: string; count: number }>({ 0: ["id", codecs.string], 1: ["count", codecs.int32] }, "T.Note"),
});
const formatted = defineDocumentType({
type: "t.formatted",
schemaId: "t.formatted.v0",
schema: note,
name: "id",
formatter: { encode: (value) => encode([(value as { id: string }).id]), decode: () => ({ id: "from-formatter", count: 0 }) },
});
test("a payload is read by its definition's formatter, a codec's, or a plain map", () => {
assert.deepEqual(decodeDocumentPayload(plain, payloadOf(encode({ id: "a", count: 1 }))), { id: "a", count: 1 });
assert.deepEqual(decodeDocumentPayload(codec, payloadOf(encode(["a", 1]))), { id: "a", count: 1 });
assert.deepEqual(decodeDocumentPayload(formatted, payloadOf(new Uint8Array([0xc0]))), { id: "from-formatter", count: 0 });
});
test("a definition that declares both a codec and a formatter is refused where its payload is read", () => {
const both = { ...codec, formatter: formatted.formatter } as typeof codec;
assert.throws(() => decodeDocumentPayload(both, payloadOf(encode(["a", 1]))), (error: Error) =>
error.message === 'CultCache document type "t.codec" declares both a codec and a formatter.');
});
test("a payload is written by its definition's codec or formatter, the way it is read", () => {
assert.deepEqual(documentPayloadBytes(encodeDocumentPayload(codec, { id: "a", count: 1 })), encode(["a", 1]));
assert.deepEqual(documentPayloadBytes(encodeDocumentPayload(plain, { id: "a", count: 1 })), encode({ id: "a", count: 1 }));
assert.deepEqual(documentPayloadBytes(encodeDocumentPayload(formatted, { id: "a", count: 1 })), encode(["a"]));
});

View file

@ -0,0 +1,61 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { decode, encode } from "@msgpack/msgpack";
import { z } from "zod";
import {
decodeDocumentPayload,
defineDocumentType,
documentPayloadBytes,
documentPayloadFromBytes,
encodeDocumentPayload,
type AnyCultCacheDocumentDefinition,
type CultCacheEnvelope,
type DocumentPayload,
} from "../src";
// The rule: a document payload is read only through its definition. The type is the enforcement, so this file is the
// proof: tsc compiles it only if every line below that carries @ts-expect-error is an error. A line that starts to
// compile makes its @ts-expect-error unused, which is itself an error, so the suite stops until the payload is opaque
// again. The function is never called; the lines are not meant to run.
export function refusedReaders(envelope: CultCacheEnvelope, definition: AnyCultCacheDocumentDefinition, payload: DocumentPayload): void {
// @ts-expect-error a plain msgpack decoder does not accept a document payload
decode(envelope.payload);
// @ts-expect-error a payload is not assignable to bytes
const bytes: Uint8Array = envelope.payload;
// @ts-expect-error nor is it a byte source a copy may be taken from
new Uint8Array(envelope.payload);
// @ts-expect-error a definition's formatter reads bytes, not a payload; the owner unwraps for it
definition.formatter!.decode(envelope.payload);
// @ts-expect-error the owner needs the definition that says how the bytes are laid out
decodeDocumentPayload(undefined, payload);
// @ts-expect-error bytes are not a payload until they enter by name
const forged: DocumentPayload = new Uint8Array();
void [bytes, forged];
}
const note = defineDocumentType({
type: "t.note",
schemaId: "t.note.v0",
schema: z.object({ id: z.string(), count: z.number() }),
name: "id",
});
test("a payload is the same bytes at run time, so frames and stores see no change", () => {
const bytes = encode({ id: "a", count: 1 });
const payload = documentPayloadFromBytes(bytes);
assert.equal(documentPayloadBytes(payload), bytes);
assert.equal(payload.byteLength, bytes.byteLength);
});
test("the owner writes a payload through the definition's schema and reads it back through the same one", () => {
const payload = encodeDocumentPayload(note, { id: "a", count: 1 });
assert.deepEqual(decode(documentPayloadBytes(payload)), { id: "a", count: 1 });
assert.deepEqual(decodeDocumentPayload(note, payload), { id: "a", count: 1 });
// A refusal names the field and never echoes the value it refused.
const refusesWithoutEchoing = (action: () => unknown) =>
assert.throws(action, (error: Error) => /count/.test(error.message) && !error.message.includes("canary-value"));
refusesWithoutEchoing(() => encodeDocumentPayload(note, { id: "a", count: "canary-value" }));
refusesWithoutEchoing(() => decodeDocumentPayload(note, documentPayloadFromBytes(encode({ id: "a", count: "canary-value" }))));
});

View file

@ -0,0 +1,789 @@
import assert from "node:assert/strict";
import { existsSync } from "node:fs";
import { mkdtemp, readFile, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { test } from "node:test";
import { decode, encode } from "@msgpack/msgpack";
import { z } from "zod";
import { codecs } from "../src";
import { MAX_SLOT_KEY, type Codec } from "../src/codec";
import { CultCache } from "../src/cult-cache";
import { readWire, WireFloat, writeWire } from "../src/wire";
import { defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
import type { AnyCultCacheDocumentDefinition, CacheBackingStore, CultCacheEnvelope } from "../src/types";
const { array, boolean, bytes, float32, float64, int32, int64, lazy, list, nullable, slots, string, union, unit } = codecs;
// The names the C# reference gives the fixture's types (CultSchemaTypeNames.FromType), as union-surface.cc carries them.
const CS = "GameCult.Caching.Tests.UnionParityVectorTests+";
// The golden is tests/vectors/union-parity, written by the C# reference. Everything below that is bytes or an expected
// outcome is read from those committed files; only the value the payload must decode to is spelled here.
function findVectors(from: string): string {
for (let dir = from; ; dir = resolve(dir, "..")) {
if (existsSync(join(dir, "tests", "vectors", "union-parity"))) return join(dir, "tests", "vectors", "union-parity");
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/union-parity not found above the test.");
}
}
const vectors = findVectors(__dirname);
const read = async (name: string) => new Uint8Array(await readFile(join(vectors, name)));
// The string the C# writer put inside every refusal payload, to prove a refusal never echoes the payload.
const CANARY = "CANARY-union-golden-4c2a";
type Node =
| { kind: "Box"; id: string; children: (Node | null)[] }
| { kind: "Text"; id: string; content: string }
| { kind: "Use"; id: string; composition: Composition; slots: (Node | null)[] }
| { kind: "Image"; id: string; uri: string; alt: string };
interface Composition { name: string; body: Node | null }
type Outcome = { kind: "Pending"; message: string } | { kind: "Completed"; sourceVersion: number } | { kind: "Denied" };
type Item = { kind: "Gem"; name: string } | { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
type CraftedItem = { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
interface Surface {
id: string;
version: number;
root: Node | null;
outcomes: (Outcome | null)[];
part: Item | null;
crafted: CraftedItem | null;
missing: Node | null;
}
const composition = slots<Composition>({ 0: ["name", string], 1: ["body", lazy(() => node)] }, `${CS}Composition`);
const node: Codec<Node | null> = union<Node>("Node", {
0: ["Box", slots<{ id: string; children: (Node | null)[] }>({ 0: ["id", string], 1: ["children", list(lazy(() => node))] })],
1: ["Text", slots<{ id: string; content: string }>({ 0: ["id", string], 1: ["content", string] })],
2: ["Use", slots<{ id: string; composition: Composition; slots: (Node | null)[] }>({
0: ["id", string], 1: ["composition", composition], 2: ["slots", list(lazy(() => node))],
})],
5: ["Image", slots<{ id: string; uri: string; alt: string }>({ 0: ["id", string], 1: ["uri", string], 3: ["alt", string] })],
}, `${CS}Node`);
const outcome = union<Outcome>("Outcome", {
0: ["Pending", slots<{ message: string }>({ 0: ["message", string] })],
1: ["Completed", slots<{ sourceVersion: number }>({ 0: ["sourceVersion", int32] })],
2: ["Denied", unit],
}, `${CS}Outcome`);
const toolArm = slots<{ name: string; uses: number }>({ 0: ["name", string], 1: ["uses", int32] });
const partArm = slots<{ name: string; count: number }>({ 0: ["name", string], 1: ["count", int32] });
const item = union<Item>("Item", {
0: ["Gem", slots<{ name: string }>({ 0: ["name", string] })],
1: ["Tool", toolArm],
2: ["Part", partArm],
}, `${CS}Item`);
const craftedItem = union<CraftedItem>("CraftedItem", { 0: ["Tool", toolArm], 1: ["Part", partArm] }, `${CS}CraftedItem`);
const surfaceCodec = slots<Surface>({
0: ["id", string],
1: ["version", int32],
2: ["root", node],
3: ["outcomes", list(outcome)],
4: ["part", item],
5: ["crafted", craftedItem],
6: ["missing", node, null],
}, `${CS}UnionSurface`);
const nodeSchema: z.ZodType<Node | null> = z.lazy(() =>
z.union([
z.null(),
z.discriminatedUnion("kind", [
z.object({ kind: z.literal("Box"), id: z.string(), children: z.array(nodeSchema) }),
z.object({ kind: z.literal("Text"), id: z.string(), content: z.string() }),
z.object({
kind: z.literal("Use"),
id: z.string(),
composition: z.object({ name: z.string(), body: nodeSchema }),
slots: z.array(nodeSchema),
}),
z.object({ kind: z.literal("Image"), id: z.string(), uri: z.string(), alt: z.string() }),
]),
]));
const surfaceSchema = z.object({
id: z.string(),
version: z.number(),
root: nodeSchema,
outcomes: z.array(z.union([
z.null(),
z.discriminatedUnion("kind", [
z.object({ kind: z.literal("Pending"), message: z.string() }),
z.object({ kind: z.literal("Completed"), sourceVersion: z.number() }),
z.object({ kind: z.literal("Denied") }),
]),
])),
part: z.union([z.null(), z.discriminatedUnion("kind", [
z.object({ kind: z.literal("Gem"), name: z.string() }),
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
])]),
crafted: z.union([z.null(), z.discriminatedUnion("kind", [
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
])]),
missing: nodeSchema,
}) as unknown as z.ZodType<Surface>;
function surfaceDocument(schemaId?: string) {
return defineDocumentType({
type: "tests.union_surface",
schemaId: schemaId ?? "tests.union_surface.v1",
schemaName: "tests.union_surface",
schemaVersion: "tests.union_surface.v1",
schema: surfaceSchema,
codec: surfaceCodec,
name: "id",
});
}
const golden: Surface = {
id: "surface",
version: 3,
root: {
kind: "Box",
id: "root",
children: [
{ kind: "Text", id: "t1", content: "hello" },
{
kind: "Use",
id: "u1",
composition: {
name: "comp",
body: {
kind: "Box",
id: "inner",
children: [
{ kind: "Text", id: "deep", content: "x" },
{ kind: "Image", id: "img", uri: "uri://a", alt: "alt" },
],
},
},
slots: [{ kind: "Text", id: "s1", content: "slot" }, null, { kind: "Box", id: "b2", children: [] }],
},
{ kind: "Image", id: "i2", uri: "uri://b", alt: "" },
],
},
outcomes: [{ kind: "Pending", message: "wait" }, { kind: "Completed", sourceVersion: 7 }, { kind: "Denied" }],
part: { kind: "Tool", name: "hammer", uses: 12 },
crafted: { kind: "Part", name: "gear", count: 4 },
missing: null,
};
const hex = (data: Uint8Array) => Buffer.from(data).toString("hex");
async function outcomeEntries() {
const entries = decode(await read("outcomes.msgpack")) as [string, string, string, number | null, string][];
return entries.map(([file, outcome, unionName, key, detail]) => ({ file, outcome, unionName, key, detail }));
}
test("the union surface vector decodes to the constructed value and re-encodes byte-identical", async () => {
const committed = await read("union-surface.msgpack");
const decoded = surfaceCodec.decode(readWire(committed));
assert.deepEqual(decoded, golden);
assert.equal(hex(writeWire(surfaceCodec.encode(decoded))), hex(committed));
assert.equal(hex(writeWire(surfaceCodec.encode(golden))), hex(committed));
});
test("a unit arm is written [key, []] and its payload must be an array", () => {
assert.deepEqual(outcome.encode({ kind: "Denied" }), [2, []]);
assert.throws(() => outcome.decode([2, null]), /Union Outcome/u);
});
test("union-surface.cc reads through SingleFileMessagePackBackingStore and writes the same payload", async () => {
const dir = await mkdtemp(join(tmpdir(), "union-parity-"));
const file = join(dir, "union-surface.cc");
await writeFile(file, await read("union-surface.cc"));
const [envelope] = await new SingleFileMessagePackBackingStore(file).pullAll();
assert.equal(hex(documentPayloadBytes(envelope.payload)), hex(await read("union-surface.msgpack")));
const document = surfaceDocument(envelope.schemaId);
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
await cache.pullAllBackingStores();
assert.deepEqual(cache.get(document, envelope.key), golden);
const written = join(dir, "written.cc");
const fresh = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(written)).build();
await fresh.put(document, envelope.key, golden);
const [rewritten] = await new SingleFileMessagePackBackingStore(written).pullAll();
assert.equal(hex(documentPayloadBytes(rewritten.payload)), hex(documentPayloadBytes(envelope.payload)));
// The reference refuses a store whose schema catalog names another type for a slot, so the catalog TS writes must be
// the one C# wrote: same canonical schema text, same member names, slots and types.
assert.equal(rewritten.catalogEntry!.canonicalSchemaJson, envelope.catalogEntry!.canonicalSchemaJson);
const shape = (members: readonly { slot: number; memberName: string; typeName: string; isName?: boolean }[]) =>
members.map((member) => [member.slot, member.memberName, member.typeName, member.isName === true]);
assert.deepEqual(shape(rewritten.catalogEntry!.members!), shape(envelope.catalogEntry!.members!));
assert.equal(rewritten.catalogEntry!.members!.length, 7);
});
test("every outcome vector is listed and holds: refusals name the union and key, never the payload", async () => {
const entries = await outcomeEntries();
assert.equal(entries.length, 9);
const document = surfaceDocument();
let refused = 0;
const carrying: string[] = [];
let readOnly = 0;
for (const { file, outcome: expected, unionName, key, detail } of entries) {
const payload = await read(file);
if (expected === "refused") {
refused += 1;
if (Buffer.from(payload).includes(CANARY)) carrying.push(file);
assert.throws(
() => surfaceCodec.decode(readWire(payload)),
(error: Error) => {
assert.ok(!error.message.includes(CANARY), `${file}: the refusal echoed the payload`);
if (detail === "arity") {
return error.message.includes(`Union ${unionName}`) && error.message.includes("expected [key, slots]");
}
return error.message.includes(`Union ${unionName}:`) && error.message.includes(`key ${key} `);
},
file,
);
const refusingCache = CultCache.builder().withDocumentType(document).withGenericStore(memoryStore([envelopeOf(payload)])).build();
await assert.rejects(() => refusingCache.pullAllBackingStores(), new RegExp(`Union ${unionName}`, "u"), file);
} else {
readOnly += 1;
assert.equal(expected, "read-only");
assert.deepEqual(surfaceCodec.decode(readWire(payload)), golden, `${file}: the extra slot is read past`);
}
}
assert.deepEqual([refused, readOnly], [5, 4]);
assert.deepEqual(carrying, ["unknown-arm-field.msgpack", "unknown-arm-list.msgpack", "unknown-arm-nested.msgpack"], "the canary is in the payloads whose arm is unknown");
});
test("a refusal lists the declared keys, so a key only the outer union declares is unknown to the inner", async () => {
const entry = (await outcomeEntries()).find((candidate) => candidate.file === "inner-union-outer-key.msgpack")!;
const payload = readWire(await read(entry.file)) as unknown[];
assert.throws(
() => surfaceCodec.decode(payload),
/Union CraftedItem: unknown arm key 2 \(declared keys: 0, 1\)\./u,
);
});
test("an arm that is not [key, slots] is refused naming the union, and a non-integer key is not echoed", () => {
for (const bad of [[1], [1, null], [1, [], []], "x", 7, {}]) {
assert.throws(() => node.decode(bad), /^Error: Union Node: expected \[key, slots\]/u);
}
assert.throws(
() => node.decode([CANARY, []]),
(error: Error) => error.message.includes("Union Node") && error.message.includes("not an integer") && !error.message.includes(CANARY),
);
});
test("an arm key that is not a declared key is refused, with no fallback", () => {
assert.throws(() => node.decode([4, []]), /Union Node: unknown arm key 4 \(declared keys: 0, 1, 2, 5\)/u);
assert.throws(() => outcome.decode([3, []]), /Union Outcome: unknown arm key 3/u);
});
test("a union encodes by kind and refuses a kind it does not declare without echoing it", () => {
assert.deepEqual(node.encode({ kind: "Text", id: "a", content: "b" }), [1, ["a", "b"]]);
assert.equal(node.encode(null), null);
assert.throws(
() => node.encode({ kind: CANARY, id: "a" } as unknown as Node),
(error: Error) => error.message.includes("Union Node") && !error.message.includes(CANARY),
);
});
test("slots: a gap writes nil, a missing slot reads its default, one without a default is refused naming the slot", () => {
const record = slots<{ a: string; c: string; d: number }>({ 0: ["a", string], 2: ["c", string], 4: ["d", int32, 9] });
assert.deepEqual(record.encode({ a: "x", c: "y", d: 1 }), ["x", null, "y", null, 1]);
assert.deepEqual(record.decode(["x", null, "y"]), { a: "x", c: "y", d: 9 });
assert.deepEqual(record.decode(["x", 5, "y", null]), { a: "x", c: "y", d: 9 });
assert.throws(() => record.decode(["x", 5, "y", null, null]), { message: "d (slot 4): expected an int32, found nil" });
assert.throws(() => record.decode(["x"]), /c \(slot 2\): the slot is missing and declares no default/u);
assert.deepEqual(record.decode(["x", null, "y", null, 1, "extra", "slots"]), { a: "x", c: "y", d: 1 });
assert.throws(() => record.decode({ a: "x" }), /expected an array of slots/u);
});
test("slots: a default may be a function and fills only a missing slot, never a nil", () => {
const record = slots<{ tags: string[] }>({ 0: ["tags", list(string), () => ["fresh"]] });
assert.deepEqual(record.decode([]), { tags: ["fresh"] });
assert.deepEqual(record.decode([null]), { tags: null });
const counted = slots<{ n: number }>({ 0: ["n", int32, 4] });
assert.deepEqual(counted.decode([]), { n: 4 });
assert.throws(() => counted.decode([null]), { message: "n (slot 0): expected an int32, found nil" });
});
test("a slot refusal names the field and slot and never the value", () => {
const record = slots<{ name: string }>({ 3: ["name", string] });
assert.throws(
() => record.decode([null, null, null, { secret: CANARY }]),
(error: Error) => error.message.startsWith("name (slot 3):") && !error.message.includes(CANARY),
);
});
test("leaf codecs refuse the wrong shape on both sides, saying what was expected and what kind was found", () => {
const table: [string, Codec<unknown>, string, unknown, string][] = [
["string", string, "a string", 5, "number"],
["int32", int32, "an int32", "5", "string"],
["int32", int32, "an int32", Number.POSITIVE_INFINITY, "a non-integer number"],
["int32", int32, "an int32", Number.NaN, "a non-integer number"],
["int32", int32, "an int32", 1.5, "a non-integer number"],
["int32", int32, "an int32", 2 ** 31, "an integer out of range"],
["int64", int64, "an int64", 2 ** 53, "a number beyond 2^53, which needs a bigint"],
["int64", int64, "an int64", 2n ** 63n, "an integer out of range"],
["float64", float64, "a float64", "1.5", "string"],
["float32", float32, "a float32", null, "nil"],
["boolean", boolean, "a boolean", "true", "string"],
["bytes", bytes, "bytes", [1, 2], "array"],
["string", string, "a string", null, "nil"],
["list", list(string), "an array", "abc", "string"],
];
for (const [label, codec, what, bad, found] of table) {
for (const side of ["encode", "decode"] as const) {
// Decode sees the wire's shapes: a JS float or a number beyond 2^53 never arrives from a payload.
if (side === "decode" && (typeof bad === "number" && (!Number.isInteger(bad) || Math.abs(bad) >= 2 ** 53))) continue;
assert.throws(() => codec[side](bad), { message: `expected ${what}, found ${found}` }, `${label} ${side}`);
}
}
for (const [codec, good] of [[string, "s"], [int32, 1], [int64, 5], [float64, 1.5], [float32, 1.5], [boolean, false], [bytes, new Uint8Array([7])], [list(int32), [1, 2]]] as [Codec<unknown>, unknown][]) {
assert.deepEqual(codec.decode(codec.encode(good)), good);
}
});
test("a list element refusal names the element, and nullable writes nil for null and undefined", () => {
const numbers = list(int32);
assert.throws(() => numbers.encode([1, "x" as unknown as number]), /^Error: element 1: expected an int32, found string$/u);
assert.throws(() => numbers.decode([1, 2, null]), /^Error: element 2: expected an int32, found nil$/u);
const maybe = nullable(int32);
assert.equal(maybe.encode(null), null);
assert.equal(maybe.encode(undefined as unknown as null), null);
assert.equal(maybe.encode(0), 0);
assert.equal(maybe.decode(null), null);
assert.equal(maybe.decode(0), 0);
assert.throws(() => maybe.decode("x"), /expected an int32/u);
});
test("a record refuses what is not a record and names the field and slot of a bad value", () => {
const record = slots<{ name: string }>({ 3: ["name", string] });
assert.throws(() => record.encode(null as unknown as { name: string }), /^Error: expected a record, found nil$/u);
assert.throws(() => record.encode("x" as unknown as { name: string }), /^Error: expected a record, found string$/u);
assert.throws(() => record.encode({ name: 5 as unknown as string }), /^Error: name \(slot 3\): expected a string, found number$/u);
});
test("a union names itself, its arm and the field when an arm's slots are refused", () => {
assert.throws(() => node.encode({ kind: "Text", id: "a", content: 5 as unknown as string }), /^Error: arm Text: content \(slot 1\): expected a string, found number$/u);
assert.throws(() => node.decode([1, ["a", 5]]), /^Error: arm Text: content \(slot 1\): expected a string, found number$/u);
assert.equal(node.encode(undefined as unknown as null), null);
assert.throws(
() => node.encode({ kind: "Nope" } as unknown as Node),
/^Error: Union Node: the value's kind is not one of its arms \(Box, Text, Use, Image\)\.$/u,
);
});
test("slot names become catalog member names in Pascal case", () => {
const record = slots<{ _hidden: string; two_words: string; Mixed9: string }>({ 0: ["_hidden", string], 1: ["two_words", string], 2: ["Mixed9", string] });
assert.deepEqual(record.members.map((member) => member.memberName), ["Hidden", "TwoWords", "Mixed9"]);
});
test("definitions are refused at definition time", () => {
assert.throws(() => union<{ kind: "A" }>("U", { 0: ["A", slots<{ kind: string }>({ 0: ["kind", string] })] }), /Union U arm "A" declares a slot named kind/u);
assert.throws(() => union<{ kind: "A" | "B" }>("U", { 0: ["A", unit], 1: ["A" as "B", unit] }), /declares the arm "A" twice/u);
assert.throws(() => union<{ kind: "A" }>("U", { x: ["A", unit] } as never), /^Error: Union U declares the key "x", which is not an integer\.$/u);
assert.throws(() => union<{ kind: "A" }>("U", { "01": ["A", unit] } as never), /key "01"/u);
assert.throws(() => union<{ kind: "A" }>("U", { "1.5": ["A", unit] } as never), /key "1\.5"/u);
assert.throws(() => union<{ kind: "A" }>("U", {} as never), /^Error: Union U declares no arms\.$/u);
assert.throws(() => slots<{ a: string; b: string }>({ 0: ["a", string], 1: ["a" as "b", string] }), /declares the field "a" twice/u);
assert.throws(() => slots<{ a: string }>({}), /declares no slots/u);
assert.throws(() => slots<{ a: string }>({ [-1]: ["a", string] }), /^Error: A slot record declares the key "-1"/u);
assert.throws(
() => defineDocumentType({
type: "t",
schema: surfaceSchema,
codec: surfaceCodec,
formatter: { encode: () => new Uint8Array(), decode: () => golden },
}),
/declares both a codec and a formatter/u,
);
});
test("a codec supplies the document's catalog members unless it is given its own", () => {
const document = defineDocumentType({
type: "t",
schema: z.object({ a: z.string(), the_b: z.string() }),
codec: slots<{ a: string; the_b: string }>({ 0: ["a", string], 3: ["the_b", string] }),
name: "a",
indexes: { byB: "the_b" },
});
assert.deepEqual(document.members, [
{ slot: 0, memberName: "A", typeName: "System.String", isName: true, indexAlias: null },
{ slot: 3, memberName: "TheB", typeName: "System.String", isName: false, indexAlias: "byB" },
]);
const given = [{ slot: 9, memberName: "Mine", typeName: "System.String" }];
const own = defineDocumentType({ type: "t", schema: z.object({ a: z.string() }), codec: slots<{ a: string }>({ 0: ["a", string] }), members: given });
assert.deepEqual(own.members, given);
});
test("schema.parse still runs on a decoded value, and a document without a codec still writes a map", async () => {
const strict = defineDocumentType({
type: "t",
schema: z.object({ a: z.string().min(3) }),
codec: slots<{ a: string }>({ 0: ["a", string] }),
});
const store = memoryStore([{ key: "k", type: "t", payload: documentPayloadFromBytes(encode(["xy"])), storedAt: new Date().toISOString() }]);
await assert.rejects(() => CultCache.builder().withDocumentType(strict).withGenericStore(store).build().pullAllBackingStores());
const plain = defineDocumentType({ type: "p", schema: z.object({ id: z.string() }) });
const holder = memoryStore([]);
await CultCache.builder().withDocumentType(plain).withGenericStore(holder).build().put(plain, "k", { id: "d" });
assert.deepEqual(decode(documentPayloadBytes(holder.entries[0].payload)), { id: "d" });
});
test("bytes and nullable round-trip through a slot record", () => {
const record = slots<{ blob: Uint8Array; maybe: string | null }>({ 0: ["blob", bytes], 1: ["maybe", nullable(string)] });
const wire = encode(record.encode({ blob: new Uint8Array([1, 2]), maybe: null }));
assert.deepEqual(record.decode(decode(wire)), { blob: new Uint8Array([1, 2]), maybe: null });
assert.throws(() => record.decode([new Uint8Array(), 4]), /maybe \(slot 1\): expected a string/u);
});
const fromHex = (text: string) => new Uint8Array(Buffer.from(text, "hex"));
// Arm keys are integers within Int32 in the reference: a float key is refused even when it holds a whole number, and a
// key outside Int32 is refused whether it is declared or read. Every payload below is [key, [7]] or [key, ["wait"]].
test("an arm key is an integer in any MessagePack integer encoding, and a float-typed key is refused", () => {
const completed = { kind: "Completed", sourceVersion: 7 };
for (const key of ["01", "cc01", "cd0001", "ce00000001", "cf0000000000000001", "d001", "d10001", "d200000001", "d30000000000000001"]) {
assert.deepEqual(outcome.decode(readWire(fromHex(`92${key}9107`))), completed, key);
}
for (const key of ["ca3f800000", "cb3ff0000000000000"]) {
assert.throws(() => outcome.decode(readWire(fromHex(`92${key}9107`))), { message: "Union Outcome: the arm key is not an integer (declared keys: 0, 1, 2)." }, key);
}
assert.throws(() => outcome.decode(readWire(fromHex("92cb800000000000000091a477616974"))), /^Error: Union Outcome: the arm key is not an integer/u, "-0.0 is a float, not key 0");
for (const key of ["ce80000000", "cf8000000000000000", "d30000000080000000"]) {
assert.throws(() => outcome.decode(readWire(fromHex(`92${key}9107`))), { message: "Union Outcome: the arm key is outside Int32 (declared keys: 0, 1, 2)." }, key);
}
assert.throws(() => outcome.decode(readWire(fromHex("92d2800000009107"))), { message: "Union Outcome: unknown arm key -2147483648 (declared keys: 0, 1, 2)." });
assert.throws(() => outcome.decode(readWire(fromHex("92ff9107"))), { message: "Union Outcome: unknown arm key -1 (declared keys: 0, 1, 2)." });
});
test("a union refuses a declared arm key outside Int32", () => {
assert.doesNotThrow(() => union<{ kind: "A" }>("U", { 2147483647: ["A", unit] }));
assert.throws(() => union<{ kind: "A" }>("U", { 2147483648: ["A", unit] }), { message: "Union U declares the key 2147483648, which is outside Int32." });
assert.throws(() => union<{ kind: "A" }>("U", { 3000000000: ["A", unit] }), /outside Int32/u);
});
test("the wire reader keeps 64-bit integers whole, tells a float from an integer, and refuses what is not a value", () => {
assert.equal(readWire(fromHex("cf0020000000000001")), 2n ** 53n + 1n);
assert.equal(readWire(fromHex("cf001fffffffffffff")), 2 ** 53 - 1);
assert.equal(readWire(fromHex("d38000000000000000")), -(2n ** 63n));
assert.equal(readWire(fromHex("cfffffffffffffffff")), 2n ** 64n - 1n);
assert.ok(readWire(fromHex("cb3ff0000000000000")) instanceof WireFloat);
assert.ok(readWire(fromHex("ca3f800000")) instanceof WireFloat);
assert.equal(readWire(fromHex("01")), 1);
assert.throws(() => readWire(fromHex("92a161")), /truncated/u, "an array that promises more than the payload holds");
assert.throws(() => readWire(fromHex("dd")), /truncated/u);
assert.throws(() => readWire(fromHex("ddffffffff")), /truncated/u, "a huge array header reserves nothing");
assert.throws(() => readWire(fromHex("a5616263")), /truncated/u);
assert.throws(() => readWire(fromHex("0102")), /bytes after its value/u);
assert.throws(() => readWire(fromHex("81a161a162")), /map, extension or reserved/u);
assert.throws(() => readWire(fromHex("c1")), /map, extension or reserved/u);
assert.throws(() => readWire(fromHex("d4010a")), /map, extension or reserved/u);
});
test("the wire writer writes the smallest integer, keeps a float's width, and round-trips what it reads", () => {
const tree = [0, 127, 128, 255, 256, 65535, 65536, 4294967295, 4294967296, -1, -32, -33, -128, -129, -32768, -32769, -2147483648, -2147483649,
2n ** 53n, 2n ** 64n - 1n, -(2n ** 63n), new WireFloat(1.5, false), new WireFloat(1.5, true), "", "a".repeat(31), "a".repeat(32), "a".repeat(256),
new Uint8Array(0), new Uint8Array(300), [], null, true, false, "héllo"];
assert.deepEqual(readWire(writeWire(tree)), tree);
assert.equal(hex(writeWire(5)), "05");
assert.equal(hex(writeWire(128)), "cc80");
assert.equal(hex(writeWire(-33)), "d0df");
assert.equal(hex(writeWire(2n ** 53n + 2n)), "cf0020000000000002");
assert.equal(hex(writeWire(new WireFloat(3, true))), "cb4008000000000000");
assert.equal(hex(writeWire(new WireFloat(1.5, false))), "ca3fc00000");
assert.equal(hex(writeWire(new Array(16).fill(null))), `dc0010${"c0".repeat(16)}`);
assert.throws(() => writeWire(1.5), /not a safe integer/u);
assert.throws(() => writeWire(2 ** 53), /not a safe integer/u);
assert.throws(() => writeWire(2n ** 64n), /wider than 64 bits/u);
assert.throws(() => writeWire({}), /this kind of value/u);
});
test("a codec and a formatter are refused together wherever the definition enters", () => {
const both = { type: "t", schema: surfaceSchema, codec: surfaceCodec, formatter: { encode: () => new Uint8Array(), decode: () => golden } };
assert.throws(() => defineDocumentType(both), /declares both a codec and a formatter/u);
assert.throws(() => CultCache.builder().withDocumentType(both), /declares both a codec and a formatter/u);
assert.throws(() => CultCache.builder().withRegistry([both]), /declares both a codec and a formatter/u);
});
test("a definition that skips defineDocumentType still writes its codec's bytes and its derived catalog", async () => {
const raw = { type: "tests.union_surface", schemaId: "tests.union_surface.v1", schema: surfaceSchema, codec: surfaceCodec, name: "id" } as unknown as AnyCultCacheDocumentDefinition;
const holder = memoryStore([]);
const cache = CultCache.builder().withDocumentType(raw).withGenericStore(holder).build();
await cache.put(raw as never, "surface", golden as never);
assert.equal(hex(documentPayloadBytes(holder.entries[0].payload)), hex(await read("union-surface.msgpack")));
const [snapshot] = cache.snapshot();
assert.deepEqual(snapshot.catalogEntry!.members!.map((member) => member.typeName).slice(0, 2), ["System.String", "System.Int32"]);
});
test("every leaf and wrapper carries the reference's type name, and a catalog is not derived from a codec that cannot name a slot", () => {
assert.deepEqual(
[string, boolean, bytes, int32, int64, float32, float64].map((codec) => codec.typeName),
["System.String", "System.Boolean", "System.Byte[]", "System.Int32", "System.Int64", "System.Single", "System.Double"],
);
assert.equal(list(int64).typeName, "System.Collections.Generic.List<System.Int64>");
assert.equal(array(string).typeName, "System.String[]");
assert.equal(nullable(int32).typeName, "System.Nullable<System.Int32>");
assert.equal(nullable(string).typeName, "System.String");
assert.equal(nullable(list(string)).typeName, "System.Collections.Generic.List<System.String>");
assert.equal(list(union<{ kind: "A" }>("U", { 0: ["A", unit] })).typeName, undefined);
const unnamed = slots<{ id: string; what: { kind: "A" } | null }>({ 0: ["id", string], 1: ["what", union<{ kind: "A" }>("U", { 0: ["A", unit] })] });
assert.throws(() => unnamed.members, /^Error: what \(slot 1\) has no C# type name for the schema catalog/u);
assert.throws(() => defineDocumentType({ type: "t", schema: z.any(), codec: unnamed as never }), /what \(slot 1\) has no C# type name/u);
const given = defineDocumentType({ type: "t", schema: z.any(), codec: unnamed as never, members: [{ slot: 0, memberName: "Id", typeName: "System.String" }] });
assert.equal(given.members!.length, 1);
});
// tests/vectors/numeric-leaves is written by the C# reference (NumericLeafVectorTests): what it reads and refuses for each
// numeric leaf, and the bytes it writes.
const numericVectors = join(vectors, "..", "numeric-leaves");
const numericLeaves: Record<string, Codec<unknown>> = { int32, int64, float32, float64 };
const doubleBitsHex = (value: number) => {
const view = new DataView(new ArrayBuffer(8));
view.setFloat64(0, value);
return Buffer.from(view.buffer).toString("hex");
};
const int64BitsHex = (value: number | bigint) => BigInt.asUintN(64, BigInt(value)).toString(16).padStart(16, "0");
const isNaNBits = (bitsHex: string) => {
const bits = BigInt(`0x${bitsHex}`);
return ((bits >> 52n) & 0x7ffn) === 0x7ffn && (bits & ((1n << 52n) - 1n)) !== 0n;
};
const isSignallingFloat32 = (payload: Uint8Array) => {
// payload is a one-slot record [float32]: 91 ca b0 b1 b2 b3
if (payload.length !== 6 || payload[1] !== 0xca) return false;
const bits = new DataView(payload.buffer, payload.byteOffset).getUint32(2);
return (bits & 0x7f800000) === 0x7f800000 && (bits & 0x007fffff) !== 0 && (bits & 0x00400000) === 0;
};
test("numeric leaves read and refuse exactly what the C# reference does", async () => {
const rows = readWire(new Uint8Array(await readFile(join(numericVectors, "reads.msgpack")))) as [string, Uint8Array, boolean, Uint8Array][];
assert.equal(rows.length, 160);
let accepted = 0;
let refused = 0;
for (const [leaf, payload, wasAccepted, bits] of rows) {
const label = `${leaf} ${hex(payload)}`;
const record = slots<{ v: unknown }>({ 0: ["v", numericLeaves[leaf]] });
if (wasAccepted && leaf === "float32" && isSignallingFloat32(payload)) {
// The one reading that differs from the reference: it reads a float32 signalling NaN, this runtime refuses it
// (float32-nan.msgpack shows the reference keeps its bits, which a JS number cannot).
assert.throws(() => record.decode(readWire(payload)), (error: Error) => !error.message.includes(hex(payload).slice(2)), label);
refused += 1;
} else if (wasAccepted) {
const read = record.decode(readWire(payload)).v as number | bigint;
const wanted = hex(bits);
if (leaf.startsWith("float")) {
if (isNaNBits(wanted)) assert.ok(Number.isNaN(read), label);
else assert.equal(doubleBitsHex(read as number), wanted, label);
} else {
assert.equal(int64BitsHex(read), wanted, label);
}
accepted += 1;
} else {
assert.throws(() => record.decode(readWire(payload)), (error: Error) => !error.message.includes(hex(payload).slice(2)), label);
refused += 1;
}
}
assert.ok(accepted > 40 && refused > 40, `accepted ${accepted}, refused ${refused}`);
assert.equal(accepted + refused, rows.length);
});
// float32-nan.msgpack: [payload, bytes the reference writes after reading it] for a float32 slot holding an infinity or a NaN.
// The reference keeps every payload bit; a JS number cannot keep a signalling NaN's, so that read is refused.
test("a float32 NaN round-trips with its bits, and a signalling one is refused on read", async () => {
const rows = readWire(new Uint8Array(await readFile(join(numericVectors, "float32-nan.msgpack")))) as [Uint8Array, Uint8Array][];
assert.equal(rows.length, 6);
const record = slots<{ v: number }>({ 0: ["v", float32] });
let refused = 0;
for (const [payload, rewritten] of rows) {
if (isSignallingFloat32(payload)) {
assert.equal(hex(rewritten), hex(payload), "the reference keeps the signalling bits");
assert.throws(() => record.decode(readWire(payload)), { message: "v (slot 0): expected a float32, found a signalling NaN" }, hex(payload));
refused += 1;
} else {
assert.equal(hex(writeWire(record.encode(record.decode(readWire(payload))))), hex(rewritten), hex(payload));
}
}
assert.equal(refused, 3);
});
test("only a NaN with its quiet bit clear is signalling: a finite float32 with a clear bit 22 reads, and a float64 slot reads any float32 NaN", () => {
const f32 = slots<{ v: number }>({ 0: ["v", float32] });
const f64 = slots<{ v: number }>({ 0: ["v", float64] });
assert.equal(f32.decode(readWire(fromHex("91ca3f800001"))).v, Math.fround(1.0000001192092896));
assert.equal(f32.decode(readWire(fromHex("91ca00000001"))).v, Math.fround(1e-45));
assert.equal(f32.decode(readWire(fromHex("91ca7f800000"))).v, Infinity);
assert.ok(Number.isNaN(f32.decode(readWire(fromHex("91ca7fc00001"))).v));
assert.ok(Number.isNaN(f64.decode(readWire(fromHex("91ca7f800001"))).v));
});
test("numeric leaves write the bytes the C# reference writes, and 64-bit values lose nothing", async () => {
const rows = readWire(new Uint8Array(await readFile(join(numericVectors, "writes.msgpack")))) as [string, string, Uint8Array][];
assert.equal(rows.length, 46);
for (const [leaf, text, expected] of rows) {
const value = leaf.startsWith("float") ? new DataView(Uint8Array.from(Buffer.from(text, "hex")).buffer).getFloat64(0) : leaf === "int32" ? Number(text) : BigInt(text);
const record = slots<{ v: unknown }>({ 0: ["v", numericLeaves[leaf]] });
assert.equal(hex(writeWire(record.encode({ v: value }))), hex(expected), `${leaf} ${text}`);
}
const wide = slots<{ v: number | bigint }>({ 0: ["v", int64] });
for (const value of [2n ** 53n + 1n, 2n ** 63n - 1n, -(2n ** 63n), -(2n ** 53n) - 1n]) {
assert.equal(wide.decode(readWire(writeWire(wide.encode({ v: value })))).v, value);
}
assert.equal(wide.decode(readWire(writeWire(wide.encode({ v: 2n })))).v, 2, "a bigint that fits comes back a number");
});
test("the wire writer's width boundaries: where a length or an integer changes its header", () => {
assert.equal(hex(writeWire(0)), "00");
assert.equal(hex(writeWire("a".repeat(31))).slice(0, 2), "bf");
assert.equal(hex(writeWire("a".repeat(32))).slice(0, 4), "d920");
assert.equal(hex(writeWire("a".repeat(255))).slice(0, 4), "d9ff");
assert.equal(hex(writeWire("a".repeat(256))).slice(0, 6), "da0100");
assert.equal(hex(writeWire("a".repeat(65535))).slice(0, 6), "daffff");
assert.equal(hex(writeWire("a".repeat(65536))).slice(0, 10), "db00010000");
assert.equal(hex(writeWire(new Uint8Array(1))), "c40100");
assert.equal(hex(writeWire(new Uint8Array(256))).slice(0, 6), "c50100");
assert.equal(hex(writeWire(new Uint8Array(65536))).slice(0, 10), "c600010000");
assert.equal(hex(writeWire(new Array(15).fill(null))), `9f${"c0".repeat(15)}`);
assert.equal(hex(writeWire(new Array(65535).fill(null))).slice(0, 6), "dcffff");
assert.equal(hex(writeWire(new Array(65536).fill(null))).slice(0, 10), "dd00010000");
assert.throws(() => writeWire(-(2n ** 63n) - 1n), { message: "cannot write an integer wider than 64 bits." });
assert.throws(() => writeWire(undefined), /this kind of value/u);
assert.equal(hex(writeWire(-(2n ** 63n))), "d38000000000000000");
});
test("the wire reader's boundaries: the largest fixarray, the 32-bit binary and string, the safe integer edge", () => {
assert.deepEqual(readWire(fromHex(`9f${"c0".repeat(15)}`)), new Array(15).fill(null));
assert.deepEqual(readWire(fromHex("c600000001aa")), new Uint8Array([0xaa]));
assert.equal(readWire(fromHex("db0000000161")), "a");
assert.equal(readWire(fromHex("d3ffe0000000000001")), -(2 ** 53 - 1));
assert.equal(readWire(fromHex("d3ffe0000000000000")), -(2n ** 53n));
assert.equal(readWire(fromHex("cf0020000000000000")), 2n ** 53n);
});
test("a refusal says what was found: nil, a float, an integer that is not the slot's, a plain kind", () => {
assert.throws(() => int32.encode(undefined as unknown as number), { message: "expected an int32, found nil" });
assert.throws(() => string.decode(new WireFloat(1, true)), { message: "expected a string, found float" });
assert.throws(() => string.decode(5n), { message: "expected a string, found integer" });
assert.throws(() => int32.decode(new WireFloat(7, true)), { message: "expected an int32, found float" });
assert.throws(() => float64.decode("1"), { message: "expected a float64, found string" });
assert.throws(() => int64.encode(1.5), { message: "expected an int64, found a non-integer number" });
});
test("nullable names a value type Nullable<T> and anything else as itself", () => {
assert.equal(nullable(boolean).typeName, "System.Nullable<System.Boolean>");
assert.equal(nullable(int64).typeName, "System.Nullable<System.Int64>");
assert.equal(nullable(float32).typeName, "System.Nullable<System.Single>");
assert.equal(nullable(float64).typeName, "System.Nullable<System.Double>");
assert.equal(nullable(bytes).typeName, "System.Byte[]");
const unnamedValueType: Codec<number> = { valueType: true, encode: (value) => value, decode: (raw) => raw as number };
assert.equal(nullable(unnamedValueType).typeName, undefined);
assert.equal(lazy(() => int32).valueType, true);
});
test("the Int32 edges of an arm key: the largest is an arm, one past either end is outside", () => {
const edge = union<{ kind: "A" }>("U", { 2147483647: ["A", unit] });
assert.deepEqual(edge.decode(readWire(fromHex("92ce7fffffff90"))), { kind: "A" });
assert.deepEqual(edge.encode({ kind: "A" }), [2147483647, []]);
assert.throws(() => edge.decode(readWire(fromHex("92ce8000000090"))), { message: "Union U: the arm key is outside Int32 (declared keys: 2147483647)." });
assert.throws(() => edge.decode(readWire(fromHex("92d3ffffffff7fffffff90"))), { message: "Union U: the arm key is outside Int32 (declared keys: 2147483647)." });
assert.throws(() => edge.decode(readWire(fromHex("92d3ffffffff8000000090"))), { message: "Union U: unknown arm key -2147483648 (declared keys: 2147483647)." });
});
function envelopeOf(payload: Uint8Array): CultCacheEnvelope {
return { key: "surface", type: "tests.union_surface", payload: documentPayloadFromBytes(payload), storedAt: new Date().toISOString() };
}
function memoryStore(entries: CultCacheEnvelope[]): CacheBackingStore & { entries: CultCacheEnvelope[] } {
return {
entries,
async pullAll() { return entries.map((entry) => ({ ...entry })); },
async push(entry) { entries.push(entry); },
async delete() {},
};
}
test("arm keys span Int32 and are listed ascending; slot keys are non-negative Int32", () => {
const keys = union<{ kind: "High" | "Zero" | "Neg" | "Low" }>("U", {
2147483647: ["High", unit],
0: ["Zero", unit],
[-1]: ["Neg", unit],
[-2147483648]: ["Low", unit],
});
for (const [kind, key] of [["High", 2147483647], ["Zero", 0], ["Neg", -1], ["Low", -2147483648]] as const) {
const written = keys.encode({ kind });
assert.deepEqual(written, [key, []]);
assert.deepEqual(keys.decode(readWire(writeWire(written))), { kind });
}
assert.throws(() => keys.decode(readWire(fromHex("920590"))), { message: "Union U: unknown arm key 5 (declared keys: -2147483648, -1, 0, 2147483647)." });
assert.throws(() => union<{ kind: "A" }>("U", { [-2147483649]: ["A", unit] }), { message: "Union U declares the key -2147483649, which is outside Int32." });
assert.throws(() => union<{ kind: "A" }>("U", { 99999999999999999999: ["A", unit] } as never), { message: "Union U declares the key 100000000000000000000, which is outside Int32." });
assert.throws(() => slots<{ a: string }>({ 2147483647: ["a", string] }), { message: "A slot record declares the key 2147483647, which is above the slot key limit 65535." });
assert.throws(() => slots<{ a: string }>({ 2147483648: ["a", string] }), { message: "A slot record declares the key 2147483648, which is above the slot key limit 65535." });
assert.throws(() => slots<{ a: string }>({ [-1]: ["a", string] }), { message: 'A slot record declares the key "-1", which is not a non-negative integer.' });
});
// A slot record is a dense array of key+1 slots, as the reference writes it, so the key is capped: the largest accepted
// key writes a 65536-slot payload, one more is refused at definition.
test("a slot key is capped at 65535: the cap is the largest slot record, one past it is refused at definition", () => {
assert.equal(MAX_SLOT_KEY, 65535);
const edge = slots<{ a: string }>({ 65535: ["a", string] });
const written = edge.encode({ a: "x" }) as unknown[];
assert.equal(written.length, 65536);
assert.equal(written[65535], "x");
assert.equal(written.filter((slot) => slot === null).length, 65535);
assert.deepEqual(edge.decode(readWire(writeWire(written))), { a: "x" });
assert.throws(() => slots<{ a: string }>({ 65536: ["a", string] }), { message: "A slot record declares the key 65536, which is above the slot key limit 65535." });
});
test("the mixed-record vector is the bytes the reference writes for a record of every leaf kind", async () => {
const mixed = slots<{ id: string; i: number; l: number | bigint; f: number; d: number }>({
0: ["id", string], 1: ["i", int32], 2: ["l", int64], 3: ["f", float32], 4: ["d", float64],
});
const committed = new Uint8Array(await readFile(join(vectors, "..", "numeric-leaves", "mixed-record.msgpack")));
assert.equal(hex(writeWire(mixed.encode({ id: "n1", i: 7, l: 8, f: 1.5, d: 2.5 }))), hex(committed));
assert.deepEqual(mixed.decode(readWire(committed)), { id: "n1", i: 7, l: 8, f: 1.5, d: 2.5 });
});
// tests/vectors/slot-nil is written by the C# reference (SlotNilVectorTests): what it reads and rewrites for a nil, a str in a
// byte[] slot, and a record short of its last slots. A runtime accepts what it accepts and rewrites the same bytes.
test("slots read and rewrite exactly what the C# reference does for nil, a str in bytes, and short records", async () => {
const inner = slots<{ x: string }>({ 0: ["x", string, ""] });
const probe = slots<{ s: string; i: number; b: Uint8Array; r: { x: string }; l: string[] }>({
0: ["s", string, ""],
1: ["i", int32, 0],
2: ["b", bytes, () => new Uint8Array()],
3: ["r", inner, null as unknown as { x: string }],
4: ["l", list(string), () => []],
});
const rows = readWire(new Uint8Array(await readFile(join(vectors, "..", "slot-nil", "reads.msgpack")))) as [string, Uint8Array, boolean, Uint8Array][];
assert.equal(rows.length, 15);
let accepted = 0;
for (const [name, payload, wasAccepted, rewritten] of rows) {
if (wasAccepted) {
assert.equal(hex(writeWire(probe.encode(probe.decode(readWire(payload))))), hex(rewritten), name);
accepted += 1;
} else {
assert.throws(() => probe.decode(readWire(payload)), (error: Error) => !error.message.includes(hex(payload).slice(2)), name);
}
}
assert.equal(accepted, 13);
});
test("a nil reads as null and writes as nil in every reference slot, and a value slot refuses it", () => {
const record = slots<{ s: string; b: Uint8Array; i: number }>({ 0: ["s", string], 1: ["b", bytes], 2: ["i", int32, 5] });
const read = record.decode([null, null, 3]);
assert.deepEqual(read, { s: null, b: null, i: 3 });
assert.deepEqual(record.encode(read), [null, null, 3]);
assert.throws(() => record.encode({ s: "a", b: new Uint8Array(), i: null as unknown as number }), { message: "i (slot 2): expected an int32, found nil" });
assert.throws(() => record.decode(["a", new Uint8Array(), null]), { message: "i (slot 2): expected an int32, found nil" });
});
test("a bytes slot reads a MessagePack str as its bytes and writes bin, and refuses a str that was not valid UTF-8", () => {
const record = slots<{ b: Uint8Array }>({ 0: ["b", bytes] });
const read = record.decode(readWire(fromHex("91a3616263")));
assert.equal(hex(read.b), "616263");
assert.equal(hex(writeWire(record.encode(read))), "91c403616263");
assert.throws(() => record.decode(readWire(fromHex("91a2c328"))), { message: "b (slot 0): expected bytes, found a string that is not valid UTF-8" });
assert.throws(() => record.decode([5]), { message: "b (slot 0): expected bytes, found number" });
});

View file

@ -27,9 +27,19 @@ const counter = await mesh.leaseRawDocument({
recordKey: "counter:main",
});
counter.watch(record => renderCounter(decodeCultNetPayload(record)));
counter.watch(record => renderCounter(decodeCultNetPayload(record, counterDocument)));
```
A record's `payload` is a `DocumentPayload`, an opaque type over the same bytes, and
`decodeCultNetPayload(record, definition)` is how it is read: through the CultCache document
definition that says whether the bytes are a plain map, a codec's slot record or a
formatter's, then through the definition's schema. A plain MessagePack decode of a payload
does not compile, and `decodeCultNetPayload` takes no definition-less form. The type stops direct reads
and assignment; it does not stop runtime narrowing (`payload instanceof Uint8Array`) or a cast, which
would read the raw bytes. For the bytes themselves use `documentPayloadBytes`. The definition
comes from `@gamecult/cultcache-ts` (this package imports only its `./document` entry, which
holds no Node built-ins).
`invoke(...)` returns the correlated provider response. Routing, schema, and
envelope failures reject immediately with `CultMeshBrowserOperationError`;
callers can branch on its stable `status` and `code` without parsing prose or

View file

@ -40,7 +40,7 @@
"node": ">=20"
},
"dependencies": {
"@msgpack/msgpack": "^3.1.3",
"@gamecult/cultcache-ts": "^0.14.0",
"cultnet-ts": "^0.2.0"
},
"devDependencies": {

View file

@ -1,4 +1,6 @@
import { decode, encode } from "@msgpack/msgpack";
import { decodeDocumentPayload } from "@gamecult/cultcache-ts/document";
import type { AnyCultCacheDocumentDefinition, CultCacheDocumentValue } from "@gamecult/cultcache-ts";
import { decodeWireValue, encodeWireValue } from "cultnet-ts/wire";
import {
encodeCultNetMessageForWire,
parseCultNetMessage,
@ -387,7 +389,7 @@ export class CultMeshBrowserClient implements AsyncDisposable {
operation: options.operation,
payloadSchema: options.payloadSchema,
payloadEncoding: "messagepack-base64",
payload: bytesToBase64(encode(options.payload)),
payload: bytesToBase64(encodeWireValue(options.payload)),
sourceRuntimeId: this.runtimeId,
targetRuntimeId: this.identity.authorityRuntimeId,
};
@ -733,7 +735,7 @@ export class CultMeshBrowserClient implements AsyncDisposable {
private send(message: CultNetMessage): void {
const socket = this.#socket;
if (!socket || socket.readyState !== 1) throw new Error("CultMesh browser route is not connected.");
const payload = encode(encodeCultNetMessageForWire(message));
const payload = encodeWireValue(encodeCultNetMessageForWire(message));
if (payload.byteLength > this.#options.maxFrameBytes) {
throw new Error(`CultMesh message exceeds the ${this.#options.maxFrameBytes}-byte limit.`);
}
@ -760,11 +762,19 @@ export class CultMeshBrowserClient implements AsyncDisposable {
}
}
export function decodeCultNetPayload<T>(record: CultNetRawDocumentRecord): T {
/**
* Reads a document record's payload through the definition that says how its bytes are laid out: a codec, a formatter or
* a plain map, then the definition's schema. The payload is opaque (`DocumentPayload`), so a plain msgpack decode of it
* does not compile; the definition is required.
*/
export function decodeCultNetPayload<TDefinition extends AnyCultCacheDocumentDefinition>(
record: CultNetRawDocumentRecord,
definition: TDefinition,
): CultCacheDocumentValue<TDefinition> {
if (record.payloadEncoding !== "messagepack") {
throw new Error(`Unsupported CultNet document payload '${record.payloadEncoding}'.`);
}
return decode(record.payload) as T;
return decodeDocumentPayload(definition, record.payload);
}
function recordMatchesSchema(record: CultNetRawDocumentRecord, requestedSchema: string): boolean {
@ -777,7 +787,7 @@ export function decodeCultNetOperationPayload<T>(response: CultNetOperationRespo
if (response.payloadEncoding !== "messagepack-base64") {
throw new Error(`Unsupported CultNet operation payload '${response.payloadEncoding ?? "unspecified"}'.`);
}
return decode(base64ToBytes(response.payload)) as T;
return decodeWireValue(base64ToBytes(response.payload)) as T;
}
function requireWebSocketEndpoint(value: string, field: string): URL {
@ -794,7 +804,7 @@ function requireWebSocketEndpoint(value: string, field: string): URL {
}
function sendSocketMessage(socket: CultMeshBrowserSocket, message: CultNetMessage, maxFrameBytes: number): void {
const payload = encode(encodeCultNetMessageForWire(message));
const payload = encodeWireValue(encodeCultNetMessageForWire(message));
if (payload.byteLength > maxFrameBytes) {
throw new Error(`CultNet message exceeds the ${maxFrameBytes}-byte limit.`);
}
@ -815,7 +825,7 @@ async function decodeSocketMessage(event: MessageEvent, maxFrameBytes: number):
if (payload.byteLength > maxFrameBytes) {
throw new Error(`CultNet frame exceeds the ${maxFrameBytes}-byte limit.`);
}
return parseCultNetMessage(decode(payload));
return parseCultNetMessage(decodeWireValue(payload));
}
async function selectOdinRoute(

View file

@ -1,7 +1,8 @@
import assert from "node:assert/strict";
import test from "node:test";
import { decode, encode } from "@msgpack/msgpack";
import { codecs, defineDocumentType, documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import { decodeWireValue, encodeWireValue } from "cultnet-ts/wire";
import {
encodeCultNetMessageForWire,
parseCultNetMessage,
@ -120,6 +121,13 @@ test("Odin rendezvous rejects a poisoned route without vetoing a certified fallb
assert.equal(selected.endpoint, signed.route.endpoint);
});
// The counter document the fake provider serves: a plain map.
const counterDocument = defineDocumentType({
type: "sample.counter_state",
schemaId: "sample.counter_state.v1",
schema: { parse: (value: unknown) => value as { count: number } },
});
test("browser client leases provider state, invokes an operation, and follows route replacement", async () => {
const first = new FakeProvider("ws://127.0.0.1:4101/mesh", 4);
const second = new FakeProvider("ws://127.0.0.1:4102/mesh", 9);
@ -139,11 +147,11 @@ test("browser client leases provider state, invokes an operation, and follows ro
schemaId: "sample.counter_state.v1",
recordKey: "counter:main",
});
assert.deepEqual(decodeCultNetPayload<{ count: number }>(lease.current!).count, 4);
assert.deepEqual(decodeCultNetPayload(lease.current!, counterDocument).count, 4);
const observed: number[] = [];
const unwatch = lease.watch(record => {
if (record) observed.push(decodeCultNetPayload<{ count: number }>(record).count);
if (record) observed.push(decodeCultNetPayload(record, counterDocument).count);
});
const response = await client.invoke({
serviceId: "sample.counter",
@ -158,7 +166,7 @@ test("browser client leases provider state, invokes an operation, and follows ro
route = second.route;
await client.refreshRoute();
assert.equal(client.state, "connected");
assert.deepEqual(decodeCultNetPayload<{ count: number }>(lease.current!).count, 9);
assert.deepEqual(decodeCultNetPayload(lease.current!, counterDocument).count, 9);
assert.deepEqual(observed, [4, 6, 9]);
assert.equal(first.activeSubscriptionCount, 0);
assert.equal(second.activeSubscriptionCount, 1);
@ -372,7 +380,7 @@ test("browser client replays one idempotent operation after route loss without d
const open = () => {
let socket: FakeSocket;
socket = new FakeSocket(wire => {
const message = parseCultNetMessage(decode(wire));
const message = parseCultNetMessage(decodeWireValue(wire));
if (message.schemaVersion === "cultmesh.session_open.v2") {
socket.deliver({
schemaVersion: "cultmesh.session_accepted.v2",
@ -400,7 +408,7 @@ test("browser client replays one idempotent operation after route loss without d
status: "accepted",
payloadSchema: "sample.increment_receipt.v1",
payloadEncoding: "messagepack-base64",
payload: Buffer.from(encode({ count: effectCount })).toString("base64"),
payload: Buffer.from(encodeWireValue({ count: effectCount })).toString("base64"),
sourceRuntimeId: "sample.counter-daemon",
};
results.set(message.messageId, response!);
@ -600,7 +608,7 @@ class FakeProvider {
}
private receive(socket: FakeSocket, wire: Uint8Array): void {
const message = parseCultNetMessage(decode(wire));
const message = parseCultNetMessage(decodeWireValue(wire));
switch (message.schemaVersion) {
case "cultmesh.session_open.v2":
this.acceptSession(socket, message);
@ -654,7 +662,7 @@ class FakeProvider {
status: "invalid",
payloadSchema: "gamecult.cultnet.operation_failure.v1",
payloadEncoding: "messagepack-base64",
payload: bytesToBase64(encode({
payload: bytesToBase64(encodeWireValue({
code: "request-schema-mismatch",
message: "Expected payload schema 'sample.increment.v1'.",
})),
@ -663,7 +671,7 @@ class FakeProvider {
});
return;
}
const payload = decode(base64ToBytes(message.payload)) as { amount: number };
const payload = decodeWireValue(base64ToBytes(message.payload)) as { amount: number };
this.#count += payload.amount;
for (const [peer, subscriptions] of this.#subscriptions) {
for (const subscriptionId of subscriptions.keys()) {
@ -684,7 +692,7 @@ class FakeProvider {
status: "accepted",
payloadSchema: "sample.increment_receipt.v1",
payloadEncoding: "messagepack-base64",
payload: bytesToBase64(encode({ count: this.#count })),
payload: bytesToBase64(encodeWireValue({ count: this.#count })),
sourceRuntimeId: this.#responseRuntimeId,
});
}
@ -698,7 +706,7 @@ class FakeProvider {
recordKey: "counter:main",
storedAt: "2026-08-17T00:00:00Z",
payloadEncoding: "messagepack",
payload: encode({ count: this.#count }),
payload: documentPayloadFromBytes(encodeWireValue({ count: this.#count })),
sourceRuntimeId: this.#runtimeId,
};
}
@ -716,7 +724,7 @@ class FakeOdin {
open(): FakeSocket {
const socket = new FakeSocket(wire => {
const message = parseCultNetMessage(decode(wire));
const message = parseCultNetMessage(decodeWireValue(wire));
if (message.schemaVersion !== "cultmesh.verse_catalog_request.v0") return;
socket.deliver({
schemaVersion: "cultmesh.verse_catalog_response.v0",
@ -785,7 +793,7 @@ function routeForCatalog(route: CultMeshBrowserRoute): NonNullable<
function handshakeOnlySocket(route: CultMeshBrowserRoute): FakeSocket {
let socket: FakeSocket;
socket = new FakeSocket(wire => {
const message = parseCultNetMessage(decode(wire));
const message = parseCultNetMessage(decodeWireValue(wire));
if (message.schemaVersion !== "cultmesh.session_open.v2") return;
socket.deliver({
schemaVersion: "cultmesh.session_accepted.v2",
@ -850,7 +858,7 @@ class FakeSocket implements CultMeshBrowserSocket {
}
deliver(message: CultNetMessage): void {
const wire = encode(encodeCultNetMessageForWire(message));
const wire = encodeWireValue(encodeCultNetMessageForWire(message));
queueMicrotask(() => this.onmessage?.({ data: wire.slice().buffer } as MessageEvent));
}
}
@ -899,7 +907,7 @@ async function createSignedRoute(endpoint: string): Promise<{
function signedHandshakeSocket(route: CultMeshBrowserRoute, providerPrivate: CryptoKey): FakeSocket {
let socket: FakeSocket;
socket = new FakeSocket(wire => {
const message = parseCultNetMessage(decode(wire));
const message = parseCultNetMessage(decodeWireValue(wire));
if (message.schemaVersion !== "cultmesh.session_open.v2") return;
void crypto.subtle.sign(
{ name: "ECDSA", hash: "SHA-256" },
@ -975,3 +983,26 @@ function base64ToBytes(value: string): Uint8Array {
const binary = atob(value);
return Uint8Array.from(binary, character => character.charCodeAt(0));
}
test("decodeCultNetPayload reads a payload through the definition it is given: plain map, codec or formatter", () => {
const record = (bytes: Uint8Array) => ({
schemaId: "t.v0", recordKey: "k", payloadEncoding: "messagepack", payload: documentPayloadFromBytes(bytes),
}) as unknown as CultNetRawDocumentRecord;
const schema = { parse: (value: unknown) => value as { count: number } };
const plain = defineDocumentType({ type: "t.plain", schemaId: "t.plain.v0", schema });
const codec = defineDocumentType({
type: "t.codec", schemaId: "t.codec.v0", schema, codec: codecs.slots<{ count: number }>({ 0: ["count", codecs.int32] }, "T.Count"),
});
const formatted = defineDocumentType({
type: "t.formatted", schemaId: "t.formatted.v0", schema,
formatter: { encode: value => encodeWireValue([value.count]), decode: payload => ({ count: (decodeWireValue(payload) as number[])[0]! }) },
});
assert.deepEqual(decodeCultNetPayload(record(encodeWireValue({ count: 7 })), plain), { count: 7 });
assert.deepEqual(decodeCultNetPayload(record(encodeWireValue([7])), codec), { count: 7 });
assert.deepEqual(decodeCultNetPayload(record(encodeWireValue([7])), formatted), { count: 7 });
});
test("decodeCultNetPayload refuses a payload that is not messagepack, naming the encoding and not the payload", () => {
const record = { schemaId: "t.v0", recordKey: "k", payloadEncoding: "json", payload: documentPayloadFromBytes(encodeWireValue("secret-value")) } as unknown as CultNetRawDocumentRecord;
assert.throws(() => decodeCultNetPayload(record, counterDocument), (error: Error) => error.message === "Unsupported CultNet document payload 'json'.");
});

View file

@ -0,0 +1,28 @@
import assert from "node:assert/strict";
import test from "node:test";
import { documentPayloadBytes, documentPayloadFromBytes, type DocumentPayload } from "@gamecult/cultcache-ts";
import { decodeWireValue } from "cultnet-ts/wire";
import type { CultNetRawDocumentRecord } from "cultnet-ts/contracts";
import { decodeCultNetPayload } from "../src/index.js";
// The rule: a document payload is read only through its definition. A CultNet record's payload is the opaque
// DocumentPayload, so tsc compiles this file only if each line carrying an expect-error directive is an error. The function is
// never called.
export function refusedReaders(record: CultNetRawDocumentRecord): void {
// @ts-expect-error the wire value decoder is for frames and values, not for a document's payload
decodeWireValue(record.payload);
// @ts-expect-error a record's payload is not assignable to bytes
const bytes: Uint8Array = record.payload;
// @ts-expect-error the browser reader needs the definition that says how the bytes are laid out
decodeCultNetPayload(record);
// @ts-expect-error bytes are not a payload until they enter by name
const forged: DocumentPayload = new Uint8Array();
void [bytes, forged];
}
test("a record's payload is the same bytes it entered with by name", () => {
const bytes = Uint8Array.from([0x90]);
assert.equal(documentPayloadBytes(documentPayloadFromBytes(bytes)), bytes);
});

View file

@ -411,9 +411,11 @@ payload binary normalization, and MessagePack decode behind the same
`CultMeshDocumentHandle` shape as local documents. The helper prefers an exact
schema id match and falls back to the requested record key, so runtimes that
alias the same logical document through different generated schema names can
still read the publication with one CultMesh call. Pass a CultCache document
definition instead of a raw schema id when you want a typed handle whose
`latest()` result is parsed through that definition.
still read the publication with one CultMesh call. The handle takes a CultCache
document definition, and its `latest()` result is read and parsed through that
definition: a payload is a `DocumentPayload`, which only
`decodeDocumentPayload(definition, payload)` reads, so no document is read
without the definition that says how its bytes are laid out.
Use `node.syncDocumentFromPeerSnapshot(...)` or
`CultMesh.syncDocumentFromPeerSnapshot(...)` when a runtime should hydrate its
@ -424,8 +426,8 @@ keep using `node.document(aliasDefinition, key)` and explicitly select
`authoritativeWriter()`, `predictionWriter()`, or `observe()`
without re-threading the RUDP peer.
The same typed-definition overload is available for
`CultMesh.documentFromStore(...)` and `CultMesh.documentFromSingleFile(...)`, so
`CultMesh.documentFromStore(...)` and `CultMesh.documentFromSingleFile(...)` take
the same definition, so
local schema publications can be read as typed reactive documents without
manually pulling records, decoding MessagePack, or parsing the payload.
When the caller wants source selection to be configuration, use

View file

@ -43,7 +43,6 @@
"node": ">=20"
},
"dependencies": {
"@msgpack/msgpack": "^3.1.2",
"@gamecult/cultcache-ts": "^0.14.0",
"cultnet-ts": "^0.2.0",
"koffi": "3.3.0"

View file

@ -1,5 +1,5 @@
import { createSocket, type Socket } from "node:dgram";
import { decode, encode } from "@msgpack/msgpack";
import { decodeWireValue, encodeWireValue } from "cultnet-ts/wire";
import {
CultCache,
@ -8,6 +8,8 @@ import {
type CacheBackingStore,
type CultCacheEnvelope,
type CultCacheDocumentValue,
type DocumentPayload,
decodeDocumentPayload,
} from "@gamecult/cultcache-ts";
import {
CultNetDocumentRegistry,
@ -126,7 +128,7 @@ export type CultMeshDocumentPublicationSource =
};
export interface CultMeshPublicationDocumentBinding {
readonly schema: string | AnyCultCacheDocumentDefinition;
readonly schema: AnyCultCacheDocumentDefinition;
readonly recordKey: string;
readonly documentId?: string;
readonly sourceId?: string;
@ -2590,54 +2592,29 @@ export function cultMeshDocumentFromCache<TDefinition extends AnyCultCacheDocume
);
}
export function cultMeshDocumentFromStore(
store: CacheBackingStore,
schemaId: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<unknown>;
export function cultMeshDocumentFromStore<TDefinition extends AnyCultCacheDocumentDefinition>(
store: CacheBackingStore,
definition: TDefinition,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
export function cultMeshDocumentFromStore(
store: CacheBackingStore,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
} = {},
): CultMeshDocumentHandle<unknown> {
const schema = typeof schemaOrDefinition === "string"
? { schemaId: schemaOrDefinition }
: cultMeshSchemaFromDefinition(schemaOrDefinition);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
const schema = cultMeshSchemaFromDefinition(definition);
const resolvedSchemaId = schema.schemaId ?? schema.type;
if (!resolvedSchemaId) {
throw new Error("CultMesh store document requires a schema id or document type.");
}
if (typeof schemaOrDefinition === "string") {
requireNonEmpty(schemaOrDefinition, "schemaId");
}
const documentId = options.documentId ?? resolvedSchemaId;
const read = async () => {
const read = async (): Promise<CultCacheDocumentValue<TDefinition>> => {
const records = await store.pullAll();
const record = resolveCultMeshStoreDocumentRecord(records, schema, schemaOrDefinition);
const record = resolveCultMeshStoreDocumentRecord(records, schema, definition);
if (!record) {
throw new Error(`CultMesh store document '${documentId}' did not contain schema ${resolvedSchemaId}.`);
}
return parseCultMeshDocumentValue(schemaOrDefinition, decode(record.payload));
return decodeDocumentPayload(definition, record.payload);
};
return cultMeshDocument(
@ -2654,77 +2631,28 @@ export function cultMeshDocumentFromStore(
);
}
export function cultMeshDocumentFromSingleFile(
path: string,
schemaId: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<unknown>;
export function cultMeshDocumentFromSingleFile<TDefinition extends AnyCultCacheDocumentDefinition>(
path: string,
definition: TDefinition,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
export function cultMeshDocumentFromSingleFile(
path: string,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
} = {},
): CultMeshDocumentHandle<unknown> {
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
requireNonEmpty(path, "path");
const store = new SingleFileMessagePackBackingStore(path);
const resolvedOptions = {
return cultMeshDocumentFromStore(store, definition, {
...options,
routeHint: options.routeHint ?? cultMeshRouteHint("shared-memory", path),
};
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromStore(store, schemaOrDefinition, resolvedOptions)
: cultMeshDocumentFromStore(store, schemaOrDefinition, resolvedOptions);
});
}
export function cultMeshDocumentFromPeerSnapshot(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
schemaId: string,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<unknown>;
export function cultMeshDocumentFromPeerSnapshot<TDefinition extends AnyCultCacheDocumentDefinition>(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
definition: TDefinition,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
export function cultMeshDocumentFromPeerSnapshot(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
@ -2733,44 +2661,34 @@ export function cultMeshDocumentFromPeerSnapshot(
pollMs?: number;
messageIdPrefix?: string;
} = {},
): CultMeshDocumentHandle<unknown> {
const schema = typeof schemaOrDefinition === "string"
? { schemaId: schemaOrDefinition }
: cultMeshSchemaFromDefinition(schemaOrDefinition);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
const schema = cultMeshSchemaFromDefinition(definition);
const resolvedSchemaId = schema.schemaId ?? schema.type;
if (!resolvedSchemaId) {
throw new Error("CultMesh peer snapshot document requires a schema id or document type.");
}
if (typeof schemaOrDefinition === "string") {
requireNonEmpty(schemaOrDefinition, "schemaId");
}
requireNonEmpty(recordKey, "recordKey");
const documentId = options.documentId ?? recordKey;
const read = async () =>
parseCultMeshDocumentValue(
schemaOrDefinition,
decodeCultNetRawDocumentPayload(
await requestCultNetRawSnapshotDocument(
peer,
resolvedSchemaId,
recordKey,
{
timeoutMs: options.timeoutMs,
messageIdPrefix: options.messageIdPrefix ?? documentId,
accepts: typeof schemaOrDefinition === "string"
? undefined
: record => {
try {
schemaOrDefinition.schema.parse(decodeCultNetRawDocumentPayload(record));
return true;
} catch {
return false;
}
},
},
),
),
const read = async (): Promise<CultCacheDocumentValue<TDefinition>> => {
const record = await requestCultNetRawSnapshotDocument(
peer,
resolvedSchemaId,
recordKey,
{
timeoutMs: options.timeoutMs,
messageIdPrefix: options.messageIdPrefix ?? documentId,
accepts: candidate => {
try {
decodeDocumentPayload(definition, candidate.payload);
return true;
} catch {
return false;
}
},
},
);
return decodeDocumentPayload(definition, record.payload);
};
return cultMeshDocument(
documentId,
@ -2786,36 +2704,10 @@ export function cultMeshDocumentFromPeerSnapshot(
);
}
export function cultMeshDocumentFromPublication(
source: CultMeshDocumentPublicationSource,
schemaId: string,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<unknown>;
export function cultMeshDocumentFromPublication<TDefinition extends AnyCultCacheDocumentDefinition>(
source: CultMeshDocumentPublicationSource,
definition: TDefinition,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
export function cultMeshDocumentFromPublication(
source: CultMeshDocumentPublicationSource,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
@ -2824,7 +2716,7 @@ export function cultMeshDocumentFromPublication(
pollMs?: number;
messageIdPrefix?: string;
} = {},
): CultMeshDocumentHandle<unknown> {
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
if (!source) {
throw new Error("CultMesh publication source is required.");
}
@ -2832,14 +2724,12 @@ export function cultMeshDocumentFromPublication(
switch (source.kind) {
case "live-publication": {
const schema = typeof schemaOrDefinition === "string"
? { schemaId: schemaOrDefinition }
: cultMeshSchemaFromDefinition(schemaOrDefinition);
const schema = cultMeshSchemaFromDefinition(definition);
const schemaId = schema.schemaId ?? schema.type;
if (!schemaId) {
throw new Error("CultMesh live publication document requires a schema id or document type.");
}
const parse = (value: unknown) => parseCultMeshDocumentValue(schemaOrDefinition, value);
const parse = (value: unknown) => definition.schema.parse(value) as CultCacheDocumentValue<TDefinition>;
return cultMeshDocument(
options.documentId ?? recordKey,
schema,
@ -2859,51 +2749,28 @@ export function cultMeshDocumentFromPublication(
);
}
case "single-file":
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromSingleFile(source.path, schemaOrDefinition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
})
: cultMeshDocumentFromSingleFile(source.path, schemaOrDefinition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
});
return cultMeshDocumentFromSingleFile(source.path, definition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
});
case "store":
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromStore(source.store, schemaOrDefinition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
})
: cultMeshDocumentFromStore(source.store, schemaOrDefinition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
});
return cultMeshDocumentFromStore(source.store, definition, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint,
sourceId: options.sourceId ?? recordKey,
pollMs: options.pollMs,
});
case "peer-snapshot":
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromPeerSnapshot(source.peer, schemaOrDefinition, recordKey, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint ?? cultMeshRouteHint("network", source.endpoint ?? "CultNet snapshot"),
sourceId: options.sourceId ?? recordKey,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
})
: cultMeshDocumentFromPeerSnapshot(source.peer, schemaOrDefinition, recordKey, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint ?? cultMeshRouteHint("network", source.endpoint ?? "CultNet snapshot"),
sourceId: options.sourceId ?? recordKey,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
});
return cultMeshDocumentFromPeerSnapshot(source.peer, definition, recordKey, {
documentId: options.documentId ?? recordKey,
routeHint: options.routeHint ?? cultMeshRouteHint("network", source.endpoint ?? "CultNet snapshot"),
sourceId: options.sourceId ?? recordKey,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
});
default:
throw new Error(`Unsupported CultMesh publication source kind '${(source as { kind?: string }).kind}'.`);
}
@ -2965,11 +2832,6 @@ export async function cultMeshSyncDocumentsFromPublication(
if (!binding) {
throw new Error("CultMesh publication binding is required.");
}
if (typeof binding.schema === "string") {
throw new Error(
`CultMesh synced publication binding '${binding.recordKey}' requires a document definition, not only a schema id.`,
);
}
const resolvedSource = binding.source ??
(typeof source === "function" ? source(binding) : source);
await node.syncDocumentFromPublication(
@ -3027,7 +2889,7 @@ export function cultMeshDocuments(
}
export function cultMeshPublicationDocument(
schema: string | AnyCultCacheDocumentDefinition,
schema: AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
@ -3069,33 +2931,19 @@ export function cultMeshDocumentsFromPublication(
}
const resolvedSource = binding.source ??
(typeof source === "function" ? source(binding) : source);
return typeof binding.schema === "string"
? cultMeshDocumentFromPublication(
resolvedSource,
binding.schema,
binding.recordKey,
{
documentId: binding.documentId ?? binding.recordKey,
sourceId: binding.sourceId ?? binding.documentId ?? binding.recordKey,
routeHint: options.routeHint,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
},
)
: cultMeshDocumentFromPublication(
resolvedSource,
binding.schema,
binding.recordKey,
{
documentId: binding.documentId ?? binding.recordKey,
sourceId: binding.sourceId ?? binding.documentId ?? binding.recordKey,
routeHint: options.routeHint,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
},
);
return cultMeshDocumentFromPublication(
resolvedSource,
binding.schema,
binding.recordKey,
{
documentId: binding.documentId ?? binding.recordKey,
sourceId: binding.sourceId ?? binding.documentId ?? binding.recordKey,
routeHint: options.routeHint,
timeoutMs: options.timeoutMs,
pollMs: options.pollMs,
messageIdPrefix: options.messageIdPrefix,
},
);
}),
);
}
@ -4106,7 +3954,7 @@ export class CultMeshNode {
messageIdPrefix: options.messageIdPrefix ?? key,
accepts: record => {
try {
definition.schema.parse(decodeCultNetRawDocumentPayload(record));
decodeDocumentPayload(definition, record.payload);
return true;
} catch {
return false;
@ -4257,11 +4105,16 @@ export interface CultMeshPeerCard {
authorityLeaseId?: string;
}
export interface CultMeshRudpDocumentPut {
/**
* A document put as `onDocumentPutRaw` receives it. `decoded` says which read the payload is ready for: `true` when the
* registry held a definition for `schemaId` and `payload` is the document read through it; `false` when it did not, and
* `payload` is the opaque `DocumentPayload`, untouched, for a handler that holds the definition itself to read with
* `decodeDocumentPayload`. Narrowing on `decoded` needs no cast in either branch.
*/
export type CultMeshRudpDocumentPut = {
schemaId: string;
recordKey: string;
storedAt: string;
payload: unknown;
sourceRuntimeId: string | null;
sourceAgentId: string | null;
sourceRole: string | null;
@ -4271,7 +4124,10 @@ export interface CultMeshRudpDocumentPut {
family: string;
port: number;
};
}
} & (
| { readonly decoded: true; readonly payload: unknown }
| { readonly decoded: false; readonly payload: DocumentPayload }
);
export interface CultMeshRudpDocumentReceipt<TDefinition extends AnyCultCacheDocumentDefinition = AnyCultCacheDocumentDefinition> {
binding: CultNetDocumentBinding<TDefinition>;
@ -5035,106 +4891,36 @@ export class CultMesh {
return cultMeshDocumentFromCache(cache, definition, key, options);
}
public static documentFromStore(
store: CacheBackingStore,
schemaId: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<unknown>;
public static documentFromStore<TDefinition extends AnyCultCacheDocumentDefinition>(
store: CacheBackingStore,
definition: TDefinition,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
public static documentFromStore(
store: CacheBackingStore,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
} = {},
): CultMeshDocumentHandle<unknown> {
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromStore(store, schemaOrDefinition, options)
: cultMeshDocumentFromStore(store, schemaOrDefinition, options);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
return cultMeshDocumentFromStore(store, definition, options);
}
public static documentFromSingleFile(
path: string,
schemaId: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<unknown>;
public static documentFromSingleFile<TDefinition extends AnyCultCacheDocumentDefinition>(
path: string,
definition: TDefinition,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
public static documentFromSingleFile(
path: string,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
pollMs?: number;
} = {},
): CultMeshDocumentHandle<unknown> {
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromSingleFile(path, schemaOrDefinition, options)
: cultMeshDocumentFromSingleFile(path, schemaOrDefinition, options);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
return cultMeshDocumentFromSingleFile(path, definition, options);
}
public static documentFromPeerSnapshot(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
schemaId: string,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<unknown>;
public static documentFromPeerSnapshot<TDefinition extends AnyCultCacheDocumentDefinition>(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
definition: TDefinition,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
public static documentFromPeerSnapshot(
peer: CultNetPeer | (() => CultNetPeer | Promise<CultNetPeer>),
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
@ -5143,42 +4929,14 @@ export class CultMesh {
pollMs?: number;
messageIdPrefix?: string;
} = {},
): CultMeshDocumentHandle<unknown> {
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromPeerSnapshot(peer, schemaOrDefinition, recordKey, options)
: cultMeshDocumentFromPeerSnapshot(peer, schemaOrDefinition, recordKey, options);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
return cultMeshDocumentFromPeerSnapshot(peer, definition, recordKey, options);
}
public static documentFromPublication(
source: CultMeshDocumentPublicationSource,
schemaId: string,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<unknown>;
public static documentFromPublication<TDefinition extends AnyCultCacheDocumentDefinition>(
source: CultMeshDocumentPublicationSource,
definition: TDefinition,
recordKey: string,
options?: {
documentId?: string;
routeHint?: CultMeshRouteHint;
sourceId?: string;
timeoutMs?: number;
pollMs?: number;
messageIdPrefix?: string;
},
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>>;
public static documentFromPublication(
source: CultMeshDocumentPublicationSource,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
routeHint?: CultMeshRouteHint;
@ -5187,10 +4945,8 @@ export class CultMesh {
pollMs?: number;
messageIdPrefix?: string;
} = {},
): CultMeshDocumentHandle<unknown> {
return typeof schemaOrDefinition === "string"
? cultMeshDocumentFromPublication(source, schemaOrDefinition, recordKey, options)
: cultMeshDocumentFromPublication(source, schemaOrDefinition, recordKey, options);
): CultMeshDocumentHandle<CultCacheDocumentValue<TDefinition>> {
return cultMeshDocumentFromPublication(source, definition, recordKey, options);
}
public static globalDocumentFromCache<TDefinition extends AnyCultCacheDocumentDefinition>(
@ -5252,7 +5008,7 @@ export class CultMesh {
}
public static publicationDocument(
schema: string | AnyCultCacheDocumentDefinition,
schema: AnyCultCacheDocumentDefinition,
recordKey: string,
options: {
documentId?: string;
@ -5955,7 +5711,7 @@ export class CultMesh {
payload: Uint8Array,
): Promise<void> {
if (record.closed) return;
const message = parseCultNetMessage(decode(payload), wireContract);
const message = parseCultNetMessage(decodeWireValue(payload), wireContract);
switch (message.schemaVersion) {
case "cultnet.snapshot_request.v0": {
if (!options.getCache) {
@ -5998,7 +5754,7 @@ export class CultMesh {
sendSchemaMessage(record, { schemaVersion: "cultnet.error.v0", error: unservable.message });
return;
}
const receipt = await options.onDocumentPutRaw(normalizeRudpDocumentPut(message, record.remote));
const receipt = await options.onDocumentPutRaw(normalizeRudpDocumentPut(message, record.remote, options.documents));
if (receipt) {
try {
sendSchemaMessage(record, receiptMessage(receipt, message.messageId));
@ -6136,7 +5892,7 @@ export class CultMesh {
/** The one reply encoder, shared by sends and by put admission. */
function encodeSchemaMessage(message: CultNetMessage): Uint8Array {
return encode(encodeCultNetMessageForWire(message, wireContract));
return encodeWireValue(encodeCultNetMessageForWire(message, wireContract));
}
/** Builds the receipt `onDocumentPutRaw` returned, under the put's message id by default. */
@ -6329,9 +6085,9 @@ export class CultMesh {
const onMessage = (message: CultNetMessage): void => {
if (message.schemaVersion !== "cultnet.document_put_raw.v0" || message.messageId !== messageId || message.document.schemaId !== receiptBinding.definition.schemaId) return;
try {
const decoded = decode(new Uint8Array(message.document.payload));
const decoded = decodeDocumentPayload(receiptBinding.definition, message.document.payload);
cleanup();
resolve(receiptBinding.definition.schema.parse(decoded) as CultCacheDocumentValue<TReceiptDefinition>);
resolve(decoded);
} catch (error) { cleanup(); reject(error); }
};
const cleanup = (): void => { clearTimeout(timeout); peer.off("message", onMessage); };
@ -6595,18 +6351,14 @@ function inferCultMeshSchemaName(schemaId: string | undefined): string | undefin
function resolveCultMeshStoreDocumentRecord(
records: readonly CultCacheEnvelope[],
schema: CultMeshDocumentSchemaDescriptor,
schemaOrDefinition: string | AnyCultCacheDocumentDefinition,
definition: AnyCultCacheDocumentDefinition,
): CultCacheEnvelope | undefined {
const schemaId = schema.schemaId ?? schema.type;
return (
records.find(candidate => schemaId !== undefined && candidate.schemaId === schemaId) ??
records.find(candidate =>
cultMeshEnvelopeMatchesSchema(candidate, schema) &&
(typeof schemaOrDefinition === "string" ||
cultMeshEnvelopeParsesAsDefinition(candidate, schemaOrDefinition))) ??
(typeof schemaOrDefinition === "string"
? undefined
: records.find(candidate => cultMeshEnvelopeParsesAsDefinition(candidate, schemaOrDefinition)))
cultMeshEnvelopeMatchesSchema(candidate, schema) && cultMeshEnvelopeParsesAsDefinition(candidate, definition)) ??
records.find(candidate => cultMeshEnvelopeParsesAsDefinition(candidate, definition))
);
}
@ -6638,7 +6390,7 @@ function cultMeshEnvelopeParsesAsDefinition(
definition: AnyCultCacheDocumentDefinition,
): boolean {
try {
definition.schema.parse(decode(envelope.payload));
decodeDocumentPayload(definition, envelope.payload);
return true;
} catch {
return false;
@ -6661,6 +6413,7 @@ function requireNonEmpty(value: string, name: string): void {
function normalizeRudpDocumentPut(
message: CultNetMessage,
remote: { address: string; family: string; port: number },
documents: CultNetDocumentRegistry,
): CultMeshRudpDocumentPut {
if (message.schemaVersion !== "cultnet.document_put_raw.v0") {
throw new Error(`Expected cultnet.document_put_raw.v0, received ${message.schemaVersion}.`);
@ -6672,16 +6425,19 @@ function normalizeRudpDocumentPut(
if (document.payloadEncoding !== "messagepack") {
throw new Error(`Unsupported raw payload encoding ${document.payloadEncoding}.`);
}
const definition = documents.getBySchemaId(document.schemaId)?.definition;
return {
schemaId: document.schemaId,
recordKey: document.recordKey,
storedAt: document.storedAt ?? new Date().toISOString(),
payload: decode(document.payload),
sourceRuntimeId: document.sourceRuntimeId ?? null,
sourceAgentId: document.sourceAgentId ?? null,
sourceRole: document.sourceRole ?? null,
tags: Array.isArray(document.tags) ? document.tags : [],
remote,
...(definition
? { decoded: true, payload: decodeDocumentPayload(definition, document.payload) }
: { decoded: false, payload: document.payload }),
};
}
@ -6774,36 +6530,10 @@ async function resolveCultNetPeer(
: peerOrProvider;
}
function decodeCultNetRawDocumentPayload(document: CultNetRawDocumentRecord): unknown {
if (document.payloadEncoding !== "messagepack") {
throw new Error(`Unsupported CultNet raw document payload encoding ${document.payloadEncoding}.`);
}
return decode(toUint8Array(document.payload));
}
function parseCultMeshDocumentValue<TDefinition extends AnyCultCacheDocumentDefinition>(
schemaOrDefinition: string | TDefinition,
value: unknown,
): unknown | CultCacheDocumentValue<TDefinition> {
return typeof schemaOrDefinition === "string"
? value
: schemaOrDefinition.schema.parse(value);
}
function isCultNetErrorMessage(message: CultNetMessage): message is CultNetErrorMessage {
return message.schemaVersion === "cultnet.error.v0";
}
function toUint8Array(value: unknown): Uint8Array {
if (value instanceof Uint8Array) {
return value;
}
if (Array.isArray(value)) {
return Uint8Array.from(value);
}
throw new Error("CultNet raw document payload was not binary.");
}
async function bindRudpSocket(host: string, options: CultMeshRudpSocketOptions): Promise<Socket> {
const socket = createSocket(host.includes(":") ? "udp6" : "udp4");
const port = options.bindPort ?? 0;

View file

@ -1,6 +1,7 @@
import { randomUUID } from "node:crypto";
import { createSocket, type Socket } from "node:dgram";
import { encode } from "@msgpack/msgpack";
import { documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import { encodeWireValue } from "cultnet-ts/wire";
import {
CultNetPeer,
CultNetRudpSocketTransportConnection,
@ -171,7 +172,9 @@ class RudpProviderConnection implements CultMeshProviderConnection {
recordKey: publication.recordKey,
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack",
payload: encode(publication.value),
// The one plain writer of a document payload: a publication carries no definition. The typed owner of this write is
// cultlib-gaps:follow_up:document-payload-encode-owner.
payload: documentPayloadFromBytes(encodeWireValue(publication.value)),
sourceRuntimeId: this.#identity.serviceInstanceId,
sourceAgentId: this.#identity.providerId,
sourceRole: "provider",

View file

@ -1,4 +1,5 @@
import { encode } from "@msgpack/msgpack";
import { encodeDocumentPayload } from "@gamecult/cultcache-ts";
import { encodeWireValue } from "cultnet-ts/wire";
import type {
CultNetDocumentPutRawMessage,
CultNetOperationRequestMessage,
@ -13,6 +14,7 @@ import {
cultMeshProviderSessionSchemas,
decodeProviderSessionPayload,
encodeProviderSessionPayload,
providerCommandDocument,
type CultMeshProviderCommandReceiptWire,
type CultMeshProviderCommandWire,
type CultMeshProviderLeaseRenewalWire,
@ -420,7 +422,7 @@ export class CultMeshProviderSessionBroker {
recordKey: `${command.providerId}/${command.serviceInstanceId}/${command.commandId}`,
storedAt: this.#clock.now().toISOString(),
payloadEncoding: "messagepack",
payload: encode(command),
payload: encodeDocumentPayload(providerCommandDocument, command),
sourceRuntimeId: this.#options.runtimeId,
sourceRole: "provider-session-broker",
},
@ -513,7 +515,7 @@ function documentTuple(document: Pick<CultNetRawDocumentRecord, "schemaId" | "re
}
function wireEquivalent(left: unknown, right: unknown): boolean {
return Buffer.from(encode(left)).equals(Buffer.from(encode(right)));
return Buffer.from(encodeWireValue(left)).equals(Buffer.from(encodeWireValue(right)));
}
function publicationEntries(record: LeaseRecord): readonly {

View file

@ -1,4 +1,5 @@
import { decode, encode } from "@msgpack/msgpack";
import { decodeDocumentPayload, defineDocumentType } from "@gamecult/cultcache-ts";
import { decodeWireValue, encodeWireValue } from "cultnet-ts/wire";
import type { CultNetRawDocumentRecord } from "cultnet-ts";
export const CULTMESH_PROVIDER_SESSION_SERVICE_ID = "gamecult.mesh.provider_session";
@ -120,27 +121,27 @@ export interface CultMeshProviderMutationAcceptanceWire {
}
export function encodeProviderSessionPayload(value: unknown): string {
return Buffer.from(encode(value)).toString("base64");
return Buffer.from(encodeWireValue(value)).toString("base64");
}
export function decodeProviderSessionPayload<T>(payload: string): T {
if (typeof payload !== "string" || payload.length === 0) {
throw new Error("CultMesh provider-session payload must be non-empty MessagePack base64.");
}
return decode(Buffer.from(payload, "base64")) as T;
return decodeWireValue(Buffer.from(payload, "base64")) as T;
}
export function encodeProviderConnectEvidence(
evidence: CultMeshProviderConnectEvidenceWire,
): Uint8Array {
assertProviderConnectEvidence(evidence);
return encode(evidence);
return encodeWireValue(evidence);
}
export function decodeProviderConnectEvidence(
payload: Uint8Array,
): CultMeshProviderConnectEvidenceWire {
const evidence = decode(payload) as unknown;
const evidence = decodeWireValue(payload) as unknown;
assertProviderConnectEvidence(evidence);
return evidence;
}
@ -155,6 +156,23 @@ export function assertProviderConnectEvidence(
}
}
// The provider command is a CultCache document like any other: written with encodeDocumentPayload and read with
// decodeDocumentPayload, so its bytes never pass through a plain decoder.
export const providerCommandDocument = defineDocumentType({
type: cultMeshProviderSessionSchemas.command,
schemaId: cultMeshProviderSessionSchemas.command,
schema: {
parse(value: unknown): CultMeshProviderCommandWire {
const command = requireObject(value, "provider command");
requireText(command.commandId, "commandId");
requireText(command.commandKind, "commandKind");
requireText(command.providerId, "providerId");
requireText(command.serviceInstanceId, "serviceInstanceId");
return command as unknown as CultMeshProviderCommandWire;
},
},
});
export function decodeProviderCommandDocument(document: CultNetRawDocumentRecord): CultMeshProviderCommandWire {
if (document.schemaId !== cultMeshProviderSessionSchemas.command) {
throw new Error(`Expected ${cultMeshProviderSessionSchemas.command}, received ${document.schemaId}.`);
@ -162,12 +180,7 @@ export function decodeProviderCommandDocument(document: CultNetRawDocumentRecord
if (document.payloadEncoding !== "messagepack") {
throw new Error(`Provider command ${document.recordKey} must use MessagePack.`);
}
const command = decode(document.payload) as Partial<CultMeshProviderCommandWire>;
requireText(command.commandId, "commandId");
requireText(command.commandKind, "commandKind");
requireText(command.providerId, "providerId");
requireText(command.serviceInstanceId, "serviceInstanceId");
return command as CultMeshProviderCommandWire;
return decodeDocumentPayload(providerCommandDocument, document.payload);
}
export function assertProviderLease(value: unknown): asserts value is CultMeshProviderLeaseWire {

View file

@ -15,6 +15,7 @@
// the same C# test decodes. A fixture authored on one side alone would only
// agree with itself.
import { documentPayloadBytes, documentPayloadFromBytes, type DocumentPayload } from "@gamecult/cultcache-ts";
import { isNullOrWhiteSpaceCSharp } from "cultnet-ts";
/** `CultMeshRealtimeWireProtocol.ApplicationProtocolName`: the QUIC ALPN. */
@ -55,7 +56,7 @@ export interface CultMeshRealtimeFrame {
producerEpoch: bigint;
sequence: bigint;
delivery: CultMeshRealtimeDelivery;
payload: Uint8Array;
payload: DocumentPayload;
}
const MAGIC = 0x31545343;
@ -123,7 +124,7 @@ export function encodeRealtimeFrame(frame: CultMeshRealtimeFrame): Uint8Array {
result.set(channel, offset); offset += channel.byteLength;
result.set(schema, offset); offset += schema.byteLength;
result.set(body, offset); offset += body.byteLength;
result.set(frame.payload, offset);
result.set(documentPayloadBytes(frame.payload), offset);
return result;
}
@ -186,6 +187,6 @@ export function decodeRealtimeFrame(bytes: Uint8Array): CultMeshRealtimeFrame {
// would compare equal or not depending on how it reached the decoder.
//
// The constructor copies and is the one type on both inputs.
payload: new Uint8Array(bytes.subarray(offset, offset + payloadLength)),
payload: documentPayloadFromBytes(new Uint8Array(bytes.subarray(offset, offset + payloadLength))),
};
}

View file

@ -0,0 +1,210 @@
import assert from "node:assert/strict";
import { mkdtemp } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { z } from "zod";
import {
codecs,
decodeDocumentPayload,
defineDocumentType,
documentPayloadBytes,
encodeDocumentPayload,
} from "@gamecult/cultcache-ts";
import type { CacheBackingStore, CultCacheEnvelope } from "@gamecult/cultcache-ts";
import {
CultNetDocumentRegistry,
defineCultNetDocumentBinding,
type CultNetPeer,
} from "cultnet-ts";
import { CultMesh, cultMeshDocumentFromSingleFile, cultMeshDocumentFromStore, type CultMeshRudpDocumentPut } from "../src/index";
// A document written in the reference's positional shape, and a plain map document beside it. Every mesh reader of a
// payload reads it the way the cache does, so both read through the mesh.
const codecNote = defineDocumentType({
type: "cultmesh.codec_note",
schemaId: "cultmesh.codec_note.v0",
schema: z.object({ noteId: z.string(), body: z.string() }),
name: "noteId",
codec: codecs.slots<{ noteId: string; body: string }>({
0: ["noteId", codecs.string],
1: ["body", codecs.string],
}, "Tests.CodecNote"),
});
const codecReceipt = defineDocumentType({
type: "cultmesh.codec_receipt",
schemaId: "cultmesh.codec_receipt.v0",
schema: z.object({ commandId: z.string(), body: z.string() }),
name: "commandId",
codec: codecs.slots<{ commandId: string; body: string }>({
0: ["commandId", codecs.string],
1: ["body", codecs.string],
}, "Tests.CodecReceipt"),
});
// The same record under another schema id: the store holds it as codec_note, so it is found by parsing, not by id.
const codecNoteAlias = defineDocumentType({
type: "cultmesh.codec_note_alias",
schemaId: "cultmesh.codec_note_alias.v0",
schema: z.object({ noteId: z.string(), body: z.string() }),
name: "noteId",
codec: codecNote.codec,
});
const plainNote = defineDocumentType({
type: "cultmesh.plain_note",
schemaId: "cultmesh.plain_note.v0",
schema: z.object({ noteId: z.string(), body: z.string() }),
name: "noteId",
});
test("the mesh reads a stored codec document as the cache does, and a plain document unchanged", async () => {
const dir = await mkdtemp(join(tmpdir(), "cultmesh-ts-codec-store-"));
const codecPath = join(dir, "codec.ccmp");
const plainPath = join(dir, "plain.ccmp");
const codecNode = await CultMesh.startNode(codecPath, { documents: [codecNote] });
await codecNode.put(codecNote, "n1", { noteId: "n1", body: "positional" });
const plainNode = await CultMesh.startNode(plainPath, { documents: [plainNote] });
await plainNode.put(plainNote, "n2", { noteId: "n2", body: "map" });
assert.deepEqual(await cultMeshDocumentFromSingleFile(codecPath, codecNote).latest(), { noteId: "n1", body: "positional" });
assert.deepEqual(await cultMeshDocumentFromSingleFile(codecPath, codecNoteAlias).latest(), { noteId: "n1", body: "positional" });
// A store holding a plain record and a codec record, the alias found by parsing takes the codec one whichever the
// single-file store (which sorts records by key) puts first: the probe has to refuse the plain record, not meet it
// second.
for (const [plainKey, codecKey] of [["a-plain", "z-codec"], ["z-plain", "a-codec"]] as const) {
const bothPath = join(dir, `both-${plainKey}.ccmp`);
const bothNode = await CultMesh.startNode(bothPath, { documents: [plainNote, codecNote] });
await bothNode.put(plainNote, plainKey, { noteId: plainKey, body: "map" });
await bothNode.put(codecNote, codecKey, { noteId: codecKey, body: "positional" });
assert.deepEqual(await cultMeshDocumentFromSingleFile(bothPath, codecNoteAlias).latest(), { noteId: codecKey, body: "positional" });
}
assert.deepEqual(await cultMeshDocumentFromSingleFile(plainPath, plainNote).latest(), { noteId: "n2", body: "map" });
});
test("the mesh syncs a codec document from a peer snapshot and returns a codec receipt", async () => {
const connectionId = 0x10203060;
const dir = await mkdtemp(join(tmpdir(), "cultmesh-ts-codec-sync-"));
const source = await CultMesh.startNode(join(dir, "source.ccmp"), { documents: [codecNote, plainNote] });
await source.put(codecNote, "n1", { noteId: "n1", body: "synced positional" });
// Two records at one key, in two shapes: a peer snapshot handle for the alias schema (which neither holds) accepts the
// one that parses as the codec.
await source.put(plainNote, "n3", { noteId: "n3", body: "map" });
await source.put(codecNote, "n3", { noteId: "n3", body: "positional" });
const target = await CultMesh.startNode(join(dir, "target.ccmp"), { documents: [codecNote] });
const commandBinding = defineCultNetDocumentBinding({ definition: codecNote });
const plainBinding = defineCultNetDocumentBinding({ definition: plainNote });
const receiptBinding = defineCultNetDocumentBinding({ definition: codecReceipt });
const puts: unknown[] = [];
const server = CultMesh.createRudpDocumentServer("cultmesh-ts-codec-server", connectionId, {
documents: new CultNetDocumentRegistry([commandBinding, plainBinding, receiptBinding]),
getCache: () => source.cache,
bindHost: "127.0.0.1", bindPort: 0, resendDelayMs: 25, resendPollMs: 5,
onDocumentPutRaw: async (document) => (puts.push(document.payload), {
binding: receiptBinding,
recordKey: document.recordKey,
value: { commandId: document.recordKey, body: "applied" },
}),
});
let peer: CultNetPeer | undefined;
try {
await server.start();
peer = await CultMesh.createRudpPeer("cultmesh-ts-codec-client", connectionId, `rudp://127.0.0.1:${server.bind.port}`, {
resendDelayMs: 25, resendPollMs: 5, connectTimeoutMs: 1_000,
});
// The accepts probe reads the snapshot's payload as the codec: a plain decode would reject every codec snapshot.
const synced = await target.syncDocumentFromPeerSnapshot(peer, codecNote, "n1", { timeoutMs: 1_000 });
assert.deepEqual(synced, { noteId: "n1", body: "synced positional" });
assert.deepEqual(target.getRequired(codecNote, "n1"), { noteId: "n1", body: "synced positional" });
// A peer snapshot handle reads the same way, and accepts the snapshot by parsing it as the codec.
assert.deepEqual(await CultMesh.documentFromPeerSnapshot(peer, codecNote, "n1", { timeoutMs: 1_000 }).latest(), { noteId: "n1", body: "synced positional" });
assert.deepEqual(
await CultMesh.documentFromPeerSnapshot(peer, codecNoteAlias, "n3", { timeoutMs: 1_000 }).latest(),
{ noteId: "n3", body: "positional" },
);
const receipt = await CultMesh.publishRudpDocumentAndWaitForReceipt(
"cultmesh-ts-codec-receipt", connectionId, `rudp://127.0.0.1:${server.bind.port}`,
commandBinding, "n2", { noteId: "n2", body: "command" }, receiptBinding,
{ messageId: "n2", receiptTimeoutMs: 1_000, resendPollMs: 5 },
);
assert.deepEqual(receipt, { commandId: "n2", body: "applied" });
// The handler is handed the document, not its wire shape: the server reads a put through the registered definition.
await CultMesh.publishRudpDocumentAndWaitForReceipt(
"cultmesh-ts-plain-receipt", connectionId, `rudp://127.0.0.1:${server.bind.port}`,
plainBinding, "n4", { noteId: "n4", body: "plain command" }, receiptBinding,
{ messageId: "n4", receiptTimeoutMs: 1_000, resendPollMs: 5 },
);
assert.deepEqual(puts, [{ noteId: "n2", body: "command" }, { noteId: "n4", body: "plain command" }]);
} finally {
peer?.close();
server.close();
}
});
test("a RUDP put naming a schema the server has no definition for reaches the handler as the untouched opaque payload", async () => {
const connectionId = 0x10203070;
const stray = defineDocumentType({
type: "cultmesh.stray_note",
schemaId: "cultmesh.stray_note.v0",
schema: z.object({ noteId: z.string() }),
name: "noteId",
codec: codecs.slots<{ noteId: string }>({ 0: ["noteId", codecs.string] }, "Tests.StrayNote"),
});
const strayBinding = defineCultNetDocumentBinding({ definition: stray });
const known = defineCultNetDocumentBinding({ definition: plainNote });
const errors: Error[] = [];
const puts: CultMeshRudpDocumentPut[] = [];
const server = CultMesh.createRudpDocumentServer("cultmesh-ts-unregistered-server", connectionId, {
documents: new CultNetDocumentRegistry([known]),
bindHost: "127.0.0.1", bindPort: 0, resendDelayMs: 25, resendPollMs: 5,
onError: (error) => errors.push(error),
onDocumentPutRaw: async (document) => { puts.push(document); },
});
let peer: CultNetPeer | undefined;
try {
await server.start();
peer = await CultMesh.createRudpPeer("cultmesh-ts-unregistered-client", connectionId, `rudp://127.0.0.1:${server.bind.port}`, {
resendDelayMs: 25, resendPollMs: 5, connectTimeoutMs: 1_000,
});
peer.send(new CultNetDocumentRegistry([strayBinding]).createRawDocumentPutMessage(strayBinding, "unregistered-1", "k", { noteId: "k" }));
const startedAt = Date.now();
while (puts.length === 0 && Date.now() - startedAt < 2_000) await new Promise((resolve) => setTimeout(resolve, 5));
assert.deepEqual(errors, []);
assert.equal(puts.length, 1);
// Not read as plain msgpack: the bytes are the codec's slot record, handed over opaque, and the definition that reads
// them is the handler's to hold.
const put = puts[0]!;
assert.ok(!put.decoded, "the registry holds no definition for this schema");
const payload = put.payload;
assert.equal(Buffer.from(documentPayloadBytes(payload)).toString("hex"), "91a16b", "a one-slot codec record, not a map");
assert.deepEqual(Buffer.from(documentPayloadBytes(payload)), Buffer.from(documentPayloadBytes(encodeDocumentPayload(stray, { noteId: "k" }))));
assert.deepEqual(decodeDocumentPayload(stray, payload), { noteId: "k" });
} finally {
peer?.close();
server.close();
}
});
test("a store read takes the record the catalog matches to the schema and the definition parses, not the first that merely parses", async () => {
const target = defineDocumentType({ type: "cultmesh.pick", schemaId: "cultmesh.pick.v1", schema: z.object({ n: z.number() }) });
const lenient = defineDocumentType({ type: "cultmesh.pick", schemaId: "cultmesh.pick.v1", schema: { parse: (value: unknown) => value } });
const entry = (schemaId: string, compatibleSchemaIds?: readonly string[]) => ({
schemaId, schemaName: schemaId, schemaVersion: `${schemaId}.v1`, contentHash: "h", canonicalSchemaJson: "{}", compatibleSchemaIds,
});
const envelope = (key: string, schemaId: string, value: unknown, compatibleSchemaIds?: readonly string[]): CultCacheEnvelope => ({
key, type: "cultmesh.pick", schemaId, storedAt: "2026-10-11T00:00:00.000Z",
payload: encodeDocumentPayload(lenient, value), catalogEntry: entry(schemaId, compatibleSchemaIds),
});
// None carries the definition's own schema id, so the exact-id find misses. The first parses and is foreign; the
// second is matched by the catalog and does not parse; the third is matched and parses.
const records = [
envelope("a", "foreign.v1", { n: 1 }),
envelope("b", "alias-bad.v1", { n: "x" }, ["cultmesh.pick.v1"]),
envelope("c", "alias-good.v1", { n: 2 }, ["cultmesh.pick.v1"]),
];
const store: CacheBackingStore = { pullAll: async () => records, push: async () => undefined, delete: async () => undefined };
assert.deepEqual(await cultMeshDocumentFromStore(store, target).latest(), { n: 2 });
});

View file

@ -4,9 +4,9 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import assert from "node:assert/strict";
import { decode, encode } from "@msgpack/msgpack";
import { decodeWireValue, encodeWireValue } from "cultnet-ts/wire";
import { z } from "zod";
import { defineDocumentType } from "@gamecult/cultcache-ts";
import { defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import {
CultNetDocumentRegistry,
CultNetPeer,
@ -22,6 +22,9 @@ import {
} from "cultnet-ts";
import { CultMesh, CultMeshRudpUnservableDocumentError } from "../src/index";
// A record's payload as the bytes a peer would put on the wire.
const payloadOf = (value: unknown) => documentPayloadFromBytes(encodeWireValue(value));
const noteDocument = defineDocumentType({
type: "cultmesh.note",
schemaId: "cultmesh.note.v0",
@ -268,7 +271,7 @@ test("CultMesh TS store document handles choose compatible foreign schema payloa
type: "runtime.generated.incompatible-note",
schemaId: "runtime.generated.incompatible-note.v1",
storedAt: "2026-06-27T00:00:00Z",
payload: encode({ nope: "not the requested shape" }),
payload: payloadOf({ nope: "not the requested shape" }),
catalogEntry: {
schemaId: "runtime.generated.incompatible-note.v1",
schemaName: "runtime.generated.incompatible-note",
@ -283,7 +286,7 @@ test("CultMesh TS store document handles choose compatible foreign schema payloa
type: "runtime.generated.cultmesh.note",
schemaId: "runtime.generated.cultmesh.note.ui.42",
storedAt: "2026-06-27T00:00:01Z",
payload: encode({
payload: payloadOf({
noteId: "note:foreign",
body: "foreign schema id, local store shape",
}),
@ -2778,7 +2781,9 @@ test("CultMesh TS RUDP document server accepts raw document puts from multiple p
maxFragmentBytes: 1024,
maxPendingReliablePackets: 16,
onDocumentPutRaw: (document) => {
received.push(`${document.sourceRuntimeId}:${document.recordKey}:${document.payload}`);
// The registry holds no definition, so the payload arrives untouched and is read here by name.
if (document.decoded) throw new Error("the registry holds no definition, so the put must arrive undecoded");
received.push(`${document.sourceRuntimeId}:${document.recordKey}:${decodeWireValue(documentPayloadBytes(document.payload))}`);
},
},
);
@ -2820,7 +2825,7 @@ test("CultMesh TS RUDP document server accepts raw document puts from multiple p
recordKey: "note:a",
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack",
payload: encode("first"),
payload: payloadOf("first"),
sourceRuntimeId: "client-a",
},
});
@ -2832,7 +2837,7 @@ test("CultMesh TS RUDP document server accepts raw document puts from multiple p
recordKey: "note:b",
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack",
payload: encode("second"),
payload: payloadOf("second"),
sourceRuntimeId: "client-b",
},
});
@ -2864,10 +2869,11 @@ test("CultMesh TS publishes one registered document to a RUDP catalog", async ()
maxFragmentBytes: 1024,
maxPendingReliablePackets: 16,
onDocumentPutRaw: (document) => {
if (document.decoded) throw new Error("the registry holds no definition, so the put must arrive undecoded");
received = {
schemaId: document.schemaId,
recordKey: document.recordKey,
payload: document.payload,
payload: decodeWireValue(documentPayloadBytes(document.payload)),
sourceRuntimeId: document.sourceRuntimeId,
};
},
@ -3018,7 +3024,12 @@ test("CultMesh TS reads remote RUDP snapshots through document handles", async (
const alias = CultMesh.documentFromPeerSnapshot(
peer,
"cultmesh.note.alias.v1",
defineDocumentType({
type: "cultmesh.note.alias",
schemaId: "cultmesh.note.alias.v1",
schema: noteDocument.schema,
name: "noteId",
}),
"note:remote",
{
documentId: "note:remote.alias",
@ -3311,7 +3322,7 @@ function notePut(
recordKey,
storedAt,
payloadEncoding: "messagepack",
payload: encode({ noteId: recordKey, body: "x".repeat(bodyLength) }),
payload: payloadOf({ noteId: recordKey, body: "x".repeat(bodyLength) }),
...(sourceRuntimeId ? { sourceRuntimeId } : {}),
},
};
@ -3324,7 +3335,7 @@ function notePut(
*/
function servedAloneBytes(put: CultNetDocumentPutRawMessage): number {
const { schemaId, recordKey, storedAt, payloadEncoding, payload } = put.document;
return encode({
return encodeWireValue({
schemaVersion: "cultnet.snapshot_response_raw.v0",
messageId: "0",
documents: [{ schemaId, recordKey, storedAt, payloadEncoding, payload }],
@ -3337,7 +3348,7 @@ function servedAloneBytes(put: CultNetDocumentPutRawMessage): number {
* asked of the server, so a server that sized another receipt is caught.
*/
function echoReceiptBytes(put: CultNetDocumentPutRawMessage): number {
return encode({
return encodeWireValue({
schemaVersion: "cultnet.document_put_raw.v0",
messageId: put.messageId,
document: {
@ -3345,7 +3356,7 @@ function echoReceiptBytes(put: CultNetDocumentPutRawMessage): number {
recordKey: put.document.recordKey,
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack",
payload: encode(decode(put.document.payload)),
payload: payloadOf(decodeWireValue(documentPayloadBytes(put.document.payload))),
sourceRuntimeId: SERVED_BOUND_SERVER,
sourceAgentId: undefined,
sourceRole: "provider",
@ -3397,7 +3408,7 @@ async function withServedBoundRig(
recordKey: document.recordKey,
storedAt: document.storedAt,
payloadEncoding: "messagepack",
payload: encode(document.payload),
payload: payloadOf(document.payload),
},
});
if (!handler.receipt) return undefined;
@ -3448,7 +3459,7 @@ function servedBodies(reply: CultNetMessage): Array<[string, number]> {
assert.equal(reply.schemaVersion, "cultnet.snapshot_response_raw.v0");
return (reply as CultNetSnapshotResponseRawMessage).documents.map((document) => [
document.recordKey,
(decode(document.payload) as { body: string }).body.length,
(decodeWireValue(documentPayloadBytes(document.payload)) as { body: string }).body.length,
]);
}
@ -3567,7 +3578,7 @@ async function withRawPeerRig(
const request = (messageId: string, recordKeys: string[]): number[] => {
const packets = session.sendMany(
"schema",
encode({ schemaVersion: "cultnet.snapshot_request.v0", messageId, recordKeys }),
encodeWireValue({ schemaVersion: "cultnet.snapshot_request.v0", messageId, recordKeys }),
{ reliable: true, ordered: true, nowMs: 2, maxFragmentBytes: 1200 },
);
packets.forEach(send);
@ -3586,7 +3597,7 @@ function refusalIn(packets: CultNetRudpPacket[]): { message: Record<string, unkn
.filter((packet) => packet.packetType === "data" && packet.channelId === "schema" && !packet.reliable)
.sort((a, b) => (a.fragmentIndex ?? 0) - (b.fragmentIndex ?? 0));
if (parts.length === 0) return undefined;
const message = decode(Buffer.concat(parts.map((part) => Buffer.from(part.payload ?? new Uint8Array())))) as Record<string, unknown>;
const message = decodeWireValue(Buffer.concat(parts.map((part) => Buffer.from(part.payload ?? new Uint8Array())))) as Record<string, unknown>;
return { message, parts };
}
@ -3655,7 +3666,7 @@ test("CultMesh TS RUDP document server honours maxPayloadBytes for replies and j
// A put whose own frame is over the limit only by the provenance it carries is served at the
// bound, so it is admitted and served.
const carried = notePut("put-at", "note:at", atBound, `runtime-${"r".repeat(200)}`);
assert.ok(encode(carried).byteLength > limit, "fixture: the put's frame is over the limit");
assert.ok(encodeWireValue(carried).byteLength > limit, "fixture: the put's frame is over the limit");
rig.peer.send(carried);
await waitForAdmission(rig, "note:at");
assert.deepEqual(servedBodies(await snapshotUnderShortestId(rig, "note:at")), [["note:at", atBound]]);

View file

@ -27,6 +27,8 @@ import { readFileSync } from "node:fs";
import { join } from "node:path";
import process from "node:process";
import { documentPayloadBytes, documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import {
CultMeshQuicRealtimeConnector,
CultMeshQuicRealtimeProvider,
@ -124,7 +126,7 @@ async function serveAsync(args: Map<string, string>): Promise<void> {
producerEpoch: 1n,
sequence: BigInt(sequence),
delivery,
payload: Buffer.from(`frame-${sequence}`, "utf8"),
payload: documentPayloadFromBytes(Buffer.from(`frame-${sequence}`, "utf8")),
};
await provider.broadcast(frame);
// Printed so the harness can compare this exact frame's TypeScript
@ -215,7 +217,7 @@ async function main(): Promise<void> {
producerEpoch: frame.producerEpoch.toString(),
sequence: frame.sequence.toString(),
delivery: frame.delivery,
payloadHex: Buffer.from(frame.payload).toString("hex"),
payloadHex: Buffer.from(documentPayloadBytes(frame.payload)).toString("hex"),
});
} finally {
clearTimeout(timer);

View file

@ -0,0 +1,49 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
decodeDocumentPayload,
defineDocumentType,
documentPayloadBytes,
documentPayloadFromBytes,
type DocumentPayload,
} from "@gamecult/cultcache-ts";
import { decodeWireValue } from "cultnet-ts/wire";
import type { CultNetRawDocumentRecord } from "cultnet-ts";
import type { CultMeshRudpDocumentPut } from "../src/index";
import { decodeRealtimeFrame, encodeRealtimeFrame, type CultMeshRealtimeFrame } from "../src/realtime-wire";
// The rule: a document payload is read only through its definition. Every payload this package touches (a CultNet
// record's, a realtime frame's) is the opaque DocumentPayload, so tsc compiles this file only if each line carrying
// an expect-error directive is an error. The function is never called.
export function refusedReaders(record: CultNetRawDocumentRecord, frame: CultMeshRealtimeFrame): void {
// @ts-expect-error the wire value decoder is for frames and values, not for a document's payload
decodeWireValue(record.payload);
// @ts-expect-error a frame's payload is not assignable to bytes
const bytes: Uint8Array = frame.payload;
// @ts-expect-error nor may a copy of it be taken as bytes
new Uint8Array(record.payload);
// @ts-expect-error bytes are not a payload until they enter by name
const forged: DocumentPayload = new Uint8Array();
void [bytes, forged];
}
// The one seam where a DocumentPayload reaches a consumer: a put for a schema the registry holds no definition for. The
// owner read of the undecoded branch needs no cast, and the decoded branch is not a DocumentPayload.
const noteDefinition = defineDocumentType({ type: "t.note", schemaId: "t.note.v0", schema: { parse: (value: unknown) => value } });
export function readPut(put: CultMeshRudpDocumentPut): unknown {
if (!put.decoded) return decodeDocumentPayload(noteDefinition, put.payload);
// @ts-expect-error a decoded put's payload is the document, not a DocumentPayload
decodeDocumentPayload(noteDefinition, put.payload);
return put.payload;
}
test("a realtime frame carries its payload as the same bytes it was given by name", () => {
const bytes = Uint8Array.from([1, 2, 3]);
const frame: CultMeshRealtimeFrame = {
channelId: "c", schemaId: "s", bodyId: "b", producerEpoch: 1n, sequence: 1n, delivery: "reliable-ordered",
payload: documentPayloadFromBytes(bytes),
};
assert.deepEqual(documentPayloadBytes(decodeRealtimeFrame(encodeRealtimeFrame(frame)).payload), bytes);
});

View file

@ -1,7 +1,8 @@
import test from "node:test";
import assert from "node:assert/strict";
import { createSocket } from "node:dgram";
import { decode } from "@msgpack/msgpack";
import { documentPayloadBytes } from "@gamecult/cultcache-ts";
import { decodeWireValue } from "cultnet-ts/wire";
import {
CultNetDocumentRegistry,
CultNetRudpSession,
@ -46,7 +47,7 @@ test("RUDP provider transport requires broker acceptance for the full lifecycle"
session.connectPayload !== undefined
&& decodeProviderConnectEvidence(session.connectPayload).sessionToken === "signed-provider-token",
acceptPublication: (_identity, publicationId, document) => {
accepted.set(publicationId, decode(document.payload));
accepted.set(publicationId, decodeWireValue(documentPayloadBytes(document.payload)));
acceptedProvenance.set(publicationId, {
runtime: document.sourceRuntimeId,
agent: document.sourceAgentId,

View file

@ -1,10 +1,16 @@
import test from "node:test";
import assert from "node:assert/strict";
import { documentPayloadBytes, documentPayloadFromBytes, encodeDocumentPayload, type DocumentPayload } from "@gamecult/cultcache-ts";
import { encodeWireValue } from "cultnet-ts/wire";
import {
cultMeshProviderSessionSchemas,
decodeProviderCommandDocument,
decodeProviderConnectEvidence,
encodeProviderConnectEvidence,
encodeProviderSessionPayload,
providerCommandDocument,
} from "../src/provider-session-wire";
const REGISTRATION_GOLDEN = "hqpwcm92aWRlcklkqGFldGhlcmlhsXNlcnZpY2VJbnN0YW5jZUlkq2FldGhlcmlhLTQyqmVuZHBvaW50SWSvYWV0aGVyaWEtcHVibGljp3ZlcnNlSWSmcHVibGljuHJlcXVlc3RlZExlYXNlRHVyYXRpb25Nc811MLBhdXRob3JpdHlMZWFzZUlkq2F1dGhvcml0eS03";
@ -41,3 +47,39 @@ test("provider Connect evidence keeps transport generation separate from authori
TOKENLESS_CONNECT_EVIDENCE_GOLDEN,
);
});
test("a provider command is a CultCache document: written and read by its definition, and a refusal names no value", () => {
const command = { commandId: "c1", commandKind: "k", providerId: "p", serviceInstanceId: "s", payload: { n: 1 } };
const payload = encodeDocumentPayload(providerCommandDocument, command);
assert.deepEqual(documentPayloadBytes(payload), encodeWireValue(command), "the wire bytes are the plain map they always were");
const record = (bytes: DocumentPayload) => ({
schemaId: cultMeshProviderSessionSchemas.command, recordKey: "p/s/c1", storedAt: "2026-10-11T00:00:00Z",
payloadEncoding: "messagepack" as const, payload: bytes,
});
assert.deepEqual(decodeProviderCommandDocument(record(payload)), command);
const refused = documentPayloadFromBytes(encodeWireValue({ ...command, providerId: "canary-provider-5d2e", commandKind: "" }));
assert.throws(
() => decodeProviderCommandDocument(record(refused)),
(error: Error) => /commandKind/.test(error.message) && !error.message.includes("canary-provider-5d2e"),
);
assert.throws(() => encodeDocumentPayload(providerCommandDocument, { ...command, commandId: " " }), /commandId/);
});
test("a provider command that is not a map, or lacks a required field, is refused naming the field and none of its values", () => {
const command = { commandId: "c1", commandKind: "k", providerId: "p", serviceInstanceId: "s", payload: 1 };
const record = (value: unknown) => ({
schemaId: cultMeshProviderSessionSchemas.command, recordKey: "p/s/c1", storedAt: "2026-10-11T00:00:00Z",
payloadEncoding: "messagepack" as const, payload: documentPayloadFromBytes(encodeWireValue(value)),
});
for (const field of ["commandId", "commandKind", "providerId", "serviceInstanceId"] as const) {
assert.throws(
() => decodeProviderCommandDocument(record({ ...command, [field]: "" })),
(error: Error) => error.message === `CultMesh provider-session ${field} must be non-empty.`,
);
}
assert.throws(
() => decodeProviderCommandDocument(record(["canary-not-a-map"])),
(error: Error) => error.message === "CultMesh provider command must be a MessagePack map.",
);
});

View file

@ -14,6 +14,8 @@ import { readFileSync } from "node:fs";
import { join } from "node:path";
import test from "node:test";
import { documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import {
canonicalRoute,
type CultMeshAuthorityRouteView,
@ -154,7 +156,7 @@ function testFrame(overrides: Partial<CultMeshRealtimeFrame> = {}): CultMeshReal
producerEpoch: 7n,
sequence: 1n,
delivery: "latest-only",
payload: new Uint8Array([3, 4, 5]),
payload: documentPayloadFromBytes(new Uint8Array([3, 4, 5])),
...overrides,
};
}

View file

@ -13,6 +13,8 @@ import { readFileSync } from "node:fs";
import { join } from "node:path";
import test from "node:test";
import { documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import {
CultMeshQuicNativeRuntime,
} from "../src/realtime-quic-native";
@ -63,7 +65,7 @@ function testFrame(overrides: Partial<CultMeshRealtimeFrame> = {}): CultMeshReal
producerEpoch: 7n,
sequence: 1n,
delivery: "latest-only",
payload: new Uint8Array([3, 4, 5]),
payload: documentPayloadFromBytes(new Uint8Array([3, 4, 5])),
...overrides,
};
}
@ -613,8 +615,8 @@ test("a peer attaching after two latest-only broadcasts for one key receives exa
if (!nativeBridgeAvailable()) return void t.skip("no native bridge available");
const provider = await startProvider();
try {
await provider.broadcast(testFrame({ sequence: 1n, payload: new Uint8Array([1]) }));
await provider.broadcast(testFrame({ sequence: 2n, payload: new Uint8Array([2]) }));
await provider.broadcast(testFrame({ sequence: 1n, payload: documentPayloadFromBytes(new Uint8Array([1])) }));
await provider.broadcast(testFrame({ sequence: 2n, payload: documentPayloadFromBytes(new Uint8Array([2])) }));
// Attach after both broadcasts have already resolved: nothing in this
// peer's outbox came from `broadcast()`'s own fan-out, only from the
@ -636,11 +638,11 @@ test("retention keeps the newer generation when an older one broadcasts afterwar
if (!nativeBridgeAvailable()) return void t.skip("no native bridge available");
const provider = await startProvider();
try {
await provider.broadcast(testFrame({ sequence: 5n, payload: new Uint8Array([5]) }));
await provider.broadcast(testFrame({ sequence: 5n, payload: documentPayloadFromBytes(new Uint8Array([5])) }));
// An older generation broadcast after a newer one must not overwrite the
// retained frame: mutating the retention comparison to a plain
// last-write-wins assignment would regress this.
await provider.broadcast(testFrame({ sequence: 3n, payload: new Uint8Array([3]) }));
await provider.broadcast(testFrame({ sequence: 3n, payload: documentPayloadFromBytes(new Uint8Array([3])) }));
const consumer = await dialProvider(provider);
try {
@ -672,7 +674,7 @@ test("a reliable-ordered frame is never retained and is not delivered to a late
// after a real latest-only frame, so the assertion below proves the
// reliable-ordered frame specifically was never retained, rather than
// merely proving the provider had nothing at all to seed.
await provider.broadcast(testFrame({ delivery: "latest-only", sequence: 1n, payload: new Uint8Array([1]) }));
await provider.broadcast(testFrame({ delivery: "latest-only", sequence: 1n, payload: documentPayloadFromBytes(new Uint8Array([1])) }));
const secondConsumer = await dialProvider(provider);
try {
const seeded = await secondConsumer.receiveFrame();

View file

@ -16,6 +16,8 @@ import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { documentPayloadBytes, documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import {
CULTMESH_REALTIME_ALPN,
CULTMESH_REALTIME_CONNECTION_CLOSE_CODE,
@ -89,7 +91,7 @@ function frameOf(fields: VectorFields): CultMeshRealtimeFrame {
producerEpoch: 0n,
sequence: 0n,
delivery: fields.delivery,
payload: payload(fields.payload),
payload: documentPayloadFromBytes(payload(fields.payload)),
};
}
@ -175,7 +177,7 @@ const sample = (overrides: Partial<CultMeshRealtimeFrame> = {}): CultMeshRealtim
producerEpoch: 1n,
sequence: 2n,
delivery: "reliable-ordered",
payload: Uint8Array.from([1, 2, 3]),
payload: documentPayloadFromBytes(Uint8Array.from([1, 2, 3])),
...overrides,
});
@ -199,7 +201,7 @@ test("the fixed header is 37 little-endian bytes in the reference's order", () =
const encoded = encodeRealtimeFrame(sample({
channelId: "ch", schemaId: "sch", bodyId: "bdy",
producerEpoch: 0x0102030405060708n, sequence: 0x1112131415161718n,
delivery: "unreliable", payload: Uint8Array.from([0xaa]),
delivery: "unreliable", payload: documentPayloadFromBytes(Uint8Array.from([0xaa])),
}));
const view = new DataView(encoded.buffer, encoded.byteOffset, encoded.byteLength);
assert.equal(view.getUint32(0, true), 0x31545343, "magic, little-endian");
@ -258,11 +260,11 @@ test("an identity of exactly 65535 UTF-8 bytes encodes and one byte more is refu
});
test("a payload at the 64 MiB ceiling encodes and one byte more is refused", () => {
const encoded = encodeRealtimeFrame(sample({ payload: ramp(CULTMESH_REALTIME_MAX_PAYLOAD_BYTES) }));
const encoded = encodeRealtimeFrame(sample({ payload: documentPayloadFromBytes(ramp(CULTMESH_REALTIME_MAX_PAYLOAD_BYTES)) }));
assert.equal(encoded.byteLength, 37 + 15 + 20 + 6 + CULTMESH_REALTIME_MAX_PAYLOAD_BYTES);
assert.equal(decodeRealtimeFrame(encoded).payload.byteLength, CULTMESH_REALTIME_MAX_PAYLOAD_BYTES);
assert.throws(
() => encodeRealtimeFrame(sample({ payload: new Uint8Array(CULTMESH_REALTIME_MAX_PAYLOAD_BYTES + 1) })),
() => encodeRealtimeFrame(sample({ payload: documentPayloadFromBytes(new Uint8Array(CULTMESH_REALTIME_MAX_PAYLOAD_BYTES + 1)) })),
/payload exceeds the QUIC wire limit/,
);
});
@ -363,16 +365,16 @@ test("decode refuses a negative payload length that the length sum cancels out",
});
test("decode reads exactly the viewed bytes of an offset view, not the buffer behind it", () => {
const good = encodeRealtimeFrame(sample({ payload: Uint8Array.from([9, 8, 7]) }));
const good = encodeRealtimeFrame(sample({ payload: documentPayloadFromBytes(Uint8Array.from([9, 8, 7])) }));
const pool = new Uint8Array(good.byteLength + 64).fill(0xa5);
pool.set(good, 17);
const view = pool.subarray(17, 17 + good.byteLength);
assert.deepEqual(decodeRealtimeFrame(view).payload, Uint8Array.from([9, 8, 7]));
assert.deepEqual(documentPayloadBytes(decodeRealtimeFrame(view).payload), Uint8Array.from([9, 8, 7]));
assert.equal(decodeRealtimeFrame(view).channelId, "aetheria.zone-1");
// And the decoded payload does not alias the frame it came from.
const decoded = decodeRealtimeFrame(good);
good[good.byteLength - 1] = 0;
assert.deepEqual(decoded.payload, Uint8Array.from([9, 8, 7]));
assert.deepEqual(documentPayloadBytes(decoded.payload), Uint8Array.from([9, 8, 7]));
});
// The `Uint8Array` case above passes with a plain `bytes.slice(...)`; this one
@ -380,12 +382,12 @@ test("decode reads exactly the viewed bytes of an offset view, not the buffer be
// `Buffer` is what a socket read hands a Node caller, so the aliasing bug this
// pins is the one that would actually reach a consumer.
test("the decoded payload copies out of a Node Buffer too, which overrides slice", () => {
const frame = Buffer.from(encodeRealtimeFrame(sample({ payload: Uint8Array.from([9, 8, 7]) })));
const frame = Buffer.from(encodeRealtimeFrame(sample({ payload: documentPayloadFromBytes(Uint8Array.from([9, 8, 7])) })));
const decoded = decodeRealtimeFrame(frame);
frame[frame.byteLength - 1] = 0;
assert.deepEqual(decoded.payload, Uint8Array.from([9, 8, 7]), "payload unchanged by a later write to the frame");
assert.equal(Buffer.isBuffer(decoded.payload), false, "the copy is a plain Uint8Array, not a Buffer");
assert.notEqual(decoded.payload.buffer, frame.buffer, "the copy has its own backing store");
assert.deepEqual(documentPayloadBytes(decoded.payload), Uint8Array.from([9, 8, 7]), "payload unchanged by a later write to the frame");
assert.equal(Buffer.isBuffer(documentPayloadBytes(decoded.payload)), false, "the copy is a plain Uint8Array, not a Buffer");
assert.notEqual(documentPayloadBytes(decoded.payload).buffer, frame.buffer, "the copy has its own backing store");
// The three identities need no such check: `TextDecoder` returns strings, which
// hold no reference to the bytes they were decoded from.
assert.equal(decoded.channelId, "aetheria.zone-1");

View file

@ -5,7 +5,8 @@
import assert from "node:assert/strict";
import { createSocket, type RemoteInfo, type Socket } from "node:dgram";
import test from "node:test";
import { encode } from "@msgpack/msgpack";
import { documentPayloadFromBytes } from "@gamecult/cultcache-ts";
import { encodeWireValue } from "cultnet-ts/wire";
import {
CultNetDocumentRegistry,
CultNetRudpSession,
@ -43,7 +44,7 @@ function put(recordKey: string) {
recordKey,
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack" as const,
payload: encode(recordKey),
payload: documentPayloadFromBytes(encodeWireValue(recordKey)),
},
};
}

View file

@ -15,6 +15,11 @@
"require": "./dist/contracts.js",
"default": "./dist/contracts.js"
},
"./wire": {
"types": "./dist/wire-value.d.ts",
"require": "./dist/wire-value.js",
"default": "./dist/wire-value.js"
},
"./idunn-runtime": {
"types": "./dist/idunn-runtime-authority.d.ts",
"require": "./dist/idunn-runtime-authority.js",

View file

@ -1,3 +1,4 @@
import type { DocumentPayload } from "@gamecult/cultcache-ts/document";
import Ajv2020, { type ValidateFunction } from "ajv/dist/2020.js";
import documentRecordSchema from "../contracts/cultnet/cultnet.document-record.schema.json";
@ -200,7 +201,7 @@ export interface CultNetRawDocumentRecord {
recordKey: string;
storedAt: string;
payloadEncoding: CultNetRawPayloadEncoding;
payload: Uint8Array;
payload: DocumentPayload;
sourceRuntimeId?: string;
sourceAgentId?: string;
sourceRole?: string;

View file

@ -1,6 +1,7 @@
import crypto from "node:crypto";
import fs from "node:fs";
import path from "node:path";
import { documentPayloadFromBytes } from "@gamecult/cultcache-ts/document";
import { decode, encode } from "@msgpack/msgpack";
import {
@ -278,7 +279,7 @@ export function createIdunnRuntimeSigner(options: IdunnRuntimeSignerOptions): Id
recordKey: expected.target,
storedAt: new Date().toISOString(),
payloadEncoding: "messagepack",
payload: new Uint8Array(signed.payload),
payload: documentPayloadFromBytes(new Uint8Array(signed.payload)),
sourceRuntimeId: expected.runtimeId,
sourceRole: "runtime-presence-health-publisher",
tags: ["cultnet.transport.rudp.v0", "runtime-presence"],

View file

@ -1,12 +1,16 @@
import { decode, encode } from "@msgpack/msgpack";
import type {
AnyCultCacheDocumentDefinition,
CultCache,
CultCacheDocumentDefinition,
CultCacheDocumentFormatter,
CultCacheDocumentValue,
CultCacheEnvelope,
} from "@gamecult/cultcache-ts";
import {
decodeDocumentPayload,
documentPayloadBytes,
documentPayloadFromBytes,
encodeDocumentPayload,
} from "@gamecult/cultcache-ts";
import {
type CultNetDocumentDeleteMessage,
@ -135,8 +139,6 @@ export class CultNetDocumentRegistry {
tags?: string[];
} = {},
): CultNetDocumentPutRawMessage {
const parsed = binding.definition.schema.parse(value);
const formatter = formatterFor(binding.definition);
return {
schemaVersion: "cultnet.document_put_raw.v0",
messageId,
@ -145,7 +147,7 @@ export class CultNetDocumentRegistry {
recordKey: documentKey,
storedAt: options.storedAt ?? new Date().toISOString(),
payloadEncoding: "messagepack",
payload: formatter.encode(parsed),
payload: encodeDocumentPayload(binding.definition, value),
sourceRuntimeId: options.sourceRuntimeId,
sourceAgentId: options.sourceAgentId,
sourceRole: options.sourceRole,
@ -175,7 +177,7 @@ export class CultNetDocumentRegistry {
continue;
}
const payload = decodeDocumentValue(binding.definition, envelope.payload);
const payload = decodeDocumentPayload(binding.definition, envelope.payload);
documents.push({
schemaId,
recordKey: envelope.key,
@ -282,7 +284,7 @@ export class CultNetDocumentRegistry {
schemaId: resolution.preserveIncomingSchemaId
? message.document.schemaId
: schemaIdForBinding(binding),
payload: new Uint8Array(message.document.payload),
payload: documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(message.document.payload))),
storedAt: message.document.storedAt,
},
};
@ -341,10 +343,10 @@ export class CultNetDocumentRegistry {
return { binding: exact, preserveIncomingSchemaId: true };
}
const payload = new Uint8Array(document.payload);
const payload = documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(document.payload)));
for (const binding of this.#uniqueBindings()) {
try {
decodeDocumentValue(binding.definition, payload);
decodeDocumentPayload(binding.definition, payload);
return { binding, preserveIncomingSchemaId: false };
} catch {
// Keep looking; a foreign schema id is acceptable only when the payload validates locally.
@ -368,7 +370,7 @@ export class CultNetDocumentRegistry {
recordKey: envelope.key,
storedAt: envelope.storedAt,
payloadEncoding: "messagepack",
payload: new Uint8Array(envelope.payload),
payload: documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(envelope.payload))),
};
}
}
@ -430,23 +432,3 @@ function resolvePayloadSchemaVersion<TDefinition extends AnyCultCacheDocumentDef
return binding.payloadSchemaVersion;
}
function decodeDocumentValue<TDefinition extends CultCacheDocumentDefinition>(
definition: TDefinition,
payload: Uint8Array,
): CultCacheDocumentValue<TDefinition> {
const formatter = formatterFor(definition);
const decoded = formatter.decode(payload);
return definition.schema.parse(decoded) as CultCacheDocumentValue<TDefinition>;
}
function formatterFor<TDefinition extends CultCacheDocumentDefinition>(
definition: TDefinition,
): CultCacheDocumentFormatter<CultCacheDocumentValue<TDefinition>> {
return (definition.formatter as CultCacheDocumentFormatter<
CultCacheDocumentValue<TDefinition>
> | undefined) ?? {
encode: (value: CultCacheDocumentValue<TDefinition>) => encode(value),
decode: (bytes: Uint8Array) => decode(bytes) as CultCacheDocumentValue<TDefinition>,
};
}

View file

@ -0,0 +1,11 @@
import { decode, encode } from "@msgpack/msgpack";
// The one msgpack entry point above the wire for packages that do not own it: a frame, an envelope, a command, any
// value that is not a document payload. A document payload is read through cultcache-ts's decodeDocumentPayload.
export function encodeWireValue(value: unknown): Uint8Array {
return encode(value);
}
export function decodeWireValue(bytes: ArrayLike<number> | ArrayBufferView | ArrayBuffer): unknown {
return decode(bytes);
}

View file

@ -4,6 +4,7 @@ import { spawnSync } from "node:child_process";
import dgram from "node:dgram";
import fs from "node:fs";
import path from "node:path";
import { documentPayloadBytes, documentPayloadFromBytes, type DocumentPayload } from "@gamecult/cultcache-ts";
import { decode, encode } from "@msgpack/msgpack";
import {
@ -133,18 +134,18 @@ test("rejects a credential file holding more than one envelope", (context) => {
);
});
function presenceSlot(document: { payload: Uint8Array }, slot: number): unknown {
return (decode(document.payload) as unknown[])[slot];
function presenceSlot(document: { payload: DocumentPayload }, slot: number): unknown {
return (decode(documentPayloadBytes(document.payload)) as unknown[])[slot];
}
// Every committed vector is asserted, dead ones included. Stateless sets sign at sequence 1.
for (const set of ["web", "service", "rudp-route"] as const) {
test(`${set}: a fresh signer answers warming, and after reportHealth('active') answers active, byte for byte as the Rust-verified vectors`, (context) => {
freezeClock(context);
assert.deepEqual(Buffer.from(signerFor(context, set).answerRouteObservation(CHALLENGE).payload), fixtureFile(set, "presence-warming.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(signerFor(context, set).answerRouteObservation(CHALLENGE).payload)), fixtureFile(set, "presence-warming.bin"));
const active = signerFor(context, set);
active.reportHealth("active");
assert.deepEqual(Buffer.from(active.answerRouteObservation(CHALLENGE).payload), fixtureFile(set, "presence-active.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(active.answerRouteObservation(CHALLENGE).payload)), fixtureFile(set, "presence-active.bin"));
});
}
@ -152,7 +153,7 @@ test("a shortfall signs the vector Rust correlates to the typed capacity disagre
freezeClock(context);
const signer = signerFor(context, "service", -1);
signer.reportHealth("active");
assert.deepEqual(Buffer.from(signer.answerRouteObservation(CHALLENGE).payload), fixtureFile("service", "presence-active-below-minimum.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(signer.answerRouteObservation(CHALLENGE).payload)), fixtureFile("service", "presence-active-below-minimum.bin"));
});
test("the signer owns the reported health: a fresh signer is warming, reportHealth moves it, the sequence keeps counting", (context) => {
@ -197,7 +198,7 @@ async function startOdinPeer(context: TestContext, endpoint: string) {
context.after(() => peer.close());
const state = {
reject: false,
received: [] as Array<{ schemaVersion?: string; document: { payload: Uint8Array } }>,
received: [] as Array<{ schemaVersion?: string; document: { payload: DocumentPayload } }>,
responseAck: undefined as number | undefined,
failure: undefined as Error | undefined,
};
@ -239,7 +240,7 @@ test("the Odin publisher publishes the signer's document to the endpoint Expecte
await publisher.publish("accepted presence");
assert.equal(peer.received[0]?.schemaVersion, "cultnet.document_put_raw.v0");
const published = decode(peer.received[0]!.document.payload) as unknown[];
const published = decode(documentPayloadBytes(peer.received[0]!.document.payload)) as unknown[];
assert.equal(published[RUNTIME_PRESENCE_SLOT.state], "warming");
assert.equal(published[RUNTIME_PRESENCE_SLOT.publisherSequence], 1);
assert.equal(peer.responseAck, 2, "the response data packet must acknowledge the publisher's reliable schema packet");
@ -257,7 +258,7 @@ test("the Odin publisher leaves a capability shortfall to the Rust correlation i
signer.reportHealth("active");
const publisher = createIdunnRuntimePresencePublisher({ signer, endpoint });
await publisher.publish("below minimum");
const capabilities = (decode(peer.received[0]!.document.payload) as unknown[])[RUNTIME_PRESENCE_SLOT.capabilities] as unknown[][];
const capabilities = (decode(documentPayloadBytes(peer.received[0]!.document.payload)) as unknown[])[RUNTIME_PRESENCE_SLOT.capabilities] as unknown[][];
assert.equal(capabilities[0]![3], expectedFacts("service").capabilities[0]!.capacity - 1);
});
@ -304,16 +305,16 @@ function statefulSigner(context: TestContext, lease = "lease.cc") {
test("stateful: warming, active and degraded match the Rust-verified vectors byte for byte, active and degraded bound to Rust's lease", (context) => {
freezeClock(context);
const signer = statefulSigner(context);
assert.deepEqual(Buffer.from(signer.answerRouteObservation(CHALLENGE).payload), fixtureFile(STATEFUL, "presence-warming.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(signer.answerRouteObservation(CHALLENGE).payload)), fixtureFile(STATEFUL, "presence-warming.bin"));
signer.reportHealth("active");
const active = signer.answerRouteObservation(CHALLENGE);
assert.deepEqual(Buffer.from(active.payload), fixtureFile(STATEFUL, "presence-active.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(active.payload)), fixtureFile(STATEFUL, "presence-active.bin"));
assert.match(String(presenceSlot(active, RUNTIME_PRESENCE_SLOT.writeLeaseSha256)), /^sha256-[0-9a-f]{64}$/);
const degraded = statefulSigner(context);
degraded.answerRouteObservation(CHALLENGE);
degraded.reportHealth("degraded");
assert.deepEqual(Buffer.from(degraded.answerRouteObservation(CHALLENGE).payload), fixtureFile(STATEFUL, "presence-degraded.bin"));
assert.deepEqual(Buffer.from(documentPayloadBytes(degraded.answerRouteObservation(CHALLENGE).payload)), fixtureFile(STATEFUL, "presence-degraded.bin"));
});
test("a warming answered on the route becomes a lease target: Rust's lease naming it is accepted", (context) => {
@ -321,7 +322,7 @@ test("a warming answered on the route becomes a lease target: Rust's lease namin
const signer = statefulSigner(context);
assert.throws(() => signer.assertWriteLease(), /does not match the current Expected incarnation/, "no warming signed yet");
signer.answerRouteObservation(CHALLENGE);
assert.equal(signer.assertWriteLease(), presenceSlot({ payload: fixtureFile(STATEFUL, "presence-active.bin") }, RUNTIME_PRESENCE_SLOT.writeLeaseSha256));
assert.equal(signer.assertWriteLease(), presenceSlot({ payload: documentPayloadFromBytes(fixtureFile(STATEFUL, "presence-active.bin")) }, RUNTIME_PRESENCE_SLOT.writeLeaseSha256));
});
for (const [lease, reason] of [["lease-other-warming.cc", "names another warming digest"], ["lease-other-target.cc", "names another target"]] as const) {

View file

@ -0,0 +1,104 @@
import assert from "node:assert/strict";
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import test from "node:test";
import { z } from "zod";
import { CultCache, codecs, defineDocumentType, documentPayloadBytes, SingleFileMessagePackBackingStore } from "@gamecult/cultcache-ts";
import { CultNetDocumentRegistry, defineCultNetDocumentBinding } from "../src";
const { float32, float64, int32, int64, slots, string } = codecs;
// tests/vectors/numeric-leaves/mixed-record.msgpack is the bytes the C# reference writes for Mixed { "n1", 7, 8, 1.5f, 2.5 }
// (NumericLeafVectorTests). A payload equal to it is a payload the reference reads.
function referenceMixedRecord(): Uint8Array {
for (let dir = __dirname; ; dir = resolve(dir, "..")) {
const file = join(dir, "tests", "vectors", "numeric-leaves", "mixed-record.msgpack");
if (existsSync(file)) return new Uint8Array(readFileSync(file));
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/numeric-leaves not found above the test.");
}
}
const mixedCodec = slots<{ id: string; i: number; l: number | bigint; f: number; d: number }>({
0: ["id", string],
1: ["i", int32],
2: ["l", int64],
3: ["f", float32],
4: ["d", float64],
});
const mixedDefinition = defineDocumentType({
type: "tests.mixed_record",
schemaId: "tests.mixed_record.v1",
schemaName: "tests.mixed_record",
schemaVersion: "tests.mixed_record.v1",
schema: z.object({ id: z.string(), i: z.number(), l: z.union([z.number(), z.bigint()]), f: z.number(), d: z.number() }),
codec: mixedCodec,
name: "id",
});
const value = { id: "n1", i: 7, l: 8, f: 1.5, d: 2.5 };
const hex = (data: Uint8Array) => Buffer.from(data).toString("hex");
function registry() {
return new CultNetDocumentRegistry([defineCultNetDocumentBinding({ definition: mixedDefinition })]);
}
test("a codec document replicates as the bytes the C# reference writes, and a snapshot of it applies on another cache", async () => {
const dir = mkdtempSync(join(tmpdir(), "cultnet-codec-"));
try {
const reference = referenceMixedRecord();
const documents = registry();
const origin = CultCache.builder().withDocumentType(mixedDefinition).withGenericStore(new SingleFileMessagePackBackingStore(join(dir, "origin.cc"))).build();
const target = CultCache.builder().withDocumentType(mixedDefinition).withGenericStore(new SingleFileMessagePackBackingStore(join(dir, "target.cc"))).build();
const put = documents.createRawDocumentPutMessage(documents.get("tests.mixed_record")!, "put-1", "n1", value);
assert.equal(hex(documentPayloadBytes(put.document.payload)), hex(reference), "a raw put carries the reference's record, not a map");
await documents.applyRawDocumentPutMessage(origin, put);
assert.equal(hex(documentPayloadBytes(origin.getRequiredEnvelope(mixedDefinition, "n1").payload)), hex(reference));
const raw = documents.createRawSnapshotResponse(origin, "snapshot-raw");
assert.equal(hex(documentPayloadBytes(raw.documents[0].payload)), hex(reference));
await documents.applyRawSnapshotResponse(target, raw);
assert.deepEqual(target.getRequired(mixedDefinition, "n1"), value);
assert.equal(hex(documentPayloadBytes(target.getRequiredEnvelope(mixedDefinition, "n1").payload)), hex(reference));
// The value-level snapshot decodes the stored record through the codec, so a peer applying it parses a record.
const values = documents.createSnapshotResponse(origin, "snapshot-values");
assert.deepEqual(values.documents[0].payload, value);
const other = CultCache.builder().withDocumentType(mixedDefinition).withGenericStore(new SingleFileMessagePackBackingStore(join(dir, "other.cc"))).build();
await documents.applySnapshotResponse(other, values);
assert.equal(hex(documentPayloadBytes(other.getRequiredEnvelope(mixedDefinition, "n1").payload)), hex(reference));
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test("a definition that carries a codec and a formatter is refused where replication reads it", () => {
const both = {
...mixedDefinition,
formatter: { encode: () => new Uint8Array(), decode: () => value },
} as unknown as typeof mixedDefinition;
const documents = new CultNetDocumentRegistry([defineCultNetDocumentBinding({ definition: both })]);
assert.throws(
() => documents.createRawDocumentPutMessage(documents.get("tests.mixed_record")!, "put-2", "n1", value),
{ message: 'CultCache document type "tests.mixed_record" declares both a codec and a formatter.' },
);
});
test("a document with neither a codec nor a formatter replicates as a map, as before", () => {
const plain = defineDocumentType({
type: "tests.plain_record",
schemaId: "tests.plain_record.v1",
schemaName: "tests.plain_record",
schemaVersion: "tests.plain_record.v1",
schema: z.object({ id: z.string() }),
name: "id",
});
const documents = new CultNetDocumentRegistry([defineCultNetDocumentBinding({ definition: plain })]);
const put = documents.createRawDocumentPutMessage(documents.get("tests.plain_record")!, "put-3", "n1", { id: "n1" });
assert.equal(hex(documentPayloadBytes(put.document.payload)), "81a26964a26e31");
});

View file

@ -6,10 +6,29 @@ import {
decodeCultNetOperationPayload,
decodeCultNetPayload,
} from "cultmesh-browser";
import { defineDocumentType } from "@gamecult/cultcache-ts/document";
// A payload is read through the definition that says how its bytes are laid out. These are plain-map definitions: the
// provider's CultCache types are integer-keyed MessagePack, and the Eve documents are validated by their own parsers.
// CounterState is [counterId, count, receipts].
const counterDocument = defineDocumentType({
type: "sample.counter_state",
schemaId: "sample.counter_state.v1",
schema: {
parse: (value: unknown): [string, number, unknown] => {
if (!Array.isArray(value) || typeof value[0] !== "string" || typeof value[1] !== "number") {
throw new Error("The counter state document is not [counterId, count, receipts].");
}
return [value[0], value[1], value[2]];
},
},
});
const eveDocument = (type: string) => defineDocumentType({ type, schemaId: `${type}.v1`, schema: { parse: (value: unknown) => value } });
const surfaceDocument = eveDocument("gamecult.eve.surface");
const advertisementDocument = eveDocument("gamecult.eve.provider_advertisement");
// CounterState is an integer-keyed MessagePack object: [counterId, count, receipts].
const counterCount = (record: Parameters<typeof decodeCultNetPayload>[0]) =>
decodeCultNetPayload<[string, number, unknown]>(record)[1];
decodeCultNetPayload(record, counterDocument)[1];
declare global {
interface Window {
@ -63,8 +82,12 @@ try {
recordKey: "sample.counter-ui",
subscriptionId: "browser-provider",
});
const provider = parseEveProviderAdvertisement(decodeCultNetPayload(advertisement.current!));
const currentSurface = parseEveSurfaceDocument(decodeCultNetPayload(surface.current!));
// Eve's generated contract type and the lowering's own interface disagree on optional plugin requirements; the
// value is the one parseEveProviderAdvertisement validated.
const provider = parseEveProviderAdvertisement(
decodeCultNetPayload(advertisement.current!, advertisementDocument),
) as unknown as NonNullable<Parameters<typeof renderEveSurface>[2]>["provider"];
const currentSurface = parseEveSurfaceDocument(decodeCultNetPayload(surface.current!, surfaceDocument));
renderEveSurface(currentSurface, host, {
activeSurfaceId: "sample.counter",
clientId: "sample.chromium",
@ -77,10 +100,8 @@ try {
payload: { amount: 1 },
idempotencyKey: window.__sampleCommandId,
});
const receipt = parseEveCommandReceipt(decodeCultNetOperationPayload(response)) as {
receiptId: string;
count: number;
};
const receipt = decodeCultNetOperationPayload<{ receiptId: string; count: number }>(response);
parseEveCommandReceipt(receipt);
window.__sampleReceipt = receipt;
receiptOutput.textContent = receipt.receiptId;
},

View file

@ -0,0 +1,54 @@
// Fails when a TypeScript test file exists that no npm script of its package runs.
//
// A package's `test` script names the compiled files it runs, either by glob or one by one, and a file that is compiled
// and listed nowhere passes every suite while proving nothing: cultcache-ts's decode-document-payload test sat in that
// state behind an explicit two-file list. Every test/**/*.test.ts maps to its dist-test/**/*.test.js, and that path has
// to match an argument of `node --test` in the package's `test` script, or in `test:interop` for test/interop.
import { readdirSync, readFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const rootManifest = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8"));
function testFiles(dir, prefix = "") {
return readdirSync(dir, { withFileTypes: true }).flatMap(entry =>
entry.isDirectory()
? testFiles(join(dir, entry.name), `${prefix}${entry.name}/`)
: entry.name.endsWith(".test.ts") ? [`${prefix}${entry.name}`] : []);
}
// The paths `node --test` is given in a script: the words of its command after `--test` that are not flags.
function nodeTestArguments(script) {
const words = script?.split("&&").map(part => part.trim().split(/\s+/)).find(part => part[0] === "node" && part.includes("--test"));
return words ? words.slice(words.indexOf("--test") + 1).filter(argument => !argument.startsWith("--")) : [];
}
function matches(pattern, path) {
const [directory, file] = [dirname(pattern), pattern.slice(dirname(pattern).length + 1)];
if (directory !== dirname(path)) return false;
return new RegExp(`^${file.split("*").map(part => part.replace(/[.+?^${}()|[\]\\]/g, "\\$&")).join("[^/]*")}$`)
.test(path.slice(directory.length + 1));
}
const unwired = [];
let checked = 0;
for (const workspace of rootManifest.workspaces) {
const packageRoot = join(repoRoot, workspace);
const scripts = JSON.parse(readFileSync(join(packageRoot, "package.json"), "utf8")).scripts ?? {};
for (const file of testFiles(join(packageRoot, "test"))) {
checked += 1;
const compiled = `dist-test/test/${file.replace(/\.ts$/, ".js")}`;
const scriptName = file.startsWith("interop/") ? "test:interop" : "test";
if (!nodeTestArguments(scripts[scriptName]).some(pattern => matches(pattern, compiled))) {
unwired.push(`${workspace}: test/${file} is not run by its package's ${scriptName} script`);
}
}
}
if (checked === 0) throw new Error("The wiring check found no test files; it is looking in the wrong place.");
if (unwired.length > 0) {
console.error(unwired.join("\n"));
process.exit(1);
}
console.log(`${checked} TypeScript test files, all run by their package's script.`);

View file

@ -1,5 +1,6 @@
import { execFileSync } from "node:child_process";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const command = process.argv[2];
if (!command) throw new Error("Usage: node scripts/run-typescript-workspaces.mjs <script>");
@ -26,6 +27,14 @@ const childEnv = process.platform === "win32"
}
: process.env;
// A test file no script runs is a suite that passes while proving nothing, so `test` starts by checking the wiring.
if (command === "test") {
execFileSync(process.execPath, [join(dirname(fileURLToPath(import.meta.url)), "check-typescript-tests-wired.mjs")], {
env: childEnv,
stdio: "inherit",
});
}
for (const workspace of workspaces) {
execFileSync(process.execPath, [npmCli, "--workspace", workspace, "run", command], {
env: childEnv,

View file

@ -78,9 +78,16 @@ try {
writeFileSync(join(consumerRoot, "runtime-smoke.cjs"), `
const assert = require("node:assert/strict");
const cache = require("@gamecult/cultcache-ts");
const documentEntry = require("@gamecult/cultcache-ts/document");
const net = require("cultnet-ts");
const mesh = require("cultmesh-ts");
assert.equal(typeof cache.defineDocumentType, "function");
// The document owner is also an entry of its own, which the browser client imports without the Node-only cache.
assert.equal(documentEntry.decodeDocumentPayload, cache.decodeDocumentPayload);
assert.equal("resolveDocumentFormatter" in cache, false);
assert.equal("resolveDocumentFormatter" in documentEntry, false);
assert.equal("codecFormatter" in cache.codecs, false);
assert.equal("codecFormatter" in documentEntry, false);
assert.equal(typeof net.CultNetPeer, "function");
assert.equal(typeof mesh.CultMesh, "function");
assert.equal(typeof mesh.CultMeshProviderSession, "function");
@ -101,6 +108,7 @@ assert.equal("putDocument" in CultMeshBrowserClient.prototype, false);
writeFileSync(join(consumerRoot, "types-smoke.ts"), `
import { defineDocumentType } from "@gamecult/cultcache-ts";
import { decodeDocumentPayload, type DocumentPayload } from "@gamecult/cultcache-ts/document";
import { CultNetPeer } from "cultnet-ts";
import {
CultMesh,
@ -112,6 +120,8 @@ import {
const point: CultMeshVec2 = CultMesh.vec2(3, 4);
const defineDocument: typeof defineDocumentType = defineDocumentType;
const decodeOwner: typeof decodeDocumentPayload = decodeDocumentPayload;
let opaque: DocumentPayload | undefined;
const peerType: typeof CultNetPeer = CultNetPeer;
const providerSessionType: typeof CultMeshProviderSession = CultMeshProviderSession;
const providerIdentity: CultMeshProviderIdentity = {
@ -123,6 +133,8 @@ const providerIdentity: CultMeshProviderIdentity = {
let providerTransport: CultMeshProviderTransport | undefined;
void point;
void defineDocument;
void decodeOwner;
void opaque;
void peerType;
void providerSessionType;
void providerIdentity;
@ -142,6 +154,7 @@ void providerTransport;
], { cwd: consumerRoot, stdio: "inherit" });
writeFileSync(join(consumerRoot, "browser-types-smoke.mts"), `
import { decodeDocumentPayload } from "@gamecult/cultcache-ts/document";
import {
CultMeshBrowserClient,
type CultMeshBrowserIdentity,
@ -157,9 +170,11 @@ const lease: CultMeshRawDocumentLeaseOptions = {
recordKey: "smoke:main",
};
const clientType: typeof CultMeshBrowserClient = CultMeshBrowserClient;
const decodeOwner: typeof decodeDocumentPayload = decodeDocumentPayload;
void identity;
void lease;
void clientType;
void decodeOwner;
`);
execFileSync(process.execPath, [
tsc,
@ -172,6 +187,29 @@ void clientType;
"browser-types-smoke.mts",
], { cwd: consumerRoot, stdio: "inherit" });
// The formatter that reads a payload around its schema is in no entry: importing it from the document entry does not compile.
writeFileSync(join(consumerRoot, "formatter-refused.mts"), `
// @ts-expect-error resolveDocumentFormatter is not exported from the document entry
import { resolveDocumentFormatter } from "@gamecult/cultcache-ts/document";
void resolveDocumentFormatter;
import { codecs } from "@gamecult/cultcache-ts";
// @ts-expect-error codecFormatter is not exported from the document entry
import { codecFormatter } from "@gamecult/cultcache-ts/document";
void codecFormatter;
// @ts-expect-error codecFormatter is not a member of the root's codecs namespace
void codecs.codecFormatter;
`);
execFileSync(process.execPath, [
tsc,
"--noEmit",
"--strict",
"--target", "ES2022",
"--module", "NodeNext",
"--moduleResolution", "NodeNext",
"--skipLibCheck",
"formatter-refused.mts",
], { cwd: consumerRoot, stdio: "inherit" });
console.log("TypeScript package closure smoke: passed");
for (const tarball of tarballs) console.log(`Packed: ${tarball}`);
} finally {

View file

@ -2,7 +2,7 @@ import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { existsSync } from "node:fs";
import { createServer as createHttpServer } from "node:http";
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { createServer as createNetServer } from "node:net";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
@ -38,8 +38,33 @@ let browser;
try {
const tsc = join(repoRoot, "node_modules", "typescript", "bin", "tsc");
await run(process.execPath, [tsc, "-p", join(repoRoot, "packages", "cultcache-ts", "tsconfig.json"), "--pretty", "false"]);
await run(process.execPath, [tsc, "-p", join(repoRoot, "packages", "cultnet-ts", "tsconfig.json"), "--pretty", "false"]);
await run(process.execPath, [tsc, "-p", join(repoRoot, "packages", "cultmesh-browser", "tsconfig.json"), "--pretty", "false"]);
// esbuild strips types, so a package signature change would otherwise surface as a browser error. The page is
// type-checked against the same module map the bundle uses.
const moduleMap = {
"@gamecult/eve-contracts": join(eveRoot, "packages", "eve-contracts", "src", "index.ts"),
"@gamecult/eve-browser-lowering": join(eveRoot, "packages", "eve-browser-lowering", "src", "index.ts"),
"cultmesh-browser": join(repoRoot, "packages", "cultmesh-browser", "src", "index.ts"),
"cultnet-ts/contracts": join(repoRoot, "packages", "cultnet-ts", "src", "contracts.ts"),
};
const sampleTsconfig = join(workRoot, "tsconfig.json");
await writeFile(sampleTsconfig, JSON.stringify({
compilerOptions: {
target: "ES2022",
module: "ES2022",
moduleResolution: "Bundler",
lib: ["ES2022", "ESNext.Disposable", "DOM", "DOM.Iterable"],
strict: true,
noEmit: true,
skipLibCheck: true,
types: [],
paths: Object.fromEntries(Object.entries(moduleMap).map(([name, target]) => [name, [target]])),
},
files: [join(sampleRoot, "browser", "main.ts")],
}));
await run(process.execPath, [tsc, "-p", sampleTsconfig, "--pretty", "false"]);
await build({
entryPoints: [join(sampleRoot, "browser", "main.ts")],
outfile: bundlePath,
@ -48,12 +73,7 @@ try {
platform: "browser",
target: "es2022",
sourcemap: true,
alias: {
"@gamecult/eve-contracts": join(eveRoot, "packages", "eve-contracts", "src", "index.ts"),
"@gamecult/eve-browser-lowering": join(eveRoot, "packages", "eve-browser-lowering", "src", "index.ts"),
"cultmesh-browser": join(repoRoot, "packages", "cultmesh-browser", "src", "index.ts"),
"cultnet-ts/contracts": join(repoRoot, "packages", "cultnet-ts", "src", "contracts.ts"),
},
alias: moduleMap,
logLevel: "warning",
});
const bundle = await readFile(bundlePath, "utf8");
@ -222,10 +242,15 @@ function startDotnet(arguments_) {
waiter.reject(new Error(`dotnet sample exited ${code}\n${output}\n${error}`));
}
});
// A line is complete once its newline has arrived: a stdout chunk can end inside a line, and the 10,000-operation
// benchmark line was parsed as JSON half-written.
function completeLines() {
return output.split(/\r?\n/u).slice(0, -1);
}
function settle() {
for (let index = waiters.length - 1; index >= 0; index--) {
const waiter = waiters[index];
const line = output.split(/\r?\n/u).find(value => value.startsWith(waiter.prefix));
const line = completeLines().find(value => value.startsWith(waiter.prefix));
if (!line) continue;
waiters.splice(index, 1);
clearTimeout(waiter.timer);
@ -238,7 +263,7 @@ function startDotnet(arguments_) {
process.stdin.write(`${line}\n`);
},
waitFor(prefix, timeoutMs = 30_000) {
const existing = output.split(/\r?\n/u).find(value => value.startsWith(prefix));
const existing = completeLines().find(value => value.startsWith(prefix));
if (existing) return Promise.resolve(existing.slice(prefix.length));
return new Promise((resolve, reject) => {
const waiter = {

View file

@ -0,0 +1,286 @@
#nullable enable
using System;
using System.Buffers;
using System.Buffers.Binary;
using System.Globalization;
using System.IO;
using System.Linq;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.Tests
{
// The numeric-leaf golden in tests/vectors/numeric-leaves: what the reference reads and refuses for an int, long, float
// and double slot, and the bytes it writes for them. The C# reference is the only writer (CULTCACHE_WRITE_VECTORS=1);
// every other run reads the committed bytes and compares them byte for byte. Other runtimes are held to these files.
public class NumericLeafVectorTests
{
private const string WriteVariable = "CULTCACHE_WRITE_VECTORS";
private static readonly string[] Leaves = { "int32", "int64", "float32", "float64" };
[MessagePackObject] public sealed class Int32Slot { [Key(0)] public int V; }
[MessagePackObject] public sealed class Int64Slot { [Key(0)] public long V; }
[MessagePackObject] public sealed class Float32Slot { [Key(0)] public float V; }
[MessagePackObject] public sealed class Float64Slot { [Key(0)] public double V; }
// A record of every leaf kind, the shape a replicated document has.
[MessagePackObject]
public sealed class Mixed
{
[Key(0)] public string Id = string.Empty;
[Key(1)] public int I;
[Key(2)] public long L;
[Key(3)] public float F;
[Key(4)] public double D;
}
private static bool Writing => Environment.GetEnvironmentVariable(WriteVariable) == "1";
private static readonly MessagePackSerializerOptions Options = CultDocumentMessagePackSerialization.OptionsFor(typeof(NumericLeafVectorTests).Assembly);
private static string VectorPath(string name)
{
for (var dir = new DirectoryInfo(TestContext.CurrentContext.TestDirectory); dir != null; dir = dir.Parent)
{
var vectors = Path.Combine(dir.FullName, "tests", "vectors");
if (Directory.Exists(vectors)) return Path.Combine(vectors, "numeric-leaves", name);
}
throw new DirectoryNotFoundException($"tests/vectors not found above {TestContext.CurrentContext.TestDirectory}.");
}
// Under the write variable the reference's bytes become the file; otherwise they must equal the committed file.
private static void Check(string name, byte[] reference)
{
var path = VectorPath(name);
if (Writing)
{
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
File.WriteAllBytes(path, reference);
return;
}
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
Assert.That(Convert.ToHexString(reference), Is.EqualTo(Convert.ToHexString(File.ReadAllBytes(path))), $"vector {name}");
}
// ---- reads: a one-slot record [value], every value fed to every leaf ----
// The value token of each read case, as MessagePack hex: integers of every width and edge, floats, and kinds no leaf reads.
private static readonly string[] ReadValues =
{
"05", "ff", "ccc8", "cd0100", "ceb2d05e00", "d0ce", "d18000", "d280000000", "d38000000000000000",
"cf0020000000000001", "cfffffffffffffffff", "cf8000000000000000", "d30020000000000001", "d3fffffffffffffff0",
// integers beyond 2^53 that a float32 slot must round once: past a tie, on a tie to even, on a tie to odd, an odd value just past exact, negative
"d31000001000000001", "cf8000008000000001", "cf8000008000000000", "cf8000018000000000", "cf8000010000000001", "cf8000010000000000", "d3efffffe800000001", "d3effffff000000001",
// an integer a float32 cannot hold that a reader holds as a plain number: 2^24 + 1 rounds to 2^24
"ce01000001",
"ca3fc00000", "cb3ff8000000000000", "cb401c000000000000", "cb7ff8000000000000", "ca7f800000", "ca7f800001", "cb7e37e43c8800759c",
"cb47efffffe0000000", "cb3fb999999999999a", "ca3dcccccd", "cb8000000000000000",
"c0", "c3", "a161", "c40100", "90", "80",
};
private static (bool Accepted, byte[] Bits) Read(string leaf, byte[] payload)
{
try
{
var bits = new byte[8];
switch (leaf)
{
case "int32": BinaryPrimitives.WriteInt64BigEndian(bits, MessagePackSerializer.Deserialize<Int32Slot>(payload, Options).V); break;
case "int64": BinaryPrimitives.WriteInt64BigEndian(bits, MessagePackSerializer.Deserialize<Int64Slot>(payload, Options).V); break;
case "float32": BinaryPrimitives.WriteInt64BigEndian(bits, BitConverter.DoubleToInt64Bits(MessagePackSerializer.Deserialize<Float32Slot>(payload, Options).V)); break;
default: BinaryPrimitives.WriteInt64BigEndian(bits, BitConverter.DoubleToInt64Bits(MessagePackSerializer.Deserialize<Float64Slot>(payload, Options).V)); break;
}
return (true, bits);
}
catch (Exception)
{
return (false, Array.Empty<byte>());
}
}
private static byte[] Payload(string valueHex) => Convert.FromHexString("91" + valueHex);
// reads.msgpack: an array of [leaf, payload (bin), accepted, bits (bin of 8: the int64 or the double's IEEE bits, big-endian; empty when refused)].
private static byte[] ReadsVector()
{
var buffer = new ArrayBufferWriter<byte>();
var writer = new MessagePackWriter(buffer);
writer.WriteArrayHeader(Leaves.Length * ReadValues.Length);
foreach (var leaf in Leaves)
foreach (var value in ReadValues)
{
var payload = Payload(value);
var (accepted, bits) = Read(leaf, payload);
writer.WriteArrayHeader(4);
writer.Write(leaf);
writer.Write(payload);
writer.Write(accepted);
writer.Write(bits);
}
writer.Flush();
return buffer.WrittenSpan.ToArray();
}
// ---- writes: the reference's own bytes for a value ----
// text is a decimal for an integer leaf and the double's IEEE bits (hex) for a float leaf.
private static readonly (string Leaf, string Text)[] WriteValues =
{
("int32", "0"), ("int32", "1"), ("int32", "127"), ("int32", "128"), ("int32", "255"), ("int32", "256"), ("int32", "65535"), ("int32", "65536"),
("int32", "-1"), ("int32", "-32"), ("int32", "-33"), ("int32", "-128"), ("int32", "-129"), ("int32", "-32768"), ("int32", "-32769"),
("int32", "2147483647"), ("int32", "-2147483648"),
("int64", "0"), ("int64", "127"), ("int64", "128"), ("int64", "4294967295"), ("int64", "4294967296"), ("int64", "9007199254740991"),
("int64", "-9007199254740991"), ("int64", "-2147483649"), ("int64", "9007199254740994"), ("int64", "9223372036854775807"),
("int64", "-9223372036854775808"), ("int64", "9007199254740993"), ("int64", "-9007199254740993"),
("float32", "3ff8000000000000"), ("float32", "3fb999999999999a"), ("float32", "0000000000000000"), ("float32", "8000000000000000"),
("float32", "4008000000000000"), ("float32", "46293e5939a08cea"), ("float32", "7ff0000000000000"), ("float32", "7ff8000000000000"),
("float64", "3ff8000000000000"), ("float64", "3fb999999999999a"), ("float64", "0000000000000000"), ("float64", "8000000000000000"),
("float64", "4008000000000000"), ("float64", "7e37e43c8800759c"), ("float64", "7ff0000000000000"), ("float64", "7ff8000000000000"),
};
private static byte[] Write(string leaf, string text)
{
double Double() => BitConverter.Int64BitsToDouble((long)ulong.Parse(text, NumberStyles.HexNumber, CultureInfo.InvariantCulture));
return leaf switch
{
"int32" => MessagePackSerializer.Serialize(new Int32Slot { V = int.Parse(text, CultureInfo.InvariantCulture) }, Options),
"int64" => MessagePackSerializer.Serialize(new Int64Slot { V = long.Parse(text, CultureInfo.InvariantCulture) }, Options),
"float32" => MessagePackSerializer.Serialize(new Float32Slot { V = (float)Double() }, Options),
_ => MessagePackSerializer.Serialize(new Float64Slot { V = Double() }, Options),
};
}
// writes.msgpack: an array of [leaf, text, bytes (bin): the one-slot record the reference writes for the value].
private static byte[] WritesVector()
{
var buffer = new ArrayBufferWriter<byte>();
var writer = new MessagePackWriter(buffer);
writer.WriteArrayHeader(WriteValues.Length);
foreach (var (leaf, text) in WriteValues)
{
writer.WriteArrayHeader(3);
writer.Write(leaf);
writer.Write(text);
writer.Write(Write(leaf, text));
}
writer.Flush();
return buffer.WrittenSpan.ToArray();
}
// mixed-record.msgpack: the reference's bytes for Mixed { "n1", 7, 8, 1.5f, 2.5 }, which other runtimes must write identically.
private static byte[] MixedRecord() => MessagePackSerializer.Serialize(new Mixed { Id = "n1", I = 7, L = 8, F = 1.5f, D = 2.5 }, Options);
// float32-nan.msgpack: an array of [payload (bin), bytes (bin)]: a one-slot float32 record holding an infinity or a NaN,
// and the bytes the reference writes after reading it. A signalling NaN (quiet bit clear) comes back unchanged.
private static readonly string[] Float32NanValues = { "ca7f800000", "ca7f800001", "ca7fa00000", "ca7fc00001", "ca7fe00000", "caff800001" };
private static byte[] Float32NanVector()
{
var buffer = new ArrayBufferWriter<byte>();
var writer = new MessagePackWriter(buffer);
writer.WriteArrayHeader(Float32NanValues.Length);
foreach (var value in Float32NanValues)
{
var payload = Payload(value);
writer.WriteArrayHeader(2);
writer.Write(payload);
writer.Write(MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Float32Slot>(payload, Options), Options));
}
writer.Flush();
return buffer.WrittenSpan.ToArray();
}
// ---- tests ----
[Test]
public void ReadsVectorMatches() => Check("reads.msgpack", ReadsVector());
[Test]
public void Float32NanVectorMatches() => Check("float32-nan.msgpack", Float32NanVector());
[Test]
public void Float32SignallingNanIsReadAndWrittenBitForBit()
{
foreach (var value in new[] { "ca7f800001", "ca7fa00000", "caff800001" })
{
var payload = Payload(value);
var again = MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Float32Slot>(payload, Options), Options);
Assert.That(Convert.ToHexString(again), Is.EqualTo(Convert.ToHexString(payload)), value);
}
}
[Test]
public void WritesVectorMatches() => Check("writes.msgpack", WritesVector());
[Test]
public void MixedRecordVectorMatches()
{
var written = MixedRecord();
Assert.That(Convert.ToHexString(written), Is.EqualTo("95A26E310708CA3FC00000CB4004000000000000"));
Check("mixed-record.msgpack", written);
var read = MessagePackSerializer.Deserialize<Mixed>(written, Options);
Assert.That((read.Id, read.I, read.L, read.F, read.D), Is.EqualTo(("n1", 7, 8L, 1.5f, 2.5)));
}
[Test]
public void ReadsCoverEveryLeafAndBothOutcomes()
{
var rows = Leaves.SelectMany(leaf => ReadValues.Select(value => (leaf, value, result: Read(leaf, Payload(value))))).ToList();
Assert.That(rows, Has.Count.EqualTo(Leaves.Length * ReadValues.Length));
foreach (var leaf in Leaves)
{
Assert.That(rows.Count(row => row.leaf == leaf && row.result.Accepted), Is.GreaterThan(5), $"{leaf} accepts");
Assert.That(rows.Count(row => row.leaf == leaf && !row.result.Accepted), Is.GreaterThan(5), $"{leaf} refuses");
}
}
// One rounding from int64 or uint64 to float, as the reference does: 2^60 + 2^36 + 1 is past the tie, so it rounds up.
[TestCase("float32", "d31000001000000001", "43b0000020000000")]
[TestCase("float32", "cf8000008000000001", "43e0000020000000")]
[TestCase("float32", "cf8000008000000000", "43e0000000000000")]
[TestCase("float32", "cf8000018000000000", "43e0000040000000")]
[TestCase("float32", "cf8000010000000001", "43e0000020000000")]
[TestCase("float32", "cf8000010000000000", "43e0000020000000")]
[TestCase("float64", "cf0020000000000001", "4340000000000000")]
[TestCase("int64", "cf0020000000000001", "0020000000000001")]
public void ReadsRoundExactlyAsTheReferenceDoes(string leaf, string value, string bitsHex)
{
var (accepted, bits) = Read(leaf, Payload(value));
Assert.That(accepted, Is.True);
Assert.That(Convert.ToHexString(bits).ToLowerInvariant(), Is.EqualTo(bitsHex));
}
[Test]
public void ValuesOutsideTheLeafAreRefused()
{
Assert.That(Read("int32", Payload("ceb2d05e00")).Accepted, Is.False);
Assert.That(Read("int64", Payload("cfffffffffffffffff")).Accepted, Is.False);
Assert.That(Read("int32", Payload("ca3fc00000")).Accepted, Is.False);
}
[Test]
public void EveryWrittenValueIsReadBackAndWrittenAgainUnchanged()
{
foreach (var (leaf, text) in WriteValues)
{
var written = Write(leaf, text);
Assert.That(Read(leaf, written).Accepted, Is.True, $"{leaf} {text}");
var again = leaf switch
{
"int32" => MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Int32Slot>(written, Options), Options),
"int64" => MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Int64Slot>(written, Options), Options),
"float32" => MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Float32Slot>(written, Options), Options),
_ => MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Float64Slot>(written, Options), Options),
};
Assert.That(Convert.ToHexString(again), Is.EqualTo(Convert.ToHexString(written)), $"{leaf} {text}");
}
}
}
}

View file

@ -0,0 +1,124 @@
#nullable enable
using System;
using System.Buffers;
using System.Collections.Generic;
using System.IO;
using GameCult.Caching.MessagePack;
using MessagePack;
using NUnit.Framework;
namespace GameCult.Caching.Tests
{
// The slot golden in tests/vectors/slot-nil: what the reference reads and rewrites for a nil or a short record in each
// kind of slot. The C# reference is the only writer (CULTCACHE_WRITE_VECTORS=1); every other run reads the committed
// bytes and compares them byte for byte. Other runtimes are held to these files.
public class SlotNilVectorTests
{
private const string WriteVariable = "CULTCACHE_WRITE_VECTORS";
[MessagePackObject] public sealed class Inner { [Key(0)] public string X = string.Empty; }
// One slot of each kind: a string, a value type, a byte[], a record and a list.
[MessagePackObject]
public sealed class Probe
{
[Key(0)] public string S = string.Empty;
[Key(1)] public int I;
[Key(2)] public byte[] B = Array.Empty<byte>();
[Key(3)] public Inner? R;
[Key(4)] public List<string> L = new();
}
private static bool Writing => Environment.GetEnvironmentVariable(WriteVariable) == "1";
private static readonly MessagePackSerializerOptions Options = CultDocumentMessagePackSerialization.OptionsFor(typeof(SlotNilVectorTests).Assembly);
// [name, payload]: the whole case table, in file order.
private static readonly (string Name, string Hex)[] Cases =
{
("full", "95a16107c402616291a17890"),
("nil-string", "95c007c4026162c090"),
("nil-bytes", "95a16107c0c090"),
("nil-record", "95a16107c4026162c090"),
("nil-list", "95a16107c4026162c0c0"),
("nil-int", "95a161c0c4026162c090"),
("bytes-str", "95a16107a3616263c090"),
("bytes-str-euro", "95a16107a3e282acc090"),
("bytes-bin", "95a16107c403616263c090"),
("short-0", "90"),
("short-1", "91a161"),
("short-2", "92a16107"),
("short-3", "93a16107c4026162"),
("short-3-nil-string", "93c007c4026162"),
("short-3-nil-int", "93a161c0c4026162"),
};
private static string VectorPath(string name)
{
for (var dir = new DirectoryInfo(TestContext.CurrentContext.TestDirectory); dir != null; dir = dir.Parent)
{
var vectors = Path.Combine(dir.FullName, "tests", "vectors");
if (Directory.Exists(vectors)) return Path.Combine(vectors, "slot-nil", name);
}
throw new DirectoryNotFoundException($"tests/vectors not found above {TestContext.CurrentContext.TestDirectory}.");
}
private static void Check(string name, byte[] reference)
{
var path = VectorPath(name);
if (Writing)
{
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
File.WriteAllBytes(path, reference);
return;
}
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
Assert.That(Convert.ToHexString(reference), Is.EqualTo(Convert.ToHexString(File.ReadAllBytes(path))), $"vector {name}");
}
private static (bool Accepted, byte[] Rewritten) Read(byte[] payload)
{
try
{
return (true, MessagePackSerializer.Serialize(MessagePackSerializer.Deserialize<Probe>(payload, Options), Options));
}
catch (MessagePackSerializationException)
{
return (false, Array.Empty<byte>());
}
}
// reads.msgpack: an array of [name, payload (bin), accepted, rewritten (bin: what the reference writes after reading it; empty when refused)].
private static byte[] ReadsVector()
{
var buffer = new ArrayBufferWriter<byte>();
var writer = new MessagePackWriter(buffer);
writer.WriteArrayHeader(Cases.Length);
foreach (var (name, hex) in Cases)
{
var payload = Convert.FromHexString(hex);
var (accepted, rewritten) = Read(payload);
writer.WriteArrayHeader(4);
writer.Write(name);
writer.Write(payload);
writer.Write(accepted);
writer.Write(rewritten);
}
writer.Flush();
return buffer.WrittenSpan.ToArray();
}
[Test]
public void ReadsVectorMatches() => Check("reads.msgpack", ReadsVector());
[Test]
public void NilIsAcceptedInAReferenceSlotAndRefusedInAValueSlot()
{
Assert.That(Read(Convert.FromHexString("95c007c4026162c090")).Accepted, Is.True);
Assert.That(Read(Convert.FromHexString("95a161c0c4026162c090")).Accepted, Is.False);
}
}
}

View file

@ -0,0 +1,16 @@
# numeric-leaves vectors
Written only by the C# reference, `tests/GameCult.Caching.Tests/NumericLeafVectorTests.cs`, with
`CULTCACHE_WRITE_VECTORS=1`. Every other run reads these files and compares them byte for byte; a changed vector is a
reviewed commit. Other runtimes decode them and are held to the same answers.
A document with one slot, `[value]`, whose slot is a C# `int`, `long`, `float` or `double` (leaf `int32`, `int64`,
`float32`, `float64`).
- `reads.msgpack`: an array of `[leaf, payload, accepted, bits]`. `payload` (bin) is the one-slot record fed to the
reference; `accepted` is whether it read; `bits` (bin, 8 bytes, big-endian) is the int64 it read, or the IEEE bits of
the double it read (a float32 widened), and empty when refused. Every value is fed to every leaf.
- `writes.msgpack`: an array of `[leaf, text, bytes]`. `text` is a decimal for an integer leaf and the double's IEEE
bits in hex for a float leaf; `bytes` (bin) is the one-slot record the reference writes for that value.
- `float32-nan.msgpack`: an array of `[payload, bytes]`. `payload` (bin) is a one-slot `float` record holding an infinity
or a NaN; `bytes` (bin) is what the reference writes after reading it. It keeps every bit, a signalling NaN's included.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -0,0 +1,12 @@
# slot-nil vectors
Written only by the C# reference, `tests/GameCult.Caching.Tests/SlotNilVectorTests.cs`, with
`CULTCACHE_WRITE_VECTORS=1`. Every other run reads this file and compares it byte for byte; a changed vector is a
reviewed commit. Other runtimes decode it and are held to the same answers.
A record of five slots: `S` string, `I` int, `B` byte[], `R` a record of one string, `L` List of string.
- `reads.msgpack`: an array of `[name, payload, accepted, rewritten]`. `payload` (bin) is the record fed to the
reference; `accepted` is whether it read; `rewritten` (bin) is what the reference writes after reading it, and empty
when refused. It covers a nil in each kind of slot, a MessagePack str in the byte[] slot, and records short of their
last slots.

Binary file not shown.