TypeScript: a record with unread slots is held read-only (slot-readonly-ts) #128
12 changed files with 976 additions and 238 deletions
|
|
@ -20,6 +20,7 @@ All notable changes to `@gamecult/cultcache-ts` are recorded here, following
|
|||
- A store whose canonical-schema catalog slot is not a string is refused as `StoreUnreadableError`.
|
||||
- The cache holds one record per key (variants:ruling:cache-refuses-retype). A `put` or `putEnvelope` that lands a second type under a key the cache holds, or under a key a store carries foreign, is refused with `SchemaConflictError` naming the key, even with no store (a memory-only cache used to hold both types at the key). A type that owns the held record's schema id (as for the store) replaces it, unless the held record is in a store other than the written type's home: that rename is refused with `SchemaConflictError` (remove the record first). Two stores holding one key, registered or foreign, refuse the load and admit nothing. A store's `delete` removes the record whose stored schema id the removal's type owns, whatever the arrived record's `type` string.
|
||||
- A store holding a record no registered type claims now loads (it used to throw "No schema is registered..."): the cache does not serve the record, lists it in `foreignRecords()`, keeps its bytes through every write, and refuses a `put` or a `delete` at its key with `SchemaConflictError` (variants:ruling:unheld-removal-refused-at-commit). Registering a type that claims its schema id serves it at the next `pullAllBackingStores`.
|
||||
- A record whose payload carries a slot past its reader's declared slots, at any depth (the record, a nested record, a list element, a union arm, a unit arm), is now held read-only: it loads and reads, and every `put`, `putEnvelope`, `update` or `delete` of its key is refused with `SchemaConflictError` before a store is touched (cultlib-gaps:ruling:extra-trailing-slots, as the C# reference). A load used to skip the extra slots silently and a later write shed them. The refusal names the schema, the key, the type that was read with extra slots and both slot counts, never a slot value, and carries them as `SchemaConflictError.unreadSlots`. An untouched record keeps its bytes through writes of other keys. Missing trailing slots keep their declared defaults and mark nothing. A `put` never marks (the value is encoded by this runtime); a `putEnvelope` of a payload carrying extra slots lands its bytes and holds the record read-only. A mark at depth 0 is consented to only when the record arrived under a schema id other than the reader's own that the reader lists compatible, and that id's catalog entry lists those slots (a writer that adds a slot and keeps the reader's id is never consented to) (`unreadSlots.explained`); every other mark refuses whatever is declared. New: `CultCache.isWritable(definition, key)` (live, advisory), `CultCache.unreadSlots(definition, key)` and the exported `CultCacheUnreadSlots`. Documents with no slot codec (a plain map, a user formatter) never mark. A `delete` is by key: a type that owns the schema id of the record held at the key as another type removes that one record (one store call, the cache dropped once the store has removed it), and a type that does not own it is refused with `SchemaConflictError`, as a `put` is; `isWritable` answers both. Implied semver: a minor bump (0.16.0) while the package is 0.x; a major one at 1.0 and after.
|
||||
|
||||
### Added
|
||||
|
||||
|
|
|
|||
|
|
@ -371,8 +371,11 @@ const sceneDocument = defineDocumentType({
|
|||
});
|
||||
```
|
||||
|
||||
- A slot is `[name, codec, default?]`. A gap is written nil and skipped on read; an extra trailing slot is skipped; a
|
||||
slot missing from the payload (or nil, when a default is declared) takes its default, and is refused without one.
|
||||
- A slot is `[name, codec, default?]`. A gap is written nil and skipped on read; an extra trailing slot is read past and
|
||||
marks the record read-only (every write or removal of its key is refused with `SchemaConflictError`, `cache.isWritable`
|
||||
asks first). The one writable case is a depth-0 extra slot of a record that arrived under a schema id other than the reader's own, which the
|
||||
reader lists in `compatibleSchemaIds`, whose catalog entry lists those slots: declaring that id consents to the loss, and a write restamps
|
||||
the record. A nested slot, or a writer that keeps the reader's own id, is never consented to. A slot missing from the payload (or nil, when a default is declared) takes its default, and is refused without one.
|
||||
- A union refuses an undeclared arm key on read, naming the union, the key and the declared keys. Nothing falls back
|
||||
to a default arm. An arm with no slots is `codecs.unit` and is written `[key, []]`.
|
||||
- A refusal names the union, the field and the slot; it never echoes the value it refused.
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ export type {
|
|||
CultCacheIndexScalar,
|
||||
InferCultCacheSchemaValue,
|
||||
CultCacheStoreRegistration,
|
||||
CultCacheUnreadSlots,
|
||||
PushAllOptions,
|
||||
} from "./types";
|
||||
export type {
|
||||
|
|
|
|||
|
|
@ -198,6 +198,66 @@ type SlotSpec<TValue, TName extends string = string> =
|
|||
|
||||
type SlotsSpec<TValue> = Readonly<Record<number, { [K in keyof TValue & string]: SlotSpec<TValue[K], K> }[keyof TValue & string]>>;
|
||||
|
||||
// The slots a payload decode read past, noted by `slots` and the unit arm while the one decode entry (decodeNoting) has a scope
|
||||
// open, and read out by that entry. Module-scoped as the reference's is thread-bound: a decode is synchronous, and a decode
|
||||
// outside the entry (a codec called directly, a CultMesh read) notes nothing. No codec signature carries it.
|
||||
export interface NotedSlots {
|
||||
/** The first type read with extra slots, outermost first. */
|
||||
readonly typeName: string;
|
||||
readonly slotCount: number;
|
||||
readonly declaredCount: number;
|
||||
readonly occurrences: number;
|
||||
/** True when every noted object was the document's own: a union adds no depth, each record or arm inside a field adds one. */
|
||||
readonly depth0Only: boolean;
|
||||
}
|
||||
|
||||
interface SlotScope {
|
||||
depth: number;
|
||||
found?: { typeName: string; slotCount: number; declaredCount: number; occurrences: number; depth0Only: boolean };
|
||||
}
|
||||
|
||||
let scope: SlotScope | undefined;
|
||||
|
||||
/** The one decode entry: runs the decode with a scope open and returns its value with what it noted. */
|
||||
export function decodeNoting<T>(decode: () => T): { value: T; unreadSlots?: NotedSlots } {
|
||||
const outer = scope;
|
||||
const mine: SlotScope = { depth: 0 };
|
||||
scope = mine;
|
||||
try {
|
||||
const value = decode();
|
||||
return mine.found ? { value, unreadSlots: mine.found } : { value };
|
||||
} finally {
|
||||
scope = outer;
|
||||
}
|
||||
}
|
||||
|
||||
function noteExtraSlots(typeName: string, slotCount: number, declaredCount: number): void {
|
||||
if (!scope) return;
|
||||
const found = scope.found;
|
||||
if (found) {
|
||||
found.occurrences += 1;
|
||||
found.depth0Only &&= scope.depth === 0;
|
||||
} else {
|
||||
scope.found = { typeName, slotCount, declaredCount, occurrences: 1, depth0Only: scope.depth === 0 };
|
||||
}
|
||||
}
|
||||
|
||||
// What a record inside a field reads at: one deeper than the record holding the field.
|
||||
function nested<T>(read: () => T): T {
|
||||
if (!scope) return read();
|
||||
scope.depth += 1;
|
||||
try {
|
||||
return read();
|
||||
} finally {
|
||||
scope.depth -= 1;
|
||||
}
|
||||
}
|
||||
|
||||
// The reference's type name for a mark: the type, not its namespace or the type it is nested in.
|
||||
function simpleTypeName(typeName: string | undefined): string {
|
||||
return typeName === undefined ? "record" : typeName.slice(Math.max(typeName.lastIndexOf("."), typeName.lastIndexOf("+")) + 1);
|
||||
}
|
||||
|
||||
const INT32_MIN = -(2 ** 31);
|
||||
const INT32_MAX = 2 ** 31 - 1;
|
||||
|
||||
|
|
@ -232,7 +292,7 @@ function pascalCase(name: string): string {
|
|||
}
|
||||
|
||||
// A positional record: slot `key` holds field `name`. Gaps are written nil and skipped on read, extra trailing slots
|
||||
// are skipped, and a slot missing from the payload takes its declared default or is refused. A default fills only a
|
||||
// are read past and noted (decodeNoting): the record that carried them is read-only, never rewritten without them, and a slot missing from the payload takes its declared default or is refused. A default fills only a
|
||||
// missing trailing slot, never a nil: as in the reference, a nil in a reference slot (string, bytes, record, union,
|
||||
// list) reads as null and writes back as nil, so a record the reference wrote re-encodes byte-identical, and a nil in
|
||||
// a value slot (int, float, bool) is refused unless the slot is nullable.
|
||||
|
|
@ -281,6 +341,9 @@ export function slots<TValue extends object>(spec: SlotsSpec<TValue>, typeName?:
|
|||
decode(raw) {
|
||||
expect(raw, "an array of slots", Array.isArray(raw));
|
||||
const input = raw as unknown[];
|
||||
if (input.length > length) {
|
||||
noteExtraSlots(simpleTypeName(typeName), input.length, length);
|
||||
}
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const field of fields) {
|
||||
const present = field.key < input.length;
|
||||
|
|
@ -291,7 +354,7 @@ export function slots<TValue extends object>(spec: SlotsSpec<TValue>, typeName?:
|
|||
? defaultOf(field)
|
||||
: input[field.key] === null && !field.codec.valueType
|
||||
? null
|
||||
: at(`${field.name} (slot ${field.key})`, () => field.codec.decode(input[field.key]));
|
||||
: at(`${field.name} (slot ${field.key})`, () => nested(() => field.codec.decode(input[field.key])));
|
||||
}
|
||||
return out as TValue;
|
||||
},
|
||||
|
|
@ -363,6 +426,9 @@ export function union<TValue extends { kind: string }>(
|
|||
if (arm === undefined) {
|
||||
throw new Error(`Union ${name}: unknown arm key ${key} (declared keys: ${declared}).`);
|
||||
}
|
||||
if (arm.body === unit && raw[1].length > 0) {
|
||||
noteExtraSlots(arm.armName, raw[1].length, 0);
|
||||
}
|
||||
const fields = arm.body === unit ? {} : at(`arm ${arm.armName}`, () => (arm.body as SlotsCodec<any>).decode(raw[1]));
|
||||
return { kind: arm.armName, ...fields } as unknown as TValue;
|
||||
},
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import {
|
||||
declaresRole,
|
||||
decodeDocumentPayload,
|
||||
decodeDocumentPayloadMarked,
|
||||
documentPayloadBytes,
|
||||
documentPayloadFromBytes,
|
||||
encodeDocumentPayload,
|
||||
|
|
@ -8,6 +8,7 @@ import {
|
|||
schemaIdentityOf,
|
||||
} from "./document";
|
||||
import { resolveDocumentFormatter } from "./document-formatter";
|
||||
import type { NotedSlots } from "./codec";
|
||||
import type { DocumentPayload } from "./document";
|
||||
import { SchemaConflictError, schemaIdForEnvelope, writtenOwns } from "./store-format";
|
||||
import type {
|
||||
|
|
@ -20,6 +21,7 @@ import type {
|
|||
CultCacheEnvelope,
|
||||
CultCacheSchemaCatalogEntry,
|
||||
CultCacheStoreRegistration,
|
||||
CultCacheUnreadSlots,
|
||||
} from "./types";
|
||||
|
||||
type Accessor = (value: unknown) => string | undefined;
|
||||
|
|
@ -27,7 +29,7 @@ type Accessor = (value: unknown) => string | undefined;
|
|||
// A registered definition reads and writes its documents through the owner, on payloads the cache has cloned.
|
||||
type RegisteredFormatter = {
|
||||
encode(value: unknown): DocumentPayload;
|
||||
decode(payload: DocumentPayload): unknown;
|
||||
decode(payload: DocumentPayload): { value: unknown; unreadSlots?: NotedSlots };
|
||||
};
|
||||
|
||||
type RegisteredDefinition = {
|
||||
|
|
@ -50,6 +52,10 @@ type HydratedEntry = CultCacheEnvelope & {
|
|||
catalogEntry: NonNullable<CultCacheEnvelope["catalogEntry"]>;
|
||||
value: unknown;
|
||||
lookups: EntryLookups;
|
||||
// What the payload decode read past: set when the record was admitted, and it makes the record read-only unless explained.
|
||||
unreadSlots?: CultCacheUnreadSlots;
|
||||
// The schema id the record arrived under, before it was admitted under its reader's: what an unread-slots refusal names.
|
||||
storedSchemaId?: string;
|
||||
};
|
||||
|
||||
// Everything a pull replaces. Built whole, then swapped in with one assignment.
|
||||
|
|
@ -84,6 +90,20 @@ type DerivedTypeLookups = {
|
|||
readonly indexMaps: Map<string, Map<string, string>>;
|
||||
};
|
||||
|
||||
// The consent a compatible-id declaration gives (cultlib-gaps:ruling:slot-mark-consent): a mark is explained only when the record
|
||||
// arrived under an id other than the reader's own that the reader lists compatible, it is at
|
||||
// depth 0 and the schema comparison of the catalog entry the record arrived with against the reader's own reported the slots it
|
||||
// carries as ignored, so its slot count is at most the highest ignored slot plus one. A record written or put through the cache
|
||||
// arrives under the reader's own entry, which ignores nothing, so no mark of it is explained.
|
||||
function explainedBy(noted: NotedSlots, reader: CultCacheSchemaCatalogEntry, arrivedId: string, arrived: CultCacheSchemaCatalogEntry | undefined): boolean {
|
||||
if (arrivedId === reader.schemaId || !(reader.compatibleSchemaIds ?? []).includes(arrivedId)) {
|
||||
return false;
|
||||
}
|
||||
const declared = new Set((reader.members ?? []).map((member) => member.slot));
|
||||
const ignored = (arrived?.members ?? []).map((member) => member.slot).filter((slot) => !declared.has(slot));
|
||||
return noted.depth0Only && noted.slotCount <= Math.max(-1, ...ignored) + 1;
|
||||
}
|
||||
|
||||
function emptyHydratedState(): HydratedState {
|
||||
return {
|
||||
entries: new Map(),
|
||||
|
|
@ -312,7 +332,7 @@ export class CultCache {
|
|||
}
|
||||
|
||||
const payload = this.#cloneBytes(entry.payload);
|
||||
const value = registered.formatter.decode(payload);
|
||||
const { value, unreadSlots } = registered.formatter.decode(payload);
|
||||
let admitted: CultCacheEnvelope = { ...entry, type, payload };
|
||||
if (registered.global) {
|
||||
const persistedKey = persistedGlobalKeys.get(type);
|
||||
|
|
@ -327,7 +347,7 @@ export class CultCache {
|
|||
admitted = { ...admitted, key: CultCache.GLOBAL_KEY };
|
||||
}
|
||||
}
|
||||
const hydrated = this.#admitEntry(next, registered, admitted, value);
|
||||
const hydrated = this.#admitEntry(next, registered, admitted, value, unreadSlots);
|
||||
const other = this.#heldUnderOtherType(next, type, hydrated.key);
|
||||
if (other !== undefined) {
|
||||
throw new Error(`Record ${hydrated.key} is held by two stores, as type "${other.type}" and as type "${type}".`);
|
||||
|
|
@ -518,10 +538,9 @@ export class CultCache {
|
|||
if (!envelope.storedAt || envelope.storedAt.trim().length === 0) {
|
||||
throw new Error(`CultCache envelope storedAt for type "${definition.type}" must be non-empty.`);
|
||||
}
|
||||
this.#requireGlobalKey(registered, envelope.key);
|
||||
|
||||
const payload = this.#cloneBytes(envelope.payload);
|
||||
const parsed = registered.formatter.decode(payload) as CultCacheDocumentValue<TDefinition>;
|
||||
const { value: parsed, unreadSlots } = registered.formatter.decode(payload) as { value: CultCacheDocumentValue<TDefinition>; unreadSlots?: NotedSlots };
|
||||
const entry: CultCacheEnvelope = {
|
||||
key: envelope.key,
|
||||
type: envelope.type,
|
||||
|
|
@ -532,7 +551,7 @@ export class CultCache {
|
|||
schemaId: registered.catalogEntry.schemaId,
|
||||
catalogEntry: registered.catalogEntry,
|
||||
};
|
||||
await this.#writeNow(registered, entry, parsed);
|
||||
await this.#writeNow(registered, entry, parsed, unreadSlots);
|
||||
return parsed;
|
||||
});
|
||||
}
|
||||
|
|
@ -585,6 +604,29 @@ export class CultCache {
|
|||
});
|
||||
}
|
||||
|
||||
/** What the record's payload carried past this build's declared slots, or undefined: the held record is read-only unless `explained`. */
|
||||
unreadSlots(definition: AnyCultCacheDocumentDefinition, key: string): CultCacheUnreadSlots | undefined {
|
||||
return this.#hydrated.entries.get(this.#entryId(this.#requireDefinition(definition).definition.type, key))?.unreadSlots;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a write of the record's own type at the key would be let through: it asks the judge #writeNow asks, of the record the
|
||||
* cache holds (a key a store carries foreign, a record of another type the write does not own or cannot replace, a global at
|
||||
* another key, a record carrying unread slots nothing consents to). It answers the WRITE form. A removal asks the judge's
|
||||
* removal form and differs in two states: an owner's `delete` of a record held in another store than its own home succeeds
|
||||
* where this is false (a write would push to the wrong store), and `delete` of a global type at a key other than __global__ is
|
||||
* the plain false of an unheld key where this is false. Live and advisory: the write or removal still decides.
|
||||
*/
|
||||
isWritable(definition: AnyCultCacheDocumentDefinition, key: string): boolean {
|
||||
const registered = this.#requireDefinition(definition);
|
||||
try {
|
||||
this.#judge(registered, key, this.#probe(registered, key));
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// What the last pull found that no registered type claims, by key ordinal. A later registration claims a record at the next pull.
|
||||
foreignRecords(): readonly CultCacheForeignRecord[] {
|
||||
return [...this.#hydrated.foreign].sort((left, right) => compareOrdinal(left.key, right.key));
|
||||
|
|
@ -680,9 +722,8 @@ export class CultCache {
|
|||
value: CultCacheDocumentValue<TDefinition>,
|
||||
): Promise<CultCacheDocumentValue<TDefinition>> {
|
||||
const registered = this.#requireDefinition(definition);
|
||||
this.#requireGlobalKey(registered, key);
|
||||
const payload = registered.formatter.encode(value);
|
||||
const parsed = registered.formatter.decode(payload) as CultCacheDocumentValue<TDefinition>;
|
||||
const { value: parsed, unreadSlots } = registered.formatter.decode(payload) as { value: CultCacheDocumentValue<TDefinition>; unreadSlots?: NotedSlots };
|
||||
const entry: CultCacheEnvelope = {
|
||||
key,
|
||||
type: definition.type,
|
||||
|
|
@ -691,26 +732,16 @@ export class CultCache {
|
|||
schemaId: registered.catalogEntry.schemaId,
|
||||
catalogEntry: registered.catalogEntry,
|
||||
};
|
||||
await this.#writeNow(registered, entry, parsed);
|
||||
await this.#writeNow(registered, entry, parsed, unreadSlots);
|
||||
return parsed;
|
||||
}
|
||||
|
||||
// Validates the whole write (home, global singleton, name and index accessors) before the store
|
||||
// is touched; once the store accepts, applying to the cache has no fallible step.
|
||||
async #writeNow(registered: RegisteredDefinition, entry: CultCacheEnvelope, value: unknown): Promise<void> {
|
||||
async #writeNow(registered: RegisteredDefinition, entry: CultCacheEnvelope, value: unknown, unreadSlots?: NotedSlots): Promise<void> {
|
||||
const home = this.#homeStore(entry.type);
|
||||
const retyped = this.#refuseRetype(entry);
|
||||
if (retyped !== undefined && this.#homeStore(retyped.type) !== home) {
|
||||
// The write owns the held record's id and would replace it, but the held record is in another store; the push
|
||||
// reaches only this type's home, so the old record would stay in its own file and load as a second holder.
|
||||
throw new SchemaConflictError(
|
||||
schemaIdForEnvelope(entry),
|
||||
[retyped.type, entry.type],
|
||||
entry.key,
|
||||
`CultCache holds record "${entry.key}" as type "${retyped.type}" in another store than the home of type "${entry.type}", so the rename cannot replace it; remove the record first.`,
|
||||
);
|
||||
}
|
||||
const hydrated = this.#admitEntry(this.#hydrated, registered, entry, value);
|
||||
const retyped = this.#judge(registered, entry.key, entry);
|
||||
const hydrated = this.#admitEntry(this.#hydrated, registered, entry, value, unreadSlots);
|
||||
this.#refuseIndexSharing(this.#hydrated, entry.type, registered.catalogEntry.schemaName, entry.key, hydrated.lookups);
|
||||
const legacy = this.#hydrated.legacyGlobals.get(entry.type);
|
||||
await home?.push(entry);
|
||||
|
|
@ -725,6 +756,85 @@ export class CultCache {
|
|||
this.#installEntry(this.#hydrated, registered, hydrated);
|
||||
}
|
||||
|
||||
// An envelope of the definition's type at the key, carrying nothing: what the judge asks about a write or a removal by that type.
|
||||
#probe(registered: RegisteredDefinition, key: string): CultCacheEnvelope {
|
||||
return {
|
||||
key, type: registered.definition.type, payload: documentPayloadFromBytes(new Uint8Array(0)), storedAt: "",
|
||||
schemaId: registered.catalogEntry.schemaId, catalogEntry: registered.catalogEntry,
|
||||
};
|
||||
}
|
||||
|
||||
// The one judge of whether a key may be written (a written envelope) or removed (none), before any store is touched: it throws the
|
||||
// refusal, and #writeNow, #deleteNow and isWritable all ask it, so the query cannot disagree with the write. For a write it
|
||||
// returns the held record of another type that the write owns and so restamps, if any. A refusal names keys and schemas,
|
||||
// never a payload or a slot value.
|
||||
#judge(registered: RegisteredDefinition, key: string, written?: CultCacheEnvelope): HydratedEntry | undefined {
|
||||
const type = registered.definition.type;
|
||||
const held = this.#hydrated.entries.get(this.#entryId(type, key));
|
||||
if (written === undefined) {
|
||||
// A removal is by key: the record it removes is the one held at the key, of this type or of another type whose schema id
|
||||
// this type owns (the record a write of this type would replace). A held record of a type it does not own is refused.
|
||||
this.#refuseForeignKey(key, type, "removed");
|
||||
this.#refuseUnread(held);
|
||||
if (held !== undefined) {
|
||||
return undefined;
|
||||
}
|
||||
const other = this.#heldUnderOtherType(this.#hydrated, type, key);
|
||||
if (other === undefined) {
|
||||
return undefined;
|
||||
}
|
||||
if (!writtenOwns(this.#probe(registered, key), schemaIdForEnvelope(other))) {
|
||||
throw new SchemaConflictError(
|
||||
registered.catalogEntry.schemaId,
|
||||
[other.type, type],
|
||||
key,
|
||||
`CultCache holds record "${key}" as type "${other.type}", so type "${type}" cannot remove it.`,
|
||||
);
|
||||
}
|
||||
this.#refuseUnread(other);
|
||||
return other;
|
||||
}
|
||||
this.#requireGlobalKey(registered, key);
|
||||
const retyped = this.#refuseRetype(written);
|
||||
this.#refuseUnread(held);
|
||||
this.#refuseUnread(retyped);
|
||||
if (retyped !== undefined && this.#homeStore(retyped.type) !== this.#homeStore(type)) {
|
||||
// The write owns the held record's id and would replace it, but the held record is in another store; the push
|
||||
// reaches only this type's home, so the old record would stay in its own file and load as a second holder.
|
||||
throw new SchemaConflictError(
|
||||
schemaIdForEnvelope(written),
|
||||
[retyped.type, type],
|
||||
key,
|
||||
`CultCache holds record "${key}" as type "${retyped.type}" in another store than the home of type "${type}", so the rename cannot replace it; remove the record first.`,
|
||||
);
|
||||
}
|
||||
return retyped;
|
||||
}
|
||||
|
||||
// The one refusal of a record held read-only for the slots its payload carries past this build's: a write or a removal of
|
||||
// its key. It names the type and the counts, never a slot value. A mark that is explained (see #admitEntry) is consented to.
|
||||
#unreadRefusal(held: HydratedEntry | undefined): SchemaConflictError | undefined {
|
||||
const unread = held?.unreadSlots;
|
||||
if (held === undefined || unread === undefined || unread.explained) {
|
||||
return undefined;
|
||||
}
|
||||
return new SchemaConflictError(
|
||||
held.storedSchemaId ?? held.schemaId ?? held.type,
|
||||
[held.catalogEntry.schemaName],
|
||||
held.key,
|
||||
`${held.catalogEntry.schemaName} record ${held.key} is read-only: ${unread.typeName} was read with ${unread.slotCount} slots where this build declares ${unread.declaredCount} ` +
|
||||
`(${unread.occurrences} object(s) in the record). Writing it would drop data a newer writer stored; read it with types that declare those slots, or write it from that build.`,
|
||||
unread,
|
||||
);
|
||||
}
|
||||
|
||||
#refuseUnread(held: HydratedEntry | undefined): void {
|
||||
const refusal = this.#unreadRefusal(held);
|
||||
if (refusal !== undefined) {
|
||||
throw refusal;
|
||||
}
|
||||
}
|
||||
|
||||
// The one key-identity judge of the cache: may this write land at its key, given what the cache holds there under
|
||||
// every registered type and the keys its stores carry foreign. The store judges the file with the same predicate
|
||||
// (writtenOwns); this judges the cache, which a store-less cache has no other judge for. Returns the held record of
|
||||
|
|
@ -782,18 +892,20 @@ export class CultCache {
|
|||
key: string,
|
||||
): Promise<boolean> {
|
||||
const registered = this.#requireDefinition(definition);
|
||||
const entry = this.#hydrated.entries.get(this.#entryId(definition.type, key));
|
||||
// variants:ruling:unheld-removal-refused-at-commit: a key a store carries foreign is refused, a key held by nobody is the
|
||||
// existing false; a key held as another type is removed by an owner of its id and refused to any other type.
|
||||
// Removal is by key, in one store call: the record held at the key of this type, else the one of another type this type owns.
|
||||
const owned = this.#judge(registered, key);
|
||||
const entry = this.#hydrated.entries.get(this.#entryId(definition.type, key)) ?? owned;
|
||||
if (!entry) {
|
||||
// variants:ruling:unheld-removal-refused-at-commit: a key a store carries foreign is refused, a key held under
|
||||
// another type or by nobody is the existing false.
|
||||
this.#refuseForeignKey(key, definition.type, "removed");
|
||||
return false;
|
||||
}
|
||||
|
||||
const home = this.#homeStore(definition.type);
|
||||
await home?.delete(this.#hydrated.legacyGlobals.get(definition.type) ?? this.#toEnvelope(entry));
|
||||
this.#hydrated.legacyGlobals.delete(definition.type);
|
||||
this.#removeEntry(this.#hydrated, registered, entry);
|
||||
// The cache drops the record only once the store has removed it.
|
||||
const home = this.#homeStore(entry.type);
|
||||
await home?.delete(this.#hydrated.legacyGlobals.get(entry.type) ?? this.#toEnvelope(entry));
|
||||
this.#hydrated.legacyGlobals.delete(entry.type);
|
||||
this.#removeEntry(this.#hydrated, this.#definitions.get(entry.type)!, entry);
|
||||
return true;
|
||||
}
|
||||
|
||||
|
|
@ -820,7 +932,7 @@ export class CultCache {
|
|||
|
||||
return {
|
||||
encode: (value: unknown) => this.#cloneBytes(encodeDocumentPayload(definition, value)),
|
||||
decode: (payload: DocumentPayload) => decodeDocumentPayload(definition, this.#cloneBytes(payload)),
|
||||
decode: (payload: DocumentPayload) => decodeDocumentPayloadMarked(definition, this.#cloneBytes(payload)),
|
||||
};
|
||||
}
|
||||
|
||||
|
|
@ -1000,6 +1112,7 @@ export class CultCache {
|
|||
registered: RegisteredDefinition,
|
||||
entry: CultCacheEnvelope,
|
||||
value: unknown,
|
||||
noted?: NotedSlots,
|
||||
): HydratedEntry {
|
||||
if (registered.global) {
|
||||
const currentGlobalKey = state.globalKeys.get(entry.type);
|
||||
|
|
@ -1020,6 +1133,8 @@ export class CultCache {
|
|||
catalogEntry: registered.catalogEntry,
|
||||
value,
|
||||
lookups,
|
||||
storedSchemaId: entry.schemaId ?? entry.type,
|
||||
...(noted ? { unreadSlots: { ...noted, explained: explainedBy(noted, registered.catalogEntry, entry.schemaId ?? entry.type, entry.catalogEntry) } } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import { decode, encode } from "@msgpack/msgpack";
|
||||
|
||||
import { MAX_PAYLOAD_DEPTH } from "./payload-depth";
|
||||
import type { SlotsCodec } from "./codec";
|
||||
import { decodeNoting, type NotedSlots, type SlotsCodec } from "./codec";
|
||||
import type { AnyCultCacheDocumentDefinition, CultCacheDocumentFormatter } from "./types";
|
||||
import { readWire, writeWire } from "./wire";
|
||||
|
||||
|
|
@ -16,6 +16,13 @@ export function resolveDocumentFormatter(definition: AnyCultCacheDocumentDefinit
|
|||
return definition.formatter ?? (definition.codec ? codecFormatter(definition.codec) : plainDocumentFormatter);
|
||||
}
|
||||
|
||||
// The one decode of a document's bytes, with the slots it read past. A plain formatter or a user formatter has no slot codec
|
||||
// and carries everything, so it notes nothing.
|
||||
export function decodeMarked(definition: AnyCultCacheDocumentDefinition, bytes: Uint8Array): { value: unknown; unreadSlots?: NotedSlots } {
|
||||
const formatter = resolveDocumentFormatter(definition);
|
||||
return decodeNoting(() => formatter.decode(bytes));
|
||||
}
|
||||
|
||||
const plainDocumentFormatter: CultCacheDocumentFormatter<unknown> = {
|
||||
// The encoder's own depth refusal (100 by default) is set above the limit, so the payload walk decides.
|
||||
encode: (value: unknown) => encode(value, { maxDepth: MAX_PAYLOAD_DEPTH * 2 }),
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import { resolveDocumentFormatter } from "./document-formatter";
|
||||
import type { NotedSlots } from "./codec";
|
||||
import { decodeMarked, resolveDocumentFormatter } from "./document-formatter";
|
||||
import { checkPayloadDepth } from "./payload-depth";
|
||||
import type {
|
||||
AnyCultCacheDocumentDefinition,
|
||||
|
|
@ -118,10 +119,20 @@ export function decodeDocumentPayload<TDefinition extends AnyCultCacheDocumentDe
|
|||
definition: TDefinition,
|
||||
payload: DocumentPayload,
|
||||
): CultCacheDocumentValue<TDefinition> {
|
||||
return decodeDocumentPayloadMarked(definition, payload).value;
|
||||
}
|
||||
|
||||
// The same read with the slots it read past (cult-cache-ts slot-readonly). The depth check runs first: a payload past the
|
||||
// limit is refused, never marked.
|
||||
export function decodeDocumentPayloadMarked<TDefinition extends AnyCultCacheDocumentDefinition>(
|
||||
definition: TDefinition,
|
||||
payload: DocumentPayload,
|
||||
): { value: CultCacheDocumentValue<TDefinition>; unreadSlots?: NotedSlots } {
|
||||
const bytes = documentPayloadBytes(payload);
|
||||
if (writesLibraryMessagePack(definition)) checkPayloadDepth(bytes, `document type "${definition.type}" payload`);
|
||||
const decoded = resolveDocumentFormatter(definition).decode(bytes);
|
||||
return definition.schema.parse(decoded) as CultCacheDocumentValue<TDefinition>;
|
||||
const { value: decoded, unreadSlots } = decodeMarked(definition, bytes);
|
||||
const value = definition.schema.parse(decoded) as CultCacheDocumentValue<TDefinition>;
|
||||
return unreadSlots ? { value, unreadSlots } : { value };
|
||||
}
|
||||
|
||||
// The one writer: the definition's schema first, then its formatter, codec or plain map.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
import { decode } from "@msgpack/msgpack";
|
||||
|
||||
import type { CultCacheEnvelope } from "./types";
|
||||
import type { CultCacheEnvelope, CultCacheUnreadSlots } from "./types";
|
||||
|
||||
export const STORE_FORMAT_VERSION = "cultcache.store.v1";
|
||||
|
||||
|
|
@ -83,13 +83,16 @@ export class SchemaConflictError extends Error {
|
|||
readonly schemaId: string;
|
||||
readonly schemaNames: string[];
|
||||
readonly recordKey: string;
|
||||
/** Set when the refusal is for unread slots; undefined for every other refusal. */
|
||||
readonly unreadSlots?: CultCacheUnreadSlots;
|
||||
|
||||
constructor(schemaId: string, schemaNames: string[], recordKey: string, message?: string) {
|
||||
constructor(schemaId: string, schemaNames: string[], recordKey: string, message?: string, unreadSlots?: CultCacheUnreadSlots) {
|
||||
super(message ?? `Schema id "${schemaId}" cannot describe record "${recordKey}": ${schemaNames.map((name) => `"${name}"`).join(", ")}.`);
|
||||
this.name = "SchemaConflictError";
|
||||
this.schemaId = schemaId;
|
||||
this.schemaNames = schemaNames;
|
||||
this.recordKey = recordKey;
|
||||
this.unreadSlots = unreadSlots;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -99,6 +99,26 @@ export interface CultCacheEnvelope {
|
|||
catalogEntry?: CultCacheSchemaCatalogEntry;
|
||||
}
|
||||
|
||||
/**
|
||||
* What a payload decode found past the slots this build declares, at any depth: the first type read with extra slots, how many
|
||||
* slots it carried against how many it declares, and how many objects in the record did. It holds no slot value, so a refusal
|
||||
* can name it. A record carrying it is read-only unless `explained`.
|
||||
*/
|
||||
export interface CultCacheUnreadSlots {
|
||||
/** The first type read with extra slots (the unit arm's arm name for a unit arm). */
|
||||
readonly typeName: string;
|
||||
readonly slotCount: number;
|
||||
readonly declaredCount: number;
|
||||
readonly occurrences: number;
|
||||
/** True when every mark sat in the document's own object. */
|
||||
readonly depth0Only: boolean;
|
||||
/**
|
||||
* True only when every mark is at depth 0 and the schema comparison of the id the record was stored under reported every slot
|
||||
* the mark carries (slotCount is at most the highest reported slot plus one): the loss a compatible-id declaration consents to.
|
||||
*/
|
||||
readonly explained: boolean;
|
||||
}
|
||||
|
||||
export interface PushAllOptions {
|
||||
soft?: boolean;
|
||||
}
|
||||
|
|
|
|||
461
packages/cultcache-ts/test/slot-readonly.test.ts
Normal file
461
packages/cultcache-ts/test/slot-readonly.test.ts
Normal file
|
|
@ -0,0 +1,461 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { copyFile, mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { test } from "node:test";
|
||||
|
||||
import { decode, encode } from "@msgpack/msgpack";
|
||||
import { z } from "zod";
|
||||
|
||||
import { codecs } from "../src";
|
||||
import { CultCache } from "../src/cult-cache";
|
||||
import { defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
|
||||
import { SingleFileMessagePackBackingStore, readSingleFileStore } from "../src/single-file-messagepack-backing-store";
|
||||
import { SchemaConflictError, StoreUnreadableError } from "../src/store-format";
|
||||
import type { CacheBackingStore, CultCacheEnvelope, CultCacheUnreadSlots } from "../src/types";
|
||||
import {
|
||||
type Surface, envelopeOf, golden, hex, memoryStore, read, slotUnitDocument, surfaceCodec, surfaceDeclaringDocument, surfaceDocument, surfaceSchema, vectors,
|
||||
} from "./union-surface-fixture";
|
||||
|
||||
// cultlib-gaps:cut:slot-readonly-ts. A payload carrying a slot past its reader's declared slots, at any depth, is read, held
|
||||
// read-only, and every write or removal of its key is refused before a store is touched. The vectors are written by the C#
|
||||
// reference: tests/vectors/union-parity/README.md, "Extra trailing slot" and "Slot stores".
|
||||
|
||||
const { slots, string } = codecs;
|
||||
const SLOT_VALUE = "extra-slot";
|
||||
|
||||
function mark(typeName: string, slotCount: number, declaredCount: number, depth0Only: boolean, explained = false, occurrences = 1): CultCacheUnreadSlots {
|
||||
return { typeName, slotCount, declaredCount, occurrences, depth0Only, explained };
|
||||
}
|
||||
|
||||
// File, then the mark the README table gives for it.
|
||||
const PAYLOAD_ROWS: [string, CultCacheUnreadSlots][] = [
|
||||
["extra-slot-record.msgpack", mark("UnionSurface", 8, 7, true)],
|
||||
["extra-slot-nested-record.msgpack", mark("Composition", 3, 2, false)],
|
||||
["extra-slot-arm.msgpack", mark("Box", 3, 2, false)],
|
||||
["extra-slot-list-element.msgpack", mark("Text", 3, 2, false)],
|
||||
["depth-skipped-at-limit.msgpack", mark("UnionSurface", 8, 7, true)],
|
||||
];
|
||||
|
||||
async function readOnlyFiles(name: string): Promise<string[]> {
|
||||
const rows = decode(await read(name)) as unknown[][];
|
||||
return rows.filter((row) => row[1] === "read-only").map((row) => row[0] as string);
|
||||
}
|
||||
|
||||
test("the read-only payload vectors are exactly the ones this walker names: four in outcomes, one in bounds", async () => {
|
||||
const listed = [...(await readOnlyFiles("outcomes.msgpack")), ...(await readOnlyFiles("bounds-outcomes.msgpack"))];
|
||||
assert.deepEqual(listed.sort(), PAYLOAD_ROWS.map(([file]) => file).sort());
|
||||
});
|
||||
|
||||
for (const [file, expected] of PAYLOAD_ROWS) {
|
||||
test(`${file}: read past the extra slot, held read-only with the mark, every write and removal of the key refused`, async () => {
|
||||
const document = surfaceDocument();
|
||||
const entries = [envelopeOf(await read(file))];
|
||||
const before = hex(documentPayloadBytes(entries[0]!.payload));
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(memoryStore(entries)).build();
|
||||
await cache.pullAllBackingStores();
|
||||
|
||||
assert.deepEqual(cache.get(document, "surface"), golden, "the extra slot is read past");
|
||||
assert.deepEqual(cache.unreadSlots(document, "surface"), expected);
|
||||
assert.equal(cache.isWritable(document, "surface"), false);
|
||||
assert.equal(cache.isWritable(document, "another"), true);
|
||||
|
||||
const refused = (error: unknown) => {
|
||||
assert.ok(error instanceof SchemaConflictError, "a SchemaConflictError");
|
||||
assert.deepEqual(error.unreadSlots, expected);
|
||||
assert.equal(error.recordKey, "surface");
|
||||
assert.ok(error.message.includes(expected.typeName), error.message);
|
||||
assert.ok(error.message.includes(`read with ${expected.slotCount} slots where this build declares ${expected.declaredCount} (${expected.occurrences} object(s)`), error.message);
|
||||
assert.ok(!error.message.includes(SLOT_VALUE), "the refusal never holds a slot value");
|
||||
return true;
|
||||
};
|
||||
await assert.rejects(() => cache.put(document, "surface", golden), refused);
|
||||
await assert.rejects(() => cache.update(document, "surface", (value) => ({ ...value!, version: 4 })), refused);
|
||||
const replacement = envelopeOf(await read("union-surface.msgpack"));
|
||||
await assert.rejects(() => cache.putEnvelope(document, replacement), refused);
|
||||
await assert.rejects(() => cache.delete(document, "surface"), refused);
|
||||
assert.equal(entries.length, 1, "nothing reached the store");
|
||||
assert.equal(hex(documentPayloadBytes(entries[0]!.payload)), before, "the held bytes are untouched");
|
||||
assert.deepEqual(cache.get(document, "surface"), golden, "the record is still held");
|
||||
|
||||
await cache.put(document, "another", golden);
|
||||
assert.equal(entries.length, 2, "another key of the same store writes");
|
||||
});
|
||||
}
|
||||
|
||||
test("a clean payload and a short one are writable: missing trailing slots keep the defaults and mark nothing", async () => {
|
||||
const document = surfaceDocument();
|
||||
const short = encode([...(decode(await read("union-surface.msgpack")) as unknown[]).slice(0, 6)]);
|
||||
const entries = [envelopeOf(await read("union-surface.msgpack")), { ...envelopeOf(short), key: "short" }];
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(memoryStore(entries)).build();
|
||||
await cache.pullAllBackingStores();
|
||||
for (const key of ["surface", "short"]) {
|
||||
assert.equal(cache.unreadSlots(document, key), undefined);
|
||||
assert.equal(cache.isWritable(document, key), true);
|
||||
}
|
||||
await cache.put(document, "short", golden);
|
||||
});
|
||||
|
||||
test("a payload past the nesting limit is refused, never marked", async () => {
|
||||
const entries = [envelopeOf(await read("depth-skipped-past-limit.msgpack"))];
|
||||
const cache = CultCache.builder().withDocumentType(surfaceDocument()).withGenericStore(memoryStore(entries)).build();
|
||||
await assert.rejects(() => cache.pullAllBackingStores(), /nesting|depth/iu);
|
||||
});
|
||||
|
||||
test("putEnvelope of a payload carrying extra slots lands the bytes and holds the record read-only", async () => {
|
||||
const document = surfaceDocument();
|
||||
const entries: ReturnType<typeof envelopeOf>[] = [];
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(memoryStore(entries)).build();
|
||||
const marked = envelopeOf(await read("extra-slot-nested-record.msgpack"));
|
||||
await cache.putEnvelope(document, marked);
|
||||
assert.equal(hex(documentPayloadBytes(entries[0]!.payload)), hex(await read("extra-slot-nested-record.msgpack")), "the bytes are carried");
|
||||
assert.deepEqual(cache.unreadSlots(document, "surface"), mark("Composition", 3, 2, false));
|
||||
assert.equal(cache.isWritable(document, "surface"), false);
|
||||
await assert.rejects(() => cache.put(document, "surface", golden), SchemaConflictError);
|
||||
});
|
||||
|
||||
test("a key a store carries foreign is not writable", async () => {
|
||||
const document = surfaceDocument();
|
||||
const foreign = { ...envelopeOf(await read("union-surface.msgpack")), key: "stranger", type: "tests.unknown", schemaId: "tests.unknown.v1" };
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(memoryStore([foreign])).build();
|
||||
await cache.pullAllBackingStores();
|
||||
assert.equal(cache.isWritable(document, "stranger"), false);
|
||||
assert.equal(cache.isWritable(document, "other"), true);
|
||||
});
|
||||
|
||||
test("a write by a type that owns the held record's id restamps it, and is refused when that record carries unread slots", async () => {
|
||||
const typed = (type: string, schemaId: string, compatibleSchemaIds: string[] = []) =>
|
||||
defineDocumentType({ type, schemaId, schemaName: type, schemaVersion: `${type}.v1`, compatibleSchemaIds, schema: surfaceSchema, codec: surfaceCodec, name: "id" });
|
||||
const older = typed("tests.surface_older", "tests.surface_older.v1");
|
||||
const newer = typed("tests.surface_newer", "tests.surface_newer.v1", ["tests.surface_older.v1"]);
|
||||
for (const [file, lands] of [["union-surface.msgpack", true], ["extra-slot-record.msgpack", false]] as const) {
|
||||
const held = { ...envelopeOf(await read(file)), type: older.type, schemaId: "tests.surface_older.v1" };
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(newer).withGenericStore(memoryStore([held])).build();
|
||||
await cache.pullAllBackingStores();
|
||||
assert.equal(cache.isWritable(newer, "surface"), lands, `${file}: isWritable agrees with the write by the type that owns the id`);
|
||||
const write = cache.put(newer, "surface", golden);
|
||||
if (lands) {
|
||||
await write;
|
||||
} else {
|
||||
await assert.rejects(() => write, (error: unknown) => error instanceof SchemaConflictError && error.unreadSlots?.typeName === "UnionSurface");
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// The store vectors. TS has no variant records: extra-slot-store.cc carries one (marked-variant), so the whole file is refused
|
||||
// by name, and the walk runs over the file without that record.
|
||||
async function copyOf(name: string): Promise<string> {
|
||||
const path = join(await mkdtemp(join(tmpdir(), "slot-readonly-")), name);
|
||||
await copyFile(join(vectors, name), path);
|
||||
return path;
|
||||
}
|
||||
|
||||
async function withoutVariant(path: string): Promise<void> {
|
||||
const [format, catalog, records] = decode(await readFile(path)) as [unknown, unknown[], unknown[][]];
|
||||
await writeFile(path, encode([format, catalog, records.filter((record) => record[0] !== "marked-variant")]));
|
||||
}
|
||||
|
||||
type SlotRow = [string, string, boolean, boolean | null, string | null, number | null, number | null, number | null, string | null];
|
||||
|
||||
async function slotRows(): Promise<SlotRow[]> {
|
||||
return decode(await read("slot-outcomes.msgpack")) as SlotRow[];
|
||||
}
|
||||
|
||||
// The catalog entry the file publishes for a schema: its id and the reference's content hash, which a store rewrites with the
|
||||
// definition's own.
|
||||
async function catalogOf(file: string, schemaName: string): Promise<[string, string]> {
|
||||
const [, catalog] = decode(await read(file)) as [unknown, string[][]];
|
||||
const entry = catalog.find((candidate) => candidate[1] === schemaName)!;
|
||||
return [entry[0]!, entry[3]!];
|
||||
}
|
||||
|
||||
async function openStore(file: string, consent: boolean) {
|
||||
const path = await copyOf(file);
|
||||
if (file === "extra-slot-store.cc") await withoutVariant(path);
|
||||
const surface = consent ? surfaceDeclaringDocument() : surfaceDocument(...(await catalogOf(file, "tests.union_surface")));
|
||||
const slotUnit = consent ? undefined : slotUnitDocument(...(await catalogOf(file, "tests.slot_unit")));
|
||||
const builder = CultCache.builder().withDocumentType(surface).withGenericStore(new SingleFileMessagePackBackingStore(path));
|
||||
const cache = (slotUnit ? builder.withDocumentType(slotUnit) : builder).build();
|
||||
await cache.pullAllBackingStores();
|
||||
return { path, cache, surface, slotUnit };
|
||||
}
|
||||
|
||||
test("extra-slot-store.cc carries a variant, which this runtime refuses by name; the rest of the file is walked", async () => {
|
||||
const path = await copyOf("extra-slot-store.cc");
|
||||
assert.throws(() => readSingleFileStore(path), StoreUnreadableError);
|
||||
});
|
||||
|
||||
test("every slot-outcomes row holds: the mark, IsWritable, and the removal refused exactly where the record is not writable", async () => {
|
||||
const rows = await slotRows();
|
||||
assert.equal(rows.length, 8);
|
||||
let walked = 0;
|
||||
for (const [file, key, writable, explained, typeName, slots, declared, occurrences, flattenRefusedBy] of rows) {
|
||||
if (flattenRefusedBy !== null) continue; // a variant: this runtime holds none
|
||||
walked += 1;
|
||||
const { path, cache, surface, slotUnit } = await openStore(file!, file === "extra-slot-consent.cc");
|
||||
const document = key === "unit-arm" ? slotUnit! : surface;
|
||||
const label = `${file} ${key}`;
|
||||
const found = cache.unreadSlots(document, key!);
|
||||
if (typeName === null) {
|
||||
assert.equal(found, undefined, label);
|
||||
} else {
|
||||
assert.ok(found, label);
|
||||
assert.deepEqual([found.explained, found.typeName, found.slotCount, found.declaredCount, found.occurrences], [explained, typeName, slots, declared, occurrences], label);
|
||||
}
|
||||
assert.equal(cache.isWritable(document, key!), writable, label);
|
||||
|
||||
const original = await readFile(path);
|
||||
if (writable) {
|
||||
assert.equal(await cache.delete(document, key!), true, label);
|
||||
} else {
|
||||
await assert.rejects(
|
||||
() => cache.delete(document, key!),
|
||||
(error: unknown) => error instanceof SchemaConflictError && error.unreadSlots?.typeName === typeName && !error.message.includes(SLOT_VALUE),
|
||||
label,
|
||||
);
|
||||
assert.equal(Buffer.compare(original, await readFile(path)), 0, `${label}: the file is unchanged`);
|
||||
}
|
||||
}
|
||||
assert.equal(walked, 7);
|
||||
});
|
||||
|
||||
test("consent: a write of the explained record lands; the over-consent write is refused; the nested mark is refused", async () => {
|
||||
const consented = await openStore("extra-slot-consent.cc", true);
|
||||
const value = consented.cache.get(consented.surface, "consented") as Surface;
|
||||
assert.deepEqual(value, golden);
|
||||
await consented.cache.put(consented.surface, "consented", value);
|
||||
assert.equal(consented.cache.unreadSlots(consented.surface, "consented"), undefined, "the write dropped the explained slot and the mark with it");
|
||||
assert.equal(readSingleFileStore(consented.path).envelopes.find((e) => e.key === "consented")!.schemaId, "tests.union_surface.v2");
|
||||
|
||||
for (const key of ["over-consent", "nested"]) {
|
||||
const refused = await openStore("extra-slot-consent.cc", true);
|
||||
await assert.rejects(
|
||||
() => refused.cache.put(refused.surface, key, golden),
|
||||
(error: unknown) => error instanceof SchemaConflictError && error.schemaId === "tests.union_surface.wide",
|
||||
`${key}: the refusal names the id the record was stored under`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("carried: after writing another record, the marked record's bytes, schema id, stored-at and every catalog entry are the originals", async () => {
|
||||
const { path, cache, surface } = await openStore("extra-slot-store.cc", false);
|
||||
const originals = readSingleFileStore(path);
|
||||
await cache.put(surface, "clean", golden);
|
||||
const after = readSingleFileStore(path);
|
||||
for (const key of ["marked", "unit-arm"]) {
|
||||
const was = originals.envelopes.find((e) => e.key === key)!;
|
||||
const now = after.envelopes.find((e) => e.key === key)!;
|
||||
assert.equal(hex(documentPayloadBytes(now.payload)), hex(documentPayloadBytes(was.payload)), key);
|
||||
assert.equal(now.schemaId, was.schemaId, key);
|
||||
assert.equal(now.storedAt, was.storedAt, key);
|
||||
assert.deepEqual(now.catalogEntry, was.catalogEntry, key);
|
||||
}
|
||||
});
|
||||
|
||||
// A writer that adds a slot and keeps its schema id (TS ids are hand-written; C# ids hash the members): the reader of that
|
||||
// same id has declared nothing, so it consents to nothing.
|
||||
test("a wider writer under the reader's own schema id is read-only, and a real round trip does not shed the slot", async () => {
|
||||
const doc = (columns: 1 | 2, compatibleSchemaIds: string[] = []) => defineDocumentType({
|
||||
type: "tests.own_id",
|
||||
schemaId: "tests.own_id.v1",
|
||||
schemaName: "tests.own_id",
|
||||
compatibleSchemaIds,
|
||||
schema: z.object({ n: z.string(), extra: z.string().optional() }),
|
||||
codec: columns === 2
|
||||
? slots<{ n: string; extra: string }>({ 0: ["n", string], 1: ["extra", string] }, "Own")
|
||||
: slots<{ n: string }>({ 0: ["n", string] }, "Own"),
|
||||
name: "n",
|
||||
});
|
||||
const path = join(await mkdtemp(join(tmpdir(), "slot-readonly-")), "own.cc");
|
||||
const wide = doc(2);
|
||||
const writer = CultCache.builder().withDocumentType(wide).withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
|
||||
await writer.put(wide, "k", { n: "n1", extra: "kept" });
|
||||
const bytes = await readFile(path);
|
||||
|
||||
for (const compatible of [[], ["tests.own_id.v1"]]) {
|
||||
const reader = doc(1, compatible);
|
||||
const cache = CultCache.builder().withDocumentType(reader).withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
|
||||
await cache.pullAllBackingStores();
|
||||
assert.deepEqual(cache.unreadSlots(reader, "k"), mark("Own", 2, 1, true), "unexplained: the arrived id is the reader's own");
|
||||
assert.equal(cache.isWritable(reader, "k"), false);
|
||||
await assert.rejects(() => cache.put(reader, "k", { n: "n2" }), SchemaConflictError);
|
||||
await assert.rejects(() => cache.delete(reader, "k"), SchemaConflictError);
|
||||
assert.equal(Buffer.compare(bytes, await readFile(path)), 0, "the file still holds the slot");
|
||||
}
|
||||
});
|
||||
|
||||
test("a record that arrived under an id the reader does not list compatible is never consented to", async () => {
|
||||
const path = await copyOf("extra-slot-consent.cc");
|
||||
const reader = surfaceDocument("tests.union_surface.v2"); // resolves by type name; lists nothing
|
||||
const cache = CultCache.builder().withDocumentType(reader).withGenericStore(new SingleFileMessagePackBackingStore(path)).build();
|
||||
await cache.pullAllBackingStores();
|
||||
assert.equal(cache.unreadSlots(reader, "consented")?.explained, false);
|
||||
assert.equal(cache.isWritable(reader, "consented"), false);
|
||||
});
|
||||
|
||||
// isWritable is the write's own judge, not a second one: for every key state it answers what put does.
|
||||
test("isWritable equals the outcome of put for every key state the write judges", async () => {
|
||||
const typed = (type: string, schemaId: string, extra: Record<string, unknown> = {}) =>
|
||||
defineDocumentType({ type, schemaId, schemaName: type, schemaVersion: `${type}.v1`, schema: surfaceSchema, codec: surfaceCodec, name: "id", ...extra });
|
||||
const older = typed("tests.surface_older", "tests.surface_older.v1");
|
||||
const newer = typed("tests.surface_newer", "tests.surface_newer.v1", { compatibleSchemaIds: ["tests.surface_older.v1"] });
|
||||
const stranger = typed("tests.surface_stranger", "tests.surface_stranger.v1");
|
||||
const global = typed("tests.surface_global", "tests.surface_global.v1", { global: true });
|
||||
const clean = (key: string, type: string, schemaId: string) => ({ ...envelopeOf(new Uint8Array()), key, type, schemaId });
|
||||
const held = async (key: string, file: string, type: string, schemaId: string) =>
|
||||
({ ...envelopeOf(await read(file)), key, type, schemaId });
|
||||
|
||||
type Row = { name: string; build: () => Promise<{ cache: CultCache; target: typeof older; key: string }> };
|
||||
const rows: Row[] = [
|
||||
{ name: "owned clean record in the writer's own store", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(newer)
|
||||
.withGenericStore(memoryStore([await held("k", "union-surface.msgpack", older.type, older.schemaId!)])).build();
|
||||
return { cache, target: newer, key: "k" };
|
||||
} },
|
||||
{ name: "owned record held in a store other than the writer's home", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(newer)
|
||||
.withBackingStore(memoryStore([await held("k", "union-surface.msgpack", older.type, older.schemaId!)]), older)
|
||||
.withBackingStore(memoryStore([]), newer).build();
|
||||
return { cache, target: newer, key: "k" };
|
||||
} },
|
||||
{ name: "a record of a type that does not own the writer's id", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(stranger)
|
||||
.withGenericStore(memoryStore([await held("k", "union-surface.msgpack", older.type, older.schemaId!)])).build();
|
||||
return { cache, target: stranger, key: "k" };
|
||||
} },
|
||||
{ name: "a global type at a key other than the global key", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(global).withGenericStore(memoryStore([])).build();
|
||||
return { cache, target: global, key: "x" };
|
||||
} },
|
||||
{ name: "a key a store carries foreign", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(older)
|
||||
.withGenericStore(memoryStore([{ ...clean("k", "tests.unknown", "tests.unknown.v1"), payload: documentPayloadFromBytes(await read("union-surface.msgpack")) }])).build();
|
||||
return { cache, target: older, key: "k" };
|
||||
} },
|
||||
{ name: "a marked record of the writer's own type", build: async () => {
|
||||
const cache = CultCache.builder().withDocumentType(older)
|
||||
.withGenericStore(memoryStore([await held("k", "extra-slot-record.msgpack", older.type, older.schemaId!)])).build();
|
||||
return { cache, target: older, key: "k" };
|
||||
} },
|
||||
{ name: "an unheld key", build: async () => ({ cache: CultCache.builder().withDocumentType(older).withGenericStore(memoryStore([])).build(), target: older, key: "k" }) },
|
||||
];
|
||||
const outcomes: [string, boolean, boolean][] = [];
|
||||
for (const row of rows) {
|
||||
const { cache, target, key } = await row.build();
|
||||
await cache.pullAllBackingStores();
|
||||
const predicted = cache.isWritable(target, key);
|
||||
const landed = await cache.put(target, key === "x" ? "x" : key, golden).then(() => true, () => false);
|
||||
assert.equal(predicted, landed, row.name);
|
||||
outcomes.push([row.name, predicted, landed]);
|
||||
}
|
||||
assert.deepEqual(outcomes.map(([, writable]) => writable), [true, false, false, false, false, false, true]);
|
||||
// The same judge refuses a removal of a key held read-only or carried foreign.
|
||||
const removals = CultCache.builder().withDocumentType(older)
|
||||
.withGenericStore(memoryStore([await held("k", "extra-slot-record.msgpack", older.type, older.schemaId!)])).build();
|
||||
await removals.pullAllBackingStores();
|
||||
await assert.rejects(() => removals.delete(older, "k"), SchemaConflictError);
|
||||
const stranger2 = CultCache.builder().withDocumentType(older)
|
||||
.withGenericStore(memoryStore([{ ...envelopeOf(await read("union-surface.msgpack")), key: "f", type: "tests.unknown", schemaId: "tests.unknown.v1" }])).build();
|
||||
await stranger2.pullAllBackingStores();
|
||||
await assert.rejects(() => stranger2.delete(older, "f"), SchemaConflictError);
|
||||
});
|
||||
|
||||
// A removal is by key (C# Remove): the owner of a held record's id removes the one record that exists, in one store call and
|
||||
// with the cache dropped only once the store has; a type that does not own it is refused like put, and isWritable agrees.
|
||||
test("delete of a key held as another type: the owner removes it once, a stranger is refused, a failing store leaves cache and store agreeing", async () => {
|
||||
const typed = (type: string, schemaId: string, extra: Record<string, unknown> = {}) =>
|
||||
defineDocumentType({ type, schemaId, schemaName: type, schemaVersion: `${type}.v1`, schema: surfaceSchema, codec: surfaceCodec, name: "id", ...extra });
|
||||
const older = typed("tests.surface_older", "tests.surface_older.v1");
|
||||
const newer = typed("tests.surface_newer", "tests.surface_newer.v1", { compatibleSchemaIds: ["tests.surface_older.v1"] });
|
||||
const stranger = typed("tests.surface_stranger", "tests.surface_stranger.v1");
|
||||
const heldRecord = async (): Promise<CultCacheEnvelope> => ({ ...envelopeOf(await read("union-surface.msgpack")), key: "k", type: older.type, schemaId: older.schemaId });
|
||||
const open = async (failing = false) => {
|
||||
const rows = [await heldRecord()];
|
||||
const calls: string[] = [];
|
||||
const store: CacheBackingStore = {
|
||||
async pullAll() { return rows.map((row) => ({ ...row })); },
|
||||
async push(entry) { rows.push(entry); },
|
||||
async delete(entry) {
|
||||
calls.push(entry.key);
|
||||
if (failing) throw new Error("store refused the removal");
|
||||
rows.splice(rows.findIndex((row) => row.key === entry.key), 1);
|
||||
},
|
||||
};
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(newer).withDocumentType(stranger).withGenericStore(store).build();
|
||||
await cache.pullAllBackingStores();
|
||||
return { cache, rows, calls };
|
||||
};
|
||||
|
||||
const owner = await open();
|
||||
assert.equal(owner.cache.isWritable(newer, "k"), true);
|
||||
assert.equal(await owner.cache.delete(newer, "k"), true, "the owner's removal removes the one record");
|
||||
assert.deepEqual([owner.rows.length, owner.calls.length, owner.cache.get(older, "k")], [0, 1, undefined], "one store call, record gone from store and cache");
|
||||
|
||||
const refused = await open();
|
||||
assert.equal(refused.cache.isWritable(stranger, "k"), false);
|
||||
await assert.rejects(() => refused.cache.delete(stranger, "k"), SchemaConflictError);
|
||||
assert.deepEqual([refused.rows.length, refused.calls.length, refused.cache.get(older, "k") !== undefined], [1, 0, true]);
|
||||
|
||||
const failing = await open(true);
|
||||
await assert.rejects(() => failing.cache.delete(newer, "k"), /store refused/u);
|
||||
assert.deepEqual([failing.rows.length, failing.calls.length, failing.cache.get(older, "k") !== undefined], [1, 1, true], "the store failed: neither side dropped it");
|
||||
|
||||
const same = await open();
|
||||
assert.equal(await same.cache.delete(older, "k"), true);
|
||||
assert.equal(await same.cache.delete(older, "k"), false, "an unheld key is still the plain false");
|
||||
});
|
||||
|
||||
// The removal form's three finer behaviours (cultlib-gaps:finding:cut-slot-readonly-ts.s4.removal-form-unpinned).
|
||||
test("an owner's delete: refused for a marked record, taken from the held record's store, and clears the held type's lookups", async () => {
|
||||
const typed = (type: string, schemaId: string, extra: Record<string, unknown> = {}) =>
|
||||
defineDocumentType({ type, schemaId, schemaName: type, schemaVersion: `${type}.v1`, schema: surfaceSchema, codec: surfaceCodec, name: "id", ...extra });
|
||||
const older = typed("tests.surface_older", "tests.surface_older.v1", { indexes: { byVersion: (value: Surface) => String(value.version) } });
|
||||
const newer = typed("tests.surface_newer", "tests.surface_newer.v1", { compatibleSchemaIds: ["tests.surface_older.v1"] });
|
||||
const heldIn = async (file: string): Promise<CultCacheEnvelope> =>
|
||||
({ ...envelopeOf(await read(file)), key: "k", type: older.type, schemaId: older.schemaId });
|
||||
const recording = (rows: CultCacheEnvelope[]) => {
|
||||
const removed: string[] = [];
|
||||
const store: CacheBackingStore = {
|
||||
async pullAll() { return rows.map((row) => ({ ...row })); },
|
||||
async push(entry) { rows.push(entry); },
|
||||
async delete(entry) { removed.push(entry.key); rows.splice(rows.findIndex((row) => row.key === entry.key), 1); },
|
||||
};
|
||||
return { store, removed, rows };
|
||||
};
|
||||
|
||||
// 1. The owner may not remove a record carrying unread slots nothing consents to.
|
||||
const marked = recording([await heldIn("extra-slot-record.msgpack")]);
|
||||
const markedCache = CultCache.builder().withDocumentType(older).withDocumentType(newer).withGenericStore(marked.store).build();
|
||||
await markedCache.pullAllBackingStores();
|
||||
await assert.rejects(() => markedCache.delete(newer, "k"), (error: unknown) => error instanceof SchemaConflictError && error.unreadSlots?.typeName === "UnionSurface");
|
||||
assert.deepEqual([marked.rows.length, marked.removed.length, markedCache.get(older, "k") !== undefined], [1, 0, true]);
|
||||
|
||||
// 2. The removal goes to the store of the record that exists, not the deleting type's; there the write form is false (it would
|
||||
// push to the wrong store) while the removal succeeds.
|
||||
const olderStore = recording([await heldIn("union-surface.msgpack")]);
|
||||
const newerStore = recording([]);
|
||||
const split = CultCache.builder().withDocumentType(older).withDocumentType(newer)
|
||||
.withBackingStore(olderStore.store, older).withBackingStore(newerStore.store, newer).build();
|
||||
await split.pullAllBackingStores();
|
||||
assert.equal(split.isWritable(newer, "k"), false, "isWritable answers the write form: the rename cannot replace a record in another store");
|
||||
assert.equal(await split.delete(newer, "k"), true);
|
||||
assert.deepEqual([olderStore.removed, newerStore.removed, split.get(older, "k")], [["k"], [], undefined]);
|
||||
|
||||
// 3. The held type's name and index lookups are cleared with the record.
|
||||
const single = recording([await heldIn("union-surface.msgpack")]);
|
||||
const cache = CultCache.builder().withDocumentType(older).withDocumentType(newer).withGenericStore(single.store).build();
|
||||
await cache.pullAllBackingStores();
|
||||
assert.equal(cache.getKeyByName(older, "surface"), "k");
|
||||
assert.equal(cache.getKeyByIndex(older, "byVersion", "3"), "k");
|
||||
await cache.delete(newer, "k");
|
||||
assert.equal(cache.getKeyByName(older, "surface"), undefined);
|
||||
assert.equal(cache.getKeyByIndex(older, "byVersion", "3"), undefined);
|
||||
assert.equal(cache.getAll(older).length, 0);
|
||||
});
|
||||
|
||||
test("a global type at a key other than the global key: isWritable is false and delete is the plain false of an unheld key", async () => {
|
||||
const global = defineDocumentType({ type: "tests.surface_global", schemaId: "tests.surface_global.v1", schemaName: "tests.surface_global", schemaVersion: "tests.surface_global.v1", schema: surfaceSchema, codec: surfaceCodec, name: "id", global: true });
|
||||
const cache = CultCache.builder().withDocumentType(global).withGenericStore(memoryStore([])).build();
|
||||
assert.equal(cache.isWritable(global, "x"), false);
|
||||
assert.equal(await cache.delete(global, "x"), false);
|
||||
});
|
||||
|
|
@ -1,8 +1,7 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { existsSync } from "node:fs";
|
||||
import { mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { join } from "node:path";
|
||||
import { test } from "node:test";
|
||||
|
||||
import { decode, encode } from "@msgpack/msgpack";
|
||||
|
|
@ -22,190 +21,14 @@ import {
|
|||
encodeDocumentPayload,
|
||||
} from "../src/document";
|
||||
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
|
||||
import type { AnyCultCacheDocumentDefinition, CacheBackingStore, CultCacheEnvelope } from "../src/types";
|
||||
import type { AnyCultCacheDocumentDefinition, CultCacheEnvelope } from "../src/types";
|
||||
|
||||
import {
|
||||
CANARY, type Node, type Surface, golden, hex, node, outcome, read, surfaceCodec, surfaceDocument, surfaceSchema, vectors, envelopeOf, memoryStore,
|
||||
} from "./union-surface-fixture";
|
||||
|
||||
const { array, boolean, bytes, float32, float64, int32, int64, lazy, list, nullable, slots, string, union, unit } = codecs;
|
||||
|
||||
// The names the C# reference gives the fixture's types (CultSchemaTypeNames.FromType), as union-surface.cc carries them.
|
||||
const CS = "GameCult.Caching.Tests.UnionParityVectorTests+";
|
||||
|
||||
// The golden is tests/vectors/union-parity, written by the C# reference. Everything below that is bytes or an expected
|
||||
// outcome is read from those committed files; only the value the payload must decode to is spelled here.
|
||||
function findVectors(from: string): string {
|
||||
for (let dir = from; ; dir = resolve(dir, "..")) {
|
||||
if (existsSync(join(dir, "tests", "vectors", "union-parity"))) return join(dir, "tests", "vectors", "union-parity");
|
||||
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/union-parity not found above the test.");
|
||||
}
|
||||
}
|
||||
const vectors = findVectors(__dirname);
|
||||
const read = async (name: string) => new Uint8Array(await readFile(join(vectors, name)));
|
||||
|
||||
// The string the C# writer put inside every refusal payload, to prove a refusal never echoes the payload.
|
||||
const CANARY = "CANARY-union-golden-4c2a";
|
||||
|
||||
type Node =
|
||||
| { kind: "Box"; id: string; children: (Node | null)[] }
|
||||
| { kind: "Text"; id: string; content: string }
|
||||
| { kind: "Use"; id: string; composition: Composition; slots: (Node | null)[] }
|
||||
| { kind: "Image"; id: string; uri: string; alt: string }
|
||||
| { kind: "Wrap"; inner: Node | null; id: string };
|
||||
interface Composition { name: string; body: Node | null }
|
||||
type Outcome = { kind: "Pending"; message: string } | { kind: "Completed"; sourceVersion: number } | { kind: "Denied" } | { kind: "Floor" } | { kind: "Ceiling" };
|
||||
// The golden's Outcome: the arms of a list carry an element id, so Denied, Floor and Ceiling are one-slot records.
|
||||
type GoldenOutcome =
|
||||
| { kind: "Pending"; message: string; id: string }
|
||||
| { kind: "Completed"; sourceVersion: number; id: string }
|
||||
| { kind: "Denied"; id: string }
|
||||
| { kind: "Floor"; id: string }
|
||||
| { kind: "Ceiling"; id: string };
|
||||
type Item = { kind: "Gem"; name: string } | { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
|
||||
type CraftedItem = { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
|
||||
interface Surface {
|
||||
id: string;
|
||||
version: number;
|
||||
root: Node | null;
|
||||
outcomes: (GoldenOutcome | null)[];
|
||||
part: Item | null;
|
||||
crafted: CraftedItem | null;
|
||||
missing: Node | null;
|
||||
}
|
||||
|
||||
const composition = slots<Composition>({ 0: ["name", string], 1: ["body", lazy(() => node)] }, `${CS}Composition`);
|
||||
const node: Codec<Node | null> = union<Node>("Node", {
|
||||
0: ["Box", slots<{ id: string; children: (Node | null)[] }>({ 0: ["id", string], 1: ["children", list(lazy(() => node))] })],
|
||||
1: ["Text", slots<{ id: string; content: string }>({ 0: ["id", string], 1: ["content", string] })],
|
||||
2: ["Use", slots<{ id: string; composition: Composition; slots: (Node | null)[] }>({
|
||||
0: ["id", string], 1: ["composition", composition], 2: ["slots", list(lazy(() => node))],
|
||||
})],
|
||||
5: ["Image", slots<{ id: string; uri: string; alt: string }>({ 0: ["id", string], 1: ["uri", string], 3: ["alt", string] })],
|
||||
6: ["Wrap", slots<{ inner: Node | null; id: string }>({ 0: ["inner", lazy(() => node)], 1: ["id", string] })],
|
||||
}, `${CS}Node`);
|
||||
const outcome = union<Outcome>("Outcome", {
|
||||
0: ["Pending", slots<{ message: string }>({ 0: ["message", string] })],
|
||||
1: ["Completed", slots<{ sourceVersion: number }>({ 0: ["sourceVersion", int32] })],
|
||||
2: ["Denied", unit],
|
||||
[-2147483648]: ["Floor", unit],
|
||||
2147483647: ["Ceiling", unit],
|
||||
}, `${CS}Outcome`);
|
||||
const idOnly = slots<{ id: string }>({ 0: ["id", string] });
|
||||
const goldenOutcome = union<GoldenOutcome>("Outcome", {
|
||||
0: ["Pending", slots<{ message: string; id: string }>({ 0: ["message", string], 1: ["id", string] })],
|
||||
1: ["Completed", slots<{ sourceVersion: number; id: string }>({ 0: ["sourceVersion", int32], 1: ["id", string] })],
|
||||
2: ["Denied", idOnly],
|
||||
[-2147483648]: ["Floor", idOnly],
|
||||
2147483647: ["Ceiling", idOnly],
|
||||
}, `${CS}Outcome`);
|
||||
const toolArm = slots<{ name: string; uses: number }>({ 0: ["name", string], 1: ["uses", int32] });
|
||||
const partArm = slots<{ name: string; count: number }>({ 0: ["name", string], 1: ["count", int32] });
|
||||
const item = union<Item>("Item", {
|
||||
0: ["Gem", slots<{ name: string }>({ 0: ["name", string] })],
|
||||
1: ["Tool", toolArm],
|
||||
2: ["Part", partArm],
|
||||
}, `${CS}Item`);
|
||||
const craftedItem = union<CraftedItem>("CraftedItem", { 0: ["Tool", toolArm], 1: ["Part", partArm] }, `${CS}CraftedItem`);
|
||||
const surfaceCodec = slots<Surface>({
|
||||
0: ["id", string],
|
||||
1: ["version", int32],
|
||||
2: ["root", node],
|
||||
3: ["outcomes", list(goldenOutcome)],
|
||||
4: ["part", item],
|
||||
5: ["crafted", craftedItem],
|
||||
6: ["missing", node, null],
|
||||
}, `${CS}UnionSurface`);
|
||||
|
||||
const nodeSchema: z.ZodType<Node | null> = z.lazy(() =>
|
||||
z.union([
|
||||
z.null(),
|
||||
z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Box"), id: z.string(), children: z.array(nodeSchema) }),
|
||||
z.object({ kind: z.literal("Text"), id: z.string(), content: z.string() }),
|
||||
z.object({
|
||||
kind: z.literal("Use"),
|
||||
id: z.string(),
|
||||
composition: z.object({ name: z.string(), body: nodeSchema }),
|
||||
slots: z.array(nodeSchema),
|
||||
}),
|
||||
z.object({ kind: z.literal("Image"), id: z.string(), uri: z.string(), alt: z.string() }),
|
||||
z.object({ kind: z.literal("Wrap"), inner: nodeSchema, id: z.string() }),
|
||||
]),
|
||||
]));
|
||||
const surfaceSchema = z.object({
|
||||
id: z.string(),
|
||||
version: z.number(),
|
||||
root: nodeSchema,
|
||||
outcomes: z.array(z.union([
|
||||
z.null(),
|
||||
z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Pending"), message: z.string(), id: z.string() }),
|
||||
z.object({ kind: z.literal("Completed"), sourceVersion: z.number(), id: z.string() }),
|
||||
z.object({ kind: z.literal("Denied"), id: z.string() }),
|
||||
z.object({ kind: z.literal("Floor"), id: z.string() }),
|
||||
z.object({ kind: z.literal("Ceiling"), id: z.string() }),
|
||||
]),
|
||||
])),
|
||||
part: z.union([z.null(), z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Gem"), name: z.string() }),
|
||||
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
|
||||
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
|
||||
])]),
|
||||
crafted: z.union([z.null(), z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
|
||||
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
|
||||
])]),
|
||||
missing: nodeSchema,
|
||||
}) as unknown as z.ZodType<Surface>;
|
||||
|
||||
function surfaceDocument(schemaId?: string) {
|
||||
return defineDocumentType({
|
||||
type: "tests.union_surface",
|
||||
schemaId: schemaId ?? "tests.union_surface.v1",
|
||||
schemaName: "tests.union_surface",
|
||||
schemaVersion: "tests.union_surface.v1",
|
||||
schema: surfaceSchema,
|
||||
codec: surfaceCodec,
|
||||
name: "id",
|
||||
});
|
||||
}
|
||||
|
||||
const golden: Surface = {
|
||||
id: "surface",
|
||||
version: 3,
|
||||
root: {
|
||||
kind: "Box",
|
||||
id: "0000000000c0",
|
||||
children: [
|
||||
{ kind: "Text", id: "0000000000c1", content: "hello" },
|
||||
{
|
||||
kind: "Use",
|
||||
id: "0000000000c2",
|
||||
composition: {
|
||||
name: "comp",
|
||||
body: {
|
||||
kind: "Box",
|
||||
id: "0000000000d0",
|
||||
children: [
|
||||
{ kind: "Text", id: "0000000000d1", content: "x" },
|
||||
{ kind: "Image", id: "0000000000d2", uri: "uri://a", alt: "alt" },
|
||||
],
|
||||
},
|
||||
},
|
||||
slots: [{ kind: "Text", id: "0000000000e1", content: "slot" }, null, { kind: "Box", id: "0000000000e2", children: [] }],
|
||||
},
|
||||
{ kind: "Image", id: "0000000000c3", uri: "uri://b", alt: "" },
|
||||
],
|
||||
},
|
||||
outcomes: [
|
||||
{ kind: "Pending", message: "wait", id: "0000000000a1" },
|
||||
{ kind: "Completed", sourceVersion: 7, id: "0000000000a2" },
|
||||
{ kind: "Denied", id: "0000000000a3" },
|
||||
],
|
||||
part: { kind: "Tool", name: "hammer", uses: 12 },
|
||||
crafted: { kind: "Part", name: "gear", count: 4 },
|
||||
missing: null,
|
||||
};
|
||||
|
||||
const hex = (data: Uint8Array) => Buffer.from(data).toString("hex");
|
||||
|
||||
async function outcomeEntries() {
|
||||
const entries = decode(await read("outcomes.msgpack")) as [string, string, string, number | null, string][];
|
||||
return entries.map(([file, outcome, unionName, key, detail]) => ({ file, outcome, unionName, key, detail }));
|
||||
|
|
@ -729,19 +552,6 @@ test("the Int32 edges of an arm key: the largest is an arm, one past either end
|
|||
assert.throws(() => edge.decode(readWire(fromHex("92d3ffffffff8000000090"))), { message: "Union U: unknown arm key -2147483648 (declared keys: 2147483647)." });
|
||||
});
|
||||
|
||||
function envelopeOf(payload: Uint8Array): CultCacheEnvelope {
|
||||
return { key: "surface", type: "tests.union_surface", payload: documentPayloadFromBytes(payload), storedAt: new Date().toISOString() };
|
||||
}
|
||||
|
||||
function memoryStore(entries: CultCacheEnvelope[]): CacheBackingStore & { entries: CultCacheEnvelope[] } {
|
||||
return {
|
||||
entries,
|
||||
async pullAll() { return entries.map((entry) => ({ ...entry })); },
|
||||
async push(entry) { entries.push(entry); },
|
||||
async delete() {},
|
||||
};
|
||||
}
|
||||
|
||||
test("arm keys span Int32 and are listed ascending; slot keys are non-negative Int32", () => {
|
||||
const keys = union<{ kind: "High" | "Zero" | "Neg" | "Low" }>("U", {
|
||||
2147483647: ["High", unit],
|
||||
|
|
|
|||
240
packages/cultcache-ts/test/union-surface-fixture.ts
Normal file
240
packages/cultcache-ts/test/union-surface-fixture.ts
Normal file
|
|
@ -0,0 +1,240 @@
|
|||
import { existsSync } from "node:fs";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import { codecs } from "../src";
|
||||
import type { Codec } from "../src/codec";
|
||||
import { defineDocumentType, documentPayloadFromBytes } from "../src/document";
|
||||
import type { CacheBackingStore, CultCacheEnvelope } from "../src/types";
|
||||
|
||||
// The union-surface fixture: the codecs, schema and value tests/vectors/union-parity was written from, shared by the
|
||||
// union-parity walker and the slot-readonly walker so the shape is spelled once.
|
||||
const { list, int32, lazy, slots, string, union, unit } = codecs;
|
||||
|
||||
|
||||
// The names the C# reference gives the fixture's types (CultSchemaTypeNames.FromType), as union-surface.cc carries them.
|
||||
export const CS = "GameCult.Caching.Tests.UnionParityVectorTests+";
|
||||
|
||||
// The golden is tests/vectors/union-parity, written by the C# reference. Everything below that is bytes or an expected
|
||||
// outcome is read from those committed files; only the value the payload must decode to is spelled here.
|
||||
export function findVectors(from: string): string {
|
||||
for (let dir = from; ; dir = resolve(dir, "..")) {
|
||||
if (existsSync(join(dir, "tests", "vectors", "union-parity"))) return join(dir, "tests", "vectors", "union-parity");
|
||||
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/union-parity not found above the test.");
|
||||
}
|
||||
}
|
||||
export const vectors = findVectors(__dirname);
|
||||
export const read = async (name: string) => new Uint8Array(await readFile(join(vectors, name)));
|
||||
|
||||
// The string the C# writer put inside every refusal payload, to prove a refusal never echoes the payload.
|
||||
export const CANARY = "CANARY-union-golden-4c2a";
|
||||
|
||||
export type Node =
|
||||
| { kind: "Box"; id: string; children: (Node | null)[] }
|
||||
| { kind: "Text"; id: string; content: string }
|
||||
| { kind: "Use"; id: string; composition: Composition; slots: (Node | null)[] }
|
||||
| { kind: "Image"; id: string; uri: string; alt: string }
|
||||
| { kind: "Wrap"; inner: Node | null; id: string };
|
||||
export interface Composition { name: string; body: Node | null }
|
||||
export type Outcome = { kind: "Pending"; message: string } | { kind: "Completed"; sourceVersion: number } | { kind: "Denied" } | { kind: "Floor" } | { kind: "Ceiling" };
|
||||
// The golden's Outcome: the arms of a list carry an element id, so Denied, Floor and Ceiling are one-slot records.
|
||||
export type GoldenOutcome =
|
||||
| { kind: "Pending"; message: string; id: string }
|
||||
| { kind: "Completed"; sourceVersion: number; id: string }
|
||||
| { kind: "Denied"; id: string }
|
||||
| { kind: "Floor"; id: string }
|
||||
| { kind: "Ceiling"; id: string };
|
||||
export type Item = { kind: "Gem"; name: string } | { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
|
||||
export type CraftedItem = { kind: "Tool"; name: string; uses: number } | { kind: "Part"; name: string; count: number };
|
||||
export interface Surface {
|
||||
id: string;
|
||||
version: number;
|
||||
root: Node | null;
|
||||
outcomes: (GoldenOutcome | null)[];
|
||||
part: Item | null;
|
||||
crafted: CraftedItem | null;
|
||||
missing: Node | null;
|
||||
}
|
||||
|
||||
export const composition = slots<Composition>({ 0: ["name", string], 1: ["body", lazy(() => node)] }, `${CS}Composition`);
|
||||
export const node: Codec<Node | null> = union<Node>("Node", {
|
||||
0: ["Box", slots<{ id: string; children: (Node | null)[] }>({ 0: ["id", string], 1: ["children", list(lazy(() => node))] }, `${CS}Box`)],
|
||||
1: ["Text", slots<{ id: string; content: string }>({ 0: ["id", string], 1: ["content", string] }, `${CS}Text`)],
|
||||
2: ["Use", slots<{ id: string; composition: Composition; slots: (Node | null)[] }>({
|
||||
0: ["id", string], 1: ["composition", composition], 2: ["slots", list(lazy(() => node))],
|
||||
})],
|
||||
5: ["Image", slots<{ id: string; uri: string; alt: string }>({ 0: ["id", string], 1: ["uri", string], 3: ["alt", string] })],
|
||||
6: ["Wrap", slots<{ inner: Node | null; id: string }>({ 0: ["inner", lazy(() => node)], 1: ["id", string] })],
|
||||
}, `${CS}Node`);
|
||||
export const outcome = union<Outcome>("Outcome", {
|
||||
0: ["Pending", slots<{ message: string }>({ 0: ["message", string] })],
|
||||
1: ["Completed", slots<{ sourceVersion: number }>({ 0: ["sourceVersion", int32] })],
|
||||
2: ["Denied", unit],
|
||||
[-2147483648]: ["Floor", unit],
|
||||
2147483647: ["Ceiling", unit],
|
||||
}, `${CS}Outcome`);
|
||||
export const idOnly = slots<{ id: string }>({ 0: ["id", string] });
|
||||
export const goldenOutcome = union<GoldenOutcome>("Outcome", {
|
||||
0: ["Pending", slots<{ message: string; id: string }>({ 0: ["message", string], 1: ["id", string] })],
|
||||
1: ["Completed", slots<{ sourceVersion: number; id: string }>({ 0: ["sourceVersion", int32], 1: ["id", string] })],
|
||||
2: ["Denied", idOnly],
|
||||
[-2147483648]: ["Floor", idOnly],
|
||||
2147483647: ["Ceiling", idOnly],
|
||||
}, `${CS}Outcome`);
|
||||
export const toolArm = slots<{ name: string; uses: number }>({ 0: ["name", string], 1: ["uses", int32] });
|
||||
export const partArm = slots<{ name: string; count: number }>({ 0: ["name", string], 1: ["count", int32] });
|
||||
export const item = union<Item>("Item", {
|
||||
0: ["Gem", slots<{ name: string }>({ 0: ["name", string] })],
|
||||
1: ["Tool", toolArm],
|
||||
2: ["Part", partArm],
|
||||
}, `${CS}Item`);
|
||||
export const craftedItem = union<CraftedItem>("CraftedItem", { 0: ["Tool", toolArm], 1: ["Part", partArm] }, `${CS}CraftedItem`);
|
||||
export const surfaceCodecOf = (typeName: string) => slots<Surface>({
|
||||
0: ["id", string],
|
||||
1: ["version", int32],
|
||||
2: ["root", node],
|
||||
3: ["outcomes", list(goldenOutcome)],
|
||||
4: ["part", item],
|
||||
5: ["crafted", craftedItem],
|
||||
6: ["missing", node, null],
|
||||
}, typeName);
|
||||
export const surfaceCodec = surfaceCodecOf(`${CS}UnionSurface`);
|
||||
|
||||
export const nodeSchema: z.ZodType<Node | null> = z.lazy(() =>
|
||||
z.union([
|
||||
z.null(),
|
||||
z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Box"), id: z.string(), children: z.array(nodeSchema) }),
|
||||
z.object({ kind: z.literal("Text"), id: z.string(), content: z.string() }),
|
||||
z.object({
|
||||
kind: z.literal("Use"),
|
||||
id: z.string(),
|
||||
composition: z.object({ name: z.string(), body: nodeSchema }),
|
||||
slots: z.array(nodeSchema),
|
||||
}),
|
||||
z.object({ kind: z.literal("Image"), id: z.string(), uri: z.string(), alt: z.string() }),
|
||||
z.object({ kind: z.literal("Wrap"), inner: nodeSchema, id: z.string() }),
|
||||
]),
|
||||
]));
|
||||
export const surfaceSchema = z.object({
|
||||
id: z.string(),
|
||||
version: z.number(),
|
||||
root: nodeSchema,
|
||||
outcomes: z.array(z.union([
|
||||
z.null(),
|
||||
z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Pending"), message: z.string(), id: z.string() }),
|
||||
z.object({ kind: z.literal("Completed"), sourceVersion: z.number(), id: z.string() }),
|
||||
z.object({ kind: z.literal("Denied"), id: z.string() }),
|
||||
z.object({ kind: z.literal("Floor"), id: z.string() }),
|
||||
z.object({ kind: z.literal("Ceiling"), id: z.string() }),
|
||||
]),
|
||||
])),
|
||||
part: z.union([z.null(), z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Gem"), name: z.string() }),
|
||||
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
|
||||
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
|
||||
])]),
|
||||
crafted: z.union([z.null(), z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("Tool"), name: z.string(), uses: z.number() }),
|
||||
z.object({ kind: z.literal("Part"), name: z.string(), count: z.number() }),
|
||||
])]),
|
||||
missing: nodeSchema,
|
||||
}) as unknown as z.ZodType<Surface>;
|
||||
|
||||
// The reference's SurfaceDeclaring: the same layout, schema tests.union_surface.v2, listing the older wide id compatible.
|
||||
export function surfaceDeclaringDocument() {
|
||||
return defineDocumentType({
|
||||
type: "tests.union_surface",
|
||||
schemaId: "tests.union_surface.v2",
|
||||
schemaName: "tests.union_surface",
|
||||
schemaVersion: "tests.union_surface.v2",
|
||||
compatibleSchemaIds: ["tests.union_surface.wide"],
|
||||
schema: surfaceSchema,
|
||||
codec: surfaceCodecOf(`${CS}SurfaceDeclaring`),
|
||||
name: "id",
|
||||
});
|
||||
}
|
||||
|
||||
export function surfaceDocument(schemaId?: string, contentHash?: string) {
|
||||
return defineDocumentType({
|
||||
...(contentHash === undefined ? {} : { contentHash }),
|
||||
type: "tests.union_surface",
|
||||
schemaId: schemaId ?? "tests.union_surface.v1",
|
||||
schemaName: "tests.union_surface",
|
||||
schemaVersion: "tests.union_surface.v1",
|
||||
schema: surfaceSchema,
|
||||
codec: surfaceCodec,
|
||||
name: "id",
|
||||
});
|
||||
}
|
||||
|
||||
export const golden: Surface = {
|
||||
id: "surface",
|
||||
version: 3,
|
||||
root: {
|
||||
kind: "Box",
|
||||
id: "0000000000c0",
|
||||
children: [
|
||||
{ kind: "Text", id: "0000000000c1", content: "hello" },
|
||||
{
|
||||
kind: "Use",
|
||||
id: "0000000000c2",
|
||||
composition: {
|
||||
name: "comp",
|
||||
body: {
|
||||
kind: "Box",
|
||||
id: "0000000000d0",
|
||||
children: [
|
||||
{ kind: "Text", id: "0000000000d1", content: "x" },
|
||||
{ kind: "Image", id: "0000000000d2", uri: "uri://a", alt: "alt" },
|
||||
],
|
||||
},
|
||||
},
|
||||
slots: [{ kind: "Text", id: "0000000000e1", content: "slot" }, null, { kind: "Box", id: "0000000000e2", children: [] }],
|
||||
},
|
||||
{ kind: "Image", id: "0000000000c3", uri: "uri://b", alt: "" },
|
||||
],
|
||||
},
|
||||
outcomes: [
|
||||
{ kind: "Pending", message: "wait", id: "0000000000a1" },
|
||||
{ kind: "Completed", sourceVersion: 7, id: "0000000000a2" },
|
||||
{ kind: "Denied", id: "0000000000a3" },
|
||||
],
|
||||
part: { kind: "Tool", name: "hammer", uses: 12 },
|
||||
crafted: { kind: "Part", name: "gear", count: 4 },
|
||||
missing: null,
|
||||
};
|
||||
export const hex = (data: Uint8Array) => Buffer.from(data).toString("hex");
|
||||
|
||||
// tests.slot_unit: a document [Name, Outcome: UnitOutcome], whose one arm Denied2 is a unit arm (extra-slot-store.cc, record
|
||||
// unit-arm). The reader's names are the reference's: a mark names Denied2.
|
||||
export const unitOutcome = union<{ kind: "Denied2" }>("UnitOutcome", { 2: ["Denied2", unit] }, `${CS}UnitOutcome`);
|
||||
export function slotUnitDocument(schemaId: string, contentHash?: string) {
|
||||
return defineDocumentType({
|
||||
...(contentHash === undefined ? {} : { contentHash }),
|
||||
type: "tests.slot_unit",
|
||||
schemaId,
|
||||
schemaName: "tests.slot_unit",
|
||||
schemaVersion: "tests.slot_unit.v1",
|
||||
schema: z.object({ name: z.string(), outcome: z.union([z.null(), z.object({ kind: z.literal("Denied2") })]) }),
|
||||
codec: slots<{ name: string; outcome: { kind: "Denied2" } | null }>({ 0: ["name", string], 1: ["outcome", unitOutcome] }, `${CS}SlotUnit`),
|
||||
name: "name",
|
||||
});
|
||||
}
|
||||
|
||||
export function envelopeOf(payload: Uint8Array): CultCacheEnvelope {
|
||||
return { key: "surface", type: "tests.union_surface", payload: documentPayloadFromBytes(payload), storedAt: new Date().toISOString() };
|
||||
}
|
||||
|
||||
export function memoryStore(entries: CultCacheEnvelope[]): CacheBackingStore & { entries: CultCacheEnvelope[] } {
|
||||
return {
|
||||
entries,
|
||||
async pullAll() { return entries.map((entry) => ({ ...entry })); },
|
||||
async push(entry) { entries.push(entry); },
|
||||
async delete() {},
|
||||
};
|
||||
}
|
||||
|
||||
Loading…
Reference in a new issue