cultcache-ts and cultnet-ts: indexes means unique, roles names selection aliases; commitId/commitCount on change messages #92
18 changed files with 1072 additions and 35 deletions
|
|
@ -13,8 +13,8 @@
|
|||
"document": { "$ref": "./cultnet.raw-document-record.schema.json" },
|
||||
"recordKey": { "type": "string", "minLength": 1 },
|
||||
"schemaId": { "type": "string", "minLength": 1 },
|
||||
"commitId": { "type": "string", "minLength": 1 },
|
||||
"commitCount": { "type": "integer", "minimum": 2 }
|
||||
"commitId": { "type": ["string", "null"], "minLength": 1 },
|
||||
"commitCount": { "type": ["integer", "null"], "minimum": 2 }
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
|
|
|
|||
|
|
@ -94,7 +94,6 @@ const itemDocument = defineDocumentType({
|
|||
schemaVersion: "gamecult.first_call_item.v1",
|
||||
schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }),
|
||||
name: "name",
|
||||
indexes: { category: "category" },
|
||||
});
|
||||
const registry = defineDocumentRegistry(itemDocument);
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,15 @@ All notable changes to `@gamecult/cultcache-ts` are recorded here, following
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
### Breaking
|
||||
|
||||
- `indexes` are unique, as C# `[CultIndex]`: a write or a load that leaves two records of a type holding one non-blank
|
||||
value under one alias is refused naming both keys and the alias, and writes nothing. Index text is the C# canonical
|
||||
text (booleans `True`/`False`, integers as digits, doubles as the exact expansion, blank values never indexed), so
|
||||
an empty string is no longer a value. `registerIndex` over held duplicates installs nothing. A new `roles` map names
|
||||
the many-row selection aliases (never indexed for lookup, same catalog `indexAlias`). The per-key migration rule is
|
||||
in the README; classify every key before the CultLib your tools load is updated. The refusal verdict is the contract, its text is informational.
|
||||
|
||||
### Changed
|
||||
|
||||
- Opening a legacy (pre-v1) store is a pure read: payloads that were not bytes are normalized in memory and the
|
||||
|
|
|
|||
|
|
@ -93,12 +93,16 @@ const itemDocument = defineDocumentType({
|
|||
schemaVersion: "item.v1",
|
||||
schema: z.object({
|
||||
name: z.string(),
|
||||
sku: z.string(),
|
||||
category: z.string(),
|
||||
value: z.number().int(),
|
||||
}),
|
||||
name: "name",
|
||||
indexes: {
|
||||
category: "category",
|
||||
sku: "sku", // unique: a sku names one item
|
||||
},
|
||||
roles: {
|
||||
category: "category", // a selection alias: many items share a category
|
||||
},
|
||||
});
|
||||
|
||||
|
|
@ -122,6 +126,7 @@ await cache.pullAllBackingStores();
|
|||
|
||||
await cache.put(itemDocument, "item:potion", {
|
||||
name: "Potion",
|
||||
sku: "POT-001",
|
||||
category: "Consumable",
|
||||
value: 50,
|
||||
});
|
||||
|
|
@ -132,7 +137,7 @@ await cache.putGlobal(settingsDocument, {
|
|||
});
|
||||
|
||||
const potion = cache.getByName(itemDocument, "Potion");
|
||||
const consumable = cache.getByIndex(itemDocument, "category", "Consumable");
|
||||
const bySku = cache.getByIndex(itemDocument, "sku", "POT-001");
|
||||
const settings = cache.getRequiredGlobal(settingsDocument);
|
||||
```
|
||||
|
||||
|
|
@ -194,7 +199,7 @@ A document type has exactly one home store
|
|||
|
||||
## Name, Index, and Global Semantics
|
||||
|
||||
`name` and `indexes` can be declared on the document definition:
|
||||
`name`, `indexes` and `roles` can be declared on the document definition:
|
||||
|
||||
```ts
|
||||
const playerDocument = defineDocumentType({
|
||||
|
|
@ -202,17 +207,70 @@ const playerDocument = defineDocumentType({
|
|||
schema: playerSchema,
|
||||
name: "displayName",
|
||||
indexes: {
|
||||
faction: "faction",
|
||||
email: "email",
|
||||
email: "email", // unique lookup
|
||||
},
|
||||
roles: {
|
||||
faction: "faction", // selection alias, many players share one
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
`indexes` are unique (variants:ruling:unique-indexes; C# `[CultIndex]`, Rust `#[cultcache(index)]`). Within one type
|
||||
and one alias no two records hold one non-blank value, judged on the result of every write and of every load; a
|
||||
refusal reads `Records <k1> and <k2> (<schema name>) both hold unique index '<alias>'; an index value names one record.`
|
||||
(keys in ordinal order, never the value), and writes nothing. The contract is the verdict, which stores and writes are
|
||||
refused; the text is informational. On a load TS judges record by record in file order and names the first collision
|
||||
it meets, where C# judges the whole store, so the two runtimes can name different pairs of one store. A blank value (absent, empty, or only whitespace) is
|
||||
never indexed and never conflicts. Two types may share an alias and a value. Index values are compared as the C#
|
||||
reference renders them: strings as is (never trimmed), `True`/`False`, integers as digits, a non-integral number as
|
||||
its exact decimal expansion; NaN and infinities have no value. `roles` name selection and reference aliases (the
|
||||
member's catalog `indexAlias`, so the catalog bytes are the same either way); they imply nothing about uniqueness and
|
||||
are never indexed for lookup, so `getByIndex` on a role answers `undefined`. An alias is declared in one of the two.
|
||||
|
||||
### Migrating an existing `indexes` key
|
||||
|
||||
Apply per key, and do it BEFORE the CultLib checkout your tools load is rebuilt or updated. A consumer that loads
|
||||
CultLib at run time (Bifrost reads `$VOIDBOT_CULTLIB_ROOT`, else the sibling `../CultLib` dist) has no pin to hold: its
|
||||
next start after the checkout moves is the bump. A consumer that pins a version applies this before it bumps the pin.
|
||||
Moving an `indexes` key to `roles` is safe on the old CultLib too when the definition lists `members` explicitly (an old
|
||||
codec definition without `members` derives `indexAlias` from `indexes` only, so a moved key would lose its catalog alias),
|
||||
so the consumer edit lands first:
|
||||
|
||||
1. Nothing looks records up by the key (`getByIndex`, `getKeyByIndex`, `getIdByIndex`; Python `get_by_index`,
|
||||
`get_key_by_index`, `node.get_by_index`, `watch_by_index`; Rust `get_by_index`): it is a role. Move it to `roles`;
|
||||
where a definition lists `members` explicitly, delete it from `indexes` and keep the member's `indexAlias` (the
|
||||
catalog is the same).
|
||||
2. Something looks it up: it stays unique only when two live records can never hold one non-blank value, because the
|
||||
record key is derived from that value alone, or the value is an outside identity (email, sku, external id).
|
||||
Enums, statuses, kinds, categories, parent or foreign ids, shared hashes and timestamps are roles; a lookup on
|
||||
one already answered an arbitrary record, so move it to a selection (`any_of`) or `getAll` plus a filter.
|
||||
3. Before the checkout moves, open each real `.cc` once with the new definitions. A refusal names both keys and the alias and
|
||||
writes nothing.
|
||||
|
||||
Misclassifying costs differently each way. A role declared unique: the second record holding a shared value is
|
||||
refused at commit, and a store already holding one refuses to open, so that store is down until the key is
|
||||
reclassified (loud, names both keys and the alias). A unique key declared a role: lookups by it answer nothing and
|
||||
duplicates stop being refused.
|
||||
|
||||
The Bifrost TS stores (at 781e6ea) call no lookup anywhere, so by rule 1 every key is a role and no key becomes unique.
|
||||
All six definitions list `members` explicitly, each with its `indexAlias`, so the edit is deleting each `indexes` block:
|
||||
the catalog is unchanged and the edit is safe on the CultLib it loads today. It must land before that checkout is
|
||||
updated:
|
||||
|
||||
| Definition | Keys | Verdict |
|
||||
|---|---|---|
|
||||
| `update-request` (`tools/agent-transport.mjs`) | `targetRepoName`, `targetAgentIdentity`, `status` | role: many requests per repo and agent; status is an enum |
|
||||
| `crossing_receipt` (`tools/bifrost-crossing-documents.mjs`) | `commandId`, `crossingKind`, `status` | role: receipts are `crossing_<commandId>_<activityKind>`, several per command; the rest are enums |
|
||||
| release authority (`tools/bifrost-repository-release-authority.mjs`) | `repositoryFullName`, `upstreamRef`, `commitSha`, `status` | role: the authority id is derived from repo, ref and sha, so one sha under two refs is two records |
|
||||
| `topic` (`tools/governance-threads.mjs`) | `jurisdictionRepoName`, `jurisdictionAgentIdentity`, `status` | role: many topics per jurisdiction |
|
||||
| `comment` (`tools/governance-threads.mjs`) | `topicId`, `authorId`, `stance` | role: many comments per topic; stance is an enum |
|
||||
| `advertisement` (`tools/provider-advertisement.mjs`) | `serviceName` | role: one record holds it today, but nothing looks it up |
|
||||
|
||||
You can also register them later if you feel like changing the machine after boot:
|
||||
|
||||
```ts
|
||||
await cache.registerNameLookup(playerDocument, "displayName");
|
||||
await cache.registerIndex(playerDocument, "faction", "faction");
|
||||
await cache.registerIndex(playerDocument, "email", "email");
|
||||
```
|
||||
|
||||
Registration, `addBackingStore`, `pullAllBackingStores` and every write and
|
||||
|
|
|
|||
|
|
@ -224,7 +224,7 @@ export class CultCache {
|
|||
return this.#serial(() => {
|
||||
const registered = this.#requireDefinition(definition);
|
||||
const nameAccessor = this.#compileAccessor(definition.type, "name", accessor);
|
||||
const derived = this.#deriveTypeLookups(definition.type, nameAccessor, registered.indexAccessors);
|
||||
const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, nameAccessor, registered.indexAccessors);
|
||||
this.#installAccessors(registered, nameAccessor, registered.indexAccessors, derived);
|
||||
return definition;
|
||||
});
|
||||
|
|
@ -243,7 +243,7 @@ export class CultCache {
|
|||
const registered = this.#requireDefinition(definition);
|
||||
const indexAccessors = new Map(registered.indexAccessors);
|
||||
indexAccessors.set(indexName, this.#compileAccessor(definition.type, `index "${indexName}"`, accessor));
|
||||
const derived = this.#deriveTypeLookups(definition.type, registered.nameAccessor, indexAccessors);
|
||||
const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, registered.nameAccessor, indexAccessors);
|
||||
this.#installAccessors(registered, registered.nameAccessor, indexAccessors, derived);
|
||||
return definition;
|
||||
});
|
||||
|
|
@ -418,6 +418,7 @@ export class CultCache {
|
|||
return key ? this.get(definition, key) : undefined;
|
||||
}
|
||||
|
||||
// A unique index: one key per non-blank value. A roles alias is never indexed and answers undefined.
|
||||
getKeyByIndex<TDefinition extends AnyCultCacheDocumentDefinition>(
|
||||
definition: TDefinition,
|
||||
indexName: string,
|
||||
|
|
@ -579,7 +580,7 @@ export class CultCache {
|
|||
if (existingBySchemaName && existingBySchemaName !== registered) {
|
||||
throw refuse(existingBySchemaName, registered.catalogEntry, `schema name "${registered.catalogEntry.schemaName}"`);
|
||||
}
|
||||
const derived = this.#deriveTypeLookups(definition.type, nameAccessor, indexAccessors);
|
||||
const derived = this.#deriveTypeLookups(definition.type, registered.catalogEntry.schemaName, nameAccessor, indexAccessors);
|
||||
|
||||
// Every check and accessor has run; install.
|
||||
this.#definitions.set(definition.type, registered);
|
||||
|
|
@ -795,11 +796,13 @@ export class CultCache {
|
|||
|
||||
switch (typeof value) {
|
||||
case "string":
|
||||
return value;
|
||||
case "number":
|
||||
return isBlankIndexText(value) ? undefined : value;
|
||||
case "boolean":
|
||||
return value ? "True" : "False";
|
||||
case "bigint":
|
||||
return String(value);
|
||||
return value.toString();
|
||||
case "number":
|
||||
return canonicalNumberText(value);
|
||||
default:
|
||||
throw new Error(`CultCache ${label} accessor for type "${type}" produced a non-scalar value.`);
|
||||
}
|
||||
|
|
@ -894,6 +897,8 @@ export class CultCache {
|
|||
}
|
||||
}
|
||||
|
||||
const lookups = this.#lookupsFor(registered.nameAccessor, registered.indexAccessors, value);
|
||||
this.#refuseIndexSharing(state, entry.type, registered.catalogEntry.schemaName, entry.key, lookups);
|
||||
return {
|
||||
...entry,
|
||||
payload: this.#cloneBytes(entry.payload),
|
||||
|
|
@ -902,10 +907,22 @@ export class CultCache {
|
|||
schemaId: registered.catalogEntry.schemaId,
|
||||
catalogEntry: registered.catalogEntry,
|
||||
value,
|
||||
lookups: this.#lookupsFor(registered.nameAccessor, registered.indexAccessors, value),
|
||||
lookups,
|
||||
};
|
||||
}
|
||||
|
||||
// The one index rule (variants:ruling:unique-indexes): within one type and one unique alias, no two records hold one
|
||||
// non-blank value. The held key at the landing record's own key is its previous self and keeps its value. The message
|
||||
// names both keys, ordinal, and the alias, never the value.
|
||||
#refuseIndexSharing(state: HydratedState, type: string, schemaName: string, key: string, lookups: EntryLookups): void {
|
||||
for (const alias of [...lookups.indexes.keys()].sort(compareOrdinal)) {
|
||||
const held = state.indexToKeyMaps.get(this.#indexId(type, alias))?.get(lookups.indexes.get(alias)!);
|
||||
if (held !== undefined && held !== key) {
|
||||
throw indexSharingRefusal(held, key, schemaName, alias);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Infallible half of admission: replaces any held record at the same key and installs the admitted one.
|
||||
#installEntry(state: HydratedState, registered: RegisteredDefinition, hydrated: HydratedEntry): void {
|
||||
const entryId = this.#entryId(hydrated.type, hydrated.key);
|
||||
|
|
@ -966,12 +983,14 @@ export class CultCache {
|
|||
// Runs the candidate accessors over every held record of the type into local maps.
|
||||
#deriveTypeLookups(
|
||||
type: string,
|
||||
schemaName: string,
|
||||
nameAccessor: Accessor | undefined,
|
||||
indexAccessors: ReadonlyMap<string, Accessor>,
|
||||
): DerivedTypeLookups {
|
||||
const entryLookups: Array<[HydratedEntry, EntryLookups]> = [];
|
||||
const nameMap = nameAccessor ? new Map<string, string>() : undefined;
|
||||
const indexMaps = new Map<string, Map<string, string>>();
|
||||
const sharing = new Map<string, string[]>();
|
||||
for (const indexName of indexAccessors.keys()) {
|
||||
indexMaps.set(indexName, new Map());
|
||||
}
|
||||
|
|
@ -983,10 +1002,23 @@ export class CultCache {
|
|||
nameMap?.set(lookups.name, entry.key);
|
||||
}
|
||||
for (const [indexName, indexValue] of lookups.indexes) {
|
||||
indexMaps.get(indexName)?.set(indexValue, entry.key);
|
||||
const map = indexMaps.get(indexName)!;
|
||||
const holder = map.get(indexValue);
|
||||
if (holder === undefined) {
|
||||
map.set(indexValue, entry.key);
|
||||
} else {
|
||||
sharing.set(`${indexName}\u0000${indexValue}`, [...(sharing.get(`${indexName}\u0000${indexValue}`) ?? [holder]), entry.key]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Judged whole: any value held by two records refuses, naming the two smallest of its keys. Held records already
|
||||
// passed the judge for every alias installed before, so only the alias being registered can be shared here.
|
||||
for (const [id, keys] of sharing) {
|
||||
keys.sort(compareOrdinal);
|
||||
throw indexSharingRefusal(keys[0], keys[1], schemaName, id.slice(0, id.indexOf("\u0000")));
|
||||
}
|
||||
|
||||
return { entryLookups, nameMap, indexMaps };
|
||||
}
|
||||
|
||||
|
|
@ -1057,3 +1089,42 @@ export class CultCache {
|
|||
return documentPayloadFromBytes(new Uint8Array(documentPayloadBytes(payload)));
|
||||
}
|
||||
}
|
||||
|
||||
function compareOrdinal(a: string, b: string): number {
|
||||
return a < b ? -1 : a > b ? 1 : 0;
|
||||
}
|
||||
|
||||
function indexSharingRefusal(keyA: string, keyB: string, schemaName: string, alias: string): Error {
|
||||
const [first, second] = compareOrdinal(keyA, keyB) <= 0 ? [keyA, keyB] : [keyB, keyA];
|
||||
return new Error(`Records ${first} and ${second} (${schemaName}) both hold unique index '${alias}'; an index value names one record.`);
|
||||
}
|
||||
|
||||
// Blank is no value: empty or only C# White_Space characters. Not JavaScript's \s, which counts U+FEFF and omits U+0085.
|
||||
const BLANK_INDEX_TEXT = /^[\t-\r \u0085\u00A0\u1680\u2000-\u200A\u2028\u2029\u202F\u205F\u3000]*$/;
|
||||
|
||||
function isBlankIndexText(value: string): boolean {
|
||||
return BLANK_INDEX_TEXT.test(value);
|
||||
}
|
||||
|
||||
// The canonical text of a number, as C# RenderCanonicalNumber: integers as their digits (-0 is 0), a non-integral finite
|
||||
// double as its exact decimal expansion, NaN and infinities as no value.
|
||||
function canonicalNumberText(value: number): string | undefined {
|
||||
if (!Number.isFinite(value)) {
|
||||
return undefined;
|
||||
}
|
||||
if (Number.isInteger(value)) {
|
||||
return BigInt(value).toString();
|
||||
}
|
||||
|
||||
// value = mantissa * 2^exponent with exponent < 0, so value = mantissa * 5^k / 10^k for k = -exponent.
|
||||
const view = new DataView(new ArrayBuffer(8));
|
||||
view.setFloat64(0, value);
|
||||
const bits = view.getBigUint64(0);
|
||||
const biasedExponent = Number((bits >> 52n) & 0x7ffn);
|
||||
const fraction = bits & 0xfffffffffffffn;
|
||||
const mantissa = biasedExponent === 0 ? fraction : fraction | (1n << 52n);
|
||||
const k = -((biasedExponent === 0 ? 1 : biasedExponent) - 1075);
|
||||
const digits = (mantissa * 5n ** BigInt(k)).toString().padStart(k + 1, "0");
|
||||
const text = `${digits.slice(0, digits.length - k)}.${digits.slice(digits.length - k)}`.replace(/0+$/, "").replace(/\.$/, "");
|
||||
return value < 0 ? `-${text}` : text;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -32,23 +32,22 @@ export function defineDocumentType<TSchema extends CultCacheSchema>(
|
|||
throw new Error(`CultCache document type "${definition.type}" declares an invalid name accessor.`);
|
||||
}
|
||||
|
||||
if (Array.isArray(definition.indexes)) {
|
||||
const seen = new Set<string>();
|
||||
for (const index of definition.indexes) {
|
||||
if (!index.name || index.name.trim().length === 0) {
|
||||
throw new Error(`CultCache document type "${definition.type}" declares an index with an empty name.`);
|
||||
}
|
||||
if (seen.has(index.name)) {
|
||||
throw new Error(`CultCache document type "${definition.type}" declares duplicate index "${index.name}".`);
|
||||
}
|
||||
seen.add(index.name);
|
||||
}
|
||||
} else if (definition.indexes) {
|
||||
for (const name of Object.keys(definition.indexes)) {
|
||||
const declared = new Set<string>();
|
||||
for (const [kind, set] of [["index", definition.indexes], ["role", definition.roles]] as const) {
|
||||
const names = new Set<string>();
|
||||
for (const name of Array.isArray(set) ? set.map((entry) => entry.name) : set ? Object.keys(set) : []) {
|
||||
if (!name || name.trim().length === 0) {
|
||||
throw new Error(`CultCache document type "${definition.type}" declares an index with an empty name.`);
|
||||
throw new Error(`CultCache document type "${definition.type}" declares ${kind === "index" ? "an index" : "a role"} with an empty name.`);
|
||||
}
|
||||
if (names.has(name)) {
|
||||
throw new Error(`CultCache document type "${definition.type}" declares duplicate ${kind} "${name}".`);
|
||||
}
|
||||
if (kind === "role" && declared.has(name)) {
|
||||
throw new Error(`CultCache document type "${definition.type}" declares "${name}" as both an index and a role; an index already names its role.`);
|
||||
}
|
||||
names.add(name);
|
||||
}
|
||||
for (const name of names) declared.add(name);
|
||||
}
|
||||
|
||||
if (definition.members) {
|
||||
|
|
@ -138,10 +137,13 @@ export function resolveDocumentMembers(definition: AnyCultCacheDocumentDefinitio
|
|||
return definition.members ?? [];
|
||||
}
|
||||
const codec = definition.codec;
|
||||
const indexAlias = (slotName: string) =>
|
||||
!Array.isArray(definition.indexes) && definition.indexes
|
||||
? Object.entries(definition.indexes).find(([, accessor]) => accessor === slotName)?.[0] ?? null
|
||||
: null;
|
||||
const indexAlias = (slotName: string) => {
|
||||
for (const set of [definition.indexes, definition.roles]) {
|
||||
const alias = !Array.isArray(set) && set ? Object.entries(set).find(([, accessor]) => accessor === slotName)?.[0] : undefined;
|
||||
if (alias !== undefined) return alias;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
return codec.members.map((member, index) => ({
|
||||
...member,
|
||||
isName: (definition.name as unknown) === codec.names[index],
|
||||
|
|
|
|||
|
|
@ -59,9 +59,21 @@ export interface CultCacheDocumentDefinition<TSchema extends CultCacheSchema = C
|
|||
members?: readonly CultCacheSchemaCatalogMember[];
|
||||
global?: boolean;
|
||||
name?: CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>;
|
||||
/**
|
||||
* Unique lookup aliases. A non-blank value names one record of the type: a write that would leave two records holding
|
||||
* one value, and a store that already holds two, is refused naming both keys and the alias. Blank values (absent,
|
||||
* empty, or only whitespace) are never indexed. `getByIndex` and `getKeyByIndex` answer from these.
|
||||
*/
|
||||
indexes?:
|
||||
| Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>>
|
||||
| readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[];
|
||||
/**
|
||||
* Selection and reference aliases: the member's catalog `indexAlias`. A role implies nothing about uniqueness, many
|
||||
* records may share a value, and it is never indexed for lookup. An alias is declared in `indexes` or `roles`, not both.
|
||||
*/
|
||||
roles?:
|
||||
| Readonly<Record<string, CultCacheDocumentAccessor<InferCultCacheSchemaValue<TSchema>>>>
|
||||
| readonly CultCacheDocumentIndexDefinition<InferCultCacheSchemaValue<TSchema>>[];
|
||||
formatter?: CultCacheDocumentFormatter<InferCultCacheSchemaValue<TSchema>>;
|
||||
codec?: SlotsCodec<InferCultCacheSchemaValue<TSchema> & object>;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -24,7 +24,6 @@ const itemDocument = defineDocumentType({
|
|||
schemaVersion: "gamecult.first_call_item.v1",
|
||||
schema: z.object({ name: z.string(), category: z.string(), value: z.number().int() }),
|
||||
name: "name",
|
||||
indexes: { category: "category" },
|
||||
});
|
||||
const registry = defineDocumentRegistry(itemDocument);
|
||||
|
||||
|
|
|
|||
504
packages/cultcache-ts/test/unique-index.test.ts
Normal file
504
packages/cultcache-ts/test/unique-index.test.ts
Normal file
|
|
@ -0,0 +1,504 @@
|
|||
import assert from "node:assert/strict";
|
||||
import { existsSync } from "node:fs";
|
||||
import { copyFile, mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { test } from "node:test";
|
||||
|
||||
import { decode } from "@msgpack/msgpack";
|
||||
import { z } from "zod";
|
||||
|
||||
import { codecs } from "../src";
|
||||
import { CultCache } from "../src/cult-cache";
|
||||
import { defineDocumentType, resolveDocumentMembers } from "../src/document";
|
||||
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
|
||||
import type { AnyCultCacheDocumentDefinition } from "../src/types";
|
||||
|
||||
// variants:ruling:unique-indexes. `indexes` are unique within one type, `roles` are selection aliases. The vectors in
|
||||
// tests/vectors/unique-index are written by the C# reference; the rest of the behaviour is pinned here.
|
||||
|
||||
function findVectors(from: string): string {
|
||||
for (let dir = from; ; dir = resolve(dir, "..")) {
|
||||
if (existsSync(join(dir, "tests", "vectors", "unique-index"))) return join(dir, "tests", "vectors", "unique-index");
|
||||
if (resolve(dir, "..") === dir) throw new Error("tests/vectors/unique-index not found above the test.");
|
||||
}
|
||||
}
|
||||
const vectors = findVectors(__dirname);
|
||||
|
||||
const CANARY = "CANARY-uidx-5e1b";
|
||||
|
||||
const skuSchema = z.object({ sku: z.any().optional(), group: z.string().optional() });
|
||||
|
||||
function item(type: string, where: "indexes" | "roles" | "none" = "indexes", schemaName = `tests.${type}`) {
|
||||
return defineDocumentType({
|
||||
type,
|
||||
schemaId: `${schemaName}.v1`,
|
||||
schemaName,
|
||||
schemaVersion: `${schemaName}.v1`,
|
||||
schema: skuSchema,
|
||||
...(where === "none" ? {} : { [where]: { sku: "sku" } }),
|
||||
});
|
||||
}
|
||||
|
||||
// A definition whose payload is never read back, so a value of any scalar kind (a bigint included) can be held and indexed.
|
||||
function scalar(type: string) {
|
||||
const held: unknown[] = [];
|
||||
return defineDocumentType({
|
||||
type,
|
||||
schemaId: `tests.${type}.v1`,
|
||||
schemaName: `tests.${type}`,
|
||||
schemaVersion: `tests.${type}.v1`,
|
||||
schema: skuSchema,
|
||||
formatter: {
|
||||
encode: (value: unknown) => new Uint8Array([held.push(value) - 1]),
|
||||
decode: (payload: Uint8Array) => held[payload[0]] as never,
|
||||
},
|
||||
indexes: { sku: "sku" },
|
||||
});
|
||||
}
|
||||
|
||||
function memory(...definitions: AnyCultCacheDocumentDefinition[]) {
|
||||
let builder = CultCache.builder();
|
||||
for (const definition of definitions) builder = builder.withDocumentType(definition);
|
||||
return builder.build();
|
||||
}
|
||||
|
||||
function refusal(keyA: string, keyB: string, schemaName: string, alias: string) {
|
||||
return `Records ${keyA} and ${keyB} (${schemaName}) both hold unique index '${alias}'; an index value names one record.`;
|
||||
}
|
||||
|
||||
test("a put of a held value at another key is refused naming both keys and the alias, and nothing moves", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-"));
|
||||
try {
|
||||
const file = join(dir, "items.cc");
|
||||
const document = item("item");
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
await cache.pullAllBackingStores();
|
||||
await cache.put(document, "zeta", { sku: CANARY });
|
||||
const before = await readFile(file);
|
||||
const snapshot = cache.snapshot();
|
||||
|
||||
await assert.rejects(cache.put(document, "alpha", { sku: CANARY }), (error: Error) => {
|
||||
assert.equal(error.message, refusal("alpha", "zeta", "tests.item", "sku"));
|
||||
assert.ok(!error.message.includes(CANARY), "the value is never echoed");
|
||||
return true;
|
||||
});
|
||||
assert.deepEqual(await readFile(file), before, "the store is byte-identical");
|
||||
assert.deepEqual(cache.snapshot(), snapshot);
|
||||
assert.equal(cache.get(document, "alpha"), undefined);
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", CANARY), "zeta");
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("rewriting a record's own key with its own value lands", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "a", { sku: "S", group: "one" });
|
||||
await cache.put(document, "a", { sku: "S", group: "two" });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
|
||||
assert.equal(cache.get(document, "a")?.group, "two");
|
||||
});
|
||||
|
||||
test("a removed or changed holder releases its value", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "b", { sku: "S" });
|
||||
await assert.rejects(cache.put(document, "a", { sku: "S" }), /Records a and b/u);
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "b", "the refused write did not take the value");
|
||||
|
||||
await cache.delete(document, "b");
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined);
|
||||
await cache.put(document, "a", { sku: "S" });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
|
||||
|
||||
await cache.put(document, "a", { sku: "T" });
|
||||
await cache.put(document, "b", { sku: "S" });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "b");
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "T"), "a");
|
||||
});
|
||||
|
||||
test("the keys in a refusal are in ordinal order whichever one lands second", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "m", { sku: "S" });
|
||||
await assert.rejects(cache.put(document, "z", { sku: "S" }), { message: refusal("m", "z", "tests.item", "sku") });
|
||||
await assert.rejects(cache.put(document, "B", { sku: "S" }), { message: refusal("B", "m", "tests.item", "sku") });
|
||||
});
|
||||
|
||||
// The C# White_Space set, and nothing else, is blank. Every UTF-16 code unit is judged, so a rule written for a few
|
||||
// characters (or for JavaScript's \s, which counts U+FEFF and omits U+0085) cannot pass.
|
||||
const CSHARP_WHITE_SPACE = (code: number) =>
|
||||
(code >= 0x09 && code <= 0x0d) || code === 0x20 || code === 0x85 || code === 0xa0 || code === 0x1680 ||
|
||||
(code >= 0x2000 && code <= 0x200a) || code === 0x2028 || code === 0x2029 || code === 0x202f || code === 0x205f || code === 0x3000;
|
||||
|
||||
test("blank is exactly the C# White_Space set: every code unit, alone, repeated and padded", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
const disagreements: string[] = [];
|
||||
for (let code = 0; code <= 0xffff; code++) {
|
||||
if (code >= 0xd800 && code <= 0xdfff) continue; // lone surrogates do not survive MessagePack text
|
||||
const character = String.fromCharCode(code);
|
||||
await cache.put(document, "x", { sku: character });
|
||||
let blank = true;
|
||||
try {
|
||||
await cache.put(document, "y", { sku: character });
|
||||
} catch {
|
||||
blank = false;
|
||||
}
|
||||
if (blank !== CSHARP_WHITE_SPACE(code)) disagreements.push(code.toString(16));
|
||||
await cache.delete(document, "x");
|
||||
await cache.delete(document, "y");
|
||||
}
|
||||
assert.deepEqual(disagreements, []);
|
||||
|
||||
// Not trimmed: a padded value is the value, distinct from its trimmed form.
|
||||
await cache.put(document, "x", { sku: " a " });
|
||||
await cache.put(document, "y", { sku: "a" });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", " a "), "x");
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "a"), "y");
|
||||
await assert.rejects(cache.put(document, "z", { sku: " a " }), /Records x and z/u);
|
||||
});
|
||||
|
||||
test("blank values are never indexed and never conflict: empty, spaces, absent", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
for (const [key, sku] of [["a", ""], ["b", " "], ["c", "\u0085"], ["d", undefined], ["e", " "]] as const) {
|
||||
await cache.put(document, key, { sku });
|
||||
}
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", ""), undefined);
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", " "), undefined);
|
||||
// U+FEFF and U+001F are values.
|
||||
await cache.put(document, "f", { sku: "" });
|
||||
await assert.rejects(cache.put(document, "g", { sku: "" }), /Records f and g/u);
|
||||
await cache.put(document, "h", { sku: "\u001F" });
|
||||
await assert.rejects(cache.put(document, "i", { sku: "\u001F" }), /Records h and i/u);
|
||||
});
|
||||
|
||||
test("two types may share an alias and a value", async () => {
|
||||
const first = item("first");
|
||||
const second = item("second");
|
||||
const cache = memory(first, second);
|
||||
await cache.put(first, "a", { sku: "S" });
|
||||
await cache.put(second, "b", { sku: "S" });
|
||||
assert.equal(cache.getKeyByIndex(first, "sku", "S"), "a");
|
||||
assert.equal(cache.getKeyByIndex(second, "sku", "S"), "b");
|
||||
});
|
||||
|
||||
test("roles may share values and are not looked up", async () => {
|
||||
const document = item("item", "roles");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "a", { sku: "S" });
|
||||
await cache.put(document, "b", { sku: "S" });
|
||||
await cache.put(document, "c", { sku: "S" });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined);
|
||||
assert.equal(cache.getByIndex(document, "sku", "S"), undefined);
|
||||
});
|
||||
|
||||
test("an alias declared in indexes and roles is refused at definition; each map is checked like indexes", () => {
|
||||
const base = { type: "bad", schema: skuSchema, schemaId: "tests.bad.v1", schemaName: "tests.bad", schemaVersion: "tests.bad.v1" };
|
||||
assert.throws(
|
||||
() => defineDocumentType({ ...base, indexes: { sku: "sku" }, roles: { sku: "sku" } }),
|
||||
{ message: 'CultCache document type "bad" declares "sku" as both an index and a role; an index already names its role.' },
|
||||
);
|
||||
assert.throws(() => defineDocumentType({ ...base, roles: { " ": "sku" } }), /declares a role with an empty name/u);
|
||||
assert.throws(
|
||||
() => defineDocumentType({ ...base, roles: [{ name: "r", accessor: "sku" }, { name: "r", accessor: "group" }] }),
|
||||
/declares duplicate role "r"/u,
|
||||
);
|
||||
assert.doesNotThrow(() => defineDocumentType({ ...base, indexes: { sku: "sku" }, roles: { group: "group" } }));
|
||||
});
|
||||
|
||||
test("a codec definition's catalog entry is identical with an alias under indexes or roles", async () => {
|
||||
const codec = codecs.slots<{ sku: string; group: string }>({ 0: ["sku", codecs.string], 1: ["group", codecs.string] }, "T.Item");
|
||||
const make = (where: "indexes" | "roles") =>
|
||||
defineDocumentType({
|
||||
type: "codec_item",
|
||||
schemaId: "tests.codec_item.v1",
|
||||
schemaName: "tests.codec_item",
|
||||
schemaVersion: "tests.codec_item.v1",
|
||||
schema: z.object({ sku: z.string(), group: z.string() }),
|
||||
codec,
|
||||
[where]: { sku: "sku" },
|
||||
});
|
||||
const catalog = async (definition: AnyCultCacheDocumentDefinition) => {
|
||||
const cache = memory(definition);
|
||||
await cache.put(definition, "a", { sku: "S", group: "g" } as never);
|
||||
return cache.snapshot()[0].catalogEntry!;
|
||||
};
|
||||
const members = (definition: AnyCultCacheDocumentDefinition) => resolveDocumentMembers(definition);
|
||||
assert.deepEqual(members(make("indexes")), members(make("roles")));
|
||||
assert.equal(members(make("roles")).find((member) => member.memberName === "Sku")?.indexAlias, "sku");
|
||||
const [unique, role] = [await catalog(make("indexes")), await catalog(make("roles"))];
|
||||
assert.equal(unique.canonicalSchemaJson, role.canonicalSchemaJson);
|
||||
assert.equal(unique.contentHash, role.contentHash);
|
||||
assert.equal(unique.schemaId, role.schemaId);
|
||||
});
|
||||
|
||||
test("vectors values.msgpack: every row's canonical text, blank rows no value", async () => {
|
||||
const rows = decode(await readFile(join(vectors, "values.msgpack")), { useBigInt64: true }) as [string, string, unknown, string | null][];
|
||||
assert.ok(rows.length >= 20);
|
||||
for (const [label, kind, value, text] of rows) {
|
||||
const document = scalar(`value_${label.replaceAll(/[^a-z0-9]/gu, "_")}`);
|
||||
const cache = memory(document);
|
||||
void kind;
|
||||
const normalized = value;
|
||||
await cache.put(document, "one", { sku: normalized });
|
||||
if (text === null) {
|
||||
await cache.put(document, "two", { sku: normalized });
|
||||
assert.equal(cache.get(document, "two") !== undefined, true, `${label}: a blank value commits twice`);
|
||||
for (const probe of ["", " ", "NaN", "0", "Infinity", "-Infinity", "null", "undefined"]) {
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", probe), undefined, `${label}: found by '${probe}'`);
|
||||
}
|
||||
} else {
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", text), "one", `${label}: found by its text`);
|
||||
await assert.rejects(cache.put(document, "two", { sku: normalized }), /Records one and two/u, label);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// An independent oracle for the double rule: the text must denote exactly the double, so N / 10^d == mantissa * 2^exponent
|
||||
// in exact integer arithmetic, with no trailing zero and no bare point. A rule written to a few spellings cannot pass.
|
||||
function exactlyDenotes(text: string, value: number): boolean {
|
||||
const match = /^(-?)(\d+)(?:\.(\d*[1-9]))?$/u.exec(text);
|
||||
if (!match) return false;
|
||||
const decimals = match[3]?.length ?? 0;
|
||||
const numerator = BigInt(match[2] + (match[3] ?? ""));
|
||||
const view = new DataView(new ArrayBuffer(8));
|
||||
view.setFloat64(0, Math.abs(value));
|
||||
const bits = view.getBigUint64(0);
|
||||
const biased = Number((bits >> 52n) & 0x7ffn);
|
||||
const mantissa = biased === 0 ? bits & 0xfffffffffffffn : (bits & 0xfffffffffffffn) | (1n << 52n);
|
||||
const exponent = (biased === 0 ? 1 : biased) - 1075;
|
||||
const lhs = numerator * (exponent >= 0 ? 1n : 2n ** BigInt(-exponent));
|
||||
const rhs = mantissa * (exponent >= 0 ? 2n ** BigInt(exponent) : 1n) * 10n ** BigInt(decimals);
|
||||
return (match[1] === "-") === (value < 0) && lhs === rhs;
|
||||
}
|
||||
|
||||
test("a number's index text is its exact value: integers as digits, doubles as the exact expansion, NaN and infinities none", async () => {
|
||||
const document = scalar("number");
|
||||
const cache = memory(document);
|
||||
const probes = [0.1, -0.1, 2.5e-7, 1.5, -123.456, 5e-324, 2.2250738585072014e-308, 1e-10, 123456789.123456789, Number.MIN_VALUE * 3,
|
||||
...Array.from({ length: 200 }, (_, i) => Math.sin(i + 1) * 10 ** ((i % 40) - 20)), 1e21 + 2 ** 20 + 0.5 * 0];
|
||||
// Round trip through lookups: the exact expansion of each probe is found, and it denotes the probe exactly.
|
||||
for (const value of probes) {
|
||||
if (Number.isInteger(value)) continue;
|
||||
await cache.put(document, "k", { sku: value });
|
||||
const candidates = candidateTexts(value);
|
||||
const found = candidates.filter((candidate) => cache.getKeyByIndex(document, "sku", candidate) === "k");
|
||||
assert.equal(found.length, 1, `${value}: exactly one spelling is its index text`);
|
||||
assert.ok(exactlyDenotes(found[0], value), `${value}: ${found[0]} denotes it exactly`);
|
||||
}
|
||||
|
||||
await cache.put(document, "k", { sku: 1e21 });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "1000000000000000000000"), "k");
|
||||
await cache.put(document, "k", { sku: -0 });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "0"), "k");
|
||||
await cache.put(document, "k", { sku: 12n });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "12"), "k");
|
||||
await cache.put(document, "k", { sku: true });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "True"), "k");
|
||||
await cache.put(document, "k", { sku: false });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "False"), "k");
|
||||
for (const value of [Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY]) {
|
||||
await cache.put(document, "p", { sku: value });
|
||||
await cache.put(document, "q", { sku: value });
|
||||
}
|
||||
});
|
||||
|
||||
// Spellings a lookup could be probed with: the exact decimal expansion computed by an arbitrary-precision route.
|
||||
function candidateTexts(value: number): string[] {
|
||||
const view = new DataView(new ArrayBuffer(8));
|
||||
view.setFloat64(0, Math.abs(value));
|
||||
const bits = view.getBigUint64(0);
|
||||
const biased = Number((bits >> 52n) & 0x7ffn);
|
||||
const mantissa = biased === 0 ? bits & 0xfffffffffffffn : (bits & 0xfffffffffffffn) | (1n << 52n);
|
||||
const k = 1075 - (biased === 0 ? 1 : biased);
|
||||
const scaled = (mantissa * 5n ** BigInt(k)).toString().padStart(k + 1, "0");
|
||||
const point = scaled.length - k;
|
||||
const text = `${scaled.slice(0, point)}.${scaled.slice(point)}`.replace(/0+$/u, "").replace(/\.$/u, "");
|
||||
return [(value < 0 ? "-" : "") + text, String(value), value.toString()].filter((spelling, i, all) => all.indexOf(spelling) === i);
|
||||
}
|
||||
|
||||
function twoDefinitions(schemaId: string, where: "indexes" | "roles") {
|
||||
return defineDocumentType({
|
||||
type: "unique_item",
|
||||
schemaId,
|
||||
schemaName: "tests.unique_item",
|
||||
schemaVersion: "tests.unique_item.v1",
|
||||
schema: z.object({ sku: z.string(), category: z.string() }),
|
||||
codec: codecs.slots<{ sku: string; category: string }>({ 0: ["sku", codecs.string], 1: ["category", codecs.string] }, "UqVecItem"),
|
||||
[where]: { Sku: "sku" },
|
||||
});
|
||||
}
|
||||
|
||||
test("vectors duplicate-store.cc and blank-store.cc hold their verdicts", async () => {
|
||||
const outcomes = decode(await readFile(join(vectors, "outcomes.msgpack"))) as [string, string | null, string, string | null, string | null][];
|
||||
assert.equal(outcomes.length, 3);
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-vec-"));
|
||||
try {
|
||||
for (const [name, alias, outcome, key1, key2] of outcomes) {
|
||||
const file = join(dir, `${outcomes.indexOf(outcomes.find((row) => row[0] === name && row[1] === alias)!)}-${name}`);
|
||||
await copyFile(join(vectors, name), file);
|
||||
const before = await readFile(file);
|
||||
const [envelope] = await new SingleFileMessagePackBackingStore(file).pullAll();
|
||||
const document = twoDefinitions(envelope.schemaId!, alias === null ? "roles" : "indexes");
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
const stateBefore = cache.snapshot();
|
||||
if (outcome === "open") {
|
||||
await cache.pullAllBackingStores();
|
||||
assert.equal(cache.snapshot().length, name === "blank-store.cc" ? 2 : 4);
|
||||
if (alias === null) assert.equal(cache.getKeyByIndex(document, "Sku", "S-1"), undefined);
|
||||
} else {
|
||||
await assert.rejects(cache.pullAllBackingStores(), (error: Error) => {
|
||||
assert.equal(error.message, refusal(key1!, key2!, "tests.unique_item", alias!));
|
||||
assert.ok(!error.message.includes("S-1"));
|
||||
return true;
|
||||
});
|
||||
assert.deepEqual(cache.snapshot(), stateBefore, "a refused load admits nothing");
|
||||
}
|
||||
assert.deepEqual(await readFile(file), before, `${name} is untouched`);
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("a refused load leaves the cache's prior state as it was, and a later good load still works", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-load-"));
|
||||
try {
|
||||
const bad = join(dir, "bad.cc");
|
||||
await copyFile(join(vectors, "duplicate-store.cc"), bad);
|
||||
const [envelope] = await new SingleFileMessagePackBackingStore(bad).pullAll();
|
||||
const document = twoDefinitions(envelope.schemaId!, "indexes");
|
||||
const cache = CultCache.builder().withDocumentType(document).withGenericStore(new SingleFileMessagePackBackingStore(bad)).build();
|
||||
await assert.rejects(cache.pullAllBackingStores(), /Records item:a and item:b/u);
|
||||
assert.equal(cache.get(document, "item:a"), undefined);
|
||||
assert.equal(cache.getKeyByIndex(document, "Sku", "S-1"), undefined);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("registerIndex over held duplicates installs nothing", async () => {
|
||||
const document = item("item", "roles");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "a", { sku: "S", group: "g" });
|
||||
await cache.put(document, "b", { sku: "S", group: "h" });
|
||||
await assert.rejects(cache.registerIndex(document, "sku", "sku"), { message: refusal("a", "b", "tests.item", "sku") });
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), undefined, "no map was installed");
|
||||
await cache.put(document, "c", { sku: "S", group: "i" });
|
||||
await cache.registerIndex(document, "group", "group");
|
||||
assert.equal(cache.getKeyByIndex(document, "group", "h"), "b");
|
||||
await assert.rejects(cache.put(document, "d", { sku: "x", group: "h" }), /Records b and d/u);
|
||||
});
|
||||
|
||||
test("registerIndex over distinct held values installs the lookup", async () => {
|
||||
const document = item("item", "none");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "a", { sku: "S" });
|
||||
await cache.registerIndex(document, "sku", "sku");
|
||||
assert.equal(cache.getKeyByIndex(document, "sku", "S"), "a");
|
||||
});
|
||||
|
||||
// ---- a type with several unique indexes: the second alias is judged, in alias order, in both judges ----
|
||||
|
||||
const pairSchema = z.object({ a: z.string().optional(), b: z.string().optional() });
|
||||
|
||||
function pair(where: { indexes?: Record<string, "a" | "b">; roles?: Record<string, "a" | "b"> }) {
|
||||
return defineDocumentType({
|
||||
type: "pair",
|
||||
schemaId: "tests.pair.v1",
|
||||
schemaName: "tests.pair",
|
||||
schemaVersion: "tests.pair.v1",
|
||||
schema: pairSchema,
|
||||
...where,
|
||||
});
|
||||
}
|
||||
|
||||
// Declared b first, so insertion order is the opposite of alias order.
|
||||
const BOTH_UNIQUE = { indexes: { b: "b", a: "a" } as Record<string, "a" | "b"> };
|
||||
|
||||
test("a collision on the second unique alias is refused, with a value on the first alias that does not collide", async () => {
|
||||
const document = pair(BOTH_UNIQUE);
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "k1", { a: "x1", b: "shared" });
|
||||
await assert.rejects(cache.put(document, "k2", { a: "x2", b: "shared" }), { message: refusal("k1", "k2", "tests.pair", "b") });
|
||||
await assert.rejects(cache.put(document, "k2", { a: "x1", b: "y" }), { message: refusal("k1", "k2", "tests.pair", "a") });
|
||||
assert.equal(cache.get(document, "k2"), undefined);
|
||||
});
|
||||
|
||||
test("when two aliases collide the first alias in ordinal order is named, and its keys are in ordinal order", async () => {
|
||||
const document = pair(BOTH_UNIQUE);
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "m", { a: "same-a", b: "same-b" });
|
||||
await assert.rejects(cache.put(document, "c", { a: "same-a", b: "same-b" }), { message: refusal("c", "m", "tests.pair", "a") });
|
||||
// The colliding holders differ per alias: a is held by m, b by n; alias a is still the one named.
|
||||
await cache.put(document, "n", { a: "other-a", b: "other-b" });
|
||||
await assert.rejects(cache.put(document, "z", { a: "same-a", b: "other-b" }), { message: refusal("m", "z", "tests.pair", "a") });
|
||||
});
|
||||
|
||||
test("a load judges every unique alias: a store collision on the second alias refuses the open", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-pair-"));
|
||||
try {
|
||||
for (const [name, records, expected] of [
|
||||
["second-only", [["k1", { a: "1", b: "s" }], ["k2", { a: "2", b: "s" }]], refusal("k1", "k2", "tests.pair", "b")],
|
||||
["both-aliases", [["m", { a: "s", b: "s" }], ["c", { a: "s", b: "s" }]], refusal("c", "m", "tests.pair", "a")],
|
||||
] as const) {
|
||||
const file = join(dir, `${name}.cc`);
|
||||
const roles = pair({ roles: { b: "b", a: "a" } });
|
||||
const writer = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
for (const [key, value] of records) await writer.put(roles, key, value);
|
||||
const before = await readFile(file);
|
||||
|
||||
const unique = pair(BOTH_UNIQUE);
|
||||
const reader = CultCache.builder().withDocumentType(unique).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
await assert.rejects(reader.pullAllBackingStores(), { message: expected }, name);
|
||||
assert.deepEqual(reader.snapshot(), [], "a refused load admits nothing");
|
||||
assert.deepEqual(await readFile(file), before);
|
||||
|
||||
const rolesReader = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
await rolesReader.pullAllBackingStores();
|
||||
assert.equal(rolesReader.snapshot().length, 2, `${name}: the same store opens under roles`);
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("registerIndex of a second unique alias judges it, naming the two smallest keys whatever order they were written in", async () => {
|
||||
const document = pair({ indexes: { a: "a" }, roles: { b: "b" } });
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "z", { a: "1", b: "s" });
|
||||
await cache.put(document, "m", { a: "2", b: "s" });
|
||||
await cache.put(document, "q", { a: "3", b: "s" });
|
||||
await assert.rejects(cache.registerIndex(document, "b", "b"), { message: refusal("m", "q", "tests.pair", "b") });
|
||||
assert.equal(cache.getKeyByIndex(document, "a", "1"), "z", "the alias already installed is intact");
|
||||
assert.equal(cache.getKeyByIndex(document, "b", "s"), undefined);
|
||||
});
|
||||
|
||||
test("verdict parity: the same stores are refused whatever the load order, and the first collision refuses (text is informational)", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-verdict-"));
|
||||
try {
|
||||
// Three records, one pair colliding; whichever position the pair has in the file, the store is refused and a clean store opens.
|
||||
const roles = pair({ roles: { a: "a" } });
|
||||
for (const [name, values, refused] of [
|
||||
["pair-first", ["s", "s", "t"], true],
|
||||
["pair-last", ["t", "s", "s"], true],
|
||||
["pair-split", ["s", "t", "s"], true],
|
||||
["distinct", ["r", "s", "t"], false],
|
||||
] as const) {
|
||||
const file = join(dir, `${name}.cc`);
|
||||
const writer = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
for (const [i, a] of values.entries()) await writer.put(roles, `r${i}`, { a });
|
||||
const unique = pair({ indexes: { a: "a" } });
|
||||
const reader = CultCache.builder().withDocumentType(unique).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
if (refused) await assert.rejects(reader.pullAllBackingStores(), /both hold unique index 'a'/u, name);
|
||||
else await reader.pullAllBackingStores();
|
||||
assert.equal(reader.snapshot().length, refused ? 0 : 3, name);
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
@ -66,6 +66,7 @@ Current message families:
|
|||
- `cultnet.snapshot_response.v0`
|
||||
- `cultnet.snapshot_response_raw.v0`
|
||||
- `cultnet.snapshot_request.v1`, `cultnet.database_subscribe.v1`, `cultnet.snapshot_response_raw.v1` (typed-selection carriers; no selection engine yet)
|
||||
- `cultnet.database_change_raw.v0` (optional `commitId` and `commitCount >= 2` mark the changes of one commit; carried and validated here, no TS code groups or applies by them)
|
||||
- `cultnet.sample.change_name.v0`
|
||||
- `cultnet.sample.chat.v0`
|
||||
|
||||
|
|
|
|||
|
|
@ -424,6 +424,10 @@ export interface CultNetDatabaseChangeRawMessage {
|
|||
document?: CultNetRawDocumentRecord;
|
||||
recordKey?: string;
|
||||
schemaId?: string;
|
||||
/** Opaque, one subscription: changes sharing a commitId are the changes of one commit. */
|
||||
commitId?: string | null;
|
||||
/** How many changes the commit carries, at least two. The C# server sends null on a change outside a grouped commit. */
|
||||
commitCount?: number | null;
|
||||
}
|
||||
|
||||
export interface CultNetSchemaDescriptor {
|
||||
|
|
@ -1397,6 +1401,15 @@ function validateDatabaseChangeRawMessage(
|
|||
);
|
||||
}
|
||||
|
||||
if (candidate.commitId !== undefined && candidate.commitId !== null &&
|
||||
(typeof candidate.commitId !== "string" || candidate.commitId.length === 0)) {
|
||||
throw new Error("Validation failed for cultnet.database_change_raw.v0: /commitId: must be a non-empty string");
|
||||
}
|
||||
if (candidate.commitCount !== undefined && candidate.commitCount !== null &&
|
||||
(typeof candidate.commitCount !== "number" || !Number.isInteger(candidate.commitCount) || candidate.commitCount < 2)) {
|
||||
throw new Error("Validation failed for cultnet.database_change_raw.v0: /commitCount: must be an integer of at least 2");
|
||||
}
|
||||
|
||||
if (candidate.changeKind === "removed") {
|
||||
requireNonEmptyString(candidate.recordKey, "/recordKey");
|
||||
requireNonEmptyString(candidate.schemaId, "/schemaId");
|
||||
|
|
|
|||
|
|
@ -54,6 +54,7 @@ import {
|
|||
validateGhostlightAgentStateGenerated,
|
||||
validateGhostlightAgentState,
|
||||
shardServes,
|
||||
type CultNetDatabaseChangeRawMessage,
|
||||
type CultNetLoginMessage,
|
||||
type GhostlightAgentStateShape,
|
||||
type GhostlightAgentStateDocument,
|
||||
|
|
@ -1688,6 +1689,55 @@ test("database subscription contracts match the C# live document lane", () => {
|
|||
assert.equal(removed.schemaVersion, "cultnet.database_change_raw.v0");
|
||||
assert.equal(removed.recordKey, "counter:main");
|
||||
|
||||
const grouped = parseCultNetMessage({
|
||||
schemaVersion: "cultnet.database_change_raw.v0",
|
||||
messageId: "change-counter-3",
|
||||
subscriptionId: "counter:browser-1",
|
||||
changeKind: "removed",
|
||||
recordKey: "counter:main",
|
||||
schemaId: "sample.counter_state.v1",
|
||||
commitId: "c1",
|
||||
commitCount: 2,
|
||||
});
|
||||
assert.equal((grouped as CultNetDatabaseChangeRawMessage).commitId, "c1");
|
||||
assert.equal((grouped as CultNetDatabaseChangeRawMessage).commitCount, 2);
|
||||
assert.deepEqual(encodeCultNetMessageForWire(grouped), grouped, "both commit fields travel");
|
||||
assert.equal((removed as CultNetDatabaseChangeRawMessage).commitId, undefined, "a change without them parses as before");
|
||||
const changeWith = (extra: Record<string, unknown>) => () => parseCultNetMessage({
|
||||
schemaVersion: "cultnet.database_change_raw.v0",
|
||||
messageId: "change-counter-4",
|
||||
subscriptionId: "counter:browser-1",
|
||||
changeKind: "removed",
|
||||
recordKey: "counter:main",
|
||||
schemaId: "sample.counter_state.v1",
|
||||
...extra,
|
||||
});
|
||||
// The C# server writes commitId and commitCount as nil on every change outside a grouped commit, so null is absent.
|
||||
const ungrouped = changeWith({ commitId: null, commitCount: null })() as CultNetDatabaseChangeRawMessage;
|
||||
assert.equal(ungrouped.commitId, null);
|
||||
assert.equal(ungrouped.commitCount, null);
|
||||
assert.doesNotThrow(changeWith({ commitId: null }));
|
||||
assert.doesNotThrow(changeWith({ commitCount: null }));
|
||||
assert.doesNotThrow(() => parseCultNetMessage({
|
||||
schemaVersion: "cultnet.database_change_raw.v0",
|
||||
messageId: "change-counter-5",
|
||||
subscriptionId: "counter:browser-1",
|
||||
changeKind: "added",
|
||||
document: {
|
||||
schemaId: "sample.counter_state.v1",
|
||||
recordKey: "counter:main",
|
||||
storedAt: "2026-08-17T00:00:00Z",
|
||||
payloadEncoding: "messagepack",
|
||||
payload: new Uint8Array([0x81, 0xa5, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x01]),
|
||||
},
|
||||
recordKey: null,
|
||||
schemaId: null,
|
||||
commitId: null,
|
||||
commitCount: null,
|
||||
}));
|
||||
for (const commitId of ["", 5]) assert.throws(changeWith({ commitId }), /\/commitId/, `commitId ${JSON.stringify(commitId)}`);
|
||||
for (const commitCount of [1, 0, 2.5, "two", -3]) assert.throws(changeWith({ commitCount }), /\/commitCount/, `commitCount ${commitCount}`);
|
||||
|
||||
const unsubscribe = parseCultNetMessage({
|
||||
schemaVersion: "cultnet.database_unsubscribe.v0",
|
||||
messageId: "unsubscribe-counter",
|
||||
|
|
|
|||
294
tests/GameCult.Caching.Tests/UniqueIndexVectorTests.cs
Normal file
294
tests/GameCult.Caching.Tests/UniqueIndexVectorTests.cs
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
#nullable enable
|
||||
using System;
|
||||
using System.Buffers;
|
||||
using System.Collections.Generic;
|
||||
using System.Globalization;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Reflection;
|
||||
using GameCult.Caching;
|
||||
using GameCult.Caching.MessagePack;
|
||||
using MessagePack;
|
||||
using NUnit.Framework;
|
||||
|
||||
namespace GameCult.Caching.Tests
|
||||
{
|
||||
// variants:ruling:unique-indexes. The canonical index text and the unique-index load verdicts in tests/vectors/unique-index.
|
||||
// The C# reference is the only writer (CULTCACHE_WRITE_VECTORS=1); every other run reads the committed bytes and
|
||||
// compares them byte for byte. Other runtimes are held to the same text and the same verdicts.
|
||||
public class UniqueIndexVectorTests
|
||||
{
|
||||
private const string WriteVariable = "CULTCACHE_WRITE_VECTORS";
|
||||
private static readonly DateTimeOffset StoredAt = new(2026, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
private static bool Writing => Environment.GetEnvironmentVariable(WriteVariable) == "1";
|
||||
|
||||
private static string VectorDirectory()
|
||||
{
|
||||
for (var dir = new DirectoryInfo(TestContext.CurrentContext.TestDirectory); dir != null; dir = dir.Parent)
|
||||
{
|
||||
var vectors = Path.Combine(dir.FullName, "tests", "vectors");
|
||||
if (Directory.Exists(vectors)) return Path.Combine(vectors, "unique-index");
|
||||
}
|
||||
|
||||
throw new DirectoryNotFoundException($"tests/vectors not found above {TestContext.CurrentContext.TestDirectory}.");
|
||||
}
|
||||
|
||||
private static string VectorPath(string name) => Path.Combine(VectorDirectory(), name);
|
||||
|
||||
private static void Check(string name, byte[] reference)
|
||||
{
|
||||
var path = VectorPath(name);
|
||||
if (Writing)
|
||||
{
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
|
||||
File.WriteAllBytes(path, reference);
|
||||
return;
|
||||
}
|
||||
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
Assert.That(Convert.ToHexString(reference), Is.EqualTo(Convert.ToHexString(File.ReadAllBytes(path))), $"vector {name}");
|
||||
}
|
||||
|
||||
private static byte[] Committed(string name)
|
||||
{
|
||||
var path = VectorPath(name);
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
return File.ReadAllBytes(path);
|
||||
}
|
||||
|
||||
// ---- values.msgpack: [label, kind, value, text-or-nil] ----
|
||||
|
||||
private sealed record Row(string Label, string Kind, object Value, string? Text);
|
||||
|
||||
private static readonly MethodInfo RenderCanonicalNumber =
|
||||
typeof(CultDocumentRegistry).GetMethod("RenderCanonicalNumber", BindingFlags.NonPublic | BindingFlags.Static)!;
|
||||
|
||||
private static object? ByIndex(CultCache cache, Type type, string alias, string value) =>
|
||||
typeof(CultCache).GetMethod(nameof(CultCache.GetByIndex))!.MakeGenericMethod(type).Invoke(cache, new object[] { alias, value });
|
||||
|
||||
private static string? Number(object raw) => (string?)RenderCanonicalNumber.Invoke(null, new[] { raw });
|
||||
|
||||
// Strings and booleans carry the text the rule states; numbers carry whatever the reference renders, never hand-typed.
|
||||
private static List<Row> Rows() => new()
|
||||
{
|
||||
new("plain", "string", "S-1", "S-1"),
|
||||
new("empty", "string", "", null),
|
||||
new("space", "string", " ", null),
|
||||
new("nel", "string", "\u0085", null),
|
||||
new("nbsp", "string", "\u00A0", null),
|
||||
new("ideographic-space", "string", "\u3000", null),
|
||||
new("vertical-tab", "string", "\u000B", null),
|
||||
new("bom", "string", "\uFEFF", "\uFEFF"),
|
||||
new("unit-separator", "string", "\u001F", "\u001F"),
|
||||
new("padded", "string", " a ", " a "),
|
||||
new("int64-max", "int64", long.MaxValue, Number(long.MaxValue)),
|
||||
new("int64-min", "int64", long.MinValue, Number(long.MinValue)),
|
||||
new("uint64-max", "uint64", ulong.MaxValue, Number(ulong.MaxValue)),
|
||||
new("double-zero", "double", 0.0, Number(0.0)),
|
||||
new("double-negative-zero", "double", -0.0, Number(-0.0)),
|
||||
new("double-tenth", "double", 0.1, Number(0.1)),
|
||||
new("double-1e21", "double", 1e21, Number(1e21)),
|
||||
new("double-2.5e-7", "double", 2.5e-7, Number(2.5e-7)),
|
||||
new("double-nan", "double", double.NaN, Number(double.NaN)),
|
||||
new("double-infinity", "double", double.PositiveInfinity, Number(double.PositiveInfinity)),
|
||||
new("bool-true", "bool", true, "True"),
|
||||
new("bool-false", "bool", false, "False"),
|
||||
};
|
||||
|
||||
private static byte[] ValuesFile()
|
||||
{
|
||||
var buffer = new ArrayBufferWriter<byte>();
|
||||
var writer = new MessagePackWriter(buffer);
|
||||
var rows = Rows();
|
||||
writer.WriteArrayHeader(rows.Count);
|
||||
foreach (var row in rows)
|
||||
{
|
||||
writer.WriteArrayHeader(4);
|
||||
writer.Write(row.Label);
|
||||
writer.Write(row.Kind);
|
||||
switch (row.Value)
|
||||
{
|
||||
case string s: writer.Write(s); break;
|
||||
case long l: writer.Write(l); break;
|
||||
case ulong u: writer.Write(u); break;
|
||||
case double d: writer.Write(d); break;
|
||||
case bool b: writer.Write(b); break;
|
||||
default: throw new InvalidOperationException(row.Kind);
|
||||
}
|
||||
|
||||
if (row.Text == null) writer.WriteNil(); else writer.Write(row.Text);
|
||||
}
|
||||
|
||||
writer.Flush();
|
||||
return buffer.WrittenSpan.ToArray();
|
||||
}
|
||||
|
||||
private static readonly Dictionary<string, Type> KindTypes = new[]
|
||||
{
|
||||
("string", typeof(string)), ("int64", typeof(long)), ("uint64", typeof(ulong)), ("double", typeof(double)), ("bool", typeof(bool))
|
||||
}.ToDictionary(pair => pair.Item1, pair => EmittedDocumentTypes.Emit(
|
||||
"UqKind" + pair.Item1, "tests.uq_kind_" + pair.Item1, "tests.uq_kind_" + pair.Item1 + ".v1",
|
||||
new[] { new EmittedDocumentTypes.Field("Value", pair.Item2, 0, Index: "v") }));
|
||||
|
||||
[Test]
|
||||
public void ValuesFileIsTheReferencesCanonicalText()
|
||||
{
|
||||
Check("values.msgpack", ValuesFile());
|
||||
|
||||
var reader = new MessagePackReader(Committed("values.msgpack"));
|
||||
var count = reader.ReadArrayHeader();
|
||||
var rows = Rows();
|
||||
Assert.That(count, Is.EqualTo(rows.Count));
|
||||
foreach (var row in rows)
|
||||
{
|
||||
Assert.That(reader.ReadArrayHeader(), Is.EqualTo(4));
|
||||
Assert.That(reader.ReadString(), Is.EqualTo(row.Label));
|
||||
Assert.That(reader.ReadString(), Is.EqualTo(row.Kind));
|
||||
reader.Skip();
|
||||
var text = reader.TryReadNil() ? null : reader.ReadString();
|
||||
Assert.That(text, Is.EqualTo(row.Text), row.Label);
|
||||
|
||||
var type = KindTypes[row.Kind];
|
||||
var registry = CultDocumentRegistry.ForTypes(new[] { type });
|
||||
using var cache = CultCacheMessagePack.Create(Path.Combine(Path.GetTempPath(), $"cultlib-uidxv-{Guid.NewGuid():N}.cc"), new CultCacheOpenOptions { Registry = registry });
|
||||
cache.Commit(batch =>
|
||||
{
|
||||
batch.Upsert(type, EmittedDocumentTypes.New(type, ("Value", row.Value)), new CultRecordKey("one"));
|
||||
batch.Upsert(type, EmittedDocumentTypes.New(type, ("Value", row.Value)), new CultRecordKey(text == null ? "two" : "one"));
|
||||
});
|
||||
if (text == null)
|
||||
{
|
||||
Assert.That(cache.Get(new CultRecordKey("two")), Is.Not.Null, $"{row.Label}: a blank value commits twice");
|
||||
foreach (var probe in new[] { "", " ", "NaN", "0", "Infinity", "∞" })
|
||||
Assert.That(ByIndex(cache, type, "v", probe), Is.Null, $"{row.Label}: found by '{probe}'");
|
||||
}
|
||||
else
|
||||
{
|
||||
Assert.That(ByIndex(cache, type, "v", text), Is.Not.Null, $"{row.Label}: found by its text");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- the stores: written through a role, judged by an index ----
|
||||
|
||||
private static readonly EmittedDocumentTypes.Field[] RoleFields =
|
||||
{
|
||||
new("Sku", typeof(string), 0, Role: "Sku"), new("Category", typeof(string), 1)
|
||||
};
|
||||
|
||||
private static readonly EmittedDocumentTypes.Field[] IndexFields =
|
||||
{
|
||||
new("Sku", typeof(string), 0, Index: "Sku"), new("Category", typeof(string), 1)
|
||||
};
|
||||
|
||||
private static readonly Type ItemRole = EmittedDocumentTypes.Emit("UqVecItemRole", "tests.unique_item", "tests.unique_item.v1", RoleFields);
|
||||
private static readonly Type ItemIndex = EmittedDocumentTypes.Emit("UqVecItemIndex", "tests.unique_item", "tests.unique_item.v1", IndexFields);
|
||||
|
||||
private static readonly (string Key, string Sku, string Category)[] Items =
|
||||
{
|
||||
("item:a", "S-1", "tools"), ("item:b", "S-1", "tools"), ("item:c", " ", "parts"), ("item:d", "\u0085", "parts")
|
||||
};
|
||||
|
||||
private static byte[] StoreBytes(string[] keys)
|
||||
{
|
||||
var directory = Path.Combine(Path.GetTempPath(), $"cultlib-uidxs-{Guid.NewGuid():N}");
|
||||
Directory.CreateDirectory(directory);
|
||||
try
|
||||
{
|
||||
var path = Path.Combine(directory, "store.cc");
|
||||
using (var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions
|
||||
{
|
||||
Registry = CultDocumentRegistry.ForTypes(new[] { ItemRole }), StoreFlushOnDispose = true
|
||||
}))
|
||||
{
|
||||
cache.Clock = () => StoredAt;
|
||||
foreach (var item in Items.Where(item => keys.Contains(item.Key)))
|
||||
cache.UpsertAsync(ItemRole, EmittedDocumentTypes.New(ItemRole, ("Sku", item.Sku), ("Category", item.Category)), new CultRecordKey(item.Key)).GetAwaiter().GetResult();
|
||||
cache.FlushAsync().GetAwaiter().GetResult();
|
||||
}
|
||||
|
||||
return File.ReadAllBytes(path);
|
||||
}
|
||||
finally
|
||||
{
|
||||
Directory.Delete(directory, recursive: true);
|
||||
}
|
||||
}
|
||||
|
||||
private static readonly (string File, string? Alias, string Outcome, string? K1, string? K2)[] Outcomes =
|
||||
{
|
||||
("duplicate-store.cc", "Sku", "refused", "item:a", "item:b"),
|
||||
("duplicate-store.cc", null, "open", null, null),
|
||||
("blank-store.cc", "Sku", "open", null, null),
|
||||
};
|
||||
|
||||
private static byte[] OutcomesFile()
|
||||
{
|
||||
var buffer = new ArrayBufferWriter<byte>();
|
||||
var writer = new MessagePackWriter(buffer);
|
||||
writer.WriteArrayHeader(Outcomes.Length);
|
||||
foreach (var row in Outcomes)
|
||||
{
|
||||
writer.WriteArrayHeader(5);
|
||||
writer.Write(row.File);
|
||||
foreach (var text in new[] { row.Alias, row.Outcome, row.K1, row.K2 })
|
||||
if (text == null) writer.WriteNil(); else writer.Write(text);
|
||||
}
|
||||
|
||||
writer.Flush();
|
||||
return buffer.WrittenSpan.ToArray();
|
||||
}
|
||||
|
||||
private static string? NextText(ref MessagePackReader reader) => reader.TryReadNil() ? null : reader.ReadString();
|
||||
|
||||
[Test]
|
||||
public void StoreVectorsHoldTheirVerdicts()
|
||||
{
|
||||
Check("duplicate-store.cc", StoreBytes(new[] { "item:a", "item:b", "item:c", "item:d" }));
|
||||
Check("blank-store.cc", StoreBytes(new[] { "item:c", "item:d" }));
|
||||
Check("outcomes.msgpack", OutcomesFile());
|
||||
|
||||
var reader = new MessagePackReader(Committed("outcomes.msgpack"));
|
||||
var count = reader.ReadArrayHeader();
|
||||
Assert.That(count, Is.EqualTo(Outcomes.Length));
|
||||
for (var i = 0; i < count; i++)
|
||||
{
|
||||
Assert.That(reader.ReadArrayHeader(), Is.EqualTo(5));
|
||||
var file = reader.ReadString()!;
|
||||
var alias = NextText(ref reader);
|
||||
var outcome = NextText(ref reader)!;
|
||||
var k1 = NextText(ref reader);
|
||||
var k2 = NextText(ref reader);
|
||||
|
||||
var directory = Path.Combine(Path.GetTempPath(), $"cultlib-uidxo-{Guid.NewGuid():N}");
|
||||
Directory.CreateDirectory(directory);
|
||||
try
|
||||
{
|
||||
var path = Path.Combine(directory, "store.cc");
|
||||
File.WriteAllBytes(path, Committed(file));
|
||||
var before = File.ReadAllBytes(path);
|
||||
var type = alias == null ? ItemRole : ItemIndex;
|
||||
var registry = CultDocumentRegistry.ForTypes(new[] { type });
|
||||
if (outcome == "open")
|
||||
{
|
||||
using var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry });
|
||||
Assert.That(cache.Get(new CultRecordKey("item:c")), Is.Not.Null, file);
|
||||
}
|
||||
else
|
||||
{
|
||||
var refused = Assert.Throws<InvalidOperationException>(() => CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = registry }).Dispose())!;
|
||||
Assert.That(refused.Message, Is.EqualTo(
|
||||
$"Records {k1} and {k2} (tests.unique_item) both hold unique index '{alias}'; an index value names one record."));
|
||||
Assert.That(refused.Message, Does.Not.Contain("S-1"));
|
||||
}
|
||||
|
||||
Assert.That(File.ReadAllBytes(path), Is.EqualTo(before), $"{file} is untouched");
|
||||
}
|
||||
finally
|
||||
{
|
||||
Directory.Delete(directory, recursive: true);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
24
tests/vectors/unique-index/README.md
Normal file
24
tests/vectors/unique-index/README.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# unique-index vectors
|
||||
|
||||
Written only by the C# reference, `tests/GameCult.Caching.Tests/UniqueIndexVectorTests.cs`, with
|
||||
`CULTCACHE_WRITE_VECTORS=1`. Every other run reads these files and compares them byte for byte; a changed vector is a
|
||||
reviewed commit. Other runtimes are held to the same canonical text and the same verdicts (`variants:ruling:unique-indexes`).
|
||||
|
||||
## Files
|
||||
|
||||
- `values.msgpack`: an array of `[label, kind, value, text-or-nil]`, kind one of `string`, `int64`, `uint64`, `double`,
|
||||
`bool` (decode integers 64-bit wide). `text` is the canonical index text of a member holding `value`: a record holding
|
||||
it is found by `GetByIndex(alias, text)`. A nil text is a blank value: no value, never indexed, never judged, found by
|
||||
no text, and two records holding it commit. Strings are as is (never trimmed), blank being empty or only C#
|
||||
`White_Space` characters (U+0009-000D, U+0020, U+0085, U+00A0, U+1680, U+2000-200A, U+2028, U+2029, U+202F, U+205F,
|
||||
U+3000); U+FEFF and U+001F are values. Booleans are `True` and `False`. Integers are their decimal digits. A
|
||||
non-integral finite double is its exact decimal expansion (written by C# `RenderExactDouble`, never hand-typed),
|
||||
`-0.0` is `0`, NaN and infinities are nil.
|
||||
- `duplicate-store.cc`: a single-file store, schema `tests.unique_item` v1, slots `[Sku: string, Category: string]`,
|
||||
stored-at pinned to `2026-01-01T00:00:00Z`. Records `item:a` and `item:b` both hold Sku `S-1`; `item:c` holds Sku
|
||||
`" "` and `item:d` holds Sku `U+0085`. Written through a role Sku.
|
||||
- `blank-store.cc`: the same, only `item:c` and `item:d`.
|
||||
- `outcomes.msgpack`: rows `[file, unique alias or nil, open|refused, k1 or nil, k2 or nil]`. Opening `file` with that
|
||||
alias a unique index (nil: a role, no unique alias) yields the verdict. A refusal reads, byte for byte,
|
||||
`Records <k1> and <k2> (tests.unique_item) both hold unique index '<alias>'; an index value names one record.`
|
||||
with `k1 < k2` ordinal, never contains the value, and leaves the file byte-identical.
|
||||
BIN
tests/vectors/unique-index/blank-store.cc
Normal file
BIN
tests/vectors/unique-index/blank-store.cc
Normal file
Binary file not shown.
BIN
tests/vectors/unique-index/duplicate-store.cc
Normal file
BIN
tests/vectors/unique-index/duplicate-store.cc
Normal file
Binary file not shown.
1
tests/vectors/unique-index/outcomes.msgpack
Normal file
1
tests/vectors/unique-index/outcomes.msgpack
Normal file
|
|
@ -0,0 +1 @@
|
|||
“•˛duplicate-store.ccŁSku§refused¦item:a¦item:b•˛duplicate-store.ccŔ¤openŔŔ•®blank-store.ccŁSku¤openŔŔ
|
||||
BIN
tests/vectors/unique-index/values.msgpack
Normal file
BIN
tests/vectors/unique-index/values.msgpack
Normal file
Binary file not shown.
Loading…
Reference in a new issue