Compare commits
54 changed files with 2506 additions and 8383 deletions
43
.github/workflows/cultmesh-portability.yml
vendored
43
.github/workflows/cultmesh-portability.yml
vendored
|
|
@ -1,18 +1,11 @@
|
|||
name: CultMesh portability and performance
|
||||
|
||||
# The forge runs one job at a time and this workflow holds the slot 14 to 30 minutes, so it runs once per change:
|
||||
# on the pull request (never again on the push that merges it; a break that merges from a stale base waits for the
|
||||
# schedule), skipped for design-note-only changes, and on a
|
||||
# daily schedule so main is still exercised when nobody opens a pull request.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- "codex/**"
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
# Not `**/*.md` or `docs/**`: docs/start/README.md is read by the first-call tests of every runtime.
|
||||
- "docs/*.md"
|
||||
- "research/**"
|
||||
- ".voidbot/**"
|
||||
schedule:
|
||||
- cron: "17 3 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
|
|
@ -25,9 +18,7 @@ jobs:
|
|||
reactive-and-resource-lifetime:
|
||||
name: ${{ matrix.os }} reactive semantics and leases
|
||||
runs-on: ${{ matrix.os }}
|
||||
# The job dies at the lower of this value and the forge runner's own cap, which Eureka is raising from 30 to 60 minutes
|
||||
# (run 1378 carried 60 and was still cancelled at 30:02); the two must both say 60 before a full run can finish.
|
||||
timeout-minutes: 60
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
|
|
@ -47,41 +38,17 @@ jobs:
|
|||
with:
|
||||
dotnet-version: 10.0.x
|
||||
|
||||
# The forge runner image (catthehacker/ubuntu:act-latest) carries Node 24 and Python 3.12 on PATH, and these two
|
||||
# setup actions downloaded them again from github.com anyway (about 2.7 and 2.8 minutes, run 639) because the
|
||||
# image's tool cache is not the one they read. They run only where the image does not carry the tools (the
|
||||
# workflow_dispatch Windows leg); the check below fails a Linux image that stops carrying them.
|
||||
- name: Set up Node.js
|
||||
if: runner.os != 'Linux'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 24
|
||||
cache: npm
|
||||
|
||||
- name: Set up Python
|
||||
if: runner.os != 'Linux'
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Check the image carries Node 24 and Python 3.12
|
||||
if: runner.os == 'Linux'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
node --version | grep -E '^v24\.'
|
||||
python --version | grep -E '^Python 3\.12\.'
|
||||
|
||||
# The image's Python is distro-managed (PEP 668): pip refuses to install into it, so the Linux leg installs the
|
||||
# CultLib packages into a venv under RUNNER_TEMP that later steps find first on PATH.
|
||||
- name: Create the Python environment
|
||||
if: runner.os == 'Linux'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
python -m venv "$RUNNER_TEMP/venv"
|
||||
echo "$RUNNER_TEMP/venv/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Install TypeScript workspace dependencies
|
||||
run: npm install --no-package-lock
|
||||
|
||||
|
|
|
|||
8
.github/workflows/quic-native.yml
vendored
8
.github/workflows/quic-native.yml
vendored
|
|
@ -9,21 +9,13 @@ name: CultMesh QUIC native
|
|||
# from the pinned debian:13 container below, never from a local or Yggdrasil
|
||||
# build.
|
||||
|
||||
# Only a change to the committed binaries, their manifest or this workflow can change what verify-committed checks,
|
||||
# so a branch push or pull request runs it for those paths alone; a tag always runs it.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "packages/cultmesh-ts/native/**"
|
||||
- ".github/workflows/quic-native.yml"
|
||||
tags:
|
||||
- "cultmesh-quic-native-v*"
|
||||
pull_request:
|
||||
paths:
|
||||
- "packages/cultmesh-ts/native/**"
|
||||
- ".github/workflows/quic-native.yml"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -26,15 +26,13 @@ All documents are CultNet map-mode: every key written, nil where absent, closed
|
|||
|
||||
### card
|
||||
|
||||
`{schemaVersion, fork_id, title, context, options[{id, text}], recommendation, depends[], leans, flag, raised_in, raised_at, deciders[], decider_count, status, ruling, ruled_from[], order, options_hash}`.
|
||||
`{schemaVersion, fork_id, title, context, options[{id, text}], recommendation, depends[], leans, flag, raised_in, deciders[], status, ruling, ruled_from[], order, options_hash}`.
|
||||
|
||||
- `fork_id`: the mind's `question` key, `<campaign>:question:<label>`.
|
||||
- **Hashed (the shown members):** `title`, `context`, `options` (each `{id, text}`, in shown order), `recommendation` (an option id, **never nil**: the stored `question` requires `recommended`), `depends` (strings, each one of the stored question's `depends` lines verbatim, at most 8, each at most 1000 UTF-8 bytes (the stored `Line`), in shown order, empty when none; the stored kind holds prose lines, so a line may read like a reference such as `campaign:kind:label` but is hashed as the text it is).
|
||||
- **Not hashed:** `fork_id` (bound beside the hash), `leans` (an option id or nil), `flag`, `raised_in`, `raised_at` (the instant the question was first admitted, an RFC3339 instant string as `answered_at`, or nil when the mind holds no admission time, as for a card projected before the field; the key is always written; it never moves when the question is edited; age is the client's: now minus `raised_at`), `deciders`, `decider_count` (the number of deciders of the call, set beside `deciders`; neither it nor the emptiness of `deciders` enters the hash, so the hash a non-decider recomputes is a decider's), `status` (`open` or `closed`), `ruling` (the ruling id when a ruling closed the fork; nil when the card is open, and nil on a closed card whose question was withdrawn or otherwise resolved with no ruling), `ruled_from` (answer ids a ruling has been admitted from; non-empty only on a closed card), `order`, and `options_hash` itself.
|
||||
- **`deciders` is read-only data.** The list is set outside the Mind, through Bifrost's admin path, and **Bifrost sends it per call**: Huginn keeps no copy (M5). The card projection copies the list the call carried. **The per-call list holds one to 64 ids** (Huginn's bound; vectors `deciders_at_the_cap_of_64`, `deciders_65`); the per-call list has no schema here, so the bound is stated for the cut that builds it. **`decider_count` is the size of that list, an integer 1 to 64, always written** (vectors `decider_count_missing`, `_zero`, `_65`, `_string`, `_null`). A fork with no decider is still refused, now as `decider_count` 0. **`deciders` holds that list for a decider and is empty for a reader who is not one: empty means "the ids are withheld from this reader", never "no deciders"** (vector `withheld_deciders`). Rule R-card: when `deciders` is non-empty it holds exactly `decider_count` ids; a visible list is whole (vectors `decider_count_fewer_than_the_ids`, `decider_count_more_than_the_ids_shown`). A question's own `audience` may only narrow it. A Mind never writes it, so a Mind cannot add itself and answer its own fork.
|
||||
- Rules a schema cannot say, each with a negative vector: a `depends` line is at most 1000 UTF-8 bytes (the schema bounds 8 lines and 1000 characters; bytes need the rule); option ids are unique; `order` is unique among a Mind's cards (a list with two cards on one `order` is refused, naming both fork ids); `recommendation` and `leans` name an option; deciders are unique; an open card names no `ruling` and its `ruled_from` is empty (a ruling that answers a fork closes it); `ruled_from` is non-empty only when `ruling` is set; a closed card may name no ruling (the question was withdrawn, superseded or deferred: step 6 reads `status`, not the ruling); `options_hash` equals the hash of the shown members; R-card (above); R-answer-result: `answer_result.answer.answerer` is never nil, being the sender's own stamped account (vector `answer_result_with_a_nil_answerer`); R-record, R-entry and R-list ("Reader views" below).
|
||||
|
||||
**Reader views.** A card and its record are shown in one of two views. The **decider view** carries the ids: `deciders` whole, every `answerer` named, and the counts equal to the ids. The **non-decider view** withholds them: `deciders` empty, every `answerer` nil, the counts kept (`decider_count`; `answerer_count` on the record). Who is a decider is Huginn's decision, not this contract's: a reader is a decider exactly when the call's subject is byte-equal to one of the call's deciders (`eureka-substrate:ruling:desk-read-visibility-withheld-ids`); Huginn's door withholds, Bifrost cannot (the card is opaque to it). There is no viewer flag on any document: the view is read off whether `deciders` is empty. This contract carries both views and refuses a view that leaks: R-record: if every `answerer` is named, `answerer_count` is the number of distinct ids; if all are nil and there is at least one answer, `answerer_count` is between 1 and the number of answers; with no answers it is 0 (vectors `answerer_count_disagrees_with_the_ids`, `answerer_count_counts_answers_not_people`, `withheld_count_over_the_answers`, `withheld_count_zero_over_answers`, `withheld_count_over_no_answers`); R-entry, the leak rule: an entry with `deciders` non-empty names every answerer, an entry with `deciders` empty names none (vectors `decider_card_with_withheld_answers`, `withheld_card_with_named_answers`, `withheld_card_with_a_named_answerer_among_nils`); R-list: all entries of one page agree on the view (vector `a_page_mixes_views`). The view does not move the page: `non_decider_page` has the counts, order, cursor and stamp of the decider page it twins, and the vectors' no-leak check finds no id string in a non-decider vector's bytes. `options_hash` is the same in both views, and the bound-answer and `stale_fork` machinery is untouched.
|
||||
- **Not hashed:** `fork_id` (bound beside the hash), `leans` (an option id or nil), `flag`, `raised_in`, `deciders`, `status` (`open` or `closed`), `ruling` (the ruling id when a ruling closed the fork; nil when the card is open, and nil on a closed card whose question was withdrawn or otherwise resolved with no ruling), `ruled_from` (answer ids a ruling has been admitted from; non-empty only on a closed card), `order`, and `options_hash` itself.
|
||||
- **`deciders` is read-only data.** The list is set outside the Mind, through Bifrost's admin path, and **Bifrost sends it per call**: Huginn keeps no copy (M5). The card projection copies the list the call carried. **The list holds one to 64 ids** (Huginn's bound), on the card and in the per-call list alike (vectors `deciders_at_the_cap_of_64`, `deciders_65`); the per-call list has no schema here, so the bound is stated for the cut that builds it. A question's own `audience` may only narrow it. A Mind never writes it, so a Mind cannot add itself and answer its own fork.
|
||||
- Rules a schema cannot say, each with a negative vector: a `depends` line is at most 1000 UTF-8 bytes (the schema bounds 8 lines and 1000 characters; bytes need the rule); option ids are unique; `order` is unique among a Mind's cards (a list with two cards on one `order` is refused, naming both fork ids); `recommendation` and `leans` name an option; deciders are unique; an open card names no `ruling` and its `ruled_from` is empty (a ruling that answers a fork closes it); `ruled_from` is non-empty only when `ruling` is set; a closed card may name no ruling (the question was withdrawn, superseded or deferred: step 6 reads `status`, not the ruling); `options_hash` equals the hash of the shown members.
|
||||
|
||||
### Options hash (canonical bytes)
|
||||
|
||||
|
|
@ -67,9 +65,7 @@ The stamped `authority` field of a stored answer is an **attribution**: the role
|
|||
|
||||
`received_at` is Huginn's admission time; "first" means first by admission order (the `n<seq>` ordinal), never by `answered_at`. `claim_in_future` is true when `answered_at` is more than five minutes after `received_at`: accepted but flagged. It is a function of the two stored values, so the writer compares against the `received_at` it stores, at the precision it stores it (a flag decided against an unrounded clock and a stored second-truncated `received_at` disagrees with the document it sits in). **Not stored, and no answer id:** a body refused at step 1 or 2 (Bifrost; it never reaches Huginn), and a body naming an unknown fork (step 3; Huginn answers it from its stored forks, but with no question there is nothing to key a record under). Each gets a `refusal` and is dropped. A malformed body is therefore never stored, and neither is an unknown fork. Every refusal from step 4 on is stored. **`unavailable`** (503) is not a refusal and not in the refusal set: Bifrost answers it when the second door or Huginn did not answer after the auth steps and the body shape passed (a bad body is still refused first). **It says the outcome is NOT CONFIRMED, never that nothing was stored**: a timeout after the request reached Huginn may have stored it. It carries no answer id. The desk retries the same request with the same `request_id` and learns the outcome: the stored one if Huginn had it, a fresh one if not. It covers **calls 1 and 2 as well as the answer POST** when Huginn cannot be reached (Transport; a read has no outcome to confirm, so a retry is plain). `retryAfterSeconds` is nil when Bifrost has no hint.
|
||||
|
||||
**The record.** `record` is `{schemaVersion, fork_id, options_hash, counted_answer_id, answerer_count, answers[]}`. `answerer_count` is an integer of at least 0: the number of distinct answerer ids over every stored answer of the fork, refusals included (a refusal stores the stamped answerer too), and it is the same number in both views (vectors `answerer_count_missing`, `_negative`, `_null`). `answers[].answerer` is a string or nil (nil only in the non-decider view).
|
||||
|
||||
**The mind's record and this document.** Huginn keeps the answer as its own record kind (`epiphany.pipeline.desk_answer.v2`, key `<campaign>:desk_answer:<question-label>.n<seq>`, the outcome as one externally tagged value, the fork as `fork`). The second door projects it to this document and nothing else crosses: `answer_id` is the key's `n<seq>` part, `fork_id` is the stored `fork`, the tagged outcome flattens into `outcome`, `of`, `retracts` and `code`, and the six refusal codes are spelt as in the schema (snake_case; the record's enum spells them in the mind's own PascalCase). On the wire an answer is named by its `n<seq>` alone, always inside the fork whose `fork_id` the document carries; a runtime that holds the full record key converts at the door, in one place. Huginn's stored `desk_answer` keeps a non-nil `answerer`; nil exists only in a record projected to a non-decider. `body_sha256` and any other digest Huginn keeps to decide "same body" stay in the record: Huginn alone decides a replay or a `request_id_conflict` and Bifrost never compares bodies, so no digest is a field of this document. A digest is a valid way to decide it only if it covers `fork_id`, `options_hash`, `answer.choice`, `answer.note`, `retracted`, `retracts` and `answered_at` (every field of the request but `request_id`), is computed by Huginn and never read from a client.
|
||||
**The mind's record and this document.** Huginn keeps the answer as its own record kind (`epiphany.pipeline.desk_answer.v2`, key `<campaign>:desk_answer:<question-label>.n<seq>`, the outcome as one externally tagged value, the fork as `fork`). The second door projects it to this document and nothing else crosses: `answer_id` is the key's `n<seq>` part, `fork_id` is the stored `fork`, the tagged outcome flattens into `outcome`, `of`, `retracts` and `code`, and the six refusal codes are spelt as in the schema (snake_case; the record's enum spells them in the mind's own PascalCase). On the wire an answer is named by its `n<seq>` alone, always inside the fork whose `fork_id` the document carries; a runtime that holds the full record key converts at the door, in one place. `body_sha256` and any other digest Huginn keeps to decide "same body" stay in the record: Huginn alone decides a replay or a `request_id_conflict` and Bifrost never compares bodies, so no digest is a field of this document. A digest is a valid way to decide it only if it covers `fork_id`, `options_hash`, `answer.choice`, `answer.note`, `retracted`, `retracts` and `answered_at` (every field of the request but `request_id`), is computed by Huginn and never read from a client.
|
||||
|
||||
**Huginn's digest, pinned in the vectors (`bodyDigests`; informative, not a wire document).** SHA-256 of one compact JSON object as `serde_json::to_string` writes it (UTF-8, no spaces), lowercase hex, keys in this order, `null` for a missing value, strings verbatim: `fork`, `options_hash`, `answered_at`, `choice`, `note`, `retracts` (the **full stored key** `<campaign>:desk_answer:<label>.n<seq>`: the door converts the wire's `n<seq>` before the digest), then, **only for the three illegal forms** (R1: a retraction carrying an answer, a retraction naming no target, a plain answer carrying a target), `illegal_retraction: true` and `retracted: true|false`. `request_id` is not in it. A plain answer and a legal retraction carry neither trailing key, so their digests are the ones Huginn main pins (`8a4d5b2e`, `9f807538`) and do not move. `9f807538` is pinned for a target no door produces (`Q1.1`); the golden for the real legal retraction is the door's form, the wire's `n1` converted to `Q1.n1` (`84576840`, vector `legal_retraction_door_form`, also main-valid). Source: Eureka's `desk-body-digest-order-20261110.md`, computed with `sha256sum`; the generator, the C# suite and the TypeScript suite recompute every digest from the input fields. **Status:** the three `retracted` goldens (`935cab1a`, `82b24ccc`, `34ef86ed`, and `79eee623` for form (i) with a null target) are the form of `cut-desk-digest-retracted.r2`, merged as Huginn #30. Eureka reports that these four and the door-form legal retraction (`84576840`) reproduce byte for byte on Huginn main `f0ca504` (the #30 fix); that run is hers, reported to us, and was not rerun here. Before #30, Huginn main (`7ea11b4`) wrote `illegal_retraction` alone, so forms (i) and (iii) collided there (`69074cc6`, pinned as `illegal_i_and_iii_alike_on_main`). The wire contract does not wait on it: the conflict scenarios are pinned in `scenarios`.
|
||||
|
||||
|
|
@ -117,8 +113,8 @@ Source: the desk's client needs (Ratatoskr PR #24 section 12, `ratatoskr-desk-sp
|
|||
- **M1.** `{mind}` is looked up in a **fixed configured list** of Mind ids, each mapped in configuration to that Mind's socket on the peer-checked door. The segment is percent-decoded once and compared, ordinally, with the list's ids: no case folding, trimming or Unicode normalisation, and a decoding failure is a miss. It is **never joined into a path, a socket name or a query**, so a value off the list opens nothing (`../x`, `a/b`, `a%2fb`, `eureka%00`, a 4 KB string, `GAMECULT-CHAT` for `gamecult-chat`, a trailing space, bytes that are not UTF-8: all unknown, 404, no socket opened).
|
||||
- **M2.** An unknown Mind gets **the same refusal as an unknown fork**: `fork_unknown`, 404, the same bytes (the document's `error` names neither the Mind nor the fork). The list of Minds is not probeable. Auth is judged first, so a caller without a valid token, scope or Active membership learns nothing about which Minds exist.
|
||||
- **M3.** On the POST the route's Mind must hold `fork_id`: step 3 is judged against **that Mind's** stored forks, so a `fork_id` the route's Mind does not hold is `fork_unknown` and nothing is stored, in either Mind. A fork cannot be answered through another Mind's route. A body that names `mind` is `body_unknown_field`: the Mind is the route's.
|
||||
- **M4.** Scopes are **not per Mind**: `desk.read` and `desk.answer` open the routes of every configured Mind; every reader sees every fork's card and outcome (a non-decider sees no account id, "Reader views"), and which forks a caller may answer is the decider list.
|
||||
- **M5. The decider list is sent per call.** Bifrost holds the admin-set list and sends it with **every answer, read and list request** to Huginn, as a typed field on the same peer-checked door; Huginn keeps no copy of it. Because it travels with read and list too, a projected card's `deciders` is the list of that call for a decider and empty for a reader who is not one, with `decider_count` travelling with both views, and step 5 judges an answer against the list the same call carried. It comes only from Bifrost's admin-set state: never from a Mind and never from any request body a client controls: no HTTP field, header or query parameter on any desk route sets or extends it, and a body naming `deciders` is `body_unknown_field` (scenario `M5_...`). A request carrying it from a Mind's own door or session is refused. **This is Bifrost and Huginn work and no vector here can test it** (the typed field and the door's wire are pending, open choice 3); the rule is stated so that the cut that builds the request has it. (Heimdall's ruling, #minds 2026-11-10 19:26Z; it replaces the earlier reading of a list Huginn receives once and holds.) **Envelope names:** the desk's envelope on that door uses the names this page uses, `mind`, `authority`, `subject` and `deciders`, with no conversion at the door (Eureka, #minds 2026-11-10 19:58Z); `deciders` is a per-call list of short ids at the envelope's top level beside `subject`, never inside the request body, and Huginn keeps no copy.
|
||||
- **M4.** Scopes are **not per Mind**: `desk.read` and `desk.answer` open the routes of every configured Mind; every reader sees every fork's card and outcome, and which forks a caller may answer is the decider list.
|
||||
- **M5. The decider list is sent per call.** Bifrost holds the admin-set list and sends it with **every answer, read and list request** to Huginn, as a typed field on the same peer-checked door; Huginn keeps no copy of it. Because it travels with read and list too, a projected card's `deciders` is the list of that call, and step 5 judges an answer against the list the same call carried. It comes only from Bifrost's admin-set state: never from a Mind and never from any request body a client controls: no HTTP field, header or query parameter on any desk route sets or extends it, and a body naming `deciders` is `body_unknown_field` (scenario `M5_...`). A request carrying it from a Mind's own door or session is refused. **This is Bifrost and Huginn work and no vector here can test it** (the typed field and the door's wire are pending, open choice 3); the rule is stated so that the cut that builds the request has it. (Heimdall's ruling, #minds 2026-11-10 19:26Z; it replaces the earlier reading of a list Huginn receives once and holds.) **Envelope names:** the desk's envelope on that door uses the names this page uses, `mind`, `authority`, `subject` and `deciders`, with no conversion at the door (Eureka, #minds 2026-11-10 19:58Z); `deciders` is a per-call list of short ids at the envelope's top level beside `subject`, never inside the request body, and Huginn keeps no copy.
|
||||
|
||||
Calls 1 and 2 are enough for a read-only desk. Every response is the pinned MessagePack document or its JSON form with the same keys (open choice 1 of `vote-api.md`: the media type is unsettled).
|
||||
|
||||
|
|
@ -126,7 +122,7 @@ Calls 1 and 2 are enough for a read-only desk. Every response is the pinned Mess
|
|||
|
||||
**List.** `list_request`: `{mind, status, cursor, limit}`. `mind` is the route's `{mind}`, a Mind id from the configured list. `status` is `open`, `recorded` or nil for both; `recorded` means a closed card. Cards come **newest first by `order`**. `order` is the card's unique rank per Mind: an integer from 0 that never changes once the card exists, later questions only appending (Huginn derives it from the Mind's questions sorted by admission ordinal, then key). A Mind never holds two cards of one `order`; a list or card set that does is refused, and the refusal names both fork ids. 0 is legal, but no cursor names it (`c0` fails the cursor pattern): a page ending at order 1 returns `c1`, and the next page returns the order-0 card alone with `next_cursor` nil. `limit` is 0 to 50; **`limit` 0 returns no entries, only the counts and the stamp**, which is what "N open" on the Mind's header needs. `cursor` is nil or `c<N>`: the cards whose `order` is below N; a client treats it as opaque and sends back the `next_cursor` it was given. `list`: `{mind, stamp, open_count, recorded_count, forks[entry], next_cursor}`. The counts cover every card the viewer may see, whatever `status` and `cursor` asked for. `stamp` is `s<N>`, N the receiver's last-seen Huginn admission ordinal for the Mind's desk documents: the same stamp means the same state, so a desk polls `limit` 0 and fetches bodies only when the stamp moves. `next_cursor` is nil on the last page. **A page may hold fewer than `limit` entries** when the next entry would push the encoded response past the receiver's response budget (Huginn: 65536 bytes); it then holds at least one, `next_cursor` is `c<order of the last entry returned>` so the next page resumes below it, and the counts and the stamp are those of the full list, unchanged. **`fork_too_large`** (HTTP 422) refuses a page whose **first** entry alone is past the budget: 422 because the request is well formed and authorised but names a resource the list cannot carry, and 413 is for a request body that is too large, which this is not; retrying returns the same refusal. Its `error` names the fork id and its `order`, for example `fork_too_large: fork <fork_id> (order <n>) alone is past the response budget`. **A refusal's `error` text is informational and not pinned**: the contract pins the code, the HTTP status, that the text names the fork id (and the order on a list), and the `c<order>` continuation on a list; the vectors' wording is a sample, and the judges check those four things, not the bytes of the text. The client continues the list with cursor `c<n>`, the cards strictly below that order, which skips this fork; it is then readable only by id (the read route returns the whole entry, and refuses with the same code only when that entry alone is past the cap, see Read); `c<n>` is valid only when n is at least 1, so for order 0 there is nothing below and the client has reached the end. `limit` 0 returns no entries and never refuses. (Ruled by Persona, 2026-11-11.) An `entry` is `{card, record}`: the card and its `record` (the answers with their outcomes and the derived `counted_answer_id`).
|
||||
|
||||
**Read.** `read_request` is `{mind, fork_id}`; the response is the `entry` for one fork of that Mind. Its `card.options_hash` is **computed by the server** from the five shown members as in "Options hash" and the client recomputes it from what it rendered; a card whose hash is not the hash of its members is no card (negative vector `card_hash_wrong`). The record's `fork_id` and `options_hash` are the card's, and `counted_answer_id` is derived, not copied (negative vectors for each). Every `desk.read` holder sees every fork's card and outcome in a configured Mind; only a decider's notes depend on the caller, going to callers in the admin-set `deciders` list, which Bifrost enforces at the door (ruling `desk-read-visibility`, option (c)); `fork_unknown` is only for a fork id that does not exist under the route's Mind (an unknown Mind answers the same, M2), so no fork is ever hidden by it, and a caller without `desk.read` is refused by the auth order above (`insufficient_scope`) before any lookup. **Huginn withholds account ids from a reader who is not a decider and returns counts ("Reader views"); Bifrost still strips notes.** Every `desk.read` holder sees cards and outcomes; notes (a decider's own words) only go to callers in the admin-set `deciders` list. The read and list shapes are unchanged and carry no audience field (`eureka-substrate:question:desk-read-visibility`, Heimdall's option (c)). **A read is refused `fork_too_large` (422) when the whole entry is past the reply cap** (Huginn's daemon caps every reply at 65536 bytes; the cap, not the page budget, is what a read meets). Its `error` names the fork id and has no order and no cursor to carry, because there is nothing to continue with (sample: `fork_too_large: fork <fork_id> alone is past the response budget`) (vector `transport.readTooLarge`; the cap is in no vector). **The answer route meets the same cap:** when an answer is `stale_fork`, Bifrost re-reads the current card to attach it, and if that read is past the cap the answer is refused `fork_too_large` (422) with the read form of the text, the same document as a read. Huginn has already stored the answer as `Refused(stale_fork)`, as for any stale answer, and it is never counted; the client is told `fork_too_large` instead of receiving the current card, and a replay of the same `request_id` gets the same 422 while the fork stays oversize. It is never `unavailable` (vector `transport.answerTooLarge`). A stripped read serves a decider's `note` as `note: null`, never omitted (map-mode: every key is written); `desk_answer.note` is already nullable in the schema, so a served record validates as it stands.
|
||||
**Read.** `read_request` is `{mind, fork_id}`; the response is the `entry` for one fork of that Mind. Its `card.options_hash` is **computed by the server** from the five shown members as in "Options hash" and the client recomputes it from what it rendered; a card whose hash is not the hash of its members is no card (negative vector `card_hash_wrong`). The record's `fork_id` and `options_hash` are the card's, and `counted_answer_id` is derived, not copied (negative vectors for each). Every `desk.read` holder sees every fork's card and outcome in a configured Mind; only a decider's notes depend on the caller, going to callers in the admin-set `deciders` list, which Bifrost enforces at the door (ruling `desk-read-visibility`, option (c)); `fork_unknown` is only for a fork id that does not exist under the route's Mind (an unknown Mind answers the same, M2), so no fork is ever hidden by it, and a caller without `desk.read` is refused by the auth order above (`insufficient_scope`) before any lookup. **Bifrost applies visibility at the door; Huginn returns the whole record to the door.** Every `desk.read` holder sees cards and outcomes; notes (a decider's own words) only go to callers in the admin-set `deciders` list. The read and list shapes are unchanged and carry no audience field (`eureka-substrate:question:desk-read-visibility`, Heimdall's option (c)). **A read is refused `fork_too_large` (422) when the whole entry is past the reply cap** (Huginn's daemon caps every reply at 65536 bytes; the cap, not the page budget, is what a read meets). Its `error` names the fork id and has no order and no cursor to carry, because there is nothing to continue with (sample: `fork_too_large: fork <fork_id> alone is past the response budget`) (vector `transport.readTooLarge`; the cap is in no vector). **The answer route meets the same cap:** when an answer is `stale_fork`, Bifrost re-reads the current card to attach it, and if that read is past the cap the answer is refused `fork_too_large` (422) with the read form of the text, the same document as a read. Huginn has already stored the answer as `Refused(stale_fork)`, as for any stale answer, and it is never counted; the client is told `fork_too_large` instead of receiving the current card, and a replay of the same `request_id` gets the same 422 while the fork stays oversize. It is never `unavailable` (vector `transport.answerTooLarge`). A stripped read serves a decider's `note` as `note: null`, never omitted (map-mode: every key is written); `desk_answer.note` is already nullable in the schema, so a served record validates as it stands.
|
||||
|
||||
**Answer.** The body is the `answer_request` and nothing else a client may set: `fork_id`, `options_hash`, `answer`, `answered_at`, `request_id` (plus `retracted` and `retracts`, written `false` and nil unless the call retracts). The response is the stored outcome: `Counted`, `Revision`, `Retraction`, or `Refused` with its code, and the **`counted_answer_id`** the fork has once this call is applied (`n<seq>` or nil; derived as in "The counted answer is derived", the same value the `record` projection carries, so the desk needs no second read). A replay returns the stored outcome with the `counted_answer_id` current at the retry, not the one it had when first stored. **A `stale_fork` refusal carries the current card** in `answer_result.card`, so the desk can show the new fork without another read. Nothing is stored for a step 1 or 2 refusal, an unknown fork (or an unknown Mind, or a fork another Mind holds) or a `request_id_conflict`. `unavailable` promises nothing either way.
|
||||
|
||||
|
|
@ -153,9 +149,7 @@ The old desk (`index.html`) read `questions/<id>` and wrote `answers/<id>` in a
|
|||
| `status` open or closed, `ruling` | derived from the resolution (status is never written): any resolution of the question closes it; the ruling is the one that answered it | `status`, `ruling`, both a projection (`ruling` is nil on an open card and on a card closed with no ruling) |
|
||||
| `order` | none (unique rank per Mind, from 0) | `order` |
|
||||
| none | `raised_in`, `asked_on` | `raised_in`; `asked_on` is not carried |
|
||||
| none | the question's admission time (no timestamp in the old format; `asked_on` is a date and stays uncarried) | `raised_at`, an instant or nil, not hashed |
|
||||
| none | none | `deciders` (outside the Mind), `ruled_from`, `options_hash` |
|
||||
| none | none | `decider_count`, `answerer_count` (counts that stand in for withheld ids) |
|
||||
| answer `{choice: key or null, note, question, updatedAt}`, one mutable document per question | none | `answer_request` then an append-only `desk_answer` |
|
||||
| `note`: her words verbatim | | `answer.note`, verbatim |
|
||||
| "Clear pick": `choice` back to null so the note stands alone | | an `answer` with `choice` nil and `note` set, as the first answer or as a `Revision` |
|
||||
|
|
@ -184,8 +178,6 @@ Option B, **Bifrost receives the answer and Huginn stores it** (the design in th
|
|||
- `request_id` (on the request and the stored `desk_answer`) and `request_id_conflict` are this page's: his file has no idempotency key, and a retry after `unavailable` needs one. Eureka's `desk_answer` kind and Huginn's request type must carry it.
|
||||
- Transport: the routes, the `{mind}` segment, the scopes, the audience and the auth order are his rulings (4.1 to 4.3a); the documents (list, read, `unavailable`) are this page's. The note limit is 4000 UTF-8 bytes here, as his 1.1 leaves the unit unsaid.
|
||||
- The recommendation is never nil. The binding file's earlier vectors D and E used a nil recommendation; its updated D and E use `"a"`, as here.
|
||||
- `raised_at` (2026-11-11) is the desk-feed addition (Bifrost PR 67): a required key, an instant or nil, outside `options_hash` (the five hashed members and every digest are unchanged; vectors `raised_at_nil`, `raised_at_epoch_integer`, `raised_at_date_only`, `raised_at_missing`). The card is spec-authored and no runtime deploys it, so the schema id stays `gamecult.fork.card.v0`.
|
||||
- Reader views (2026-11-11) are the desk-feed addition behind Huginn PR 56 and `eureka-substrate:ruling:desk-read-visibility-withheld-ids`: the card gains `decider_count` and `deciders` may be empty; the record gains `answerer_count` and `answerer` may be nil, the counts beside the collections they summarise (not on the list envelope, which neither the read entry nor the `stale_fork` card travels in). `options_hash` and every digest are unchanged. The schema ids stay `gamecult.fork.card.v0` and `gamecult.fork.record.v0` for the same reason as `raised_at`.
|
||||
- Hash vectors: A, B, C, D, E, F and G of the binding file are reproduced digest for digest by the Python generator, the C# suite and the TypeScript suite (the generator prints `same` for each; none differs).
|
||||
|
||||
## Open choices
|
||||
|
|
@ -194,7 +186,7 @@ Option B, **Bifrost receives the answer and Huginn stores it** (the design in th
|
|||
2. NFC across runtimes relies on each runtime's normaliser; a pinned table, as `write-receipt.md` does for letters, would remove the reliance. The risk is only for code points assigned after a runtime's Unicode version.
|
||||
3. How Huginn learns the verified subject: the second door's authenticated context (the peer credential names Bifrost the service, not the member), so the subject must travel beside the request. This contract has no envelope for it.
|
||||
4. Whether `fork_unknown` and shape refusals should leave an audit record somewhere other than a `desk_answer`.
|
||||
5. Whether `leans` and `flag`, which the booth shows, should join the hash. The binding file's five members do not include them; a changed lean would not make an answer stale. `raised_at` is immutable, so it was kept out of the hash for the same reason it need not be in: it can never make a shown card stale.
|
||||
5. Whether `leans` and `flag`, which the booth shows, should join the hash. The binding file's five members do not include them; a changed lean would not make an answer stale.
|
||||
6. What the person sees of a retracted counted answer when the next answer counts: this contract only says the count moves.
|
||||
7. `received_at` is not in the binding file's stored fields; it is kept for `claim_in_future` and could be dropped if the `n<seq>` answer ordinal alone decided answer order.
|
||||
8. The stamped `authority` field is an attribution, not an authority (above); renaming it `attribution` would say so, at the price of a new key the door must also refuse in a body.
|
||||
|
|
@ -203,4 +195,3 @@ Option B, **Bifrost receives the answer and Huginn stores it** (the design in th
|
|||
11. While a known Mind's door is down its routes answer `unavailable` (503) and an unknown Mind answers `fork_unknown` (404), so the two are distinguishable for that period. M2 makes them identical only when every door is up; making them identical when one is down means answering `unavailable` for any `{mind}` not on the list, which tells a client nothing about which Minds exist but also hides a typo. Heimdall's to rule; the vectors do not pin it.
|
||||
12. A fork's answers are unbounded in number, so a fork's entry can grow past the reply cap and become unreadable (`fork_too_large` on read, and on list when it is the first entry of a page). Whether the contract should cap the answers per fork is for Persona and Heimdall to rule; nothing here decides it.
|
||||
13. The read form of `fork_too_large` is a target, not current behaviour: Huginn's door answers an over-cap read as `Refused(ResponseTooLarge{bytes, limit})`, which carries no fork id and is not `fork_too_large`. Bifrost maps it to `fork_too_large` (422) using the route's `fork_id`; until it does, a read past the cap reaches the client as 503 `unavailable`. The same holds for the answer route: a `stale_fork` answer whose card re-read is past the cap is a Bifrost target too (`fork_too_large` 422 over a stored `Refused(stale_fork)`), and until it is built the client sees 503 `unavailable`.
|
||||
14. What a non-decider can still infer: the card's `status` and `order`, the `authority` role on each answer (a role, not an id, and the ruling leaves it whole), and `answerer_count`. Whether a role plus a count is too much to show is Eureka's ruling to revisit.
|
||||
|
|
|
|||
|
|
@ -70,9 +70,7 @@
|
|||
"leans",
|
||||
"flag",
|
||||
"raised_in",
|
||||
"raised_at",
|
||||
"deciders",
|
||||
"decider_count",
|
||||
"status",
|
||||
"ruling",
|
||||
"ruled_from",
|
||||
|
|
@ -145,30 +143,15 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
"raised_at": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/instant"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"deciders": {
|
||||
"type": "array",
|
||||
"minItems": 0,
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"maxItems": 64
|
||||
},
|
||||
"decider_count": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 64
|
||||
},
|
||||
"status": {
|
||||
"enum": [
|
||||
"open",
|
||||
|
|
@ -390,14 +373,7 @@
|
|||
"type": "boolean"
|
||||
},
|
||||
"answerer": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"authority": {
|
||||
"$ref": "#/$defs/id"
|
||||
|
|
@ -483,14 +459,7 @@
|
|||
"type": "boolean"
|
||||
},
|
||||
"answerer": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"authority": {
|
||||
"$ref": "#/$defs/id"
|
||||
|
|
@ -562,14 +531,7 @@
|
|||
"type": "boolean"
|
||||
},
|
||||
"answerer": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"authority": {
|
||||
"$ref": "#/$defs/id"
|
||||
|
|
@ -669,14 +631,7 @@
|
|||
"type": "boolean"
|
||||
},
|
||||
"answerer": {
|
||||
"oneOf": [
|
||||
{
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
"$ref": "#/$defs/id"
|
||||
},
|
||||
"authority": {
|
||||
"$ref": "#/$defs/id"
|
||||
|
|
@ -780,7 +735,6 @@
|
|||
"fork_id",
|
||||
"options_hash",
|
||||
"counted_answer_id",
|
||||
"answerer_count",
|
||||
"answers"
|
||||
],
|
||||
"properties": {
|
||||
|
|
@ -803,10 +757,6 @@
|
|||
}
|
||||
]
|
||||
},
|
||||
"answerer_count": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"answers": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
|
|
|
|||
|
|
@ -701,7 +701,6 @@ checked by the operator in the editor.
|
|||
- Run `MintElementIds` once over `GameData/Aetheria.cc`.
|
||||
- AetherDb migrations skip variants or use the variant API (Q1a).
|
||||
- Re-author the families listed in section 6.
|
||||
- Hull families live in each ship's own `.cc`, not in `GameData/Aetheria.cc`, because hulls are mod-owned packages (Nibu's ruling about her data, 2026-11-11, #minds).
|
||||
|
||||
### C5. CultNet deltas (C# reference)
|
||||
|
||||
|
|
@ -1609,7 +1608,7 @@ The one door per runtime, and who walks through it:
|
|||
- **Shared paths.** cultnet-ts reaches the header decision only through `readSingleFileStore` (AUTH:4, :321) and checks only record count, schema id and type. cultnet-ts therefore needs a cultcache-ts release that exports it, which is the release that ships with this change; the manifests move to that version then (c0-ships-with-c2a), not before.
|
||||
- **Cut line.** `merge-r1` h3 (Rust) and h4 (TS, cultnet-ts); spec `cut-merge-r1.r3` authority map.
|
||||
- **Pinned by.** `tests/vectors/document-variants-c2a/readability/manifest.txt`, walked by `StoreReadabilityTests.OpenReadsExactlyTheFilesTheVectorsSayItReads` (C#), `CultMeshSingleFileReadabilityTests.AWriteReplacesAFileExactlyWhenItReads` (Mesh), `a_file_is_replaced_exactly_when_it_reads` (RS:5132), "SingleFileMessagePackBackingStore replaces a file exactly when it reads" (TS test :2321) and `test_single_file_replaces_a_file_exactly_when_it_reads` (Python). AUTH cases: `packages/cultnet-ts/test/idunn-runtime-authority.test.ts`. Declared divergences in the manifest: `legacy-envelopes.bin` (C# refuses; the others read and replace it) and `variant-slot-v3.bin` (C# reads; the others refuse).
|
||||
- **Known gaps.** `variants:follow_up:range-check-in-ci` (`npm run test:pack:ts` builds every admitted tag and checks exports, but `.github/workflows/publish-packages.yml` does not run it).
|
||||
- **Known gaps.** `variants:follow_up:c0-bad-canonical-in-manifest` (`current-catalog-bad-canonical.msgpack` is not in the manifest; TS reads it, Rust refuses). `variants:follow_up:legacy-vectors-all-runtimes` (the three `legacy-catalog-*` c0 vectors have a verdict only in C# tests; all four runtimes refuse them, probed). `variants:follow_up:range-check-in-ci` (`npm run test:pack:ts` builds every admitted tag and checks exports, but `.github/workflows/publish-packages.yml` does not run it).
|
||||
|
||||
### B. Existence and links (what "nothing at the path" means)
|
||||
|
||||
|
|
@ -1640,7 +1639,7 @@ The one door per runtime, and who walks through it:
|
|||
- **Shared paths.** Open, commit, Mesh read and write.
|
||||
- **Cut line.** `mesh-single-file-reader` h1 (the C# clause). Rust, TS and Python were correct by construction of their reads.
|
||||
- **Pinned by.** C#: `ADirectoryAtTheStorePathIsAnIoErrorOnOpenAndOnCommitNotAnEmptyStore` (`StoreReadabilityTests.cs:236`). Mesh: `ADirectoryAtThePathIsAnIoErrorOnReadAndWriteAndIsLeftAlone` (:149). Rust: `a_write_over_a_directory_stops_at_the_read_and_leaves_no_staging_file` RS:5726.
|
||||
- **Known gaps.** None open. The `@directory` manifest row is walked by all four runtimes: every operation including delete is refused and leaves the directory untouched; Rust pins an I/O failure rather than a typed unreadable store.
|
||||
- **Known gaps.** TS and Python have no test for a directory at the store path. No follow-up is registered; the natural home is `write-set-vectors` with the other readability pins.
|
||||
|
||||
### E. Lock
|
||||
|
||||
|
|
|
|||
|
|
@ -40,9 +40,6 @@ Probes: scratch tests in this worktree, removed. C# filtered runs on Starfire (o
|
|||
| P5 cache {A,B}: remove, then add in a second commit | C | n/a | n/a | n/a |
|
||||
| P6 N2 over an N1 record (undeclared name drift), cache {N2} / cacheless reg {N2} | C restamped / C restamped | not probed | not probed | C (cache) |
|
||||
|
||||
The C#-written probe the leaves are held to is `tests/vectors/key-identity` (12 rows: retype, foreign carry, key in two stores), walked by
|
||||
`KeyIdentityVectorTests`; each leaf cut corrects its own column above.
|
||||
|
||||
What this establishes:
|
||||
|
||||
- C#: one record per key. The only path that retypes is the cache arm (P1): `CacheBackingStore.RefuseUndeclared`
|
||||
|
|
|
|||
|
|
@ -9,10 +9,6 @@
|
|||
|
||||
### Breaking
|
||||
|
||||
- A single-file store holding two records under one key is unreadable: the v1 reader and the legacy envelope-array reader refuse it with `StoreUnreadableError` (a key is compared by its exact code points, no case folding or normalisation). A catalog slot whose canonical schema is not a string is refused the same way. A store that loaded before can now be refused: repair it with the runtime that wrote the duplicate.
|
||||
- A `put` or `push_all` that would land a second type under a key the store holds is refused with `SchemaConflictError`, and the file is left as it was. Retype a key by deleting it, then putting the new type. A write replaces the record at a key only when the written type owns it: its catalog entry declares the stored schema id (its own, or one it lists compatible), so a renamed type that lists the old id compatible may replace the record, or it has the stored record's schema name. This runtime does not compare shapes, so a same-name type is never refused for dropping persisted slots as C# refuses it.
|
||||
- Writes of the same type replace by key: the last record wins, in a batch as in the file.
|
||||
|
||||
- `indexes` are unique, as C# `[CultIndex]`: a write, load or `register_index` that leaves two records of a type holding one non-blank value under one alias is refused (`CultCacheError`) naming both keys and the alias, and writes nothing. Index text is the C# canonical text (`True`/`False`, integers as digits, floats as the exact expansion, blank values never indexed), so an empty or whitespace string is no longer a value. A new `roles` map names the many-row selection aliases (never indexed for lookup, same catalog `index_alias`). Classify every `indexes` key per the README before updating. The verdict is the contract, the text is informational.
|
||||
|
||||
- A single-file store written as `cultcache.store.v3` is read and written: a store the C# runtime marked v3 (it holds a record with an element id: an owned list element with a `[CultElementId]` member) keeps its header here, and is carried through every rewrite with its ids; this runtime never drops the mark. A runtime from before this change refuses a v3 store by name. Rolling back to an older release after a store has been written as v3 is one-way: there is no carry-back, so restore from a copy taken before upgrading.
|
||||
|
|
|
|||
|
|
@ -317,7 +317,7 @@ class CultCache:
|
|||
structures, running every check and every user extractor; the cache's state is replaced
|
||||
only once that build finishes, so a refused load admits nothing."""
|
||||
loaded_values: dict[str, dict[str, Any]] = {}
|
||||
read_values: dict[tuple[str, str], Any] = {} # (type, key) -> value, one per key, the last read wins; the judge is order-free
|
||||
read_order: dict[tuple[str, str], Any] = {} # (type, key) -> value, in the order the stores yielded them
|
||||
loaded_envelopes: dict[str, dict[str, CultCacheEnvelope]] = {}
|
||||
loaded_legacy_global_keys: dict[str, str] = {}
|
||||
routes, generic = self._state.stores_by_type, self._state.generic_store
|
||||
|
|
@ -356,10 +356,10 @@ class CultCache:
|
|||
envelope = self._stamped(document, envelope)
|
||||
value = document.decode_payload(envelope.payload)
|
||||
loaded_values.setdefault(envelope.type, {})[envelope.key] = value
|
||||
read_values[(envelope.type, envelope.key)] = value
|
||||
read_order[(envelope.type, envelope.key)] = value
|
||||
loaded_envelopes.setdefault(envelope.type, {})[envelope.key] = envelope
|
||||
loaded_names, loaded_indexes, loaded_lookup_keys = self._derive_lookups(
|
||||
[(type, key, value) for (type, key), value in read_values.items()], self._state.name_extractors, self._state.index_extractors
|
||||
[(type, key, value) for (type, key), value in read_order.items()], self._state.name_extractors, self._state.index_extractors
|
||||
)
|
||||
# Plain attribute assignments: nothing fallible runs once the swap begins.
|
||||
(
|
||||
|
|
@ -627,7 +627,7 @@ class CultCache:
|
|||
name_extractors: dict[str, str | Any],
|
||||
index_extractors: dict[str, dict[str, str | Any]],
|
||||
) -> tuple[_Names, _Indexes, _LookupKeysByType]:
|
||||
"""Builds name and index lookups for the given (type, key, value) records, judged whole and order-free, in fresh dicts, touching no cache state."""
|
||||
"""Builds name and index lookups for the given (type, key, value) records, judged in the order given, in fresh dicts, touching no cache state."""
|
||||
names: _Names = {}
|
||||
indexes: _Indexes = {}
|
||||
lookup_keys: _LookupKeysByType = {}
|
||||
|
|
@ -636,7 +636,7 @@ class CultCache:
|
|||
keys = self._lookup_keys_for(value, name_extractors.get(type), index_extractors.get(type, {}))
|
||||
lookup_keys.setdefault(type, {})[key] = keys
|
||||
landing.append((type, key, keys))
|
||||
# Judged whole and order-free, over every type at once and on the file alone (never against what the cache held): a held value can
|
||||
# Judged whole, over every type at once and on the file alone (never against what the cache held): a held value can
|
||||
# have been mutated in place since it was admitted, and another process may have moved a value between records.
|
||||
self._refuse_index_sharing(landing, {}, ())
|
||||
for type in lookup_keys:
|
||||
|
|
@ -653,35 +653,28 @@ class CultCache:
|
|||
held: dict[tuple[str, str, str], str],
|
||||
replaced: Iterable[tuple[str, str]],
|
||||
) -> None:
|
||||
"""The one index rule (variants:ruling:unique-indexes): within one type and one unique alias, no two records hold one
|
||||
non-blank value, judged on the state the admission would leave and never on staging, file or pull order. The holders
|
||||
of a value are the landing keys projecting it plus the held key unless this admission replaces it. Of every
|
||||
(type, alias, value) with two or more distinct holders, the refusal names the least by (alias, its least key, its
|
||||
second-least key, schema name), each compared by UTF-16 code unit, never the value."""
|
||||
"""The one index rule (variants:ruling:unique-indexes), in the order of C# RefuseIndexSharing: within one type and one
|
||||
unique alias, no two records hold one non-blank value, judged on the result. The holders of a value are the landing
|
||||
keys projecting it (in landing order) plus the held key unless this admission replaces it; two distinct holders
|
||||
refuse, naming the two smallest keys (UTF-16 code-unit order) and the alias, never the value. Aliases are judged in
|
||||
UTF-16 order, values in the order they landed."""
|
||||
released = set(replaced)
|
||||
landed: dict[tuple[str, str, str], list[str]] = {}
|
||||
for type, key, keys in landing:
|
||||
for index, index_value in keys.indexes.items():
|
||||
landed.setdefault((type, index, index_value), []).append(key)
|
||||
violations: list[tuple[bytes, bytes, bytes, bytes, str, str, str, str]] = []
|
||||
for (type, index, index_value), keys in landed.items():
|
||||
for (type, index, index_value), keys in sorted(landed.items(), key=lambda item: _utf16_order(item[0][1])):
|
||||
held_key = held.get((type, index, index_value))
|
||||
if held_key is not None and (type, held_key) not in released:
|
||||
keys = [*keys, held_key]
|
||||
distinct = sorted(keys, key=_utf16_order)
|
||||
distinct = sorted(dict.fromkeys(keys), key=_utf16_order)
|
||||
if len(distinct) > 1:
|
||||
document = self._state.documents.get(type)
|
||||
schema_name = (document.schema_name if document is not None else None) or type
|
||||
violations.append(
|
||||
(_utf16_order(index), _utf16_order(distinct[0]), _utf16_order(distinct[1]), _utf16_order(schema_name),
|
||||
index, distinct[0], distinct[1], schema_name)
|
||||
raise CultCacheError(
|
||||
f"Records {distinct[0]} and {distinct[1]} ({schema_name}) both hold unique index '{index}'; "
|
||||
"an index value names one record."
|
||||
)
|
||||
if violations:
|
||||
*_, index, first, second, schema_name = min(violations, key=lambda violation: violation[:4])
|
||||
raise CultCacheError(
|
||||
f"Records {first} and {second} ({schema_name}) both hold unique index '{index}'; "
|
||||
"an index value names one record."
|
||||
)
|
||||
|
||||
def _remove_lookups(self, type: str, key: str) -> None:
|
||||
"""Removes the lookups the held record produced at admission, handing a name to another held
|
||||
|
|
|
|||
|
|
@ -245,44 +245,11 @@ def _derive_catalog(records: list[tuple[CultCacheEnvelope, bool]]) -> list[CultC
|
|||
return [entry for entry, _ in chosen.values()]
|
||||
|
||||
|
||||
def _retyped(held: CultCacheEnvelope, written: CultCacheEnvelope) -> SchemaConflictError:
|
||||
"""The refusal of a write that would put a second type under a key the store already holds (C# CacheBackingStore.Retyped)."""
|
||||
return SchemaConflictError(
|
||||
_schema_id_for(held),
|
||||
[held.type],
|
||||
held.key,
|
||||
message=(
|
||||
f"Record '{held.key}' is a {held.type} (schema id '{_schema_id_for(held)}'); writing a {written.type} there would "
|
||||
"retype the key. Remove the record first, in the same batch or an earlier commit, then add the other type's."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _written_owns(held: CultCacheEnvelope, written: CultCacheEnvelope) -> bool:
|
||||
"""Whether a write of `written` may replace `held` at its key (C# WrittenOwns): the written type's entry declares the held
|
||||
record's schema id (its own id or one it lists compatible), or it is the type of the same schema name. The same-name half
|
||||
stands in for C#'s "reads the published entry whole": this runtime does not compare shapes, so it never refuses a
|
||||
same-name type for dropping slots."""
|
||||
stored_id = _schema_id_for(held)
|
||||
entry = written.catalog_entry if written.catalog_entry is not None else _default_catalog_entry(written)
|
||||
return stored_id == entry.schema_id or stored_id in entry.compatible_schema_ids or held.type == written.type
|
||||
|
||||
|
||||
def _encode_snapshot(
|
||||
supplied: list[CultCacheEnvelope], arrived: list[CultCacheEnvelope], format_version: str
|
||||
) -> list[Any]:
|
||||
# A key names one record whatever its type, so a supplied record replaces the arrived one by key alone, and only of the
|
||||
# type that owns it: retyping a key is a removal and an add, so a record another type owns under the key refuses the write.
|
||||
held_by_key = {envelope.key: envelope for envelope in arrived}
|
||||
landing: dict[str, CultCacheEnvelope] = {}
|
||||
for envelope in supplied:
|
||||
for held in (landing.get(envelope.key), held_by_key.get(envelope.key)):
|
||||
if held is not None and not _written_owns(held, envelope):
|
||||
raise _retyped(held, envelope)
|
||||
landing[envelope.key] = envelope
|
||||
supplied = list(landing.values())
|
||||
replaced = set(landing)
|
||||
kept = [envelope for envelope in arrived if envelope.key not in replaced]
|
||||
replaced = {(envelope.type, envelope.key) for envelope in supplied}
|
||||
kept = [envelope for envelope in arrived if (envelope.type, envelope.key) not in replaced]
|
||||
envelopes = [*kept, *supplied]
|
||||
catalog = [
|
||||
_encode_catalog_entry(entry)
|
||||
|
|
@ -433,7 +400,6 @@ def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None
|
|||
catalog_by_schema_id.setdefault(compatible_schema_id, entry)
|
||||
|
||||
envelopes: list[CultCacheEnvelope] = []
|
||||
seen: set[str] = set()
|
||||
for raw_record in decoded[2]:
|
||||
if not isinstance(raw_record, list) or len(raw_record) < 4:
|
||||
raise ValueError("CultCache persisted records must be MessagePack arrays")
|
||||
|
|
@ -446,9 +412,6 @@ def _decode_snapshot(decoded: Any) -> tuple[str, list[CultCacheEnvelope]] | None
|
|||
key, schema_id, stored_at, payload = raw_record
|
||||
if not isinstance(key, str) or not key:
|
||||
raise ValueError("CultCache persisted records must declare a key")
|
||||
if key in seen:
|
||||
raise ValueError(f"CultCache record {key!r} appears more than once")
|
||||
seen.add(key)
|
||||
if not isinstance(schema_id, str) or not schema_id:
|
||||
raise ValueError("CultCache persisted records must declare a schema id")
|
||||
if not isinstance(stored_at, str) or not stored_at:
|
||||
|
|
@ -474,13 +437,9 @@ def _decode_legacy_envelopes(decoded: Any) -> list[CultCacheEnvelope]:
|
|||
if not isinstance(decoded, list):
|
||||
raise ValueError("CultCache MessagePack store is not a recognized snapshot")
|
||||
envelopes: list[CultCacheEnvelope] = []
|
||||
seen: set[Any] = set()
|
||||
for item in decoded:
|
||||
if not isinstance(item, dict):
|
||||
raise ValueError("CultCache legacy envelopes must be MessagePack maps")
|
||||
if item.get("key") in seen:
|
||||
raise ValueError(f"CultCache record {item['key']!r} appears more than once")
|
||||
seen.add(item.get("key"))
|
||||
try:
|
||||
payload = item["payload"]
|
||||
envelopes.append(
|
||||
|
|
@ -525,14 +484,12 @@ def _decode_catalog_entry(value: Any) -> CultCacheSchemaCatalogEntry:
|
|||
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid compatible schema ids')
|
||||
if not isinstance(members, list):
|
||||
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has invalid members')
|
||||
if not isinstance(canonical_schema_json, str):
|
||||
raise ValueError(f'CultCache schema catalog entry "{schema_id}" has an invalid canonical schema')
|
||||
return CultCacheSchemaCatalogEntry(
|
||||
schema_id=schema_id,
|
||||
schema_name=schema_name,
|
||||
schema_version=schema_version if isinstance(schema_version, str) and schema_version else f"{schema_name}.v1",
|
||||
content_hash=content_hash if isinstance(content_hash, str) and content_hash else schema_id,
|
||||
canonical_schema_json=canonical_schema_json,
|
||||
canonical_schema_json=canonical_schema_json if isinstance(canonical_schema_json, str) else "",
|
||||
compatible_schema_ids=tuple(compatible_schema_ids),
|
||||
members=tuple(_decode_catalog_member(member) for member in members),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -1567,19 +1567,15 @@ class CultCacheTests(unittest.TestCase):
|
|||
# Every vector in the folder has a manifest row.
|
||||
self.assertEqual(
|
||||
sorted(path.name for path in root.glob("*.bin")),
|
||||
sorted(vector for vector, *_ in rows if not vector.startswith(("..", "@"))),
|
||||
sorted(vector for vector, *_ in rows if not vector.startswith("..")),
|
||||
)
|
||||
envelopes = SingleFileMessagePackBackingStore(self._C2A_VECTORS / "v3-base.msgpack").pull_all()
|
||||
for vector, _, _, _, python in rows:
|
||||
# "@directory": the walker makes an empty directory at the store path; it is refused and left untouched.
|
||||
content = None if vector == "@directory" else (root / vector).read_bytes()
|
||||
content = (root / vector).read_bytes()
|
||||
for operation in ("pull_all", "push_all", "push", "delete"):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
store_path = Path(tmp) / "store.msgpack"
|
||||
if content is None:
|
||||
store_path.mkdir()
|
||||
else:
|
||||
store_path.write_bytes(content)
|
||||
store_path.write_bytes(content)
|
||||
store = SingleFileMessagePackBackingStore(store_path)
|
||||
run = {
|
||||
"pull_all": store.pull_all,
|
||||
|
|
@ -1602,129 +1598,12 @@ class CultCacheTests(unittest.TestCase):
|
|||
self.assertIn(envelopes[0].key, keys, f"{vector} {operation}")
|
||||
else:
|
||||
self.assertNotIn(envelopes[0].key, keys, f"{vector} {operation}")
|
||||
elif content is None:
|
||||
# A path that cannot be reached as a file stays an OSError (_read_store): not a store that failed to decode.
|
||||
with self.assertRaises(OSError, msg=f"{vector} {operation}"):
|
||||
run()
|
||||
self.assertTrue(store_path.is_dir() and not any(store_path.iterdir()), f"{vector} {operation} touched the directory")
|
||||
else:
|
||||
with self.assertRaises(StoreUnreadableError, msg=f"{vector} {operation}") as refused:
|
||||
run()
|
||||
self.assertIsNotNone(refused.exception.__cause__, f"{vector} {operation}")
|
||||
self.assertEqual(store_path.read_bytes(), content, f"{vector} {operation} rewrote a file it cannot read")
|
||||
|
||||
@staticmethod
|
||||
def _rec(type: str, key: str, second: int = 0) -> CultCacheEnvelope:
|
||||
return CultCacheEnvelope(key=key, type=type, payload=b"\x90", stored_at=f"2026-09-30T00:00:0{second}Z")
|
||||
|
||||
def test_a_supplied_record_replaces_an_arrived_one_of_the_same_type_by_key(self) -> None:
|
||||
import msgpack # type: ignore
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
store = SingleFileMessagePackBackingStore(path)
|
||||
store.push(self._rec("first", "k"))
|
||||
store.push(self._rec("first", "k", 1))
|
||||
store.push_all([self._rec("first", "k", 2), self._rec("first", "k", 3)])
|
||||
records = msgpack.unpackb(path.read_bytes(), raw=False)[2]
|
||||
self.assertEqual([record[0] for record in records], ["k"])
|
||||
self.assertEqual([(e.type, e.stored_at) for e in store.pull_all()], [("first", "2026-09-30T00:00:03Z")])
|
||||
|
||||
def test_a_write_that_would_retype_a_key_is_refused_and_leaves_the_file(self) -> None:
|
||||
from cultcache_py import SchemaConflictError
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
store = SingleFileMessagePackBackingStore(path)
|
||||
store.push(self._rec("first", "k"))
|
||||
content = path.read_bytes()
|
||||
for run in (
|
||||
lambda: store.push(self._rec("second", "k", 1)),
|
||||
lambda: store.push_all([self._rec("other", "j"), self._rec("second", "k", 1)]),
|
||||
):
|
||||
with self.assertRaises(SchemaConflictError) as refused:
|
||||
run()
|
||||
self.assertEqual(refused.exception.record_key, "k")
|
||||
self.assertEqual(refused.exception.schema_names, ["first"])
|
||||
self.assertIn("would retype the key", str(refused.exception))
|
||||
self.assertEqual(path.read_bytes(), content)
|
||||
# Retyping is a removal and an add.
|
||||
store.delete("first", "k")
|
||||
store.push(self._rec("second", "k", 1))
|
||||
self.assertEqual([e.type for e in SingleFileMessagePackBackingStore(path).pull_all()], ["second"])
|
||||
|
||||
def test_a_retype_is_judged_by_the_stored_schema_id_the_written_type_declares_not_by_name(self) -> None:
|
||||
from cultcache_py import SchemaConflictError
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
store = SingleFileMessagePackBackingStore(path)
|
||||
store.push(self._writer_record("k", "old", "id-old", self._writer_entry("id-old", "old", "h1", ("id-old",))))
|
||||
content = path.read_bytes()
|
||||
# A renamed type that lists the stored id compatible owns the record, so its write replaces it.
|
||||
renamed = self._writer_record("k", "new", "id-new", self._writer_entry("id-new", "new", "h2", ("id-new", "id-old")))
|
||||
# A type that lists nothing of the stored id and has another name does not.
|
||||
stranger = self._writer_record("k", "stranger", "id-x", self._writer_entry("id-x", "stranger", "h3", ("id-x",)))
|
||||
for run in (lambda: store.push(stranger), lambda: store.push_all([stranger])):
|
||||
with self.assertRaises(SchemaConflictError):
|
||||
run()
|
||||
self.assertEqual(path.read_bytes(), content)
|
||||
store.push(renamed)
|
||||
self.assertEqual([(e.type, e.schema_id) for e in SingleFileMessagePackBackingStore(path).pull_all()], [("new", "id-new")])
|
||||
|
||||
def test_a_batch_naming_one_key_under_two_types_is_refused_whole(self) -> None:
|
||||
from cultcache_py import SchemaConflictError
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
store = SingleFileMessagePackBackingStore(path)
|
||||
with self.assertRaises(SchemaConflictError):
|
||||
store.push_all([self._rec("first", "k"), self._rec("second", "k", 1)])
|
||||
self.assertFalse(path.exists())
|
||||
|
||||
def test_a_cache_put_of_a_second_type_under_a_key_is_refused_and_persists_nothing_new(self) -> None:
|
||||
from cultcache_py import SchemaConflictError
|
||||
|
||||
other = define_document_type("other", encode=lambda item: asdict(item), decode=lambda raw: Item(**raw))
|
||||
document = item_doc()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
|
||||
def build() -> CultCache:
|
||||
cache = (
|
||||
CultCache.builder()
|
||||
.register_document_type(document)
|
||||
.register_document_type(other)
|
||||
.add_generic_store(SingleFileMessagePackBackingStore(path))
|
||||
.build()
|
||||
)
|
||||
cache.pull_all_backing_stores()
|
||||
return cache
|
||||
|
||||
cache = build()
|
||||
cache.put(document, "shared", Item(name="A", category="c", value=1))
|
||||
content = path.read_bytes()
|
||||
with self.assertRaises(SchemaConflictError):
|
||||
cache.put(other, "shared", Item(name="B", category="c", value=2))
|
||||
self.assertEqual(path.read_bytes(), content)
|
||||
self.assertIsNone(cache.get(other, "shared"))
|
||||
reloaded = build()
|
||||
self.assertEqual(reloaded.get_required(document, "shared").value, 1)
|
||||
self.assertIsNone(reloaded.get(other, "shared"))
|
||||
cache.delete(document, "shared")
|
||||
cache.put(other, "shared", Item(name="B", category="c", value=2))
|
||||
self.assertEqual(build().get_required(other, "shared").value, 2)
|
||||
|
||||
def test_a_catalog_entry_with_a_non_string_canonical_schema_is_unreadable(self) -> None:
|
||||
# Direct probe of the c0 vector; its manifest row is added by the TS cut.
|
||||
vector = self._C2A_VECTORS.parent / "document-variants-c0" / "current-catalog-bad-canonical.msgpack"
|
||||
content = vector.read_bytes()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.msgpack"
|
||||
path.write_bytes(content)
|
||||
with self.assertRaises(StoreUnreadableError):
|
||||
SingleFileMessagePackBackingStore(path).pull_all()
|
||||
self.assertEqual(path.read_bytes(), content)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
|
|||
|
|
@ -470,76 +470,6 @@ class VectorTests(unittest.TestCase):
|
|||
self.assertEqual(path.read_bytes(), before)
|
||||
|
||||
|
||||
def test_refusals(self) -> None:
|
||||
ids = {
|
||||
"tests.unique_item": "sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882",
|
||||
"tests.uq_pair": "sha256:2538cceafb6dcbce22ef11b932ecb804af5f96bc3aaedff692a52d3640433e60",
|
||||
"tests.uq_alias": "sha256:77a77a394ebd8bf71e14b9b6f2cea4978c25d4e705fd12364f660edbf6c63b1f",
|
||||
"tests.uq_wide": "sha256:b79eecd11eed5e456902f095d2748da7f59125c4c8972b0da7dddc81d07fc03f",
|
||||
"tests.uq_alpha": "sha256:53aeaf2ea9b25e85ade6b15fc04c7653e7b206c87e258f1aa74dd32a840a1389",
|
||||
"tests.uq_beta": "sha256:45ed62a64c91f6d815d3ec975c4a4a1b2ce07b6e7262a32d828673cf10899586",
|
||||
}
|
||||
aliases = {
|
||||
"tests.uq_pair": {"Sku": "sku", "Alt": "second"},
|
||||
"tests.uq_alias": {"Sku": "sku", "alt": "second"},
|
||||
"tests.uq_wide": {"\uff21": "sku", "\U00010400": "second"}, # U+FF21 and U+10400: code-unit order puts the astral alias first
|
||||
}
|
||||
documents = {
|
||||
name: define_database_entry_type(
|
||||
name,
|
||||
[database_entry_field("sku", 0, codec=t.string), database_entry_field("second", 1, codec=t.string)],
|
||||
indexes=aliases.get(name, {"Sku": "sku"}),
|
||||
roles={},
|
||||
schema_id=schema_id,
|
||||
schema_name=name,
|
||||
)
|
||||
for name, schema_id in ids.items()
|
||||
}
|
||||
rows = msgpack.unpackb((VECTORS / "refusals.msgpack").read_bytes(), raw=False)
|
||||
walked = 0
|
||||
for label, ops, expected in rows:
|
||||
if any(op[0] == "commit" and len({schema for _, schema, _, _ in op[1]}) > 1 for op in ops):
|
||||
continue # put_envelopes takes one type: no batch across types (rows 3 and 26)
|
||||
with self.subTest(label=label), tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "store.cc"
|
||||
state: dict[str, CultCache] = {}
|
||||
|
||||
def perform(op) -> None:
|
||||
if op[0] == "open":
|
||||
if op[1] is not None:
|
||||
shutil.copyfile(VECTORS / op[1], path)
|
||||
state["cache"] = cache_of(*documents.values(), store=SingleFileMessagePackBackingStore(path))
|
||||
state["cache"].pull_all_backing_stores()
|
||||
elif op[0] == "pull":
|
||||
shutil.copyfile(VECTORS / op[1], path)
|
||||
state["cache"].pull_all_backing_stores()
|
||||
else:
|
||||
document = documents[op[1][0][1]]
|
||||
state["cache"].put_envelopes(
|
||||
document, [envelope(document, key, {"sku": sku, "second": second}) for key, _, sku, second in op[1]]
|
||||
)
|
||||
|
||||
for op in ops[:-1]:
|
||||
perform(op)
|
||||
if expected is None:
|
||||
perform(ops[-1])
|
||||
else:
|
||||
last = ops[-1]
|
||||
if last[0] == "open":
|
||||
before = (VECTORS / last[1]).read_bytes()
|
||||
elif last[0] == "pull":
|
||||
before = (VECTORS / last[1]).read_bytes()
|
||||
else:
|
||||
before = path.read_bytes() if path.exists() else b""
|
||||
k1, k2, schema, alias = expected
|
||||
with self.assertRaises(CultCacheError) as caught:
|
||||
perform(last)
|
||||
self.assertEqual(str(caught.exception), refusal(k1, k2, schema, alias))
|
||||
self.assertEqual(path.read_bytes() if path.exists() else b"", before)
|
||||
walked += 1
|
||||
self.assertEqual(walked, 35)
|
||||
|
||||
|
||||
class ListStore(RecordingStore):
|
||||
"""A store whose contents the test swaps between loads: another process rewrote the file."""
|
||||
|
||||
|
|
@ -585,6 +515,22 @@ class BatchAndOrderTests(unittest.TestCase):
|
|||
)
|
||||
self.assertNotIn("item:j", cache.snapshot().get("item", {}))
|
||||
|
||||
def test_values_are_named_in_landing_order(self) -> None:
|
||||
document = item_doc()
|
||||
cache = cache_of(document)
|
||||
rows = [("item:a", "B"), ("item:b", "B"), ("item:x", "A"), ("item:y", "A")]
|
||||
with self.assertRaises(CultCacheError) as caught:
|
||||
cache.put_envelopes(document, [envelope(document, key, Item(sku=sku)) for key, sku in rows])
|
||||
self.assertEqual(str(caught.exception), refusal("item:a", "item:b", "item", "sku"))
|
||||
|
||||
def test_keys_sort_by_utf16_code_unit_not_code_point(self) -> None:
|
||||
document = item_doc()
|
||||
cache = cache_of(document)
|
||||
astral, bmp = "k\U0001F600", "k\uff01" # code points: bmp < astral; UTF-16 units: D83D < FF01
|
||||
with self.assertRaises(CultCacheError) as caught:
|
||||
cache.put_envelopes(document, [envelope(document, bmp, Item(sku="S")), envelope(document, astral, Item(sku="S"))])
|
||||
self.assertEqual(str(caught.exception), refusal(astral, bmp, "item", "sku"))
|
||||
|
||||
def test_a_load_judges_every_type_at_once_aliases_in_ordinal_order(self) -> None:
|
||||
first = item_doc("first", indexes={"zeta": "sku"}, roles={})
|
||||
second = item_doc("second", indexes={"alpha": "sku"}, roles={})
|
||||
|
|
@ -607,7 +553,7 @@ class BatchAndOrderTests(unittest.TestCase):
|
|||
cache.put_envelopes(document, [envelope(document, f"item:{n}", row) for n, row in enumerate(rows)])
|
||||
self.assertEqual(str(caught.exception), refusal("item:0", "item:1", "item", astral))
|
||||
|
||||
def test_a_load_names_the_least_violation_whatever_order_the_store_yields(self) -> None:
|
||||
def test_a_load_judges_records_in_the_order_the_file_yields_them(self) -> None:
|
||||
alpha, beta = item_doc("alpha", indexes={"k": "sku"}, roles={}), item_doc("beta", indexes={"k": "sku"}, roles={})
|
||||
loose_alpha, loose_beta = item_doc("alpha", indexes={}, roles={}), item_doc("beta", indexes={}, roles={})
|
||||
store = ListStore(
|
||||
|
|
@ -622,7 +568,7 @@ class BatchAndOrderTests(unittest.TestCase):
|
|||
cache = cache_of(alpha, beta, store=store)
|
||||
with self.assertRaises(CultCacheError) as caught:
|
||||
cache.pull_all_backing_stores()
|
||||
self.assertEqual(str(caught.exception), refusal("a:1", "a:2", "alpha", "k"))
|
||||
self.assertEqual(str(caught.exception), refusal("b:1", "b:2", "beta", "k"))
|
||||
|
||||
def test_register_index_refuses_an_alias_the_definition_declares_as_a_role(self) -> None:
|
||||
document = item_doc()
|
||||
|
|
|
|||
|
|
@ -14,10 +14,6 @@ only in git history.
|
|||
- `bool` index and lookup text renders `True` / `False`, as C# does, where it rendered `true` / `false`.
|
||||
- `DatabaseEntry::UNIQUE_INDEXES` replaces `cult_indexes` as the one index declaration and changes type (its members are the unique aliases); a hand implementation overriding `cult_indexes` no longer compiles.
|
||||
- `register_entry_type` over a type that already holds records re-judges them under the new declaration (unique indexes and the global rule) and refuses, changing nothing, when they violate it. A declaration with an index also refuses when a held record cannot be decoded as the new type; a declaration without one does not read the records, so it does not.
|
||||
- A single-file store holds one record per key. The v1 and legacy readers refuse a store with two records under one key (`CultCacheStoreUnreadable`, the record `Undecodable`): the store does not open, and a write over it refuses and leaves the file. A consumer that holds such a file must repair it with the runtime that wrote it before opening it here.
|
||||
- Every single-file writer (`push`, `push_all`, `insert_entry_if_absent`, the compare-exchange and snapshot-append family, and the cache's puts and batches) refuses to put a second type under a key the store holds, with `SchemaConflictError` naming the key and both types, writing nothing. Retyping a key is a delete, then an add (one batch may hold both). A write owns the held record when its schema id equals the id the held record is stored under, as C# `WrittenOwns` asks; a type renamed under a stable id therefore replaces its own record, and a write of another type name under another id is refused (`push` of the same type name under another id still replaces the record, as before; `compare_and_swap_entry`, and a `push_all` batch holding one type under two ids, refuse it). Registered types here declare no compatible ids, so equality is the whole test. Writers used to leave two records under the key.
|
||||
- `compare_and_swap_entry` over a single file refuses an in-place retype (expected `A/k`, replacement `B/k` under another schema id) with the same `SchemaConflictError`, as `compare_exchange` does, where it used to retype the record and return `true`. A lost race still returns `false` first.
|
||||
- A write of the same type under a held key replaces the record by key, and in one `push_all` the last given of several entries under a key and a schema id is the record, whatever their type names. The redb stores are unchanged and still key by type and key.
|
||||
|
||||
### Added
|
||||
|
||||
|
|
|
|||
|
|
@ -143,8 +143,6 @@ use serde::de::DeserializeOwned;
|
|||
use std::any::Any;
|
||||
use std::collections::BTreeMap;
|
||||
use std::collections::BTreeSet;
|
||||
use std::collections::HashMap;
|
||||
use std::collections::HashSet;
|
||||
use std::fs;
|
||||
use std::fs::File;
|
||||
use std::fs::OpenOptions;
|
||||
|
|
@ -931,8 +929,7 @@ fn store_header(bytes: &[u8]) -> StoreHeader {
|
|||
}
|
||||
|
||||
/// A write the catalog cannot describe: records of different types under one schema id. This writer derives one entry per carried
|
||||
/// schema id from its records, so that is its only conflict besides a single-file write that would put a second type under a
|
||||
/// key the store already holds (the key is the record's identity there). Nothing is written. It names the id, the two names and a record key.
|
||||
/// schema id from its records, so that is its only conflict. Nothing is written. It names the id, the two names and a record key.
|
||||
#[derive(Debug)]
|
||||
pub struct SchemaConflictError {
|
||||
pub schema_id: String,
|
||||
|
|
@ -1187,10 +1184,6 @@ impl SingleFileMessagePackBackingStore {
|
|||
if entries[index] != *expected {
|
||||
return Ok(false);
|
||||
}
|
||||
// An in-place retype is a retype: the replacement must own the schema id the held record is stored under.
|
||||
if schema_identity(&entries[index]) != schema_identity(&replacement) {
|
||||
return Err(retype_conflict(&entries[index], &replacement));
|
||||
}
|
||||
entries[index] = replacement;
|
||||
entries.sort_by_key(entry_id);
|
||||
self.write_all_unlocked(&entries)?;
|
||||
|
|
@ -1498,8 +1491,7 @@ impl SingleFileMessagePackBackingStore {
|
|||
Err(error) if io_not_found(&error) => DiskStore::absent(),
|
||||
other => other?,
|
||||
};
|
||||
let entries = settle_one_record_per_key(entries, &disk)?;
|
||||
let bytes = encode_store_snapshot(&entries, &disk)
|
||||
let bytes = encode_store_snapshot(entries, &disk)
|
||||
.and_then(|snapshot| rmp_serde::to_vec(&snapshot).context("failed to encode MessagePack"))
|
||||
.map_err(|error| error.context(failed(CultCacheStoreWriteFailedKind::Rejected)))?;
|
||||
self.replace_unlocked(&bytes)
|
||||
|
|
@ -1753,7 +1745,6 @@ impl CacheBackingStore for SingleFileMessagePackBackingStore {
|
|||
fn push(&mut self, entry: &CultCacheEnvelope) -> Result<()> {
|
||||
self.with_exclusive_lock(|| {
|
||||
let mut entries = self.read_all_unlocked()?;
|
||||
// A write replaces the record of its own type under its key; a second type under the key reaches the commit guard.
|
||||
entries.retain(|candidate| entry_id(candidate) != entry_id(entry));
|
||||
entries.push(entry.clone());
|
||||
entries.sort_by_key(entry_id);
|
||||
|
|
@ -1777,13 +1768,7 @@ impl CacheBackingStore for SingleFileMessagePackBackingStore {
|
|||
return Ok(());
|
||||
}
|
||||
|
||||
// Of entries sharing a key and a schema id the last one given is the record, whatever their type names (a type
|
||||
// renamed under a stable id); a second schema id under a key reaches the commit guard.
|
||||
let mut by_id = BTreeMap::<(String, String), &CultCacheEnvelope>::new();
|
||||
for entry in entries {
|
||||
by_id.insert((schema_identity(entry).to_string(), entry.key.clone()), entry);
|
||||
}
|
||||
let mut entries: Vec<CultCacheEnvelope> = by_id.into_values().cloned().collect();
|
||||
let mut entries = entries.to_vec();
|
||||
entries.sort_by_key(entry_id);
|
||||
self.write_all_unlocked(&entries)
|
||||
})
|
||||
|
|
@ -2753,16 +2738,6 @@ pub struct CultCache {
|
|||
}
|
||||
|
||||
type IndexProjector = fn(&[u8]) -> Result<Vec<(&'static str, String)>>;
|
||||
/// Orders by UTF-16 code unit (C# `StringComparer.Ordinal`), not by UTF-8 bytes or code point.
|
||||
fn utf16_cmp(left: &str, right: &str) -> std::cmp::Ordering {
|
||||
left.encode_utf16().cmp(right.encode_utf16())
|
||||
}
|
||||
|
||||
/// Orders violations (alias, least key, second key, schema) field by field, each by UTF-16.
|
||||
fn violation_cmp(left: &[&str; 4], right: &[&str; 4]) -> std::cmp::Ordering {
|
||||
left.iter().zip(right).map(|(a, b)| utf16_cmp(a, b)).find(|order| order.is_ne()).unwrap_or(std::cmp::Ordering::Equal)
|
||||
}
|
||||
|
||||
type IndexSlot = (String, String, String);
|
||||
type IndexHolders = BTreeMap<IndexSlot, String>;
|
||||
|
||||
|
|
@ -3449,36 +3424,22 @@ impl CultCache {
|
|||
groups[index].1.insert(key);
|
||||
}
|
||||
}
|
||||
let mut least: Option<[&str; 4]> = None;
|
||||
for (slot, holders) in &mut groups {
|
||||
if let Some(held_key) = held.get(&*slot)
|
||||
groups.sort_by(|left, right| left.0.1.cmp(&right.0.1));
|
||||
for (slot, mut holders) in groups {
|
||||
if let Some(held_key) = held.get(&slot)
|
||||
&& !last.contains_key(&(slot.0.as_str(), held_key.as_str()))
|
||||
{
|
||||
holders.insert(held_key.clone());
|
||||
}
|
||||
}
|
||||
let mut ordered: Vec<(&IndexSlot, Vec<&String>)> = Vec::new();
|
||||
for (slot, holders) in &groups {
|
||||
let mut keys: Vec<&String> = holders.iter().collect();
|
||||
keys.sort_by(|left, right| utf16_cmp(left, right));
|
||||
ordered.push((slot, keys));
|
||||
}
|
||||
for (slot, keys) in &ordered {
|
||||
if let [first, second, ..] = keys.as_slice() {
|
||||
let schema = self.definitions.get(&slot.0).copied().unwrap_or(&slot.0);
|
||||
let violation = [slot.1.as_str(), first.as_str(), second.as_str(), schema];
|
||||
if least.is_none_or(|best| violation_cmp(&violation, &best).is_lt()) {
|
||||
least = Some(violation);
|
||||
}
|
||||
let mut ordered = holders.iter();
|
||||
if let (Some(first), Some(second)) = (ordered.next(), ordered.next()) {
|
||||
return Err(anyhow!(
|
||||
"Records {first} and {second} ({}) both hold unique index '{}'; an index value names one record.",
|
||||
self.definitions.get(&slot.0).copied().unwrap_or(&slot.0),
|
||||
slot.1
|
||||
));
|
||||
}
|
||||
}
|
||||
if let Some([alias, first, second, schema]) = least {
|
||||
return Err(anyhow!(
|
||||
"Records {first} and {second} ({schema}) both hold unique index '{alias}'; an index value names one record."
|
||||
));
|
||||
}
|
||||
for (slot, mut keys) in ordered.into_iter().map(|(slot, keys)| (slot.clone(), keys)) {
|
||||
let key = keys.remove(0).clone();
|
||||
let key = holders.into_iter().next().expect("a group holds at least one key");
|
||||
change.taken.push((slot, key));
|
||||
}
|
||||
Ok(change)
|
||||
|
|
@ -3724,65 +3685,7 @@ impl DiskStore {
|
|||
/// The legacy envelope array: an array of envelope maps. Its acceptance is the one verdict on an array whose first slot
|
||||
/// is not a store header, for reading it and for replacing it.
|
||||
fn decode_legacy_envelopes(bytes: &[u8]) -> Result<Vec<CultCacheEnvelope>> {
|
||||
let envelopes: Vec<CultCacheEnvelope> =
|
||||
decode_grown(bytes).context("failed to decode legacy CultCache envelope array")?;
|
||||
require_unique_keys(&envelopes)?;
|
||||
Ok(envelopes)
|
||||
}
|
||||
|
||||
/// A record's identity in a store is its key string alone, compared ordinal: a store holding a key twice is unreadable,
|
||||
/// whatever the format of the file that holds it.
|
||||
fn require_unique_keys(envelopes: &[CultCacheEnvelope]) -> Result<()> {
|
||||
let mut keys = HashSet::<&str>::new();
|
||||
for envelope in envelopes {
|
||||
if !keys.insert(envelope.key.as_str()) {
|
||||
bail!("CultCache record {:?} appears more than once", envelope.key);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// What a record's stored schema id is: the id it carries, else its type (a registered type's id is its entry type). This
|
||||
/// runtime's registered types declare no compatible ids, so a written record owns a held one exactly when the ids are equal:
|
||||
/// the question C# `WrittenOwns` asks of the stored id, not a comparison of type names.
|
||||
fn schema_identity(entry: &CultCacheEnvelope) -> &str {
|
||||
entry.schema_id.as_deref().unwrap_or(&entry.r#type)
|
||||
}
|
||||
|
||||
/// The refusal of a write that would retype a key: the written record does not own the schema id the held one is stored under.
|
||||
/// Retyping is remove-then-add (variants:ruling:cache-refuses-retype).
|
||||
fn retype_conflict(held: &CultCacheEnvelope, written: &CultCacheEnvelope) -> anyhow::Error {
|
||||
anyhow::Error::new(SchemaConflictError {
|
||||
schema_id: schema_identity(written).to_string(),
|
||||
schema_names: vec![held.r#type.clone(), written.r#type.clone()],
|
||||
record_key: written.key.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
/// The commit guard every single-file write passes: a store holds one record per key. Two entries under a key whose schema
|
||||
/// ids differ are a retype and are refused with the key and both types, nothing written. Two under one schema id are one
|
||||
/// record: a written entry replaces the held one it owns (a type renamed under a stable id), the held one being what the file
|
||||
/// already carries.
|
||||
fn settle_one_record_per_key(entries: &[CultCacheEnvelope], disk: &DiskStore) -> Result<Vec<CultCacheEnvelope>> {
|
||||
let arrived: BTreeMap<(String, String), &CultCacheEnvelope> = disk.envelopes.iter().map(|entry| (entry_id(entry), entry)).collect();
|
||||
let held = |entry: &CultCacheEnvelope| arrived.get(&entry_id(entry)) == Some(&entry);
|
||||
let mut settled = Vec::<&CultCacheEnvelope>::new();
|
||||
let mut at_key = HashMap::<&str, usize>::new();
|
||||
for entry in entries {
|
||||
match at_key.get(entry.key.as_str()) {
|
||||
None => {
|
||||
at_key.insert(entry.key.as_str(), settled.len());
|
||||
settled.push(entry);
|
||||
}
|
||||
Some(&slot) if schema_identity(settled[slot]) != schema_identity(entry) => return Err(retype_conflict(settled[slot], entry)),
|
||||
Some(&slot) => {
|
||||
if !held(entry) || held(settled[slot]) {
|
||||
settled[slot] = entry;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(settled.into_iter().cloned().collect())
|
||||
decode_grown(bytes).context("failed to decode legacy CultCache envelope array")
|
||||
}
|
||||
|
||||
/// Decodes a store whose header the caller has already read as a readable one: its envelopes and its catalog.
|
||||
|
|
@ -3821,7 +3724,6 @@ fn decode_store_snapshot(bytes: &[u8]) -> Result<(Vec<CultCacheEnvelope>, Vec<Pe
|
|||
})
|
||||
})
|
||||
.collect::<Result<Vec<_>>>()?;
|
||||
require_unique_keys(&envelopes)?;
|
||||
Ok((envelopes, snapshot.1))
|
||||
}
|
||||
|
||||
|
|
@ -4293,159 +4195,32 @@ mod tests {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn settings_one() -> Settings {
|
||||
Settings { theme: "green".to_string(), retries: 1 }
|
||||
}
|
||||
|
||||
fn note_one() -> Note {
|
||||
Note { title: "same key".to_string(), body: "different type".to_string() }
|
||||
}
|
||||
|
||||
fn retype_refusal(error: &anyhow::Error) -> &SchemaConflictError {
|
||||
error.downcast_ref::<SchemaConflictError>().unwrap_or_else(|| panic!("{error:#}"))
|
||||
}
|
||||
|
||||
fn reloaded(path: &Path) -> Result<Vec<CultCacheEnvelope>> {
|
||||
SingleFileMessagePackBackingStore::new(path).pull_all()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_second_type_under_a_stored_key_is_refused_and_the_store_and_cache_stay() -> Result<()> {
|
||||
fn entry_identity_is_polymorphic_by_type_and_key() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let store_path = temp.path().join("cache.cc");
|
||||
let mut cache = CultCache::new();
|
||||
cache.register_registry(TestEntries)?;
|
||||
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&store_path))?;
|
||||
|
||||
cache.put("shared", &settings_one())?;
|
||||
let before = std::fs::read(&store_path)?;
|
||||
let error = cache.put("shared", ¬e_one()).unwrap_err();
|
||||
assert_eq!(retype_refusal(&error).record_key, "shared");
|
||||
assert_eq!(std::fs::read(&store_path)?, before, "the refused write changed the file");
|
||||
assert_eq!(cache.snapshot().len(), 1);
|
||||
cache.put(
|
||||
"shared",
|
||||
&Settings {
|
||||
theme: "green".to_string(),
|
||||
retries: 1,
|
||||
},
|
||||
)?;
|
||||
cache.put(
|
||||
"shared",
|
||||
&Note {
|
||||
title: "same key".to_string(),
|
||||
body: "different type".to_string(),
|
||||
},
|
||||
)?;
|
||||
|
||||
assert_eq!(cache.snapshot().len(), 2);
|
||||
assert_eq!(cache.get_required::<Note>("shared")?.title, "same key");
|
||||
assert_eq!(cache.get_required::<Settings>("shared")?.theme, "green");
|
||||
assert!(cache.get::<Note>("shared")?.is_none());
|
||||
|
||||
// Retyping is remove-then-add, and what was written reads back.
|
||||
assert!(cache.delete::<Settings>("shared")?);
|
||||
cache.put("shared", ¬e_one())?;
|
||||
let held = reloaded(&store_path)?;
|
||||
assert_eq!(held.len(), 1);
|
||||
assert_eq!(held[0].r#type, Note::TYPE);
|
||||
|
||||
// The same type under the key replaces it.
|
||||
cache.put("shared", &Note { title: "again".to_string(), body: String::new() })?;
|
||||
let held = reloaded(&store_path)?;
|
||||
assert_eq!(held.len(), 1);
|
||||
assert_eq!(cache.get_required::<Note>("shared")?.title, "again");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_batch_holding_a_second_type_under_a_key_is_refused_whole() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let store_path = temp.path().join("batch.cc");
|
||||
let mut cache = CultCache::new();
|
||||
cache.register_registry(TestEntries)?;
|
||||
cache.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&store_path))?;
|
||||
cache.put("kept", &settings_one())?;
|
||||
let before = std::fs::read(&store_path)?;
|
||||
|
||||
let (settings, _) = cache.prepare_entry("shared", &settings_one())?;
|
||||
let (note, _) = cache.prepare_entry("shared", ¬e_one())?;
|
||||
let error = cache.put_prepared_batch(vec![settings.clone(), note.clone()]).unwrap_err();
|
||||
assert_eq!(retype_refusal(&error).record_key, "shared");
|
||||
assert_eq!(std::fs::read(&store_path)?, before);
|
||||
assert_eq!(cache.snapshot().len(), 1);
|
||||
|
||||
// The store's own batch write refuses it too, in either order.
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&store_path);
|
||||
for batch in [[settings.clone(), note.clone()], [note, settings]] {
|
||||
let error = store.push_all(&batch, PushAllOptions::default()).unwrap_err();
|
||||
assert_eq!(retype_refusal(&error).record_key, "shared");
|
||||
assert_eq!(std::fs::read(&store_path)?, before);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_stores_own_push_refuses_a_second_type_under_a_key() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let store_path = temp.path().join("push.cc");
|
||||
let mut cache = CultCache::new();
|
||||
cache.register_registry(TestEntries)?;
|
||||
let (settings, _) = cache.prepare_entry("shared", &settings_one())?;
|
||||
let (note, _) = cache.prepare_entry("shared", ¬e_one())?;
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&store_path);
|
||||
store.push(&settings)?;
|
||||
let before = std::fs::read(&store_path)?;
|
||||
let error = store.push(¬e).unwrap_err();
|
||||
let mut names = retype_refusal(&error).schema_names.clone();
|
||||
names.sort();
|
||||
assert_eq!(names, vec![Note::TYPE.to_string(), Settings::TYPE.to_string()]);
|
||||
assert_eq!(std::fs::read(&store_path)?, before);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_single_file_writer_commits_a_second_type_under_a_held_key() -> Result<()> {
|
||||
let mut cache = CultCache::new();
|
||||
cache.register_registry(TestEntries)?;
|
||||
let (held, _) = cache.prepare_entry("k", &settings_one())?;
|
||||
let (other_key, _) = cache.prepare_entry("c", &settings_one())?;
|
||||
let (note, _) = cache.prepare_entry("k", ¬e_one())?;
|
||||
let absent_note = CultCacheExpectedEnvelope { key: "k".to_string(), r#type: Note::TYPE.to_string(), current: None };
|
||||
type Writer = fn(&mut SingleFileMessagePackBackingStore, &[CultCacheEnvelope; 3], &CultCacheExpectedEnvelope) -> Result<bool>;
|
||||
let writers: Vec<(&str, Writer)> = vec![
|
||||
("push", |s, e, _| s.push(&e[2]).map(|()| true)),
|
||||
("push_all", |s, e, _| s.push_all(&[e[0].clone(), e[2].clone()], PushAllOptions::default()).map(|()| true)),
|
||||
("insert_entry_if_absent", |s, e, _| s.insert_entry_if_absent(e[2].clone())),
|
||||
("compare_and_swap_entry", |s, e, _| s.compare_and_swap_entry(&e[0], e[2].clone())),
|
||||
("compare_exchange", |s, e, x| s.compare_exchange(std::slice::from_ref(x), &[e[2].clone()])),
|
||||
("compare_and_swap_batch", |s, e, _| s.compare_and_swap_batch(&[], vec![e[2].clone()])),
|
||||
("append_if_snapshot_unchanged", |s, e, _| s.append_if_snapshot_unchanged(&[e[0].clone(), e[1].clone()], vec![e[2].clone()])),
|
||||
("replace_and_append_if_snapshot_unchanged", |s, e, _| {
|
||||
s.replace_and_append_if_snapshot_unchanged(&[e[0].clone(), e[1].clone()], vec![e[2].clone()])
|
||||
}),
|
||||
("replace_and_delete_if_snapshot_unchanged", |s, e, _| {
|
||||
s.replace_and_delete_if_snapshot_unchanged(&[e[0].clone(), e[1].clone()], vec![e[2].clone()], std::slice::from_ref(&e[1]))
|
||||
}),
|
||||
("compare_exchange_snapshot", |s, e, _| s.compare_exchange_snapshot(&[e[1].clone(), e[0].clone()], &[e[0].clone(), e[2].clone()])),
|
||||
("try_compare_exchange_snapshot", |s, e, _| {
|
||||
s.try_compare_exchange_snapshot(&[e[1].clone(), e[0].clone()], &[e[0].clone(), e[2].clone()]).map(|_| true)
|
||||
}),
|
||||
];
|
||||
let envelopes = [held.clone(), other_key.clone(), note];
|
||||
for (name, write) in writers {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.cc");
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&path);
|
||||
store.push_all(&[held.clone(), other_key.clone()], PushAllOptions::default())?;
|
||||
let before = std::fs::read(&path)?;
|
||||
let error = write(&mut store, &envelopes, &absent_note).expect_err(name);
|
||||
assert_eq!(retype_refusal(&error).record_key, "k", "{name}");
|
||||
assert_eq!(std::fs::read(&path)?, before, "{name} changed the file");
|
||||
assert!(store.pull_all().is_ok(), "{name}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_legacy_array_with_a_key_under_two_types_is_not_decodable() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let store_path = temp.path().join("legacy.cc");
|
||||
let record = |kind: &str| {
|
||||
CultCacheEnvelope {
|
||||
key: "a".to_string(),
|
||||
r#type: kind.to_string(),
|
||||
payload: rmp_serde::to_vec(&Vec::<u8>::new()).unwrap(),
|
||||
stored_at: "2026-01-01T00:00:00Z".to_string(),
|
||||
schema_id: None,
|
||||
}
|
||||
};
|
||||
std::fs::write(&store_path, rmp_serde::to_vec_named(&vec![record("old.one"), record("old.two")])?)?;
|
||||
let error = SingleFileMessagePackBackingStore::new(&store_path).pull_all().unwrap_err();
|
||||
assert_eq!(unreadable_kind(&error), Some(CultCacheStoreUnreadableKind::Undecodable), "{error:#}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
@ -6109,26 +5884,6 @@ mod tests {
|
|||
}
|
||||
|
||||
|
||||
// The writer keeps one record per key: another type under a held key is refused and the file stays; the same type replaces.
|
||||
#[test]
|
||||
fn a_write_of_another_type_under_an_existing_key_is_refused_and_the_same_type_replaces() -> Result<()> {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.msgpack");
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&path);
|
||||
let mut other = snapshot_envelope("k", b"two");
|
||||
other.r#type = "snapshot.other".to_string();
|
||||
other.schema_id = Some("snapshot.other".to_string());
|
||||
store.push(&snapshot_envelope("k", b"one"))?;
|
||||
let before = std::fs::read(&path)?;
|
||||
assert!(store.push(&other).unwrap_err().downcast_ref::<SchemaConflictError>().is_some());
|
||||
assert!(store.push_all(&[snapshot_envelope("k", b"one"), other.clone()], PushAllOptions::default()).is_err());
|
||||
assert_eq!(std::fs::read(&path)?, before);
|
||||
store.push(&snapshot_envelope("k", b"three"))?;
|
||||
let read = store.pull_all()?;
|
||||
assert_eq!((read.len(), read[0].payload.as_slice()), (1, b"three".as_slice()));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// A file is replaced by a rewrite exactly when this runtime's own reader opens it: one verdict per file, asked by open,
|
||||
// push_all and push alike. The bytes and every runtime's verdict are tests/vectors/document-variants-c2a/readability.
|
||||
// A refusal is a CultCacheStoreUnreadable, and a replaced file carries the header its old content decides.
|
||||
|
|
@ -6140,44 +5895,25 @@ mod tests {
|
|||
for line in manifest.lines().filter(|line| !line.is_empty() && !line.starts_with('#')) {
|
||||
let cells: Vec<&str> = line.split_whitespace().collect();
|
||||
let (vector, reads) = (cells[0], cells[2] == "reads");
|
||||
if !vector.starts_with("..") && !vector.starts_with('@') {
|
||||
if !vector.starts_with("..") {
|
||||
listed.push(vector.to_string());
|
||||
}
|
||||
// `@directory`: the walker puts an empty directory at the store path; it is refused and left untouched.
|
||||
let directory = vector == "@directory";
|
||||
let bytes = if directory { Vec::new() } else { std::fs::read(root.join(vector))? };
|
||||
let bytes = std::fs::read(root.join(vector))?;
|
||||
// A v3 store keeps its marker through a rewrite; every other file that reads is written v1.
|
||||
let header = if vector.contains("v3") { "cultcache.store.v3" } else { STORE_FORMAT_V1 };
|
||||
let operations: &[&str] = if directory {
|
||||
&["open", "soft_push_all", "push_all", "push", "delete"]
|
||||
} else {
|
||||
&["open", "soft_push_all", "push_all", "push"]
|
||||
};
|
||||
for &operation in operations {
|
||||
for operation in ["open", "soft_push_all", "push_all", "push"] {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.msgpack");
|
||||
if directory {
|
||||
std::fs::create_dir(&path)?;
|
||||
} else {
|
||||
std::fs::write(&path, &bytes)?;
|
||||
}
|
||||
std::fs::write(&path, &bytes)?;
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&path);
|
||||
let outcome = match operation {
|
||||
"open" => store.pull_all().map(|_| ()),
|
||||
"soft_push_all" => store.push_all(&[snapshot_envelope("x", b"one")], PushAllOptions { soft: true }),
|
||||
"push_all" => store.push_all(&[snapshot_envelope("x", b"one")], PushAllOptions::default()),
|
||||
"delete" => store.delete(&snapshot_envelope("x", b"one")),
|
||||
_ => store.push(&snapshot_envelope("x", b"one")),
|
||||
};
|
||||
let what = format!("{vector} {operation}");
|
||||
assert_eq!(outcome.is_ok(), reads, "{what}: {outcome:?}");
|
||||
if directory {
|
||||
// A directory is an I/O failure, never a typed unreadable store.
|
||||
let error = outcome.unwrap_err();
|
||||
assert!(io_error_in(&error).is_some() && unreadable_kind(&error).is_none(), "{what}: {error:#}");
|
||||
assert!(path.is_dir() && std::fs::read_dir(&path)?.next().is_none(), "{what} touched the directory");
|
||||
continue;
|
||||
}
|
||||
if operation == "soft_push_all" && reads {
|
||||
// A soft flush leaves a store it reads byte for byte.
|
||||
assert_eq!(std::fs::read(&path)?, bytes, "{what} rewrote a store it reads");
|
||||
|
|
@ -6672,93 +6408,6 @@ mod tests {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
// The retype refusal asks what C# WrittenOwns asks: does the written record own the schema id the held one is stored under.
|
||||
// A type renamed under a stable id owns it, so its write replaces the record; another id does not, and is refused.
|
||||
#[test]
|
||||
fn a_write_owning_the_stored_schema_id_replaces_the_record_under_a_new_name() -> Result<()> {
|
||||
let envelope = |r#type: &str, schema_id: &str, payload: &[u8]| CultCacheEnvelope {
|
||||
key: "a".into(),
|
||||
r#type: r#type.into(),
|
||||
payload: payload.to_vec(),
|
||||
stored_at: "2026-09-30T00:00:00Z".into(),
|
||||
schema_id: Some(schema_id.into()),
|
||||
};
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.msgpack");
|
||||
store_file(&path, &[("t.s", "t.old")], &[("a", "t.s", b"one".to_vec())])?;
|
||||
let before = std::fs::read(&path)?;
|
||||
let error = SingleFileMessagePackBackingStore::new(&path).push(&envelope("t.other", "t.other", b"x")).unwrap_err();
|
||||
assert_eq!(retype_refusal(&error).record_key, "a");
|
||||
assert_eq!(std::fs::read(&path)?, before);
|
||||
|
||||
SingleFileMessagePackBackingStore::new(&path).push(&envelope("t.s", "t.s", b"two"))?;
|
||||
let held = SingleFileMessagePackBackingStore::new(&path).pull_all()?;
|
||||
assert_eq!(held.len(), 1);
|
||||
assert_eq!((held[0].r#type.as_str(), held[0].payload.as_slice()), ("t.s", b"two".as_slice()));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// Where a held record and a written one share a schema id under different type names, the written one is the record
|
||||
// whichever name sorts first: the writers hand settle_one_record_per_key their entries sorted by type name.
|
||||
#[test]
|
||||
fn a_renamed_type_replaces_the_held_record_whether_its_name_sorts_before_or_after() -> Result<()> {
|
||||
for new_name in ["a.new", "z.new"] {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.msgpack");
|
||||
store_file(&path, &[("t.s", "t.old")], &[("a", "t.s", b"one".to_vec())])?;
|
||||
let written = CultCacheEnvelope {
|
||||
key: "a".into(),
|
||||
r#type: new_name.into(),
|
||||
payload: b"two".to_vec(),
|
||||
stored_at: "2026-09-30T00:00:00Z".into(),
|
||||
schema_id: Some("t.s".into()),
|
||||
};
|
||||
SingleFileMessagePackBackingStore::new(&path).push(&written)?;
|
||||
let held = SingleFileMessagePackBackingStore::new(&path).pull_all()?;
|
||||
assert_eq!(held.len(), 1, "{new_name}");
|
||||
assert_eq!(held[0].payload, b"two".to_vec(), "{new_name}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// In one push_all, entries under a key and a schema id with different type names resolve by the order given, not by
|
||||
// the order of the names.
|
||||
#[test]
|
||||
fn push_all_resolves_same_id_entries_with_different_names_by_the_order_given() -> Result<()> {
|
||||
let envelope = |r#type: &str, payload: &[u8]| CultCacheEnvelope {
|
||||
key: "a".into(),
|
||||
r#type: r#type.into(),
|
||||
payload: payload.to_vec(),
|
||||
stored_at: "2026-09-30T00:00:00Z".into(),
|
||||
schema_id: Some("t.s".into()),
|
||||
};
|
||||
for (first, second) in [("a.name", "b.name"), ("b.name", "a.name")] {
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.msgpack");
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&path);
|
||||
store.push_all(&[envelope(first, b"first"), envelope(second, b"second")], PushAllOptions::default())?;
|
||||
let held = store.pull_all()?;
|
||||
assert_eq!(held.len(), 1, "{first} then {second}");
|
||||
assert_eq!(held[0].payload, b"second".to_vec(), "{first} then {second}");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// Of same-type entries sharing a key in one push_all the last one given is the record.
|
||||
#[test]
|
||||
fn push_all_keeps_the_last_of_two_same_type_entries_under_a_key() -> Result<()> {
|
||||
let mut cache = CultCache::new();
|
||||
cache.register_registry(TestEntries)?;
|
||||
let (first, _) = cache.prepare_entry("k", &settings_one())?;
|
||||
let (last, _) = cache.prepare_entry("k", &Settings { theme: "red".to_string(), retries: 2 })?;
|
||||
let temp = tempfile::tempdir()?;
|
||||
let path = temp.path().join("store.cc");
|
||||
let mut store = SingleFileMessagePackBackingStore::new(&path);
|
||||
store.push_all(&[first, last.clone()], PushAllOptions::default())?;
|
||||
assert_eq!(store.pull_all()?.into_iter().map(|e| e.payload).collect::<Vec<_>>(), vec![last.payload]);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "tests.stable", schema = "tests.old_name")]
|
||||
struct StableId {
|
||||
|
|
|
|||
|
|
@ -80,50 +80,6 @@ struct VBool {
|
|||
v: bool,
|
||||
}
|
||||
|
||||
/// The refusals vector's other schemas: `Sku` is unique in each; `Alt` is a second unique index in `uq_pair` (alias
|
||||
/// `Alt`) and `uq_alias` (alias `alt`, sorting after `Sku` only under an ordinal compare); alpha and beta are plain.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "uq_pair", schema = "tests.uq_pair")]
|
||||
struct UqPair {
|
||||
#[cultcache(key = 0, index = "Sku")]
|
||||
sku: String,
|
||||
#[cultcache(key = 1, index = "Alt")]
|
||||
alt: String,
|
||||
}
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "uq_alias", schema = "tests.uq_alias")]
|
||||
struct UqAlias {
|
||||
#[cultcache(key = 0, index = "Sku")]
|
||||
sku: String,
|
||||
#[cultcache(key = 1, index = "alt")]
|
||||
alt: String,
|
||||
}
|
||||
/// Aliases U+FF21 and U+10400: UTF-16 puts the astral alias first (D801 DC00 < FF21), code-point order the other way.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "uq_wide", schema = "tests.uq_wide")]
|
||||
struct UqWide {
|
||||
#[cultcache(key = 0, index = "\u{FF21}")]
|
||||
sku: String,
|
||||
#[cultcache(key = 1, index = "\u{10400}")]
|
||||
alt: String,
|
||||
}
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "uq_alpha", schema = "tests.uq_alpha")]
|
||||
struct UqAlpha {
|
||||
#[cultcache(key = 0, index = "Sku")]
|
||||
sku: String,
|
||||
#[cultcache(key = 1)]
|
||||
second: String,
|
||||
}
|
||||
#[derive(Clone, Debug, PartialEq, Eq, DatabaseEntry)]
|
||||
#[cultcache(type = "uq_beta", schema = "tests.uq_beta")]
|
||||
struct UqBeta {
|
||||
#[cultcache(key = 0, index = "Sku")]
|
||||
sku: String,
|
||||
#[cultcache(key = 1)]
|
||||
second: String,
|
||||
}
|
||||
|
||||
fn unique(sku: &str) -> Unique {
|
||||
Unique { sku: sku.into(), category: "c".into() }
|
||||
}
|
||||
|
|
@ -383,6 +339,35 @@ fn a_batch_is_judged_whole() {
|
|||
assert_eq!(cache.get_by_index::<Unique>("Sku", "D1").unwrap(), cache.get::<Unique>("e").unwrap());
|
||||
}
|
||||
|
||||
// The verdict and the named pair do not depend on key order, staging order, or alias declaration order.
|
||||
#[test]
|
||||
fn the_refusal_names_the_ordinal_pair_whatever_the_order() {
|
||||
let orders: [[&str; 3]; 6] =
|
||||
[["a", "b", "c"], ["a", "c", "b"], ["b", "a", "c"], ["b", "c", "a"], ["c", "a", "b"], ["c", "b", "a"]];
|
||||
for order in orders {
|
||||
let mut cache = memory_cache();
|
||||
let rows: Vec<_> = order.iter().map(|key| (*key, unique("S"))).collect();
|
||||
let text = error_text(cache.put_prepared_batch(staged(&cache, &rows)));
|
||||
assert_eq!(text, refusal("a", "b", "tests.unique_item", "Sku"), "{order:?}");
|
||||
assert!(skus(&cache).is_empty());
|
||||
}
|
||||
// Both aliases collide in one batch: the alias sorting first is named, though `B` is declared first.
|
||||
for order in [["x", "y"], ["y", "x"]] {
|
||||
let mut cache = memory_cache();
|
||||
let rows: Vec<_> = order.iter().map(|key| (*key, multi(Some("s"), Some("t")))).collect();
|
||||
let text = error_text(cache.put_prepared_batch(staged(&cache, &rows)));
|
||||
assert_eq!(text, refusal("x", "y", "Multi", "A"));
|
||||
}
|
||||
// A held key and a landing key: the pair is ordinal, not held-first.
|
||||
let mut cache = memory_cache();
|
||||
cache.put("m", &unique("S")).unwrap();
|
||||
assert_eq!(error_text(cache.put("z", &unique("S"))), refusal("m", "z", "tests.unique_item", "Sku"));
|
||||
assert_eq!(error_text(cache.put("a", &unique("S"))), refusal("a", "m", "tests.unique_item", "Sku"));
|
||||
// A value that differs only by case or by a surrounding space is another value.
|
||||
cache.put("n", &unique("s")).unwrap();
|
||||
cache.put("o", &unique("S ")).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_load_of_a_duplicate_image_is_refused_and_keeps_the_prior_view() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
|
|
@ -576,97 +561,6 @@ fn vectors_stores_hold_their_verdicts() {
|
|||
}
|
||||
}
|
||||
|
||||
/// One op of a refusals row: `["open", file-or-nil]`, `["pull", file]` or `["commit", [[key, schema, sku, second], ...]]`.
|
||||
#[derive(Deserialize)]
|
||||
#[serde(untagged)]
|
||||
enum Arg {
|
||||
Records(Vec<(String, String, String, String)>),
|
||||
File(Option<String>),
|
||||
}
|
||||
|
||||
type RefusalRow = (String, Vec<(String, Arg)>, Option<(String, String, String, String)>);
|
||||
|
||||
/// Rows `[label, ops, refusal]`: the pair a refusal names is a function of the state the admission would leave
|
||||
/// (`tests/vectors/unique-index/README.md`, RULE), whatever the staging, file or pull order, key case or alias case.
|
||||
#[test]
|
||||
fn vectors_refusals_hold_the_rule() {
|
||||
let bytes = std::fs::read(vectors().join("refusals.msgpack")).unwrap();
|
||||
let rows: Vec<RefusalRow> = rmp_serde::from_slice(&bytes).unwrap();
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let mut walked = 0;
|
||||
for (index, (label, ops, refused)) in rows.into_iter().enumerate() {
|
||||
let path = dir.path().join(format!("row-{index}.cc"));
|
||||
let mut cache: Option<CultCache> = None;
|
||||
let mut failure = None;
|
||||
for (at, (kind, arg)) in ops.iter().enumerate() {
|
||||
let last = at + 1 == ops.len();
|
||||
let mut before = path.exists().then(|| std::fs::read(&path).unwrap());
|
||||
let result = match (kind.as_str(), arg) {
|
||||
("open", Arg::File(file)) => {
|
||||
if let Some(file) = file {
|
||||
std::fs::copy(vectors().join(file), &path).unwrap();
|
||||
}
|
||||
let mut opened = CultCache::new();
|
||||
opened.register_entry_type::<Unique>().unwrap();
|
||||
opened.register_entry_type::<UqPair>().unwrap();
|
||||
opened.register_entry_type::<UqAlias>().unwrap();
|
||||
opened.register_entry_type::<UqWide>().unwrap();
|
||||
opened.register_entry_type::<UqAlpha>().unwrap();
|
||||
opened.register_entry_type::<UqBeta>().unwrap();
|
||||
opened.add_generic_backing_store(SingleFileMessagePackBackingStore::new(&path)).unwrap();
|
||||
let pulled = opened.pull_all_backing_stores();
|
||||
cache = Some(opened);
|
||||
pulled
|
||||
}
|
||||
("pull", Arg::File(Some(file))) => {
|
||||
std::fs::copy(vectors().join(file), &path).unwrap();
|
||||
before = Some(std::fs::read(&path).unwrap());
|
||||
cache.as_mut().unwrap().pull_all_backing_stores()
|
||||
}
|
||||
("commit", Arg::Records(records)) => {
|
||||
let cache = cache.as_mut().unwrap();
|
||||
let staged: Vec<_> = records
|
||||
.iter()
|
||||
.map(|(key, schema, sku, second)| {
|
||||
let (sku, second) = (sku.clone(), second.clone());
|
||||
match schema.as_str() {
|
||||
"tests.unique_item" => cache.prepare_entry(key, &Unique { sku, category: second }).map(|(envelope, _)| envelope),
|
||||
"tests.uq_pair" => cache.prepare_entry(key, &UqPair { sku, alt: second }).map(|(envelope, _)| envelope),
|
||||
"tests.uq_alias" => cache.prepare_entry(key, &UqAlias { sku, alt: second }).map(|(envelope, _)| envelope),
|
||||
"tests.uq_wide" => cache.prepare_entry(key, &UqWide { sku, alt: second }).map(|(envelope, _)| envelope),
|
||||
"tests.uq_alpha" => cache.prepare_entry(key, &UqAlpha { sku, second }).map(|(envelope, _)| envelope),
|
||||
"tests.uq_beta" => cache.prepare_entry(key, &UqBeta { sku, second }).map(|(envelope, _)| envelope),
|
||||
other => panic!("{label}: unexpected schema {other}"),
|
||||
}
|
||||
.unwrap()
|
||||
})
|
||||
.collect();
|
||||
cache.put_prepared_batch(staged)
|
||||
}
|
||||
other => panic!("{label}: unexpected op {:?}", other.0),
|
||||
};
|
||||
if last {
|
||||
failure = Some((result, before));
|
||||
} else {
|
||||
result.unwrap_or_else(|error| panic!("{label}: op {at} should succeed: {error:#}"));
|
||||
}
|
||||
}
|
||||
let (result, before) = failure.unwrap();
|
||||
match refused {
|
||||
None => result.unwrap_or_else(|error| panic!("{label}: should succeed: {error:#}")),
|
||||
Some((k1, k2, schema, alias)) => {
|
||||
let error = result.err().unwrap_or_else(|| panic!("{label}: should refuse"));
|
||||
assert_eq!(format!("{error:#}"), refusal(&k1, &k2, &schema, &alias), "{label}");
|
||||
if let Some(before) = before {
|
||||
assert_eq!(std::fs::read(&path).unwrap(), before, "{label}: the file is byte-identical");
|
||||
}
|
||||
}
|
||||
}
|
||||
walked += 1;
|
||||
}
|
||||
assert_eq!(walked, 37);
|
||||
}
|
||||
|
||||
// A pull replaces the holder table with the stored image's: a value another writer changed is released, not kept.
|
||||
#[test]
|
||||
fn a_second_pull_replaces_the_holders_with_the_stored_image() {
|
||||
|
|
|
|||
|
|
@ -14,10 +14,6 @@ All notable changes to `@gamecult/cultcache-ts` are recorded here, following
|
|||
an empty string is no longer a value. `registerIndex` over held duplicates installs nothing. A new `roles` map names
|
||||
the many-row selection aliases (never indexed for lookup, same catalog `indexAlias`). The per-key migration rule is
|
||||
in the README; classify every key before the CultLib your tools load is updated. The refusal verdict is the contract, its text is informational.
|
||||
- The single-file reader (v1 and the legacy envelope array) refuses a store that holds two records under one key as `StoreUnreadableError`; a store that opened before may now be refused. Repair it by removing the extra record with the runtime that wrote it.
|
||||
- A `put`, `push` or `pushAll` that would land a second type under a held key (against the file, or within one batch) is refused with `SchemaConflictError`, and nothing is written. The written type may replace the record when its catalog entry carries the stored schema id as its own or lists it compatible (a type renamed under a stable id restamps the record, as C# does); otherwise retyping a key is a delete, then a put. The C# allowance for an undeclared older id of the same schema name that reads whole is not ported: a same-named type keeps passing, including one that drops persisted slots, which C# refuses.
|
||||
- Same-type writes replace by key; in one `pushAll` batch the last entry for a key wins.
|
||||
- A store whose canonical-schema catalog slot is not a string is refused as `StoreUnreadableError`.
|
||||
|
||||
### Changed
|
||||
|
||||
|
|
|
|||
|
|
@ -47,7 +47,6 @@ type EntryLookups = {
|
|||
};
|
||||
|
||||
type HydratedEntry = CultCacheEnvelope & {
|
||||
catalogEntry: NonNullable<CultCacheEnvelope["catalogEntry"]>;
|
||||
value: unknown;
|
||||
lookups: EntryLookups;
|
||||
};
|
||||
|
|
@ -317,16 +316,6 @@ export class CultCache {
|
|||
}
|
||||
}
|
||||
|
||||
// Judged whole, once: the refusal names the least violation of the state this load would leave, whatever the
|
||||
// order of the stores and of the entries in them.
|
||||
refuseLeastIndexSharing(function* () {
|
||||
for (const entry of next.entries.values()) {
|
||||
for (const [alias, value] of entry.lookups.indexes) {
|
||||
yield { type: entry.type, schemaName: entry.catalogEntry.schemaName, alias, value, key: entry.key };
|
||||
}
|
||||
}
|
||||
}());
|
||||
|
||||
this.#hydrated = next;
|
||||
});
|
||||
}
|
||||
|
|
@ -663,7 +652,6 @@ export class CultCache {
|
|||
async #writeNow(registered: RegisteredDefinition, entry: CultCacheEnvelope, value: unknown): Promise<void> {
|
||||
const home = this.#homeStore(entry.type);
|
||||
const hydrated = this.#admitEntry(this.#hydrated, registered, entry, value);
|
||||
this.#refuseIndexSharing(this.#hydrated, entry.type, registered.catalogEntry.schemaName, entry.key, hydrated.lookups);
|
||||
const legacy = this.#hydrated.legacyGlobals.get(entry.type);
|
||||
await home?.push(entry);
|
||||
if (home && legacy) {
|
||||
|
|
@ -915,6 +903,7 @@ export class CultCache {
|
|||
}
|
||||
|
||||
const lookups = this.#lookupsFor(registered.nameAccessor, registered.indexAccessors, value);
|
||||
this.#refuseIndexSharing(state, entry.type, registered.catalogEntry.schemaName, entry.key, lookups);
|
||||
return {
|
||||
...entry,
|
||||
payload: this.#cloneBytes(entry.payload),
|
||||
|
|
@ -1006,6 +995,7 @@ export class CultCache {
|
|||
const entryLookups: Array<[HydratedEntry, EntryLookups]> = [];
|
||||
const nameMap = nameAccessor ? new Map<string, string>() : undefined;
|
||||
const indexMaps = new Map<string, Map<string, string>>();
|
||||
const sharing = new Map<string, string[]>();
|
||||
for (const indexName of indexAccessors.keys()) {
|
||||
indexMaps.set(indexName, new Map());
|
||||
}
|
||||
|
|
@ -1018,16 +1008,21 @@ export class CultCache {
|
|||
}
|
||||
for (const [indexName, indexValue] of lookups.indexes) {
|
||||
const map = indexMaps.get(indexName)!;
|
||||
if (!map.has(indexValue)) map.set(indexValue, entry.key);
|
||||
const holder = map.get(indexValue);
|
||||
if (holder === undefined) {
|
||||
map.set(indexValue, entry.key);
|
||||
} else {
|
||||
sharing.set(`${indexName}\u0000${indexValue}`, [...(sharing.get(`${indexName}\u0000${indexValue}`) ?? [holder]), entry.key]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Judged whole, by the same judge as a load: the least violation of the state the registration would leave.
|
||||
refuseLeastIndexSharing(function* () {
|
||||
for (const [entry, lookups] of entryLookups) {
|
||||
for (const [alias, value] of lookups.indexes) yield { type, schemaName, alias, value, key: entry.key };
|
||||
}
|
||||
}());
|
||||
// Judged whole: any value held by two records refuses, naming the two smallest of its keys. Held records already
|
||||
// passed the judge for every alias installed before, so only the alias being registered can be shared here.
|
||||
for (const [id, keys] of sharing) {
|
||||
keys.sort(compareOrdinal);
|
||||
throw indexSharingRefusal(keys[0], keys[1], schemaName, id.slice(0, id.indexOf("\u0000")));
|
||||
}
|
||||
|
||||
return { entryLookups, nameMap, indexMaps };
|
||||
}
|
||||
|
|
@ -1104,34 +1099,6 @@ function compareOrdinal(a: string, b: string): number {
|
|||
return a < b ? -1 : a > b ? 1 : 0;
|
||||
}
|
||||
|
||||
// The whole-state judge (variants:question:unique-index-refusal-pair-rule): over every (type, alias, value) held at two
|
||||
// or more distinct keys, throws for the one least by (alias, least key, second-least key, schema name), each compared by
|
||||
// UTF-16 code unit. A function of the state, never of the order the rows arrive in.
|
||||
function refuseLeastIndexSharing(rows: Iterable<{ type: string; schemaName: string; alias: string; value: string; key: string }>): void {
|
||||
const groups = new Map<string, { schemaName: string; alias: string; keys: string[] }>();
|
||||
for (const { type, schemaName, alias, value, key } of rows) {
|
||||
const id = JSON.stringify([type, alias, value]);
|
||||
const group = groups.get(id);
|
||||
if (group) group.keys.push(key);
|
||||
else groups.set(id, { schemaName, alias, keys: [key] });
|
||||
}
|
||||
let least: { schemaName: string; alias: string; first: string; second: string } | undefined;
|
||||
for (const { schemaName, alias, keys } of groups.values()) {
|
||||
if (keys.length < 2) continue;
|
||||
keys.sort(compareOrdinal);
|
||||
const candidate = { schemaName, alias, first: keys[0], second: keys[1] };
|
||||
if (!least || compareLeast(candidate, least) < 0) least = candidate;
|
||||
}
|
||||
if (least) throw indexSharingRefusal(least.first, least.second, least.schemaName, least.alias);
|
||||
}
|
||||
|
||||
function compareLeast(
|
||||
a: { schemaName: string; alias: string; first: string; second: string },
|
||||
b: { schemaName: string; alias: string; first: string; second: string },
|
||||
): number {
|
||||
return compareOrdinal(a.alias, b.alias) || compareOrdinal(a.first, b.first) || compareOrdinal(a.second, b.second) || compareOrdinal(a.schemaName, b.schemaName);
|
||||
}
|
||||
|
||||
function indexSharingRefusal(keyA: string, keyB: string, schemaName: string, alias: string): Error {
|
||||
const [first, second] = compareOrdinal(keyA, keyB) <= 0 ? [keyA, keyB] : [keyB, keyA];
|
||||
return new Error(`Records ${first} and ${second} (${schemaName}) both hold unique index '${alias}'; an index value names one record.`);
|
||||
|
|
|
|||
|
|
@ -60,8 +60,9 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
|
|||
const disk = this.#readDisk();
|
||||
this.#format = disk.format;
|
||||
const existing = disk.envelopes;
|
||||
requireOneTypePerKey(this.filePath, [entry], existing);
|
||||
const filtered = existing.filter((candidate) => candidate.key !== entry.key);
|
||||
const filtered = existing.filter(
|
||||
(candidate) => !(candidate.type === entry.type && candidate.key === entry.key),
|
||||
);
|
||||
await this.#writeAll([entry], filtered);
|
||||
});
|
||||
}
|
||||
|
|
@ -102,9 +103,7 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
|
|||
this.#format = STORE_FORMAT_VERSION;
|
||||
}
|
||||
|
||||
requireOneTypePerKey(this.filePath, entries, []);
|
||||
// A key is held once: of same-type entries sharing a key in one batch, the last is kept.
|
||||
await this.#writeAll([...new Map(entries.map((entry) => [entry.key, entry])).values()], []);
|
||||
await this.#writeAll(entries, []);
|
||||
});
|
||||
}
|
||||
|
||||
|
|
@ -138,36 +137,6 @@ export class SingleFileMessagePackBackingStore implements CacheBackingStore {
|
|||
}
|
||||
}
|
||||
|
||||
// As C# WrittenOwns' declared half: the written type owns the stored schema id when its catalog entry carries that id as its own
|
||||
// or lists it compatible, so a type renamed under a stable id restamps the record it already holds. (The C# half that lets an
|
||||
// undeclared older id of the same schema name through when it reads whole is not ported; a same-named type keeps passing.)
|
||||
function writtenOwns(written: CultCacheEnvelope, storedId: string): boolean {
|
||||
const entry = written.catalogEntry;
|
||||
return entry !== undefined && (entry.schemaId === storedId || (entry.compatibleSchemaIds ?? []).includes(storedId));
|
||||
}
|
||||
|
||||
// A store holds a key once, and the reader refuses a file that holds it twice, so a write that would put a second type under a
|
||||
// key (against a record on disk, or within its own batch) is refused before anything is written. Retyping a record is a delete
|
||||
// of the old type followed by a put of the new one.
|
||||
function requireOneTypePerKey(path: string, supplied: CultCacheEnvelope[], held: CultCacheEnvelope[]): void {
|
||||
const typeByKey = new Map<string, CultCacheEnvelope>();
|
||||
for (const entry of held) {
|
||||
typeByKey.set(entry.key, entry);
|
||||
}
|
||||
for (const entry of supplied) {
|
||||
const prior = typeByKey.get(entry.key);
|
||||
if (prior !== undefined && prior.type !== entry.type && !writtenOwns(entry, schemaIdForEnvelope(prior))) {
|
||||
throw new SchemaConflictError(
|
||||
schemaIdForEnvelope(entry),
|
||||
[prior.type, entry.type],
|
||||
entry.key,
|
||||
`CultCache store ${path} holds record "${entry.key}" as type "${prior.type}", so type "${entry.type}" cannot be written under the same key; remove the record first.`,
|
||||
);
|
||||
}
|
||||
typeByKey.set(entry.key, entry);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The one reader of a CultCache single-file store, for this package's store and for any consumer that must read a
|
||||
* store file (an authority bundle, say) rather than decode it itself. A path with nothing at it is an absent, empty,
|
||||
|
|
@ -235,9 +204,8 @@ type DecodedSnapshot = {
|
|||
};
|
||||
|
||||
function encodeSnapshot(supplied: CultCacheEnvelope[], arrived: CultCacheEnvelope[], format: StoreFormat): unknown[] {
|
||||
// A key is held once: a supplied record replaces the arrived record under its key whatever the type.
|
||||
const replaced = new Set(supplied.map((entry) => entry.key));
|
||||
const kept = arrived.filter((entry) => !replaced.has(entry.key));
|
||||
const replaced = new Set(supplied.map((entry) => `${entry.type}::${entry.key}`));
|
||||
const kept = arrived.filter((entry) => !replaced.has(`${entry.type}::${entry.key}`));
|
||||
const entries = [...kept, ...supplied];
|
||||
const catalog = catalogEntriesFor([
|
||||
...kept.map((envelope) => ({ envelope, registered: false })),
|
||||
|
|
@ -448,7 +416,6 @@ function decodeStoreFile(data: Uint8Array): Omit<SingleFileStoreRead, "absent">
|
|||
throw new Error("CultCache file is not a recognized CultCache MessagePack store.");
|
||||
}
|
||||
|
||||
requireUniqueKeys(legacy);
|
||||
const normalized = legacy.map((entry) => ({ ...entry, payload: normalizePayload(entry.payload) }));
|
||||
return { format: STORE_FORMAT_VERSION, envelopes: envelopeArraySchema.parse(normalized).map((entry) => ({ ...entry, payload: documentPayloadFromBytes(entry.payload) })) as CultCacheEnvelope[] };
|
||||
}
|
||||
|
|
@ -482,21 +449,9 @@ function decodeSnapshot(decoded: unknown): DecodedSnapshot | undefined {
|
|||
}
|
||||
|
||||
const records = recordsRaw.map(decodeRecord);
|
||||
requireUniqueKeys(records);
|
||||
return { format: decoded[0], catalogBySchemaId, records };
|
||||
}
|
||||
|
||||
// A store holds a key once, whatever the type: a repeat is refused in a v1 snapshot and in a legacy envelope array alike.
|
||||
function requireUniqueKeys(records: Array<{ key: string }>): void {
|
||||
const seen = new Set<string>();
|
||||
for (const { key } of records) {
|
||||
if (seen.has(key)) {
|
||||
throw new Error(`CultCache record "${key}" appears more than once.`);
|
||||
}
|
||||
seen.add(key);
|
||||
}
|
||||
}
|
||||
|
||||
function decodeCatalogEntry(value: unknown): CultCacheSchemaCatalogEntry {
|
||||
if (!Array.isArray(value)) {
|
||||
throw new Error("CultCache schema catalog entries must be MessagePack arrays.");
|
||||
|
|
@ -524,16 +479,12 @@ function decodeCatalogEntry(value: unknown): CultCacheSchemaCatalogEntry {
|
|||
throw new Error(`CultCache schema catalog entry "${schemaId}" has invalid members.`);
|
||||
}
|
||||
|
||||
if (typeof canonicalSchemaJson !== "string") {
|
||||
throw new Error(`CultCache schema catalog entry "${schemaId}" has an invalid canonical schema.`);
|
||||
}
|
||||
|
||||
return {
|
||||
schemaId,
|
||||
schemaName,
|
||||
schemaVersion: isNonEmptyString(schemaVersion) ? schemaVersion : `${schemaName}.v1`,
|
||||
contentHash: isNonEmptyString(contentHash) ? contentHash : schemaId,
|
||||
canonicalSchemaJson,
|
||||
canonicalSchemaJson: typeof canonicalSchemaJson === "string" ? canonicalSchemaJson : "",
|
||||
compatibleSchemaIds,
|
||||
members: members.map(decodeCatalogMember),
|
||||
};
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ import { z } from "zod";
|
|||
import { CultCache } from "../src/cult-cache";
|
||||
import { inspectCultCacheBytes } from "../src/cult-cache-inspector";
|
||||
import { defineDocumentRegistry, defineDocumentType, documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
|
||||
import { SingleFileMessagePackBackingStore, readSingleFileStore, readStoreBytes } from "../src/single-file-messagepack-backing-store";
|
||||
import { SingleFileMessagePackBackingStore, readStoreBytes } from "../src/single-file-messagepack-backing-store";
|
||||
import { SchemaConflictError, StoreUnreadableError, refuseUnreadStoreBytes } from "../src/store-format";
|
||||
import type { AnyCultCacheDocumentDefinition, CacheBackingStore, CultCacheEnvelope, CultCacheSchema } from "../src/types";
|
||||
|
||||
|
|
@ -143,63 +143,6 @@ test("CultCache supports registry bootstrap, global documents, and lookup by nam
|
|||
});
|
||||
});
|
||||
|
||||
test("a put that would land a second type under a persisted key is refused, the cache and the file are unchanged, and the store reloads", async () => {
|
||||
const alpha = defineDocumentType({ type: "alpha", schema: z.object({ v: z.number() }) });
|
||||
const beta = defineDocumentType({ type: "beta", schema: z.object({ v: z.number() }) });
|
||||
const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "cache.msgpack");
|
||||
const registry = defineDocumentRegistry(alpha, beta);
|
||||
const open = () => CultCache.builder().withRegistry(registry).withGenericStore(new SingleFileMessagePackBackingStore(storePath)).build();
|
||||
|
||||
const cache = open();
|
||||
await cache.put(alpha, "shared", { v: 1 });
|
||||
const before = [...(await readFile(storePath))];
|
||||
await assert.rejects(() => cache.put(beta, "shared", { v: 2 }), SchemaConflictError);
|
||||
assert.deepEqual([...(await readFile(storePath))], before);
|
||||
assert.deepEqual(cache.get(alpha, "shared"), { v: 1 });
|
||||
assert.equal(cache.get(beta, "shared"), undefined);
|
||||
|
||||
const reloaded = open();
|
||||
await reloaded.pullAllBackingStores();
|
||||
assert.deepEqual(reloaded.get(alpha, "shared"), { v: 1 });
|
||||
|
||||
assert.equal(await cache.delete(alpha, "shared"), true);
|
||||
await cache.put(beta, "shared", { v: 2 });
|
||||
const retyped = open();
|
||||
await retyped.pullAllBackingStores();
|
||||
assert.equal(retyped.get(alpha, "shared"), undefined);
|
||||
assert.deepEqual(retyped.get(beta, "shared"), { v: 2 });
|
||||
});
|
||||
|
||||
test("a put by a type renamed under the stored id, which lists that id compatible, restamps the record; an unrelated type is refused", async () => {
|
||||
const old = defineDocumentType({ type: "ren.old", schema: z.object({ v: z.number() }), schemaId: "ren.id.v1", schemaName: "ren.old" });
|
||||
const renamed = defineDocumentType({
|
||||
type: "ren.new",
|
||||
schema: z.object({ v: z.number() }),
|
||||
schemaId: "ren.id.v2",
|
||||
schemaName: "ren.new",
|
||||
compatibleSchemaIds: ["ren.id.v2", "ren.id.v1"],
|
||||
});
|
||||
const other = defineDocumentType({ type: "ren.other", schema: z.object({ v: z.number() }), schemaId: "ren.id.other", schemaName: "ren.other" });
|
||||
const storePath = join(await mkdtemp(join(tmpdir(), "cultcache-ts-")), "cache.msgpack");
|
||||
const open = (...types: (typeof old)[]) =>
|
||||
types
|
||||
.reduce((builder, type) => builder.withDocumentType(type), CultCache.builder())
|
||||
.withGenericStore(new SingleFileMessagePackBackingStore(storePath))
|
||||
.build();
|
||||
|
||||
await open(old).put(old, "k", { v: 1 });
|
||||
const cache = open(renamed, other as unknown as typeof old);
|
||||
await cache.pullAllBackingStores();
|
||||
assert.deepEqual(cache.get(renamed, "k"), { v: 1 });
|
||||
await assert.rejects(() => cache.put(other, "k", { v: 9 }), SchemaConflictError);
|
||||
await cache.put(renamed, "k", { v: 2 });
|
||||
|
||||
const reloaded = open(renamed);
|
||||
await reloaded.pullAllBackingStores();
|
||||
assert.deepEqual(reloaded.get(renamed, "k"), { v: 2 });
|
||||
assert.equal(readSingleFileStore(storePath).envelopes.filter((envelope) => envelope.key === "k").length, 1);
|
||||
});
|
||||
|
||||
test("CultCache can register name and index lookups after entries already exist", async () => {
|
||||
const noteDocument = defineDocumentType({
|
||||
type: "note",
|
||||
|
|
@ -2582,24 +2525,17 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
|
|||
.filter((line) => line.length > 0 && !line.startsWith("#"))
|
||||
.map((line) => line.split(/\s+/u));
|
||||
// Every vector in the folder has a manifest row.
|
||||
const listed = rows.map(([vector]) => vector!).filter((vector) => !vector.startsWith("..") && !vector.startsWith("@")).sort();
|
||||
const listed = rows.map(([vector]) => vector!).filter((vector) => !vector.startsWith("..")).sort();
|
||||
assert.deepEqual((await readdir(root)).filter((name) => name.endsWith(".bin")).sort(), listed);
|
||||
const dir = await mkdtemp(join(tmpdir(), "cultcache-readability-"));
|
||||
const seed = await copyVector(dir, join(c2aVectors, "v3-base.msgpack"), "seed.msgpack");
|
||||
const envelopes = await new SingleFileMessagePackBackingStore(seed).pullAll();
|
||||
for (const [vector, , , typescript] of rows) {
|
||||
const reads = typescript === "reads";
|
||||
// "@directory": the walker makes an empty directory at the store path; the runtime refuses to open or replace it and leaves it be.
|
||||
const isDirectory = vector === "@directory";
|
||||
const bytes = isDirectory ? Buffer.alloc(0) : await readFile(join(root, vector));
|
||||
const bytes = await readFile(join(root, vector));
|
||||
for (const operation of ["pullAll", "softPushAll", "pushAll", "push", "delete"] as const) {
|
||||
const file = join(dir, `${vector.replace(/[^a-z0-9.-]/giu, "_")}.${operation}.msgpack`);
|
||||
if (isDirectory) {
|
||||
assert.equal(typescript, "refuses", "a directory is never a store");
|
||||
await mkdir(file);
|
||||
} else {
|
||||
await writeFile(file, bytes);
|
||||
}
|
||||
await writeFile(file, bytes);
|
||||
const store = new SingleFileMessagePackBackingStore(file);
|
||||
const run = () =>
|
||||
operation === "pullAll"
|
||||
|
|
@ -2634,10 +2570,6 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
|
|||
} else {
|
||||
assert.ok(!keys.includes(envelopes[0]!.key), `${vector} ${operation}`);
|
||||
}
|
||||
} else if (isDirectory) {
|
||||
await assert.rejects(run, (error) => !(error instanceof StoreUnreadableError), `${vector} ${operation}: an I/O failure, not a typed refusal`);
|
||||
assert.ok((await lstat(file)).isDirectory(), `${vector} ${operation} replaced the directory`);
|
||||
assert.deepEqual(await readdir(file), [], `${vector} ${operation} wrote into the directory`);
|
||||
} else {
|
||||
await assert.rejects(
|
||||
run,
|
||||
|
|
@ -2650,25 +2582,6 @@ test("SingleFileMessagePackBackingStore replaces a file exactly when it reads",
|
|||
}
|
||||
});
|
||||
|
||||
// One key is held once: a push whose type does not own the stored schema id is refused, so the writer cannot emit a store its own
|
||||
// reader refuses; one whose type owns it (its own id, or one it lists compatible, as C# WrittenOwns) restamps the record in place.
|
||||
test("SingleFileMessagePackBackingStore push asks whether the written type owns the stored id, not what the types are called", async () => {
|
||||
const seed = CultCache.builder().withDocumentType(ownsId).withDocumentType(listsId).build();
|
||||
await seed.put(ownsId, "k", { name: "owner" });
|
||||
await seed.put(listsId, "k", { name: "lister" });
|
||||
const file = join(await mkdtemp(join(tmpdir(), "cultcache-push-key-")), "store.msgpack");
|
||||
const store = new SingleFileMessagePackBackingStore(file);
|
||||
const idsOf = async () => ((decode(await readFile(file)) as unknown[][])[2] as unknown[][]).map((record) => [record[0], record[1]]);
|
||||
|
||||
await store.push(seed.getRequiredEnvelope(listsId, "k"));
|
||||
await assert.rejects(() => store.push(seed.getRequiredEnvelope(ownsId, "k")), SchemaConflictError);
|
||||
assert.deepEqual(await idsOf(), [["k", "id.lister"]]);
|
||||
|
||||
await store.pushAll([seed.getRequiredEnvelope(listsId, "k")]);
|
||||
await store.push(seed.getRequiredEnvelope(listsId, "k"));
|
||||
assert.equal((await store.pullAll()).length, 1);
|
||||
});
|
||||
|
||||
// Owner beats lister: a definition that owns an id and one that lists it as compatible register together in either order. A record
|
||||
// under the id is the owner's, and a lister's only when nothing owns it; two listers and no owner name no single definition.
|
||||
const ownsId = defineDocumentType({ type: "tests.owns", schema: z.object({ name: z.string() }), schemaId: "id.owned", schemaName: "tests.owns" });
|
||||
|
|
|
|||
|
|
@ -12,7 +12,6 @@ import { decode, encode } from "@msgpack/msgpack";
|
|||
|
||||
import { documentPayloadBytes, documentPayloadFromBytes } from "../src/document";
|
||||
import { SingleFileMessagePackBackingStore } from "../src/single-file-messagepack-backing-store";
|
||||
import { SchemaConflictError, StoreUnreadableError } from "../src/store-format";
|
||||
import type { CultCacheEnvelope } from "../src/types";
|
||||
|
||||
import {
|
||||
|
|
@ -572,84 +571,15 @@ const snapshotOf = async (store: SingleFileMessagePackBackingStore): Promise<str
|
|||
.map((entry) => `${entry.type}/${entry.key}:${[...documentPayloadBytes(entry.payload)].at(-1)}`)
|
||||
.sort();
|
||||
|
||||
test("push replaces the record under the same key and type and keeps every other; retyping is delete then push", async () => {
|
||||
test("push replaces the record at the same type and key and keeps every other", async () => {
|
||||
const directory = await scratch();
|
||||
try {
|
||||
const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc"));
|
||||
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("c", "u", 3)]);
|
||||
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("a", "u", 3)]);
|
||||
await store.push(entryOf("a", "t", 9));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "u/c:3"]);
|
||||
await store.push(entryOf("d", "t", 4));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "t/d:4", "u/c:3"]);
|
||||
await store.delete(entryOf("a", "t"));
|
||||
await store.push(entryOf("a", "u", 5));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/b:2", "t/d:4", "u/a:5", "u/c:3"]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("a push that would land a second type under an existing key is refused and the file is byte for byte unchanged", async () => {
|
||||
const directory = await scratch();
|
||||
try {
|
||||
const path = join(directory, "store.cc");
|
||||
const store = new SingleFileMessagePackBackingStore(path);
|
||||
await store.pushAll([entryOf("k", "t", 1), entryOf("m", "t", 2)]);
|
||||
const before = [...(await readFile(path))];
|
||||
await assert.rejects(
|
||||
() => store.push(entryOf("k", "u", 9)),
|
||||
(error: unknown) => error instanceof SchemaConflictError && error.recordKey === "k" && /holds record "k" as type "t"/u.test(error.message),
|
||||
);
|
||||
assert.deepEqual([...(await readFile(path))], before);
|
||||
assert.deepEqual(await snapshotOf(new SingleFileMessagePackBackingStore(path)), ["t/k:1", "t/m:2"]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("a pushAll batch that carries one key under two types is refused and writes nothing", async () => {
|
||||
const directory = await scratch();
|
||||
try {
|
||||
const path = join(directory, "store.cc");
|
||||
const store = new SingleFileMessagePackBackingStore(path);
|
||||
await assert.rejects(() => store.pushAll([entryOf("k", "t", 1), entryOf("k", "u", 2)]), SchemaConflictError);
|
||||
assert.equal((await readdir(directory)).includes("store.cc"), false);
|
||||
await store.pushAll([entryOf("a", "t", 1)]);
|
||||
const before = [...(await readFile(path))];
|
||||
await assert.rejects(() => store.pushAll([entryOf("k", "t", 1), entryOf("k", "u", 2)]), SchemaConflictError);
|
||||
assert.deepEqual([...(await readFile(path))], before);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("a pushAll batch that carries one key twice under one type keeps the last and the file reads back with one record", async () => {
|
||||
const directory = await scratch();
|
||||
try {
|
||||
const path = join(directory, "store.cc");
|
||||
await new SingleFileMessagePackBackingStore(path).pushAll([entryOf("k", "t", 1), entryOf("a", "t", 3), entryOf("k", "t", 2)]);
|
||||
assert.deepEqual(await snapshotOf(new SingleFileMessagePackBackingStore(path)), ["t/a:3", "t/k:2"]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("a legacy envelope array that holds a key under two types is refused on open, as a v1 store is", async () => {
|
||||
const directory = await scratch();
|
||||
try {
|
||||
const path = join(directory, "legacy.cc");
|
||||
const bytes = encode([
|
||||
{ key: "a", type: "t", payload: { n: 1 }, storedAt: "2026-10-20T00:00:00Z" },
|
||||
{ key: "a", type: "u", payload: { n: 2 }, storedAt: "2026-10-20T00:00:00Z" },
|
||||
]);
|
||||
await writeFile(path, bytes);
|
||||
const store = new SingleFileMessagePackBackingStore(path);
|
||||
await assert.rejects(
|
||||
() => store.pullAll(),
|
||||
(error: unknown) => error instanceof StoreUnreadableError && /"a" appears more than once/u.test(error.message),
|
||||
);
|
||||
await assert.rejects(() => store.push(entryOf("b")), StoreUnreadableError);
|
||||
assert.deepEqual([...(await readFile(path))], [...bytes]);
|
||||
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "u/a:3"]);
|
||||
await store.push(entryOf("c", "t", 4));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/a:9", "t/b:2", "t/c:4", "u/a:3"]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
|
|
@ -659,12 +589,11 @@ test("delete removes only the record at that type and key", async () => {
|
|||
const directory = await scratch();
|
||||
try {
|
||||
const store = new SingleFileMessagePackBackingStore(join(directory, "store.cc"));
|
||||
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("c", "u", 3)]);
|
||||
await store.pushAll([entryOf("a", "t", 1), entryOf("b", "t", 2), entryOf("a", "u", 3)]);
|
||||
await store.delete(entryOf("a", "t"));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/c:3"]);
|
||||
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/a:3"]);
|
||||
await store.delete(entryOf("zzz", "t"));
|
||||
await store.delete(entryOf("c", "t"));
|
||||
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/c:3"]);
|
||||
assert.deepEqual(await snapshotOf(store), ["t/b:2", "u/a:3"]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -119,6 +119,14 @@ test("a removed or changed holder releases its value", async () => {
|
|||
assert.equal(cache.getKeyByIndex(document, "sku", "T"), "a");
|
||||
});
|
||||
|
||||
test("the keys in a refusal are in ordinal order whichever one lands second", async () => {
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "m", { sku: "S" });
|
||||
await assert.rejects(cache.put(document, "z", { sku: "S" }), { message: refusal("m", "z", "tests.item", "sku") });
|
||||
await assert.rejects(cache.put(document, "B", { sku: "S" }), { message: refusal("B", "m", "tests.item", "sku") });
|
||||
});
|
||||
|
||||
// The C# White_Space set, and nothing else, is blank. Every UTF-16 code unit is judged, so a rule written for a few
|
||||
// characters (or for JavaScript's \s, which counts U+FEFF and omits U+0085) cannot pass.
|
||||
const CSHARP_WHITE_SPACE = (code: number) =>
|
||||
|
|
@ -357,77 +365,6 @@ test("vectors duplicate-store.cc and blank-store.cc hold their verdicts", async
|
|||
}
|
||||
});
|
||||
|
||||
// refusals.msgpack: rows [label, ops, refusal] written by the C# reference. TypeScript has no batch commit
|
||||
// (variants:idea:unique-index-runtimes.ts-batch-commit), so it performs every row whose commits each hold one record:
|
||||
// 5, 8-13, 19-22, 24, 25, 27, 29, 31, 32, 33, 35 and 37 of the README (20).
|
||||
const REFUSAL_SCHEMAS: [string, string, string, Record<string, "sku" | "second">][] = [
|
||||
["unique_item", "tests.unique_item", "sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882", { Sku: "sku" }],
|
||||
["uq_pair", "tests.uq_pair", "sha256:2538cceafb6dcbce22ef11b932ecb804af5f96bc3aaedff692a52d3640433e60", { Sku: "sku", Alt: "second" }],
|
||||
["uq_alias", "tests.uq_alias", "sha256:77a77a394ebd8bf71e14b9b6f2cea4978c25d4e705fd12364f660edbf6c63b1f", { Sku: "sku", alt: "second" }],
|
||||
["uq_wide", "tests.uq_wide", "sha256:b79eecd11eed5e456902f095d2748da7f59125c4c8972b0da7dddc81d07fc03f", { "\uFF21": "sku", "\u{10400}": "second" }],
|
||||
["uq_alpha", "tests.uq_alpha", "sha256:53aeaf2ea9b25e85ade6b15fc04c7653e7b206c87e258f1aa74dd32a840a1389", { Sku: "sku" }],
|
||||
["uq_beta", "tests.uq_beta", "sha256:45ed62a64c91f6d815d3ec975c4a4a1b2ce07b6e7262a32d828673cf10899586", { Sku: "sku" }],
|
||||
];
|
||||
|
||||
type RefusalOp = ["open", string | null] | ["pull", string] | ["commit", [string, string, string, string][]];
|
||||
|
||||
test("vectors refusals.msgpack: every row TS can perform", async () => {
|
||||
const rows = decode(await readFile(join(vectors, "refusals.msgpack"))) as [string, RefusalOp[], [string, string, string, string] | null][];
|
||||
const definitions = new Map(
|
||||
REFUSAL_SCHEMAS.map(([type, schemaName, schemaId, indexes]) => [
|
||||
schemaName,
|
||||
defineDocumentType({
|
||||
type,
|
||||
schemaId,
|
||||
schemaName,
|
||||
schemaVersion: `${schemaName}.v1`,
|
||||
schema: z.object({ sku: z.string(), second: z.string() }),
|
||||
codec: codecs.slots<{ sku: string; second: string }>({ 0: ["sku", codecs.string], 1: ["second", codecs.string] }, `Uq${type}`),
|
||||
indexes,
|
||||
}),
|
||||
]),
|
||||
);
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-refusals-"));
|
||||
let walked = 0;
|
||||
try {
|
||||
for (const [label, ops, refused] of rows) {
|
||||
if (ops.some((op) => op[0] === "commit" && op[1].length > 1)) continue;
|
||||
const file = join(dir, `${walked}.cc`);
|
||||
let cache: CultCache | undefined;
|
||||
const perform = async (op: RefusalOp) => {
|
||||
if (op[0] === "open") {
|
||||
if (op[1] !== null) await copyFile(join(vectors, op[1]), file);
|
||||
let builder = CultCache.builder().withGenericStore(new SingleFileMessagePackBackingStore(file));
|
||||
for (const definition of definitions.values()) builder = builder.withDocumentType(definition);
|
||||
cache = builder.build();
|
||||
await cache.pullAllBackingStores();
|
||||
} else if (op[0] === "pull") {
|
||||
await copyFile(join(vectors, op[1]), file);
|
||||
await cache!.pullAllBackingStores();
|
||||
} else {
|
||||
for (const [key, schema, sku, second] of op[1]) await cache!.put(definitions.get(schema)!, key, { sku, second });
|
||||
}
|
||||
};
|
||||
for (const op of ops.slice(0, -1)) await perform(op);
|
||||
const last = ops[ops.length - 1];
|
||||
if (refused === null) {
|
||||
await perform(last);
|
||||
} else {
|
||||
const [k1, k2, schema, alias] = refused;
|
||||
const before = last[0] === "commit" || (last[0] === "open" && last[1] === null)
|
||||
? (existsSync(file) ? await readFile(file) : Buffer.alloc(0))
|
||||
: await readFile(join(vectors, last[1]!));
|
||||
await assert.rejects(perform(last), { message: refusal(k1, k2, schema, alias) }, label);
|
||||
assert.deepEqual(existsSync(file) ? await readFile(file) : Buffer.alloc(0), before, `${label}: the file is untouched`);
|
||||
}
|
||||
walked++;
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
assert.equal(walked, 20);
|
||||
});
|
||||
|
||||
test("a refused load leaves the cache's prior state as it was, and a later good load still works", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-load-"));
|
||||
try {
|
||||
|
|
@ -551,33 +488,27 @@ test("registerIndex of a second unique alias judges it, naming the two smallest
|
|||
assert.equal(cache.getKeyByIndex(document, "b", "s"), undefined);
|
||||
});
|
||||
|
||||
// ---- the single-put judge's two orders, which no pair-order vector row reaches from TS (a vector row's second record
|
||||
// arrives in a batch that TS cannot perform): aliases by UTF-16 code unit, then the key pair by UTF-16 code unit ----
|
||||
|
||||
test("a put colliding on two aliases names the one least by UTF-16 code unit, not by code point", async () => {
|
||||
// U+10400 is the surrogate pair D801 DC00, so it sorts before U+FF21 by code unit and after it by code point.
|
||||
const document = pair({ indexes: { "Ａ": "a", "\u{10400}": "b" } });
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "h1", { a: "s1", b: "t1" });
|
||||
await cache.put(document, "h2", { a: "s2", b: "t2" });
|
||||
await assert.rejects(cache.put(document, "p", { a: "s1", b: "t2" }), { message: refusal("h2", "p", "tests.pair", "\u{10400}") });
|
||||
assert.equal(cache.get(document, "p"), undefined);
|
||||
});
|
||||
|
||||
test("a put names its key pair by UTF-16 code unit, not by normalised form", async () => {
|
||||
// "e" U+0302 is the NFD form of U+00EA: equal when normalised, and less by code unit.
|
||||
const document = item("item");
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "ê", { sku: "v" });
|
||||
await assert.rejects(cache.put(document, "ê", { sku: "v" }), { message: refusal("ê", "ê", "tests.item", "sku") });
|
||||
assert.equal(cache.get(document, "ê"), undefined);
|
||||
});
|
||||
|
||||
test("a put colliding on two aliases orders them by UTF-16 code unit, not by normalised form", async () => {
|
||||
// U+00EA normalises to e U+0302, which is less than e U+0303 when normalised and greater by code unit.
|
||||
const document = pair({ indexes: { "ê": "a", "ẽ": "b" } });
|
||||
const cache = memory(document);
|
||||
await cache.put(document, "h1", { a: "s1", b: "t1" });
|
||||
await cache.put(document, "h2", { a: "s2", b: "t2" });
|
||||
await assert.rejects(cache.put(document, "p", { a: "s1", b: "t2" }), { message: refusal("h2", "p", "tests.pair", "ẽ") });
|
||||
test("verdict parity: the same stores are refused whatever the load order, and the first collision refuses (text is informational)", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "uidx-verdict-"));
|
||||
try {
|
||||
// Three records, one pair colliding; whichever position the pair has in the file, the store is refused and a clean store opens.
|
||||
const roles = pair({ roles: { a: "a" } });
|
||||
for (const [name, values, refused] of [
|
||||
["pair-first", ["s", "s", "t"], true],
|
||||
["pair-last", ["t", "s", "s"], true],
|
||||
["pair-split", ["s", "t", "s"], true],
|
||||
["distinct", ["r", "s", "t"], false],
|
||||
] as const) {
|
||||
const file = join(dir, `${name}.cc`);
|
||||
const writer = CultCache.builder().withDocumentType(roles).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
for (const [i, a] of values.entries()) await writer.put(roles, `r${i}`, { a });
|
||||
const unique = pair({ indexes: { a: "a" } });
|
||||
const reader = CultCache.builder().withDocumentType(unique).withGenericStore(new SingleFileMessagePackBackingStore(file)).build();
|
||||
if (refused) await assert.rejects(reader.pullAllBackingStores(), /both hold unique index 'a'/u, name);
|
||||
else await reader.pullAllBackingStores();
|
||||
assert.equal(reader.snapshot().length, refused ? 0 : 3, name);
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -7,9 +7,6 @@ All notable changes to `@gamecult/cultnet-ts` are recorded here, following
|
|||
|
||||
### Added
|
||||
|
||||
- Fork card (`contracts/cultnet/governance/fork-card.md`): `raised_at`, a required key holding an RFC3339 instant string or nil (the question's first admission time), outside `options_hash`; the card vectors are re-pinned with the key, three negatives (epoch integer, date-only string, missing key) and a nil-`raised_at` valid card; every `optionsHash` digest is unchanged.
|
||||
- Fork card reader views (`fork-card.md`, "Reader views"): the card gains `decider_count` (required, 1 to 64) and `deciders` may be empty (ids withheld from a non-decider); the record gains `answerer_count` (distinct answerers over every stored answer, refusals included); `answerer` is nullable inside a record shown to a non-decider. Card and record vectors are re-pinned (card map header `DE0011` to `DE0012`, record `85` to `86`); withheld twins, the R-card, R-record, R-entry, R-list and R-answer-result rule vectors and a no-leak byte check are new; every `optionsHash` and `bodyDigests` digest is unchanged.
|
||||
- Fork card vectors: a nil `answerer` is now pinned for Revision and Retraction as well as Counted and Refused (`deskAnswer` `withheld_answerer_revision`, `withheld_answerer_retraction`; entries `open_counted_revision_and_retraction` and its withheld twin; a list scenario pair; negatives for a nil answerer in an `answer_result` and for a named Retraction or an over-count in a withheld entry). Additive: every existing vector, `optionsHash` and `bodyDigests` digest is unchanged.
|
||||
- `WriteTracker`: a write sent after `sessionEnd` settles unconfirmed `session_ended` when it is sent (it stayed unanswered forever). The replay runner now fails on an event key outside `scenarios.semantics` and on a vector count that differs from the file's `counts`; seven tracker vectors are new. A blank text in a raw frame is Unicode White_Space, not `String.prototype.trim()`.
|
||||
- Write receipt, client side (`contracts/cultnet/interop/write-receipt.md`): `cultnet.write_admitted.v0` and `cultnet.error.v1` (v0 plus `messageId`; its closed code set adds `limit_reached`, `CULTNET_ERROR_V1_CODES`) parse and encode, and `CULTNET_WRITE_RECEIPT_MESSAGE_VERSIONS` / `helloAdvertisesWriteReceipts` name the hello capability. `cultnet.error.v0` and its code set are unchanged. `WriteTracker` settles each write as admitted, refused or unconfirmed (windows `RECEIPT_RESULT_WINDOW_MS` 5000 and `LEGACY_RESULT_WINDOW_MS` 500, counted from the send; late answers recorded as `admitted_late` / `refused_late`), and `tools/replay-write-receipt-vectors.mjs` replays the vectors against it (see README, "Write receipt replay runner"). The interop schemas are read from `contracts/cultnet/interop/`.
|
||||
- `message_invalid` joins the closed `cultnet.error.v0` code set, with `cultNetMessageInvalidError(field, reason)` (`details.field` names the request field at fault), byte-identical to the C# `ForMessageInvalid`.
|
||||
|
|
|
|||
|
|
@ -90,7 +90,7 @@ const jstr = (s: string): string => {
|
|||
};
|
||||
type Option = { id: string; text: string };
|
||||
type Shown = { title: string; context: string; options: Option[]; recommendation: string; depends: string[] };
|
||||
type Card = Shown & { fork_id: string; leans: string | null; raised_at: string | null; deciders: string[]; decider_count: number; status: string; ruling: string | null; ruled_from: string[]; options_hash: string };
|
||||
type Card = Shown & { fork_id: string; leans: string | null; deciders: string[]; status: string; ruling: string | null; ruled_from: string[]; options_hash: string };
|
||||
// RFC 8785 over the five shown members, every string NFC-normalised first; keys sorted; no numbers in the member set.
|
||||
const nfc = (s: string): string => s.normalize("NFC");
|
||||
const canonical = (c: Shown): Buffer =>
|
||||
|
|
@ -104,7 +104,6 @@ const cardRuleErrors = (c: Card): string[] => {
|
|||
if (!ids.includes(c.recommendation)) errors.push("recommendation is not an option");
|
||||
if (c.leans !== null && !ids.includes(c.leans)) errors.push("leans is not an option");
|
||||
if (new Set(c.deciders).size !== c.deciders.length) errors.push("duplicate deciders");
|
||||
if (c.deciders.length > 0 && c.deciders.length !== c.decider_count) errors.push("R-card: shown deciders are not exactly decider_count ids");
|
||||
if (c.depends.some((d) => Buffer.byteLength(d, "utf8") > 1000)) errors.push("a depends line is over 1000 UTF-8 bytes");
|
||||
if (c.status === "open" && c.ruling !== null) errors.push("an open card names a ruling");
|
||||
if (c.ruling === null && c.ruled_from.length > 0) errors.push("ruled_from without a ruling");
|
||||
|
|
@ -122,15 +121,6 @@ test("fork card: options hash is SHA-256 of the canonical JSON, computed here, a
|
|||
const byName = new Map<string, string>(fork.optionsHash.map((h: { name: string; sha256: string }) => [h.name, h.sha256]));
|
||||
const base = byName.get("heimdall_A_five_members")!;
|
||||
assert.equal(byName.get("hidden_members_do_not_move_it"), base);
|
||||
const card0 = fork.card.valid[0].message as Card;
|
||||
for (const raised_at of [null, "2027-01-02T03:04:05.123Z"]) {
|
||||
const stamped: Card = { ...card0, raised_at };
|
||||
assert.equal(optionsHash(stamped), card0.options_hash, "raised_at stays outside the hash");
|
||||
}
|
||||
for (const [deciders, decider_count] of [[[], 2], [card0.deciders, 64]] as const) {
|
||||
const shown: Card = { ...card0, deciders: [...deciders], decider_count };
|
||||
assert.equal(optionsHash(shown), card0.options_hash, "decider_count and the emptiness of deciders stay outside the hash");
|
||||
}
|
||||
assert.equal(byName.get("heimdall_E_decomposed_equals_D"), byName.get("heimdall_D_nfc_escapes_astral"), "an NFD and an NFC spelling hash alike");
|
||||
for (const moved of ["heimdall_B_recommendation_changed", "heimdall_C_option_order_changed", "heimdall_F_option_id_changed", "heimdall_G_depends_emptied", "option_text_changed",
|
||||
"title_changed", "context_changed", "depends_order_changed", "depends_order_changed_reversed"]) {
|
||||
|
|
@ -181,7 +171,7 @@ test("fork card: mutating any shown member, an option id included, changes the h
|
|||
assert.notEqual(optionsHash(copy), base, `mutation ${i}`);
|
||||
}
|
||||
const hidden = structuredClone(card);
|
||||
hidden.fork_id = "other:question:id"; hidden.leans = "yes"; hidden.deciders = []; hidden.decider_count = 7; hidden.status = "closed"; hidden.ruled_from = ["n1"];
|
||||
hidden.fork_id = "other:question:id"; hidden.leans = "yes"; hidden.deciders = ["x"]; hidden.status = "closed"; hidden.ruled_from = ["n1"];
|
||||
assert.equal(optionsHash(hidden), base, "fork_id and the unshown members stay outside the hash");
|
||||
});
|
||||
|
||||
|
|
@ -205,11 +195,6 @@ test("fork card: documents validate and encode to the vector bytes", () => {
|
|||
assert.ok(check(forkSchema, "answerResult", v.message), v.name);
|
||||
assert.equal(hexOf(v.message), v.messagePackHex, v.name);
|
||||
assert.equal(v.message.card !== null, v.name === "stale_fork", "only a stale refusal shows the new fork");
|
||||
assert.deepEqual(resultRuleErrors(v.message), [], v.name);
|
||||
}
|
||||
for (const v of fork.resultsInvalid) {
|
||||
assert.ok(!check(forkSchema, "answerResult", v.message) || resultRuleErrors(v.message).length > 0, `${v.name}: ${v.why}`);
|
||||
assert.equal(hexOf(v.message), v.messagePackHex, v.name);
|
||||
}
|
||||
for (const v of fork.unavailable) {
|
||||
assert.ok(check(forkSchema, "unavailable", v.message), v.name);
|
||||
|
|
@ -238,16 +223,12 @@ test("fork card: every negative is refused by the schema or a stated rule", () =
|
|||
}
|
||||
});
|
||||
|
||||
// R-answer-result: the answer in an answer_result is the sender's own, stamped with their account, never nil (nil exists only in a record shown to a non-decider).
|
||||
const resultRuleErrors = (r: { answer: { answerer: string | null } }): string[] => (r.answer.answerer === null ? ["R-answer-result: the answerer is nil"] : []);
|
||||
|
||||
// Rules a JSON Schema cannot say: a note is at most 4000 UTF-8 bytes (maxLength counts code points and is only the looser bound).
|
||||
const noteRuleErrors = (def: string, message: any): string[] => {
|
||||
const note = def === "answerRequest" ? message?.answer?.note : def === "deskAnswer" ? message?.note : null;
|
||||
return note != null && Buffer.byteLength(note, "utf8") > 4000 ? ["note is over 4000 UTF-8 bytes"] : [];
|
||||
};
|
||||
|
||||
const OP_ID = "e0d1c2b3-a495-4687-8879-6a5b4c3d2e1f";
|
||||
const TAILS: Record<string, string> = {
|
||||
unauthenticated: "a verified session is required",
|
||||
body_forbidden_field: "answerer and authority are set by the receiver",
|
||||
|
|
@ -319,7 +300,7 @@ function judge(start: Card, steps: { kind: string; body: Record<string, any> | n
|
|||
}
|
||||
responses.push(down && reached ? unavailable() : response);
|
||||
}
|
||||
return { responses, record: { schemaVersion: "gamecult.fork.record.v0", fork_id: card.fork_id, options_hash: optionsHash(card), counted_answer_id: counted(), answerer_count: new Set(answers.map((a) => a.answerer)).size, answers } };
|
||||
return { responses, record: { schemaVersion: "gamecult.fork.record.v0", fork_id: card.fork_id, options_hash: optionsHash(card), counted_answer_id: counted(), answers } };
|
||||
}
|
||||
|
||||
test("fork card: every scenario replays to the vector's responses and record, bytes included", () => {
|
||||
|
|
@ -331,7 +312,6 @@ test("fork card: every scenario replays to the vector's responses and record, by
|
|||
assert.deepEqual(record, sc.record, sc.name);
|
||||
assert.ok(check(forkSchema, "record", record), sc.name);
|
||||
assert.equal(hexOf(record), sc.recordHex, sc.name);
|
||||
for (const s of sc.steps) if (s.response?.answer) assert.deepEqual(resultRuleErrors(s.response), [], sc.name);
|
||||
for (const s of sc.steps) if (s.response) assert.ok(check(forkSchema, "answerResult", s.response) || check(forkSchema, "refusal", s.response) || check(forkSchema, "unavailable", s.response), sc.name);
|
||||
}
|
||||
});
|
||||
|
|
@ -398,34 +378,15 @@ test("fork card: operator authority needs a cited Counted answer record", () =>
|
|||
});
|
||||
|
||||
// ---- transport: list, read, auth (fork-card.md "Transport") ----
|
||||
type Entry = { card: Card; record: { fork_id: string; options_hash: string; counted_answer_id: string | null; answerer_count: number; answers: Record<string, any>[] } };
|
||||
const entryRuleErrors = (e: Entry): string[] => {
|
||||
const entryRuleErrors = (e: { card: Card; record: { fork_id: string; options_hash: string; counted_answer_id: string | null; answers: Record<string, any>[] } }): string[] => {
|
||||
const errors = cardRuleErrors(e.card);
|
||||
if (e.record.fork_id !== e.card.fork_id) errors.push("record of another fork");
|
||||
if (e.record.options_hash !== e.card.options_hash) errors.push("record under another hash");
|
||||
const counted = e.record.answers.find((a) => (a.outcome === "Counted" || a.outcome === "Revision") && a.options_hash === e.card.options_hash
|
||||
&& !e.record.answers.some((r) => r.outcome === "Retraction" && r.retracts === a.answer_id));
|
||||
if ((counted?.answer_id ?? null) !== e.record.counted_answer_id) errors.push("counted_answer_id is not derived");
|
||||
// R-record: named answerers are counted by distinct id, nil ones by a bound
|
||||
const named = e.record.answers.filter((a) => a.answerer !== null).length, total = e.record.answers.length, n = e.record.answerer_count;
|
||||
if (total === 0 && n !== 0) errors.push("R-record: no answers and a count other than 0");
|
||||
else if (named === total && total > 0 && n !== new Set(e.record.answers.map((a) => a.answerer)).size) errors.push("R-record: answerer_count is not the distinct answerers");
|
||||
else if (named === 0 && total > 0 && (n < 1 || n > total)) errors.push("R-record: a withheld count outside 1..answers");
|
||||
// R-entry: the card's view and the record's view agree, so a withheld card never rides beside a named account
|
||||
if (e.card.deciders.length > 0 && named < total) errors.push("R-entry: a decider-view card with withheld answerers");
|
||||
if (e.card.deciders.length === 0 && named > 0) errors.push("R-entry: a withheld card beside a named answerer");
|
||||
return errors;
|
||||
};
|
||||
// R-list: all entries of one page are one reader's view
|
||||
const viewRuleErrors = (forks: { card: Card }[]): string[] => (new Set(forks.map((f) => f.card.deciders.length > 0)).size > 1 ? ["R-list: a page mixes views"] : []);
|
||||
// The non-decider twin of a decider-view entry: ids withheld, counts kept.
|
||||
const withheld = (e: Entry): Entry => ({ ...e, card: { ...e.card, deciders: [] }, record: { ...e.record, answers: e.record.answers.map((a) => ({ ...a, answerer: null })) } });
|
||||
const idsOf = (e: Entry): string[] => [...e.card.deciders, ...e.record.answers.map((a) => a.answerer as string)];
|
||||
const bytesHaveNoId = (hex: string, ids: string[], name: string): void => {
|
||||
const bytes = Buffer.from(hex, "hex");
|
||||
assert.ok(ids.length > 0, name);
|
||||
for (const id of ids) assert.ok(!bytes.includes(Buffer.from(id, "utf8")), `${name}: the withheld bytes carry ${id}`);
|
||||
};
|
||||
|
||||
// Order is a unique rank per Mind: the refusal names both fork ids, the first holder and the second.
|
||||
const orderErrors = (cards: { fork_id: string; order: number }[]): string[] => {
|
||||
|
|
@ -504,13 +465,6 @@ test("fork transport: requests and entries validate and encode to the vector byt
|
|||
for (const v of t[section].valid) {
|
||||
assert.ok(check(forkSchema, def, v.message), v.name);
|
||||
if (def === "entry") assert.deepEqual(entryRuleErrors(v.message), [], v.name);
|
||||
if (def === "entry" && v.twin !== undefined) {
|
||||
const twin = t.entry.valid.find((x: any) => x.name === v.twin);
|
||||
assert.deepEqual(v.message, withheld(twin.message), `${v.name}: the twin of ${v.twin}`);
|
||||
assert.equal(v.message.record.answerer_count, twin.message.record.answerer_count, "a decider and a non-decider read the same count");
|
||||
bytesHaveNoId(v.messagePackHex, idsOf(twin.message), v.name);
|
||||
assert.ok(twin.messagePackHex.includes(Buffer.from(idsOf(twin.message)[0], "utf8").toString("hex").toUpperCase()), "the decider view does carry the ids");
|
||||
}
|
||||
assert.equal(hexOf(v.message), v.messagePackHex, v.name);
|
||||
}
|
||||
for (const v of t[section].invalid) {
|
||||
|
|
@ -519,10 +473,9 @@ test("fork transport: requests and entries validate and encode to the vector byt
|
|||
}
|
||||
}
|
||||
for (const v of t.listInvalid) {
|
||||
const refused = !check(forkSchema, "list", v.message) || orderErrors(v.message.forks.map((f: any) => f.card)).length > 0 || viewRuleErrors(v.message.forks).length > 0;
|
||||
const refused = !check(forkSchema, "list", v.message) || orderErrors(v.message.forks.map((f: any) => f.card)).length > 0;
|
||||
assert.ok(refused, `${v.name}: ${v.why}`);
|
||||
if (v.refusal !== undefined) assert.deepEqual(orderErrors(v.message.forks.map((f: any) => f.card)), [v.refusal], v.name);
|
||||
if (v.name === "a_page_mixes_views") assert.equal(viewRuleErrors(v.message.forks).length, 1, v.name);
|
||||
assert.equal(hexOf(v.message), v.messagePackHex, v.name);
|
||||
}
|
||||
});
|
||||
|
|
@ -531,26 +484,12 @@ test("fork transport: every list request replays to the vector's response, newes
|
|||
for (const sc of fork.transport.listScenarios) {
|
||||
for (const e of sc.entries) assert.deepEqual(entryRuleErrors(e), [], sc.name);
|
||||
assert.deepEqual(orderErrors(sc.entries.map((e: any) => e.card)), [], sc.name);
|
||||
if (sc.twin !== undefined) {
|
||||
// the non-decider page is the decider page with the ids withheld: counts, order, cursor and stamp equal, byte for byte but the withheld members
|
||||
const twin = fork.transport.listScenarios.find((x: any) => x.name === sc.twin);
|
||||
assert.deepEqual(sc.entries, twin.entries.map(withheld), sc.name);
|
||||
assert.equal(sc.steps.length, twin.steps.length);
|
||||
sc.steps.forEach((step: any, i: number) => {
|
||||
const other = twin.steps[i].response;
|
||||
assert.deepEqual(step.response, { ...other, forks: other.forks.map(withheld) }, sc.name);
|
||||
assert.ok(twin.entries.length > 0 && step.response.forks.length > 0);
|
||||
bytesHaveNoId(step.responseHex, twin.entries.flatMap(idsOf), sc.name);
|
||||
});
|
||||
assert.ok(twin.steps[0].responseHex.includes(Buffer.from(OP_ID, "utf8").toString("hex").toUpperCase()), "the decider page does carry the ids");
|
||||
}
|
||||
let previous: { next_cursor: string | null } | null = null;
|
||||
for (const step of sc.steps) {
|
||||
const response = judgeList(sc.entries, sc.ordinal, step.request);
|
||||
assert.deepEqual(response, step.response, sc.name);
|
||||
assert.ok(check(forkSchema, "list", response), sc.name);
|
||||
assert.equal(hexOf(response), step.responseHex, sc.name);
|
||||
assert.deepEqual(viewRuleErrors(response.forks), [], sc.name);
|
||||
const orders = response.forks.map((f: any) => f.card.order);
|
||||
assert.deepEqual(orders, [...orders].sort((a: number, b: number) => b - a), sc.name);
|
||||
if (previous !== null) assert.equal(step.request.cursor, previous.next_cursor, `${sc.name}: the second request follows the first page's cursor`);
|
||||
|
|
|
|||
|
|
@ -115,8 +115,6 @@ test("a client reconnecting from the same endpoint replaces only its own session
|
|||
assert.throws(() => server.send(oldPeer, "schema", enc("stale")), /not connected/);
|
||||
assert.equal(server.peers.length, 2);
|
||||
|
||||
// The server accepting is not the client hearing it: send only once the client's new session is connected.
|
||||
await waitFor(() => a.connection.connected, "the reconnecting client's handshake");
|
||||
a.connection.send("schema", enc("after"));
|
||||
b.connection.send("schema", enc("still"));
|
||||
await waitFor(() => a.received.includes("re:after") && b.received.includes("re:still"), "replies after the reconnect");
|
||||
|
|
|
|||
|
|
@ -173,11 +173,9 @@ def chash(c): return sha(canonical_json(c["title"], c["context"], c["options"],
|
|||
def card(**kw):
|
||||
c = {"schemaVersion": "gamecult.fork.card.v0", "fork_id": FID, "title": "Slot mark consent", "context": "Should a slot mark be shared?",
|
||||
"options": OPT, "recommendation": "no", "depends": DEP, "leans": None, "flag": None,
|
||||
"raised_in": {"kind": "CutSpec", "id": "cultlib-gaps:cut_spec:cut-slot-tolerance.r4"}, "raised_at": "2026-11-10T09:30:00Z", "deciders": [OP], "decider_count": 1,
|
||||
"raised_in": {"kind": "CutSpec", "id": "cultlib-gaps:cut_spec:cut-slot-tolerance.r4"}, "deciders": [OP],
|
||||
"status": "open", "ruling": None, "ruled_from": [], "order": 3, "options_hash": ""}
|
||||
c.update(kw)
|
||||
if "decider_count" not in kw: c["decider_count"] = len(c["deciders"]) # the decider view: the count equals the ids; a withheld card passes its own
|
||||
c["options_hash"] = chash(c); return c
|
||||
c.update(kw); c["options_hash"] = chash(c); return c
|
||||
CARD = card(); H = CARD["options_hash"]
|
||||
|
||||
hashes = []
|
||||
|
|
@ -204,7 +202,7 @@ hv("title_changed", card(title="Slot mark consent?"))
|
|||
hv("context_changed", card(context="Should slot marks be shared?"))
|
||||
hv("depends_order_changed", card(depends=DEP + ["x:question:y"]))
|
||||
hv("depends_order_changed_reversed", card(depends=["x:question:y"] + DEP))
|
||||
hv("hidden_members_do_not_move_it", card(leans="yes", flag="provisional", deciders=[], decider_count=2, status="closed", ruling="r:1", ruled_from=["rc-1"], order=9, raised_in=None, raised_at="2027-01-02T03:04:05.123Z", fork_id="other:question:id"))
|
||||
hv("hidden_members_do_not_move_it", card(leans="yes", flag="provisional", deciders=[OP, HM], status="closed", ruling="r:1", ruled_from=["rc-1"], order=9, raised_in=None, fork_id="other:question:id"))
|
||||
hv("control_characters_and_markup_are_escaped_minimally", card(title='q " b \\ t \t n \n bell \u0007 del \u007f', context="</script> \u2028 &"))
|
||||
hv("astral_option_id_and_text", card(options=[{"id": "\U0001F600", "text": "\U0001D11E"}, OPT[1]]))
|
||||
assert [h["sha256"] for h in hashes if h["heimdall"] == "D"] == [h["sha256"] for h in hashes if h["heimdall"] == "E"]
|
||||
|
|
@ -220,7 +218,6 @@ def body(choice=None, note=None, h=None, at="2026-11-12T19:02:00Z", fork=FID, re
|
|||
return {"schemaVersion": "gamecult.fork.answer_request.v0", "fork_id": fork, "options_hash": h or H, "answer": a,
|
||||
"retracted": retracts is not None, "retracts": retracts, "answered_at": at, "request_id": rid or next_rid()}
|
||||
def retraction(target, **kw): return body(retracts=target, **kw)
|
||||
NULLANS_RID = "0000b001-0000-4000-8000-00000000b001"
|
||||
NOTE = "Keep it local: a slot mark is the author's call, and sharing it is a separate act."
|
||||
def srv(aid, who, at, auth="Operator"): return {"answer_id": aid, "answerer": who, "authority": auth, "received_at": at}
|
||||
TAIL = {"unauthenticated": "a verified session is required", "body_forbidden_field": "answerer and authority are set by the receiver", "body_unknown_field": "the body is a closed shape",
|
||||
|
|
@ -240,10 +237,7 @@ def da(outcome, link, code, b, server):
|
|||
"options_hash": b["options_hash"], "choice": a["choice"], "note": a["note"], "answered_at": b["answered_at"], "received_at": server["received_at"],
|
||||
"claim_in_future": future(b["answered_at"], server["received_at"]), "answerer": server["answerer"], "authority": server["authority"], "request_id": b["request_id"]}
|
||||
def result(answer, c=None): return {"schemaVersion": "gamecult.fork.answer_result.v0", "answer": answer, "card": c, "counted_answer_id": None} # sc() fills counted_answer_id
|
||||
def record(counted, answers, h=H, fork=FID, answerer_count=None):
|
||||
# answerer_count: distinct answerers over every stored answer, refusals included; a withheld record passes the count it kept
|
||||
n = len({a["answerer"] for a in answers}) if answerer_count is None else answerer_count
|
||||
return {"schemaVersion": "gamecult.fork.record.v0", "fork_id": fork, "options_hash": h, "counted_answer_id": counted, "answerer_count": n, "answers": answers}
|
||||
def record(counted, answers, h=H, fork=FID): return {"schemaVersion": "gamecult.fork.record.v0", "fork_id": fork, "options_hash": h, "counted_answer_id": counted, "answers": answers}
|
||||
T = lambda m, sec=5: f"2026-11-12T19:{m:02d}:{sec:02d}Z"
|
||||
def step(kind, body_=None, server=None, response=None, **kw):
|
||||
d = {"kind": kind, "body": body_, "server": server, "response": response}; d.update(kw); return d
|
||||
|
|
@ -391,38 +385,15 @@ def fcard(n, status="open", **kw):
|
|||
return c
|
||||
def fbody(c, choice, aid_note):
|
||||
return body(choice, aid_note, h=c["options_hash"], fork=c["fork_id"])
|
||||
def with_count(e, n): return dict(e, record=dict(e["record"], answerer_count=n))
|
||||
def withheld_answers(e):
|
||||
e = json.loads(json.dumps(e))
|
||||
for a in e["record"]["answers"]: a["answerer"] = None
|
||||
return e
|
||||
def fentry(c, answers=(), counted=None):
|
||||
return {"schemaVersion": "gamecult.fork.entry.v0", "card": c, "record": record(counted, list(answers), c["options_hash"], c["fork_id"])}
|
||||
def withheld(e):
|
||||
"""The non-decider twin of a decider-view entry: ids withheld, counts kept (fork-card.md, Reader views)."""
|
||||
e = json.loads(json.dumps(e)); e["card"]["deciders"] = []
|
||||
for a in e["record"]["answers"]: a["answerer"] = None
|
||||
return e
|
||||
q5 = fcard(5); q4 = fcard(4); q2 = fcard(2)
|
||||
q3 = fcard(3, "closed", ruling="cultlib-gaps:resolution:question.q3.n1", ruled_from=["n1"]); q1 = fcard(1, "closed", ruling="cultlib-gaps:resolution:question.q1.n1")
|
||||
a5 = da("Counted", None, None, fbody(q5, "no", NOTE), srv("n1", OP, T(2)))
|
||||
a3 = da("Counted", None, None, fbody(q3, "yes", NOTE), srv("n1", OP, T(2)))
|
||||
r3 = da("Refused", None, "fork_closed", fbody(q3, "no", NOTE), srv("n2", HM, T(4), "Steward"))
|
||||
ENTRIES = [fentry(q5, [a5], "n1"), fentry(q4), fentry(q3, [a3, r3], "n1"), fentry(q2), fentry(q1)]
|
||||
a5b = da("Revision", "n1", None, fbody(q5, "yes", NOTE), srv("n2", OP, T(4)))
|
||||
TWICE = fentry(q5, [a5, a5b], "n1") # two answers, one account
|
||||
q0 = fcard(0)
|
||||
# Revision and Retraction answers are nullable like Counted and Refused: one fork holding a Counted, its Revision and a Retraction of the Counted (appended after the older entries; nothing above moves)
|
||||
q6 = fcard(6); _RID = RID[0] # the request_id counter is restored after this block: no later body changes
|
||||
a6 = da("Counted", None, None, fbody(q6, "no", NOTE), srv("n1", OP, T(2)))
|
||||
a6r = da("Revision", "n1", None, fbody(q6, "yes", NOTE), srv("n2", OP, T(4)))
|
||||
a6x = da("Retraction", "n1", None, body(retracts="n1", h=q6["options_hash"], fork=q6["fork_id"]), srv("n3", OP, T(6)))
|
||||
RID[0] = _RID
|
||||
REVRET = fentry(q6, [a6, a6r, a6x], "n2")
|
||||
WREVRET = withheld(REVRET)
|
||||
NAMED_RETRACTION = json.loads(json.dumps(WREVRET)); NAMED_RETRACTION["record"]["answers"][2]["answerer"] = HM
|
||||
WENTRIES = [withheld(e) for e in ENTRIES]
|
||||
MIXED = json.loads(json.dumps(WENTRIES[2])); MIXED["record"]["answers"][0]["answerer"] = OP # one answer named among nils
|
||||
ORDER0 = [fentry(fcard(3)), fentry(fcard(2)), fentry(fcard(1)), fentry(q0)] # a Mind whose oldest card has order 0
|
||||
q4b = dict(fcard(4), fork_id="gamecult-chat:question:q4b") # a second card on order 4 in the same Mind
|
||||
def order_shared(entries):
|
||||
|
|
@ -449,10 +420,7 @@ LISTS = [
|
|||
{"name": "recorded_only_paged", "ordinal": 412, "entries": ENTRIES, "steps": [lstep(ENTRIES, 412, lreq(status="recorded", limit=1)), lstep(ENTRIES, 412, lreq(status="recorded", cursor="c3", limit=1))]},
|
||||
{"name": "limit_zero_is_the_count_and_stamp_only", "ordinal": 412, "entries": ENTRIES, "steps": [lstep(ENTRIES, 412, lreq(limit=0))]},
|
||||
{"name": "order_zero_is_legal_but_no_cursor_names_it", "ordinal": 9, "entries": ORDER0, "steps": [lstep(ORDER0, 9, lreq(limit=3)), lstep(ORDER0, 9, lreq(cursor="c1", limit=3))]},
|
||||
{"name": "non_decider_page", "ordinal": 412, "entries": WENTRIES, "twin": "all_forks_one_page", "steps": [lstep(WENTRIES, 412, lreq(limit=50))]},
|
||||
{"name": "a_mind_with_no_forks", "ordinal": 7, "entries": [], "steps": [lstep([], 7, lreq(mind="nibu", limit=10))]},
|
||||
{"name": "counted_revision_and_retraction_page", "ordinal": 413, "entries": [REVRET], "steps": [lstep([REVRET], 413, lreq(limit=10))]},
|
||||
{"name": "non_decider_counted_revision_and_retraction_page", "ordinal": 413, "entries": [WREVRET], "twin": "counted_revision_and_retraction_page", "steps": [lstep([WREVRET], 413, lreq(limit=10))]},
|
||||
]
|
||||
# A page may stop short of `limit` when the next entry would pass the receiver's response budget; a list whose first
|
||||
# entry alone passes it is refused `fork_too_large` (422). The budget is not in the vector: a step records what the receiver answered, and the
|
||||
|
|
@ -579,10 +547,8 @@ fork = {
|
|||
"card": {"valid": [{"name": "operator_fork", "message": CARD, "messagePackHex": hx(CARD)},
|
||||
{"name": "leans_flag_closed_two_deciders", "message": card(leans="yes", flag="provisional: Self expects this to move", deciders=[OP, HM], depends=[], raised_in=None, status="closed", ruling="x:resolution:y", ruled_from=["n1"], order=0),
|
||||
"messagePackHex": hx(card(leans="yes", flag="provisional: Self expects this to move", deciders=[OP, HM], depends=[], raised_in=None, status="closed", ruling="x:resolution:y", ruled_from=["n1"], order=0))},
|
||||
{"name": "raised_at_nil", "message": card(raised_at=None), "messagePackHex": hx(card(raised_at=None))},
|
||||
{"name": "depends_at_both_caps", "message": card(depends=["a"] * 7 + ["\u00e9" * 500]), "messagePackHex": hx(card(depends=["a"] * 7 + ["\u00e9" * 500]))},
|
||||
{"name": "closed_without_a_ruling_the_question_was_withdrawn", "message": card(status="closed"), "messagePackHex": hx(card(status="closed"))},
|
||||
{"name": "withheld_deciders", "message": card(deciders=[], decider_count=2), "messagePackHex": hx(card(deciders=[], decider_count=2))},
|
||||
{"name": "deciders_at_the_cap_of_64", "message": card(deciders=DECIDERS_64), "messagePackHex": hx(card(deciders=DECIDERS_64))}],
|
||||
"invalid": [bad(n, w, m) for n, w, m in [
|
||||
("depends_nine_lines", "depends holds at most 8 lines (the stored question's limit)", card(depends=["a"] * 9)),
|
||||
|
|
@ -590,13 +556,7 @@ fork = {
|
|||
("depends_line_1001_bytes", "rule: one byte over the limit (500 two-byte characters and one more; not expressible in the schema)", card(depends=["\u00e9" * 500 + "a"])),
|
||||
("depends_line_over_1000_bytes", "rule: a depends line is at most 1000 UTF-8 bytes (the stored Line limit; not expressible in the schema)", card(depends=["\u00e9" * 501])),
|
||||
("one_option", "two or more options", card(options=[OPT[0]], recommendation="yes")),
|
||||
("decider_count_zero", "a fork with no decider cannot be answered; an empty deciders list now means withheld, so the count refuses it", card(deciders=[], decider_count=0)),
|
||||
("decider_count_missing", "map-mode writes every key: decider_count is required", {k: v for k, v in card().items() if k != "decider_count"}),
|
||||
("decider_count_65", "decider_count is at most 64 (Bifrost's per-call list)", card(deciders=[], decider_count=65)),
|
||||
("decider_count_string", "decider_count is an integer", card(deciders=[], decider_count="2")),
|
||||
("decider_count_null", "decider_count is never nil: it states the count even when the ids are withheld", card(deciders=[], decider_count=None)),
|
||||
("decider_count_fewer_than_the_ids", "rule R-card: a card that shows deciders shows all of them, so the count equals the ids (not expressible in the schema)", card(deciders=[OP, HM], decider_count=1)),
|
||||
("decider_count_more_than_the_ids_shown", "rule R-card: a visible list must be whole; a partial list is no view (not expressible in the schema)", card(deciders=[OP, HM], decider_count=3)),
|
||||
("no_deciders", "a fork with no decider cannot be answered", card(deciders=[])),
|
||||
("null_recommendation", "every card recommends an option", dict(card(), recommendation=None)),
|
||||
("unknown_key", "closed document", dict(card(), note="x")),
|
||||
("option_with_a_label", "options are {id,text}; the old format's display label is not a member (fork-card.md, Old format)", card(options=[dict(OPT[0], label="Yes"), OPT[1]])),
|
||||
|
|
@ -605,15 +565,12 @@ fork = {
|
|||
("duplicate_option_ids", "rule: option ids are unique (not expressible in the schema)", card(options=[OPT[0], dict(OPT[1], id=OPT[0]["id"])])),
|
||||
("recommendation_not_an_option", "rule: recommendation is an option id (not expressible in the schema)", card(recommendation="maybe")),
|
||||
("leans_not_an_option", "rule: leans is nil or an option id (not expressible in the schema)", card(leans="maybe")),
|
||||
("duplicate_deciders", "rule: deciders are unique (not expressible in the schema)", card(deciders=[OP, OP], decider_count=2)),
|
||||
("duplicate_deciders", "rule: deciders are unique (not expressible in the schema)", card(deciders=[OP, OP])),
|
||||
("ruled_from_on_an_open_card", "rule: a ruling that answers a fork closes it, so an open card cites no answer in ruled_from (not expressible in the schema)", card(ruled_from=["n1"])),
|
||||
("open_card_names_a_ruling", "rule: only a closed card names a ruling (not expressible in the schema)", card(ruling="cultlib-gaps:resolution:question.slot-mark-consent.n1")),
|
||||
("ruled_from_without_a_ruling", "rule: ruled_from lists answers a ruling was admitted from, so it needs the ruling (not expressible in the schema)", card(status="closed", ruled_from=["n1"])),
|
||||
("options_hash_of_another_card", "rule: options_hash equals the hash of the shown members (not expressible in the schema)", dict(card(), options_hash=STALE_H)),
|
||||
("raised_at_epoch_integer", "raised_at is an RFC3339 instant string or nil, not epoch milliseconds", card(raised_at=1762767000000)),
|
||||
("raised_at_date_only", "raised_at is an instant with a time and a Z, not a date", card(raised_at="2026-11-10")),
|
||||
("raised_at_missing", "map-mode writes every key: raised_at is required, nil when unknown", {k: v for k, v in card().items() if k != "raised_at"}),
|
||||
("deciders_65", "deciders holds at most 64 (Huginn's bound)", card(deciders=DECIDERS_64 + ["00000041-0000-4000-8000-000000000041"], decider_count=65)),
|
||||
("deciders_65", "deciders holds at most 64 (Huginn's bound)", card(deciders=DECIDERS_64 + ["00000041-0000-4000-8000-000000000041"])),
|
||||
]]},
|
||||
"request": {"valid": [{"name": n, "message": m, "messagePackHex": hx(m)} for n, m in
|
||||
[("choice_and_note", x1), ("choice_alone", pick), ("note_alone_clear_pick", words), ("retraction", retraction("n1")),
|
||||
|
|
@ -639,11 +596,8 @@ fork = {
|
|||
]]},
|
||||
"deskAnswer": {"valid": [{"name": n, "message": m, "messagePackHex": hx(m)} for n, m in
|
||||
[("counted", CNT), ("revision", REV), ("retraction", da("Retraction", "n1", None, retraction("n1"), sr("n8"))),
|
||||
("withheld_answerer", dict(CNT, answerer=None)),
|
||||
("refused_stale", da("Refused", None, "stale_fork", stale_body(), sx1)),
|
||||
("refused_closed_retraction_names_its_target", da("Refused", None, "fork_closed", retraction("n1"), sr("n9"))),
|
||||
("withheld_answerer_revision", dict(REV, answerer=None)),
|
||||
("withheld_answerer_retraction", dict(da("Retraction", "n1", None, retraction("n1", rid=NULLANS_RID), sr("n8")), answerer=None))]],
|
||||
("refused_closed_retraction_names_its_target", da("Refused", None, "fork_closed", retraction("n1"), sr("n9")))]],
|
||||
"invalid": [dict(bad(n, w, m), messagePackHex=hx(m)) for n, w, m in [
|
||||
("counted_with_a_code", "only Refused carries a code", dict(CNT, code="fork_closed")),
|
||||
("refused_without_a_code", "Refused carries a reason code", dict(REF, code=None)),
|
||||
|
|
@ -658,10 +612,6 @@ fork = {
|
|||
("outcome_lower_case", "outcome tags are Counted, Revision, Retraction, Refused", dict(CNT, outcome="counted")),
|
||||
("no_answerer", "answerer is stamped on every stored answer", dict(CNT, answerer="")),
|
||||
]]},
|
||||
"resultsInvalid": [dict(bad(n, w, m), messagePackHex=hx(m)) for n, w, m in [
|
||||
("answer_result_with_a_nil_answerer", "rule R-answer-result: the answer is the sender's own, stamped with their account; a nil answerer exists only inside a record shown to a non-decider (not expressible in the schema)", dict(result(dict(CNT, answerer=None)), counted_answer_id="n1")),
|
||||
("answer_result_with_a_nil_answerer_on_a_revision", "rule R-answer-result holds for every outcome: a Revision the sender just made is stamped with their account (not expressible in the schema)", dict(result(dict(REV, answerer=None)), counted_answer_id="n2")),
|
||||
("answer_result_with_a_nil_answerer_on_a_retraction", "rule R-answer-result holds for every outcome: a Retraction the sender just made is stamped with their account (not expressible in the schema)", dict(result(dict(da("Retraction", "n1", None, retraction("n1", rid=NULLANS_RID), sr("n8")), answerer=None)), counted_answer_id=None))]],
|
||||
"results": [{"name": r["name"], "httpStatus": r["httpStatus"], "message": r["message"], "messagePackHex": hx(r["message"])} for r in STORED_REFUSALS],
|
||||
"doorRefusals": [{"name": c, "httpStatus": HTTP[c], "message": refusal_doc(c), "messagePackHex": hx(refusal_doc(c))} for c in TAIL] + [{"name": "fork_too_large", "httpStatus": HTTP["fork_too_large"], "message": SHORT_PAGES[1]["steps"][0]["refusal"], "messagePackHex": SHORT_PAGES[1]["steps"][0]["refusalHex"]}],
|
||||
"unavailable": [{"name": "second_door_down", "httpStatus": 503, "message": unavailable_doc(30), "messagePackHex": hx(unavailable_doc(30))},
|
||||
|
|
@ -681,38 +631,20 @@ fork = {
|
|||
("fork_id_empty", "a read names a fork", {"schemaVersion": "gamecult.fork.read_request.v0", "mind": "gamecult-chat", "fork_id": ""}),
|
||||
("mind_empty", "a read names a Mind", {"schemaVersion": "gamecult.fork.read_request.v0", "mind": "", "fork_id": FID}),
|
||||
("mind_missing", "map-mode writes every key; the Mind is the route's", {"schemaVersion": "gamecult.fork.read_request.v0", "fork_id": FID})]]},
|
||||
"entry": {"valid": [{"name": n, "message": e, "messagePackHex": hx(e)} for n, e in [("open_no_answers", ENTRIES[1]), ("open_counted", ENTRIES[0]), ("closed_with_ruling_and_a_refusal", ENTRIES[2])]]
|
||||
+ [dict({"name": n, "message": e, "messagePackHex": hx(e)}, twin=t) for n, e, t in [("withheld_open_counted", WENTRIES[0], "open_counted"), ("withheld_closed_with_ruling_and_a_refusal", WENTRIES[2], "closed_with_ruling_and_a_refusal"), ("withheld_no_answers", WENTRIES[1], "open_no_answers")]]
|
||||
+ [{"name": "open_counted_revision_and_retraction", "message": REVRET, "messagePackHex": hx(REVRET)},
|
||||
dict({"name": "withheld_counted_revision_and_retraction", "message": WREVRET, "messagePackHex": hx(WREVRET)}, twin="open_counted_revision_and_retraction")],
|
||||
"entry": {"valid": [{"name": n, "message": e, "messagePackHex": hx(e)} for n, e in [("open_no_answers", ENTRIES[1]), ("open_counted", ENTRIES[0]), ("closed_with_ruling_and_a_refusal", ENTRIES[2])]],
|
||||
"invalid": [dict(bad(n, w, m), messagePackHex=hx(m)) for n, w, m in [
|
||||
("entry_without_record", "an entry holds the card and its record", {k: v for k, v in ENTRIES[0].items() if k != "record"}),
|
||||
("record_of_another_fork", "rule: the record names the card's fork (not expressible in the schema)", dict(ENTRIES[0], record=dict(ENTRIES[0]["record"], fork_id="gamecult-chat:question:q9"))),
|
||||
("record_under_another_hash", "rule: the record is under the card's options_hash (not expressible in the schema)", dict(ENTRIES[0], record=dict(ENTRIES[0]["record"], options_hash=STALE_H))),
|
||||
("counted_answer_not_derived", "rule: counted_answer_id is the first live Counted or Revision answer under the card's hash (not expressible in the schema)", dict(ENTRIES[0], record=dict(ENTRIES[0]["record"], counted_answer_id=None))),
|
||||
("answerer_count_missing", "map-mode writes every key: answerer_count is required", dict(ENTRIES[0], record={k: v for k, v in ENTRIES[0]["record"].items() if k != "answerer_count"})),
|
||||
("answerer_count_negative", "answerer_count is not negative", with_count(ENTRIES[0], -1)),
|
||||
("answerer_count_null", "answerer_count is never nil: a decider and a non-decider read the same number", with_count(ENTRIES[0], None)),
|
||||
("answerer_count_disagrees_with_the_ids", "rule R-record: in the decider view answerer_count is the number of distinct answerers (two here, not three; not expressible in the schema)", with_count(ENTRIES[2], 3)),
|
||||
("answerer_count_counts_answers_not_people", "rule R-record: two answers by one account are one answerer (not expressible in the schema)", with_count(TWICE, 2)),
|
||||
("withheld_card_with_a_named_answerer_among_nils", "rule R-entry: a withheld card beside one named answerer, even among nils (not expressible in the schema)", MIXED),
|
||||
("withheld_count_over_the_answers", "rule R-record: withheld answers are at most as many answerers as answers (two answers, count 3; not expressible in the schema)", with_count(WENTRIES[2], 3)),
|
||||
("withheld_count_zero_over_answers", "rule R-record: with at least one answer the count is at least 1 (not expressible in the schema)", with_count(WENTRIES[0], 0)),
|
||||
("withheld_count_over_no_answers", "rule R-record: with no answers the count is 0 in both views (not expressible in the schema)", with_count(WENTRIES[1], 1)),
|
||||
("decider_card_with_withheld_answers", "rule R-entry: a card that shows deciders is the decider view, so every answerer is named (not expressible in the schema)", withheld_answers(ENTRIES[0])),
|
||||
("withheld_card_with_named_answers", "rule R-entry, the leak: a card with deciders withheld beside an answer that still names its account (not expressible in the schema)", dict(ENTRIES[0], card=WENTRIES[0]["card"])),
|
||||
("card_hash_wrong", "rule: the server computes options_hash and a client recomputes it; a card with another hash is no card", dict(ENTRIES[1], card=dict(ENTRIES[1]["card"], options_hash=STALE_H), record=dict(ENTRIES[1]["record"], options_hash=STALE_H))),
|
||||
("withheld_revision_and_retraction_beside_a_named_retraction", "rule R-entry: a withheld card beside one named answerer, a Retraction's included (not expressible in the schema)", NAMED_RETRACTION),
|
||||
("withheld_revision_and_retraction_count_over_the_answers", "rule R-record: withheld answers are at most as many answerers as answers (three answers, count 4; not expressible in the schema)", with_count(WREVRET, 4))]]},
|
||||
("card_hash_wrong", "rule: the server computes options_hash and a client recomputes it; a card with another hash is no card", dict(ENTRIES[1], card=dict(ENTRIES[1]["card"], options_hash=STALE_H), record=dict(ENTRIES[1]["record"], options_hash=STALE_H)))]]},
|
||||
"listInvalid": [dict(bad(n, w, m), messagePackHex=hx(m)) for n, w, m in [
|
||||
("next_cursor_malformed", "a cursor is c<N>", dict(LISTS[0]["steps"][0]["response"], next_cursor="more")),
|
||||
("stamp_malformed", "a stamp is s<N>", dict(LISTS[0]["steps"][0]["response"], stamp="412")),
|
||||
("count_negative", "counts are not negative", dict(LISTS[0]["steps"][0]["response"], open_count=-1)),
|
||||
("a_page_mixes_views", "rule R-list: a page is one reader's view, so every entry agrees on whether the deciders are shown (not expressible in the schema)",
|
||||
dict(LISTS[1]["steps"][0]["response"], forks=[WENTRIES[0]] + LISTS[1]["steps"][0]["response"]["forks"][1:]))]]
|
||||
("count_negative", "counts are not negative", dict(LISTS[0]["steps"][0]["response"], open_count=-1))]]
|
||||
+ [dict(bad("two_cards_share_one_order", "rule: order is a unique rank per Mind; the refusal names both fork ids (not expressible in the schema)", DUP_LIST),
|
||||
messagePackHex=hx(DUP_LIST), refusal=order_shared(DUP_LIST["forks"])[0])],
|
||||
"listScenarios": [dict({"name": l["name"], "ordinal": l["ordinal"], "entries": l["entries"], "steps": l["steps"]}, **({"twin": l["twin"]} if "twin" in l else {})) for l in LISTS],
|
||||
"listScenarios": [{"name": l["name"], "ordinal": l["ordinal"], "entries": l["entries"], "steps": l["steps"]} for l in LISTS],
|
||||
"listShortPages": SHORT_PAGES,
|
||||
"readTooLarge": READ_TOO_LARGE,
|
||||
"answerTooLarge": ANSWER_TOO_LARGE,
|
||||
|
|
|
|||
|
|
@ -310,7 +310,7 @@ public static class CultDocumentMessagePackSerialization
|
|||
/// <summary>
|
||||
/// Reads a store file: exactly one complete MessagePack array of three slots (header, schema catalog, records), or an empty
|
||||
/// array, which is an empty store. A truncated file, bytes after the array, a value that is not an array, a first slot that
|
||||
/// is not a header string, a store with fewer or more slots, a record whose schema the catalog does not publish, two records under one key, or a body
|
||||
/// is not a header string, a store with fewer or more slots, a record whose schema the catalog does not publish, or a body
|
||||
/// that does not decode are refused with
|
||||
/// <see cref="CultStoreUnreadableException"/>, so a reader never takes part of a file for the whole and never skips a slot
|
||||
/// it does not understand. The rewriting paths use this same reader as their verdict on whether a file may be replaced.
|
||||
|
|
@ -366,7 +366,6 @@ public static class CultDocumentMessagePackSerialization
|
|||
}
|
||||
|
||||
RequirePublishedSchemas(snapshot);
|
||||
RequireUniqueKeys(snapshot);
|
||||
return (snapshot, catalogBytes, recordBytes);
|
||||
}
|
||||
catch (CultStoreUnreadableException)
|
||||
|
|
@ -398,17 +397,6 @@ public static class CultDocumentMessagePackSerialization
|
|||
}
|
||||
}
|
||||
|
||||
// A key names one record: identity is the key string alone, compared by ordinal code units (no case folding, no normalisation).
|
||||
private static void RequireUniqueKeys(CultPersistedStoreSnapshot snapshot)
|
||||
{
|
||||
var seen = new HashSet<string>(StringComparer.Ordinal);
|
||||
foreach (var record in snapshot.Records)
|
||||
{
|
||||
if (!seen.Add(record.Key))
|
||||
throw new CultStoreUnreadableException($"Record '{record.Key}' appears more than once.");
|
||||
}
|
||||
}
|
||||
|
||||
// Slot 0 of a non-empty store, read before anything after it: a binary or nil stamp is refused as such, and a string stamp is
|
||||
// judged by the caller's rule when it passes one. Bytes that do not open with an array and a string are left to the full read.
|
||||
private static void JudgeStamp(byte[] payload, Action<string>? requireReadable)
|
||||
|
|
|
|||
|
|
@ -1,371 +0,0 @@
|
|||
#nullable enable
|
||||
using System;
|
||||
using System.Buffers;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text.RegularExpressions;
|
||||
using GameCult.Caching;
|
||||
using GameCult.Caching.MessagePack;
|
||||
using MessagePack;
|
||||
using NUnit.Framework;
|
||||
|
||||
namespace GameCult.Caching.Tests
|
||||
{
|
||||
// tests/vectors/key-identity: a cache holds one record per key whatever its type, and a store holding a record no registered
|
||||
// type claims keeps it (variants:ruling:cache-refuses-retype, write-ownership-by-written-type, unheld-removal-refused-at-commit).
|
||||
// The C# reference is the only writer: under CULTCACHE_WRITE_VECTORS=1 the rows are run and what the reference answers is
|
||||
// written; every other run walks the committed rows against the reference and compares the files byte for byte.
|
||||
public class KeyIdentityVectorTests
|
||||
{
|
||||
private const string Vector = "key-identity";
|
||||
|
||||
private static readonly Type Alpha = Kind("KiAlpha", "tests.ki_alpha");
|
||||
private static readonly Type Beta = Kind("KiBeta", "tests.ki_beta");
|
||||
private static readonly Type Gamma = Kind("KiGamma", "tests.ki_gamma");
|
||||
|
||||
private static Type Kind(string type, string schema) =>
|
||||
EmittedDocumentTypes.Emit(type, schema, schema + ".v1", new[] { new EmittedDocumentTypes.Field("Text", typeof(string), 0) });
|
||||
|
||||
private static readonly Dictionary<string, Type> Types = new() { ["alpha"] = Alpha, ["beta"] = Beta, ["gamma"] = Gamma };
|
||||
|
||||
private sealed record Op(string Kind, string? File = null, string[]? Types = null, string? Type = null, string? Key = null, string? Text = null);
|
||||
|
||||
private sealed record Row(string Label, string[] Registered, Op[] Ops);
|
||||
|
||||
// [refusal key or nil, held [type, key, text], file [key, schema name] or nil, foreign keys, kept foreign keys], all sorted.
|
||||
private sealed record Expect(string? Refusal, string[][] Held, string[][]? File, string[] Foreign, string[] Kept);
|
||||
|
||||
private static Op Open(string? file) => new("open", file);
|
||||
private static Op Attach(string file, params string[] home) => new("attach", file, home);
|
||||
private static Op Reopen() => new("reopen");
|
||||
private static Op Put(string type, string key, string text) => new("put", Type: type, Key: key, Text: text);
|
||||
private static Op Delete(string type, string key) => new("delete", Type: type, Key: key);
|
||||
|
||||
private static readonly string[] AB = { "alpha", "beta" };
|
||||
private static readonly string[] A = { "alpha" };
|
||||
|
||||
// Only the ops are typed here; every verdict is what the reference answers.
|
||||
private static readonly Row[] Table =
|
||||
{
|
||||
new("retype-refused-held-in-file", AB, new[] { Open(null), Put("alpha", "k", "a"), Put("beta", "k", "b") }),
|
||||
new("retype-by-remove-then-add", AB, new[] { Open(null), Put("alpha", "k", "a"), Delete("alpha", "k"), Put("beta", "k", "b") }),
|
||||
new("retype-refused-memory-cache", AB, new[] { Open("memory"), Put("alpha", "k", "a"), Put("beta", "k", "b") }),
|
||||
new("same-type-replaces", A, new[] { Open(null), Put("alpha", "k", "a"), Put("alpha", "k", "b") }),
|
||||
new("retype-refused-after-reload", AB, new[] { Open(null), Put("alpha", "k", "a"), Reopen(), Put("beta", "k", "b") }),
|
||||
new("foreign-loads", A, new[] { Open("foreign-store.cc") }),
|
||||
new("foreign-kept-by-a-write", A, new[] { Open("foreign-store.cc"), Put("alpha", "m", "m") }),
|
||||
new("foreign-key-write-refused", A, new[] { Open("foreign-store.cc"), Put("alpha", "k", "x") }),
|
||||
new("foreign-key-delete-refused", A, new[] { Open("foreign-store.cc"), Delete("alpha", "k") }),
|
||||
new("foreign-claimed-by-a-later-registration", new[] { "alpha", "gamma" }, new[] { Open("foreign-store.cc") }),
|
||||
new("key-in-two-stores-refused", AB, new[] { Open("memory"), Attach("catalog-store.cc", "alpha"), Attach("run-store.cc", "beta") }),
|
||||
new("same-type-replaces-memory-cache", A, new[] { Open("memory"), Put("alpha", "k", "a"), Put("alpha", "k", "b") }),
|
||||
};
|
||||
|
||||
// The stores C# writes: [file name, registered types, (type, key, text) records]. Never typed by a runtime.
|
||||
private static readonly (string File, string[] Registered, (string Type, string Key, string Text)[] Records)[] Stores =
|
||||
{
|
||||
("foreign-store.cc", new[] { "alpha", "gamma" }, new[] { ("alpha", "j", "a"), ("gamma", "k", "g") }),
|
||||
("catalog-store.cc", new[] { "alpha" }, new[] { ("alpha", "qx", "a") }),
|
||||
("run-store.cc", new[] { "beta" }, new[] { ("beta", "qx", "b") }),
|
||||
};
|
||||
|
||||
private static CultDocumentRegistry RegistryOf(IEnumerable<string> names) => CultDocumentRegistry.ForTypes(names.Select(name => Types[name]).ToArray());
|
||||
|
||||
private static object Doc(string type, string text) => EmittedDocumentTypes.New(Types[type], ("Text", text));
|
||||
|
||||
private static byte[] StoreBytes(string[] registered, (string Type, string Key, string Text)[] records)
|
||||
{
|
||||
var directory = Path.Combine(Path.GetTempPath(), $"cultlib-kiv-{Guid.NewGuid():N}");
|
||||
Directory.CreateDirectory(directory);
|
||||
try
|
||||
{
|
||||
var path = Path.Combine(directory, "store.cc");
|
||||
using (var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = RegistryOf(registered), StoreFlushOnDispose = true }))
|
||||
{
|
||||
cache.Clock = () => VectorFiles.StoredAt;
|
||||
foreach (var record in records)
|
||||
cache.UpsertAsync(Types[record.Type], Doc(record.Type, record.Text), new CultRecordKey(record.Key)).GetAwaiter().GetResult();
|
||||
cache.FlushAsync().GetAwaiter().GetResult();
|
||||
}
|
||||
|
||||
return File.ReadAllBytes(path);
|
||||
}
|
||||
finally
|
||||
{
|
||||
Directory.Delete(directory, recursive: true);
|
||||
}
|
||||
}
|
||||
|
||||
private static CultPersistedStoreSnapshot Snapshot(byte[] bytes) => CultDocumentMessagePackSerialization.DeserializeSnapshot(bytes);
|
||||
|
||||
private static string Name(CultPersistedStoreSnapshot snapshot, CultPersistedRecord record) =>
|
||||
snapshot.SchemaCatalog.Single(entry => entry.SchemaId == record.SchemaId).SchemaName;
|
||||
|
||||
private static string Short(string schemaName) => Types.Single(pair => pair.Key == schemaName.Replace("tests.ki_", "")).Key;
|
||||
|
||||
// The key as a delimited token: not a piece of a longer word, schema name or path.
|
||||
private static bool Delimited(string message, string key) => Regex.IsMatch(message, $@"(?<![A-Za-z0-9_]){Regex.Escape(key)}(?![A-Za-z0-9_])");
|
||||
|
||||
private static string RecordBytes(CultPersistedRecord record) => $"{record.SchemaId}|{record.StoredAt}|{Convert.ToBase64String(record.Payload)}";
|
||||
|
||||
// Runs the ops and returns what the reference answers. Every op but the last must succeed; the last may refuse.
|
||||
private static Expect Observe(Row row)
|
||||
{
|
||||
var directory = Path.Combine(Path.GetTempPath(), $"cultlib-kiw-{Guid.NewGuid():N}");
|
||||
Directory.CreateDirectory(directory);
|
||||
var registry = RegistryOf(row.Registered);
|
||||
string? path = null;
|
||||
byte[]? before = null;
|
||||
CultCache? cache = null;
|
||||
try
|
||||
{
|
||||
Exception? refusal = null;
|
||||
string? refusedKey = null;
|
||||
for (var i = 0; i < row.Ops.Length; i++)
|
||||
{
|
||||
var op = row.Ops[i];
|
||||
try
|
||||
{
|
||||
switch (op.Kind)
|
||||
{
|
||||
case "open" or "reopen":
|
||||
cache?.Dispose();
|
||||
if (op.Kind == "open" && op.File == "memory")
|
||||
{
|
||||
cache = new CultCache(registry, CultCacheMessagePack.CreateCodec(registry));
|
||||
break;
|
||||
}
|
||||
|
||||
if (op.Kind == "open")
|
||||
{
|
||||
path = Path.Combine(directory, "store.cc");
|
||||
if (op.File != null) File.WriteAllBytes(path, VectorFiles.Committed(Vector, op.File));
|
||||
before = File.Exists(path) ? File.ReadAllBytes(path) : null;
|
||||
}
|
||||
|
||||
cache = CultCacheMessagePack.Create(path!, new CultCacheOpenOptions { Registry = registry, StoreFlushOnDispose = true });
|
||||
break;
|
||||
case "attach":
|
||||
{
|
||||
var attached = Path.Combine(directory, op.File!);
|
||||
File.WriteAllBytes(attached, VectorFiles.Committed(Vector, op.File!));
|
||||
var held = cache!.AllStoredDocuments.Select(stored => stored.Key.Value).ToHashSet(StringComparer.Ordinal);
|
||||
var shared = Snapshot(File.ReadAllBytes(attached)).Records.Select(record => record.Key).Where(held.Contains).ToArray();
|
||||
try
|
||||
{
|
||||
cache.AddBackingStore(new SingleFileMessagePackBackingStore(attached), op.Types!.Select(name => Types[name]).ToArray());
|
||||
cache.PullAllBackingStoresAsync().GetAwaiter().GetResult();
|
||||
}
|
||||
catch (InvalidOperationException) when (shared.Length == 1)
|
||||
{
|
||||
Assert.That(File.ReadAllBytes(attached), Is.EqualTo(VectorFiles.Committed(Vector, op.File!)), "a refused attach leaves the store as it was");
|
||||
throw;
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
case "put":
|
||||
cache!.Commit(batch => batch.Upsert(Types[op.Type!], Doc(op.Type!, op.Text!), new CultRecordKey(op.Key!)));
|
||||
break;
|
||||
case "delete":
|
||||
cache!.Commit(batch => batch.Remove(new CultRecordKey(op.Key!)));
|
||||
break;
|
||||
default:
|
||||
throw new InvalidOperationException(op.Kind);
|
||||
}
|
||||
}
|
||||
catch (Exception failure) when (i == row.Ops.Length - 1 &&
|
||||
(failure is CultSchemaConflictException || (op.Kind == "attach" && failure is InvalidOperationException)))
|
||||
{
|
||||
refusal = failure;
|
||||
}
|
||||
}
|
||||
|
||||
if (refusal is CultSchemaConflictException conflict)
|
||||
{
|
||||
refusedKey = conflict.RecordKey;
|
||||
Assert.That(row.Ops[^1].Key, Is.EqualTo(conflict.RecordKey), $"{row.Label}: the refusal names the key written or removed");
|
||||
}
|
||||
|
||||
if (refusal is InvalidOperationException && refusal is not CultSchemaConflictException)
|
||||
{
|
||||
// The key is read out of the refusal's own text, not taken from the row: the second-store refusal opens "Record <key> is held by".
|
||||
var named = Regex.Match(refusal.Message, @"^Record (\S+) is held by");
|
||||
Assert.That(named.Success, Is.True, $"{row.Label}: the refusal opens by naming a key: {refusal.Message}");
|
||||
refusedKey = named.Groups[1].Value;
|
||||
var shared = Snapshot(VectorFiles.Committed(Vector, "run-store.cc")).Records.Select(record => record.Key).ToArray();
|
||||
Assert.That(shared, Does.Contain(refusedKey), $"{row.Label}: the key the refusal names is the shared one");
|
||||
Assert.That(Regex.Matches(refusal.Message, Regex.Escape(refusedKey)).Count, Is.EqualTo(Regex.Matches(refusal.Message, $@"(?<![A-Za-z0-9_]){Regex.Escape(refusedKey)}(?![A-Za-z0-9_])").Count),
|
||||
$"{row.Label}: the key occurs in the refusal only as a delimited token, never inside a schema name or other text");
|
||||
}
|
||||
|
||||
if (refusal != null)
|
||||
Assert.That(Delimited(refusal.Message, refusedKey ?? "\0"), Is.True, $"{row.Label}: the refusal names the key as a token");
|
||||
|
||||
var heldNow = cache!.AllStoredDocuments
|
||||
.Select(stored => new[] { Short(stored.Descriptor.SchemaName), stored.Key.Value, (string)EmittedDocumentTypes.Read(stored.Document, "Text")! })
|
||||
.OrderBy(entry => entry[0], StringComparer.Ordinal).ThenBy(entry => entry[1], StringComparer.Ordinal).ToArray();
|
||||
var foreign = cache.BackingStores.SelectMany(store => store.ForeignRecords).Select(record => record.Key).OrderBy(key => key, StringComparer.Ordinal).ToArray();
|
||||
|
||||
string[][]? file = null;
|
||||
var kept = Array.Empty<string>();
|
||||
if (path != null)
|
||||
{
|
||||
cache.FlushAsync().GetAwaiter().GetResult();
|
||||
var after = Snapshot(File.ReadAllBytes(path));
|
||||
file = after.Records.OrderBy(record => record.Key, StringComparer.Ordinal).Select(record => new[] { record.Key, Name(after, record) }).ToArray();
|
||||
var start = before == null ? null : Snapshot(before);
|
||||
kept = foreign.Where(key =>
|
||||
{
|
||||
var original = start?.Records.SingleOrDefault(record => record.Key == key);
|
||||
return original != null && RecordBytes(original) == RecordBytes(after.Records.Single(record => record.Key == key));
|
||||
}).ToArray();
|
||||
}
|
||||
|
||||
return new Expect(refusedKey, heldNow, file, foreign, kept);
|
||||
}
|
||||
finally
|
||||
{
|
||||
cache?.Dispose();
|
||||
Directory.Delete(directory, recursive: true);
|
||||
}
|
||||
}
|
||||
|
||||
// ---- rows.msgpack: [label, registered, ops, expect] ----
|
||||
|
||||
private static void WriteStrings(ref MessagePackWriter writer, IEnumerable<string?> values)
|
||||
{
|
||||
var list = values.ToArray();
|
||||
writer.WriteArrayHeader(list.Length);
|
||||
foreach (var value in list)
|
||||
if (value == null) writer.WriteNil(); else writer.Write(value);
|
||||
}
|
||||
|
||||
private static void WriteRows(ref MessagePackWriter writer, Row row, Expect expect)
|
||||
{
|
||||
writer.WriteArrayHeader(4);
|
||||
writer.Write(row.Label);
|
||||
WriteStrings(ref writer, row.Registered);
|
||||
writer.WriteArrayHeader(row.Ops.Length);
|
||||
foreach (var op in row.Ops)
|
||||
{
|
||||
var fields = op.Kind switch
|
||||
{
|
||||
"open" => new object?[] { op.Kind, op.File },
|
||||
"attach" => new object?[] { op.Kind, op.File, op.Types },
|
||||
"reopen" => new object?[] { op.Kind },
|
||||
"put" => new object?[] { op.Kind, op.Type, op.Key, op.Text },
|
||||
_ => new object?[] { op.Kind, op.Type, op.Key },
|
||||
};
|
||||
writer.WriteArrayHeader(fields.Length);
|
||||
foreach (var field in fields)
|
||||
switch (field)
|
||||
{
|
||||
case null: writer.WriteNil(); break;
|
||||
case string[] list: WriteStrings(ref writer, list); break;
|
||||
default: writer.Write((string)field); break;
|
||||
}
|
||||
}
|
||||
|
||||
writer.WriteArrayHeader(5);
|
||||
if (expect.Refusal == null) writer.WriteNil(); else WriteStrings(ref writer, new[] { expect.Refusal });
|
||||
writer.WriteArrayHeader(expect.Held.Length);
|
||||
foreach (var held in expect.Held) WriteStrings(ref writer, held);
|
||||
if (expect.File == null) writer.WriteNil();
|
||||
else
|
||||
{
|
||||
writer.WriteArrayHeader(expect.File.Length);
|
||||
foreach (var record in expect.File) WriteStrings(ref writer, record);
|
||||
}
|
||||
|
||||
WriteStrings(ref writer, expect.Foreign);
|
||||
WriteStrings(ref writer, expect.Kept);
|
||||
}
|
||||
|
||||
private static byte[] RowsFile()
|
||||
{
|
||||
var buffer = new ArrayBufferWriter<byte>();
|
||||
var writer = new MessagePackWriter(buffer);
|
||||
writer.WriteArrayHeader(Table.Length);
|
||||
foreach (var row in Table)
|
||||
WriteRows(ref writer, row, Observe(row));
|
||||
writer.Flush();
|
||||
return buffer.WrittenSpan.ToArray();
|
||||
}
|
||||
|
||||
private static string[] ReadStrings(ref MessagePackReader reader)
|
||||
{
|
||||
var values = new string[reader.ReadArrayHeader()];
|
||||
for (var i = 0; i < values.Length; i++) values[i] = reader.ReadString()!;
|
||||
return values;
|
||||
}
|
||||
|
||||
private static (Row Row, Expect Expect) ReadRow(ref MessagePackReader reader)
|
||||
{
|
||||
Assert.That(reader.ReadArrayHeader(), Is.EqualTo(4));
|
||||
var label = reader.ReadString()!;
|
||||
var registered = ReadStrings(ref reader);
|
||||
var ops = new Op[reader.ReadArrayHeader()];
|
||||
for (var i = 0; i < ops.Length; i++)
|
||||
{
|
||||
var length = reader.ReadArrayHeader();
|
||||
var kind = reader.ReadString()!;
|
||||
ops[i] = kind switch
|
||||
{
|
||||
"open" => new Op(kind, reader.TryReadNil() ? null : reader.ReadString()),
|
||||
"attach" => new Op(kind, reader.ReadString(), ReadStrings(ref reader)),
|
||||
"reopen" => new Op(kind),
|
||||
"put" => new Op(kind, Type: reader.ReadString(), Key: reader.ReadString(), Text: reader.ReadString()),
|
||||
"delete" => new Op(kind, Type: reader.ReadString(), Key: reader.ReadString()),
|
||||
_ => throw new InvalidOperationException(kind),
|
||||
};
|
||||
Assert.That(length, Is.EqualTo(kind switch { "reopen" => 1, "open" => 2, "delete" => 3, "attach" => 3, _ => 4 }), $"{label}: op {kind}");
|
||||
}
|
||||
|
||||
Assert.That(reader.ReadArrayHeader(), Is.EqualTo(5));
|
||||
var refusal = reader.TryReadNil() ? null : ReadStrings(ref reader).Single();
|
||||
var held = new string[reader.ReadArrayHeader()][];
|
||||
for (var i = 0; i < held.Length; i++) held[i] = ReadStrings(ref reader);
|
||||
string[][]? file = null;
|
||||
if (!reader.TryReadNil())
|
||||
{
|
||||
file = new string[reader.ReadArrayHeader()][];
|
||||
for (var i = 0; i < file.Length; i++) file[i] = ReadStrings(ref reader);
|
||||
}
|
||||
|
||||
var foreign = ReadStrings(ref reader);
|
||||
var kept = ReadStrings(ref reader);
|
||||
return (new Row(label, registered, ops), new Expect(refusal, held, file, foreign, kept));
|
||||
}
|
||||
|
||||
private static string Render(Expect expect) =>
|
||||
$"refusal={expect.Refusal ?? "-"} held={string.Join(";", expect.Held.Select(h => string.Join(",", h)))} " +
|
||||
$"file={(expect.File == null ? "-" : string.Join(";", expect.File.Select(f => string.Join(",", f))))} " +
|
||||
$"foreign={string.Join(",", expect.Foreign)} kept={string.Join(",", expect.Kept)}";
|
||||
|
||||
[Test]
|
||||
public void KeyIdentityRowsHoldTheirVerdicts()
|
||||
{
|
||||
foreach (var store in Stores)
|
||||
VectorFiles.Check(Vector, store.File, StoreBytes(store.Registered, store.Records));
|
||||
VectorFiles.Check(Vector, "rows.msgpack", RowsFile());
|
||||
|
||||
var reader = new MessagePackReader(VectorFiles.Committed(Vector, "rows.msgpack"));
|
||||
var count = reader.ReadArrayHeader();
|
||||
Assert.That(count, Is.EqualTo(Table.Length));
|
||||
var walked = 0;
|
||||
var failed = new List<string>();
|
||||
for (var i = 0; i < count; i++)
|
||||
{
|
||||
var (row, expect) = ReadRow(ref reader);
|
||||
var observed = Observe(row);
|
||||
if (Render(observed) == Render(expect)) walked++;
|
||||
else failed.Add($"{row.Label}: committed {Render(expect)} but the reference answers {Render(observed)}");
|
||||
}
|
||||
|
||||
Assert.That(failed, Is.Empty);
|
||||
Assert.That(walked, Is.EqualTo(12), "rows walked");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -7,7 +7,6 @@ using System.Linq;
|
|||
using GameCult.Caching.MessagePack;
|
||||
using MessagePack;
|
||||
using NUnit.Framework;
|
||||
using NUnit.Framework.Constraints;
|
||||
using static GameCult.Caching.Tests.ElementIdTests;
|
||||
|
||||
namespace GameCult.Caching.Tests
|
||||
|
|
@ -62,39 +61,6 @@ namespace GameCult.Caching.Tests
|
|||
.Select(line => line.Split((char[]?)null, StringSplitOptions.RemoveEmptyEntries))
|
||||
.Select(cells => new TestCaseData(cells[0], cells[CSharp] == "reads").SetName($"{cells[0]} {cells[CSharp]}"));
|
||||
|
||||
// The row `@directory` has no bytes: the walker makes an empty directory at the store path, and a runtime must refuse it untouched.
|
||||
private const string DirectoryRow = "@directory";
|
||||
|
||||
private static byte[] Place(string path, string vector)
|
||||
{
|
||||
if (vector != DirectoryRow)
|
||||
{
|
||||
var bytes = File.ReadAllBytes(Path.Combine(VectorRoot(), vector));
|
||||
File.WriteAllBytes(path, bytes);
|
||||
return bytes;
|
||||
}
|
||||
|
||||
if (File.Exists(path))
|
||||
File.Delete(path);
|
||||
Directory.CreateDirectory(path);
|
||||
return Array.Empty<byte>();
|
||||
}
|
||||
|
||||
// A directory at the store path is refused as IOException ("not a store file", CultCache.cs): not a store that is unreadable.
|
||||
private static IResolveConstraint Refusal(string vector) =>
|
||||
vector == DirectoryRow ? Throws.TypeOf<IOException>() : Throws.TypeOf<CultStoreUnreadableException>();
|
||||
|
||||
private static void AssertUntouched(string path, string vector, byte[] bytes, string message)
|
||||
{
|
||||
if (vector == DirectoryRow)
|
||||
{
|
||||
Assert.That(Directory.Exists(path), message);
|
||||
Assert.That(Directory.EnumerateFileSystemEntries(path), Is.Empty, message);
|
||||
}
|
||||
else
|
||||
Assert.That(File.ReadAllBytes(path), Is.EqualTo(bytes), message);
|
||||
}
|
||||
|
||||
private static bool HoldsAVariant(byte[] bytes) =>
|
||||
CultDocumentMessagePackSerialization.DeserializeSnapshot(bytes).Records.Any(record => record.Variant != null);
|
||||
|
||||
|
|
@ -108,33 +74,18 @@ namespace GameCult.Caching.Tests
|
|||
return path;
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void ADuplicateKeyIsRefusedNamingTheKeyAndTheFile()
|
||||
{
|
||||
var path = Path.Combine(_directory, "dup.cc");
|
||||
File.WriteAllBytes(path, File.ReadAllBytes(Path.Combine(VectorRoot(), "dup-key-same-schema-v3.bin")));
|
||||
|
||||
Assert.That(() => CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry }),
|
||||
Throws.TypeOf<CultStoreUnreadableException>()
|
||||
.With.Property(nameof(CultStoreUnreadableException.Path)).EqualTo(path)
|
||||
.And.Message.Contains("more than once"));
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(Vectors))]
|
||||
public void OpenReadsExactlyTheFilesTheVectorsSayItReads(string vector, bool reads)
|
||||
{
|
||||
var path = Path.Combine(_directory, "open.cc");
|
||||
var bytes = Place(path, vector);
|
||||
File.WriteAllBytes(path, File.ReadAllBytes(Path.Combine(VectorRoot(), vector)));
|
||||
|
||||
void Open() => CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry }).Dispose();
|
||||
|
||||
if (reads)
|
||||
Assert.DoesNotThrow(Open);
|
||||
else
|
||||
{
|
||||
Assert.That(Open, vector == DirectoryRow ? Refusal(vector) : Throws.TypeOf<CultStoreUnreadableException>().With.Property(nameof(CultStoreUnreadableException.Path)).EqualTo(path));
|
||||
AssertUntouched(path, vector, bytes, "a store this runtime cannot open was changed by the attempt");
|
||||
}
|
||||
Assert.That(Open, Throws.TypeOf<CultStoreUnreadableException>().With.Property(nameof(CultStoreUnreadableException.Path)).EqualTo(path));
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(Vectors))]
|
||||
|
|
@ -142,7 +93,8 @@ namespace GameCult.Caching.Tests
|
|||
{
|
||||
var path = Seed("flush.cc");
|
||||
using var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry });
|
||||
var bytes = Place(path, vector);
|
||||
var bytes = File.ReadAllBytes(Path.Combine(VectorRoot(), vector));
|
||||
File.WriteAllBytes(path, bytes);
|
||||
var store = cache.BackingStores[0];
|
||||
|
||||
if (reads && HoldsAVariant(bytes))
|
||||
|
|
@ -162,8 +114,8 @@ namespace GameCult.Caching.Tests
|
|||
return;
|
||||
}
|
||||
|
||||
Assert.That(() => store.PushAll(), Refusal(vector));
|
||||
AssertUntouched(path, vector, bytes, "a file this runtime cannot read was rewritten");
|
||||
Assert.That(() => store.PushAll(), Throws.TypeOf<CultStoreUnreadableException>());
|
||||
Assert.That(File.ReadAllBytes(path), Is.EqualTo(bytes), "a file this runtime cannot read was rewritten");
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(Vectors))]
|
||||
|
|
@ -171,7 +123,8 @@ namespace GameCult.Caching.Tests
|
|||
{
|
||||
var path = Seed("commit.cc");
|
||||
using var cache = CultCacheMessagePack.Create(path, new CultCacheOpenOptions { Registry = Registry });
|
||||
var bytes = Place(path, vector);
|
||||
var bytes = File.ReadAllBytes(Path.Combine(VectorRoot(), vector));
|
||||
File.WriteAllBytes(path, bytes);
|
||||
|
||||
if (reads && HoldsAVariant(bytes))
|
||||
{
|
||||
|
|
@ -188,8 +141,8 @@ namespace GameCult.Caching.Tests
|
|||
return;
|
||||
}
|
||||
|
||||
Assert.That(() => cache.Commit(batch => batch.Upsert(typeof(IdDeck), new IdDeck { Name = "e" }, new CultRecordKey("e"))), Refusal(vector));
|
||||
AssertUntouched(path, vector, bytes, "a file this runtime cannot read was rewritten");
|
||||
Assert.That(() => cache.Commit(batch => batch.Upsert(typeof(IdDeck), new IdDeck { Name = "e" }, new CultRecordKey("e"))), Throws.TypeOf<CultStoreUnreadableException>());
|
||||
Assert.That(File.ReadAllBytes(path), Is.EqualTo(bytes), "a file this runtime cannot read was rewritten");
|
||||
}
|
||||
|
||||
private string DirectoryStore(string name)
|
||||
|
|
@ -314,9 +267,6 @@ namespace GameCult.Caching.Tests
|
|||
Assert.That(() => cache.Commit(batch => batch.Upsert(typeof(IdDeck), new IdDeck { Name = "e" }, new CultRecordKey("e"))),
|
||||
Throws.TypeOf<IOException>().With.Message.Contains("directory"));
|
||||
|
||||
Assert.That(() => cache.Commit(batch => batch.Remove(new CultRecordKey("e"))),
|
||||
Throws.TypeOf<IOException>().With.Message.Contains("directory"));
|
||||
|
||||
Assert.That(Directory.GetFileSystemEntries(path), Is.Empty);
|
||||
Assert.That(Directory.GetFileSystemEntries(later), Is.Empty);
|
||||
}
|
||||
|
|
@ -717,7 +667,7 @@ namespace GameCult.Caching.Tests
|
|||
[Test]
|
||||
public void EveryVectorInTheFolderHasAManifestRow()
|
||||
{
|
||||
var rows = Vectors().Select(row => (string)row.Arguments[0]).Where(name => !name.StartsWith("..", StringComparison.Ordinal) && name != DirectoryRow).OrderBy(name => name, StringComparer.Ordinal);
|
||||
var rows = Vectors().Select(row => (string)row.Arguments[0]).Where(name => !name.StartsWith("..", StringComparison.Ordinal)).OrderBy(name => name, StringComparer.Ordinal);
|
||||
var files = Directory.GetFiles(VectorRoot(), "*.bin").Select(Path.GetFileName).OrderBy(name => name, StringComparer.Ordinal);
|
||||
Assert.That(rows, Is.EqualTo(files));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -18,16 +18,43 @@ namespace GameCult.Caching.Tests
|
|||
// compares them byte for byte. Other runtimes are held to the same text and the same verdicts.
|
||||
public class UniqueIndexVectorTests
|
||||
{
|
||||
private static readonly DateTimeOffset StoredAt = VectorFiles.StoredAt;
|
||||
private static bool Writing => VectorFiles.Writing;
|
||||
private const string WriteVariable = "CULTCACHE_WRITE_VECTORS";
|
||||
private static readonly DateTimeOffset StoredAt = new(2026, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
private static bool Writing => Environment.GetEnvironmentVariable(WriteVariable) == "1";
|
||||
|
||||
private static string VectorDirectory() => VectorFiles.Directory("unique-index");
|
||||
private static string VectorDirectory()
|
||||
{
|
||||
for (var dir = new DirectoryInfo(TestContext.CurrentContext.TestDirectory); dir != null; dir = dir.Parent)
|
||||
{
|
||||
var vectors = Path.Combine(dir.FullName, "tests", "vectors");
|
||||
if (Directory.Exists(vectors)) return Path.Combine(vectors, "unique-index");
|
||||
}
|
||||
|
||||
throw new DirectoryNotFoundException($"tests/vectors not found above {TestContext.CurrentContext.TestDirectory}.");
|
||||
}
|
||||
|
||||
private static string VectorPath(string name) => Path.Combine(VectorDirectory(), name);
|
||||
|
||||
private static void Check(string name, byte[] reference) => VectorFiles.Check("unique-index", name, reference);
|
||||
private static void Check(string name, byte[] reference)
|
||||
{
|
||||
var path = VectorPath(name);
|
||||
if (Writing)
|
||||
{
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
|
||||
File.WriteAllBytes(path, reference);
|
||||
return;
|
||||
}
|
||||
|
||||
private static byte[] Committed(string name) => VectorFiles.Committed("unique-index", name);
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
Assert.That(Convert.ToHexString(reference), Is.EqualTo(Convert.ToHexString(File.ReadAllBytes(path))), $"vector {name}");
|
||||
}
|
||||
|
||||
private static byte[] Committed(string name)
|
||||
{
|
||||
var path = VectorPath(name);
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
return File.ReadAllBytes(path);
|
||||
}
|
||||
|
||||
// ---- values.msgpack: [label, kind, value, text-or-nil] ----
|
||||
|
||||
|
|
@ -289,12 +316,6 @@ namespace GameCult.Caching.Tests
|
|||
EmittedDocumentTypes.Emit("UqVecAliasIndex", "tests.uq_alias", "tests.uq_alias.v1",
|
||||
new EmittedDocumentTypes.Field[] { new("Sku", typeof(string), 0, Index: "Sku"), new("Alt", typeof(string), 1, Index: "alt") }),
|
||||
"Alt"),
|
||||
new("tests.uq_wide",
|
||||
EmittedDocumentTypes.Emit("UqVecWideRole", "tests.uq_wide", "tests.uq_wide.v1",
|
||||
new EmittedDocumentTypes.Field[] { new("Sku", typeof(string), 0, Role: "\uFF21"), new("Alt", typeof(string), 1, Role: "\U00010400") }),
|
||||
EmittedDocumentTypes.Emit("UqVecWideIndex", "tests.uq_wide", "tests.uq_wide.v1",
|
||||
new EmittedDocumentTypes.Field[] { new("Sku", typeof(string), 0, Index: "\uFF21"), new("Alt", typeof(string), 1, Index: "\U00010400") }),
|
||||
"Alt"),
|
||||
Plain("Alpha", "tests.uq_alpha"),
|
||||
Plain("Beta", "tests.uq_beta"),
|
||||
};
|
||||
|
|
@ -312,7 +333,6 @@ namespace GameCult.Caching.Tests
|
|||
private const string Item = "tests.unique_item";
|
||||
private const string Pair = "tests.uq_pair";
|
||||
private const string Alias = "tests.uq_alias";
|
||||
private const string Wide = "tests.uq_wide";
|
||||
private const string Alpha = "tests.uq_alpha";
|
||||
private const string Beta = "tests.uq_beta";
|
||||
|
||||
|
|
@ -331,16 +351,6 @@ namespace GameCult.Caching.Tests
|
|||
private static readonly Rec[] RepullBefore = Items_(("item:a1", "A"), ("item:b1", "C"));
|
||||
private static readonly Rec[] RepullAfter = Items_(("item:a1", "A"), ("item:b1", "B"), ("item:b2", "B"), ("item:c1", "A"));
|
||||
private static readonly Rec[] KeyCaseRecords = Items_(("a", "S"), ("z", "S"), ("B", "T"), ("C", "T"));
|
||||
private static readonly Rec[] HolderCaseRecords = Items_(("a", "S"), ("A", "S"));
|
||||
private static readonly Rec[] CrossSchemaRecords = { new("x:1", Alpha, "P"), new("y:1", Beta, "P") };
|
||||
// Distinct under any trim, case-fold, NFC or NFKC of the value: 'S' and 's', 'S' and 'S ', NFC and NFD, fullwidth S.
|
||||
private static readonly Rec[] DistinctValueRecords = Items_(
|
||||
("k:1", "S"), ("k:2", "s"), ("k:3", "S "), ("k:4", "\u00E9"), ("k:5", "e\u0301"), ("k:6", "\uFF33"), ("k:7", " S"));
|
||||
private static readonly Rec[] NormalKeyRecords = Items_(("\u00E9", "S"), ("e\u0301", "S"));
|
||||
// Three holders of one value: code-unit order is e+U+0302, f, U+00E9; NFC would order f, U+00E9, e+U+0302 and NFD U+00E9, e+U+0302, f.
|
||||
private static readonly Rec[] ThreeHolderRecords = Items_(("e\u0302", "S"), ("f", "S"), ("\u00E9", "S"));
|
||||
// Both records violate both aliases: by code unit U+10400 (D801 DC00) precedes U+FF21, by code point U+FF21 does.
|
||||
private static readonly Rec[] WideRecords = { new("item:a", Wide, "S", "X"), new("item:b", Wide, "S", "X") };
|
||||
private static readonly Rec[] KeyPunctuationRecords = Items_(("_", "S"), ("a", "S"), ("Z", "T"), ("b", "T"));
|
||||
private static readonly Rec[] SurrogateGroupRecords = Items_(("\uE000", "S"), ("\uE000b", "S"), ("\U0001F600", "T"), ("\U0001F600b", "T"));
|
||||
|
||||
|
|
@ -350,9 +360,6 @@ namespace GameCult.Caching.Tests
|
|||
("repull-before.cc", RepullBefore), ("repull-after.cc", RepullAfter), ("alias-store.cc", AliasRecords),
|
||||
("key-case-store.cc", KeyCaseRecords), ("key-punctuation-store.cc", KeyPunctuationRecords),
|
||||
("surrogate-groups-store.cc", SurrogateGroupRecords),
|
||||
("holder-case-store.cc", HolderCaseRecords),
|
||||
("cross-schema-store.cc", CrossSchemaRecords), ("distinct-values-store.cc", DistinctValueRecords),
|
||||
("normal-keys-store.cc", NormalKeyRecords), ("normal-three-store.cc", ThreeHolderRecords), ("wide-store.cc", WideRecords),
|
||||
};
|
||||
|
||||
private static readonly Refusal SurrogateRefusal = new("k\U0001F600", "k\uE000", Item, "Sku");
|
||||
|
|
@ -361,10 +368,6 @@ namespace GameCult.Caching.Tests
|
|||
private static readonly Refusal AliasRefusal = new("item:a", "item:b", Alias, "Sku");
|
||||
private static readonly Refusal RepullRefusal = new("item:a1", "item:c1", Item, "Sku");
|
||||
private static readonly Refusal KeyCaseRefusal = new("B", "C", Item, "Sku");
|
||||
private static readonly Refusal HolderCaseRefusal = new("A", "a", Item, "Sku");
|
||||
private static readonly Refusal NormalKeyRefusal = new("e\u0301", "\u00E9", Item, "Sku");
|
||||
private static readonly Refusal ThreeHolderRefusal = new("e\u0302", "f", Item, "Sku");
|
||||
private static readonly Refusal WideRefusal = new("item:a", "item:b", Wide, "\U00010400");
|
||||
private static readonly Refusal KeyPunctuationRefusal = new("Z", "b", Item, "Sku");
|
||||
private static readonly Refusal SurrogateGroupRefusal = new("\U0001F600", "\U0001F600b", Item, "Sku");
|
||||
|
||||
|
|
@ -399,29 +402,6 @@ namespace GameCult.Caching.Tests
|
|||
new("load-key-case", new[] { Open("key-case-store.cc") }, KeyCaseRefusal),
|
||||
new("load-key-punctuation", new[] { Open("key-punctuation-store.cc") }, KeyPunctuationRefusal),
|
||||
new("load-surrogate-groups", new[] { Open("surrogate-groups-store.cc") }, SurrogateGroupRefusal),
|
||||
// Rows 23-24: two holders of one value that differ only by case are two holders; a case-folding dedupe would see one.
|
||||
new("batch-holders-by-case", new[] { Open(), Commit(HolderCaseRecords) }, HolderCaseRefusal),
|
||||
new("load-holders-by-case", new[] { Open("holder-case-store.cc") }, HolderCaseRefusal),
|
||||
// Row 25: the held `a` is replaced only by the exact key `a`, never by `A`.
|
||||
new("batch-replaced-by-case", new[] { Open(), Commit(Items_(("a", "S"))), Commit(Items_(("A", "S"))) }, HolderCaseRefusal),
|
||||
// Rows 26-31: identity is exact. One value in two schemas is two groups; values differing in case, trailing or leading space, normalisation form or width are two values; keys differing only by normalisation form are two holders.
|
||||
new("batch-value-in-two-schemas", new[] { Open(), Commit(CrossSchemaRecords) }, null),
|
||||
new("load-value-in-two-schemas", new[] { Open("cross-schema-store.cc") }, null),
|
||||
new("batch-distinct-values", new[] { Open(), Commit(DistinctValueRecords) }, null),
|
||||
new("load-distinct-values", new[] { Open("distinct-values-store.cc") }, null),
|
||||
new("batch-keys-by-normal-form", new[] { Open(), Commit(NormalKeyRecords) }, NormalKeyRefusal),
|
||||
new("load-keys-by-normal-form", new[] { Open("normal-keys-store.cc") }, NormalKeyRefusal),
|
||||
// Rows 32-33: one landing record sharing two unique aliases with a held one; the least alias by code unit is named ('Alt' before 'Sku'; 'Sku' before 'alt').
|
||||
new("put-onto-a-held-pair", new[] { Open(), Commit(new Rec("m", Pair, "S", "X")), Commit(new Rec("a", Pair, "S", "X")) },
|
||||
new Refusal("a", "m", Pair, "Alt")),
|
||||
new("put-onto-a-held-alias-pair", new[] { Open(), Commit(new Rec("m", Alias, "S", "X")), Commit(new Rec("a", Alias, "S", "X")) },
|
||||
new Refusal("a", "m", Alias, "Sku")),
|
||||
// Rows 34-35: three holders of one value; the two least by code unit are named, which no NFC, NFD or culture sort of the keys yields.
|
||||
new("batch-three-holders-normal-form", new[] { Open(), Commit(ThreeHolderRecords) }, ThreeHolderRefusal),
|
||||
new("load-three-holders-normal-form", new[] { Open("normal-three-store.cc") }, ThreeHolderRefusal),
|
||||
// Rows 36-37: two records violating two aliases, U+FF21 and U+10400; aliases compare by code unit, so U+10400 is named.
|
||||
new("batch-alias-code-unit", new[] { Open(), Commit(WideRecords) }, WideRefusal),
|
||||
new("load-alias-code-unit", new[] { Open("wide-store.cc") }, WideRefusal),
|
||||
};
|
||||
|
||||
private static byte[] RefusalStoreBytes(Rec[] records)
|
||||
|
|
@ -542,7 +522,6 @@ namespace GameCult.Caching.Tests
|
|||
var count = reader.ReadArrayHeader();
|
||||
Assert.That(count, Is.EqualTo(Cases.Length));
|
||||
var walked = 0;
|
||||
var failed = new List<string>();
|
||||
for (var i = 0; i < count; i++)
|
||||
{
|
||||
Assert.That(reader.ReadArrayHeader(), Is.EqualTo(3));
|
||||
|
|
@ -603,10 +582,6 @@ namespace GameCult.Caching.Tests
|
|||
|
||||
walked++;
|
||||
}
|
||||
catch (AssertionException failure)
|
||||
{
|
||||
failed.Add($"{label}: {failure.Message}");
|
||||
}
|
||||
finally
|
||||
{
|
||||
cache?.Dispose();
|
||||
|
|
@ -614,8 +589,7 @@ namespace GameCult.Caching.Tests
|
|||
}
|
||||
}
|
||||
|
||||
Assert.That(failed, Is.Empty, "rows that failed");
|
||||
Assert.That(walked, Is.EqualTo(37));
|
||||
Assert.That(walked, Is.EqualTo(22));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,50 +0,0 @@
|
|||
#nullable enable
|
||||
using System;
|
||||
using System.IO;
|
||||
using NUnit.Framework;
|
||||
|
||||
namespace GameCult.Caching.Tests
|
||||
{
|
||||
// The one place a C#-written vector directory is found, written and compared. The C# reference is the only writer
|
||||
// (CULTCACHE_WRITE_VECTORS=1, never set in CI); every other run reads the committed bytes and compares them byte for byte.
|
||||
internal static class VectorFiles
|
||||
{
|
||||
private const string WriteVariable = "CULTCACHE_WRITE_VECTORS";
|
||||
|
||||
internal static readonly DateTimeOffset StoredAt = new(2026, 1, 1, 0, 0, 0, TimeSpan.Zero);
|
||||
|
||||
internal static bool Writing => Environment.GetEnvironmentVariable(WriteVariable) == "1";
|
||||
|
||||
internal static string Directory(string vector)
|
||||
{
|
||||
for (var dir = new DirectoryInfo(TestContext.CurrentContext.TestDirectory); dir != null; dir = dir.Parent)
|
||||
{
|
||||
var vectors = System.IO.Path.Combine(dir.FullName, "tests", "vectors");
|
||||
if (System.IO.Directory.Exists(vectors)) return System.IO.Path.Combine(vectors, vector);
|
||||
}
|
||||
|
||||
throw new DirectoryNotFoundException($"tests/vectors not found above {TestContext.CurrentContext.TestDirectory}.");
|
||||
}
|
||||
|
||||
internal static void Check(string vector, string name, byte[] reference)
|
||||
{
|
||||
var path = System.IO.Path.Combine(Directory(vector), name);
|
||||
if (Writing)
|
||||
{
|
||||
System.IO.Directory.CreateDirectory(System.IO.Path.GetDirectoryName(path)!);
|
||||
File.WriteAllBytes(path, reference);
|
||||
return;
|
||||
}
|
||||
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
Assert.That(Convert.ToHexString(reference), Is.EqualTo(Convert.ToHexString(File.ReadAllBytes(path))), $"vector {name}");
|
||||
}
|
||||
|
||||
internal static byte[] Committed(string vector, string name)
|
||||
{
|
||||
var path = System.IO.Path.Combine(Directory(vector), name);
|
||||
Assert.That(File.Exists(path), Is.True, $"vector {name} is committed");
|
||||
return File.ReadAllBytes(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -16,8 +16,6 @@ namespace GameCult.Mesh.Tests;
|
|||
public sealed class CultMeshSingleFileReadabilityTests
|
||||
{
|
||||
private const int CSharp = 1;
|
||||
// The row `@directory` has no bytes: the walker makes an empty directory at the store path, and the write must refuse it untouched.
|
||||
private const string DirectoryRow = "@directory";
|
||||
private string _root = null!;
|
||||
|
||||
[SetUp]
|
||||
|
|
@ -69,15 +67,8 @@ public sealed class CultMeshSingleFileReadabilityTests
|
|||
foreach (var (name, write) in writes)
|
||||
{
|
||||
var path = Path.Combine(_root, name.Replace(' ', '-') + ".cc");
|
||||
var isDirectory = vector == DirectoryRow;
|
||||
var bytes = Array.Empty<byte>();
|
||||
if (isDirectory)
|
||||
Directory.CreateDirectory(path);
|
||||
else
|
||||
{
|
||||
bytes = File.ReadAllBytes(Path.Combine(VectorRoot(), vector));
|
||||
File.WriteAllBytes(path, bytes);
|
||||
}
|
||||
var bytes = File.ReadAllBytes(Path.Combine(VectorRoot(), vector));
|
||||
File.WriteAllBytes(path, bytes);
|
||||
|
||||
if (reads)
|
||||
{
|
||||
|
|
@ -105,18 +96,8 @@ public sealed class CultMeshSingleFileReadabilityTests
|
|||
continue;
|
||||
}
|
||||
|
||||
// A directory at the path is refused as IOException ("not a store file"), not as an unreadable store.
|
||||
if (isDirectory)
|
||||
Assert.That(() => write(path), Throws.TypeOf<IOException>(), name);
|
||||
else
|
||||
Assert.That(() => write(path), Throws.TypeOf<CultStoreUnreadableException>(), name);
|
||||
if (isDirectory)
|
||||
{
|
||||
Directory.Exists(path).Should().BeTrue(name + " replaced a directory");
|
||||
Directory.EnumerateFileSystemEntries(path).Should().BeEmpty(name + " wrote into a directory");
|
||||
}
|
||||
else
|
||||
File.ReadAllBytes(path).Should().Equal(bytes, name + " rewrote a file this runtime cannot read");
|
||||
Assert.That(() => write(path), Throws.TypeOf<CultStoreUnreadableException>(), name);
|
||||
File.ReadAllBytes(path).Should().Equal(bytes, name + " rewrote a file this runtime cannot read");
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -134,7 +115,7 @@ public sealed class CultMeshSingleFileReadabilityTests
|
|||
[Test]
|
||||
public void EveryVectorInTheFolderHasAManifestRow()
|
||||
{
|
||||
var rows = Vectors().Select(row => (string)row.Arguments[0]).Where(name => !name.StartsWith("..", StringComparison.Ordinal) && name != DirectoryRow).OrderBy(name => name, StringComparer.Ordinal);
|
||||
var rows = Vectors().Select(row => (string)row.Arguments[0]).Where(name => !name.StartsWith("..", StringComparison.Ordinal)).OrderBy(name => name, StringComparer.Ordinal);
|
||||
var files = Directory.GetFiles(VectorRoot(), "*.bin").Select(Path.GetFileName).OrderBy(name => name, StringComparer.Ordinal);
|
||||
rows.Should().Equal(files);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -178,7 +178,6 @@ namespace GameCult.Networking.Tests
|
|||
if (c["leans"] is not null && !ids.Contains(S(c["leans"]))) yield return "leans is not an option";
|
||||
var deciders = c["deciders"]!.AsArray().Select(S).ToList();
|
||||
if (deciders.Distinct().Count() != deciders.Count) yield return "duplicate deciders";
|
||||
if (deciders.Count > 0 && deciders.Count != c["decider_count"]!.GetValue<int>()) yield return "R-card: shown deciders are not exactly decider_count ids";
|
||||
if (c["depends"]!.AsArray().Any(d => Encoding.UTF8.GetByteCount(S(d)) > 1000)) yield return "a depends line is over 1000 UTF-8 bytes";
|
||||
if (S(c["status"]) == "open" && c["ruling"] is not null) yield return "an open card names a ruling";
|
||||
if (c["ruling"] is null && c["ruled_from"]!.AsArray().Count > 0) yield return "ruled_from without a ruling";
|
||||
|
|
@ -259,17 +258,8 @@ namespace GameCult.Networking.Tests
|
|||
Assert.That(HashOf(copy), Is.Not.EqualTo(baseline), $"mutation {i}");
|
||||
}
|
||||
var hidden = Clone(card);
|
||||
hidden["fork_id"] = "other:question:id"; hidden["leans"] = "yes"; hidden["deciders"] = new JsonArray(); hidden["decider_count"] = 7; hidden["status"] = "closed"; hidden["ruled_from"] = new JsonArray("n1"); hidden["raised_at"] = "2027-01-02T03:04:05.123Z";
|
||||
hidden["fork_id"] = "other:question:id"; hidden["leans"] = "yes"; hidden["status"] = "closed"; hidden["ruled_from"] = new JsonArray("n1");
|
||||
Assert.That(HashOf(hidden), Is.EqualTo(baseline), "fork_id and the unshown members stay outside the hash");
|
||||
hidden["raised_at"] = null;
|
||||
Assert.That(HashOf(hidden), Is.EqualTo(baseline), "a nil raised_at stays outside the hash");
|
||||
foreach (var (ids, count) in new[] { (0, 2), (1, 64) })
|
||||
{
|
||||
var counted = Clone(card)!;
|
||||
counted["deciders"] = new JsonArray(card["deciders"]!.AsArray().Take(ids).Select(d => JsonValue.Create(S(d))).ToArray());
|
||||
counted["decider_count"] = count;
|
||||
Assert.That(HashOf(counted), Is.EqualTo(baseline), "decider_count and the emptiness of deciders stay outside the hash");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
|
|
@ -299,12 +289,6 @@ namespace GameCult.Networking.Tests
|
|||
Assert.That(Errors(ForkSchema, "answerResult", v!["message"]), Is.Empty, S(v["name"]));
|
||||
Assert.That(HexOf(v["message"]), Is.EqualTo(S(v["messagePackHex"])), S(v["name"]));
|
||||
Assert.That(v["message"]!["card"] is not null, Is.EqualTo(S(v["name"]) == "stale_fork"), "only a stale refusal shows the new fork");
|
||||
Assert.That(ResultRuleErrors(v["message"]!), Is.Empty, S(v["name"]));
|
||||
}
|
||||
foreach (var v in Fork["resultsInvalid"]!.AsArray())
|
||||
{
|
||||
Assert.That(Errors(ForkSchema, "answerResult", v!["message"]).Count > 0 || ResultRuleErrors(v["message"]!).Any(), Is.True, $"{S(v["name"])}: {S(v["why"])}");
|
||||
Assert.That(HexOf(v["message"]), Is.EqualTo(S(v["messagePackHex"])), S(v["name"]));
|
||||
}
|
||||
Assert.That(Fork["doorRefusals"]!.AsArray().Select(r => S(r!["name"])).OrderBy(x => x, StringComparer.Ordinal), Is.EqualTo(Codes("refusal")));
|
||||
foreach (var v in Fork["unavailable"]!.AsArray())
|
||||
|
|
@ -335,12 +319,6 @@ namespace GameCult.Networking.Tests
|
|||
}
|
||||
}
|
||||
|
||||
// R-answer-result: the answer in an answer_result is the sender's own, stamped with their account, never nil (nil exists only in a record shown to a non-decider).
|
||||
private static IEnumerable<string> ResultRuleErrors(JsonNode r)
|
||||
{
|
||||
if (r["answer"]!["answerer"] is null) yield return "R-answer-result: the answerer is nil";
|
||||
}
|
||||
|
||||
// Rules a JSON Schema cannot say: a note is at most 4000 UTF-8 bytes (maxLength counts code points and is only the looser bound).
|
||||
private static IEnumerable<string> NoteRuleErrors(string def, JsonNode? message)
|
||||
{
|
||||
|
|
@ -448,8 +426,7 @@ namespace GameCult.Networking.Tests
|
|||
}
|
||||
var record = new JsonObject
|
||||
{
|
||||
["schemaVersion"] = "gamecult.fork.record.v0", ["fork_id"] = Clone(card["fork_id"]), ["options_hash"] = HashOf(card), ["counted_answer_id"] = Counted(),
|
||||
["answerer_count"] = answers.Select(a => S(a!["answerer"])).Distinct().Count(), ["answers"] = answers
|
||||
["schemaVersion"] = "gamecult.fork.record.v0", ["fork_id"] = Clone(card["fork_id"]), ["options_hash"] = HashOf(card), ["counted_answer_id"] = Counted(), ["answers"] = answers
|
||||
};
|
||||
return (responses, record);
|
||||
}
|
||||
|
|
@ -497,7 +474,6 @@ namespace GameCult.Networking.Tests
|
|||
Assert.That(JsonNode.DeepEquals(record, sc["record"]), Is.True, $"{name}: {record.ToJsonString()}");
|
||||
Assert.That(Errors(ForkSchema, "record", record), Is.Empty, name);
|
||||
Assert.That(HexOf(record), Is.EqualTo(S(sc["recordHex"])), name);
|
||||
foreach (var st in steps) if (st!["response"]?["answer"] is not null) Assert.That(ResultRuleErrors(st["response"]!), Is.Empty, name);
|
||||
foreach (var step in steps)
|
||||
{
|
||||
var r = step!["response"];
|
||||
|
|
@ -555,38 +531,6 @@ namespace GameCult.Networking.Tests
|
|||
var counted = answers.Where(a => (S(a!["outcome"]) is "Counted" or "Revision") && S(a["options_hash"]) == S(card["options_hash"])
|
||||
&& !answers.Any(r => S(r!["outcome"]) == "Retraction" && S(r["retracts"]) == S(a["answer_id"]))).Select(a => S(a!["answer_id"])).FirstOrDefault();
|
||||
if ((record["counted_answer_id"] is null ? null : S(record["counted_answer_id"])) != counted) yield return "counted_answer_id is not derived";
|
||||
// R-record: named answerers are counted by distinct id, nil ones by a bound
|
||||
var named = answers.Count(a => a!["answerer"] is not null); var total = answers.Count; var n = record["answerer_count"]!.GetValue<long>();
|
||||
if (total == 0 && n != 0) yield return "R-record: no answers and a count other than 0";
|
||||
else if (named == total && total > 0 && n != answers.Select(a => S(a!["answerer"])).Distinct().Count()) yield return "R-record: answerer_count is not the distinct answerers";
|
||||
else if (named == 0 && total > 0 && (n < 1 || n > total)) yield return "R-record: a withheld count outside 1..answers";
|
||||
// R-entry: the card's view and the record's view agree, so a withheld card never rides beside a named account
|
||||
var shown = card["deciders"]!.AsArray().Count > 0;
|
||||
if (shown && named < total) yield return "R-entry: a decider-view card with withheld answerers";
|
||||
if (!shown && named > 0) yield return "R-entry: a withheld card beside a named answerer";
|
||||
}
|
||||
|
||||
// R-list: all entries of one page are one reader's view.
|
||||
private static IEnumerable<string> ViewRuleErrors(JsonArray forks)
|
||||
{
|
||||
if (forks.Select(f => f!["card"]!["deciders"]!.AsArray().Count > 0).Distinct().Count() > 1) yield return "R-list: a page mixes views";
|
||||
}
|
||||
|
||||
// The non-decider twin of a decider-view entry: ids withheld, counts kept.
|
||||
private static JsonNode Withheld(JsonNode e)
|
||||
{
|
||||
var w = Clone(e)!;
|
||||
w["card"]!["deciders"] = new JsonArray();
|
||||
foreach (var a in w["record"]!["answers"]!.AsArray()) a!["answerer"] = null;
|
||||
return w;
|
||||
}
|
||||
private static List<string> IdsOf(JsonNode e) =>
|
||||
e["card"]!["deciders"]!.AsArray().Select(S).Concat(e["record"]!["answers"]!.AsArray().Select(a => S(a!["answerer"]))).ToList();
|
||||
private static void NoIdInBytes(string hex, IEnumerable<string> ids, string name)
|
||||
{
|
||||
var list = ids.ToList();
|
||||
Assert.That(list, Is.Not.Empty, name);
|
||||
foreach (var id in list) Assert.That(hex, Does.Not.Contain(Convert.ToHexString(Encoding.UTF8.GetBytes(id))), $"{name}: the withheld bytes carry {id}");
|
||||
}
|
||||
|
||||
// Order is a unique rank per Mind: the refusal names both fork ids, the first holder and the second.
|
||||
|
|
@ -697,14 +641,6 @@ namespace GameCult.Networking.Tests
|
|||
{
|
||||
Assert.That(Errors(ForkSchema, def, v!["message"]), Is.Empty, S(v["name"]));
|
||||
if (def == "entry") Assert.That(EntryRuleErrors(v["message"]!), Is.Empty, S(v["name"]));
|
||||
if (def == "entry" && v["twin"] is not null)
|
||||
{
|
||||
var twin = t[section]!["valid"]!.AsArray().Single(x => S(x!["name"]) == S(v["twin"]));
|
||||
Assert.That(JsonNode.DeepEquals(v["message"], Withheld(twin!["message"]!)), Is.True, $"{S(v["name"])}: the twin of {S(v["twin"])}");
|
||||
Assert.That(v["message"]!["record"]!["answerer_count"]!.GetValue<long>(), Is.EqualTo(twin["message"]!["record"]!["answerer_count"]!.GetValue<long>()), "a decider and a non-decider read the same count");
|
||||
NoIdInBytes(S(v["messagePackHex"]), IdsOf(twin["message"]!), S(v["name"]));
|
||||
Assert.That(S(twin["messagePackHex"]), Does.Contain(Convert.ToHexString(Encoding.UTF8.GetBytes(IdsOf(twin["message"]!)[0]))), "the decider view does carry the ids");
|
||||
}
|
||||
Assert.That(HexOf(v["message"]), Is.EqualTo(S(v["messagePackHex"])), S(v["name"]));
|
||||
}
|
||||
foreach (var v in t[section]!["invalid"]!.AsArray())
|
||||
|
|
@ -716,9 +652,7 @@ namespace GameCult.Networking.Tests
|
|||
foreach (var v in t["listInvalid"]!.AsArray())
|
||||
{
|
||||
var shared = OrderErrors(v!["message"]!["forks"]!.AsArray().Select(f => f!["card"]));
|
||||
var views = ViewRuleErrors(v["message"]!["forks"]!.AsArray()).ToList();
|
||||
Assert.That(Errors(ForkSchema, "list", v["message"]).Count > 0 || shared.Count > 0 || views.Count > 0, Is.True, $"{S(v["name"])}: {S(v["why"])}");
|
||||
if (S(v["name"]) == "a_page_mixes_views") Assert.That(views, Has.Count.EqualTo(1));
|
||||
Assert.That(Errors(ForkSchema, "list", v["message"]).Count > 0 || shared.Count > 0, Is.True, $"{S(v["name"])}: {S(v["why"])}");
|
||||
if (v["refusal"] is not null) Assert.That(shared, Is.EqualTo(new[] { S(v["refusal"]) }), S(v["name"]));
|
||||
Assert.That(HexOf(v["message"]), Is.EqualTo(S(v["messagePackHex"])), S(v["name"]));
|
||||
}
|
||||
|
|
@ -733,29 +667,12 @@ namespace GameCult.Networking.Tests
|
|||
var entries = sc["entries"]!.AsArray();
|
||||
foreach (var e in entries) Assert.That(EntryRuleErrors(e!), Is.Empty, name);
|
||||
Assert.That(OrderErrors(entries.Select(e => e!["card"])), Is.Empty, name);
|
||||
if (sc["twin"] is not null)
|
||||
{
|
||||
// the non-decider page is the decider page with the ids withheld: counts, order, cursor and stamp equal, byte for byte but the withheld members
|
||||
var twin = Fork["transport"]!["listScenarios"]!.AsArray().Single(x => S(x!["name"]) == S(sc["twin"]))!;
|
||||
Assert.That(JsonNode.DeepEquals(sc["entries"], new JsonArray(twin["entries"]!.AsArray().Select(e => Withheld(e!)).ToArray())), Is.True, name);
|
||||
var mine = sc["steps"]!.AsArray(); var theirs = twin["steps"]!.AsArray();
|
||||
Assert.That(mine.Count, Is.EqualTo(theirs.Count), name);
|
||||
for (var i = 0; i < mine.Count; i++)
|
||||
{
|
||||
var expected = Clone(theirs[i]!["response"])!;
|
||||
expected["forks"] = new JsonArray(expected["forks"]!.AsArray().Select(e => Withheld(e!)).ToArray());
|
||||
Assert.That(JsonNode.DeepEquals(mine[i]!["response"], expected), Is.True, name);
|
||||
NoIdInBytes(S(mine[i]!["responseHex"]), twin["entries"]!.AsArray().SelectMany(e => IdsOf(e!)), name);
|
||||
}
|
||||
Assert.That(S(theirs[0]!["responseHex"]), Does.Contain(Convert.ToHexString(Encoding.UTF8.GetBytes(IdsOf(twin["entries"]![0]!)[0]))), "the decider page does carry the ids");
|
||||
}
|
||||
JsonNode? previous = null;
|
||||
foreach (var step in sc["steps"]!.AsArray())
|
||||
{
|
||||
var response = JudgeList(entries, sc["ordinal"]!.GetValue<long>(), step!["request"]!);
|
||||
Assert.That(JsonNode.DeepEquals(response, step["response"]), Is.True, $"{name}: {response.ToJsonString()}");
|
||||
Assert.That(Errors(ForkSchema, "list", response), Is.Empty, name);
|
||||
Assert.That(ViewRuleErrors(response["forks"]!.AsArray()), Is.Empty, name);
|
||||
Assert.That(HexOf(response), Is.EqualTo(S(step["responseHex"])), name);
|
||||
var orders = response["forks"]!.AsArray().Select(f => f!["card"]!["order"]!.GetValue<long>()).ToList();
|
||||
Assert.That(orders, Is.Ordered.Descending, name);
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
“²cultcache.store.v3‘—¬vectors.item¬vectors.item¯vectors.item.v1¬vectors.itemÙL{"schemaName":"vectors.item","schemaVersion":"vectors.item.v1","members":[]}‘¬vectors.item<65>“”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha”¤beta¬vectors.item¹2026-09-29T00:00:01+00:00Ä’¤beta”¥Alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha
|
||||
|
|
@ -1 +0,0 @@
|
|||
“²cultcache.store.v3‘—¬vectors.item¬vectors.item¯vectors.item.v1¬vectors.itemÙL{"schemaName":"vectors.item","schemaVersion":"vectors.item.v1","members":[]}‘¬vectors.item<65>’”¢Ã©¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha”£eÌ<65>¬vectors.item¹2026-09-29T00:00:01+00:00Ä’¤beta
|
||||
|
|
@ -1 +0,0 @@
|
|||
“²cultcache.store.v3‘—¬vectors.item¬vectors.item¯vectors.item.v1¬vectors.itemÙL{"schemaName":"vectors.item","schemaVersion":"vectors.item.v1","members":[]}‘¬vectors.item<65>“”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha”¤beta¬vectors.item¹2026-09-29T00:00:01+00:00Ä’¤beta”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha
|
||||
|
|
@ -1 +0,0 @@
|
|||
“²cultcache.store.v3’—¬vectors.item¬vectors.item¯vectors.item.v1¬vectors.itemÙL{"schemaName":"vectors.item","schemaVersion":"vectors.item.v1","members":[]}‘¬vectors.item<65>—vectors.othervectors.other°vectors.other.v1vectors.otherÙN{"schemaName":"vectors.other","schemaVersion":"vectors.other.v1","members":[]}‘vectors.other<65>“”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha”¥alphavectors.other¹2026-09-29T00:00:00+00:00Ä’¥alpha”¤beta¬vectors.item¹2026-09-29T00:00:01+00:00Ä’¤beta
|
||||
|
|
@ -1 +0,0 @@
|
|||
“²cultcache.store.v3‘—¬vectors.item¬vectors.item¯vectors.item.v1¬vectors.itemÙL{"schemaName":"vectors.item","schemaVersion":"vectors.item.v1","members":[]}‘¬vectors.item<65>“”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha”¤beta¬vectors.item¹2026-09-29T00:00:01+00:00Ä’¤beta”¥alpha¬vectors.item¹2026-09-29T00:00:00+00:00Ä’¥alpha
|
||||
|
|
@ -5,7 +5,6 @@ A file may be replaced by a flush or a commit exactly when the runtime's own rea
|
|||
covers open, flush and commit. Each vector is the bytes named; the trailing and truncated ones are cut from v3-base.
|
||||
"""
|
||||
import pathlib
|
||||
import unicodedata
|
||||
|
||||
import msgpack
|
||||
|
||||
|
|
@ -13,21 +12,12 @@ here = pathlib.Path(__file__).resolve().parent
|
|||
whole = (here.parent / "v3-base.msgpack").read_bytes()
|
||||
pack = lambda value: msgpack.packb(value, use_bin_type=True)
|
||||
base = msgpack.unpackb(whole, raw=False)
|
||||
# A second published schema, and a record of it under the key alpha.
|
||||
other_entry = ["vectors.other", "vectors.other", "vectors.other.v1", "vectors.other", '{"schemaName":"vectors.other","schemaVersion":"vectors.other.v1","members":[]}', ["vectors.other"], []]
|
||||
other_record = ["alpha", "vectors.other"] + base[2][0][2:]
|
||||
# The same visible key as U+00E9 (NFC) and U+0065 U+0301 (NFD): equal after normalisation, unequal ordinally.
|
||||
nfc, nfd = "\u00e9", "e\u0301"
|
||||
assert unicodedata.normalize("NFC", nfd) == unicodedata.normalize("NFC", nfc) and nfc != nfd
|
||||
assert "Alpha".casefold() == "alpha" and "Alpha" != "alpha"
|
||||
|
||||
vectors = {
|
||||
"zero-byte": b"",
|
||||
"empty-array": pack([]),
|
||||
# A legacy envelope array: an array of maps. Rust, TypeScript and Python read it; C# has no legacy reader.
|
||||
"legacy-envelopes": pack([{"key": "old", "type": "old.type", "payload": pack([]), "storedAt": "2026-01-01T00:00:00Z"}]),
|
||||
# A legacy envelope array holding key a under two types: a key is one record, whatever the type. Refused like a v3 store with a shared key.
|
||||
"legacy-dup-key": pack([{"key": "a", "type": "old.one", "payload": pack([]), "storedAt": "2026-01-01T00:00:00Z"}, {"key": "a", "type": "old.two", "payload": pack([]), "storedAt": "2026-01-01T00:00:00Z"}]),
|
||||
"legacy-trailing": None, # filled below: the legacy array plus bytes after it
|
||||
"ints": pack([1, 2, 3]),
|
||||
"nil-first": pack([None, "x"]),
|
||||
|
|
@ -65,16 +55,6 @@ vectors = {
|
|||
"variant-slot-v3": pack([base[0], base[1], base[2] + [["gamma"] + base[2][0][1:3] + [b"", ["alpha", [[0, [[0, ""]], "", "gamma"]]]]]]),
|
||||
# v3-base with a fourth top-level slot after the records.
|
||||
"extra-top-slot": pack(base + [0]),
|
||||
# A store holding two records under one key is unreadable, whatever else they agree or differ on. Every record names a
|
||||
# schema the catalog publishes, so the shared key is the only defect.
|
||||
"dup-key-same-schema-v3": pack([base[0], base[1], base[2] + [base[2][0]]]),
|
||||
# Two records under one key, each naming a different published schema: the key alone is the identity.
|
||||
"dup-key-other-schema-v3": pack([base[0], base[1] + [other_entry], [base[2][0], other_record, base[2][1]]]),
|
||||
# alpha twice with different payload bytes: last-wins would silently pick one.
|
||||
"dup-key-different-payload-v3": pack([base[0], base[1], base[2] + [base[2][0][:3] + [b"\x92\xa5alpha\x09"]]]),
|
||||
# Keys are ordinal strings: no case folding and no Unicode normalisation, so these are distinct keys and the store reads.
|
||||
"distinct-keys-case-v3": pack([base[0], base[1], base[2] + [["Alpha"] + base[2][0][1:]]]),
|
||||
"distinct-keys-nfd-v3": pack([base[0], base[1], [[nfc] + base[2][0][1:], [nfd] + base[2][1][1:]]]),
|
||||
}
|
||||
vectors["legacy-trailing"] = vectors["legacy-envelopes"] + b"\x01\x02\x03"
|
||||
for name, data in vectors.items():
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
’„£key¡a¤type§old.one§payloadÄ<01>¨storedAt´2026-01-01T00:00:00Z„£key¡a¤type§old.two§payloadÄ<01>¨storedAt´2026-01-01T00:00:00Z
|
||||
|
|
@ -1,12 +1,9 @@
|
|||
# Whether each runtime's own reader opens the file. A flush and a commit replace a file exactly when it reads; where a runtime has a soft flush (TypeScript, Rust) it leaves a file that reads byte for byte and refuses one that does not.
|
||||
# A store holding two records under one key is unreadable; keys are ordinal strings (no case folding, no normalisation), so distinct spellings are distinct keys.
|
||||
# Columns: file, csharp, rust, typescript, python. `../v3-base.msgpack` is the one valid store.
|
||||
# `@directory` is no file: the walker makes an empty directory at the store path and every runtime must refuse to open it and to write over it (flush, commit, delete), leaving the directory empty.
|
||||
../v3-base.msgpack reads reads reads reads
|
||||
zero-byte.bin refuses refuses refuses refuses
|
||||
empty-array.bin reads reads reads reads
|
||||
legacy-envelopes.bin refuses reads reads reads
|
||||
legacy-dup-key.bin refuses refuses refuses refuses
|
||||
legacy-trailing.bin refuses refuses refuses refuses
|
||||
ints.bin refuses refuses refuses refuses
|
||||
nil-first.bin refuses refuses refuses refuses
|
||||
|
|
@ -33,13 +30,3 @@ missing-schema-versioned-v3.bin refuses refuses refuses refuses
|
|||
compatible-id-only-v3.bin reads reads reads reads
|
||||
own-id-over-compatible-v3.bin reads reads reads reads
|
||||
compatible-before-owner-v3.bin reads reads reads reads
|
||||
dup-key-same-schema-v3.bin refuses refuses refuses refuses
|
||||
dup-key-other-schema-v3.bin refuses refuses refuses refuses
|
||||
dup-key-different-payload-v3.bin refuses refuses refuses refuses
|
||||
distinct-keys-case-v3.bin reads reads reads reads
|
||||
distinct-keys-nfd-v3.bin reads reads reads reads
|
||||
../../document-variants-c0/current-catalog-bad-canonical.msgpack refuses refuses refuses refuses
|
||||
../../document-variants-c0/legacy-catalog-plain.msgpack refuses refuses refuses refuses
|
||||
../../document-variants-c0/legacy-catalog-extra-slot.msgpack refuses refuses refuses refuses
|
||||
../../document-variants-c0/legacy-catalog-trailing.msgpack refuses refuses refuses refuses
|
||||
@directory refuses refuses refuses refuses
|
||||
|
|
|
|||
|
|
@ -1,47 +0,0 @@
|
|||
# key-identity vectors
|
||||
|
||||
Written only by the C# reference, `tests/GameCult.Caching.Tests/KeyIdentityVectorTests.cs`, with
|
||||
`CULTCACHE_WRITE_VECTORS=1` (never set in CI). Every other run reads these files, runs every row against the reference and
|
||||
compares the files byte for byte; a changed vector is a reviewed commit. Other runtimes are held to the same verdicts
|
||||
(`variants:ruling:cache-refuses-retype`, `variants:ruling:write-ownership-by-written-type`,
|
||||
`variants:ruling:unheld-removal-refused-at-commit`).
|
||||
|
||||
RULE: a cache holds one record per key whatever its type; a write of a type that does not own the held record's schema id at
|
||||
that key is refused as a schema conflict naming the key, with the cache, its store file and every foreign record
|
||||
byte-identical; retyping is a delete then a put. A store that holds a record no registered type claims loads: the cache does
|
||||
not serve it, lists its key as foreign, keeps its bytes through every write, and refuses a write or a removal at its key.
|
||||
Registering a type that declares its schema id claims it at the next load.
|
||||
|
||||
## Files
|
||||
|
||||
- Schemas: `tests.ki_alpha`, `tests.ki_beta`, `tests.ki_gamma` (`alpha`, `beta`, `gamma` below), each one string slot
|
||||
`[0: Text]`, version the name plus `.v1`. The schema ids C# writes in the store catalogs and on every record:
|
||||
- `tests.ki_alpha`: `sha256:b20703a98f49c253898404ab50ba2c630720f0b8a5e1a41614d5650be22799fe`
|
||||
- `tests.ki_beta`: `sha256:dee4b727cf2ee04e61ab635ff0c5f3dca7345a411dc5f1018e583ab16009cf94`
|
||||
- `tests.ki_gamma`: `sha256:fbc26e351e00c9e7185e8de171c7486aba707c229c949f0aae6b889288af5f6d`
|
||||
- `foreign-store.cc`: a single-file store, alpha `a` at key `j`, gamma `g` at key `k`, stored-at pinned to
|
||||
`2026-01-01T00:00:00Z`. `catalog-store.cc`: alpha `a` at `qx`. `run-store.cc`: beta `b` at `qx` (row 11). The row-11 key is `qx` because it must
|
||||
occur in the refusal's message only as the key, never inside a schema name (`k` is inside `ki_alpha`).
|
||||
- `rows.msgpack`: an array of `[label, registered, ops, expect]`.
|
||||
- `registered`: the types the cache registers, a subset of `[alpha, beta, gamma]`.
|
||||
- `ops`, in order: `[open, file-or-nil-or-memory]` (a cache on a fresh copy of the vector file; nil is a fresh path with no
|
||||
file; the string `memory` is a cache with no store), `[attach, file, [home types]]` (route one more store, a fresh copy
|
||||
of the vector file, to those home types, and pull every store), `[reopen]` (dispose, then a new cache on the same path),
|
||||
`[put, type, key, text]`, `[delete, type, key]` (each one commit).
|
||||
- `expect` is `[refusal, held, file, foreign, kept]`. Every op but the last succeeds; the last succeeds or refuses, and in the generator only a schema conflict (or, for an attach, an invalid-operation refusal) counts as a refusal: any other exception fails the run.
|
||||
`refusal` is nil or `[key]`: a typed schema-conflict refusal naming the key for a put or a delete (rows 1, 3, 5, 8, 9),
|
||||
and for row 11 any refusal (an invalid-operation error, not a schema conflict) whose message contains the key as a delimited token (not inside a schema name or other word; the C# walker also pins its own `Record <key> is held by` opening, which a leaf need not copy); the key in `expect` is the one the stores share, the store file it was refused from left byte-identical.
|
||||
`held` is the cache's records, sorted by type then key, as `[type, key, text]`. `file` is the store file's records after
|
||||
the ops, sorted by key, as `[key, schema name]`, nil for a cache with no store. `foreign` is the keys the cache lists as
|
||||
foreign, sorted. `kept` is the foreign keys whose stored record (schema id, stored-at, payload) after the ops equals the
|
||||
file's before the first op.
|
||||
|
||||
## Rows
|
||||
|
||||
1 retype-refused-held-in-file, 2 retype-by-remove-then-add, 3 retype-refused-memory-cache, 4 same-type-replaces, 5
|
||||
retype-refused-after-reload, 6 foreign-loads, 7 foreign-kept-by-a-write, 8 foreign-key-write-refused, 9
|
||||
foreign-key-delete-refused, 10 foreign-claimed-by-a-later-registration (registered `[alpha, gamma]`: row 6's store loads
|
||||
with nothing foreign), 11 key-in-two-stores-refused, 12 same-type-replaces-memory-cache (a cache with no store replaces a same-type record: admitted).
|
||||
|
||||
Every runtime walks 12 rows and asserts the count; none is exempt (every row is single-record ops). A leaf never registers
|
||||
gamma in rows 6 and 8; only row 10 does.
|
||||
Binary file not shown.
Binary file not shown.
|
|
@ -1 +0,0 @@
|
|||
œ”»retype-refused-held-in-file’¥alpha¤beta“’¤openÀ”£put¥alpha¡k¡a”£put¤beta¡k¡b•‘¡k‘“¥alpha¡k¡a‘’¡k®tests.ki_alpha<68><61>”¹retype-by-remove-then-add’¥alpha¤beta”’¤openÀ”£put¥alpha¡k¡a“¦delete¥alpha¡k”£put¤beta¡k¡b•À‘“¤beta¡k¡b‘’¡ktests.ki_beta<74><61>”»retype-refused-memory-cache’¥alpha¤beta“’¤open¦memory”£put¥alpha¡k¡a”£put¤beta¡k¡b•‘¡k‘“¥alpha¡k¡aÀ<61><C380>”²same-type-replaces‘¥alpha“’¤openÀ”£put¥alpha¡k¡a”£put¥alpha¡k¡b•À‘“¥alpha¡k¡b‘’¡k®tests.ki_alpha<68><61>”»retype-refused-after-reload’¥alpha¤beta”’¤openÀ”£put¥alpha¡k¡a‘¦reopen”£put¤beta¡k¡b•‘¡k‘“¥alpha¡k¡a‘’¡k®tests.ki_alpha<68><61>”foreign-loads‘¥alpha‘’¤open°foreign-store.cc•À‘“¥alpha¡j¡a’’¡j®tests.ki_alpha’¡k®tests.ki_gamma‘¡k‘¡k”·foreign-kept-by-a-write‘¥alpha’’¤open°foreign-store.cc”£put¥alpha¡m¡m•À’“¥alpha¡j¡a“¥alpha¡m¡m“’¡j®tests.ki_alpha’¡k®tests.ki_gamma’¡m®tests.ki_alpha‘¡k‘¡k”¹foreign-key-write-refused‘¥alpha’’¤open°foreign-store.cc”£put¥alpha¡k¡x•‘¡k‘“¥alpha¡j¡a’’¡j®tests.ki_alpha’¡k®tests.ki_gamma‘¡k‘¡k”ºforeign-key-delete-refused‘¥alpha’’¤open°foreign-store.cc“¦delete¥alpha¡k•‘¡k‘“¥alpha¡j¡a’’¡j®tests.ki_alpha’¡k®tests.ki_gamma‘¡k‘¡k”Ù'foreign-claimed-by-a-later-registration’¥alpha¥gamma‘’¤open°foreign-store.cc•À’“¥alpha¡j¡a“¥gamma¡k¡g’’¡j®tests.ki_alpha’¡k®tests.ki_gamma<6D><61>”¹key-in-two-stores-refused’¥alpha¤beta“’¤open¦memory“¦attach°catalog-store.cc‘¥alpha“¦attach¬run-store.cc‘¤beta•‘¢qx‘“¥alpha¢qx¡aÀ<61><C380>”¿same-type-replaces-memory-cache‘¥alpha“’¤open¦memory”£put¥alpha¡k¡a”£put¥alpha¡k¡b•À‘“¥alpha¡k¡bÀ<62><C380>
|
||||
Binary file not shown.
|
|
@ -19,15 +19,14 @@ reviewed commit. Other runtimes are held to the same canonical text and the same
|
|||
`" "` and `item:d` holds Sku `U+0085`. Written through a role Sku.
|
||||
- `blank-store.cc`: the same, only `item:c` and `item:d`.
|
||||
- `pair-store.cc`, `surrogate-store.cc`, `types-store.cc`, `repull-before.cc`, `repull-after.cc`, `alias-store.cc`,
|
||||
`key-case-store.cc`, `key-punctuation-store.cc`, `surrogate-groups-store.cc`, `holder-case-store.cc`, `cross-schema-store.cc`, `distinct-values-store.cc`, `normal-keys-store.cc`, `normal-three-store.cc`, `wide-store.cc`: single-file stores written like
|
||||
`key-case-store.cc`, `key-punctuation-store.cc`, `surrogate-groups-store.cc`: single-file stores written like
|
||||
`duplicate-store.cc`, through roles, for the rows of `refusals.msgpack` that open or pull them. Schemas
|
||||
`tests.unique_item`, `tests.uq_pair` (Sku and Alt both unique), `tests.uq_alias` (Sku and `alt` both unique),
|
||||
`tests.uq_wide` (Sku and Alt both unique, aliases U+FF21 and U+10400), `tests.uq_alpha` and `tests.uq_beta`, each slots `[0: Sku string, 1: second string]`, schema version the name plus
|
||||
`tests.uq_alpha` and `tests.uq_beta`, each slots `[0: Sku string, 1: second string]`, schema version the name plus
|
||||
`.v1`. The schema ids C# writes in the store catalogs and on every record:
|
||||
- `tests.unique_item`: `sha256:93bb890de83069061af696b23f94fd32fb3c2dad3c91ee645e8add56085f1882`
|
||||
- `tests.uq_pair`: `sha256:2538cceafb6dcbce22ef11b932ecb804af5f96bc3aaedff692a52d3640433e60`
|
||||
- `tests.uq_alias`: `sha256:77a77a394ebd8bf71e14b9b6f2cea4978c25d4e705fd12364f660edbf6c63b1f`
|
||||
- `tests.uq_wide`: `sha256:b79eecd11eed5e456902f095d2748da7f59125c4c8972b0da7dddc81d07fc03f`
|
||||
- `tests.uq_alpha`: `sha256:53aeaf2ea9b25e85ade6b15fc04c7653e7b206c87e258f1aa74dd32a840a1389`
|
||||
- `tests.uq_beta`: `sha256:45ed62a64c91f6d815d3ec975c4a4a1b2ce07b6e7262a32d828673cf10899586`
|
||||
- `outcomes.msgpack`: rows `[file, unique alias or nil, open|refused, k1 or nil, k2 or nil]`. Opening `file` with that
|
||||
|
|
@ -52,32 +51,19 @@ last succeeds and the last refuses with exactly the RULE text built from those f
|
|||
byte-identical. Schemas are those of the store files above; `second` is `Category` (plain) except in `tests.uq_pair`
|
||||
and `tests.uq_alias`, where it is `Alt` (unique, alias `Alt` and `alt` respectively).
|
||||
|
||||
The 13 rows of the spec are a floor; rows 14-37 extend it with orderings that a code-point, case-insensitive or
|
||||
The 13 rows of the spec are a floor; rows 14-22 extend it with orderings that a code-point, case-insensitive or
|
||||
culture compare would flip. Rows, in file order: 1 batch-later-value-staged-first, 2 batch-surrogate-keys, 3
|
||||
batch-interleaved-types, 4 batch-two-aliases, 5 held-then-landing, 6 held-in-the-least-group, 7 swap-off-a-held-value,
|
||||
8 load-two-aliases, 9 load-surrogate-keys, 10 load-interleaved-types, 11 repull-held-pair, 12 load-repull-after, 13
|
||||
commit-onto-a-loaded-holder, 14 batch-key-case (`B` before `a`), 15 batch-key-punctuation (`Z` before `_` before `a`),
|
||||
16 batch-key-case-in-a-group, 17 batch-surrogate-groups (U+1F600 before U+E000, the reverse of code-point order), 18
|
||||
batch-alias-case (alias `Sku` before `alt`), 19 load-alias-case, 20 load-key-case, 21 load-key-punctuation, 22
|
||||
load-surrogate-groups, 23 batch-holders-by-case (`a` and `A` holding one value: `A` before `a`), 24 load-holders-by-case, 25 batch-replaced-by-case (held `a`, then a commit of `A` on its value: `A` does not replace `a`),
|
||||
26 batch-value-in-two-schemas (one value in `tests.uq_alpha` and `tests.uq_beta` at two keys: two groups, success), 27
|
||||
load-value-in-two-schemas, 28 batch-distinct-values (`S`, `s`, `S `, ` S`, U+00E9, `e`+U+0301, U+FF33 are seven values: success), 29
|
||||
load-distinct-values, 30 batch-keys-by-normal-form (keys U+00E9 and `e`+U+0301 holding one value are two holders, `e`+U+0301 first),
|
||||
31 load-keys-by-normal-form, 32 put-onto-a-held-pair (held `m` and landing `a` share Sku and Alt in `tests.uq_pair`, one record per
|
||||
commit: `Alt` is named before `Sku`), 33 put-onto-a-held-alias-pair (the same in `tests.uq_alias`: `Sku` before `alt`), 34 batch-three-holders-normal-form
|
||||
(keys `e`+U+0302, `f` and U+00E9 holding one value: the two least by code unit, `e`+U+0302 and `f`, are named; an NFC, NFD or culture sort of the keys names another pair), 35
|
||||
load-three-holders-normal-form, 36 batch-alias-code-unit (two records each holding the same value under both aliases of `tests.uq_wide`, U+FF21 and U+10400: by UTF-16 code
|
||||
unit U+10400 (D801 DC00) precedes U+FF21, by code point it follows it, so the refusal names alias U+10400), 37 load-alias-code-unit.
|
||||
Rows 19-22, 24, 27, 29, 31, 35 and 37 open a store holding the records of rows 18, 14, 15, 17, 23, 26, 28, 30, 34 and 36, so a runtime with no batch
|
||||
load-surrogate-groups. Rows 19-22 open a store holding the records of rows 18, 14, 15 and 17, so a runtime with no batch
|
||||
commit still walks alias and cross-group key order.
|
||||
|
||||
A runtime performs every row its API can perform and asserts the count it walks. These are the counts a runtime must
|
||||
assert, not a statement of which runtimes walk them today:
|
||||
A runtime performs every row its API can perform and asserts the count it walked:
|
||||
|
||||
- C# and Rust: all 37.
|
||||
- Python: 35, every row but 3 and 26 (`put_envelopes` takes one type, so no batch of two types).
|
||||
- TypeScript: 20, the rows with no multi-record commit: 5, 8, 9, 10, 11, 12, 13, 19, 20, 21, 22, 24, 25, 27, 29, 31, 32, 33, 35 and 37
|
||||
(no batch commit, so no row 1-4, 6, 7, 14-18, 23, 26, 28, 30, 34 or 36; `variants:idea:unique-index-runtimes.ts-batch-commit`).
|
||||
|
||||
Not pinned: the schema-name tiebreak. Record keys are unique across a whole cache, so two types never hold the same key and
|
||||
two groups can never tie on alias and key pair; the final `schema` comparison cannot decide a refusal.
|
||||
- C# and Rust: all 22.
|
||||
- Python: 21, every row but 3 (`put_envelopes` takes one type).
|
||||
- TypeScript: 11, rows 5, 8, 9, 10, 11, 12, 13, 19, 20, 21 and 22 (no batch commit, so no row 1-4, 6, 7 or 14-18;
|
||||
`variants:idea:unique-index-runtimes.ts-batch-commit`).
|
||||
|
|
|
|||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -2,12 +2,6 @@
|
|||
|
||||
All notable changes to this package are documented in this file.
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Breaking
|
||||
|
||||
- A single-file store holding two records under one key is unreadable: the reader refuses it with `CultStoreUnreadableException` naming the key (keys compare by ordinal code units, no case folding or normalisation), so a cache, `ReadStoreSnapshot` and a flush refuse the file and leave it as it is. Before, the later record silently replaced the earlier. A directory store reads its manifest through the same reader and refuses a duplicate key the same way. Repair the duplicate with the runtime that wrote it before reading the store here.
|
||||
|
||||
## [3.0.0] - 2026-11-11
|
||||
|
||||
### Breaking
|
||||
|
|
|
|||
Loading…
Reference in a new issue