From 2de070652dde4357c2bb16cce5931b1061e89724 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Mon, 19 Oct 2026 14:27:25 +0200 Subject: [PATCH 1/3] Research: content-addressed HTTP answers (IPFS, OCI, Nix, Bazel, git; Fetch CORS and encoding), for cultmesh-ts-content C2 Co-Authored-By: Claude Opus 5.5 --- ...26-10-19-content-addressed-http-answers.md | 170 ++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100644 docs/research/2026-10-19-content-addressed-http-answers.md diff --git a/docs/research/2026-10-19-content-addressed-http-answers.md b/docs/research/2026-10-19-content-addressed-http-answers.md new file mode 100644 index 00000000..3feac45f --- /dev/null +++ b/docs/research/2026-10-19-content-addressed-http-answers.md @@ -0,0 +1,170 @@ +# Content-addressed HTTP answers: what the specs and sources say + +Fork served: **cultmesh-ts-content C2 provider HTTP answers** (session label +self-2026-10-19-cm-content). Eyes pass, 2026-10-19. Facts with evidence pointers +only; no recommendation. Line numbers refer to the raw files fetched on +2026-10-19 (master/main at that date), so they drift. + +Source shorthands: + +- **TG** = IPFS Trustless Gateway spec, https://specs.ipfs.tech/http-gateways/trustless-gateway/ (source: https://raw.githubusercontent.com/ipfs/specs/main/src/http-gateways/trustless-gateway.md) +- **PG** = IPFS Path Gateway spec, https://specs.ipfs.tech/http-gateways/path-gateway/ (source: https://raw.githubusercontent.com/ipfs/specs/main/src/http-gateways/path-gateway.md) +- **OCI** = OCI Distribution Spec, https://raw.githubusercontent.com/opencontainers/distribution-spec/main/spec.md +- **Fetch** = WHATWG Fetch Standard, https://fetch.spec.whatwg.org/ +- **RFC 9110/9111/9112**, https://www.rfc-editor.org/rfc/rfc9110.txt (and 9111, 9112, 8246) + +Two things the draft model page said "from memory" are checked at the end +(section 8). + +## 1. IPFS Trustless Gateway (`application/vnd.ipld.raw`) + +| Case | What the spec says | Pointer | +|---|---|---| +| Found | Body is "An opaque bytes matching the requested block CID"; "The Body hash MUST match the Multihash from the requested CID." | TG "Block Responses (application/vnd.ipld.raw)" (src l.341-346) | +| Unknown/absent hash | `404 Not Found` MUST be returned when the root block is not in the gateway's storage; applies to HEAD for any CID and GET of raw blocks. Body not specified. For non-recursive gateways it "definitively signals" the content is not in the dataset. | TG "Response Status Codes / 404 Not Found" (src l.274-283); "Recursion" (l.606-612) | +| Malformed request | TG defines no status for a malformed CID. It says servers SHOULD "validate CID format and encoding before processing requests". PG says `400 Bad Request` is a "generic client error", example "malformed CID". TG gives 400 for missing `Accept` in strict trustless mode (l.108) and for an out-of-range `entity-bytes` (l.210). | TG "Security Considerations / HTTP Servers" (src l.616-628); PG "Response Status Codes" | +| Stored object failing its digest | **Not specified for servers.** The only requirement is client-side: clients MUST verify block digest against the CID before processing or storing. TG `500` is "only for genuine server errors, not for content unavailability" (storage failure, resource exhaustion, unexpected internal errors); corruption is not named. | TG "HTTP Clients / Block Validation" (src l.629-631); "500 Internal Server Error" (l.294-296) | +| Range | TG defines no Range support for raw blocks. PG says Range is "for requesting specific byte range of UnixFS files and raw blocks"; gateways SHOULD support single-range, multi-range optional; success is `206` with `Content-Range`; `Accept-Ranges` is optional, `none` is the example for non-deterministic CAR order. | PG "Range (request header)" (src l.184-193), "Content-Range" (l.626), "Accept-Ranges" (l.632-640) | +| HEAD | `HEAD /ipfs/{cid}` is GET without payload; missing root block is 404 for HEAD too. PG: gateways SHOULD do minimal work to produce headers such as `Content-Length`. For the probe path, all HEAD headers are optional. | TG "HEAD /ipfs/{cid}" (src l.68-72), "404" (l.274-283), "Dedicated Probe Paths" (l.525-548); PG HEAD text | +| CORS | **TG defines none.** PG only says gateways serving cross-origin traffic SHOULD list `Ipfs-Uri` (and `X-Ipfs-Path`) in `Access-Control-Expose-Headers`. Kubo's config doc says its default `Gateway.HTTPHeaders` is `{}` "+ implicit CORS headers from `boxo/gateway#AddAccessControlHeaders` and ipfs/specs#423"; the values themselves are not in that file. | PG header sections; https://raw.githubusercontent.com/ipfs/kubo/master/docs/config.md l.1484-1488 | +| Cache-Control | Response value is in PG, not TG: `Cache-Control: public, max-age=29030400, immutable` "MUST be returned for every immutable resource under `/ipfs/` namespace". (Request-side `Cache-Control: only-if-cached` returns `412` when the root block is not immediately available.) | PG "Cache-Control (response header)" (src l.488-500); TG "Cache-Control: only-if-cached" (l.120-126) | +| ETag | TG: "MUST be returned and follow the recommendations in" PG. PG default is the requested CID in quotes: `Etag: "bafy…foo"`; format-specific variants add a suffix (`"bafy…foo.raw"`); non-deterministic streams use `W/`. PG: gateway MUST compare `If-None-Match` against its `Etag` (answer 304). TG: clients SHOULD NOT assume ETags are portable across gateways. | TG "Etag (response header)" (src l.324-338); PG l.130-136, l.453-486 | +| Content-Type | "MUST be returned"; for raw blocks the type is `application/vnd.ipld.raw` (IANA registered). No charset parameter is specified. | TG "Content-Type (response header)" (src l.300-308); IANA entry linked at l.343 | +| Other headers | `Content-Disposition` MUST be `attachment` with `filename=".bin"` for raw blocks, "to ensure requested bytes are not rendered by a web browser". PG: `X-Content-Type-Options: nosniff` should be returned with `application/vnd.ipld.raw` and `.car` (from the PG fetch summary; line not re-grepped). | TG "Content-Disposition" (src l.310-316); PG "X-Content-Type-Options" | +| Content-Encoding / Length | PG: `Content-Length` "Represents the length of returned HTTP payload", and warns "the value may differ from the real size of requested data if compression or chunked `Transfer-Encoding` are used", citing ipfs/specs#461 (an open TODO in the spec). | PG "Content-Length (response header)" (src l.613-624) | +| Size cap | Clients SHOULD cap block size; the spec names 2 MiB as the p2p block limit (see Block Limits). | TG "Block Limits" (src l.558-573) and "HTTP Clients" (l.629-640) | + +## 2. OCI Distribution Spec, blob `GET`/`HEAD /v2//blobs/` + +| Case | What the spec says | Pointer | +|---|---|---| +| Found (GET) | "MUST provide the expected blob, with a response code that MUST be `200 OK`." Response MUST contain `Docker-Content-Digest`; "If present, the value of this header MUST be a digest matching that of the response body." Clients MAY ignore it, but if used MUST verify it; "Clients SHOULD verify that the response body matches the requested digest." | OCI "Pulling blobs" (src l.180-195) | +| Unknown | `404 Not Found` MUST be returned (GET and HEAD). The endpoint table lists only `200` success and `404` failure. Error code names `BLOB_UNKNOWN` (code-1) exist in the error table; this section does not require a JSON body. | OCI l.194, l.209-216; Endpoints table l.846 (end-2); Error Codes table l.891 | +| Malformed digest | **Not specified for blob pull.** The table lists only 404. (Other endpoints: PUT blob 400 on digest problems; DELETE blob 404/400/405.) `DIGEST_INVALID` is defined as "provided digest did not match uploaded content" (upload semantics). | OCI table l.846-858; code-4 l.894 | +| Stored object failing its digest | **Not specified for servers.** Client-side verification only (above). | OCI l.188-193 | +| Range | "A registry SHOULD support the `Range` request header in accordance with RFC 9110 (section 14)." No `206`/`416`/`Accept-Ranges` text for blob pull (416 appears only in the upload flow, l.392, 421). | OCI l.196 | +| HEAD | HEAD on an existing blob MUST return `200`; MUST contain `Docker-Content-Digest` and `Content-Length` ("the size in bytes"); not found is `404`. Note: older registries may omit the digest header; clients "can reasonably assume the digest algorithm used is sha256". | OCI "Checking if content exists" (src l.198-216) | +| CORS | **Not in spec.** | whole OCI spec (no `Access-Control` occurrence found in the `grep` of the file) | +| Cache-Control/ETag | **Not in spec.** | OCI spec | +| Content-Type | **Not specified for blob GET**; `application/octet-stream` appears only for upload requests. | OCI upload sections | +| Redirects | Registries may answer blob GET with a redirect; clients must not forward `Authorization` across hosts. | OCI pull text (from the WebFetch of the same file; not re-grepped) | + +Known failures near this area (NVD records, fetched 2026-10-19): + +- CVE-2024-3727 (containers/image, Red Hat): "integrity-check flaw" allowing unexpected authenticated registry access, resource exhaustion, local path traversal. It affects the client library, not a registry server. https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2024-3727 +- CVE-2026-85747 (distribution/distribution, fixed 3.1.2): the client's manifest `Get()` does not hash the body against the requested digest, so a malicious registry can substitute content. https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-85747 (status "Awaiting Analysis") +- CVE-2026-93317 (Docker build): a registry or OCI-layout blob source could supply content not matching the claimed digest and have it cached under that digest. https://services.nvd.nist.gov/rest/json/cves/2.0?cveId=CVE-2026-93317 (status "Awaiting Analysis") +- Searched for a path traversal CVE in a registry server's digest-to-path mapping: none surfaced. + +## 3. Nix binary cache (HTTP) + +The manual's Binary Cache page defines layout only (a `nix-cache-info` file, a +`.narinfo` per object with "a (usually relative) URL to the corresponding +compressed NAR", the NAR). It states no status codes, headers, CORS, Range or +Cache-Control. https://nix.dev/manual/nix/2.35/protocols/binary-cache/ (page head). +The behavior below is from the Nix client/writer source. + +| Case | What the source shows | Pointer | +|---|---|---| +| Found | Writer stores NARs at `nar/.nar`, i.e. named by the hash of the compressed file; narinfo records `FileHash`, `FileSize`, plus `NarHash`. | https://raw.githubusercontent.com/NixOS/nix/master/src/libstore/binary-cache-store.cc l.156-181 | +| Absent | Client maps HTTP `404` and `410` to `NotFound` ("The file is definitely not there"); `403` to `Forbidden`; `408`/`429` to `Transient`. `fileExists` issues a **HEAD** and treats `NotFound` **and `Forbidden`** as absent, "S3 buckets return 403 if a file doesn't exist and the bucket is unlistable". | https://raw.githubusercontent.com/NixOS/nix/master/src/libstore/filetransfer.cc l.826-838; .../http-binary-cache-store.cc l.184-202 | +| Malformed request | Not specified. | n/a | +| Failing digest | Client side only; the writer's `FileHash`/`NarHash` are checked by the consumer. No server answer defined. | binary-cache-store.cc l.175-181, l.313 | +| Range | Client records `acceptRanges` for retries (client-side use of Range). No server rule. | filetransfer.cc l.219 | +| HEAD | Used for existence probing (above). | http-binary-cache-store.cc l.190 | +| CORS / Cache-Control / ETag | Not in spec or the files read. | n/a | +| Content-Type | Writer uploads `text/x-nix-narinfo` (narinfo), `text/x-nix-cache-info`, and `application/x-nix-nar` (NAR). | binary-cache-store.cc l.57, l.135, l.270 | +| Content-Encoding | Writer can upload with a `Content-Encoding` header naming the compression (http-binary-cache-store.cc l.232). Client sets `CURLOPT_ACCEPT_ENCODING ""` (accept any supported) and tracks a `hasContentEncoding` flag. The NAR itself is stored already compressed (named by compressed-file hash), so transport encoding and stored compression are separate. | filetransfer.cc l.223-224, l.616 | + +## 4. Bazel remote cache, HTTP + +| Case | What the source/spec says | Pointer | +|---|---|---| +| Protocol | Bazel docs: "Binary data (BLOB) is uploaded via PUT requests and downloaded via GET requests"; action results under `/ac/`, files under `/cas/`; any "HTTP/1.1 server that supports PUT and GET" can serve. The doc page states **no** status codes, no digest verification on PUT, no Content-Type, no compression, no Range. HEAD is not mentioned there. | https://bazel.build/remote/caching "HTTP caching protocol" | +| bazel-remote README | CAS key is "the lowercase SHA256 hash of the entry"; `/cas/` or `//cas/` (instance "optional but ignored"); "HEAD requests can be used to confirm whether a key exists"; the empty blob is "always available, even if the cache is empty" (HEAD example: `200` with `Content-Length: 0`); GET with `Accept-Encoding: zstd` "may" return zstd data; PUT of zstd requires `Content-Encoding: zstd` and `X-Digest-SizeBytes`. | https://raw.githubusercontent.com/buchgr/bazel-remote/master/README.md (HTTP section) | +| Found | `200`, `Content-Type: application/octet-stream`, `Content-Length` set to the stored size (uncompressed path). | https://raw.githubusercontent.com/buchgr/bazel-remote/master/server/http.go l.275-282 | +| Absent | `404`, body "Not found" via `http.Error`. | http.go l.268-271 | +| Malformed | Path must match `^/?(.*/)?(ac/\|cas/)([a-f0-9]{64})$` (lowercase 64-hex); otherwise `400` with the escaped URL echoed in the message (`html.EscapeString`). | http.go l.33, l.101-112, l.216-223 | +| Failing digest (read) | No digest check on the GET path in this handler (reader returned by `cache.Get`, `io.Copy` to the client). A failure after headers are sent is logged and cannot change the status ("we've already started writing data"). | http.go l.284-297 | +| Digest check (write) | PUT checks size match (`400` "sizes don't match") and `Content-Encoding` is `identity`/`zstd`/absent (else `400`); the remaining digest verification is in code not read here. | http.go l.306-345, l.372-388 | +| Compression | When zstd is chosen the handler sets `Content-Encoding: zstd` and **omits `Content-Length`** ("TODO: calculate Content-Length for compressed blobs too"). | http.go l.246-250, l.275-281 | +| HEAD | `200` with `Content-Length` of the size, or `404`. | http.go l.468-491 | +| Range / CORS / Cache-Control / ETag | Not present in the handler excerpts read. | n/a | + +## 5. Git over HTTP + +Smart HTTP does not address single objects. Dumb HTTP does: URL +`$GIT_URL/objects/XX/YYYY…` for a loose object. +https://git-scm.com/docs/http-protocol (introduction example). + +| Case | What the spec/source says | Pointer | +|---|---|---| +| Absent | Server "MUST NOT respond with `200 OK`"; "SHOULD respond with `404 Not Found`, `410 Gone`, or any other suitable HTTP status code which does not imply the resource exists". Access denied MUST be `403`. | git http-protocol "General Request Processing" | +| Caching | Clients MAY reuse a cached entity per `Cache-Control`/`Expires` (follow RFC 2616); servers MAY return `304`; clients treat `304` as `200`. Smart services must prevent caching. The page states no Cache-Control value for object files. | same section | +| Content-Type | Specified only for `info/refs` (`text/plain; charset=utf-8` SHOULD, clients MUST NOT validate); nothing for object files. | same page | +| Failing digest | Client side: the dumb walker compares the received object's computed id with the requested id and errors `File %s has bad hash`. | https://raw.githubusercontent.com/git/git/master/http-walker.c l.540-541 | +| Missing target | The client treats a missing-target result as `HTTP_MISSING_TARGET` (http.c l.1944). | https://raw.githubusercontent.com/git/git/master/http.c | +| Content-Encoding | Client sets `CURLOPT_ENCODING ""` at http.c l.2284 (accept any encoding curl supports, curl decodes transparently) and sends `Pragma: no-cache` (l.1452). The http-protocol page says nothing about gzip for objects. | http.c | +| Range, CORS, ETag | Not in the page. | n/a | + +## 6. Browser fetch facts (verifying reader) + +| Fact | Evidence | +|---|---| +| CORS-safelisted response-header names are exactly `Cache-Control`, `Content-Language`, `Content-Length`, `Content-Type`, `Expires`, `Last-Modified`, `Pragma`. So `Content-Length` is readable cross-origin from a CORS response without `Access-Control-Expose-Headers`. `ETag` is **not** on the list; it needs `Access-Control-Expose-Headers` to be visible. | Fetch 2.2.2 "Headers", CORS-safelisted response-header name (https://fetch.spec.whatwg.org/#cors-safelisted-response-header-name) | +| The CORS-exposed header-name list comes from `Access-Control-Expose-Headers`; a CORS filtered response exposes safelisted names plus that list. | Fetch 2.2.6 "Responses" (https://fetch.spec.whatwg.org/#concept-response-cors-exposed-header-name-list); 3.3.3 | +| CORS check: failure if `Access-Control-Allow-Origin` is absent; "If request's credentials mode is not "include" and origin is `*`, then return success." With credentials mode `include`, `*` is not enough (needs exact origin plus `Access-Control-Allow-Credentials: true`). | Fetch 4.10 "CORS check"; 3.3.5 | +| A CORS-preflight happens for non-safelisted methods/headers (`GET`/`HEAD` with no custom headers are simple requests). | Fetch 3.3 (general); not re-quoted here | +| Fetch decodes content codings before exposing the body. In HTTP-network fetch: it records encoded size from transmitted bytes, sets `bytes` to "the result of handling content codings", then increases decoded size by the decoded length. Unsupported codings: bytes are returned as-is; decoding error: failure. | Fetch 4.7 "HTTP-network fetch"; 2.2.4 "handle content codings" (https://fetch.spec.whatwg.org/#handle-content-codings) | +| The spec itself notes: "This makes the `Content-Length` header unreliable to the extent that it was reliable to begin with." (the header describes the encoded body; the body the page reads is decoded.) | Fetch 4.7 (fetch_t.txt l.9670) | +| When a request has a `Range` header, the browser appends `Accept-Encoding: identity`, "This avoids a failure when handling content codings with a part of an encoded response." | Fetch 4.6 "HTTP-network-or-cache fetch" (l.8760-8767) | +| HTTP framing: with a valid `Content-Length`, a response is incomplete if fewer octets arrive than `Content-Length`; a client MUST record it as incomplete. `Content-Length` "refers specifically to the amount of data enclosed" (encoded form on the wire). | RFC 9112 section 8 "Handling Incomplete Messages"; RFC 9110 section 8.6 | +| `Cache-Control: no-transform`: an intermediary "MUST NOT transform the content". | RFC 9111 section 5.2.2.6 | +| `immutable` extension: identifies responses that will not be updated during freshness lifetime so clients need not revalidate. | RFC 8246 section 2 | + +## 7. HTTP core facts that apply to all five systems + +| Fact | Evidence | +|---|---| +| Range: "A server MAY ignore the Range header field"; a server MUST ignore Range for methods without defined range handling (GET is the only one). | RFC 9110 section 14.2 "Range" (l.6476-6484) | +| HEAD: "server SHOULD send the same header fields" as for GET; a `Content-Length` on HEAD MUST equal the GET length if sent. | RFC 9110 section 9.3.2 (l.3993), section 8.6 (l.3225) | +| `304`: a `Content-Length` in 304 MUST equal what a 200 would carry, if present. | RFC 9110 section 8.6 | +| `Content-Length` MUST NOT be sent on `204`. | RFC 9110 section 8.6 | + +## 8. Cross-check against the draft model page's from-memory claims + +| Draft claim (map l.44-49) | Finding | +|---|---| +| IPFS trustless gateway answers raw blocks with `Cache-Control: public, max-age=29030400, immutable` | In **PG**, not TG: the `/ipfs/` immutable value is a MUST there; TG itself does not state the response value. | +| IPFS uses the CID as `ETag` | PG: default `Etag: ""` quoted; variants add suffixes (raw: `".raw"`). | +| IPFS returns 400 for an unparseable CID | PG lists `400` as "generic client error" with malformed-CID as the example. TG itself lists no malformed-CID rule. | +| OCI `GET`/`HEAD` returns 200 with the digest header | Confirmed (`Docker-Content-Digest` MUST on GET and HEAD; `Content-Length` MUST on HEAD). | +| OCI 404 `BLOB_UNKNOWN` | 404 confirmed; `BLOB_UNKNOWN` exists as a code name; the blob-pull text does not mandate a JSON body. | +| OCI 400 `DIGEST_INVALID` on a bad digest | **Not in the blob-pull text.** Endpoint table lists only 404 for blob GET/HEAD; `DIGEST_INVALID` means "digest did not match uploaded content". | +| OCI Range (206/416) optional | Spec says "SHOULD support" Range per RFC 9110; no 206/416/Accept-Ranges text for pull. | +| Nix `.narinfo` 200 or 404, probed with HEAD | Client source: HEAD for existence; 404 and 410 both map to NotFound; 403 also treated as absent in `fileExists`. | +| Map's own claims (open `*` CORS, 500 on corrupt, Range ignored, Cache-Control immutable) | `*` with non-`include` credentials passes the CORS check (Fetch 4.10). No system surveyed specifies a status for a corrupt stored object. Ignoring Range is permitted (RFC 9110 14.2). | + +## 9. Where the systems agree and differ (matrix of what was found) + +| Item | IPFS TG/PG | OCI | Nix | Bazel (bazel-remote) | Git dumb | +|---|---|---|---|---|---| +| Absent | 404 (MUST, TG) | 404 (MUST) | 404/410 -> NotFound; 403 also "absent" in HEAD probe | 404 "Not found" | 404/410/other non-200 | +| Malformed id | PG 400 generic; TG silent | silent for blob pull | silent | 400 with message | silent | +| Corrupt stored object (server) | silent; client MUST verify | silent; client SHOULD verify | silent; client verifies hashes | no read-path check in handler | silent; client errors "bad hash" | +| Range | PG: 206 supported (raw blocks); TG silent | SHOULD support | client-side retry only | not in handler | not in spec | +| HEAD | same as GET minus body; 404 | 200 + digest + Content-Length MUST | used to probe | 200 + Content-Length or 404 | not in spec | +| CORS | none in TG/PG (Kubo adds implicit headers) | none | none | none | none | +| Cache-Control on content | PG: `public, max-age=29030400, immutable` MUST | none | none | none | defers to RFC 2616 | +| ETag | PG: quoted CID | none | none | none | 304 allowed | +| Content-Type | `application/vnd.ipld.raw` | unspecified (pull) | `application/x-nix-nar` etc. | `application/octet-stream` | unspecified (objects) | +| Content-Length vs encoding | PG warns they may differ | n/a | transport `Content-Encoding` separate from stored compression | zstd path omits Content-Length | client auto-decodes | +| nosniff / attachment | TG: Content-Disposition attachment MUST; PG: nosniff | none | none | none | none | + +## 10. Limits of this pass + +- Fetch spec 4.7 was read from the HTML text; the exact algorithm line that fails the fetch on a body shorter than Content-Length was not found in the Fetch text (framing is RFC 9112 section 8, which says the client records the message incomplete). +- bazel-remote digest verification on PUT was not read to the end; the read-path (GET) has no digest check in the handler. +- Kubo's actual default CORS header values were not retrieved (only the pointer to boxo and ipfs/specs#423). +- git CVEs were not searched; Nix and Bazel CVEs were not searched. +- OCI "CORS: not in spec" is from a search of the spec file, not from a registry implementation (distribution/distribution's own headers were not read). From 035d851127dd4b6cad098b52c7297a9da51070d6 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Mon, 19 Oct 2026 15:20:47 +0200 Subject: [PATCH 2/3] Docs: cultmesh-ts-content map pass 2 (C2/C3 facts F19-F27); target verify-before-use judges received bytes, not length headers Co-Authored-By: Claude Opus 5.5 --- docs/cultmesh-ts-content-map.md | 341 ++++++++++++++++++++--------- docs/cultmesh-ts-content-target.md | 6 +- 2 files changed, 241 insertions(+), 106 deletions(-) diff --git a/docs/cultmesh-ts-content-map.md b/docs/cultmesh-ts-content-map.md index 43f2ab03..592d0b19 100644 --- a/docs/cultmesh-ts-content-map.md +++ b/docs/cultmesh-ts-content-map.md @@ -1,16 +1,19 @@ # TypeScript CultMesh Content Plane: Map Campaign `cultmesh-ts-content`. Imagination wrote this on 2026-10-19 at CultLib -`8990bd46` (origin/main `a88c629e` plus the target doc). The target is -`docs/cultmesh-ts-content-target.md`. This page holds the body facts, the model +`8990bd46` (origin/main `a88c629e` plus the target doc), and refreshed it the +same day at `2de07065` for C2 and C3, reading C1 from `b8b825e1`. The target +is `docs/cultmesh-ts-content-target.md`. The HTTP prior art is +`docs/research/2026-10-19-content-addressed-http-answers.md` (cited below as +**R**, with its section number). This page holds the body facts, the model page, the rationale and the cut sequence. Specs and questions live in the typed mind, not here. ## 1. Body facts -Each fact was probed by running code or read from source at `8990bd46`. -Scratch probes ran under the session scratchpad (`imagination/`), not in the -repo. +F1-F18 were probed by running code or read from source at `8990bd46`. F19-F27 +were probed or read at `b8b825e1` (C1's head) and `2de07065`. Scratch probes +ran under the session scratchpad (`imagination/`), not in the repo. | # | Fact | Evidence | |---|---|---| @@ -32,127 +35,257 @@ repo. | F16 | **No CultLib TS package except `cultcache-ts` has a publish job.** `publish-packages.yml` triggers on `cultcache-ts-v*`, `cultcache-py-v*`, `cultnet-py-v*` and `cultmesh-py-v*` only. Tags `cultnet-ts-v0.1.1` and `cultmesh-ts-v0.1.1` exist, but no workflow consumes them. `npm view` returns 404 anonymously for `cultnet-ts`, `cultmesh-ts`, `cultmesh-browser` and even `@gamecult/cultcache-ts` (`@gamecult/cultcache-ts` may be private; unverified). The doctrine's rule of at most three tags per push stands (SKILL.md:674). | source; `npm view` | | F17 | StreamPixels consumes CultLib as the git submodule `vendor/CultLib` (gitlink `30ee8b97`) with `file:` deps and `pnpm.overrides`. Its CI and Idunn recipes build the vendored workspaces (`npm ci --ignore-scripts`, then build `cultcache-ts`, `cultnet-ts` and `cultmesh-ts`). `apps/web` depends only on `cultnet-ts` today, not on `cultmesh-browser`. | StreamPixels `.gitmodules`, `apps/*/package.json`, CI workflow lines 62-95 | | F18 | The Rust reader sets two guards that the TS reader should match. A caller cap `max_bytes` is checked before any request. And a zero-size chunk is refused unless it is the only one, because a manifest of zero-size chunks would be a request flood. | source `cultnet-rs/src/content.rs:364-385` | +| F19 | `BuildChunkUri` on more endpoint shapes, as `AbsoluteUri` (what `HttpClient` sends). `https://h` and `https://h/` both give `https://h/?chunkHash=…&recordKey=k`. `https://h:443/x` drops the default port. `https://H.Example/X/` lowercases the host and keeps the path's case. `https://h/x#frag` keeps the fragment after the query. `https://h/x?a=%20b+c` keeps the existing query verbatim and joins with `&`. **`https://h/x?` (an empty query) gives `https://h/x?&chunkHash=…`**, because `UriBuilder.Query` is `?`, which is not blank. `https://[::1]:8443/c` keeps the bracketed host. `Uri.EscapeDataString("!'()*~-._ +é𐐀")` is `%21%27%28%29%2A~-._%20%2B%C3%A9%F0%90%90%80`: unlike JS `encodeURIComponent`, it escapes `!'()*`. | probe: `dotnet fsi` on .NET 10, the three lines of `BuildChunkUri` (`CultMeshHttpsContentTransport.cs:114-122`) copied verbatim | +| F20 | Node 24.15 global `fetch` (undici) against a raw-socket server. A gzip body (`Content-Encoding: gzip`, `Content-Length: 29`) arrives as 1,000 decoded bytes, while `headers.get("content-length")` still says `29`. A body shorter than its `Content-Length` (400 of 1,000) rejects while reading: `TypeError: terminated`, cause code `UND_ERR_RES_CONTENT_LENGTH_MISMATCH`. A body longer than its `Content-Length` (1,000 against 400) is cut silently to 400 by HTTP framing. With no `Content-Length`, all 1,000 bytes arrive. So a reader that trusts the header refuses valid gzipped content, and it learns nothing the received bytes do not tell it (R §6: Fetch decodes content codings, and its own note calls `Content-Length` unreliable). | probe: `imagination/fetchprobe.mjs` | +| F21 | The C# TCP content server is the reference for what a provider does per request (`SendChunkAsync`, `CultMeshTcpContentTransport.cs:384-430`). It runs `NormalizeHash` on the requested hash, refuses a non-blank `RecordKey` that is not `CreateRecordKey(hash)` (ordinal compare, message "record key disagrees with its content hash"), resolves the chunk **by hash** (`_resolveChunk(hash)`), and runs `CultMeshCdn.ValidateChunkPayload` before it sends a byte. So **verify-before-serve on every request is the reference behaviour**. C# chunk storage is one record per chunk, keyed by the chunk record key (`CultMeshDatabaseContentProvider`, `CultMeshContentTransfer.cs:41-71`). Per-chunk storage is the reference layout. | source | +| F22 | C1's exports at `b8b825e1` (`packages/cultnet-ts/src/content.ts`): `normalizeHash`, `parseSha256Hex`, `sha256Hex`, `chunkRecordKey`, `artifactRecordKey` (async), `validateManifest`, `encodeManifest`, `decodeManifest`, `packContent`, `CULTMESH_CDN_MANIFEST_SCHEMA`, `DEFAULT_CHUNK_SIZE_BYTES` (4,194,304), `EMPTY_SHA256`, and the types `CultMeshCdnChunkRef`, `CultMeshCdnArtifactManifest`, `CultMeshCdnPackOptions` and `CultMeshCdnPackedChunk`. `sha256Hex` throws a plain `Error` ("WebCrypto SHA-256 is unavailable.") when `crypto.subtle` is missing. `validateManifest` takes the whole manifest (it reads `metadata`). `cultnet-ts/authority` already exports `isLoopbackEndpoint` (`cultmesh-authority.ts:377`: `localhost`, `loopback`, `::1`, `::ffff:7f00:1`, `127.x.x.x`), and `cultmesh-browser` already imports it (`index.ts:23`). | `git show b8b825e1:…`; cut report `cut-c1-shapes.h1` landed names | +| F23 | The consumer (StreamPixels `dfbe2a6`, `docs/web-character-customization-map.md`) stores one whole-artifact file per content hash at `/srv/streampixels/content/`, builds manifests at service boot, and hashes every referenced blob at boot (lines 149-159, 175, 317-325). Its artifacts are 10-125 KB (W10), well under the 4 MiB default chunk. It also builds manifests for `catalog.cc` and `character-artifacts.cc`, whose sizes the map does not give. It mounts the handler at `GET /api/content` on its existing server (line 320). Its sketch, `createHttpsContentHandler(store)` (line 293), takes `(chunkHash, recordKey)` and returns bytes or not-found. | StreamPixels `git show dfbe2a6:docs/web-character-customization-map.md` | +| F24 | Build shapes for the two packages. `cultmesh-ts` is CommonJS (`module` and `moduleResolution` `Node16`), and its `package.json` exports only `.` (lines 7-13). `cultmesh-browser` is ESM (`"type": "module"`, `moduleResolution` `Bundler`) and is one file, `src/index.ts`, with no relative imports. A new relative import must therefore be spelled `./content.js`, or Node's ESM loader cannot find it at test time. Both packages' `test` scripts run the glob `dist-test/test/*.test.js`, which the wiring check accepts (`scripts/check-typescript-tests-wired.mjs`). CI runs both in `cultmesh-portability.yml`: the browser at lines 119-120, and `cultmesh-ts` at lines 121-129 under the step name "Verify the realtime frame codec with the TypeScript codec". `esbuild` resolves from the root `devDependencies`, as C1's browser-bundle test shows. | source at `b8b825e1` | +| F25 | The existing TLS fixture `packages/cultmesh-ts/test/fixtures/quic-test.p12` is a self-signed P-256 certificate with subject `CN=cultmesh-quic-native-tests` and no subjectAltName (fixtures README). Node's TLS hostname check therefore refuses it for `127.0.0.1` even when it is trusted through `NODE_EXTRA_CA_CERTS`. A C# handler that pins by certificate hash does not care about the name. | source: fixtures README | +| F26 | `CultMeshCdn.ValidateChunkPayload`, `HashBytes`, `ValidateManifest` and `NormalizeHash` are `internal`, and `GameCult.Mesh` declares no `InternalsVisibleTo`. A test in `tests/GameCult.Mesh.Tests` can therefore verify chunk bytes only with `SHA256.HashData`, or through a public consumer of the connector. The public surface is `CreateRecordKey`, `PackArtifact`, `PublishAsync` and `ReadArtifact(Async)`. | grep of `CultMeshCdn.cs`, `src/GameCult.Mesh` | +| F27 | No CultLib workflow runs `cultmesh-ts` interop tests. `cultnet-interop.yml` (windows-latest, main only) runs `cultnet-ts`'s `test/interop`, and its QUIC native case spawns `dotnet test --filter NativeConnectorReceivesFromExternalProvider` through `runDotnetTestFilter` (`cultnet-interop.test.ts:550,2826`). `cultmesh-ts` has a `test/interop/` folder (a QUIC peer script) but no `test:interop` script. | grep of `.github/workflows`, source | ## 2. Model page There is one row per persistent or wire-visible kind. Identity is what names the kind, lifecycle is what happens to it over time, and authority is who decides. The HTTP answers are new design: no runtime has an HTTPS content server (target, -"Why now"). They are grounded in three content-addressed HTTP stores, as I -recall their specs; I did not fetch the specs in this pass. The IPFS trustless -gateway answers raw blocks with `Cache-Control: public, max-age=29030400, -immutable`, uses the CID as `ETag`, and returns 400 for an unparseable CID. The -OCI distribution spec's `GET`/`HEAD /v2//blobs/` returns 200 with -the digest header, 404 `BLOB_UNKNOWN` and 400 `DIGEST_INVALID`, and treats Range -(206/416) as optional. Nix binary caches serve `.narinfo` with 200 or 404, -and `nix` probes them with HEAD. My confidence in those three shapes is -moderate to high. +"Why now"). They rest on R, which fetched the specs and sources on 2026-10-19 +and corrected this page's first draft (R §8). In short: + +- **404 for absent** is universal (R §9). Nix also reads 410 and S3's 403 as + absent; we send 404 only. +- **400 for a malformed id** comes from the IPFS path gateway (generic client + error, with a malformed CID as its example) and bazel-remote (a hash that is + not 64 lowercase hex). The OCI blob-pull text defines no 400; its + `DIGEST_INVALID` is an upload code, so the first draft was wrong to cite it. +- **No system specifies a server answer for a stored object that fails its + digest** (R §9). Every one puts verification on the client. bazel-remote + streams without checking, so a failure after the headers have gone "cannot + change the status" (R §4). That is the scar `verify-before-serve` avoids by + buffering and hashing before the status line. +- **Range** may be ignored (RFC 9110 §14.2). OCI says registries SHOULD + support it, which the first draft called "optional". We ignore it. +- **`Cache-Control: public, max-age=29030400, immutable`** is the IPFS path + gateway's MUST for immutable content (R §1). The trustless gateway itself + does not state it. +- **The Fetch facts** decide the reader (R §6). Fetch decodes content codings + before the page sees the body, so `Content-Length` describes the encoded + body, which F20 reproduces in Node. `Content-Length` is CORS-safelisted, and + `ETag` is not. `Access-Control-Allow-Origin: *` passes the CORS check for any + request whose credentials mode is not `include`. | Kind | Identity (what names it) | Lifecycle | Authority (who decides; forbidden writers) | |---|---|---|---| -| **Manifest** `gamecult.mesh.cdn_artifact_manifest.v1` | The record key (row below), derived from `artifactId`, `version` and `contentHash`. On the wire it is the 10-array of F1. Its namespace is the schema id. Two manifests for one body under different ids or versions are distinct and legitimate. | Immutable once packed. A new body or version is a new manifest and a new key. Nothing revises one in place, and deletion is not in scope (no GC). `createdAtUtc` is set at pack, and a caller may pin it so that repacking is reproducible. | The packer (`packContent` in `cultnet-ts/content`) is the only TS writer. The encoder and decoder in the same module are the only TS byte owners. **Forbidden:** a second TS manifest type or encoder in `cultmesh-ts`, `cultmesh-browser` or a consumer, and any URL or endpoint field. | -| **Chunk ref** `CultMeshCdnChunkRef` | Its `chunkHash` (SHA-256 of the chunk bytes) within one manifest, plus `offset`. Two refs may share a hash, which is legal and served once. | Immutable inside its manifest. `recordKey` is derived and may be blank on read (C# and Rust fall back to `mesh:cdn:chunk:`). | Packer writes; `validateManifest` judges contiguity, non-negative sizes and the size sum (C# `ValidateManifestShape` order). Reader and provider never edit one. | -| **Chunk record key** `mesh:cdn:chunk:` | Fully derived from `normalizeHash(chunkHash)`. It is injective only over valid hex, because C# accepts any non-blank string (F7). | Derived on demand, never stored independently of a manifest. | `chunkRecordKey` in `cultnet-ts/content`. On the HTTP wire the provider **checks** a non-blank `recordKey` against it and never derives a file from it. | -| **Artifact record key** `mesh:cdn:artifact:::` | Derived by `StableToken` (F8). It is not injective: `A/b` and `a b` both give `a-b`, and the content hash at the end disambiguates. | Derived on demand. | `artifactRecordKey` in `cultnet-ts/content` mirrors C# per UTF-16 unit. Truth comes from the C#-written `contracts/cultmesh/cdn-record-key-vectors.json`, and TS never authors expectations for it. | -| **Content hash / chunk hash** | Lowercase 64-hex SHA-256. On input, `normalizeHash` accepts `sha256:`, case and surrounding whitespace, as C# does. The HTTP boundary and the reader also demand `/^[0-9a-f]{64}$/`. | Immutable. It *is* the identity. | `sha256Hex` (WebCrypto `crypto.subtle`) in `cultnet-ts/content` is the only TS hasher. The provider and the reader call it and keep no copy. | -| **Stored chunk** (provider disk) | File `/`, flat, one file per distinct chunk hash. For a single-chunk artifact (every StreamPixels artifact, since 125 KB is well under the 4 MiB default chunk) this is the body file StreamPixels already plans (`/srv/streampixels/content/`). | Written by the publisher outside the provider (StreamPixels' publish script) and never written by the provider. Immutable. Deletion has no owner (GC is out of scope). A corrupt or oversized file stays on disk and is never served. | The publisher writes; **the provider is read-only**. The provider verifies each read against its name before serving (`verify-before-serve`). **Forbidden:** any path derived from `recordKey`, the URL path or an unvalidated hash. | -| **Provider store index** | **None.** The file name is the index: the store maps hex to `/` on each request after the hex check. There is no in-memory table, so nothing can go stale. | Not applicable: it is computed per request. | `createDirectoryChunkStore(root)`, a read-only lookup port. A host may supply another `ContentChunkStore`, and the provider verifies whatever any store returns. | -| **The empty chunk** `e3b0c442…b855` | The SHA-256 of zero bytes. | Intrinsic: it needs no file. | The provider answers it as 200 with `Content-Length: 0` without consulting the store, so empty artifacts work against any store. The reader verifies it locally and sends no request. | -| **Wire request** | `GET` or `HEAD ?chunkHash=&recordKey=`. Extra query parameters are ignored (an endpoint may carry its own, F5). The mount path belongs to the host. | Stateless. | Built by the C# connector or the TS reader. The TS reader builds the URL **byte-identical** to `BuildChunkUri` (escaping as `Uri.EscapeDataString`), so C# and browser clients share HTTP cache entries. | -| **Answer 200** (GET) | The body is exactly the verified chunk bytes. | Immutable per URL. | Provider. Headers: `Content-Length: `, `Content-Type: application/octet-stream`, `X-Content-Type-Options: nosniff`, `Cache-Control: public, max-age=31536000, immutable, no-transform`, `ETag: ""` and `Access-Control-Allow-Origin: *`. `no-transform` stops a proxy from compressing the body, which would make `Content-Length` disagree with the decoded size (F6). `nosniff` with octet-stream keeps a public chunk from being rendered as HTML on the host's origin. | -| **Answer 200** (HEAD) | The same headers as GET, with no body. | Same as GET. | Provider. It verifies exactly as GET does, so HEAD can never advertise a corrupt chunk. | -| **Answer 304** | The request's `If-None-Match` contains `""` or `*`. | Same as 200. | Provider, answered before the store is touched. It is safe because the ETag is the content identity. | -| **Answer 400** | `chunkHash` missing, repeated, or not 64 hex after `normalizeHash`; or a non-blank `recordKey` that is not `mesh:cdn:chunk:` (the C# server's `record key disagrees` rule). | Stateless. | Provider. Short `text/plain` reason, no echo of input, `nosniff`. This follows OCI `DIGEST_INVALID` and IPFS's 400 for a bad CID. | -| **Answer 404** | A valid hash whose chunk the store does not hold. | Stateless; a later publish can turn it into 200. | Provider (OCI `BLOB_UNKNOWN`, Nix narinfo 404). The answer carries `Cache-Control: no-store`, so an absence is never cached as immutable. | -| **Answer 405** | Any method other than GET, HEAD or OPTIONS. | Stateless. | Provider, with `Allow: GET, HEAD, OPTIONS`. | -| **Answer 204** (OPTIONS) | A CORS preflight. The reader's GET is a simple request and never preflights. This answer exists for clients that add headers. | Stateless. | Provider: `Access-Control-Allow-Origin: *`, `Access-Control-Allow-Methods: GET, HEAD`, `Access-Control-Allow-Headers: If-None-Match`, `Access-Control-Max-Age: 86400`. | -| **Answer 500** | The store returned bytes whose SHA-256 is not the requested hash, or a file larger than `maxChunkBytes`. | Persists until the operator replaces the file. | Provider. It sends no body bytes and calls the host's `onCorruptChunk(hash)` hook so the host can fail health. It is 500 and not 404 because this is a server fault the operator must see, and a 404 would hide corruption as absence. | -| **Range** | Not supported. A `Range` header is ignored, the answer is 200 with the full chunk (RFC 9110 permits this), and no `Accept-Ranges` header is sent. | Not applicable. | Provider. The chunk is the resumption unit, and resumable TS state is out of scope. OCI treats Range as optional too. | -| **Reader result / error** | `Uint8Array` of the whole body, or one `CultMeshContentError` with `code` ∈ {`manifest-invalid`, `too-large`, `insecure-endpoint`, `webcrypto-unavailable`, `http-status`, `length-mismatch`, `chunk-hash-mismatch`, `content-hash-mismatch`}. An abort rejects with the signal's own reason (`AbortError`). | Transient, held in memory only. No byte of a failed chunk or body is returned, in whole or in part (`verify-before-use`). | `fetchVerifiedContent` in `cultmesh-browser`. Its concurrency window defaults to 4 and is capped at 32 (the C# transfer owner's bounds, `content-sessions.md`). `maxBytes` defaults to 256 MiB and is checked before any request. `https:` is required, except `http:` on loopback hosts for development. | -| **Endpoint** | A URL the caller resolved through CultMesh. | Per call; never persisted in content. | The caller. **Forbidden:** any manifest, chunk-ref or record field carrying it (`no-urls-in-content`). | -| **Vector files** | `content-vectors.cs-written.json` (C#), `.rs-written.json` (Rust), new `content-vectors.ts-written.json` (TS, manifests only, own inputs), new `cdn-record-key-vectors.json` (C#, record-key traps). | Each is rewritten only by its writer under `CULTNET_WRITE_VECTORS=1`. CI never sets the variable: the committed files are the contract. | Each writer owns its file. C# judges the TS file, and TS judges both C# files. The cs-written file gains no kind or label (F10). | +| **Manifest** `gamecult.mesh.cdn_artifact_manifest.v1` | The record key (row below), derived from `artifactId`, `version` and `contentHash`. On the wire it is the 10-array of F1. Its namespace is the schema id. Two manifests for one body under different ids or versions are distinct and legitimate. | Immutable once packed. A new body or version is a new manifest and a new key. Nothing revises one in place, and deletion is not in scope (no GC). `createdAtUtc` is set at pack, and a caller may pin it so that repacking is reproducible. | `packContent` in `cultnet-ts/content` (C1, `b8b825e1`) is the only TS writer. The encoder and decoder in the same module are the only TS byte owners. The reader takes a manifest value and never builds or edits one. **Forbidden:** a second TS manifest type or encoder in `cultmesh-ts`, `cultmesh-browser` or a consumer, and any URL or endpoint field. | +| **Chunk ref** `CultMeshCdnChunkRef` | Its `chunkHash` (SHA-256 of the chunk bytes) within one manifest, plus `offset`. Two refs may share a hash: this is legal, and the reader fetches each ref. | Immutable inside its manifest. `recordKey` is derived and may be blank on read. A blank key goes on the wire as `recordKey=` (F5), and the provider accepts it. | The packer writes. `validateManifest` (C1) judges contiguity, non-negative sizes and the size sum. The reader and provider never edit one. | +| **Chunk record key** `mesh:cdn:chunk:` | Fully derived from `normalizeHash(chunkHash)`. It is injective only over valid hex, because C# accepts any non-blank string (F7). | Derived on demand, never stored independently of a manifest. | `chunkRecordKey` in `cultnet-ts/content`. On the HTTP wire the provider **checks** a non-blank `recordKey` against it, as the C# TCP server does (F21), and never derives a file from it. | +| **Artifact record key** `mesh:cdn:artifact:::` | Derived by `StableToken` (F8). It is not injective: `A/b` and `a b` both give `a-b`, and the content hash at the end disambiguates. | Derived on demand. | `artifactRecordKey` in `cultnet-ts/content` (async; C1 deviation). Truth comes from the C#-written `contracts/cultmesh/cdn-record-key-vectors.json`. Neither C2 nor C3 uses it. | +| **Content hash / chunk hash** | Lowercase 64-hex SHA-256. On input, `normalizeHash` accepts `sha256:`, any case and surrounding whitespace, as C# does. The HTTP boundary and the reader also demand 64 hex through `parseSha256Hex`. | Immutable. It *is* the identity. | `sha256Hex` and `parseSha256Hex` (C1) are the only TS hasher and the only hex rule. The provider and reader call them and keep no copy. The negative greps pin this. | +| **Stored chunk** (provider disk) | File `/`, flat, one file per distinct chunk hash. This is the reference layout (F21: C# stores one record per chunk). For a single-chunk artifact the chunk hash equals the content hash, so StreamPixels' whole-artifact file `/srv/streampixels/content/` (F23) is already a stored chunk. A multi-chunk artifact's publisher writes `packContent(...).chunks`, one file each. | Written by the publisher outside the provider, and never written by the provider. Immutable. Deletion has no owner (GC is out of scope). A corrupt or oversized file stays on disk and is never served. | The publisher writes; **the provider is read-only**. The provider verifies every read against its name before serving (`verify-before-serve`, the C# per-request behaviour of F21). A symlink or a replaced file cannot leak anything, because only bytes that hash to the requested name are ever served. **Forbidden:** any path derived from `recordKey`, the URL path or an unvalidated hash. | +| **Provider store index** | **None.** The file name is the index. The directory store maps hex to `/` on each request, after its own `parseSha256Hex` equality check. No in-memory table exists, so nothing can go stale. | Not applicable: it is computed per request. | `ContentChunkStore` is the port: `readChunk(chunkHash, maxBytes) → Promise`. `createDirectoryChunkStore(root)` stats before reading and rejects a file over `maxBytes` with `ContentChunkTooLargeError`, so an oversized file is never read into memory. A host may supply another store, for example one indexed from manifests, and the handler verifies whatever any store returns. | +| **The empty chunk** `e3b0c442…b855` | The SHA-256 of zero bytes. | Intrinsic: it needs no file. | The provider answers it as 200 with `Content-Length: 0` without consulting the store, so empty artifacts work against any store (bazel-remote does the same, R §4). The reader verifies it locally and sends no request. | +| **Wire request** | `GET` or `HEAD ?chunkHash=&recordKey=`. Extra query parameters are ignored, because an endpoint may carry its own (F5). The mount path belongs to the host, and the handler never reads it. | Stateless. | Built by the C# connector or the TS reader. The TS reader builds the URL **byte-identical** to `BuildChunkUri`, `Uri.EscapeDataString` escaping (F19) and the empty-query `?&` join included, so C# and browser clients share HTTP cache entries. | +| **Content coding** | Identity, always. | Not applicable. | Provider: no `Content-Encoding`, and `Cache-Control` carries `no-transform`, which forbids an intermediary to transform the content (RFC 9111 §5.2.2.6). That keeps the C# connector's `Content-Length` check meaningful (F6). The reader does not rely on it: it judges the bytes it received (F20). | +| **Answer 200** (GET) | The body is exactly the verified chunk bytes. | Immutable per URL. | Provider. Headers: `Content-Length: `, `Content-Type: application/octet-stream`, `X-Content-Type-Options: nosniff`, `Cache-Control: public, max-age=31536000, immutable, no-transform` and `Access-Control-Allow-Origin: *`. Octet-stream with `nosniff` keeps a public chunk from rendering as HTML on the host's origin. The IPFS `Content-Disposition: attachment` adds nothing over that for a fetch client, so it is not sent. | +| **Answer 200** (HEAD) | The same headers as GET, with no body. | Same as GET. | Provider. HEAD verifies exactly as GET does, so it can never advertise a corrupt chunk. RFC 9110 §8.6 requires HEAD's `Content-Length` to equal GET's. | +| **No ETag, no 304** | Not applicable. | Not applicable. | Provider. `immutable` with a one-year max-age means browsers do not revalidate. `ETag` is not CORS-exposed (R §6), and the URL is already the content identity. An `ETag` without a 304 path is noise, and a 304 path is a second answer to test. The first draft had both, and both are cut. | +| **Answer 400** | `chunkHash` missing, repeated, or not 64 hex after `normalizeHash`. Or `recordKey` repeated, or non-blank and not exactly `mesh:cdn:chunk:` (the C# server's "record key disagrees" rule, F21). | Stateless. | Provider. Body is a fixed `text/plain; charset=utf-8` reason with no echo of input (bazel-remote echoes the URL; we do not). Headers: `nosniff`, `Cache-Control: no-store`, `Access-Control-Allow-Origin: *`. | +| **Answer 404** | A valid hash whose chunk the store does not hold. | Stateless; a later publish can turn it into 200. | Provider. Same error headers as 400. `no-store` keeps an absence from being cached as immutable. | +| **Answer 405** | Any method but GET and HEAD, OPTIONS included. | Stateless. | Provider, with `Allow: GET, HEAD`. The reader's GET is a CORS simple request (no custom headers, credentials omitted) and never preflights. A client that adds headers would preflight, and its preflight is refused, which is correct because the provider honours no request header. The first draft's 204 preflight answer is cut. | +| **Answer 500** | The store returned bytes whose SHA-256 is not the requested hash; the store holds a file over `maxChunkBytes`; or the store failed. | Persists until the operator replaces the file. | Provider. It sends a fixed reason and none of the chunk's bytes, then calls the host's `onServerFault({ chunkHash, kind, error })` with `kind` ∈ {`hash-mismatch`, `too-large`, `store-error`}, so the host can fail health. A throwing hook does not change the answer. It is 500 and not 404 because the IPFS trustless gateway reserves 500 for "genuine server errors" (R §1), and corruption is one the operator must see, while a 404 would hide it as absence. | +| **Range** | Not supported. A `Range` header is ignored, and the answer is 200 with the full chunk (RFC 9110 §14.2). No `Accept-Ranges` header is sent. | Not applicable. | Provider. The chunk is the resumption unit, and resumable TS state is out of scope. | +| **Provider memory** | At most `maxChunkBytes` buffered per in-flight request. | Per request. | Provider. Buffering is forced, because the status line cannot be sent before the hash is known (the bazel-remote scar above). `maxChunkBytes` defaults to `DEFAULT_CHUNK_SIZE_BYTES` (4 MiB) and must lie in 1..2147483647. The host bounds concurrency (server `maxConnections` or a reverse proxy), and the README says so. | +| **Reader result / error** | A `Uint8Array` of the whole body, or one `CultMeshContentError` with `code` ∈ {`manifest-invalid`, `too-large`, `insecure-endpoint`, `webcrypto-unavailable`, `network`, `http-status`, `length-mismatch`, `chunk-hash-mismatch`, `content-hash-mismatch`}, plus `chunkHash` and `status` where they apply. A caller abort rejects with the signal's own `reason`, unwrapped. Bad options reject with `RangeError`. | Transient, held in memory only. No byte of a failed chunk or body is returned, in whole or in part (`verify-before-use`). | `fetchVerifiedContent` (and `fetchVerifiedChunk`) in `cultmesh-browser`. Concurrency defaults to 4 and is bounded 1..32 (the C# `CultMeshContentTransferOptions` bounds). `maxBytes` defaults to 256 MiB and is checked before any request. The scheme must be `https:`, or `http:` where `isLoopbackEndpoint` (F22) holds. Only status 200 is accepted. **`Content-Length` is not consulted** (F20). The body is read through a stream into the chunk's slot, a body that runs past the chunk's size is cancelled at the first excess byte, and one that ends short is refused. | +| **Endpoint** | A URL the caller resolved through CultMesh. | Per call; never persisted in content. | The caller. **Forbidden:** any manifest, chunk-ref or record field carrying it (`no-urls-in-content`). Error messages never echo it, because an endpoint query may carry a tenant or a token. | +| **Vector files** | `content-vectors.cs-written.json` (C#), `.rs-written.json` (Rust), `content-vectors.ts-written.json` (TS, manifests only, own inputs) and `cdn-record-key-vectors.json` (C#, record-key traps), all landed by C1. | Each is rewritten only by its writer under `CULTNET_WRITE_VECTORS=1`. CI never sets the variable: the committed files are the contract. | Each writer owns its file. C2 and C3 add no vector file. HTTP behaviour is pinned by tests and, in C4, by the C# connector itself. | -No cell above is a guess, with two marked exceptions: the prior-art shapes -(recalled, not fetched) and `@gamecult/cultcache-ts`'s npm visibility (F16). +Two cells rest on reasoning rather than a probe: that the host, not the +provider, bounds concurrency, and the cost of hashing on HEAD. Both are +defaults a later cut can revise without changing any shape. ## 3. Rationale **Where the CDN shapes live: `cultnet-ts/src/content.ts`, exported as -`cultnet-ts/content`.** Both consumers already depend on `cultnet-ts`, and its -subpaths are the repo's established browser-safe entry points (F14). A shapes -module inside `cultmesh-browser` cannot be imported by `cultmesh-ts`, which is -CommonJS. A shapes module inside `cultmesh-ts` drags a Node package into the -browser. A new package would add a release surface for roughly 250 lines. The -placement also matches Rust, where `cultnet-rs/src/content.rs` owns the same -rules. An esbuild `platform: "browser"` bundle test pins the browser-safety -claim. +`cultnet-ts/content`.** C1 landed this at `b8b825e1`. Both consumers already +depend on `cultnet-ts`, and its subpaths are the repo's established +browser-safe entry points (F14). The placement matches Rust, where +`cultnet-rs/src/content.rs` owns the same rules. -**Hashing is WebCrypto in one place.** `crypto.subtle` exists in browsers (secure -contexts) and in Node 20+, so one hasher serves the packer, the provider and the -reader. The packer is therefore async. WebCrypto has no incremental digest, so -the whole-body hash needs the body in memory. That is acceptable because the -target keeps artifacts in memory in the browser, and a Node publisher of very -large artifacts would want a streaming packer. That packer is not in scope. +**Hashing is WebCrypto in one place.** `crypto.subtle` exists in browsers +(secure contexts) and in Node 20+, so one hasher serves the packer, the +provider and the reader. WebCrypto has no incremental digest, so every hash +needs its bytes in memory. That fits all three users: the reader holds the +body (the target keeps artifacts in memory), the provider holds one chunk at a +time, and the packer holds the body. -**Decoding follows the siblings, not a canonical-bytes tribunal.** The manifest -decoder accepts any integer width (as C# does), refuses duplicate metadata keys -(as C# and Rust do, using `mapKeyConverter`), keeps metadata as ordered entries, -keeps int64 exact, and refuses unsafe integers. Before decoding, it runs the -payload depth walk (ruling `payload-depth-limit-256`). That needs -`checkPayloadDepth` exported from `@gamecult/cultcache-ts/document`, a one-name -export from its existing owner, rather than a second walk in `cultnet-ts` -(F4). +**Decoding follows the siblings.** C1's report records two places where its +decoder is stricter than MessagePack-CSharp: an exact field count, and refusal +of nil strings. Ruling `leaf-acceptance-follows-reference` may make that a +finding. That belongs to C1's Soul pass, not to C2 or C3, which take manifest +values and never decode bytes. -**No TS CultCache document definition for manifests yet.** The shapes module -gives bytes, values and keys. Defining `gamecult.mesh.cdn_artifact_manifest` as -a `cultcache-ts` document needs a formatter, and it would bring the C# `[CultIndex]` -declarations on `Kind`, `Version` and `ContentHash` with it. Ruling -`variants:unique-indexes` says `[CultIndex]` means unique, which would bar two -manifests of one kind in a store. I did not probe whether the reference already -trips on this. No consumer asks for a TS store of manifests: StreamPixels builds -manifests at boot. This is recorded as a follow-up, not done. +**No TS CultCache document definition for manifests yet.** This is still +follow-up `ts-manifest-document-definition`. -**Provider: `cultmesh-ts/content`, a pure core plus a Node adapter.** The core is -`createContentChunkHandler(store, options) => (request) => Promise`, -where a request is `{ method, url, headers }` and an answer is -`{ status, headers, body? }`. It is framework-neutral, testable without a -socket, and mountable on `node:https`, Express, Fastify or Next. A `toNodeListener` -adapter of about 15 lines covers the bare `node:https` case. The name says -"chunk" because the handler answers chunks only. StreamPixels' sketch, -`createHttpsContentHandler`, implied it does TLS, and it does not: the host or -a reverse proxy terminates TLS. The store is a narrow port, `readChunk(hex) → -bytes | undefined`. The core verifies whatever any store returns, so -`verify-before-serve` does not depend on the store being honest. +**Provider: `cultmesh-ts/content`, a request-to-answer core, one store and one +Node adapter, in one file.** `createContentChunkHandler(store, options)` +returns `(request: { method, url }) => Promise<{ status, headers, body? }>`. +The request carries no headers, because the provider honours none: Range is +ignored, and conditional requests have no answer now that ETag is cut. The +core reads only the query, never the path, which is what makes +`chunk-only-route` structural rather than checked. `toNodeListener(handler)` +is about 15 lines for `node:http(s)`, and Express, Fastify or Next mount the +core the same way. The name says "chunk" because the handler answers chunks +only. StreamPixels' sketch name, `createHttpsContentHandler` (F23), implied it +does TLS, and it does not: the host terminates TLS. One file, not three, +because the store and adapter are about 50 lines between them and have no +consumer apart from the core. `cultmesh-ts` is already a Node package, so +`node:fs` in the file costs nothing. -**Reader: `fetchVerifiedContent(endpoint, manifest, options)` in -`cultmesh-browser`.** Before any request it validates everything: shape, cap, -the zero-size rule (F18), and hex hashes. It fetches in a bounded window with -`credentials: "omit"` and an internal `AbortController` chained to the caller's -signal. It refuses a mismatched `Content-Length` before reading. With no -`Content-Length`, it reads through a bounded stream reader. It hashes each -chunk, places it at its offset in a private buffer, and returns that buffer -only after the whole-body hash matches. The C# connector's URL shape is copied -byte for byte (F5), so the pin is a fixture taken from the probe output. +**Store layout: per-chunk flat files.** This is the reference layout (F21), +and it needs no index that can go stale. It coincides with StreamPixels' +whole-artifact files, because every StreamPixels artifact is a single chunk +(F23). The alternative is a store indexed from manifests, mapping a chunk hash +to (artifact file, offset, size), so that multi-chunk artifacts could be +stored whole. It is not shipped, because it adds a derived index that must be +rebuilt when manifests change, and no consumer needs it. A host that wants it +writes about 30 lines against the port, and the handler still verifies every +byte. **Consumer note for StreamPixels B3:** its `catalog.cc` and +`character-artifacts.cc` manifests must stay single-chunk, so their files are +the stored chunk. Either they are packed with `chunkSizeBytes` at least the +file size and the provider's `maxChunkBytes` is raised to match, or B3 +publishes the chunk files. -**The cross-process test runs from TS, as F13 does.** A `node --test` interop -test starts the TS provider over `node:https` with a throwaway self-signed -certificate. It then runs one env-gated NUnit test that drives the unmodified -`CultMeshHttpsContentTransportConnector`, with a handler pinning that -certificate, and checks every chunk with `CultMeshCdn`'s own validation. The TS -reader reads the same chunks from the same provider in the same test. +**Verify on every request, not once at boot.** The invariant says "serves +only bytes whose SHA-256 equals the requested chunk hash", and a boot check +cannot see a file replaced after boot. C# verifies per request (F21). A +"verified at boot" cache would be a cache pretending to be truth. StreamPixels' +boot check is its own deploy gate (missing blob means failed health), and it +stays useful beside per-request verification. The cost is one SHA-256 of at +most 4 MiB per request, and the immutable one-year caching keeps repeats away +from the origin. -**"Released" needs an operator ruling (question admitted).** The target says -"the npm packages StreamPixels vendors", but StreamPixels vendors a gitlink, not -npm (F17). None of the three packages has a publish job, and none is visible on -npm (F16). +**Reader: `fetchVerifiedContent(endpoint, manifest, options)` and +`fetchVerifiedChunk(endpoint, chunkHash, sizeBytes, options)` in +`cultmesh-browser`, over one internal per-chunk primitive.** The second is the +"fetch by hash" a general consumer expects for a single-chunk artifact. It +builds no manifest: it validates the hash and size, sends +`recordKey=chunkRecordKey(hash)` so its URL equals a C#-packed manifest's, and +runs the same primitive. The whole-body check is the chunk check, because for +a single chunk the two hashes are the same. Before any request, +`fetchVerifiedContent` runs `validateManifest`, `parseSha256Hex` on every +hash, the zero-size rule (F18) and the `maxBytes` cap. It then fills a private +buffer of `sizeBytes` through a sliding window of at most +`maxConcurrentChunkRequests` requests. It uses `credentials: "omit"` and an +internal `AbortController` chained to the caller's signal, so the first +failure cancels every request still in flight. It hashes each chunk in place +and returns the buffer only after the whole-body hash matches. + +**Why the reader ignores `Content-Length`.** The target says the reader +"refuses a response whose length differs from the advertised size before +reading it all". The header cannot carry that duty. Fetch decodes content +codings, and a proxy that gzips in spite of `no-transform` turns a correct +1,000-byte chunk into a 29-byte `Content-Length` (F20, R §6). In a browser the +reader cannot even see `Content-Encoding` cross-origin, because it is not +safelisted, so it cannot tell that case apart. What the invariant protects is +bounded reading and no acceptance of a wrong length. The streaming cap +delivers both from the bytes actually received: the reader stops at the first +byte past the chunk's size, and refuses a short body at its end. The target +sentence should be reworded (reported to Self as a contradiction). + +**C3 does not wait for C2.** C3's live test serves from a stub `node:http` +loopback server written in the test (about 30 lines). The stub sends what the +model page says, plus the misbehaviours: gzip, a short body, an endless body, +and wrong bytes. The real provider meets the real reader in C4. This keeps +the two cuts parallel, and it lets C3 test misbehaviours the real provider can +never produce. + +**"Released"** is ruled: `release-shape-gitlink` (Defaulted by Self). ## 4. Cut sequence -One line per cut. Only C1 is specified. C2 and C3 run in parallel after C1 -merges, each in its own worktree. +C1 (`cut-c1-shapes.r1`, report `h1` under a Soul pass), C2 +(`cut-c2-provider.r1`) and C3 (`cut-c3-reader.r1`) are specified in the mind. +C2 and C3 run in parallel once C1 merges, each in its own worktree from C1's +merge. Both are based on `b8b825e1`, and they touch disjoint files except +`cultmesh-portability.yml`, where their hunks lie 15 lines apart. What +remains: -1. **C1 `c1-shapes`**: `cultnet-ts/content` owns the manifest and chunk-ref types, the codec, `normalizeHash`/`parseSha256Hex`, both record-key grammars, `sha256Hex` and `packContent`. TS joins the content vectors in both directions, C# writes record-key vectors, and the C# content-vector suite enters CI. **Specified.** -2. **C2 `c2-provider`**: `cultmesh-ts/content` holds `createContentChunkHandler`, `createDirectoryChunkStore` and `toNodeListener`, with the model page's answers as behavioural tests, path-traversal negatives and verify-before-serve mutants. -3. **C3 `c3-reader`**: `cultmesh-browser` gets `fetchVerifiedContent` and `CultMeshContentError`. It is tested against a fetch double and a live `node:http` loopback provider (C2's, once merged), and covers abort, concurrency bound, empty artifact and URL parity with F5. -4. **C4 `c4-cross-process`**: the TS interop test serves over `node:https`; an env-gated NUnit test drives the unmodified C# connector; the TS reader reads the same chunks; CI lane per F13. -5. **C5 `c5-release`**: version bumps for `cultcache-ts`, `cultnet-ts`, `cultmesh-ts` and `cultmesh-browser`, with CHANGELOGs, then a Soul pass, then tags (no more than three per push). The shape follows the release ruling. +4. **C4 `c4-cross-process`**, waiting on C2 and C3. Line-level plan for the + pass that specifies it: + - New fixture `packages/cultmesh-ts/test/fixtures/content-https-test.p12` + plus `content-https-test-cert.pem`, a self-signed P-256 certificate with + subjectAltName `IP:127.0.0.1, DNS:localhost` (F25 rules out reusing the + QUIC pair). The recipe goes in the fixtures README beside the QUIC one. + - New `packages/cultmesh-ts/test/interop/content-https.test.ts`, and a + `test:interop` script in `cultmesh-ts/package.json` that runs + `dist-test/test/interop/*.test.js`, as `cultnet-ts`'s does. The test is + skipped when `dotnet` is absent. It packs a 3.5-chunk artifact (chunk + size 64 KiB), an empty artifact and a single-chunk artifact with C1's + `packContent`, and writes the chunks into a temporary directory store. + It serves them through `createContentChunkHandler` and `toNodeListener` + on `node:https` with the new fixture on `127.0.0.1:0`. It writes the + manifests' bytes (`encodeManifest`) to a temporary file. + - C# leg: it spawns `dotnet test tests/GameCult.Mesh.Tests --filter + HttpsConnectorFetchesFromExternalTypeScriptProvider` with + `CULTMESH_HTTPS_CONTENT_ENDPOINT`, `CULTMESH_HTTPS_CONTENT_MANIFESTS` + (the file path) and `CULTMESH_HTTPS_CONTENT_CERT_SHA256`. Reuse or lift + the `runDotnetTestFilter` shape from `cultnet-interop.test.ts:2826` + (F27), and assert on "Test Run Successful." and "Passed: 1", not on a + skip. + - New C# test in `tests/GameCult.Mesh.Tests/CultMeshHttpsContentInteropTests.cs`: + `Assert.Ignore` without the variable, as F13 does. It builds the + **unmodified** `CultMeshHttpsContentTransportConnector` with a + `HttpClientHandler` whose `ServerCertificateCustomValidationCallback` + accepts exactly the pinned SHA-256, then `ConnectAsync` with a + `CultMeshTransportCandidate { Endpoint }` and a session target, then + `CopyChunkToAsync` for every chunk ref of every manifest (deserialized + by the public MessagePack path the vector tests use). It checks each + chunk with `SHA256.HashData` and the whole body against `ContentHash` + (F26: the validators are internal). It also asserts that an unknown + hash raises `HttpRequestException` with 404. The first step of the pass + probes whether `CultMeshContentTransferService` can drive the connector + without Odin; if it can, it verifies instead of the test's own hashing. + - TS reader leg: a child `node` process with `NODE_EXTRA_CA_CERTS` set to + the fixture PEM runs `fetchVerifiedContent` from built + `cultmesh-browser/dist` against the same `https://127.0.0.1:` + endpoint, through the real global `fetch` (no injection), and prints + the SHA-256 of each body. The parent compares them with the manifests'. + A child process, because the trust anchor must be set before the + process starts. `cultmesh-ts` gains `cultmesh-browser` as a workspace + devDependency, which is test-only. + - Corrupt leg: one stored file is flipped after serving starts. The C# + connector then fails on the 500 (`EnsureSuccessStatusCode`), the reader + rejects with `http-status` 500, and `onServerFault` fired with + `hash-mismatch`. + - CI: a step in `cultmesh-portability.yml` after the content-vector step + (`npm run test:interop --workspace packages/cultmesh-ts`). That job + already has .NET 10 and Node on both OSes. + - Negative: the git diff of `src/GameCult.Mesh/` over the cut prints + nothing (the connector is unmodified). + - Size: about 350 lines and one fixture pair; one Hands pass. +5. **C5 `c5-release`**, waiting on C4 and the Soul passes of C2 to C4. Ruling + `release-shape-gitlink`: merged to CultLib `main` with version bumps, + CHANGELOGs and git tags, and no npm publish. Line-level plan: + - Bumps (minor; every change is additive): `cultnet-ts` 0.2.0 to 0.3.0 + (`./content`), `cultmesh-ts` 0.1.1 to 0.2.0 (`./content`), and `cultmesh-browser` 0.3.0 to 0.4.0 (the reader). + `@gamecult/cultcache-ts` gains only the `checkPayloadDepth` re-export: + a patch bump, 0.14.0 to 0.14.1 (its version at `b8b825e1`). + - Dependency ranges: `cultmesh-ts` and `cultmesh-browser` move + `cultnet-ts` to `^0.3.0`. `cultnet-ts`'s `^0.14.0` on + `@gamecult/cultcache-ts` already admits 0.14.1 and stays. `npm install + --no-package-lock` and the four workspace tests run after the bumps. + - A `CHANGELOG.md` per bumped package. None exists in any package at + `b8b825e1`, so C5 creates four, each naming the live surface and not the + history (Keep a Changelog headings). + - A Soul pass over the release diff, then four tags: `cultcache-ts-v…`, + `cultnet-ts-v0.3.0`, `cultmesh-ts-v0.2.0` and `cultmesh-browser-v0.4.0`. + These go in **two pushes** (at most three tags per push, SKILL.md): + `cultcache-ts` first, because its tag triggers `publish-packages.yml` + (F16), and the three plane tags second. Whether the `cultcache-ts` tag's + npm publish job still runs, and to where, is checked before that push; + follow-up `npm-publish-ts-packages` owns the rest. + - The consumer's pickup (StreamPixels bumping `vendor/CultLib` to the + tagged commit) is StreamPixels' B-cuts, not this campaign. + - Size: about 60 lines of version and CHANGELOG text; a short Hands pass. diff --git a/docs/cultmesh-ts-content-target.md b/docs/cultmesh-ts-content-target.md index 246950af..e1ee042a 100644 --- a/docs/cultmesh-ts-content-target.md +++ b/docs/cultmesh-ts-content-target.md @@ -61,8 +61,10 @@ URLs; clients resolve content through CultMesh. fetches content from the TypeScript provider unmodified, in a cross-process test. The TypeScript reader fetches the same chunks from the same provider. - `verify-before-use`: the reader releases no byte that failed its chunk hash - or the whole-body hash, and refuses a response whose length differs from the - advertised size before reading it all. + or the whole-body hash. It refuses a body longer than the advertised size at + its first excess byte, and a shorter one at its end. It does not trust + response length headers, because Fetch decodes content codings and a proxy + can rewrite them (research 2026-10-19, section 6). - `verify-before-serve`: the provider serves only bytes whose SHA-256 equals the requested chunk hash. - `chunk-only-route`: the provider answers only chunk requests. A path, a From 64862d4a159e4c400891cdadc22773a98c3c71b0 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Mon, 19 Oct 2026 19:26:18 +0200 Subject: [PATCH 3/3] Docs: cultmesh-ts-content map pass 3 (C4 facts F28-F33, cross-process probes, C5 dependency note) Co-Authored-By: Claude Opus 5.5 --- docs/cultmesh-ts-content-map.md | 109 +++++++++++++++----------------- 1 file changed, 52 insertions(+), 57 deletions(-) diff --git a/docs/cultmesh-ts-content-map.md b/docs/cultmesh-ts-content-map.md index 592d0b19..9aae6eba 100644 --- a/docs/cultmesh-ts-content-map.md +++ b/docs/cultmesh-ts-content-map.md @@ -12,7 +12,9 @@ typed mind, not here. ## 1. Body facts F1-F18 were probed by running code or read from source at `8990bd46`. F19-F27 -were probed or read at `b8b825e1` (C1's head) and `2de07065`. Scratch probes +were probed or read at `b8b825e1` (C1's head) and `2de07065`. F28-F33 were +probed on 2026-10-19 over the C2+C3 merge tree (`7adbf3cb` merged with C3's +head, extracted by `git archive`), on Windows and in a Linux container. Scratch probes ran under the session scratchpad (`imagination/`), not in the repo. | # | Fact | Evidence | @@ -44,6 +46,12 @@ ran under the session scratchpad (`imagination/`), not in the repo. | F25 | The existing TLS fixture `packages/cultmesh-ts/test/fixtures/quic-test.p12` is a self-signed P-256 certificate with subject `CN=cultmesh-quic-native-tests` and no subjectAltName (fixtures README). Node's TLS hostname check therefore refuses it for `127.0.0.1` even when it is trusted through `NODE_EXTRA_CA_CERTS`. A C# handler that pins by certificate hash does not care about the name. | source: fixtures README | | F26 | `CultMeshCdn.ValidateChunkPayload`, `HashBytes`, `ValidateManifest` and `NormalizeHash` are `internal`, and `GameCult.Mesh` declares no `InternalsVisibleTo`. A test in `tests/GameCult.Mesh.Tests` can therefore verify chunk bytes only with `SHA256.HashData`, or through a public consumer of the connector. The public surface is `CreateRecordKey`, `PackArtifact`, `PublishAsync` and `ReadArtifact(Async)`. | grep of `CultMeshCdn.cs`, `src/GameCult.Mesh` | | F27 | No CultLib workflow runs `cultmesh-ts` interop tests. `cultnet-interop.yml` (windows-latest, main only) runs `cultnet-ts`'s `test/interop`, and its QUIC native case spawns `dotnet test --filter NativeConnectorReceivesFromExternalProvider` through `runDotnetTestFilter` (`cultnet-interop.test.ts:550,2826`). `cultmesh-ts` has a `test/interop/` folder (a QUIC peer script) but no `test:interop` script. | grep of `.github/workflows`, source | +| F28 | **The real provider meets the real reader across processes, on Windows and Linux.** C2's handler, store and `toNodeListener` served on `node:https` with a self-signed P-256 certificate (subjectAltName `IP:127.0.0.1, DNS:localhost`, `CA:FALSE`, `serverAuth`, 100-year validity). They served a 4-chunk artifact (3.5 x 64 KiB) and an empty one. C3's `fetchVerifiedContent` ran in a child `node` with `NODE_EXTRA_CA_CERTS` set to that certificate, through the real global `fetch`, and returned both bodies with the right hashes. Without the variable it rejected `network` (cause `DEPTH_ZERO_SELF_SIGNED_CERT`). Behind a test wrapper that flips byte 0 of every non-empty 200 body, it rejected `chunk-hash-mismatch`. Platforms: Windows (Node 24.15) and a `node:24-trixie` container (Node 24.21). | probe: `imagination/probe/{serve,read,drive}.mjs` over the C2+C3 merge tree `8d1482c8` | +| F29 | **The C# consumer that verifies is `CultMeshContentTransferService`, and it runs against an HTTPS route without Odin.** The probed path: `CultMeshDiscoveryService` over a test `RouteSource()`, then `CultMeshSessionManager` with `LocalDevelopment` trust and the unmodified `CultMeshHttpsContentTransportConnector`, then `CultMeshSessionContentProvider`, then `FetchAsync(manifest)`. This is the shape of `CultMeshContentSessionTests.TcpContentSessionStreamsLargeBodyAndReusesCommittedBody` with the TCP connector swapped for the HTTPS one. It fetched both artifacts from the TS provider, decoding the **TS-encoded** manifest bytes with `CultDocumentMessagePackSerialization.DeserializeUntyped`. Trust was a `SocketsHttpHandler` whose `SslOptions.CertificateChainPolicy` is `CustomRootTrust` with only the fixture certificate in `CustomTrustStore` (`RevocationMode.NoCheck`). That is real chain and name validation, with no callback and no system roots. The default handler failed with `UntrustedRoot`. Against the tampering wrapper, `FetchAsync` threw `AggregateException` over `InvalidDataException` ("CDN artifact chunk hash does not match its manifest."), leaving only `..partial`, never `.body`. The connector itself does not verify (F6). The C# verifier is the transfer service's `BoundedHashingWriteStream` per chunk and its whole-file hash. The results on the Linux container (self-contained linux-x64 publish) were identical. | probe: `imagination/csprobe/Program.cs` and `probe/drive-cs.mjs`; source `CultMeshContentTransfer.cs:115-330`, `CultMeshContentTransports.cs:138-160` | +| F30 | The two consumers request the same URLs. C# sent five requests: four chunks, plus the empty chunk `e3b0…`, which the provider answers intrinsically. Each was `/api/content?chunkHash=&recordKey=mesh%3Acdn%3Achunk%3A`. The reader sent the same four chunk URLs, and nothing for the empty chunk, which it verifies locally. | same probes, the provider's request log | +| F31 | **Build order decides where the cross-process test lives.** `scripts/run-typescript-workspaces.mjs` builds `cultcache-ts, cultnet-ts, cultmesh-ts, cultmesh-browser`, and each package's `test` compiles `test/**/*.ts`, `test/interop` included. A `cultmesh-ts` test that imports `cultmesh-browser` therefore fails to compile on a clean checkout, while a `cultmesh-browser` test that imports `cultmesh-ts/content` follows the order. Named ESM imports of the CommonJS `cultmesh-ts/content` work from an ESM file: all four provider names import as functions. At `7adbf3cb`, `cultmesh-portability.yml` runs the browser step (lines 119-120) **before** the `cultmesh-ts` step (129-130), which is the opposite of the script's order. The wiring check requires `test/interop/*.test.ts` to be run by a `test:interop` script. | probe (node, ESM import); source | +| F32 | Node's `execFile("dotnet")` resolves `dotnet.exe` on Windows without an extension. The C3 merge into `7adbf3cb` is clean at C3 head `67ff3f64` (`git merge-tree` gives `666162a3`). It touches the workflow only at comment lines 105-107, and the reader's exports keep their names. The probe's body pattern `(i*31+7)&255` repeats every 256 bytes, so all of its 64 KiB chunks shared one hash. A parity fixture needs non-periodic bytes, or chunk order is invisible. | probe (node); `git merge-tree` | +| F33 | `cultmesh-portability.yml` triggers on `push` only for `main` and `codex/**`. A `hands/**` branch runs it through `pull_request` or `workflow_dispatch`, and both triggers exist. The job `reactive-and-resource-lifetime` has .NET 10 and Node 24 on `windows-latest` and `ubuntu-latest`, and it already builds `tests/GameCult.Mesh.Tests` in earlier steps. | source at `7adbf3cb` | ## 2. Model page @@ -98,7 +106,7 @@ and corrected this page's first draft (R §8). In short: | **Provider memory** | At most `maxChunkBytes` buffered per in-flight request. | Per request. | Provider. Buffering is forced, because the status line cannot be sent before the hash is known (the bazel-remote scar above). `maxChunkBytes` defaults to `DEFAULT_CHUNK_SIZE_BYTES` (4 MiB) and must lie in 1..2147483647. The host bounds concurrency (server `maxConnections` or a reverse proxy), and the README says so. | | **Reader result / error** | A `Uint8Array` of the whole body, or one `CultMeshContentError` with `code` ∈ {`manifest-invalid`, `too-large`, `insecure-endpoint`, `webcrypto-unavailable`, `network`, `http-status`, `length-mismatch`, `chunk-hash-mismatch`, `content-hash-mismatch`}, plus `chunkHash` and `status` where they apply. A caller abort rejects with the signal's own `reason`, unwrapped. Bad options reject with `RangeError`. | Transient, held in memory only. No byte of a failed chunk or body is returned, in whole or in part (`verify-before-use`). | `fetchVerifiedContent` (and `fetchVerifiedChunk`) in `cultmesh-browser`. Concurrency defaults to 4 and is bounded 1..32 (the C# `CultMeshContentTransferOptions` bounds). `maxBytes` defaults to 256 MiB and is checked before any request. The scheme must be `https:`, or `http:` where `isLoopbackEndpoint` (F22) holds. Only status 200 is accepted. **`Content-Length` is not consulted** (F20). The body is read through a stream into the chunk's slot, a body that runs past the chunk's size is cancelled at the first excess byte, and one that ends short is refused. | | **Endpoint** | A URL the caller resolved through CultMesh. | Per call; never persisted in content. | The caller. **Forbidden:** any manifest, chunk-ref or record field carrying it (`no-urls-in-content`). Error messages never echo it, because an endpoint query may carry a tenant or a token. | -| **Vector files** | `content-vectors.cs-written.json` (C#), `.rs-written.json` (Rust), `content-vectors.ts-written.json` (TS, manifests only, own inputs) and `cdn-record-key-vectors.json` (C#, record-key traps), all landed by C1. | Each is rewritten only by its writer under `CULTNET_WRITE_VECTORS=1`. CI never sets the variable: the committed files are the contract. | Each writer owns its file. C2 and C3 add no vector file. HTTP behaviour is pinned by tests and, in C4, by the C# connector itself. | +| **Vector files** | `content-vectors.cs-written.json` (C#), `.rs-written.json` (Rust), `content-vectors.ts-written.json` (TS, manifests only, own inputs) and `cdn-record-key-vectors.json` (C#, record-key traps), all landed by C1. | Each is rewritten only by its writer under `CULTNET_WRITE_VECTORS=1`. CI never sets the variable: the committed files are the contract. | Each writer owns its file. C2 and C3 add no vector file. HTTP behaviour is pinned by tests and, in C4, by the C# transfer service driving the unmodified connector (F29). | Two cells rest on reasoning rather than a probe: that the host, not the provider, bounds concurrency, and the cost of hashing on HEAD. Both are @@ -201,6 +209,29 @@ and wrong bytes. The real provider meets the real reader in C4. This keeps the two cuts parallel, and it lets C3 test misbehaviours the real provider can never produce. +**C4: the proof lives in `cultmesh-browser/test/interop`, with the C# half +in `GameCult.Mesh.Tests`.** The orchestrator is Node, following the QUIC +precedent (F13): it starts the provider, runs both consumers and judges what +the provider saw. It lives in `cultmesh-browser` and not `cultmesh-ts`, +because the workspace build order compiles `cultmesh-ts` first (F31). The +browser package can import `cultmesh-ts/content` with types. The reverse +would need either an untyped script or a build-order exception. The provider +runs in its own process rather than the orchestrator's, so each of the three +parties is a separate OS process, and the orchestrator never shares an event +loop with the server it judges. The C# half drives the connector through +`CultMeshContentTransferService` (F29), because the connector alone does not +verify (F6). A parity test that stopped at the connector would show that C# +receives bytes, not that C# refuses bad ones. TLS trust is a committed +fixture certificate, as the QUIC fixture is (F25). The C# side anchors it +through `CustomRootTrust`, which keeps chain and name validation and adds no +code path that can accept a different certificate. A hash-pinning +`ServerCertificateCustomValidationCallback` would skip name validation and +add a callback that has to be read to be trusted. Corruption comes in two +kinds. On disk, the real provider refuses it with 500 (C2's +verify-before-serve). In flight, a test wrapper flips bytes after the +provider has verified them. The in-flight leg is what proves that both +consumers verify. + **"Released"** is ruled: `release-shape-gitlink` (Defaulted by Self). ## 4. Cut sequence @@ -212,59 +243,19 @@ merge. Both are based on `b8b825e1`, and they touch disjoint files except `cultmesh-portability.yml`, where their hunks lie 15 lines apart. What remains: -4. **C4 `c4-cross-process`**, waiting on C2 and C3. Line-level plan for the - pass that specifies it: - - New fixture `packages/cultmesh-ts/test/fixtures/content-https-test.p12` - plus `content-https-test-cert.pem`, a self-signed P-256 certificate with - subjectAltName `IP:127.0.0.1, DNS:localhost` (F25 rules out reusing the - QUIC pair). The recipe goes in the fixtures README beside the QUIC one. - - New `packages/cultmesh-ts/test/interop/content-https.test.ts`, and a - `test:interop` script in `cultmesh-ts/package.json` that runs - `dist-test/test/interop/*.test.js`, as `cultnet-ts`'s does. The test is - skipped when `dotnet` is absent. It packs a 3.5-chunk artifact (chunk - size 64 KiB), an empty artifact and a single-chunk artifact with C1's - `packContent`, and writes the chunks into a temporary directory store. - It serves them through `createContentChunkHandler` and `toNodeListener` - on `node:https` with the new fixture on `127.0.0.1:0`. It writes the - manifests' bytes (`encodeManifest`) to a temporary file. - - C# leg: it spawns `dotnet test tests/GameCult.Mesh.Tests --filter - HttpsConnectorFetchesFromExternalTypeScriptProvider` with - `CULTMESH_HTTPS_CONTENT_ENDPOINT`, `CULTMESH_HTTPS_CONTENT_MANIFESTS` - (the file path) and `CULTMESH_HTTPS_CONTENT_CERT_SHA256`. Reuse or lift - the `runDotnetTestFilter` shape from `cultnet-interop.test.ts:2826` - (F27), and assert on "Test Run Successful." and "Passed: 1", not on a - skip. - - New C# test in `tests/GameCult.Mesh.Tests/CultMeshHttpsContentInteropTests.cs`: - `Assert.Ignore` without the variable, as F13 does. It builds the - **unmodified** `CultMeshHttpsContentTransportConnector` with a - `HttpClientHandler` whose `ServerCertificateCustomValidationCallback` - accepts exactly the pinned SHA-256, then `ConnectAsync` with a - `CultMeshTransportCandidate { Endpoint }` and a session target, then - `CopyChunkToAsync` for every chunk ref of every manifest (deserialized - by the public MessagePack path the vector tests use). It checks each - chunk with `SHA256.HashData` and the whole body against `ContentHash` - (F26: the validators are internal). It also asserts that an unknown - hash raises `HttpRequestException` with 404. The first step of the pass - probes whether `CultMeshContentTransferService` can drive the connector - without Odin; if it can, it verifies instead of the test's own hashing. - - TS reader leg: a child `node` process with `NODE_EXTRA_CA_CERTS` set to - the fixture PEM runs `fetchVerifiedContent` from built - `cultmesh-browser/dist` against the same `https://127.0.0.1:` - endpoint, through the real global `fetch` (no injection), and prints - the SHA-256 of each body. The parent compares them with the manifests'. - A child process, because the trust anchor must be set before the - process starts. `cultmesh-ts` gains `cultmesh-browser` as a workspace - devDependency, which is test-only. - - Corrupt leg: one stored file is flipped after serving starts. The C# - connector then fails on the 500 (`EnsureSuccessStatusCode`), the reader - rejects with `http-status` 500, and `onServerFault` fired with - `hash-mismatch`. - - CI: a step in `cultmesh-portability.yml` after the content-vector step - (`npm run test:interop --workspace packages/cultmesh-ts`). That job - already has .NET 10 and Node on both OSes. - - Negative: the git diff of `src/GameCult.Mesh/` over the cut prints - nothing (the connector is unmodified). - - Size: about 350 lines and one fixture pair; one Hands pass. +4. **C4 `c4-parity`** (`cut-c4-parity.r1`, branch + `hands/cultmesh-ts-content-c4-parity` from `7adbf3cb` plus the C3 merge), + waiting on C2 and C3. The spec carries the plan. In one paragraph: a + node:test orchestrator in `cultmesh-browser/test/interop` packs three + artifacts (multi-chunk, single and empty) and spawns the C2 provider as its + own `node:https` process with three mounts: honest, tampered and corrupt + store. It runs the C3 reader in a child with `NODE_EXTRA_CA_CERTS`, once + more without it as a control, and then `dotnet test` over a new + `CultMeshHttpsContentInteropTests`. That suite drives the unmodified + connector through `CultMeshContentTransferService` and trusts the committed + fixture through `CustomRootTrust`. The orchestrator then judges URL parity + and the provider's fault log. CI gains the step on both OSes, and the + browser and `cultmesh-ts` steps swap to follow the workspace order (F31). 5. **C5 `c5-release`**, waiting on C4 and the Soul passes of C2 to C4. Ruling `release-shape-gitlink`: merged to CultLib `main` with version bumps, CHANGELOGs and git tags, and no npm publish. Line-level plan: @@ -274,8 +265,12 @@ remains: a patch bump, 0.14.0 to 0.14.1 (its version at `b8b825e1`). - Dependency ranges: `cultmesh-ts` and `cultmesh-browser` move `cultnet-ts` to `^0.3.0`. `cultnet-ts`'s `^0.14.0` on - `@gamecult/cultcache-ts` already admits 0.14.1 and stays. `npm install - --no-package-lock` and the four workspace tests run after the bumps. + `@gamecult/cultcache-ts` already admits 0.14.1 and stays. C4 gives + `cultmesh-browser` a devDependency `cultmesh-ts: ^0.1.1`, which does not + admit 0.2.0, so it moves to `^0.2.0` in the same commit. Otherwise npm + looks for 0.2.0 on the registry and the install fails. `npm install + --no-package-lock`, the four workspace tests and the browser's + `test:interop` run after the bumps. - A `CHANGELOG.md` per bumped package. None exists in any package at `b8b825e1`, so C5 creates four, each naming the live surface and not the history (Keep a Changelog headings).