GameCult/Heimdall
Watch
9
0
Fork
You've already forked Heimdall
 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Read Idunn stores in the layout Idunn actually writes

Browse source 
cultcache-ts and cultcache-rs write different single-file store layouts,
and the TypeScript client cannot read the Rust one at all -- its schema
rejects it with "expected object, received array". Verified against the
real enrolled identity on yggdrasil, not inferred.

  cultcache-rs  [ [key, type, payload, stored_at, schema_id], ... ]
  cultcache-ts  [ formatVersion, catalog, records ] with named envelopes

Both readers I wrote for Idunn-written files assumed the named-object
shape. They were wrong, and their tests passed only because the fixtures
were built to match the assumption rather than the writer -- the same
failure the census keeps recording, where the check agrees with the guess.

src/idunn-store.ts now owns that decode, consumed by the write-lease reader
and the presence-identity reader, and says plainly that it exists because
the two CultLib forks disagree. If they are reconciled, it should be
deleted in favour of the client.

The write-lease fixture is rebuilt as a positional envelope, and the
descriptor test now wraps a genuine machine-bound payload the way
cultcache-rs writes it instead of enrolling through the TypeScript client
and reading it back.

55 pass, 1 skipped, 12 files; tsc clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Meta Vens 2026-09-06 16:07:47 +02:00
commit 8c1aa9f9e8
5 changed files with 119 additions and 68 deletions
Download patch file Download diff file Expand all files Collapse all files

59
src/idunn-store.ts Normal file
View file

@ -0,0 +1,59 @@
/**
* Reading the `.cc` stores Idunn writes.
*
* Idunn writes with `cultcache-rs`, which lays a single-file store out as a
* bare MessagePack array of positional envelopes:
*
* [ [key, type, payload, stored_at, schema_id], ... ]
*
* `cultcache-ts` writes a different shape — `[formatVersion, catalog, records]`
* with envelopes as named objects — and **cannot read the Rust layout at all**;
* its schema rejects it with "expected object, received array". So Heimdall
* cannot use its own CultCache client to read an Idunn-written store, and this
* module exists to read exactly that one layout.
*
* That divergence is a defect in CultLib, not a property of the format, and
* this module is a consumer-side reader for as long as it stands. If the two
* forks are reconciled, delete this and use the client.
*/
import { decode } from "@msgpack/msgpack";
const ENVELOPE_FIELD_COUNT = 5;
const ENVELOPE_PAYLOAD = 2;
/**
* Decode the payloads of every envelope in an Idunn-written store, in order.
*
* Accepts bytes rather than a path so callers can read from a file, or from a
* file descriptor systemd passed them.
*/
export function readIdunnStorePayloads(bytes: Uint8Array, label: string): Uint8Array[] {
const store = decode(bytes);
if (!Array.isArray(store)) {
throw new Error(`${label} is not a cultcache-rs single-file store`);
}
return store.map((envelope, index) => {
if (!Array.isArray(envelope) || envelope.length !== ENVELOPE_FIELD_COUNT) {
throw new Error(
`${label} envelope ${index} is not the ${ENVELOPE_FIELD_COUNT}-field positional contract`
);
}
const payload = envelope[ENVELOPE_PAYLOAD];
if (!(payload instanceof Uint8Array)) {
throw new Error(`${label} envelope ${index} has no binary payload`);
}
return payload;
});
}
/** The single record an Idunn-written store is expected to hold. */
export function readIdunnStoreRecord(bytes: Uint8Array, label: string): Uint8Array {
const payloads = readIdunnStorePayloads(bytes, label);
if (payloads.length !== 1) {
throw new Error(`${label} must contain exactly one record, found ${payloads.length}`);
}
return payloads[0]!;
}

26
src/process-write-lease.ts
View file

@ -20,6 +20,7 @@ import { readFile } from "node:fs/promises";
import path from "node:path";
import { decode } from "@msgpack/msgpack";
import { readIdunnStoreRecord } from "./idunn-store.js";
const WRITE_LEASE_SCHEMA = "idunn.process_write_lease.v1";
const EXPECTED_INCARNATION_SCHEMA = "idunn.expected_incarnation.v2";
@ -78,27 +79,14 @@ function stringField(fields: unknown[], index: number, label: string): string {
}
/**
* Envelopes are `{ key, type, payload, storedAt }`; only the payload carries
* the positional record. Read the file directly rather than through the store
* helper: this must work on a store owned by another UID, and it must never
* create the file it is inspecting.
* Read the record out of an Idunn-written store. Reads the file directly
* rather than through the CultCache client: this must work on a store owned by
* another UID, it must never create the file it is inspecting, and
* `cultcache-ts` cannot parse the layout `cultcache-rs` writes anyway — see
* `idunn-store.ts`.
*/
async function readEnvelopePayload(file: string, label: string): Promise<Uint8Array> {
const bytes = await readFile(file);
const entries = decode(bytes);
const envelopes = Array.isArray(entries) ? entries : [entries];
const last = envelopes[envelopes.length - 1];
if (!last || typeof last !== "object") {
throw new Error(`${label} holds no CultCache envelope`);
}
const payload = (last as Record<string, unknown>).payload;
if (!(payload instanceof Uint8Array)) {
throw new Error(`${label} envelope has no binary payload`);
}
return payload;
return readIdunnStoreRecord(await readFile(file), label);
}
/**

17
src/provider-health-identity.ts
View file

@ -10,6 +10,7 @@ import {
import { readFileSync } from "node:fs";
import { chmod, readFile } from "node:fs/promises";
import { SingleFileMessagePackBackingStore, type CultCacheEnvelope } from "cultcache-ts";
import { readIdunnStoreRecord } from "./idunn-store.js";
const PRIVATE_SCHEMA = "gamecult.provider_health_identity.private.v1";
const PRIVATE_KEY = "gamecult-provider-health-identity";
@ -129,18 +130,10 @@ export async function openProviderHealthIdentity(
return openOrEnrollProviderHealthIdentity(path);
}
const store = decode(readFileSync(descriptor));
const envelopes = Array.isArray(store) ? store : [store];
if (envelopes.length !== 1) {
throw new Error(
"Idunn runtime presence identity must contain exactly one record."
);
}
const payload = (envelopes[0] as Record<string, unknown>)?.payload;
if (!(payload instanceof Uint8Array)) {
throw new Error("Idunn runtime presence identity has no binary payload.");
}
const payload = readIdunnStoreRecord(
readFileSync(descriptor),
"Idunn runtime presence identity"
);
return decodePrivateIdentity(payload, await readMachineId());
}

80
tests/presence-identity-descriptor.test.ts
View file

@ -1,8 +1,10 @@
import { mkdtemp, readFile } from "node:fs/promises";
import { mkdtemp, readFile, writeFile } from "node:fs/promises";
import { openSync, closeSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { encode } from "@msgpack/msgpack";
import { SingleFileMessagePackBackingStore } from "cultcache-ts";
import { describe, expect, it } from "vitest";
import {
@ -17,22 +19,43 @@ const machineIdAvailable = await readFile("/etc/machine-id", "utf8").then(
);
/**
* The identity is machine-id bound, so these can only run where a machine-id
* exists. On Windows the enrolment path itself is unavailable, and the
* descriptor selection below is the part worth proving anyway.
* The identity is machine-id bound, so enrolment only works where a machine-id
* exists. These run on Linux and are skipped elsewhere.
*/
describe.skipIf(!machineIdAvailable)("presence identity from a passed descriptor", () => {
it("reads the identity Idunn passed rather than enrolling its own", async () => {
it("reads an Idunn-written store, not a cultcache-ts one", async () => {
const dir = await mkdtemp(path.join(os.tmpdir(), "heimdall-presence-"));
const store = path.join(dir, "runtime-presence-identity.cc");
const enrolled = await openOrEnrollProviderHealthIdentity(store);
const fd = openSync(store, "r");
// Enrol with the TypeScript client to get a genuine machine-bound payload,
// then re-wrap that payload the way cultcache-rs writes a store: a bare
// array of positional [key, type, payload, stored_at, schema_id] envelopes.
// Idunn writes with the Rust client, and cultcache-ts cannot read that
// layout, so wrapping it here is what makes this test resemble reality.
const tsStore = path.join(dir, "ts-enrolled.cc");
const enrolled = await openOrEnrollProviderHealthIdentity(tsStore);
const records = await new SingleFileMessagePackBackingStore(tsStore).pullAll();
expect(records).toHaveLength(1);
const idunnStore = path.join(dir, "runtime-presence-identity.cc");
await writeFile(
idunnStore,
encode([
[
"gamecult-provider-health-identity",
"gamecult.provider_health_identity.private.v1",
records[0]!.payload,
"2026-09-06T00:00:00.000Z",
"gamecult.provider_health_identity.private.v1",
],
])
);
const fd = openSync(idunnStore, "r");
try {
// The lookup maps a name index onto 3 + index, so the name list has to be
// padded until the presence entry lands on whatever descriptor open()
// actually returned. Assuming it is 3 is how this test would pass on one
// machine and read the wrong descriptor on another.
// The lookup maps a name index onto 3 + index, so the name list is padded
// until the presence entry lands on whatever descriptor open() actually
// returned. Assuming it is 3 is how this passes on one machine and reads
// the wrong descriptor on another.
const names = [
...Array.from({ length: fd - 3 }, (_, i) => `pad-${i}`),
"gamecult-runtime-presence-identity",
@ -44,9 +67,9 @@ describe.skipIf(!machineIdAvailable)("presence identity from a passed descriptor
process.pid
);
// Same key as the enrolled one means it came from the descriptor. A
// fresh self-enrolment would produce a different key, which is exactly
// the failure that would make Idunn refuse this daemon's health.
// Same key means it came from the descriptor. A fresh self-enrolment
// would produce a different key -- which is exactly the failure that
// makes Idunn refuse this daemon's health and stall promotion.
expect(providerHealthPublicIdentity(opened).publicKeyHex).toBe(
providerHealthPublicIdentity(enrolled).publicKeyHex
);
@ -59,41 +82,26 @@ describe.skipIf(!machineIdAvailable)("presence identity from a passed descriptor
describe("presence identity descriptor selection", () => {
const dir = os.tmpdir();
it("falls back to the path when systemd passed nothing", async () => {
// No LISTEN_FDNAMES means no Idunn: the ordinary self-enrolling path
// applies and must not be treated as an error.
const target = path.join(dir, `heimdall-no-fd-${process.pid}.cc`);
await expect(
openProviderHealthIdentity(target, {}, process.pid)
).resolves.toBeDefined().catch(() => {
// On a host without a machine-id enrolment cannot succeed; the point is
// that it took the path branch rather than trying to read a descriptor.
});
});
it("ignores descriptors addressed to another process", async () => {
// LISTEN_PID naming a different process means these descriptors are not
// ours. Consuming them would read a sibling's identity.
const target = path.join(dir, `heimdall-wrong-pid-${process.pid}.cc`);
// ours; consuming them would read a sibling's identity. The call must take
// the path branch, so it must not fail with a descriptor-read error.
await openProviderHealthIdentity(
target,
path.join(dir, `heimdall-wrong-pid-${process.pid}.cc`),
{ LISTEN_FDNAMES: "gamecult-runtime-presence-identity", LISTEN_PID: String(process.pid + 1) },
process.pid
).catch((error) => {
// Either it enrolled via the path, or it failed for a machine-id reason.
// What must not happen is a descriptor read.
expect(String(error)).not.toContain("presence identity");
expect(String(error)).not.toContain("Idunn runtime presence identity");
});
});
it("ignores a descriptor set that does not name the presence identity", async () => {
const target = path.join(dir, `heimdall-other-fd-${process.pid}.cc`);
await openProviderHealthIdentity(
target,
path.join(dir, `heimdall-other-fd-${process.pid}.cc`),
{ LISTEN_FDNAMES: "some-other-credential", LISTEN_PID: String(process.pid) },
process.pid
).catch((error) => {
expect(String(error)).not.toContain("presence identity");
expect(String(error)).not.toContain("Idunn runtime presence identity");
});
});
});

5
tests/process-write-lease.test.ts
View file

@ -14,7 +14,10 @@ import { resolveWriteLease } from "../src/process-write-lease.js";
* than a convenient object.
*/
function envelope(key: string, type: string, payload: Uint8Array): Uint8Array {
return encode([{ key, type, payload, storedAt: "2026-09-06T00:00:00.000Z" }]);
// cultcache-rs writes positional envelopes: [key, type, payload, stored_at,
// schema_id]. Building the fixture the way Idunn actually writes it is the
// point -- an object-shaped fixture agrees with a wrong reader.
return encode([[key, type, payload, "2026-09-06T00:00:00.000Z", type]]);
}
function expectedIncarnation(target: string, incarnationId: string): Uint8Array {