diff --git a/README.md b/README.md index fb6d9ea..f85e797 100644 --- a/README.md +++ b/README.md @@ -39,12 +39,17 @@ Landed right now: Heimdall JWTs without phoning home on every guarded request - a read-only Odin/Eve provider advertisement export that names redacted CultCache witness document shapes and paths without touching live secrets +- a daemon-owned CultCache boundary witness at `GC_ACCESS_CULTCACHE_PATH` + containing the Heimdall provider advertisement, command boundary, transport + profile, and Idunn daemon-health summary +- optional Idunn daemon-health publication over `cultnet.transport.rudp.v0` + when `GC_ACCESS_IDUNN_RUDP_HEALTH` is configured Still not landed: - end-to-end app integrations in consumer repos - refresh/revocation flows and admin surfaces -- runtime CultCache `.cc` witness writing or CultMesh publication +- full redacted per-document Heimdall auth witness export or CultMesh publication Canonical docs: diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index b4cec3e..4c769f0 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -57,6 +57,9 @@ What exists right now is: only public provider, required access, and queue protection - explicit state/doctrine scaffolding so the plan does not immediately turn into soup +- a daemon-owned boundary witness store plus optional Idunn RUDP health pulse + so Heimdall can advertise provider/boundary truth to Odin and Idunn without + pretending HTTP/status checks own the daemon ## Primary objective diff --git a/docs/verse-service-contract.md b/docs/verse-service-contract.md index 58b37c0..6138625 100644 --- a/docs/verse-service-contract.md +++ b/docs/verse-service-contract.md @@ -40,8 +40,8 @@ Heimdall currently has durable Postgres storage for deployed auth/control-plane state. That is acceptable for live auth operations, but it is not enough for the GameCult Verse service contract by itself. -The next state cut is a CultCache `.cc` witness/export path for Heimdall-owned -auth/control-plane documents: +The next full state cut is a redacted CultCache `.cc` witness/export path for +Heimdall-owned auth/control-plane documents: - `heimdall.account.v0` - `heimdall.linked_identity.v0` @@ -53,8 +53,12 @@ auth/control-plane documents: - `heimdall.app_profile.v0` - `heimdall.managed_credential_projection.v0` -The `.cc` store does not need to replace Postgres in the first pass. The first -pass should export a typed, redacted, operator-safe witness of Heimdall state so +The `.cc` store does not need to replace Postgres in the first pass. The live +first pass is narrower: it publishes a daemon-owned boundary store for Odin and +Idunn. That store carries the Heimdall provider advertisement, command +boundary, transport profile, and daemon-health summary without exporting +per-account auth truth. The next pass should extend that runtime-owned boundary +store into a typed, redacted, operator-safe witness of Heimdall auth state so CultMesh/Odin/Eve can inspect service truth without receiving provider secrets. Sensitive fields must be handled explicitly: @@ -89,41 +93,48 @@ refreshing an entitlement snapshot. Heimdall accepts or denies those actions. 1. Define Heimdall CultCache document shapes for redacted auth/control-plane witness state. -2. Add a read-only export command from the current store into a `.cc` witness. -3. Publish the witness through CultMesh with secret-safe projections only. -4. Add an Eve DSL provider over the witness and existing health/discovery data. -5. Register Heimdall's provider surface with Odin. -6. Only after the witness path is stable, decide whether any live state should +2. Publish a daemon-owned boundary `.cc` store plus Idunn daemon-health + summary from the live runtime. +3. Extend the boundary store into redacted auth/document witness exports. +4. Publish the witness through CultMesh with secret-safe projections only. +5. Add an Eve DSL provider over the witness and existing health/discovery data. +6. Register Heimdall's provider surface with Odin. +7. Only after the witness path is stable, decide whether any live state should move from Postgres into a CultCache-backed primary store. The invariant: Heimdall owns shared auth truth, not app-domain truth. CultCache and CultMesh make Heimdall inspectable; they do not leak secrets or move product state into the auth authority. -## Read-Only Witness First Cut +## Runtime Boundary First Cut -The current repo cut defines the read-only witness/ad surface without wiring a -runtime writer or migrating live state. +The current repo cut now publishes a daemon-owned boundary witness without +migrating live auth truth out of Postgres or the HTTP auth surface. Authority map: - Owner: Heimdall remains the only owner of auth/control-plane mutation. -- Inputs: static provider catalog, static app profiles, and the redacted witness - descriptor table in `src/verse-witness.ts`. -- Outputs: `gamecult.eve.provider_advertisement.v1` JSON printed by - `pnpm export:provider-advertisement`. -- Derived state: advertised CultCache paths and schema IDs are witness/export - targets. They do not prove that a live `.cc` writer exists yet. -- Forbidden writers: Odin probes, Eve renderers, dashboards, and this export - command cannot mutate accounts, linked identities, grants, sessions, tokens, - completions, entitlements, audit events, or app profiles. -- Shared paths: future CultCache export, CultMesh publication, and Eve lowering - should reuse the same document IDs and redaction doctrine. +- Inputs: static provider catalog, static app profiles, deployment/runtime + config, and the redacted witness descriptor table in `src/verse-witness.ts`. +- Outputs: the existing read-only `pnpm export:provider-advertisement` JSON plus + a runtime-owned `GC_ACCESS_CULTCACHE_PATH` store containing + `gamecult.eve.provider_advertisement.v1`, `heimdall.command_boundary.v1`, + `heimdall.transport_profile.v1`, and `idunn.daemon_health.v1` summary state. +- Derived state: `/healthz`, JWKS, discovery, systemd, nginx routing, and the + boundary store are projections. They must not mutate auth truth or replace + explicit Heimdall API ownership. +- Forbidden writers: Odin probes, Eve renderers, dashboards, Idunn, and the + boundary store cannot mutate accounts, linked identities, grants, sessions, + tokens, completions, entitlements, audit events, or app profiles. +- Shared paths: the runtime boundary store, future redacted auth witness + exports, CultMesh publication, and Eve lowering should reuse the same + document IDs and redaction doctrine. - Deletion line: if a future writer needs app-domain fields or provider tokens - to make the surface useful, the design is wrong; keep those fields out or move - the concern back to the owning app/service. + to make the surface useful, the design is wrong; keep those fields out or + move the concern back to the owning app/service. -The advertised witness descriptors currently name these redacted `.cc` targets: +The boundary store currently names the daemon-owned publication seam. The next +redacted auth witness cut should add these `.cc` targets: - `heimdall.account.v0` at `cultcache/heimdall/accounts/{accountId}.cc` - `heimdall.linked_identity.v0` at diff --git a/notes/fresh-workspace-handoff.md b/notes/fresh-workspace-handoff.md index a75b468..5f9ff58 100644 --- a/notes/fresh-workspace-handoff.md +++ b/notes/fresh-workspace-handoff.md @@ -113,6 +113,13 @@ Do not trust this file for the exact live HEAD. Always check git. for the StreamPixels migration slice - the intended first deployment shape is a Heimdall service on Yggdrasil behind nginx, not an embedded cross-runtime shared library fantasy +- Heimdall now writes a daemon-owned boundary witness store at + `GC_ACCESS_CULTCACHE_PATH` containing provider advertisement, command + boundary, transport profile, and daemon-health summary state +- Heimdall can now publish `heimdall.cultnet-rudp-provider-health` to Idunn + over `cultnet.transport.rudp.v0` when `GC_ACCESS_IDUNN_RUDP_HEALTH` is + configured; HTTP `/healthz`, JWKS, discovery, systemd, and nginx routing + remain compatibility witnesses ## Critical Doctrine @@ -128,6 +135,18 @@ Do not trust this file for the exact live HEAD. Always check git. Do not continue implementation automatically from a rehydrate-only request. +If the user asks to continue on the Idunn/swarm transport cut, the current next +move is deployment and live verification: + +- ship the `CultLib/packages/cultnet-ts` snapshot through the Yggdrasil + Heimdall deploy lane +- configure persistent `GC_ACCESS_CULTCACHE_PATH` under `/srv/heimdall` +- configure `GC_ACCESS_IDUNN_RUDP_HEALTH=10.77.0.2:17870` +- redeploy Heimdall on Yggdrasil +- update `check-heimdall.sh` to verify the witness store and the new `/healthz` + fields +- confirm live Idunn accepts `yggdrasil-heimdall` from `10.77.0.1` + If the user asks to continue on Spotiverse, the current next move is the real browser `/auth/login` flow through public Heimdall. The Yggdrasil SSH tunnel must include reverse forward diff --git a/package.json b/package.json index fcaf3bd..cf44c9a 100644 --- a/package.json +++ b/package.json @@ -16,6 +16,8 @@ }, "dependencies": { "@fastify/cors": "^11.2.0", + "@msgpack/msgpack": "^3.1.3", + "cultnet-ts": "file:../CultLib/packages/cultnet-ts", "fastify": "^5.8.5", "pg": "^8.20.0" }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index e56ffb6..9e60119 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -11,6 +11,12 @@ importers: '@fastify/cors': specifier: ^11.2.0 version: 11.2.0 + '@msgpack/msgpack': + specifier: ^3.1.3 + version: 3.1.3 + cultnet-ts: + specifier: file:../CultLib/packages/cultnet-ts + version: file:../CultLib/packages/cultnet-ts fastify: specifier: ^5.8.5 version: 5.8.5 @@ -228,6 +234,10 @@ packages: '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} + '@msgpack/msgpack@3.1.3': + resolution: {integrity: sha512-47XIizs9XZXvuJgoaJUIE2lFoID8ugvc0jzSHP+Ptfk8nTbnR8g788wv48N03Kx0UkAv559HWRQ3yzOgzlRNUA==} + engines: {node: '>= 18'} + '@napi-rs/wasm-runtime@1.1.4': resolution: {integrity: sha512-3NQNNgA1YSlJb/kMH1ildASP9HW7/7kYnRI2szWJaofaS1hWmbGI4H+d3+22aGzXXN9IJ+n+GiFVcGipJP18ow==} peerDependencies: @@ -439,6 +449,12 @@ packages: resolution: {integrity: sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==} engines: {node: '>=18'} + cultcache-ts@file:../CultLib/packages/cultcache-ts: + resolution: {directory: ../CultLib/packages/cultcache-ts, type: directory} + + cultnet-ts@file:../CultLib/packages/cultnet-ts: + resolution: {directory: ../CultLib/packages/cultnet-ts, type: directory} + define-data-property@1.1.4: resolution: {integrity: sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==} engines: {node: '>= 0.4'} @@ -1037,6 +1053,9 @@ packages: yallist@4.0.0: resolution: {integrity: sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==} + zod@3.25.76: + resolution: {integrity: sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==} + snapshots: '@emnapi/core@1.10.0': @@ -1163,6 +1182,8 @@ snapshots: '@jridgewell/sourcemap-codec@1.5.5': {} + '@msgpack/msgpack@3.1.3': {} + '@napi-rs/wasm-runtime@1.1.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.10.0)': dependencies: '@emnapi/core': 1.10.0 @@ -1339,6 +1360,17 @@ snapshots: cookie@1.1.1: {} + cultcache-ts@file:../CultLib/packages/cultcache-ts: + dependencies: + '@msgpack/msgpack': 3.1.3 + zod: 3.25.76 + + cultnet-ts@file:../CultLib/packages/cultnet-ts: + dependencies: + '@msgpack/msgpack': 3.1.3 + ajv: 8.20.0 + cultcache-ts: file:../CultLib/packages/cultcache-ts + define-data-property@1.1.4: dependencies: es-define-property: 1.0.1 @@ -1858,3 +1890,5 @@ snapshots: xtend@4.0.2: {} yallist@4.0.0: {} + + zod@3.25.76: {} diff --git a/src/app.ts b/src/app.ts index 2c29710..e5eb0e2 100644 --- a/src/app.ts +++ b/src/app.ts @@ -525,6 +525,10 @@ export async function buildApp(options: BuildAppOptions = {}): Promise entry.trim()).filter(Boolean) ?? []; } +function readOptionalString(envValue: string | undefined): string | undefined { + const value = envValue?.trim(); + return value ? value : undefined; +} + export function loadConfig(env: NodeJS.ProcessEnv = process.env): HeimdallConfig { + const sourceRoot = fileURLToPath(new URL("../", import.meta.url)); + const workspaceRoot = path.resolve(sourceRoot, ".."); const host = env.HOST ?? "127.0.0.1"; const port = readInt(env.PORT, 4100); const publicBaseUrl = trimTrailingSlash(env.GC_ACCESS_BASE_URL ?? `http://${host}:${port}`); const issuer = trimTrailingSlash(env.GC_ACCESS_ISSUER ?? publicBaseUrl); + const dataRoot = env.GC_ACCESS_DATA_ROOT ?? path.join(workspaceRoot, ".heimdall-data"); const storageBackend = env.GC_ACCESS_STORAGE_BACKEND === "postgres" || env.GC_ACCESS_DATABASE_URL ? "postgres" : "memory"; const providersConfig = Object.fromEntries( @@ -116,8 +132,14 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): HeimdallConfig serviceName: "heimdall", host, port, + workspaceRoot, + dataRoot, + cultCachePath: env.GC_ACCESS_CULTCACHE_PATH ?? path.join(dataRoot, "cultcache", "heimdall.service.cc"), publicBaseUrl, issuer, + daemonId: env.GC_ACCESS_IDUNN_DAEMON ?? "yggdrasil-heimdall", + idunnRudpHealth: readOptionalString(env.GC_ACCESS_IDUNN_RUDP_HEALTH), + idunnHealthContract: env.GC_ACCESS_IDUNN_HEALTH_CONTRACT ?? "heimdall.cultnet-rudp-provider-health", sessionTtlSeconds: readInt(env.GC_ACCESS_SESSION_TTL_SECONDS, 3600), refreshTtlSeconds: readInt(env.GC_ACCESS_REFRESH_TTL_SECONDS, 60 * 60 * 24 * 30), stateTtlSeconds: readInt(env.GC_ACCESS_STATE_TTL_SECONDS, 600), diff --git a/src/idunn-rudp-health.ts b/src/idunn-rudp-health.ts new file mode 100644 index 0000000..9124a5a --- /dev/null +++ b/src/idunn-rudp-health.ts @@ -0,0 +1,190 @@ +import dgram from "node:dgram"; +import { encode } from "@msgpack/msgpack"; +import { + encodeCultNetMessageForWire, + encodeRudpPacket, + type CultNetDocumentPutRawMessage, + type CultNetRudpPacket, +} from "cultnet-ts"; +import type { HeimdallConfig } from "./config.js"; + +const CULTNET_RUDP_PROTOCOL_ID = "cultnet.transport.rudp.v0"; +const IDUNN_HEALTH_RUDP_CONNECTION_ID = 0x1d0d0001; +const RUDP_HEALTH_CONNECT_ATTEMPTS = 3; +const RUDP_PULSE_POST_SEND_GRACE_MS = 1000; +const RUDP_CONNECT_TO_DATA_GRACE_MS = 300; + +type Endpoint = { + host: string; + port: number; +}; + +type IdunnHealthInput = { + daemonId: string; + state: string; + detail: string; + observedAt: string; +}; + +export async function publishIdunnRudpHealth(config: HeimdallConfig, health: IdunnHealthInput): Promise { + if (!config.idunnRudpHealth) { + return; + } + + const endpoint = parseEndpoint(config.idunnRudpHealth); + let lastError: unknown; + for (let attempt = 1; attempt <= RUDP_HEALTH_CONNECT_ATTEMPTS; attempt += 1) { + try { + await publishIdunnRudpHealthOnce(endpoint, config, health); + return; + } catch (error) { + lastError = error; + if ((error as Error & { code?: string }).code !== "ETIMEDOUT") { + throw error; + } + } + } + throw lastError; +} + +async function publishIdunnRudpHealthOnce( + endpoint: Endpoint, + config: HeimdallConfig, + health: IdunnHealthInput, +): Promise { + const socket = dgram.createSocket(endpointFamily(endpoint.host)); + + try { + await bindSocket(socket, endpoint); + const connect = buildConnectPacket(); + await sendPacket(socket, endpoint, connect); + await sleep(RUDP_CONNECT_TO_DATA_GRACE_MS); + const payload = buildDocumentPutPayload(config, health); + await sendPacket(socket, endpoint, buildSchemaDataPacket(payload)); + await sleep(RUDP_PULSE_POST_SEND_GRACE_MS); + } finally { + socket.close(); + } +} + +function buildDocumentPutPayload(config: HeimdallConfig, health: IdunnHealthInput): Uint8Array { + const recordPayload = encode([ + health.daemonId, + health.state, + health.detail, + health.observedAt, + config.idunnHealthContract, + "daemon-published", + CULTNET_RUDP_PROTOCOL_ID, + ]); + const message: CultNetDocumentPutRawMessage = { + schemaVersion: "cultnet.document_put_raw.v0", + messageId: `heimdall-health:${health.daemonId}:${health.observedAt.replace(/[:.]/g, "-")}`, + document: { + schemaId: "idunn.daemon_health", + recordKey: health.daemonId, + storedAt: health.observedAt, + payloadEncoding: "messagepack", + payload: recordPayload, + sourceRuntimeId: "heimdall-service", + sourceRole: "daemon-health-publisher", + tags: [CULTNET_RUDP_PROTOCOL_ID], + }, + }; + return encode(encodeCultNetMessageForWire(message, "cultnet.schema.v0")); +} + +function buildConnectPacket(): CultNetRudpPacket { + return { + packetType: "connect", + connectionId: IDUNN_HEALTH_RUDP_CONNECTION_ID, + sequence: 1, + ack: 0, + ackMask: 0, + channelId: "control", + reliable: true, + ordered: true, + sequenced: false, + payload: new Uint8Array(), + }; +} + +function buildSchemaDataPacket(payload: Uint8Array): CultNetRudpPacket { + return { + packetType: "data", + connectionId: IDUNN_HEALTH_RUDP_CONNECTION_ID, + sequence: 2, + ack: 0, + ackMask: 0, + channelId: "schema", + reliable: true, + ordered: true, + sequenced: false, + payload, + }; +} + +async function bindSocket(socket: dgram.Socket, endpoint: Endpoint): Promise { + await new Promise((resolve, reject) => { + socket.once("error", reject); + socket.bind(0, localBindHost(endpoint.host), () => { + socket.off("error", reject); + resolve(); + }); + }); +} + +function localBindHost(remoteHost: string): string { + if (remoteHost === "localhost" || remoteHost.startsWith("127.")) { + return "127.0.0.1"; + } + if (remoteHost === "::1") { + return "::1"; + } + return remoteHost.includes(":") ? "::" : "0.0.0.0"; +} + +async function sendPacket(socket: dgram.Socket, endpoint: Endpoint, input: CultNetRudpPacket): Promise { + const wire = encodeRudpPacket(input); + await new Promise((resolve, reject) => { + socket.send(wire, endpoint.port, endpoint.host, (error) => { + if (error) { + reject(error); + } else { + resolve(); + } + }); + }); +} + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +function endpointFamily(host: string): "udp4" | "udp6" { + return host.includes(":") ? "udp6" : "udp4"; +} + +function parseEndpoint(value: string): Endpoint { + const text = value.trim(); + const ipv6 = text.match(/^\[([^\]]+)\]:(\d+)$/); + if (ipv6) { + return { host: ipv6[1]!, port: parsePort(ipv6[2]!) }; + } + const index = text.lastIndexOf(":"); + if (index <= 0) { + throw new Error(`Idunn RUDP endpoint must be host:port, got "${value}".`); + } + return { + host: text.slice(0, index), + port: parsePort(text.slice(index + 1)), + }; +} + +function parsePort(value: string): number { + const port = Number(value); + if (!Number.isInteger(port) || port <= 0 || port > 65535) { + throw new Error(`Idunn RUDP endpoint port is invalid: ${value}`); + } + return port; +} diff --git a/src/index.ts b/src/index.ts index eeb1f85..417b469 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,13 +1,36 @@ import { buildApp } from "./app.js"; import { loadConfig } from "./config.js"; +import { publishIdunnRudpHealth } from "./idunn-rudp-health.js"; +import { createHeimdallRuntimePulse, buildHeimdallHealthDetail, publishHeimdallVerseState } from "./verse-state.js"; const config = loadConfig(); const app = await buildApp({ config }); +const versePulseIntervalMs = 60_000; try { await app.listen({ host: config.host, port: config.port }); app.log.info(`Heimdall listening on ${config.host}:${config.port}`); + await publishVerseState().catch((error) => { + console.error("Heimdall verse publication failed on startup:", error); + }); + const timer = setInterval(() => { + void publishVerseState().catch((error) => { + console.error("Heimdall verse publication failed on interval:", error); + }); + }, versePulseIntervalMs); + timer.unref?.(); } catch (error) { app.log.error(error); process.exitCode = 1; } + +async function publishVerseState(): Promise { + const pulse = createHeimdallRuntimePulse(config); + await publishHeimdallVerseState(config, pulse); + await publishIdunnRudpHealth(config, { + daemonId: config.daemonId, + state: "active", + detail: buildHeimdallHealthDetail(config, pulse), + observedAt: pulse.updatedAt, + }); +} diff --git a/src/verse-state.ts b/src/verse-state.ts new file mode 100644 index 0000000..2ea4b37 --- /dev/null +++ b/src/verse-state.ts @@ -0,0 +1,203 @@ +import { mkdir, rename, rm, writeFile } from "node:fs/promises"; +import path from "node:path"; +import { encode } from "@msgpack/msgpack"; +import { appSlugs, providers } from "./contracts.js"; +import type { HeimdallConfig } from "./config.js"; +import { buildHeimdallProviderAdvertisement } from "./verse-witness.js"; + +const STORE_FORMAT_VERSION = "cultcache.store.v1"; + +type CultCacheRecord = { + key: string; + schemaId: string; + schemaName: string; + schemaVersion: string; + payload: unknown; + storedAt: string; +}; + +export type HeimdallRuntimePulse = { + updatedAt: string; + configuredProviders: string[]; + storageBackend: string; + appProfileCount: number; + cultCachePath: string; +}; + +export function createHeimdallRuntimePulse(config: HeimdallConfig, updatedAt = new Date().toISOString()): HeimdallRuntimePulse { + return { + updatedAt, + configuredProviders: providers.filter((provider) => { + const entry = config.providers[provider]; + return Boolean(entry?.clientId && entry.clientSecret); + }), + storageBackend: config.storage.backend, + appProfileCount: appSlugs.length, + cultCachePath: config.cultCachePath, + }; +} + +export async function publishHeimdallVerseState(config: HeimdallConfig, pulse: HeimdallRuntimePulse): Promise { + const records: CultCacheRecord[] = [ + { + key: "heimdall", + schemaId: "gamecult.eve.provider_advertisement.v1", + schemaName: "gamecult.eve.provider_advertisement", + schemaVersion: "gamecult.eve.provider_advertisement.v1", + payload: buildRuntimeProviderAdvertisement(config, pulse), + storedAt: pulse.updatedAt, + }, + { + key: "heimdall", + schemaId: "heimdall.command_boundary.v1", + schemaName: "heimdall.command_boundary", + schemaVersion: "heimdall.command_boundary.v1", + payload: buildCommandBoundary(config, pulse), + storedAt: pulse.updatedAt, + }, + { + key: "heimdall", + schemaId: "heimdall.transport_profile.v1", + schemaName: "heimdall.transport_profile", + schemaVersion: "heimdall.transport_profile.v1", + payload: buildTransportProfile(config, pulse), + storedAt: pulse.updatedAt, + }, + { + key: config.daemonId, + schemaId: "idunn.daemon_health", + schemaName: "idunn.daemon_health", + schemaVersion: "idunn.daemon_health.v1", + payload: buildDaemonHealthRecord(config, pulse), + storedAt: pulse.updatedAt, + }, + ]; + + await writeCultCacheSnapshot(config.cultCachePath, records); +} + +export function buildHeimdallHealthDetail(config: HeimdallConfig, pulse: HeimdallRuntimePulse): string { + const healthTransport = config.idunnRudpHealth ? "CultNet/RUDP" : "CultCache witness only"; + return `Heimdall auth runtime active; storage=${pulse.storageBackend}; providers=${pulse.configuredProviders.length}/${providers.length}; apps=${pulse.appProfileCount}; healthTransport=${healthTransport}`; +} + +function buildRuntimeProviderAdvertisement(config: HeimdallConfig, pulse: HeimdallRuntimePulse) { + const advertisement = buildHeimdallProviderAdvertisement({ updatedAt: pulse.updatedAt }); + const witnessPath = path.relative(config.workspaceRoot, config.cultCachePath).replace(/\\/g, "/"); + return { + ...advertisement, + status: "daemon_live" as const, + version: "daemon-live-v1", + provider: { + ...advertisement.provider, + transport: config.idunnRudpHealth + ? "CultCache witness store + daemon-published Idunn health over CultNet/RUDP." + : "CultCache witness store; Idunn health still falls back to local witness publication.", + }, + controlSurface: { + ...advertisement.controlSurface, + controls: { + ...advertisement.controlSurface.controls, + reason: "This runtime publication advertises provider shape and lifecycle boundaries. Auth mutation remains behind explicit Heimdall HTTP APIs.", + }, + }, + runtime: { + storageBackend: pulse.storageBackend, + configuredProviders: pulse.configuredProviders, + appProfileCount: pulse.appProfileCount, + cultCachePath: witnessPath, + idunnHealthContract: config.idunnHealthContract, + idunnRudpHealth: config.idunnRudpHealth ?? null, + }, + }; +} + +function buildCommandBoundary(config: HeimdallConfig, pulse: HeimdallRuntimePulse) { + return { + schema: "heimdall.command_boundary.v1", + boundaryId: "heimdall", + daemonId: config.daemonId, + providerId: "heimdall", + updatedAt: pulse.updatedAt, + owner: "Heimdall auth runtime", + lifecycleAuthority: "idunn.yggdrasil-source-app.deploy", + healthPublication: { + contract: config.idunnHealthContract, + transport: config.idunnRudpHealth ? "cultnet.transport.rudp.v0" : "cultcache-store", + publicationSource: config.idunnRudpHealth ? "daemon-published" : "daemon-published-cultcache", + stateOwner: "Heimdall auth runtime", + }, + commands: [], + forbiddenWriters: [ + "Odin and Idunn may observe Heimdall boundary state but do not mutate auth/control-plane truth.", + "HTTP /healthz, JWKS, discovery, systemd, and nginx routing are compatibility witnesses, not daemon truth.", + ], + compatibility: { + httpHealth: "/healthz", + jwks: "/.well-known/jwks.json", + discovery: "/.well-known/heimdall-configuration", + yggdrasilDeployLane: "E:/Projects/Odin/scripts/deploy-yggdrasil-heimdall.cmd", + status: "fallback-witness-only", + }, + }; +} + +function buildTransportProfile(config: HeimdallConfig, pulse: HeimdallRuntimePulse) { + return { + schema: "heimdall.transport_profile.v1", + profileId: "heimdall", + daemonId: config.daemonId, + providerId: "heimdall", + updatedAt: pulse.updatedAt, + targetTransport: "cultnet.transport.rudp.v0", + currentTransport: config.idunnRudpHealth + ? "cultcache-redacted-witness + daemon-published-rudp-health + compatibility-http" + : "cultcache-redacted-witness + compatibility.ssh-systemd-http", + healthTransport: config.idunnRudpHealth ? "cultnet.transport.rudp.v0" : "cultcache-store", + stateTransport: "cultcache-store", + rendererTransport: "browser-http-jwks-discovery-lowering", + storageBackend: pulse.storageBackend, + cutLine: config.idunnRudpHealth + ? "Heimdall now publishes daemon-owned witness state and Idunn health; systemd, /healthz, JWKS, discovery, and nginx routing can be demoted to deployment/debug witnesses once Odin consumes the typed store." + : "Heimdall now publishes a daemon-owned witness store; the remaining transport debt is Idunn health publication over CultNet/RUDP and Odin ingestion of the typed witness surface.", + }; +} + +function buildDaemonHealthRecord(config: HeimdallConfig, pulse: HeimdallRuntimePulse) { + return [ + config.daemonId, + "active", + buildHeimdallHealthDetail(config, pulse), + pulse.updatedAt, + config.idunnHealthContract, + config.idunnRudpHealth ? "daemon-published" : "daemon-published-cultcache", + config.idunnRudpHealth ? "cultnet.transport.rudp.v0" : "cultcache-store", + ]; +} + +async function writeCultCacheSnapshot(filePath: string, records: CultCacheRecord[]): Promise { + await mkdir(path.dirname(filePath), { recursive: true }); + const catalog = records.map((record) => [ + record.schemaId, + record.schemaName, + record.schemaVersion, + record.schemaId, + JSON.stringify({ + schemaName: record.schemaName, + schemaVersion: record.schemaVersion, + members: [], + }), + [record.schemaId], + [], + ]); + const entries = records.map((record) => [record.key, record.schemaId, record.storedAt, encode(record.payload)]); + const tempPath = `${filePath}.tmp-${process.pid}-${Date.now()}`; + + try { + await writeFile(tempPath, encode([STORE_FORMAT_VERSION, catalog, entries])); + await rename(tempPath, filePath); + } catch (error) { + await rm(tempPath, { force: true }).catch(() => undefined); + throw error; + } +} diff --git a/src/verse-witness.ts b/src/verse-witness.ts index 2860010..9e7641c 100644 --- a/src/verse-witness.ts +++ b/src/verse-witness.ts @@ -128,12 +128,12 @@ export interface EveProviderAdvertisementDocument { verseId: "gamecult"; rootVerse: "asgard"; canonicalService: "asgard.heimdall"; - locatedService: "asgard.starfire.heimdall"; - cultMeshAddress: "asgard.starfire.heimdall/eve/tui"; + locatedService: "asgard.yggdrasil.heimdall"; + cultMeshAddress: "asgard.yggdrasil.heimdall/eve/tui"; title: "Heimdall"; description: string; version: string; - status: "read_only_witness_planned"; + status: "read_only_witness_planned" | "daemon_live"; updatedAt: string; provider: { id: "heimdall"; @@ -169,8 +169,8 @@ export function buildHeimdallProviderAdvertisement(options: { updatedAt: string verseId: "gamecult", rootVerse: "asgard", canonicalService: "asgard.heimdall", - locatedService: "asgard.starfire.heimdall", - cultMeshAddress: "asgard.starfire.heimdall/eve/tui", + locatedService: "asgard.yggdrasil.heimdall", + cultMeshAddress: "asgard.yggdrasil.heimdall/eve/tui", title: "Heimdall", description: "Shared GameCult auth/control-plane authority with redacted CultCache witness projections.", version: "read-only-witness-v0", @@ -193,7 +193,7 @@ export function buildHeimdallProviderAdvertisement(options: { updatedAt: string transport: "CultMesh provider advertisement; first cut is a read-only fixture/export.", }, controlSurface: { - primary: "asgard.starfire.heimdall/eve/tui", + primary: "asgard.yggdrasil.heimdall/eve/tui", controls: { mode: "read-only", writes: false, @@ -201,9 +201,9 @@ export function buildHeimdallProviderAdvertisement(options: { updatedAt: string }, }, endpoints: [ - "asgard.starfire.heimdall/eve/tui", - "asgard.starfire.heimdall/eve/gui", - "asgard.starfire.heimdall/witness", + "asgard.yggdrasil.heimdall/eve/tui", + "asgard.yggdrasil.heimdall/eve/gui", + "asgard.yggdrasil.heimdall/witness", ], routes: [ { diff --git a/state/evidence.jsonl b/state/evidence.jsonl index c01dd68..ca86bd8 100644 --- a/state/evidence.jsonl +++ b/state/evidence.jsonl @@ -28,3 +28,4 @@ {"ts": "2026-06-02T18:36:46+00:00", "type": "deployment", "status": "ok", "note": "Public Yggdrasil Heimdall was deployed with the Spotify/Spotiverse provider code, Spotify credentials from secrets/spotify.txt, a generated Spotiverse app shared secret, and GC_ACCESS_APP_SPOTIVERSE_BACKEND_CALLBACK_URLS set to the Yggdrasil loopback reverse-forward callback. Local Spotiverse now points at public Heimdall, and the Yggdrasil SSH tunnel exposes 127.0.0.1:18796 back to local Spotiverse :8796."} {"ts": "2026-06-08T20:02:27+00:00", "type": "implementation", "status": "accepted", "note": "Added Bifrost as a Heimdall app profile for Discord/Patreon member access. Bifrost owns KTLST role and Patreon tier policies through backend-callback OAuth starts; Heimdall owns provider OAuth/token custody and generic configured backend callback allowlisting."} {"ts": "2026-06-09T19:21:17+00:00", "type": "implementation", "status": "accepted", "note": "Bifrost patron support sync reuses Heimdall's existing linked Patreon identity and membership reader. Heimdall exposes app-authenticated /v1/apps/bifrost/patron-support/sync, refreshes Patreon custody when needed, maps active paid tier membership into a signed RecurringSupportSnapshot, and posts it to Bifrost's /heimdall/patron-support/events intake; Bifrost owns points, tiers, voting weight, and ledger meaning."} +{"ts": "2026-06-16T16:59:54+00:00", "type": "implementation", "status": "accepted", "note": "Heimdall now writes a daemon-owned CultCache boundary store with provider advertisement, command boundary, transport profile, and daemon-health summary state, and can publish heimdall.cultnet-rudp-provider-health to Idunn over cultnet.transport.rudp.v0 when configured."} diff --git a/state/map.yaml b/state/map.yaml index 391de09..4bbeb3f 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -32,7 +32,7 @@ invariants: - Host apps verify signed claims locally and keep resource ownership checks local. - StreamPixels can reuse shared auth authority without centralizing streamer-audience data. - Spotiverse consumes Spotify as a Heimdall-managed credential; Spotiverse does not own Spotify refresh-token custody. - - Heimdall's first Odin/Eve provider advertisement is read-only: it names redacted CultCache witness document shapes and paths, but does not write `.cc` state or mutate auth truth. + - Heimdall's first Odin/Eve daemon boundary publication remains secret-safe: it writes only provider advertisement, command boundary, transport profile, and daemon-health summary state, not per-account auth truth. - Persistent state cruft is the same failure class as architecture cruft. data_flow: @@ -58,6 +58,7 @@ active_subgoals: - Keep the shared architecture explicit and separate from app profiles. - Preserve the default boundary that app-domain data remains app-owned. - Align the plan to Heimdall as a Yggdrasil-hosted authority service. + - Publish Heimdall daemon health and boundary state for Idunn/Odin without moving auth truth out of Heimdall. - Verify the Discord and Patreon Repixelizer auth paths with real provider credentials, stable signing, persistent storage, and live browser callback verification. - Wire StreamPixels through Heimdall for Twitch/YouTube viewer identity and managed creator-provider credentials without moving StreamPixels domain state into Heimdall. @@ -81,6 +82,7 @@ accepted_design: - StreamPixels viewer and creator provider OAuth should start through Heimdall; StreamPixels redeems one-time completion codes, owns local profile/connector binding, and resolves current provider access tokens from Heimdall by app-authenticated managed-credential calls. - Bifrost is a Heimdall app profile for Discord/Patreon member access. Bifrost owns the KTLST Discord role policy and Patreon tier policy; Heimdall owns provider OAuth, token custody, entitlement evaluation, and configured backend callback trust. - Bifrost patron support sync reuses Heimdall's existing linked Patreon identity and membership reader. Heimdall turns active paid Patreon tier evidence into signed support facts; Bifrost owns point/tier/vote derivation. + - Heimdall's first runtime Verse cut publishes a service-owned CultCache boundary store and optional Idunn RUDP health pulse; full redacted auth-document witness export remains a later pass. landed_baseline: - Shared architecture extracted into its own repo and renamed to Heimdall. @@ -99,6 +101,7 @@ landed_baseline: - Bifrost now has a Heimdall app profile for Discord/Patreon member access, and backend callback trust is checked through configured app callback allowlists before legacy app-specific defaults. - Heimdall exposes `POST /v1/apps/bifrost/patron-support/sync` as an app-authenticated bridge from linked Patreon membership evidence to Bifrost's signed patron support intake endpoint. - A read-only `gamecult.eve.provider_advertisement.v1` export now advertises Heimdall provider/app surfaces plus redacted CultCache witness document paths for Odin/Eve discovery. + - Heimdall now writes a daemon-owned `GC_ACCESS_CULTCACHE_PATH` boundary store containing provider advertisement, command boundary, transport profile, and daemon-health summary state, and it can publish `heimdall.cultnet-rudp-provider-health` to Idunn over `cultnet.transport.rudp.v0` when configured. rejected_paths: - name: per_app_auth_copy_paste @@ -109,5 +112,5 @@ rejected_paths: reason: reusing auth mechanics does not justify centralizing app-domain audience or product data current_status: - summary: Heimdall and StreamPixels are deployed on Yggdrasil for the StreamPixels auth migration slice. Heimdall has Twitch/YouTube provider credentials, the shared StreamPixels app secret, generic Twitch/YouTube identity and managed-credential custody runtimes, and a live fix for Twitch token responses that return scopes as arrays. Live StreamPixels Twitch sign-in through Heimdall has succeeded and redeemed a completion code. StreamPixels delegates Twitch/YouTube OAuth to Heimdall, redeems one-time completion codes server-side, writes its own httpOnly viewer session, stores local profile/creator binding state, and asks Heimdall for current provider access tokens when runtime code needs them. Spotiverse is added as a Heimdall app profile with Spotify as a managed credential provider. Bifrost is added as a Heimdall app profile for Discord/Patreon member access through backend-callback handoff, with Bifrost owning KTLST role and Patreon tier policies while Heimdall owns provider OAuth/token custody and callback allowlist enforcement. Bifrost patron support sync now reuses Heimdall's linked Patreon identity and membership reader to emit signed recurring-support facts to Bifrost; Bifrost still owns ledgers, patron points, voting weight, and audit meaning. The key boundary still holds: Heimdall owns provider OAuth and credential custody; host apps own local sessions, route/resource checks, runtime behavior, and app-domain data. - next_action: Deploy Heimdall with `GC_ACCESS_APP_BIFROST_SHARED_SECRET`, `GC_ACCESS_BIFROST_PATRON_SUPPORT_ENDPOINT`, and `GC_ACCESS_BIFROST_PATRON_SUPPORT_SECRET`, then trigger `/v1/apps/bifrost/patron-support/sync` for a linked Patreon account and verify Bifrost records the recurring support snapshot. After that, return to Spotiverse and StreamPixels browser verification. + summary: Heimdall and StreamPixels are deployed on Yggdrasil for the StreamPixels auth migration slice. Heimdall has Twitch/YouTube provider credentials, the shared StreamPixels app secret, generic Twitch/YouTube identity and managed-credential custody runtimes, and a live fix for Twitch token responses that return scopes as arrays. Live StreamPixels Twitch sign-in through Heimdall has succeeded and redeemed a completion code. StreamPixels delegates Twitch/YouTube OAuth to Heimdall, redeems one-time completion codes server-side, writes its own httpOnly viewer session, stores local profile/creator binding state, and asks Heimdall for current provider access tokens when runtime code needs them. Spotiverse is added as a Heimdall app profile with Spotify as a managed credential provider. Bifrost is added as a Heimdall app profile for Discord/Patreon member access through backend-callback handoff, with Bifrost owning KTLST role and Patreon tier policies while Heimdall owns provider OAuth/token custody and callback allowlist enforcement. Bifrost patron support sync now reuses Heimdall's linked Patreon identity and membership reader to emit signed recurring-support facts to Bifrost; Bifrost still owns ledgers, patron points, voting weight, and audit meaning. Heimdall now also has a daemon-owned boundary witness store and optional Idunn RUDP health pulse for the swarm transport cut. The key boundary still holds: Heimdall owns provider OAuth and credential custody; host apps own local sessions, route/resource checks, runtime behavior, and app-domain data. + next_action: Ship the CultLib snapshot through the Yggdrasil Heimdall deploy lane, deploy with a persistent `GC_ACCESS_CULTCACHE_PATH` plus `GC_ACCESS_IDUNN_RUDP_HEALTH=10.77.0.2:17870`, then update `check-heimdall.sh` to verify the witness store and confirm live Idunn accepts `yggdrasil-heimdall` from `10.77.0.1`. After that, return to Bifrost patron-support sync and the remaining browser verification passes. diff --git a/tests/app.test.ts b/tests/app.test.ts index 1275cea..2b1bb61 100644 --- a/tests/app.test.ts +++ b/tests/app.test.ts @@ -13,8 +13,14 @@ function createTestConfig(): HeimdallConfig { serviceName: "heimdall", host: "127.0.0.1", port: 4100, + workspaceRoot: "E:/Projects/Heimdall", + dataRoot: "E:/Projects/Heimdall/.heimdall-data", + cultCachePath: "E:/Projects/Heimdall/.heimdall-data/cultcache/heimdall.service.cc", publicBaseUrl: "https://heimdall.gamecult.org", issuer: "https://heimdall.gamecult.org", + daemonId: "yggdrasil-heimdall", + idunnRudpHealth: undefined, + idunnHealthContract: "heimdall.cultnet-rudp-provider-health", sessionTtlSeconds: 3600, refreshTtlSeconds: 60 * 60 * 24 * 30, stateTtlSeconds: 600, @@ -175,7 +181,7 @@ describe("Heimdall service", () => { ], }) ); - }); + }, 15_000); it("builds signed OAuth start state for supported providers", async () => { const app = await buildApp({ config: createTestConfig() }); diff --git a/tests/security.test.ts b/tests/security.test.ts index 154f3f0..1aa7c02 100644 --- a/tests/security.test.ts +++ b/tests/security.test.ts @@ -11,8 +11,14 @@ function createTestConfig(overrides: Partial = {}): HeimdallConf serviceName: "heimdall", host: "127.0.0.1", port: 4100, + workspaceRoot: "E:/Projects/Heimdall", + dataRoot: "E:/Projects/Heimdall/.heimdall-data", + cultCachePath: "E:/Projects/Heimdall/.heimdall-data/cultcache/heimdall.service.cc", publicBaseUrl: "https://heimdall.gamecult.org", issuer: "https://heimdall.gamecult.org", + daemonId: "yggdrasil-heimdall", + idunnRudpHealth: undefined, + idunnHealthContract: "heimdall.cultnet-rudp-provider-health", sessionTtlSeconds: 3600, refreshTtlSeconds: 60 * 60 * 24 * 30, stateTtlSeconds: 600, diff --git a/tests/verse-runtime.test.ts b/tests/verse-runtime.test.ts new file mode 100644 index 0000000..b009592 --- /dev/null +++ b/tests/verse-runtime.test.ts @@ -0,0 +1,66 @@ +import { mkdtemp, readFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { decode } from "@msgpack/msgpack"; +import { describe, expect, it } from "vitest"; +import { loadConfig } from "../src/config.js"; +import { createHeimdallRuntimePulse, publishHeimdallVerseState } from "../src/verse-state.js"; + +describe("Heimdall verse runtime publication", () => { + it("writes provider advertisement, command boundary, transport profile, and daemon health to the witness store", async () => { + const tmp = await mkdtemp(path.join(os.tmpdir(), "heimdall-verse-runtime-")); + const config = loadConfig({ + GC_ACCESS_BASE_URL: "https://heimdall.gamecult.org", + GC_ACCESS_CULTCACHE_PATH: path.join(tmp, "cultcache", "heimdall.service.cc"), + GC_ACCESS_DATABASE_URL: "postgres://heimdall:test@localhost/heimdall", + GC_ACCESS_IDUNN_RUDP_HEALTH: "10.77.0.2:17870", + GC_ACCESS_PROVIDER_DISCORD_CLIENT_ID: "discord-client", + GC_ACCESS_PROVIDER_DISCORD_CLIENT_SECRET: "discord-secret", + GC_ACCESS_PROVIDER_TWITCH_CLIENT_ID: "twitch-client", + GC_ACCESS_PROVIDER_TWITCH_CLIENT_SECRET: "twitch-secret", + }); + const pulse = createHeimdallRuntimePulse(config, "2026-06-16T17:10:00.000Z"); + + await publishHeimdallVerseState(config, pulse); + + const contents = await readFile(config.cultCachePath); + const decoded = decode(contents) as [string, unknown[], Array<[string, string, string, Uint8Array]>]; + + expect(decoded[0]).toBe("cultcache.store.v1"); + const entries = decoded[2]; + const bySchema = new Map(entries.map((entry) => [entry[1], decode(entry[3])])); + + expect(bySchema.get("gamecult.eve.provider_advertisement.v1")).toMatchObject({ + providerId: "heimdall", + status: "daemon_live", + locatedService: "asgard.yggdrasil.heimdall", + runtime: { + storageBackend: "postgres", + configuredProviders: ["discord", "twitch"], + appProfileCount: 4, + }, + }); + expect(bySchema.get("heimdall.command_boundary.v1")).toMatchObject({ + boundaryId: "heimdall", + daemonId: "yggdrasil-heimdall", + healthPublication: { + contract: "heimdall.cultnet-rudp-provider-health", + transport: "cultnet.transport.rudp.v0", + }, + }); + expect(bySchema.get("heimdall.transport_profile.v1")).toMatchObject({ + daemonId: "yggdrasil-heimdall", + targetTransport: "cultnet.transport.rudp.v0", + healthTransport: "cultnet.transport.rudp.v0", + }); + expect(bySchema.get("idunn.daemon_health")).toEqual([ + "yggdrasil-heimdall", + "active", + "Heimdall auth runtime active; storage=postgres; providers=2/6; apps=4; healthTransport=CultNet/RUDP", + "2026-06-16T17:10:00.000Z", + "heimdall.cultnet-rudp-provider-health", + "daemon-published", + "cultnet.transport.rudp.v0", + ]); + }); +});