Heimdall/docs/implementation-plan.md

276 lines
11 KiB
Markdown
Raw Normal View History

2026-04-26 11:41:27 +01:00
# Heimdall Implementation Plan
2026-04-26 05:18:35 +01:00
## What this file is
2026-04-26 11:41:27 +01:00
This file is the forward plan and active hypothesis ledger for Heimdall.
2026-04-26 05:18:35 +01:00
It is not the canonical shared architecture note. That lives in
`docs/architecture.md`.
If the code or the architecture note disagrees with this plan, trust the code
and the architecture note first, then fix this file.
## Current machine
2026-04-26 12:25:11 +01:00
There is now a first landed Heimdall skeleton.
2026-04-26 05:18:35 +01:00
What exists right now is:
2026-04-26 11:41:27 +01:00
- the shared architecture note
2026-04-26 12:25:11 +01:00
- the concrete service contract in `docs/service-contract.md`
2026-04-26 05:18:35 +01:00
- app-binding notes for Repixelizer and StreamPixels
2026-04-26 12:25:11 +01:00
- a standalone TypeScript/Fastify service skeleton under `src/`
- Ed25519 JWT signing plus JWKS/discovery exposure
- OAuth start/callback contract surfaces with signed `state`
- app-profile-driven claim issuance for Repixelizer and StreamPixels
2026-04-26 12:50:07 +01:00
- in-memory/Postgres auth-control-plane storage
- persisted accounts, linked identities, sessions, grants, entitlement
snapshots, and audit records
- a real Discord provider callback slice for Repixelizer access
2026-04-26 17:00:57 +01:00
- a direct backend callback handoff for same-host app integrations
- a one-time browser completion handoff so auth callback pages can post back to
2026-04-26 17:00:57 +01:00
the opener instead of leaking final app auth in URL fragments, now retained
as fallback rather than the preferred same-host path
- file-backed signing-key loading/bootstrap for stable service identity
- AES-256-GCM token sealing for managed provider credentials at rest
- a local verifier helper that app backends can use against Heimdall JWKS
2026-04-29 11:51:29 +01:00
- Repixelizer has consumed the direct backend callback flow in its hosted web
layer and verifies Heimdall Ed25519 access tokens locally
- signed refresh tokens and `/v1/apps/{appSlug}/sessions/refresh` let app
backends reissue short-lived app claims without restarting provider OAuth
- Discord and Patreon access-token refresh landed for Repixelizer entitlement
re-evaluation
2026-04-29 15:00:55 +01:00
- generic Twitch and YouTube OAuth callback runtimes for StreamPixels identity
and managed creator connections
- an app-authenticated managed-credential resolve endpoint so StreamPixels can
call provider APIs without storing refresh tokens locally
- managed Twitch/YouTube/Joystick credentials refresh inside Heimdall before resolution
when they are expired or near expiry
2026-04-29 15:00:55 +01:00
- StreamPixels now starts viewer and creator Twitch/YouTube OAuth through
Heimdall, redeems one-time completion codes server-side, and keeps local
profile/creator connector binding state in StreamPixels
2026-04-29 12:41:43 +01:00
- Heimdall is deployed on Yggdrasil at `https://heimdall.gamecult.org` with
nginx/TLS, Postgres storage, stable file-backed signing, configured token
custody, and public health/discovery/JWKS checks passing
- Repixelizer is deployed on Yggdrasil at
`https://repixelizer.gamecult.org` with Heimdall mode enabled, Discord as the
only public provider, required access, and queue protection
2026-04-26 12:25:11 +01:00
- explicit state/doctrine scaffolding so the plan does not immediately turn into
soup
- a daemon-owned boundary witness store plus optional Idunn RUDP health pulse
so Heimdall can advertise provider/boundary truth to Odin and Idunn without
pretending HTTP/status checks own the daemon
- the `gamecult.heimdall.access` Eve plugin, persistent auth attempts, and a
loopback-only authenticated CultNet command boundary for Ghostlight
2026-04-26 05:18:35 +01:00
## Primary objective
2026-04-26 11:41:27 +01:00
Build Heimdall as a shared auth authority service that can:
2026-04-26 05:18:35 +01:00
- own provider OAuth and identity linking
2026-04-26 11:41:27 +01:00
- issue signed sessions and app-facing claims
2026-04-26 05:18:35 +01:00
- refresh Discord and Patreon entitlements
- evaluate per-app capability rules
2026-04-26 11:41:27 +01:00
- expose a clean contract to app backends without owning their domain data
2026-04-26 05:18:35 +01:00
## Current priorities
### 0. Complete the Ghostlight Eve access cut
Landed in Heimdall:
- public plugin manifest, advertisement, state schemas, fixtures, pure ABI, and
browser reference adapter
- persistent attempt lifecycle in memory and Postgres
- encrypted/HMAC-bound `heimdall.auth.begin` and `heimdall.auth.complete`
- exact source-runtime admission, nonce/expiry checks, idempotent retries, and
single-use completion redemption
- Odin-safe plugin and private-route metadata without secret-bearing state
Remaining outside this repo:
- complete the two-account and eight-account Ghostlight human canaries
- pressure refresh, logout, stale completion, and Odin-outage behavior through
the deployed browser host
Live proof:
- Heimdall `1086aee01169bf60e8a492b2740db1c6f3e8cabf` with CultLib
`5cefa0db0079a8e3ee22f29d7b9e6e5aa60912a9` is active on Yggdrasil
- Odin's typed snapshot returns the provider, command boundary, plugin, and
transport profile under their unique record keys
- Ghostlight accepted a canonical anonymous `heimdall.auth.begin` by resolving
that boundary through Odin; the navigation receipt exposed only advertised
Discord and Heimdall origins
2026-04-26 11:41:27 +01:00
### 1. Lock the service boundary
2026-04-26 05:18:35 +01:00
Goals:
2026-04-26 11:41:27 +01:00
- keep Heimdall scoped to auth/control-plane machinery
2026-04-26 05:18:35 +01:00
- keep app-domain data in app-owned stores
- keep app profiles thin and explicit
2026-04-29 11:51:29 +01:00
### 2. Keep the contract tight while it goes live
2026-04-26 05:18:35 +01:00
Goals:
2026-04-26 11:41:27 +01:00
- define signed claim/session shapes
- define OAuth start/callback/link surfaces
- define grant and entitlement persistence boundaries
- define how app backends verify claims locally
2026-04-29 11:51:29 +01:00
This now exists in reference form and Repixelizer has a Python consumer in
its hosted web layer.
2026-04-26 05:18:35 +01:00
2026-04-29 11:51:29 +01:00
### 3. Launch Repixelizer with Discord first
2026-04-26 05:18:35 +01:00
Goals:
2026-04-29 11:51:29 +01:00
- deploy Heimdall on Yggdrasil with stable signing, encrypted token custody,
and persistent Postgres storage
- configure the Discord OAuth app redirect URI to
`https://heimdall.gamecult.org/v1/oauth/discord/callback`
2026-04-29 13:22:01 +01:00
- configure Heimdall's Discord client credentials for the Repixelizer-owned
Discord OAuth app
2026-04-29 11:51:29 +01:00
- configure Repixelizer for `GC_ACCESS_MODE=heimdall` and
2026-04-29 14:39:11 +01:00
`GC_ACCESS_ALLOWED_PROVIDERS=discord,patreon`
2026-04-29 13:22:01 +01:00
- configure Repixelizer's Discord access policy with its GameCult guild id and
allowed role ids; Repixelizer sends that policy to Heimdall during the
backend-callback OAuth start
2026-04-29 11:51:29 +01:00
- run the live browser OAuth flow and confirm Repixelizer receives the backend
callback, verifies the claim locally, adopts a local session, gates hosted
access, and stamps queued jobs with `account_id` / `session_id`
2026-04-29 12:41:43 +01:00
Deployment/configuration status:
- the deployment and service wiring are done
- public health/discovery/JWKS/config checks pass
2026-04-29 14:39:11 +01:00
- public Repixelizer auth-start returns Discord and Patreon authorization URLs
using the Heimdall callback and Repixelizer backend handoff
2026-04-29 12:41:43 +01:00
- the Discord authorization URL requests only `identify guilds.members.read`
2026-04-29 14:39:11 +01:00
- the Patreon authorization URL requests only `identity identity[email]`
- the remaining launch check is the real interactive provider browser flow
2026-04-29 12:41:43 +01:00
2026-04-29 11:51:29 +01:00
Current Discord policy:
- Heimdall grants Repixelizer `entitlement.app_access` only when the Discord
2026-04-29 13:22:01 +01:00
member has one of the role ids Repixelizer sent in its caller-owned
entitlement policy
- launch policy is intentionally role-gated: the GameCult Discord can remain
2026-04-29 13:07:44 +01:00
public, but Repixelizer access is limited to members with either `KLTST`
or the Patreon-synced `Inner Sanctum` role
2026-04-26 05:18:35 +01:00
2026-04-29 14:39:11 +01:00
Current Patreon policy:
- Heimdall grants Repixelizer `entitlement.app_access` only when Patreon returns
a currently entitled tier titled `Inner Sanctum`
- the policy is title-based, not amount-based
2026-04-26 11:41:27 +01:00
### 4. Migrate StreamPixels without flattening its good local seams
2026-04-26 05:18:35 +01:00
Goals:
2026-04-29 15:00:55 +01:00
- move provider/linking authority to Heimdall
- keep viewer profiles, creator memberships, connector bindings, diagnostics,
runtime behavior, and audience data
2026-04-26 05:18:35 +01:00
in StreamPixels
2026-04-26 11:41:27 +01:00
- keep creator/operator route authorization local to StreamPixels after claim
verification
2026-04-29 15:00:55 +01:00
Landed:
- Heimdall handles Twitch/YouTube OAuth exchange and identity resolution
- Heimdall stores managed provider tokens and exposes current access-token
resolution to app backends through an app shared secret
- StreamPixels delegates viewer claim/link and creator connector OAuth starts
to Heimdall
- StreamPixels redeems completion codes through its service, then owns local
viewer profile linking or creator connector binding
Still to verify:
- real browser StreamPixels viewer claim/link
- real creator connector attach for Twitch and YouTube
- runtime polling/subscription behavior using Heimdall-resolved credentials
2026-04-26 11:41:27 +01:00
### 5. Capture the Yggdrasil deployment shape early
Goals:
- treat Heimdall as a localhost service behind nginx on Yggdrasil
- support same-host callers first
- avoid per-request network chatter for routine app authorization
2026-04-26 05:18:35 +01:00
## Proposed phases
2026-04-26 11:41:27 +01:00
### Phase 1: Contract and schema scaffold
2026-04-26 12:25:11 +01:00
Landed in the first pass:
- signed claim/session payload shape
- Ed25519 JWKS verification surface
- OAuth `state` and callback contract surfaces
- app profile / capability rule inputs and outputs
Still to do in this phase:
- key rotation policy instead of just stable key loading
- token custody retrieval and refresh have landed (see `docs/architecture.md`, Managed credential custody); the open pieces are the follow-ups listed there
2026-04-26 11:41:27 +01:00
### Phase 2: Heimdall service skeleton
2026-04-26 05:18:35 +01:00
2026-04-26 12:25:11 +01:00
Landed in the first pass:
- provider OAuth start route scaffolding
- callback route scaffolding with signed-state validation
- session/claim issuance contract
- capability evaluation for shared app capabilities
Next in this phase:
- connection lifecycle and token-rotation work
2026-04-26 05:18:35 +01:00
Landed for the Ghostlight private-command consumer:
- exact persisted-session validation for refresh
- revision-bound logout revocation with stale-refresh rejection
2026-04-26 11:41:27 +01:00
### Phase 3: Repixelizer binding
2026-04-26 05:18:35 +01:00
2026-04-29 11:51:29 +01:00
Landed in Repixelizer:
- Heimdall hosted auth mode
- direct backend callback receipt
- local Ed25519 JWT verification against Heimdall JWKS
- httpOnly local session adoption
- hosted landing-page provider buttons
- hosted route and queue/job ownership checks
Still to do for this phase:
- deploy and configure both services
- verify the real Discord OAuth path end to end
2026-04-26 05:18:35 +01:00
2026-04-26 11:41:27 +01:00
### Phase 4: StreamPixels authority migration
2026-04-26 05:18:35 +01:00
2026-04-26 11:41:27 +01:00
- replace provider/linking authority paths with Heimdall calls where they truly
belong
- keep viewer/session expansion, creator memberships, and operator route
enforcement local where they are app-domain concerns
2026-04-26 05:18:35 +01:00
2026-04-26 11:41:27 +01:00
### Phase 5: Ops and deploy
2026-04-26 05:18:35 +01:00
2026-04-26 11:41:27 +01:00
- define nginx/systemd/env/runbook surfaces for Heimdall on Yggdrasil
- keep callers and state stores localhost-first where practical
2026-04-26 05:18:35 +01:00
## Guardrails
- do not let auth reuse become data mixing
2026-04-26 11:41:27 +01:00
- do not turn Heimdall into a mandatory round-trip for every guarded app route
- do not fantasize about one shared cross-runtime embedded library; the apps can
share contracts without sharing a runtime
2026-04-26 05:18:35 +01:00
- do not duplicate the shared architecture note into app profiles
2026-04-26 11:41:27 +01:00
- do not rip out app-local ownership and authorization logic that still belongs
in the app
2026-04-26 05:18:35 +01:00
- keep one clear hypothesis per pass
- commit completed work before the planning notes start breeding