diff --git a/docs/cultnet-transport-parity.md b/docs/cultnet-transport-parity.md index 92cb27e5..b1f0f099 100644 --- a/docs/cultnet-transport-parity.md +++ b/docs/cultnet-transport-parity.md @@ -350,7 +350,9 @@ Current progress: Verse role. The facade can now also create a connected `CultNetPeer` from a direct RUDP endpoint, peer card, or authorized peer lookup, so schema and shard catalog request helpers ride the RUDP transport without callers hand-wrapping - the socket transport. + the socket transport. The TypeScript interop peer now also serves and dials + the schema-v0 interop flow over `interop-rudp` while keeping TCP-framed as the + compatibility lane; discovery and hello responses advertise both profiles. - TypeScript now also has a `CultNetSchemaCatalog` for remote schema descriptors plus `CultNetPeer` request/fetch/sync helpers for `cultnet.schema_catalog_request.v0` / `cultnet.schema_catalog_response.v0`. diff --git a/docs/runtime-parity-scope.md b/docs/runtime-parity-scope.md index 74393f60..cb6cd67d 100644 --- a/docs/runtime-parity-scope.md +++ b/docs/runtime-parity-scope.md @@ -32,7 +32,7 @@ CultMesh. | --- | --- | --- | --- | | C# | Reference implementation and production game/server substrate | CultCache typed documents and SoA, CultNet schema-v0, LiteNetLib production path with adapter profile, shared transport profiles, shared RUDP socket transport, connected RUDP CultMesh client helpers, RUDP reconnect policy/controller and reconnect-loop helper, LiteNetLib client reconnect-controller adoption, schema-client transport port for service request/response clients, CultMesh node/session/catalog/stream/simulation/RUDP surfaces, authorized peer lookup | No need to abandon LiteNetLib before the CultNet adapter fully owns production service adoption | | Python | Package runtime, local daemon endpoint, raw interop peer, typed state participant | CultCache/CultNet/CultMesh package surface, branded CultMesh node/schema/shard/Verse/peer/authority/stream/RUDP ergonomics, authorized peer lookup, shard catalogs/logs, schema-transport factory for raw client request/subscription flows, local TCP and RUDP CultMesh server listeners, session/prediction/simulation fact helpers, RUDP codec/session/socket transport, RUDP reconnect policy/controller and reconnect-loop helper, schema-v0 MessagePack over RUDP evidence, benchmark evidence against C# cache paths | Universal C# throughput, full C# daemon/server stack, and RUDP as the only public service pipe | -| TypeScript | Node/browser-adjacent projection and interop runtime | CultCache parity tests, CultNet schema-v0 peer, advertised schema/shard catalog wire contracts plus request/sync helpers, transport profiles, TCP-framed transport port and peer helper, shared RUDP socket transport, RUDP reconnect policy/controller, RUDP reconnect-loop helper, branded CultMesh node/schema/shard/Verse/peer/authority/stream/RUDP ergonomics, authorized peer lookup | Full C# game session/server stack | +| TypeScript | Node/browser-adjacent projection and interop runtime | CultCache parity tests, CultNet schema-v0 peer, advertised schema/shard catalog wire contracts plus request/sync helpers, transport profiles, TCP-framed transport port and peer helper, shared RUDP socket transport, RUDP interop serve/dial path, RUDP reconnect policy/controller, RUDP reconnect-loop helper, branded CultMesh node/schema/shard/Verse/peer/authority/stream/RUDP ergonomics, authorized peer lookup | Full C# game session/server stack | | Rust | Low-level cache/transport/runtime substrate | CultCache/CultNet interop lane, schema and shard catalog helpers, transport profiles, TCP-framed port, shared RUDP codec/session/socket transport, connected RUDP CultMesh client helpers, RUDP reconnect policy/controller and reconnect-loop helper, schema-v0 MessagePack over RUDP evidence, small CultMesh facade for schema registry, shard catalog, peer cards, authority leases, authorized peer lookup, RUDP endpoint parsing, and peer-card/authorized-peer RUDP client construction | Full CultMesh game-session/server facade, stream catalog, or production game-session server | | Kotlin | JVM/Android client substrate | Typed MessagePack cache, built-in schema catalog advertisement, schema/shard catalogs with shared schema-message transport fetch/sync helpers, authority leases, authorized peer lookup, stream catalog, branded `CultMesh` facade, WebSocket transport profile/connection helper, RUDP client/server sugar, connected RUDP CultMesh client helpers, RUDP reconnect policy/controller and reconnect-loop helper, schema-v0 MessagePack over RUDP evidence | C# SoA store, full C# server daemon, or Unity hot-loop memory layout | diff --git a/packages/cultnet-ts/README.md b/packages/cultnet-ts/README.md index 2cb36fa7..5ff66e7e 100644 --- a/packages/cultnet-ts/README.md +++ b/packages/cultnet-ts/README.md @@ -254,6 +254,9 @@ maps them back to the exact C# union payload shape at the wire boundary. `createTcpFramedCultNetPeer(...)` wraps a Node `Duplex` in the shared `tcp_framed` profile before schema messages reach `CultNetPeer`. Passing a raw `Duplex` to `CultNetPeer` remains as compatibility for older callers. +- The interop peer still keeps the TCP-framed lane for compatibility, but its + serve and dial commands also exercise schema-v0 over the shared RUDP socket + transport and advertise both transport profiles. ## Development diff --git a/packages/cultnet-ts/test/interop/cultnet-interop-peer.ts b/packages/cultnet-ts/test/interop/cultnet-interop-peer.ts index 8d031744..0af43301 100644 --- a/packages/cultnet-ts/test/interop/cultnet-interop-peer.ts +++ b/packages/cultnet-ts/test/interop/cultnet-interop-peer.ts @@ -10,13 +10,22 @@ import { CultCache, SingleFileMessagePackBackingStore, defineDocumentType, type import { CultNetDocumentRegistry, CultNetPeer, + CultNetRudpPacket, + CultNetRudpSession, + CultNetRudpSocketTransportConnection, CultNetSchemaRegistry, TcpFramedTransportConnection, cultNetBuiltinSchemaRegistry, + createRudpTransportProfile, createTcpFramedTransportProfile, + decodeRudpPacket, defineCultNetDocumentBinding, + encodeCultNetMessageForWire, + encodeRudpPacket, + parseCultNetMessage, type CultNetDocumentBinding, type CultNetDocumentPutRawMessage, + type CultNetHelloMessage, type CultNetMessage, type CultNetSchemaCatalogRequestMessage, type CultNetSchemaCatalogResponseMessage, @@ -67,6 +76,9 @@ import { type InteropNote, } from "./cultnet-interop-shared"; +const RUDP_CONNECTION_ID = 0x43554c54; +const textEncoder = new TextEncoder(); + async function main(): Promise { const [mode, ...rest] = process.argv.slice(2); if (!mode) { @@ -98,12 +110,13 @@ async function serve(args: Map): Promise { const advertiseHost = requireArg(args, "advertise-host"); const bindHost = args.get("bind-host") ?? "127.0.0.1"; const tcpPort = optionalIntArg(args, "tcp-port"); + const rudpPort = optionalIntArg(args, "rudp-port", tcpPort ?? undefined); const discoveryPort = optionalIntArg(args, "discovery-port"); const discoveryGroup = requireArg(args, "discovery-group"); const schemaPath = requireArg(args, "schema-path"); - if (!tcpPort || !discoveryPort) { - throw new Error("serve mode requires --tcp-port and --discovery-port"); + if (!tcpPort || !rudpPort || !discoveryPort) { + throw new Error("serve mode requires --tcp-port, --rudp-port, and --discovery-port"); } const interopSchema = await loadInteropSchemaRegistration(schemaPath); @@ -156,6 +169,7 @@ async function serve(args: Map): Promise { agentId, advertiseHost, tcpPort, + rudpPort, cache, documentRegistry, customSchemas, @@ -185,12 +199,28 @@ async function serve(args: Map): Promise { agentId, advertiseHost, tcpPort, + rudpPort, }); } catch { // ignore unrelated multicast noise instead of becoming dramatic about it } }); + const rudpSocket = await startRudpInteropServer({ + runtimeId, + runtimeKind, + displayName, + agentId, + advertiseHost, + tcpPort, + rudpPort, + bindHost, + cache, + documentRegistry, + customSchemas, + noteSchemaId: interopSchema.schemaId, + }); + await Promise.all([ new Promise((resolve, reject) => { tcpServer.once("error", reject); @@ -213,18 +243,26 @@ async function serve(args: Map): Promise { runtimeId, runtimeKind, tcpPort, + rudpPort, discoveryPort, discoveryGroup, }); await waitForTermination(async () => { + rudpSocket.close(); udpSocket.close(); await closeServer(tcpServer); }); } +interface SchemaPeer { + send(message: CultNetMessage): void; + sendHello(message: CultNetHelloMessage): void; + sendSchemaCatalogResponse(message: CultNetSchemaCatalogResponseMessage): void; +} + async function handleServerMessage(input: { - peer: CultNetPeer; + peer: SchemaPeer; message: CultNetMessage; runtimeId: string; runtimeKind: string; @@ -232,6 +270,7 @@ async function handleServerMessage(input: { agentId: string; advertiseHost: string; tcpPort: number; + rudpPort: number; cache: CultCache; documentRegistry: CultNetDocumentRegistry; customSchemas: CultNetSchemaRegistry; @@ -246,6 +285,7 @@ async function handleServerMessage(input: { agentId, advertiseHost, tcpPort, + rudpPort, cache, documentRegistry, customSchemas, @@ -274,6 +314,10 @@ async function handleServerMessage(input: { transportId: "interop-tcp", host: advertiseHost, port: tcpPort, + }), createRudpTransportProfile(runtimeId, { + transportId: "interop-rudp", + host: advertiseHost, + port: rudpPort, })], supportsSchemaCatalog: true, }); @@ -301,7 +345,7 @@ async function handleServerMessage(input: { } async function handleRawPut(input: { - peer: CultNetPeer; + peer: Pick; message: CultNetDocumentPutRawMessage; runtimeId: string; agentId: string; @@ -369,6 +413,155 @@ async function handleRawPut(input: { } } +interface RudpInteropServerOptions { + runtimeId: string; + runtimeKind: string; + displayName: string; + agentId: string; + advertiseHost: string; + tcpPort: number; + rudpPort: number; + bindHost: string; + cache: CultCache; + documentRegistry: CultNetDocumentRegistry; + customSchemas: CultNetSchemaRegistry; + noteSchemaId: string; +} + +interface RudpInteropRemote { + session: CultNetRudpSession; +} + +async function startRudpInteropServer(options: RudpInteropServerOptions): Promise { + const socket = createSocket("udp4"); + const remotes = new Map(); + socket.on("error", (error) => writeLog("rudpError", { runtimeId: options.runtimeId, error: error.message })); + socket.on("message", (wire, remote) => { + void handleRudpDatagram(socket, remotes, options, wire, remote); + }); + + await new Promise((resolve, reject) => { + socket.once("error", reject); + socket.bind(options.rudpPort, options.bindHost, () => resolve()); + }); + + const resendTimer = setInterval(() => { + for (const [remoteKey, peer] of remotes) { + const [host, portValue] = remoteKey.split(":"); + const port = Number(portValue); + for (const packet of peer.session.dueResends(Date.now())) { + sendRudpPacket(socket, host ?? "", port, packet); + } + } + }, 25); + resendTimer.unref?.(); + socket.once("close", () => clearInterval(resendTimer)); + return socket; +} + +async function handleRudpDatagram( + socket: DgramSocket, + remotes: Map, + options: RudpInteropServerOptions, + wire: Buffer, + remote: RemoteInfo, +): Promise { + let packet: CultNetRudpPacket; + try { + packet = decodeRudpPacket(wire); + } catch { + return; + } + if (packet.connectionId !== RUDP_CONNECTION_ID) { + return; + } + + const remoteKey = `${remote.address}:${remote.port}`; + let peer = remotes.get(remoteKey); + if (packet.packetType === "connect") { + peer = { + session: new CultNetRudpSession({ + connectionId: RUDP_CONNECTION_ID, + resendDelayMs: 25, + }), + }; + remotes.set(remoteKey, peer); + sendRudpPacket(socket, remote.address, remote.port, peer.session.acceptConnect(packet, Date.now(), encodeText("cultnet-interop-rudp"))); + return; + } + if (!peer) { + return; + } + + try { + const result = peer.session.receive(packet, Date.now()); + if (result.reply) { + sendRudpPacket(socket, remote.address, remote.port, result.reply); + } + if (result.disconnected) { + remotes.delete(remoteKey); + return; + } + const schemaPeer = new RudpSchemaPeer(socket, remote, peer.session); + for (const frame of result.delivered) { + if (frame.channelId !== "schema") { + continue; + } + const message = parseCultNetMessage(decode(frame.payload), INTEROP_WIRE_CONTRACT); + await handleServerMessage({ + peer: schemaPeer, + message, + runtimeId: options.runtimeId, + runtimeKind: options.runtimeKind, + displayName: options.displayName, + agentId: options.agentId, + advertiseHost: options.advertiseHost, + tcpPort: options.tcpPort, + rudpPort: options.rudpPort, + cache: options.cache, + documentRegistry: options.documentRegistry, + customSchemas: options.customSchemas, + noteSchemaId: options.noteSchemaId, + }); + } + if (packet.packetType === "data" || result.delivered.length > 0) { + sendRudpPacket(socket, remote.address, remote.port, peer.session.createAck()); + } + } catch (error) { + writeLog("rudpMessageError", { + runtimeId: options.runtimeId, + error: error instanceof Error ? error.message : String(error), + }); + } +} + +class RudpSchemaPeer implements SchemaPeer { + constructor( + private readonly socket: DgramSocket, + private readonly remote: RemoteInfo, + private readonly session: CultNetRudpSession, + ) {} + + send(message: CultNetMessage): void { + const payload = encode(encodeCultNetMessageForWire(message, INTEROP_WIRE_CONTRACT)); + for (const packet of this.session.sendMany("schema", payload, { + reliable: true, + ordered: true, + nowMs: Date.now(), + })) { + sendRudpPacket(this.socket, this.remote.address, this.remote.port, packet); + } + } + + sendHello(message: CultNetHelloMessage): void { + this.send(message); + } + + sendSchemaCatalogResponse(message: CultNetSchemaCatalogResponseMessage): void { + this.send(message); + } +} + function createCatalogResponse( customSchemas: CultNetSchemaRegistry, request: CultNetSchemaCatalogRequestMessage, @@ -407,6 +600,7 @@ async function respondToProbe(input: { agentId: string; advertiseHost: string; tcpPort: number; + rudpPort: number; }): Promise { const { socket, @@ -418,6 +612,7 @@ async function respondToProbe(input: { agentId, advertiseHost, tcpPort, + rudpPort, } = input; const announceMessage: DiscoveryAnnounce = { @@ -435,6 +630,10 @@ async function respondToProbe(input: { transportId: "interop-tcp", host: advertiseHost, port: tcpPort, + }), createRudpTransportProfile(runtimeId, { + transportId: "interop-rudp", + host: advertiseHost, + port: rudpPort, })], supportsSchemaCatalog: true, }; @@ -519,11 +718,12 @@ async function dial(args: Map): Promise { const agentId = requireArg(args, "agent-id"); const targetHost = requireArg(args, "target-host"); const targetPort = optionalIntArg(args, "target-port"); + const targetRudpPort = optionalIntArg(args, "target-rudp-port"); const schemaPath = requireArg(args, "schema-path"); const timeoutMs = optionalIntArg(args, "timeout-ms", 4000) ?? 4000; - if (!targetPort) { - throw new Error("dial mode requires --target-port"); + if (!targetPort && !targetRudpPort) { + throw new Error("dial mode requires --target-port or --target-rudp-port"); } const interopSchema = await loadInteropSchemaRegistration(schemaPath); @@ -538,16 +738,9 @@ async function dial(args: Map): Promise { .build(); const documentRegistry = new CultNetDocumentRegistry(Object.values(documents)); - const socket = await connectTo(targetHost, targetPort); - const transport = new TcpFramedTransportConnection( - socket, - createTcpFramedTransportProfile(runtimeId, { - transportId: "interop-tcp", - host: targetHost, - port: targetPort, - }), - ); - const peer = new CultNetPeer(transport, { wireContract: INTEROP_WIRE_CONTRACT }); + const peer = targetRudpPort + ? await connectRudpPeer(runtimeId, targetHost, targetRudpPort, timeoutMs) + : await connectTcpPeer(runtimeId, targetHost, targetPort ?? raise("missing TCP target port")); writeTrace("dialSendHello", { runtimeId }); const remoteHelloWait = waitForMessage(peer, (message) => message.schemaVersion === "cultnet.hello.v0", timeoutMs); @@ -643,7 +836,8 @@ async function dial(args: Map): Promise { mode: "dial", runtimeId, targetHost, - targetPort, + targetPort: targetRudpPort ?? targetPort, + transport: targetRudpPort ? "rudp" : "tcp_framed", remoteHello, hasInteropSchema, retrievedNote: note, @@ -757,6 +951,64 @@ async function connectTo(host: string, port: number): Promise { }); } +async function connectTcpPeer(runtimeId: string, targetHost: string, targetPort: number): Promise { + const socket = await connectTo(targetHost, targetPort); + const transport = new TcpFramedTransportConnection( + socket, + createTcpFramedTransportProfile(runtimeId, { + transportId: "interop-tcp", + host: targetHost, + port: targetPort, + }), + ); + return new CultNetPeer(transport, { wireContract: INTEROP_WIRE_CONTRACT }); +} + +async function connectRudpPeer( + runtimeId: string, + targetHost: string, + targetPort: number, + timeoutMs: number, +): Promise { + const socket = createSocket("udp4"); + await new Promise((resolve, reject) => { + socket.once("error", reject); + socket.bind(0, "127.0.0.1", () => resolve()); + }); + const transport = new CultNetRudpSocketTransportConnection({ + runtimeId, + socket, + mode: "client", + remoteHost: targetHost, + remotePort: targetPort, + connectionId: RUDP_CONNECTION_ID, + transportId: "interop-rudp", + resendDelayMs: 25, + }); + transport.connect(encodeText("cultnet-interop-rudp")); + await waitUntil(() => transport.connected, timeoutMs, "RUDP interop dial handshake"); + return new CultNetPeer(transport, { wireContract: INTEROP_WIRE_CONTRACT }); +} + +async function waitUntil(predicate: () => boolean, timeoutMs: number, label: string): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (predicate()) { + return; + } + await delay(5); + } + throw new Error(`Timed out waiting for ${label}.`); +} + +function sendRudpPacket(socket: DgramSocket, host: string, port: number, packet: CultNetRudpPacket): void { + socket.send(encodeRudpPacket(packet), port, host); +} + +function encodeText(value: string): Uint8Array { + return new Uint8Array(textEncoder.encode(value)); +} + async function closeServer(server: Server): Promise { await new Promise((resolve, reject) => { server.close((error) => { diff --git a/packages/cultnet-ts/test/interop/cultnet-interop.test.ts b/packages/cultnet-ts/test/interop/cultnet-interop.test.ts index 4ac205d4..3f680d36 100644 --- a/packages/cultnet-ts/test/interop/cultnet-interop.test.ts +++ b/packages/cultnet-ts/test/interop/cultnet-interop.test.ts @@ -398,6 +398,7 @@ test("CultNet TS/Rust/C#/Python peers discover each other and exchange raw state "rust-client-dial", "ts-peer", "ts-client-dial", + "ts-rudp-client-dial", "ts-rust-dial", "ts-csharp-dial", "python-peer", @@ -519,6 +520,25 @@ test("CultNet TS/Rust/C#/Python peers discover each other and exchange raw state assert.equal(tsDial.mutationReceipt.accepted, true); assert.equal(tsDial.fireReceipt.shotsFired, 1); + logInteropPhase("schema-v0", "typescript dials typescript over rudp"); + const tsRudpDial = await runJsonCommand("ts-rudp-dial", process.execPath, [ + tsPeerScript, + "dial", + "--runtime-id", "ts-rudp-client", + "--runtime-kind", "node", + "--display-name", "TS RUDP Dialer", + "--agent-id", "ts-rudp-client-agent", + "--target-host", "127.0.0.1", + "--target-rudp-port", String(tsPort), + "--schema-path", interopSchemaPath, + ], cultNetTsRoot); + assert.equal(tsRudpDial.transport, "rudp"); + assert.equal(tsRudpDial.remoteHello.runtimeId, "ts-peer"); + assert.equal(tsRudpDial.hasInteropSchema, true); + assert.equal(tsRudpDial.retrievedNote.authorRuntimeId, "ts-peer"); + assert.ok(tsRudpDial.mutatedNote.tags.includes("decorated:ts-rudp-client")); + assert.equal(tsRudpDial.fireReceipt.accepted, true); + logInteropPhase("schema-v0", "rust dials csharp"); const rustDial = await runJsonCommand("rust-dial", rustBinaryPath, [ "dial",