Add replay-backed retained session checkpoints

This commit is contained in:
Meta Vens 2026-07-15 08:08:02 +01:00
commit 949f98af52
15 changed files with 707 additions and 43 deletions

View file

@ -9,10 +9,10 @@ algorithms usable as durable, typed GameCult world truth. Unity, renderers, and
game products consume Ymir state and contact facts; they do not become physics
authorities.
The Box3D parity facade currently witnesses a bounded released-behavior slice.
The production command path still runs the transitional managed solver and is
not Box3D-authoritative until one retained Box3D session embedded in the
Aetheria daemon owns every step, query, and contact fact.
The production runtime path is a retained Box3D session. The parity facade
witnesses released behavior at the native boundary; Ymir adds stable ids,
typed commands and facts, replay memory, and deterministic projection without
installing a second solver.
## Target Ownership
@ -84,8 +84,9 @@ and typed Begin/Hit/End facts. It never treats omission as removal. Aetheria
embeds one session per run and zone; its old process-wide simulator objects
are not session identifiers. Revision-checked
retained-session circle overlap and cast queries observe the current stable-id
body projection without mutating it. Checkpoint reconstruction and non-Windows
RID artifacts remain cutover work.
body projection without mutating it. Replay-backed checkpoint reconstruction is
implemented; host-owned incremental journal persistence and non-Windows RID
artifacts remain cutover work.
See [the architecture map](docs/architecture.md) and
[the executable parity contract](docs/box3d-parity.md). The
@ -117,7 +118,8 @@ smoke is still required before that is a repeatable packaging guarantee. Linux
and macOS packages must be produced and smoked on their own release workers
before those RIDs are advertised.
The current package version is `0.3.0`. ABI v4 exposes Box3D body type,
The current package version is `0.3.0`. ABI v5 exposes the exact pinned native
build identity plus Box3D body type,
bullet, field-participation, and collision-filter configuration without
exposing Box3D handles. These are Ymir contract fields, but their behavior is
Box3D behavior.

View file

@ -56,11 +56,13 @@ step, and disposal operations. It emits revisioned receipts and typed
Begin/Hit/End facts with stable Ymir identities. Ordinary retained steps do not
accept complete body snapshots.
The current CLI still has no named session registry, CultMesh command
lowerings, durable receipt ledger, or reconstruction
checkpoint. Its CultCache and Eve output is a regenerated diagnostic
projection. The sections marked as target contract describe that daemon cut;
they are not advertisements of current CLI capability.
The embedded retained-session library now has a replay-backed reconstruction
checkpoint and durable-in-checkpoint command receipt ledger. Host-owned private
incremental journal storage and atomic frame/resume commits remain integration
work. Complete replay history is not product state and must not ride in public
client frames. The current CLI still has no named session registry or CultMesh
command lowerings; its CultCache and Eve output remains a regenerated
diagnostic projection.
## Target Inputs
@ -175,6 +177,17 @@ lifecycle identity are binary contracts, not tolerance checks. Authoritative
contact facts resume only after canonical history has been replayed through the
checkpoint tick; a checkpoint alone cannot emit gameplay contact facts.
`gamecult.ymir.session_checkpoint.replay.v1` implements that promise. It stores
the initial definitions, complete typed command journal, logical session
generation, command digest, body-generation counter, world verification
projection, active contact-episode verification records, and pinned runtime
fingerprint. Restore replays into a new Box3D world with outward publication
suppressed, then compares the reconstructed boundary exactly before returning a
live session. The bounded binary codec rejects incompatible, corrupt, or
trailing data and checksums the complete encoded payload. ABI v5 validates the
exact pinned native build id, not merely Box3D's public version. Native Box3D
identities never enter the checkpoint.
`gamecult.ymir.world_state.v2` appends Box3D body type, bullet,
field-participation, and collision filter arrays to the v1 layout. The v1 and
v0 readers are explicit migration paths; keys are not reassigned. Torque is
@ -200,7 +213,8 @@ Box3D-backed sessions prove:
3. overlap and shape-cast lowering;
4. continuous collision event timing and same-step Box3D cast witnesses;
5. transient force and torque application;
6. checkpoint reconstruction and replay;
6. checkpoint reconstruction and replay (implemented; journal compaction and
long-running storage remain operational work);
7. Aetheria gravity, tractor, pickup, and payload smokes.
There is no managed fallback after that cut. A native load or compatibility
@ -213,9 +227,9 @@ failure fails closed and reports the missing capability.
session facts.
- Negative source checks prove custom solver and spatial-index authorities are
absent after cutover.
- Aetheria smokes must prove gravity, contact-gated destruction loot, tractor
attraction, pickup collection, and fast payload hits through the retained
contract. Current gravity, pickup-contact, cast, and overlap witnesses cover
only the snapshot/query compatibility path.
- Aetheria smokes prove gravity, tractor attraction, contact-gated pickup
collection/rejection, physical payload hits, and restart continuation through
the retained contract. Destruction loot remains Aetheria gameplay state
gated by Ymir contact facts where collection occurs.
- Released-client witnesses prove generic Eve clients only observe Ymir and
Aetheria provider contracts, never native implementation details.

View file

@ -21,16 +21,16 @@ the overlap, cast, pair-step, torque-lifetime, version, and ABI-layout
operations used by the harness. This keeps Box3D's alpha C ABI out of Ymir's
public managed contracts.
Production ABI version 4 uses caller-owned blittable buffers, explicit cdecl,
Production ABI version 5 uses caller-owned blittable buffers, explicit cdecl,
fixed-width status/count values, compile-time C layout assertions, and a
managed/native layout sentinel before opening a session. It adds explicit
managed/native layout and pinned-build-identity sentinel before opening a session. It adds explicit
retained body mutations, one-shot force and torque operations, and a
session-owned Begin/Hit/End event buffer. The private buffer retains Box3D's
stored contact key only long enough to correlate its separate event arrays;
public Ymir facts never expose native contact identity. Planar direction maps to Box3D
rotation about Y; angular velocity and transient torque remain Box3D state and
commands rather than managed integration.
ABI v4 additionally carries Box3D static/kinematic/dynamic selection, bullet
ABI v5 additionally carries Box3D static/kinematic/dynamic selection, bullet
CCD, category/mask/group filtering, and per-body radial-field participation.
`tests/Ymir.Box3D.Parity` builds that facade through CMake and invokes it with

View file

@ -85,9 +85,21 @@ chronology.
- Aetheria cargo, damage, loot, feedback, and scoring remain gameplay state
derived from Ymir facts.
The current in-process receipt ledger is not a durable reconnect ledger. It is
kept for the lifetime of the session and is intentionally not advertised as a
reconnect or checkpoint guarantee.
The replay checkpoint preserves the logical session generation, complete typed
command journal, receipt ledger, body-generation counter, world projection, and
active contact episodes. Restore creates a fresh native Box3D world and replays
that journal through the checkpoint boundary before facts may leave the
session. The world and episode records verify replay; they do not replace it or
infer contact continuity from body proximity.
`gamecult.ymir.session_checkpoint.replay.v1` is encoded by the bounded binary
`YmirSessionCheckpointCodec`. The checkpoint carries the pinned Box3D/ABI/
precision fingerprint and fails closed on incompatible provenance, corrupt or
trailing data, journal divergence, world divergence, or contact-lineage
divergence. A SHA-256 checksum binds session identity, provenance, journal,
world verifier, and active episodes. ABI v5 independently validates the exact
pinned native build id before opening a session. It contains no native Box3D
handle or contact key.
## Forbidden Writers
@ -102,10 +114,9 @@ reconnect or checkpoint guarantee.
## Shared Paths
Direct commands, agent commands, and spawns use the same explicit in-process
session operations. Imports, reconnects, and replay must lower through those
operations after the owning product's registry and reconstruction contract exist.
Restore will establish initial state; it will not remain a parallel writer.
Direct commands, agent commands, spawns, and checkpoint replay use the same
explicit session operations. Restore establishes a fresh native world through
those operations; it is not a parallel state writer.
Retained queries will observe a named revision without mutating it. Snapshot
reads cannot be fed back without an explicit restore, teleport, velocity, or
configuration operation.
@ -133,10 +144,11 @@ is derived process state: it never appears in a public fact or checkpoint.
2. Public managed retained session with no ordinary full synchronization:
done.
3. Revision-checked retained-session overlap and cast queries: done.
4. Embed one retained session per Aetheria run and zone, then delete its process-wide
simulator identity, manual tractor geometry, per-projectile mini-worlds,
and tuple contact dedup.
5. For optional standalone hosting, add a session registry, generation-bearing
4. One embedded retained world and physical-payload session per Aetheria run
and zone: done.
5. Persist replay journals privately and incrementally behind a bounded resume
descriptor; complete history must not ride in public product frames.
6. For optional standalone hosting, add a session registry, generation-bearing
routed commands, idempotent Create, durable receipts, and CultMesh lowering.
The isolated `YmirSimulator` remains a compatibility API. It is never the
@ -160,5 +172,5 @@ End fact.
- different insertion orders produce the same semantic fact order.
- queries observe the requested retained revision and cannot inject bodies.
- disposed or stale session generations fail closed.
- restored continuation must reproduce bodies and fact identity before Ymir
advertises retained-session checkpoint support.
- restored continuation reproduces command receipts, bodies, active contact
identity, and subsequent Begin/End facts before authoritative publication.

View file

@ -1206,7 +1206,12 @@ YMIR_EXPORT ymir_box3d_torque_result YMIR_CALL ymir_box3d_torque_lifetime(float
YMIR_EXPORT uint32_t YMIR_CALL ymir_box3d_get_abi_version(void)
{
return 4;
return 5;
}
YMIR_EXPORT const char* YMIR_CALL ymir_box3d_get_build_id(void)
{
return "box3d-8441b4a06d6d09dcfb0b0f704df4d847d1437b92-abi5-f32";
}
YMIR_EXPORT int32_t YMIR_CALL ymir_box3d_get_abi_layout(

View file

@ -22,6 +22,10 @@ internal static partial class Box3DNative
[UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]
internal static partial uint ymir_box3d_get_abi_version();
[LibraryImport(LibraryName)]
[UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]
internal static partial nint ymir_box3d_get_build_id();
[LibraryImport(LibraryName)]
[UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]
internal static partial Box3DStatus ymir_box3d_get_abi_layout(

View file

@ -0,0 +1,16 @@
namespace Ymir.Box3D;
public static class Box3DRuntime
{
public const string NativeBuildId =
"box3d-8441b4a06d6d09dcfb0b0f704df4d847d1437b92-abi5-f32";
public static string ValidatedBuildId
{
get
{
Box3DSession.EnsureCompatibleNativeLibrary();
return NativeBuildId;
}
}
}

View file

@ -69,7 +69,7 @@ internal readonly record struct Box3DContactFact(
internal sealed class Box3DSession : IDisposable
{
internal const uint AbiVersion = 4;
internal const uint AbiVersion = 5;
internal const int DefaultSubstepCount = 4;
private readonly Box3DSessionHandle _handle;
@ -331,6 +331,13 @@ internal sealed class Box3DSession : IDisposable
throw new InvalidOperationException($"Ymir Box3D ABI mismatch. Managed expects {AbiVersion}, native provides {abi}.");
}
var buildId = Marshal.PtrToStringAnsi(Box3DNative.ymir_box3d_get_build_id());
if (!string.Equals(buildId, Box3DRuntime.NativeBuildId, StringComparison.Ordinal))
{
throw new InvalidOperationException(
$"Ymir Box3D build mismatch. Managed expects '{Box3DRuntime.NativeBuildId}', native provides '{buildId ?? "<null>"}'.");
}
ThrowIfFailed(
Box3DNative.ymir_box3d_get_abi_layout(
out var bodyInputSize,

View file

@ -1,3 +1,5 @@
using Ymir.Box3D;
namespace Ymir.Core;
public sealed record YmirSessionCreateRequest(
@ -5,6 +7,48 @@ public sealed record YmirSessionCreateRequest(
IReadOnlyList<PhysicsBody> InitialBodies,
float InitialTime = 0.0f);
public static class YmirSessionCheckpointContract
{
public const string FormatId = "gamecult.ymir.session_checkpoint.replay.v1";
public static string RuntimeFingerprint =>
$"gamecult.ymir.replay.v1|{Box3DRuntime.ValidatedBuildId}";
}
public sealed record YmirSessionCheckpoint(
string FormatId,
string RuntimeFingerprint,
string SessionId,
string SessionGeneration,
long Revision,
long StepIndex,
float Time,
long NextBodyGeneration,
IReadOnlyList<PhysicsBody> InitialBodies,
float InitialTime,
IReadOnlyList<YmirSessionJournalEntry> Journal,
string JournalDigest,
YmirWorld World,
IReadOnlyList<YmirContactEpisodeCheckpoint> ActiveContactEpisodes);
public sealed record YmirContactEpisodeCheckpoint(
string BodyA,
long BodyAGeneration,
string BodyB,
long BodyBGeneration,
long StartStepIndex,
long Ordinal);
public abstract record YmirSessionJournalEntry;
public sealed record YmirSpawnBodyJournalEntry(YmirSpawnBodyCommand Command) : YmirSessionJournalEntry;
public sealed record YmirRemoveBodyJournalEntry(YmirRemoveBodyCommand Command) : YmirSessionJournalEntry;
public sealed record YmirTeleportBodyJournalEntry(YmirTeleportBodyCommand Command) : YmirSessionJournalEntry;
public sealed record YmirSetBodyVelocityJournalEntry(YmirSetBodyVelocityCommand Command) : YmirSessionJournalEntry;
public sealed record YmirConfigureBodyJournalEntry(YmirConfigureBodyCommand Command) : YmirSessionJournalEntry;
public sealed record YmirApplyForceJournalEntry(YmirApplyForceCommand Command) : YmirSessionJournalEntry;
public sealed record YmirApplyTorqueJournalEntry(YmirApplyTorqueCommand Command) : YmirSessionJournalEntry;
public sealed record YmirStepSessionJournalEntry(YmirStepSessionCommand Command) : YmirSessionJournalEntry;
public sealed record YmirCommandHeader(string CommandId, long ExpectedRevision);
public sealed record YmirSpawnBodyCommand(YmirCommandHeader Header, PhysicsBody Body);

View file

@ -53,7 +53,7 @@ public sealed class YmirOperatorStateDocument
public string BatchKernel { get; set; } = "";
[Key(5)]
public string Persistence { get; set; } = "CultCache gamecult.ymir.world_state.v2; explicit v1/v0 read migration";
public string Persistence { get; set; } = "Checksummed replay checkpoint v1; host-owned private journal persistence; world_state.v2 inspection snapshot";
[Key(6)]
public string UpdatedAtUtc { get; set; } = "";
@ -90,13 +90,14 @@ public static class YmirServicePublication
"stable GameCult id to transient Box3D handle projection",
"Box3D-backed circle overlap and cast queries over submitted bodies",
"deterministic result ordering",
"checksummed replay checkpoint reconstruction bound to the pinned native build",
"CultCache world snapshot v2 with explicit v1/v0 read migration",
"diagnostic Eve publication"
],
DoesNotOwn =
[
"daemon-owned named session registry or CultMesh command lowering yet",
"complete checkpoint reconstruction yet",
"host-owned durable journal storage or atomic frame persistence",
"physics algorithm invention",
"Box3D solver and collision semantics",
"CultMesh provider sessions or discovery during cutover",
@ -116,7 +117,7 @@ public static class YmirServicePublication
StateOwner = "In-process retained Box3D sessions, isolated snapshot steps, and diagnostic projections",
NumericSubstrate = "Box3D v0.1.0 (C17)",
BatchKernel = "Box3D native solver; Ymir does not own physics algorithms",
Persistence = "CultCache gamecult.ymir.world_state.v2; explicit v1/v0 read migration",
Persistence = "Checksummed replay checkpoint v1; host-owned private journal persistence; world_state.v2 inspection snapshot",
UpdatedAtUtc = FormatUtc(now)
};

View file

@ -19,6 +19,9 @@ public sealed class YmirSession : IDisposable
private readonly Dictionary<string, PhysicsBody> _bodies = new(StringComparer.Ordinal);
private readonly Dictionary<Box3DContactKey, ActiveEpisode> _episodesByNativeContact = new();
private readonly Dictionary<string, LedgerEntry> _ledger = new(StringComparer.Ordinal);
private readonly List<YmirSessionJournalEntry> _journal = [];
private readonly PhysicsBody[] _initialBodies;
private readonly float _initialTime;
private readonly string _sessionId;
private readonly string _sessionGeneration;
private ulong _nextNativeKey = 1;
@ -30,7 +33,7 @@ public sealed class YmirSession : IDisposable
private bool _disposed;
private bool _faulted;
private YmirSession(YmirSessionCreateRequest request)
private YmirSession(YmirSessionCreateRequest request, string? restoredSessionGeneration = null)
{
ArgumentNullException.ThrowIfNull(request);
if (string.IsNullOrWhiteSpace(request.SessionId))
@ -43,14 +46,18 @@ public sealed class YmirSession : IDisposable
}
_sessionId = request.SessionId;
_sessionGeneration = Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture);
_sessionGeneration = string.IsNullOrWhiteSpace(restoredSessionGeneration)
? Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture)
: restoredSessionGeneration;
_time = request.InitialTime;
_initialTime = request.InitialTime;
var initial = request.InitialBodies ?? throw new ArgumentNullException(nameof(request.InitialBodies));
ValidateUniqueBodyIds(initial);
_initialBodies = initial.OrderBy(body => body.Id, StringComparer.Ordinal).ToArray();
try
{
foreach (var body in initial.OrderBy(body => body.Id, StringComparer.Ordinal))
foreach (var body in _initialBodies)
{
ValidateBody(body, allowTorque: false);
SpawnCore(body);
@ -66,6 +73,26 @@ public sealed class YmirSession : IDisposable
public static YmirSession Create(YmirSessionCreateRequest request) => new(request);
public static YmirSession Restore(YmirSessionCheckpoint checkpoint)
{
ValidateCheckpointHeader(checkpoint);
var session = new YmirSession(
new YmirSessionCreateRequest(checkpoint.SessionId, checkpoint.InitialBodies, checkpoint.InitialTime),
checkpoint.SessionGeneration);
try
{
foreach (var entry in checkpoint.Journal)
session.Replay(entry);
session.VerifyCheckpoint(checkpoint);
return session;
}
catch
{
session.Dispose();
throw;
}
}
public YmirSessionInfo Info
{
get
@ -78,12 +105,48 @@ public sealed class YmirSession : IDisposable
}
}
public YmirSessionCheckpoint Checkpoint()
{
lock (_gate)
{
ObjectDisposedException.ThrowIf(_disposed, this);
if (_faulted)
throw new InvalidOperationException("A faulted Ymir session cannot publish a checkpoint.");
var activeEpisodes = _episodesByNativeContact.Values
.Distinct()
.Select(EpisodeCheckpoint)
.OrderBy(value => value.BodyA, StringComparer.Ordinal)
.ThenBy(value => value.BodyAGeneration)
.ThenBy(value => value.BodyB, StringComparer.Ordinal)
.ThenBy(value => value.BodyBGeneration)
.ThenBy(value => value.StartStepIndex)
.ThenBy(value => value.Ordinal)
.ToArray();
return new YmirSessionCheckpoint(
YmirSessionCheckpointContract.FormatId,
YmirSessionCheckpointContract.RuntimeFingerprint,
_sessionId,
_sessionGeneration,
_revision,
_stepIndex,
_time,
_nextBodyGeneration,
_initialBodies.ToArray(),
_initialTime,
_journal.ToArray(),
JournalDigest(),
SnapshotCore(),
activeEpisodes);
}
}
public YmirCommandReceipt Spawn(YmirSpawnBodyCommand command)
{
ArgumentNullException.ThrowIfNull(command);
lock (_gate)
{
var fingerprint = Fingerprint(command.Header, "body.spawn", writer => WriteBody(writer, command.Body));
RecordCommand(command.Header, new YmirSpawnBodyJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.spawn") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.spawn", fingerprint);
@ -108,6 +171,7 @@ public sealed class YmirSession : IDisposable
lock (_gate)
{
var fingerprint = Fingerprint(command.Header, "body.remove", writer => WriteString(writer, command.BodyId));
RecordCommand(command.Header, new YmirRemoveBodyJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.remove") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.remove", fingerprint);
@ -139,6 +203,7 @@ public sealed class YmirSession : IDisposable
Write(writer, command.Position);
Write(writer, command.Direction);
});
RecordCommand(command.Header, new YmirTeleportBodyJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.teleport") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.teleport", fingerprint);
@ -174,6 +239,7 @@ public sealed class YmirSession : IDisposable
Write(writer, command.LinearVelocity);
writer.Write(command.AngularVelocity);
});
RecordCommand(command.Header, new YmirSetBodyVelocityJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.set_velocity") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.set_velocity", fingerprint);
@ -219,6 +285,7 @@ public sealed class YmirSession : IDisposable
writer.Write(command.Restitution);
writer.Write(command.IsKinematic);
});
RecordCommand(command.Header, new YmirConfigureBodyJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.configure") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.configure", fingerprint);
@ -268,6 +335,7 @@ public sealed class YmirSession : IDisposable
WriteString(writer, command.BodyId);
Write(writer, command.Force);
});
RecordCommand(command.Header, new YmirApplyForceJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.apply_force") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.apply_force", fingerprint);
@ -300,6 +368,7 @@ public sealed class YmirSession : IDisposable
WriteString(writer, command.BodyId);
writer.Write(command.Torque);
});
RecordCommand(command.Header, new YmirApplyTorqueJournalEntry(command));
if (Preflight(command.Header, fingerprint, "body.apply_torque") is { } previous)
{
return ReceiptFrom(previous, command.Header, "body.apply_torque", fingerprint);
@ -348,6 +417,7 @@ public sealed class YmirSession : IDisposable
WriteField(writer, field);
}
});
RecordCommand(command.Header, new YmirStepSessionJournalEntry(command));
if (Preflight(command.Header, fingerprint, "session.step") is { } previous)
{
if (previous.Result is YmirSessionStepResult replay)
@ -818,6 +888,113 @@ public sealed class YmirSession : IDisposable
_bodies.Values.OrderBy(body => body.Id, StringComparer.Ordinal).ToArray(),
_fields.ToArray());
private void RecordCommand(YmirCommandHeader header, YmirSessionJournalEntry entry)
{
if (!_ledger.ContainsKey(header.CommandId))
_journal.Add(entry);
}
private void Replay(YmirSessionJournalEntry entry)
{
switch (entry)
{
case YmirSpawnBodyJournalEntry value: Spawn(value.Command); break;
case YmirRemoveBodyJournalEntry value: Remove(value.Command); break;
case YmirTeleportBodyJournalEntry value: Teleport(value.Command); break;
case YmirSetBodyVelocityJournalEntry value: SetVelocity(value.Command); break;
case YmirConfigureBodyJournalEntry value: Configure(value.Command); break;
case YmirApplyForceJournalEntry value: ApplyForce(value.Command); break;
case YmirApplyTorqueJournalEntry value: ApplyTorque(value.Command); break;
case YmirStepSessionJournalEntry value: Step(value.Command); break;
default: throw new InvalidOperationException($"Unknown Ymir journal entry '{entry?.GetType().FullName ?? "<null>"}'.");
}
}
private void VerifyCheckpoint(YmirSessionCheckpoint checkpoint)
{
var world = SnapshotCore();
var active = _episodesByNativeContact.Values
.Distinct()
.Select(EpisodeCheckpoint)
.OrderBy(value => value.BodyA, StringComparer.Ordinal)
.ThenBy(value => value.BodyAGeneration)
.ThenBy(value => value.BodyB, StringComparer.Ordinal)
.ThenBy(value => value.BodyBGeneration)
.ThenBy(value => value.StartStepIndex)
.ThenBy(value => value.Ordinal)
.ToArray();
var expectedBodies = (checkpoint.World.Bodies ?? Array.Empty<PhysicsBody>())
.OrderBy(value => value.Id, StringComparer.Ordinal).ToArray();
var expectedFields = checkpoint.World.Fields ?? Array.Empty<RadialField>();
if (_revision != checkpoint.Revision || _stepIndex != checkpoint.StepIndex ||
_time != checkpoint.Time || _nextBodyGeneration != checkpoint.NextBodyGeneration ||
world.Time != checkpoint.World.Time || !world.Bodies.SequenceEqual(expectedBodies) ||
!world.Fields.SequenceEqual(expectedFields) ||
!active.SequenceEqual(checkpoint.ActiveContactEpisodes) ||
!string.Equals(JournalDigest(), checkpoint.JournalDigest, StringComparison.Ordinal))
{
throw new InvalidOperationException(
$"Ymir checkpoint replay diverged for session '{checkpoint.SessionId}'.");
}
}
private string JournalDigest()
{
using var stream = new MemoryStream();
using (var writer = new BinaryWriter(stream, Encoding.UTF8, leaveOpen: true))
{
writer.Write(1);
writer.Write(_journal.Count);
foreach (var entry in _journal)
{
var header = JournalHeader(entry);
WriteString(writer, entry.GetType().Name);
WriteString(writer, header.CommandId);
if (!_ledger.TryGetValue(header.CommandId, out var ledger))
throw new InvalidOperationException($"Ymir journal command '{header.CommandId}' has no ledger result.");
WriteString(writer, ledger.Fingerprint);
}
}
return Convert.ToHexString(SHA256.HashData(stream.GetBuffer().AsSpan(0, checked((int)stream.Length))))
.ToLowerInvariant();
}
private static YmirCommandHeader JournalHeader(YmirSessionJournalEntry entry) => entry switch
{
YmirSpawnBodyJournalEntry value => value.Command.Header,
YmirRemoveBodyJournalEntry value => value.Command.Header,
YmirTeleportBodyJournalEntry value => value.Command.Header,
YmirSetBodyVelocityJournalEntry value => value.Command.Header,
YmirConfigureBodyJournalEntry value => value.Command.Header,
YmirApplyForceJournalEntry value => value.Command.Header,
YmirApplyTorqueJournalEntry value => value.Command.Header,
YmirStepSessionJournalEntry value => value.Command.Header,
_ => throw new InvalidOperationException($"Unknown Ymir journal entry '{entry?.GetType().FullName ?? "<null>"}'.")
};
private static YmirContactEpisodeCheckpoint EpisodeCheckpoint(ActiveEpisode episode) => new(
episode.Pair.BodyA,
episode.Pair.BodyAGeneration,
episode.Pair.BodyB,
episode.Pair.BodyBGeneration,
episode.StartStepIndex,
episode.Ordinal);
private static void ValidateCheckpointHeader(YmirSessionCheckpoint checkpoint)
{
ArgumentNullException.ThrowIfNull(checkpoint);
if (!string.Equals(checkpoint.FormatId, YmirSessionCheckpointContract.FormatId, StringComparison.Ordinal) ||
!string.Equals(checkpoint.RuntimeFingerprint, YmirSessionCheckpointContract.RuntimeFingerprint, StringComparison.Ordinal))
throw new InvalidOperationException("Ymir checkpoint format or runtime provenance is incompatible.");
if (string.IsNullOrWhiteSpace(checkpoint.SessionId) || string.IsNullOrWhiteSpace(checkpoint.SessionGeneration) ||
checkpoint.Revision < 0 || checkpoint.StepIndex < 0 || checkpoint.NextBodyGeneration < 1 ||
!float.IsFinite(checkpoint.Time) || checkpoint.Time < 0 ||
!float.IsFinite(checkpoint.InitialTime) || checkpoint.InitialTime < 0 ||
checkpoint.InitialBodies == null || checkpoint.Journal == null || checkpoint.World == null ||
checkpoint.ActiveContactEpisodes == null || string.IsNullOrWhiteSpace(checkpoint.JournalDigest))
throw new InvalidOperationException("Ymir checkpoint header is incomplete or invalid.");
}
private bool TryBody(string? id, out ulong key, out PhysicsBody body)
{
if (!string.IsNullOrWhiteSpace(id) &&

View file

@ -0,0 +1,317 @@
using System.Security.Cryptography;
using System.Text;
namespace Ymir.Core;
public static class YmirSessionCheckpointCodec
{
private const uint Magic = 0x43534D59; // YMSC
private const int Version = 1;
private const int MaximumPayloadBytes = 256 * 1024 * 1024;
private const int MaximumCollectionCount = 4 * 1024 * 1024;
private const int MaximumStringBytes = 1024 * 1024;
private const int ChecksumBytes = 32;
public static byte[] Encode(YmirSessionCheckpoint checkpoint)
{
ArgumentNullException.ThrowIfNull(checkpoint);
using var stream = new MemoryStream();
using var writer = new BinaryWriter(stream, Encoding.UTF8, leaveOpen: true);
writer.Write(Magic);
writer.Write(Version);
WriteString(writer, checkpoint.FormatId);
WriteString(writer, checkpoint.RuntimeFingerprint);
WriteString(writer, checkpoint.SessionId);
WriteString(writer, checkpoint.SessionGeneration);
writer.Write(checkpoint.Revision);
writer.Write(checkpoint.StepIndex);
writer.Write(checkpoint.Time);
writer.Write(checkpoint.NextBodyGeneration);
WriteBodies(writer, checkpoint.InitialBodies);
writer.Write(checkpoint.InitialTime);
WriteJournal(writer, checkpoint.Journal);
WriteString(writer, checkpoint.JournalDigest);
WriteWorld(writer, checkpoint.World);
WriteEpisodes(writer, checkpoint.ActiveContactEpisodes);
writer.Flush();
if (stream.Length > MaximumPayloadBytes - ChecksumBytes)
throw new InvalidOperationException("Ymir checkpoint exceeds the supported payload size.");
var content = stream.ToArray();
var checksum = SHA256.HashData(content);
return [.. content, .. checksum];
}
public static YmirSessionCheckpoint Decode(ReadOnlySpan<byte> payload)
{
if (payload.Length <= ChecksumBytes || payload.Length > MaximumPayloadBytes)
throw new InvalidDataException("Ymir checkpoint payload size is invalid.");
var content = payload[..^ChecksumBytes];
var expectedChecksum = payload[^ChecksumBytes..];
Span<byte> actualChecksum = stackalloc byte[ChecksumBytes];
SHA256.HashData(content, actualChecksum);
if (!CryptographicOperations.FixedTimeEquals(actualChecksum, expectedChecksum))
throw new InvalidDataException("Ymir checkpoint checksum is invalid.");
try
{
using var stream = new MemoryStream(content.ToArray(), writable: false);
using var reader = new BinaryReader(stream, Encoding.UTF8, leaveOpen: true);
if (reader.ReadUInt32() != Magic || reader.ReadInt32() != Version)
throw new InvalidDataException("Ymir checkpoint codec header is incompatible.");
var checkpoint = new YmirSessionCheckpoint(
ReadString(reader),
ReadString(reader),
ReadString(reader),
ReadString(reader),
reader.ReadInt64(),
reader.ReadInt64(),
reader.ReadSingle(),
reader.ReadInt64(),
ReadBodies(reader),
reader.ReadSingle(),
ReadJournal(reader),
ReadString(reader),
ReadWorld(reader),
ReadEpisodes(reader));
if (stream.Position != stream.Length)
throw new InvalidDataException("Ymir checkpoint contains trailing data.");
return checkpoint;
}
catch (InvalidDataException)
{
throw;
}
catch (Exception error) when (error is EndOfStreamException or IOException or ArgumentException or OverflowException)
{
throw new InvalidDataException("Ymir checkpoint payload is corrupt.", error);
}
}
private static void WriteJournal(BinaryWriter writer, IReadOnlyList<YmirSessionJournalEntry> entries)
{
WriteCount(writer, entries?.Count ?? -1);
foreach (var entry in entries!)
{
switch (entry)
{
case YmirSpawnBodyJournalEntry value:
writer.Write((byte)1); WriteHeader(writer, value.Command.Header); WriteBody(writer, value.Command.Body); break;
case YmirRemoveBodyJournalEntry value:
writer.Write((byte)2); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); break;
case YmirTeleportBodyJournalEntry value:
writer.Write((byte)3); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); WriteVec2(writer, value.Command.Position); WriteVec2(writer, value.Command.Direction); break;
case YmirSetBodyVelocityJournalEntry value:
writer.Write((byte)4); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); WriteVec2(writer, value.Command.LinearVelocity); writer.Write(value.Command.AngularVelocity); break;
case YmirConfigureBodyJournalEntry value:
writer.Write((byte)5); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); writer.Write(value.Command.Radius); writer.Write(value.Command.Mass); writer.Write(value.Command.IsStatic); writer.Write(value.Command.Restitution); writer.Write(value.Command.IsKinematic); break;
case YmirApplyForceJournalEntry value:
writer.Write((byte)6); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); WriteVec2(writer, value.Command.Force); break;
case YmirApplyTorqueJournalEntry value:
writer.Write((byte)7); WriteHeader(writer, value.Command.Header); WriteString(writer, value.Command.BodyId); writer.Write(value.Command.Torque); break;
case YmirStepSessionJournalEntry value:
writer.Write((byte)8); WriteHeader(writer, value.Command.Header); writer.Write(value.Command.DeltaTime); WriteFields(writer, value.Command.Fields); writer.Write(value.Command.SubstepCount); break;
default:
throw new InvalidOperationException($"Unknown Ymir checkpoint journal entry '{entry?.GetType().FullName ?? "<null>"}'.");
}
}
}
private static YmirSessionJournalEntry[] ReadJournal(BinaryReader reader)
{
var count = ReadCount(reader);
var values = new YmirSessionJournalEntry[count];
for (var index = 0; index < count; index++)
{
values[index] = reader.ReadByte() switch
{
1 => new YmirSpawnBodyJournalEntry(new YmirSpawnBodyCommand(ReadHeader(reader), ReadBody(reader)!)),
2 => new YmirRemoveBodyJournalEntry(new YmirRemoveBodyCommand(ReadHeader(reader), ReadString(reader))),
3 => new YmirTeleportBodyJournalEntry(new YmirTeleportBodyCommand(ReadHeader(reader), ReadString(reader), ReadVec2(reader), ReadVec2(reader))),
4 => new YmirSetBodyVelocityJournalEntry(new YmirSetBodyVelocityCommand(ReadHeader(reader), ReadString(reader), ReadVec2(reader), reader.ReadSingle())),
5 => new YmirConfigureBodyJournalEntry(new YmirConfigureBodyCommand(ReadHeader(reader), ReadString(reader), reader.ReadSingle(), reader.ReadSingle(), reader.ReadBoolean(), reader.ReadSingle(), reader.ReadBoolean())),
6 => new YmirApplyForceJournalEntry(new YmirApplyForceCommand(ReadHeader(reader), ReadString(reader), ReadVec2(reader))),
7 => new YmirApplyTorqueJournalEntry(new YmirApplyTorqueCommand(ReadHeader(reader), ReadString(reader), reader.ReadSingle())),
8 => new YmirStepSessionJournalEntry(new YmirStepSessionCommand(ReadHeader(reader), reader.ReadSingle(), ReadFields(reader)!, reader.ReadInt32())),
var kind => throw new InvalidDataException($"Unknown Ymir checkpoint journal kind {kind}.")
};
}
return values;
}
private static void WriteWorld(BinaryWriter writer, YmirWorld world)
{
ArgumentNullException.ThrowIfNull(world);
writer.Write(world.Time);
WriteBodies(writer, world.Bodies);
WriteFields(writer, world.Fields);
}
private static YmirWorld ReadWorld(BinaryReader reader)
{
var time = reader.ReadSingle();
var bodies = ReadBodies(reader);
var fields = ReadFields(reader) ?? throw new InvalidDataException("Ymir checkpoint world fields cannot be null.");
return new YmirWorld(time, bodies, fields);
}
private static void WriteBodies(BinaryWriter writer, IReadOnlyList<PhysicsBody> bodies)
{
WriteCount(writer, bodies?.Count ?? -1);
foreach (var body in bodies!) WriteBody(writer, body);
}
private static PhysicsBody[] ReadBodies(BinaryReader reader)
{
var count = ReadCount(reader);
var values = new PhysicsBody[count];
for (var index = 0; index < count; index++)
values[index] = ReadBody(reader) ?? throw new InvalidDataException("Ymir body list contains null.");
return values;
}
private static void WriteBody(BinaryWriter writer, PhysicsBody? body)
{
writer.Write(body != null);
if (body == null) return;
WriteString(writer, body.Id);
WriteVec2(writer, body.Position);
WriteVec2(writer, body.Velocity);
writer.Write(body.Radius);
writer.Write(body.Mass);
writer.Write(body.IsStatic);
writer.Write(body.Restitution);
writer.Write(body.Direction.HasValue);
if (body.Direction is { } direction) WriteVec2(writer, direction);
writer.Write(body.AngularVelocity);
writer.Write(body.Torque);
writer.Write(body.IsKinematic);
writer.Write(body.IsBullet);
writer.Write(body.ParticipatesInFields);
writer.Write(body.CollisionCategoryBits);
writer.Write(body.CollisionMaskBits);
writer.Write(body.CollisionGroupIndex);
}
private static PhysicsBody? ReadBody(BinaryReader reader)
{
if (!reader.ReadBoolean()) return null;
var id = ReadString(reader);
var position = ReadVec2(reader);
var velocity = ReadVec2(reader);
var radius = reader.ReadSingle();
var mass = reader.ReadSingle();
var isStatic = reader.ReadBoolean();
var restitution = reader.ReadSingle();
var direction = reader.ReadBoolean() ? ReadVec2(reader) : (Vec2?)null;
return new PhysicsBody(
id, position, velocity, radius, mass, isStatic, restitution, direction,
reader.ReadSingle(), reader.ReadSingle(), reader.ReadBoolean(), reader.ReadBoolean(),
reader.ReadBoolean(), reader.ReadUInt64(), reader.ReadUInt64(), reader.ReadInt32());
}
private static void WriteFields(BinaryWriter writer, IReadOnlyList<RadialField>? fields)
{
writer.Write(fields != null);
if (fields == null) return;
WriteCount(writer, fields.Count);
foreach (var field in fields)
{
writer.Write(field != null);
if (field == null) continue;
WriteString(writer, field.Id);
WriteVec2(writer, field.Position);
writer.Write(field.Strength);
writer.Write(field.Radius);
}
}
private static RadialField[]? ReadFields(BinaryReader reader)
{
if (!reader.ReadBoolean()) return null;
var count = ReadCount(reader);
var values = new RadialField[count];
for (var index = 0; index < count; index++)
{
if (!reader.ReadBoolean())
{
values[index] = null!;
continue;
}
values[index] = new RadialField(ReadString(reader), ReadVec2(reader), reader.ReadSingle(), reader.ReadSingle());
}
return values;
}
private static void WriteEpisodes(BinaryWriter writer, IReadOnlyList<YmirContactEpisodeCheckpoint> episodes)
{
WriteCount(writer, episodes?.Count ?? -1);
foreach (var value in episodes!)
{
WriteString(writer, value.BodyA); writer.Write(value.BodyAGeneration);
WriteString(writer, value.BodyB); writer.Write(value.BodyBGeneration);
writer.Write(value.StartStepIndex); writer.Write(value.Ordinal);
}
}
private static YmirContactEpisodeCheckpoint[] ReadEpisodes(BinaryReader reader)
{
var count = ReadCount(reader);
var values = new YmirContactEpisodeCheckpoint[count];
for (var index = 0; index < count; index++)
values[index] = new YmirContactEpisodeCheckpoint(
ReadString(reader), reader.ReadInt64(), ReadString(reader), reader.ReadInt64(),
reader.ReadInt64(), reader.ReadInt64());
return values;
}
private static void WriteHeader(BinaryWriter writer, YmirCommandHeader header)
{
WriteString(writer, header.CommandId);
writer.Write(header.ExpectedRevision);
}
private static YmirCommandHeader ReadHeader(BinaryReader reader) => new(ReadString(reader), reader.ReadInt64());
private static void WriteVec2(BinaryWriter writer, Vec2 value)
{
writer.Write(value.X);
writer.Write(value.Y);
}
private static Vec2 ReadVec2(BinaryReader reader) => new(reader.ReadSingle(), reader.ReadSingle());
private static void WriteString(BinaryWriter writer, string value)
{
if (value == null) throw new InvalidOperationException("Ymir checkpoint strings cannot be null.");
var bytes = Encoding.UTF8.GetBytes(value);
if (bytes.Length > MaximumStringBytes)
throw new InvalidOperationException("Ymir checkpoint string exceeds the supported size.");
writer.Write(bytes.Length);
writer.Write(bytes);
}
private static string ReadString(BinaryReader reader)
{
var length = reader.ReadInt32();
if (length < 0 || length > MaximumStringBytes)
throw new InvalidDataException("Ymir checkpoint string length is invalid.");
var bytes = reader.ReadBytes(length);
if (bytes.Length != length)
throw new EndOfStreamException();
return Encoding.UTF8.GetString(bytes);
}
private static void WriteCount(BinaryWriter writer, int count)
{
if (count < 0 || count > MaximumCollectionCount)
throw new InvalidOperationException("Ymir checkpoint collection count is invalid.");
writer.Write(count);
}
private static int ReadCount(BinaryReader reader)
{
var count = reader.ReadInt32();
return count is < 0 or > MaximumCollectionCount
? throw new InvalidDataException("Ymir checkpoint collection count is invalid.")
: count;
}
}

View file

@ -124,7 +124,7 @@ public sealed class Box3DParityTests
[Fact]
public unsafe void RetainedSessionCommandsPreserveTypedContactLifecycle()
{
Assert.Equal(4u, Box3DOracle.ymir_box3d_get_abi_version());
Assert.Equal(5u, Box3DOracle.ymir_box3d_get_abi_version());
Assert.Equal(0, Box3DOracle.ymir_box3d_session_create(out var session));
try
{

View file

@ -25,7 +25,8 @@ public sealed class YmirServicePublicationTests
Assert.Equal("Box3D v0.1.0 (C17)", diagnostics.NumericSubstrate);
Assert.Contains("does not own physics algorithms", diagnostics.BatchKernel, StringComparison.Ordinal);
Assert.Contains("world_state.v2", diagnostics.Persistence, StringComparison.Ordinal);
Assert.Contains("v0 read migration", diagnostics.Persistence, StringComparison.Ordinal);
Assert.Contains("replay checkpoint v1", diagnostics.Persistence, StringComparison.Ordinal);
Assert.Contains("host-owned private journal persistence", diagnostics.Persistence, StringComparison.Ordinal);
Assert.Equal("box3d-retained-session", diagnostics.Status);
}

View file

@ -92,6 +92,70 @@ public sealed class YmirSessionTests
Assert.Equal("b", firstBegin.BodyB);
}
[Fact]
public void ReplayCheckpointPreservesContactLineageAndCommandLedger()
{
using var original = YmirSession.Create(new YmirSessionCreateRequest(
"checkpoint-contact",
[
Body("ship", positionX: 0.0f, isStatic: true),
Body("pickup", positionX: 1.5f)
]));
var began = original.Step(Step("begin", 0, 0.01f));
var firstBegin = Assert.Single(began.ContactFacts, fact => fact.Kind == YmirContactFactKind.Begin);
var checkpoint = original.Checkpoint();
var payload = YmirSessionCheckpointCodec.Encode(checkpoint);
checkpoint = YmirSessionCheckpointCodec.Decode(payload);
using var restored = YmirSession.Restore(checkpoint);
var duplicate = restored.Step(Step("begin", 0, 0.01f));
Assert.Equal(began.Receipt.ReceiptId, duplicate.Receipt.ReceiptId);
Assert.Equal(began.Receipt.Outcome, duplicate.Receipt.Outcome);
Assert.Equal(began.Receipt.BeforeRevision, duplicate.Receipt.BeforeRevision);
Assert.Equal(began.Receipt.AfterRevision, duplicate.Receipt.AfterRevision);
Assert.Equal(began.Receipt.ProducedFactIds, duplicate.Receipt.ProducedFactIds);
Assert.Equal(began.ContactFacts.Select(value => value.FactId), duplicate.ContactFacts.Select(value => value.FactId));
Assert.Equal(checkpoint.SessionGeneration, restored.Info.SessionGeneration);
var conflict = restored.Step(Step("begin", 0, 0.02f));
Assert.Equal(YmirCommandError.CommandIdConflict, conflict.Receipt.Error);
var persistent = restored.Step(Step("persistent", 1, 0.01f));
Assert.DoesNotContain(persistent.ContactFacts, fact => fact.Kind == YmirContactFactKind.Begin);
restored.Teleport(new YmirTeleportBodyCommand(
Header("separate", 2), "pickup", new Vec2(10.0f, 0.0f), new Vec2(0.0f, 1.0f)));
var ended = restored.Step(Step("end", 3, 0.01f));
var end = Assert.Single(ended.ContactFacts, fact => fact.Kind == YmirContactFactKind.End);
Assert.Equal(firstBegin.ContactId, end.ContactId);
restored.Teleport(new YmirTeleportBodyCommand(
Header("return", 4), "pickup", new Vec2(1.5f, 0.0f), new Vec2(0.0f, 1.0f)));
var beganAgain = restored.Step(Step("begin-again", 5, 0.01f));
var secondBegin = Assert.Single(beganAgain.ContactFacts, fact => fact.Kind == YmirContactFactKind.Begin);
Assert.NotEqual(firstBegin.ContactId, secondBegin.ContactId);
}
[Fact]
public void ReplayCheckpointRestoresPendingForceAndRejectsCorruptJournal()
{
using var original = YmirSession.Create(new YmirSessionCreateRequest(
"checkpoint-force", [Body("ship")]));
original.ApplyForce(new YmirApplyForceCommand(Header("force", 0), "ship", new Vec2(10.0f, 0.0f)));
var checkpoint = original.Checkpoint();
using var restored = YmirSession.Restore(checkpoint);
var stepped = restored.Step(Step("step", 1, 0.1f));
Assert.True(stepped.World.Bodies.Single().Velocity.X > 0.0f);
var corrupt = checkpoint with { Journal = [] };
Assert.Throws<InvalidOperationException>(() => YmirSession.Restore(corrupt));
var wrongRuntime = checkpoint with { RuntimeFingerprint = "some-other-solver" };
Assert.Throws<InvalidOperationException>(() => YmirSession.Restore(wrongRuntime));
var corruptIdentity = YmirSessionCheckpointCodec.Encode(checkpoint);
corruptIdentity[32] ^= 0xff;
Assert.Throws<InvalidDataException>(() => YmirSessionCheckpointCodec.Decode(corruptIdentity));
Assert.Throws<InvalidDataException>(() =>
YmirSessionCheckpointCodec.Decode(YmirSessionCheckpointCodec.Encode(checkpoint).Concat(new byte[] { 1 }).ToArray()));
}
[Fact]
public void RemoveAndRecreateChangesBodyGeneration()
{