Improve managed content throughput
This commit is contained in:
parent
26b4d1eadc
commit
a311cbe8cb
8 changed files with 292 additions and 62 deletions
|
|
@ -41,3 +41,15 @@ network descriptor without changing logical identity.
|
|||
The content protocol uses one bounded response per manifest chunk. A snapshot
|
||||
response containing `gamecult.mesh.cdn_artifact_chunk.v1` payload records is a
|
||||
legacy transport path and must not be used for bulk delivery.
|
||||
|
||||
The transfer owner may keep a bounded window of requests in flight per content
|
||||
hash (four by default, never more than 32). Candidate chunks can arrive out of
|
||||
order, but only the transfer owner writes them, and it flushes and checkpoints
|
||||
them in manifest-offset order. If one request fails, the owner observes the
|
||||
rest of that window before returning; fetched-but-uncommitted bytes never become
|
||||
trusted resume state. Concurrent artifacts have independent windows.
|
||||
|
||||
RUDP schema hosts should drain immediately available transport work with
|
||||
`PollAvailableAsync`. Its result distinguishes transport items consumed from
|
||||
application messages dispatched, so ACK and connection traffic counts as
|
||||
progress without impersonating a message or triggering an idle delay.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
# Distributed CDN Documents
|
||||
|
||||
CultMesh CDN artifacts are typed CultCache documents, not a separate asset
|
||||
transport. The same manifest and chunk records can be written by an asset
|
||||
pipeline daemon, replicated through CultNet raw snapshots or shard logs, and
|
||||
read by Unity, TypeScript, or another runtime that implements the CultCache
|
||||
document contract.
|
||||
CultMesh CDN artifacts use typed CultCache storage and a separate managed
|
||||
content-delivery session. Manifests may travel through snapshots or shard logs;
|
||||
artifact payload chunks remain in provider storage and cross the network through
|
||||
`cultmesh.content.v1`. Unity, TypeScript, and other runtimes consume the same
|
||||
manifest and verification contract without making snapshots a bulk byte lane.
|
||||
|
||||
This is the canonical path for moving Aetheria assets into CultLib. Starbridge
|
||||
Unity and Starbridge Electron should consume the same records so UI textures,
|
||||
|
|
@ -67,11 +67,11 @@ updates.
|
|||
Clients should:
|
||||
|
||||
1. Fetch or receive a manifest by record key, tag, or index.
|
||||
2. Fetch every chunk record referenced by `manifest.Chunks`.
|
||||
3. Verify each chunk hash before copying bytes.
|
||||
4. Reassemble chunks by `Offset`.
|
||||
5. Verify the full `ContentHash`.
|
||||
6. Interpret bytes according to `MimeType` plus domain metadata.
|
||||
2. Give it to `CultMeshContentTransferService` with one or more content providers.
|
||||
3. Let that owner request missing chunks through managed content sessions.
|
||||
4. Use only the atomically promoted `<content-hash>.body` returned after chunk
|
||||
and full-body verification.
|
||||
5. Interpret bytes according to `MimeType` plus domain metadata.
|
||||
|
||||
Single-channel data stays single-channel. For example, a gravity map can use
|
||||
raw bytes with metadata such as `channels=r32` instead of pretending to be an
|
||||
|
|
@ -147,27 +147,20 @@ await CultMesh.PublishCdnArtifactAsync(node.Cache, artifact);
|
|||
For build distribution, use the same API with `Kind = CultMeshCdnArtifactKinds.Build`
|
||||
and metadata such as `platform=windows-x64` or `channel=internal`.
|
||||
|
||||
## CultNet Replication
|
||||
## CultNet Publication And Delivery
|
||||
|
||||
CDN documents use the regular raw document lane:
|
||||
Publish manifests and chunks into the provider cache:
|
||||
|
||||
```csharp
|
||||
var registry = CultMesh.CreateCdnDocumentRegistry(node.Cache.Registry);
|
||||
var request = registry.CreateSnapshotRequest(
|
||||
"cdn-request",
|
||||
schemaIds:
|
||||
[
|
||||
node.Cache.Registry.GetRequired<CultMeshCdnArtifactManifest>().SchemaId,
|
||||
node.Cache.Registry.GetRequired<CultMeshCdnArtifactChunk>().SchemaId
|
||||
],
|
||||
recordKeys: artifact.RecordKeys.Select(key => key.Value));
|
||||
|
||||
var response = registry.CreateRawSnapshotResponse(node.Cache, "cdn-response", request);
|
||||
await CultMesh.PublishCdnArtifactAsync(providerCache, artifact);
|
||||
using var contentServer = new CultMeshContentServer(schemaServer, providerCache);
|
||||
```
|
||||
|
||||
The TypeScript client should mirror the C# schema names, schema versions,
|
||||
record-key rules, chunk hashing, and verification order. C# remains the
|
||||
reference implementation.
|
||||
Snapshots may publish the manifest record, catalog, and body-control documents.
|
||||
They must not publish `CultMeshCdnArtifactChunk` payload records for bulk
|
||||
delivery. Consumers use `CultMeshSessionContentProvider` and
|
||||
`CultMeshContentTransferService`; C# remains the reference implementation for
|
||||
record keys, hashing, resumability, and atomic promotion.
|
||||
|
||||
## Entity Prefabs
|
||||
|
||||
|
|
@ -187,7 +180,8 @@ the same package graph into their native entity/component model.
|
|||
Unity and TypeScript clients should implement the same rules:
|
||||
|
||||
- Fetch prefab packages by record key, tag, or version catalog.
|
||||
- Fetch every referenced artifact manifest and chunk.
|
||||
- Fetch every referenced artifact manifest, then delegate payload transfer to
|
||||
the generic content-transfer owner.
|
||||
- Verify signed manifests before trusting package membership.
|
||||
- Verify chunk hashes and full artifact hashes before import.
|
||||
- Cache artifacts locally by content hash.
|
||||
|
|
|
|||
|
|
@ -551,9 +551,10 @@ atomic promotion primitive. Concurrent requests for one hash share the same
|
|||
per-content transfer lock and verified work.
|
||||
|
||||
**Cut line:** the legacy distributed reader no longer owns a sequential chunk
|
||||
loop or whole-artifact allocation. The first slice intentionally leaves
|
||||
adaptive parallelism, quotas, pinning, signer policy, arbitrary streaming
|
||||
destinations, and cleanup scheduling to later Phase 4 work.
|
||||
loop or whole-artifact allocation. The transfer owner now opens a bounded
|
||||
per-content request window while preserving one ordered writer/checkpointer.
|
||||
Adaptive scheduling, quotas, pinning, signer policy, arbitrary streaming
|
||||
destinations, and cleanup scheduling remain later Phase 4 work.
|
||||
|
||||
**State transition:** each verified chunk flushes a typed checkpoint. Restart
|
||||
rehydrates that record, rehashes every claimed range, and fetches corrupt or
|
||||
|
|
@ -563,8 +564,10 @@ partial state.
|
|||
|
||||
**Verification layer:** focused tests cover durable cancellation/restart,
|
||||
provider corruption and failover, forged checkpoint rejection, final-hash
|
||||
failure, atomic visibility, and concurrent request coalescing. Existing CDN
|
||||
tests cover the compatibility reader.
|
||||
failure, atomic visibility, concurrent request coalescing, bounded concurrent
|
||||
fetch, ordered checkpointing under out-of-order completion, and observation of
|
||||
the whole request window after an early failure. Existing CDN tests cover the
|
||||
compatibility reader.
|
||||
|
||||
### Phase 4 managed content-session slice
|
||||
|
||||
|
|
@ -600,8 +603,10 @@ descriptors; the managed content protocol carries bounded payload responses.
|
|||
**Verification layer:** Mesh tests transfer a multi-chunk body through one
|
||||
managed session, assert zero snapshot requests, assert warm committed reuse
|
||||
without another network request, and round-trip both wire messages through the
|
||||
schema dispatcher. The released EveUnity/Aetheria cold witness remains the real
|
||||
RUDP consumer proof required to close the slice.
|
||||
schema dispatcher. RUDP schema-host tests distinguish control-packet progress
|
||||
from dispatched messages and drain bounded available work without idle sleeps.
|
||||
The released EveUnity/Aetheria cold witness remains the real consumer proof
|
||||
required to close the slice.
|
||||
|
||||
### Phase 5 verified CDN mapped-body slice
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue