connector-usage-only: store-failure tests, README front door
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
af5da3dfaa
commit
dcb5f109f7
4 changed files with 62 additions and 26 deletions
23
README.md
23
README.md
|
|
@ -4,6 +4,11 @@ The connector kit for GameCult minds: one typed, encrypted, replay-safe contract
|
|||
between a caller that needs a model turn and a daemon that holds the provider
|
||||
credential, so no project wires its own provider calls.
|
||||
|
||||
The connector meters nothing in dollars. It holds the subscription token, lends it, and
|
||||
publishes the plan's usage windows (`usage_readout`, and `plan_quota` on every lease grant) for the
|
||||
minds to read and decide their own burn. The one usage-derived refusal is the provider's own
|
||||
(`ProviderRejected`), repeated until its reset.
|
||||
|
||||
Status: the `model-connector` crate and the `claude-connector` backend crate are live. No
|
||||
connector binary exists yet (the Claude connector binary is first, then an OpenAI-compatible one).
|
||||
|
||||
|
|
@ -11,7 +16,7 @@ connector binary exists yet (the Claude connector binary is first, then an OpenA
|
|||
|
||||
Crate `crates/model-connector`:
|
||||
|
||||
- `src/contract.rs` - the v3 wire contract (`gamecult.model.*.v3`): `ProviderRequest`, `TransportInvocation`,
|
||||
- `src/contract.rs` - the v4 wire contract (`gamecult.model.*.v4`): `ProviderRequest`, `TransportInvocation`,
|
||||
`TransportResult`, `TransportReceipt`, the typed `StopReason`, `FailureKind` and `Refusal`,
|
||||
`InputItem::ReasoningState` (a provider's opaque reasoning block, replayed by callers
|
||||
verbatim via `TransportResult::transcript_items`; the service refuses another provider's
|
||||
|
|
@ -23,10 +28,10 @@ Crate `crates/model-connector`:
|
|||
`cultnet.operation_request.v0` / `operation_response.v0` for service
|
||||
`gamecult.model.connector`, operation `invoke` (or `usage_readout`), its `message_id` the
|
||||
request id and its payload the base64 of the sealed envelope. `usage_readout` answers a
|
||||
`UsageReadout` (`gamecult.model.usage_readout.v1`): each caller's spent and reserved
|
||||
micro-USD against its ceiling in its current window, and the backend's plan quota;
|
||||
`UsageReadout` (`gamecult.model.usage_readout.v1`): the backend's plan quota windows, each with
|
||||
utilization, reset, status and age;
|
||||
`ConnectorClient::usage_readout` asks.
|
||||
- `schemas/` - one JSON Schema per v3 document, generated from the types.
|
||||
- `schemas/` - one JSON Schema per v4 document, generated from the types.
|
||||
- `src/client.rs` - `ConnectorClient`, the one-request caller side.
|
||||
- `src/service.rs` (feature `daemon`) - `TransportService`: caller admission, durable
|
||||
replay (a retried request is answered byte-identically and the provider runs once),
|
||||
|
|
@ -49,7 +54,7 @@ Crate `crates/claude-connector` (library only; the binary that wires it to `serv
|
|||
group); `SessionStore` and `StderrSink` are the other ports. `ClaudeCodeConfig` is what the
|
||||
binary hands in; the credential is read from a systemd credential and reaches the child on a
|
||||
pipe (`CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR`), never in an environment. `QuotaCell` keeps the
|
||||
plan windows the CLI's `rate_limit_event`s report, for the spend readout.
|
||||
plan windows the CLI's `rate_limit_event`s report, for the usage readout and the lease grant.
|
||||
|
||||
Frame arithmetic is public in `src/envelope.rs`, measured on the carriage frame: `frame_bytes_for_payload(n)` is the frame a
|
||||
caller payload bound of `n` needs, `payload_bytes_for_frame(frame)` is the largest payload bound
|
||||
|
|
@ -58,7 +63,7 @@ a frame allows, and `MAX_IDENTITY_BYTES` (256) bounds caller ids, request ids an
|
|||
needed); the smallest legal frame is `frame_bytes_for_payload(MIN_PAYLOAD_BYTES)`. A request id past
|
||||
the bound is refused on the wire as `Refusal::Malformed`, the reply echoing the id (the connection is dropped only if that reply cannot fit the frame); `ConnectorClient` refuses such an id locally with `RequestIdTooLong`.
|
||||
|
||||
The default feature set compiles contract, envelope and client only. The kit speaks v3
|
||||
The default feature set compiles contract, envelope and client only. The kit speaks v4
|
||||
only: a v2 frame gets no answer.
|
||||
|
||||
## Verify
|
||||
|
|
@ -68,9 +73,9 @@ cargo test --locked --workspace
|
|||
cargo test --locked --workspace --features model-connector/daemon
|
||||
```
|
||||
|
||||
`crates/model-connector/tests/wire_golden_v3.rs` pins the v3 documents and one sealed carriage
|
||||
frame to the fixtures in `crates/model-connector/tests/fixtures/wire_v3` (regenerate with
|
||||
`WIRE_V3_REGENERATE=1`; see the fixtures' README), and
|
||||
`crates/model-connector/tests/wire_golden_v4.rs` pins the v4 documents and one sealed carriage
|
||||
frame to the fixtures in `crates/model-connector/tests/fixtures/wire_v4` (regenerate with
|
||||
`WIRE_V4_REGENERATE=1`; see the fixtures' README), and
|
||||
`crates/model-connector/tests/published_schemas.rs` fails when `schemas/` differs from the types
|
||||
(`MODEL_CONNECTOR_REGENERATE_SCHEMAS=1` rewrites it).
|
||||
|
||||
|
|
|
|||
|
|
@ -1438,7 +1438,8 @@ fn usage_sums_model_usage_across_models() {
|
|||
panic!("completed expected");
|
||||
};
|
||||
assert_eq!(*summed, usage(12, 37, 1083, 24, Some(8)));
|
||||
// The receipt is priced from usage alone: 12*3 + 37*0.3 + 1083*6 + 24*15 micro-USD, rounded up.
|
||||
// The receipt is the usage alone: the CLI's own total never reaches it.
|
||||
assert_eq!(parts(&cheap).1, parts(&dear).1);
|
||||
|
||||
// No model reported thinking tokens: none is claimed.
|
||||
let plain = r#"{"claude-haiku-5-5":{"inputTokens":1,"outputTokens":1,"cacheReadInputTokens":0,"cacheCreationInputTokens":0}}"#;
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ impl PeerUnits for FakeUnits {
|
|||
struct Harness {
|
||||
dir: PathBuf,
|
||||
socket: PathBuf,
|
||||
service: Arc<Mutex<TransportService>>,
|
||||
_service: Arc<Mutex<TransportService>>,
|
||||
quota: Arc<QuotaCell>,
|
||||
tokens: Arc<FakeTokens>,
|
||||
/// The fake behind the listener, when it has one.
|
||||
|
|
@ -162,18 +162,6 @@ fn receive(stream: &UnixStream) -> (Vec<u8>, Vec<OwnedFd>) {
|
|||
}
|
||||
}
|
||||
|
||||
fn connect(harness: &Harness) -> UnixStream {
|
||||
let stream = UnixStream::connect(&harness.socket).unwrap();
|
||||
stream
|
||||
.set_read_timeout(Some(Duration::from_secs(5)))
|
||||
.unwrap();
|
||||
stream
|
||||
}
|
||||
|
||||
fn send(stream: &mut UnixStream, message: &LeaseMessage) {
|
||||
stream.write_all(&encode_frame(message)).unwrap();
|
||||
}
|
||||
|
||||
fn request(ttl_ms: u64) -> LeaseMessage {
|
||||
LeaseMessage::ClaudeLeaseRequest(LeaseRequest {
|
||||
expires_at_unix_ms: now_ms() + ttl_ms,
|
||||
|
|
@ -182,7 +170,7 @@ fn request(ttl_ms: u64) -> LeaseMessage {
|
|||
|
||||
/// What one request to the listener came back with.
|
||||
struct Answer {
|
||||
stream: UnixStream,
|
||||
_stream: UnixStream,
|
||||
raw: Vec<u8>,
|
||||
message: Option<LeaseMessage>,
|
||||
fds: Vec<OwnedFd>,
|
||||
|
|
@ -202,7 +190,7 @@ fn ask_at(socket: &std::path::Path, ttl_ms: u64) -> Answer {
|
|||
let (raw, fds) = receive(&stream);
|
||||
let message = decode_frame(&raw);
|
||||
Answer {
|
||||
stream,
|
||||
_stream: stream,
|
||||
raw,
|
||||
message,
|
||||
fds,
|
||||
|
|
@ -316,7 +304,7 @@ fn start(setup: Setup) -> Harness {
|
|||
let harness = Harness {
|
||||
dir,
|
||||
socket,
|
||||
service,
|
||||
_service: service,
|
||||
quota,
|
||||
tokens,
|
||||
units: Some(units),
|
||||
|
|
|
|||
|
|
@ -2158,4 +2158,46 @@ mod tests {
|
|||
Some(Operation::UsageReadout)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_store_that_cannot_be_written_executes_nothing_and_holds_no_slot() {
|
||||
let store = FlakyStore::new(0);
|
||||
let allowance = store.allowance();
|
||||
let mut service =
|
||||
TransportService::with_store(store, "fixture-sse", [caller(CALLER, KEY, 1)], 5_000)
|
||||
.unwrap();
|
||||
let invocation = invocation_for(CALLER, "request-1");
|
||||
assert!(matches!(
|
||||
service.begin_unplanned(&raw_envelope(&invocation), 1_000, FRAME),
|
||||
Err(ServiceError::ReplayStore(_))
|
||||
));
|
||||
allowance.store(10, Ordering::SeqCst);
|
||||
assert!(matches!(
|
||||
service.begin_unplanned(&raw_envelope(&invocation), 1_000, FRAME),
|
||||
Ok(TransportAdmission::Execute(_))
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_failed_completion_write_answers_the_store_failure_and_frees_the_slot() {
|
||||
let store = FlakyStore::new(1);
|
||||
let allowance = store.allowance();
|
||||
let mut service =
|
||||
TransportService::with_store(store, "fixture-sse", [caller(CALLER, KEY, 1)], 5_000)
|
||||
.unwrap();
|
||||
let (claim, invocation) = claim_for(&mut service, "request-1");
|
||||
let result = text_result(claim.call(), "done");
|
||||
assert!(matches!(
|
||||
service.complete(*claim, result, FRAME),
|
||||
Err(ServiceError::ReplayStore(_))
|
||||
));
|
||||
// The slot is free (one concurrent request is admitted), and the retry
|
||||
// of the unfinished id is told it is indeterminate, not in flight.
|
||||
allowance.store(10, Ordering::SeqCst);
|
||||
let again = service
|
||||
.begin_unplanned(&raw_envelope(&invocation), 1_000, FRAME)
|
||||
.unwrap();
|
||||
assert_eq!(refusal_of(again, &invocation), Refusal::Indeterminate);
|
||||
let _ = claim_for(&mut service, "request-2");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue