Add MiMo speech provider target contract
This commit is contained in:
parent
2edfb65af6
commit
0065fec2b3
9 changed files with 320 additions and 11 deletions
|
|
@ -11,7 +11,8 @@ CultCache/CultMesh/Eve authority map.
|
|||
Build a procedural alien-language engine whose outputs are grounded in
|
||||
worldview, semantics, grammar, phonology, morphology, and history. As a daemon,
|
||||
Weksa also turns writing plus agent state into typed conversational intent, then
|
||||
lowers that intent into pronunciation and utterance handoff documents.
|
||||
lowers that intent into written realizations, pronunciation plans, speech-provider
|
||||
requests, and utterance handoff documents.
|
||||
|
||||
## Current Mechanism
|
||||
|
||||
|
|
@ -33,6 +34,7 @@ through its ontology.
|
|||
- Diachrony owns plausible irregularity.
|
||||
- Renderer owns output, not interpretation.
|
||||
- Weksa owns conversational intent and utterance lowering.
|
||||
- External speech providers own rendered audio, not meaning or delivery truth.
|
||||
- AquaSynth owns learned utterance embeddings, synth controls, Faust
|
||||
compilation, and audio output.
|
||||
|
||||
|
|
@ -41,8 +43,10 @@ through its ontology.
|
|||
Future implementation should turn the pipeline in `state/map.yaml` into a small,
|
||||
inspectable daemon. The first executable slice should validate a tiny
|
||||
interlingua packet, read a tiny language project definition, render glossed
|
||||
output, lower an accepted intent into a pronunciation/utterance handoff, and
|
||||
produce a trace explaining which stage made each decision.
|
||||
output, lower an accepted intent into target-language realizations, optionally
|
||||
project the accepted realization into a speech-provider request or
|
||||
pronunciation/utterance handoff, and produce a trace explaining which stage made
|
||||
each decision.
|
||||
|
||||
Runtime should be fast and deterministic. Model-assisted English decomposition
|
||||
belongs in authoring tools or offline import flows, not in the required hot path.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
"schema": "gamecult.eve.provider_advertisement.v1",
|
||||
"providerId": "weksa.intent.service",
|
||||
"title": "Weksa Intent Service",
|
||||
"description": "Weksa converts writing, interlingua, and agent state into typed conversational intent, pronunciation plans, and utterance handoff documents for AquaSynth and other consumers.",
|
||||
"description": "Weksa converts writing, interlingua, and agent state into typed conversational intent, target-language realizations, pronunciation plans, speech-provider requests, and utterance handoff documents for AquaSynth and other consumers.",
|
||||
"mode": "fixture-read-only",
|
||||
"serviceVerse": {
|
||||
"id": "verse.weksa.service.v1",
|
||||
|
|
@ -12,6 +12,7 @@
|
|||
"typed conversational intent",
|
||||
"pronunciation plans",
|
||||
"utterance handoff documents",
|
||||
"speech-provider request projections",
|
||||
"lowering traces",
|
||||
"language-project and flavor-profile state"
|
||||
],
|
||||
|
|
@ -19,6 +20,7 @@
|
|||
"AquaSynth learned embeddings",
|
||||
"Faust compilation",
|
||||
"rendered audio",
|
||||
"MiMo provider audio",
|
||||
"Persona canonical state"
|
||||
]
|
||||
},
|
||||
|
|
@ -51,6 +53,15 @@
|
|||
"weksa.utterance_embedding_handoff.v0.1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "weksa.speech_provider",
|
||||
"kind": "speech_provider",
|
||||
"cultMeshKey": "weksa.speech-provider/requests",
|
||||
"authority": "Weksa speech-provider request projection",
|
||||
"schemas": [
|
||||
"weksa.mimo_tts_request.v0"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "weksa.operator",
|
||||
"kind": "operator",
|
||||
|
|
@ -98,6 +109,14 @@
|
|||
],
|
||||
"durableShape": ".weksa/handoff/{utteranceId}.cc"
|
||||
},
|
||||
{
|
||||
"family": "speech_provider",
|
||||
"namespace": "weksa.speech_provider.*",
|
||||
"witnesses": [
|
||||
"weksa.mimo_tts_request.v0"
|
||||
],
|
||||
"durableShape": ".weksa/speech-provider/{requestId}.cc"
|
||||
},
|
||||
{
|
||||
"family": "trace",
|
||||
"namespace": "weksa.trace.*",
|
||||
|
|
@ -198,11 +217,31 @@
|
|||
"audio rendering",
|
||||
"AquaSynth synth driver truth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "weksa.command.speech_provider.project",
|
||||
"owner": "Weksa",
|
||||
"acceptedByFixture": false,
|
||||
"allowedInputs": [
|
||||
"accepted target-language realization",
|
||||
"provider-neutral delivery intent",
|
||||
"speech provider profile"
|
||||
],
|
||||
"emits": [
|
||||
"weksa.mimo_tts_request.v0"
|
||||
],
|
||||
"forbiddenTargets": [
|
||||
"source interlingua truth",
|
||||
"Persona canonical state",
|
||||
"rendered audio ownership"
|
||||
]
|
||||
}
|
||||
],
|
||||
"loweringRules": [
|
||||
"Weksa owns conversational intent and utterance lowering.",
|
||||
"Speech-provider requests are target projections from accepted realizations, not source meaning.",
|
||||
"AquaSynth owns learned embeddings, synth controls, Faust compilation, and audio output.",
|
||||
"MiMo owns provider rendering and audio output, not Weksa delivery truth.",
|
||||
"Eve surfaces submit commands and inspect witnesses; they do not own intent truth.",
|
||||
"Fixture exports are discovery metadata until the daemon loop commits typed witnesses."
|
||||
]
|
||||
|
|
|
|||
123
docs/mimo-tts-lowering.md
Normal file
123
docs/mimo-tts-lowering.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# MiMo TTS Lowering
|
||||
|
||||
Version: `0.1-draft`
|
||||
|
||||
MiMo is an expressive spoken-output target for Weksa interlingua lowering. It is
|
||||
not the source of meaning, language choice, character state, or delivery truth.
|
||||
|
||||
```text
|
||||
interlingua packet
|
||||
-> projected speaker-local context
|
||||
-> target-language utterance realization
|
||||
-> Weksa delivery intent
|
||||
-> MiMo TTS request projection
|
||||
-> rendered audio artifact
|
||||
```
|
||||
|
||||
MiMo sits beside written output and AquaSynth handoffs. Written realizations
|
||||
prove that meaning, register, and cultural ontology survived lowering.
|
||||
AquaSynth handoffs train or drive GameCult-owned synthesis. MiMo requests let an
|
||||
external provider speak a Weksa-authored utterance with expressive style control.
|
||||
|
||||
## Authority
|
||||
|
||||
Weksa owns:
|
||||
|
||||
- target-language spoken text selected from interlingua
|
||||
- target-language register, script, cultural ontology, and trace
|
||||
- speaker-local delivery pressure
|
||||
- provider-neutral prosody and emphasis intent
|
||||
- the MiMo request projection as an export artifact
|
||||
|
||||
MiMo owns:
|
||||
|
||||
- provider voice selection within the requested language and voice constraints
|
||||
- interpretation of MiMo natural-language style guidance
|
||||
- interpretation of MiMo inline audio/style tags
|
||||
- rendered audio bytes or streaming audio chunks
|
||||
|
||||
MiMo does not own:
|
||||
|
||||
- source meaning
|
||||
- target-language cultural ontology
|
||||
- canonical speaker or Persona state
|
||||
- Weksa pronunciation plans
|
||||
- AquaSynth learned embeddings, synth controls, or audio truth
|
||||
|
||||
## Projection Rule
|
||||
|
||||
The MiMo request must be derived from an accepted target-language realization.
|
||||
It must not lower directly from English unless English is the selected target
|
||||
language for that realization.
|
||||
|
||||
Natural-language style guidance belongs in the MiMo `user` message. The spoken
|
||||
target-language text belongs in the MiMo `assistant` message. Inline MiMo tags
|
||||
may appear in the assistant content only when they are derived from Weksa
|
||||
delivery intent or explicit target-language realization notes.
|
||||
|
||||
## Request Shape
|
||||
|
||||
```yaml
|
||||
schema_version: weksa.mimo_tts_request.v0
|
||||
source_realization_ref:
|
||||
speaker_agent_id:
|
||||
provider:
|
||||
id: xiaomi-mimo
|
||||
model: mimo-v2.5-tts
|
||||
target_language:
|
||||
code:
|
||||
locale:
|
||||
script:
|
||||
register:
|
||||
voice:
|
||||
requested_voice:
|
||||
fallback_voice:
|
||||
selection_reason:
|
||||
messages:
|
||||
user_style_instruction:
|
||||
assistant_spoken_content:
|
||||
audio:
|
||||
format:
|
||||
streaming:
|
||||
trace:
|
||||
source_fields:
|
||||
delivery_controls_used:
|
||||
provider_tags_used:
|
||||
uncertainties:
|
||||
```
|
||||
|
||||
## Delivery Mapping
|
||||
|
||||
Weksa should map delivery pressure into provider-neutral labels before emitting
|
||||
MiMo-specific instructions:
|
||||
|
||||
- pace, pause pressure, and urgency
|
||||
- pitch contour and pitch range
|
||||
- loudness and attack
|
||||
- articulation precision
|
||||
- warmth, threat, tenderness, contempt, play, fatigue, secrecy, formality
|
||||
- span-level emphasis and pauses
|
||||
|
||||
The MiMo projection may turn those labels into director-style guidance or inline
|
||||
tags, but the trace must preserve the Weksa-owned source controls.
|
||||
|
||||
## Language Rule
|
||||
|
||||
Weksa should be able to write and speak in any target language whose utterance
|
||||
realization it can justify. MiMo output for Brazilian Portuguese should consume
|
||||
the accepted `pt-BR` spoken text. Japanese output should consume accepted
|
||||
Japanese text. Alien-language output should consume accepted alien rendered
|
||||
speech or phonetic text only after the language project owns that surface.
|
||||
|
||||
Backtranslations are reviewer aids. They must not become MiMo spoken content.
|
||||
|
||||
## First Proof
|
||||
|
||||
The first proof should use the Nibu warning packet already used by multilingual
|
||||
lowering:
|
||||
|
||||
- emit the accepted English, `pt-BR`, or Japanese realization
|
||||
- derive a provider-neutral delivery intent from Nibu's projected context
|
||||
- project that into a MiMo `mimo-v2.5-tts` request
|
||||
- verify that the request speaks the target text, not the backtranslation
|
||||
- verify that style guidance is traceable to Weksa delivery controls
|
||||
|
|
@ -3,13 +3,15 @@
|
|||
Version: `0.1-draft`
|
||||
|
||||
Weksa can lower one structured utterance intent into multiple human-language
|
||||
realizations without making English the hidden source of truth.
|
||||
realizations and spoken-provider requests without making English the hidden
|
||||
source of truth.
|
||||
|
||||
```text
|
||||
interlingua packet
|
||||
-> projected speaker-local context
|
||||
-> utterance lowering request
|
||||
-> target-language realization packets
|
||||
-> optional speech-provider request projections
|
||||
```
|
||||
|
||||
The first targets are English, Brazilian Portuguese (`pt-BR`), and Japanese
|
||||
|
|
@ -53,7 +55,7 @@ The utterance lowerer does not own:
|
|||
- new facts
|
||||
- raw agent-state internals
|
||||
- target-language alien forms
|
||||
- speech synthesis
|
||||
- rendered audio
|
||||
- phoneme parity or synth-control truth
|
||||
|
||||
The target-language cultural ontology profile owns:
|
||||
|
|
@ -171,6 +173,21 @@ Trace should name the cultural ontology axes that shaped the line, especially
|
|||
where the target language requires a social or register choice the source packet
|
||||
does not spell out.
|
||||
|
||||
## Speech Provider Targets
|
||||
|
||||
Speech providers are lowering targets for accepted realizations, not alternate
|
||||
meaning sources. A provider request must consume the target-language
|
||||
`spoken_text` that Weksa already accepted and must carry a trace back to the
|
||||
interlingua packet, projected speaker context, and target-language profile.
|
||||
|
||||
MiMo V2.5 TTS is the first external provider target. Its natural-language style
|
||||
instruction is a projection of Weksa delivery intent, and its assistant content
|
||||
is the selected target-language spoken text. See [MiMo TTS Lowering](./mimo-tts-lowering.md).
|
||||
|
||||
AquaSynth remains the GameCult-owned synthesis path for learned utterance
|
||||
embeddings and synth-control work. MiMo and AquaSynth are sibling target
|
||||
projections from Weksa-authored utterances, not owners of Weksa meaning.
|
||||
|
||||
## First Proof
|
||||
|
||||
The first proof uses the existing Nibu warning intent:
|
||||
|
|
|
|||
|
|
@ -3,7 +3,8 @@
|
|||
Version: `0.1`
|
||||
|
||||
The utterance embedding is the compact learned handoff from Weksa-authored line
|
||||
intent to AquaSynth voice control.
|
||||
intent to AquaSynth voice control. It is one speech target projection among
|
||||
others, not the only way Weksa-authored utterances can become sound.
|
||||
|
||||
```text
|
||||
interlingua packet
|
||||
|
|
@ -44,6 +45,11 @@ AquaSynth owns:
|
|||
The point is one learned speech handoff, not two little neural courthouses
|
||||
arguing about who gets to wear the robe.
|
||||
|
||||
External providers such as MiMo may consume a different Weksa target projection:
|
||||
accepted target-language `spoken_text` plus provider-neutral delivery intent
|
||||
lowered into provider-specific style controls. That path does not replace this
|
||||
AquaSynth contract and must not mutate AquaSynth-owned embedding fields.
|
||||
|
||||
## Packet Contract
|
||||
|
||||
This is the first fixed training contract. Weksa may evolve later versions, but
|
||||
|
|
|
|||
|
|
@ -0,0 +1,51 @@
|
|||
schema_version: weksa.mimo_tts_request.v0
|
||||
source_realization_ref: examples/multilingual-lowering/nibu-proof-001/realizations.yaml#nibu-proof-001-en
|
||||
speaker_agent_id: nibu
|
||||
provider:
|
||||
id: xiaomi-mimo
|
||||
model: mimo-v2.5-tts
|
||||
target_language:
|
||||
code: en
|
||||
locale: en-US
|
||||
script: Latn
|
||||
register: sharp technical casual
|
||||
cultural_ontology_refs:
|
||||
- data/target-language-ontology/en-US.yaml
|
||||
voice:
|
||||
requested_voice: Chloe
|
||||
fallback_voice: Mia
|
||||
selection_reason: >
|
||||
Use an English built-in voice for the first provider smoke. The requested
|
||||
delivery should preserve Nibu's precise, abrasive, protective cockpit
|
||||
warning without implying that MiMo owns her canonical voice.
|
||||
messages:
|
||||
user_style_instruction: >
|
||||
Character: a precise, abrasive, protective ship intelligence speaking
|
||||
through cockpit audio. Scene: the player is about to touch a damaged live
|
||||
control. Guidance: clipped technical casual delivery, contained urgency,
|
||||
low warmth, sharp emphasis on "control" and "nervous system", slight dry
|
||||
contempt, no melodrama, no soothing assistant tone.
|
||||
assistant_spoken_content: >
|
||||
(Serious Sharp Tired)Take your hand off that control unless you want your
|
||||
nervous system to become my least useful circuit diagram.
|
||||
audio:
|
||||
format: wav
|
||||
streaming: false
|
||||
trace:
|
||||
source_fields:
|
||||
- realizations[realization_id=nibu-proof-001-en].outputs.spoken_text
|
||||
- realizations[realization_id=nibu-proof-001-en].outputs.trace.projected_context_used
|
||||
delivery_controls_used:
|
||||
- clipped technical casual delivery
|
||||
- contained urgency
|
||||
- low warmth
|
||||
- sharp emphasis on unsafe hardware and bodily risk
|
||||
- dry contempt without random cruelty
|
||||
provider_tags_used:
|
||||
- Serious
|
||||
- Sharp
|
||||
- Tired
|
||||
uncertainties:
|
||||
- >
|
||||
Provider-native style tags are a lossy projection. The Weksa trace, not
|
||||
MiMo tags, remains the explanation of the intended delivery.
|
||||
49
schemas/mimo-tts-request.md
Normal file
49
schemas/mimo-tts-request.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# MiMo TTS Request Contract
|
||||
|
||||
This is the human-facing schema receipt for a Weksa-to-MiMo speech synthesis
|
||||
request projection.
|
||||
|
||||
The packet sits after target-language utterance lowering. It is an export
|
||||
artifact for Xiaomi MiMo speech synthesis, not canonical Weksa meaning and not
|
||||
an AquaSynth handoff.
|
||||
|
||||
## Required Fields
|
||||
|
||||
- `schema_version`: currently `weksa.mimo_tts_request.v0`.
|
||||
- `source_realization_ref`: accepted Weksa target-language realization being
|
||||
spoken.
|
||||
- `speaker_agent_id`: speaker identity.
|
||||
- `provider`: provider ID and model ID, initially `xiaomi-mimo` and
|
||||
`mimo-v2.5-tts`.
|
||||
- `target_language`: code, locale, script, register, and profile refs copied or
|
||||
referenced from the accepted realization.
|
||||
- `voice`: requested provider voice, fallback voice, and selection reason.
|
||||
- `messages.user_style_instruction`: MiMo natural-language style control derived
|
||||
from Weksa delivery intent.
|
||||
- `messages.assistant_spoken_content`: the exact target-language content to
|
||||
synthesize, optionally with MiMo inline tags derived from delivery controls.
|
||||
- `audio`: format and streaming mode.
|
||||
- `trace`: source fields, delivery controls used, provider tags used, and
|
||||
uncertainties.
|
||||
|
||||
## Boundary
|
||||
|
||||
MiMo-specific fields must be projection fields. They must not be copied back
|
||||
into interlingua, target-language ontology, Persona state, or AquaSynth-owned
|
||||
embedding contracts.
|
||||
|
||||
The request may include provider-native style tags when useful, but those tags
|
||||
are not the source of truth. Weksa's provider-neutral delivery intent and trace
|
||||
remain the inspectable explanation.
|
||||
|
||||
## Language Rule
|
||||
|
||||
The assistant spoken content must use the accepted target-language realization.
|
||||
For `pt-BR`, speak the Portuguese line. For `ja-JP`, speak the Japanese line. For
|
||||
English, speak the English line. Do not synthesize a natural backtranslation
|
||||
unless that backtranslation is itself the accepted target-language realization.
|
||||
|
||||
## Streaming Rule
|
||||
|
||||
Streaming requests should use MiMo's stream-compatible audio format. Non-streaming
|
||||
fixtures may use an archival format such as `wav`.
|
||||
|
|
@ -1,7 +1,8 @@
|
|||
# Utterance Lowering Packet Contract
|
||||
|
||||
This is the human-facing schema receipt for lowering one Weksa interlingua
|
||||
packet into one or more target-language utterance realizations.
|
||||
packet into one or more target-language utterance realizations and optional
|
||||
downstream target projections.
|
||||
|
||||
Executable schema code does not exist yet. When it does, it must reconcile with
|
||||
this document and
|
||||
|
|
@ -30,6 +31,9 @@ It combines:
|
|||
- `projected_context_ref`: source reference for the consumed projection.
|
||||
- `lowering_controls`: constraints shared by all target lowerers.
|
||||
- `realizations`: non-empty list of target-language outputs.
|
||||
- `target_projections`: optional downstream projections derived from accepted
|
||||
realizations, such as AquaSynth utterance embedding handoffs or MiMo TTS
|
||||
requests.
|
||||
|
||||
## Target Language
|
||||
|
||||
|
|
@ -88,3 +92,8 @@ Trace should identify:
|
|||
This packet may feed pronunciation planning or AquaSynth handoff work later, but
|
||||
it does not contain phoneme-parity, vocal-tract, reference-synth, or realtime
|
||||
voice artifacts.
|
||||
|
||||
MiMo TTS requests are allowed as derived target projections. They must reference
|
||||
an accepted realization, speak that realization's target-language text, and keep
|
||||
provider-native style tags out of source meaning. MiMo owns rendered audio;
|
||||
Weksa owns the delivery intent and trace used to create the request.
|
||||
|
|
|
|||
|
|
@ -12,9 +12,10 @@ objective:
|
|||
Create a durable workspace where Weksa can develop from a persistent map
|
||||
before runtime architecture decisions are made.
|
||||
current: >
|
||||
Emit concrete v0.1 utterance handoff artifacts for AquaSynth's first
|
||||
speech-training curriculum while keeping Weksa's ownership limited to
|
||||
inspected inputs.
|
||||
Treat written output, AquaSynth utterance embeddings, and MiMo expressive
|
||||
speech requests as sibling target projections from accepted interlingua
|
||||
lowerings while keeping Weksa's ownership limited to inspected intent,
|
||||
delivery controls, and trace.
|
||||
|
||||
constraints:
|
||||
- Runtime implementation language is undecided.
|
||||
|
|
@ -49,6 +50,9 @@ constraints:
|
|||
prosody/emphasis hints, and Ghostlight/Epiphany-shaped character-state
|
||||
vectors; AquaSynth owns the learned utterance embedding encoder, gradient
|
||||
descent, and synth-control decoding.
|
||||
- MiMo V2.5 TTS is an external speech-provider target. Weksa may project an
|
||||
accepted target-language realization plus delivery intent into MiMo style
|
||||
controls, but MiMo owns only provider rendering and audio output.
|
||||
- Large offline text-embedding jobs should use `bge-m3` on the neighboring
|
||||
Ollama inference box documented in `E:\Projects\gamecult-ops` rather than
|
||||
running on the local Weksa workspace by default.
|
||||
|
|
@ -81,6 +85,9 @@ data_flow:
|
|||
owns the target pronunciation layer
|
||||
- optionally encode the line as a learned expression vector for downstream
|
||||
synth control
|
||||
- optionally project an accepted realization into a provider-specific
|
||||
speech request such as MiMo TTS without making the provider an owner of
|
||||
source meaning
|
||||
- apply alien ontology and cultural salience
|
||||
- select grammatical commitments
|
||||
- derive morphology and lexical families
|
||||
|
|
@ -272,6 +279,10 @@ landed_baseline:
|
|||
English text semantics may use `bge-m3`; IPA strings and alien phonetic
|
||||
material belong in PanPhon sequence evidence that AquaSynth compresses into
|
||||
the phonetic realization embedding.
|
||||
- MiMo V2.5 TTS is accepted as a speech-provider target projection from
|
||||
Weksa utterance realizations. MiMo requests speak the accepted target text
|
||||
and receive Weksa-derived style guidance; backtranslations and provider
|
||||
tags do not become source meaning.
|
||||
- First Nibu interlingua-to-English lowering fixture.
|
||||
- First Nibu multilingual utterance lowering fixture at
|
||||
`examples/multilingual-lowering/nibu-proof-001/`, covering English,
|
||||
|
|
|
|||
Loading…
Reference in a new issue