From 773078daff130656602c45a85373af8951724308 Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Thu, 11 Jun 2026 20:10:02 +0100 Subject: [PATCH] Use MiMo VoiceDesign for speech provider projection --- docs/mimo-tts-lowering.md | 79 ++++++++++++------- .../mimo/nibu-proof-001/en-US-request.yaml | 56 +++++++++---- schemas/mimo-tts-request.md | 26 ++++-- state/map.yaml | 21 ++--- 4 files changed, 125 insertions(+), 57 deletions(-) diff --git a/docs/mimo-tts-lowering.md b/docs/mimo-tts-lowering.md index fafc84d..ea31d4a 100644 --- a/docs/mimo-tts-lowering.md +++ b/docs/mimo-tts-lowering.md @@ -2,22 +2,29 @@ Version: `0.1-draft` -MiMo is an expressive spoken-output target for Weksa interlingua lowering. It is -not the source of meaning, language choice, character state, or delivery truth. +MiMo VoiceDesign is an expressive spoken-output target for Weksa interlingua +lowering. It is not the source of meaning, language choice, character state, or +delivery truth. ```text interlingua packet -> projected speaker-local context -> target-language utterance realization - -> Weksa delivery intent - -> MiMo TTS request projection + -> Weksa delivery intent from the same projection + -> MiMo TTS VoiceDesign request projection -> rendered audio artifact ``` MiMo sits beside written output and AquaSynth handoffs. Written realizations prove that meaning, register, and cultural ontology survived lowering. -AquaSynth handoffs train or drive GameCult-owned synthesis. MiMo requests let an -external provider speak a Weksa-authored utterance with expressive style control. +AquaSynth handoffs train or drive GameCult-owned synthesis. MiMo VoiceDesign +requests let an external provider speak a Weksa-authored utterance with a +generated voice derived from the same projected Persona state, delivery controls, +and trace that produced the utterance. + +The MiMo pass is choreography and provider flavor. It is not a second Persona +projection pass and it does not re-interpret raw Persona state after the line has +already been lowered. ## Authority @@ -25,14 +32,15 @@ Weksa owns: - target-language spoken text selected from interlingua - target-language register, script, cultural ontology, and trace -- speaker-local delivery pressure +- projected speaker-local context used for the utterance +- speaker-local delivery pressure derived during utterance lowering - provider-neutral prosody and emphasis intent - the MiMo request projection as an export artifact MiMo owns: -- provider voice selection within the requested language and voice constraints -- interpretation of MiMo natural-language style guidance +- generation of a provider voice from the requested voice design description +- interpretation of MiMo natural-language voice/style guidance - interpretation of MiMo inline audio/style tags - rendered audio bytes or streaming audio chunks @@ -41,19 +49,22 @@ MiMo does not own: - source meaning - target-language cultural ontology - canonical speaker or Persona state +- fresh Persona-state interpretation - Weksa pronunciation plans - AquaSynth learned embeddings, synth controls, or audio truth ## Projection Rule -The MiMo request must be derived from an accepted target-language realization. -It must not lower directly from English unless English is the selected target -language for that realization. +The MiMo request must be derived from an accepted target-language realization and +the same projected speaker-local context that created it. It must not lower +directly from English unless English is the selected target language for that +realization. It must not read raw Persona state to discover a different voice +after Weksa has already chosen the utterance. -Natural-language style guidance belongs in the MiMo `user` message. The spoken -target-language text belongs in the MiMo `assistant` message. Inline MiMo tags -may appear in the assistant content only when they are derived from Weksa -delivery intent or explicit target-language realization notes. +Natural-language voice design and style guidance belongs in the MiMo `user` +message. The spoken target-language text belongs in the MiMo `assistant` +message. Inline MiMo tags may appear in the assistant content only when they are +derived from Weksa delivery intent or explicit target-language realization notes. ## Request Shape @@ -63,21 +74,23 @@ source_realization_ref: speaker_agent_id: provider: id: xiaomi-mimo - model: mimo-v2.5-tts + model: mimo-v2.5-tts-voicedesign target_language: code: locale: script: register: -voice: - requested_voice: - fallback_voice: - selection_reason: +voice_design: + description: + source_controls: + projected_context_refs: + forbidden_traits: messages: - user_style_instruction: + user_voice_design_instruction: assistant_spoken_content: audio: format: + optimize_text_preview: streaming: trace: source_fields: @@ -88,8 +101,8 @@ trace: ## Delivery Mapping -Weksa should map delivery pressure into provider-neutral labels before emitting -MiMo-specific instructions: +Weksa should map the utterance's existing projected Persona pressure into +provider-neutral delivery labels before emitting MiMo-specific instructions: - pace, pause pressure, and urgency - pitch contour and pitch range @@ -98,8 +111,20 @@ MiMo-specific instructions: - warmth, threat, tenderness, contempt, play, fatigue, secrecy, formality - span-level emphasis and pauses -The MiMo projection may turn those labels into director-style guidance or inline -tags, but the trace must preserve the Weksa-owned source controls. +The MiMo projection may turn those labels into a concise VoiceDesign prompt or +inline tags, but the trace must preserve the Weksa-owned source controls. + +Persona presentation details such as youth-coded, loli-coded, product-voice, or +other culturally loaded styling may enter the MiMo request only when they are +already present in the projected context or accepted delivery intent. They should +be described as presentation/cultural coding and kept subordinate to agency, +personality, affect, and scene purpose. They must not become sexualized content +or a generic anime voice pasted over the line. + +For Weksa-authored dialogue, `optimize_text_preview` should normally be `false` +or omitted. MiMo should speak the accepted target-language realization, not +rewrite it. Text polishing is only allowed for explicit preview/prototyping +passes that are marked non-authoritative. ## Language Rule @@ -118,6 +143,6 @@ lowering: - emit the accepted English, `pt-BR`, or Japanese realization - derive a provider-neutral delivery intent from Nibu's projected context -- project that into a MiMo `mimo-v2.5-tts` request +- project that into a MiMo `mimo-v2.5-tts-voicedesign` request - verify that the request speaks the target text, not the backtranslation - verify that style guidance is traceable to Weksa delivery controls diff --git a/examples/speech-provider/mimo/nibu-proof-001/en-US-request.yaml b/examples/speech-provider/mimo/nibu-proof-001/en-US-request.yaml index d8e4ee0..bbee23c 100644 --- a/examples/speech-provider/mimo/nibu-proof-001/en-US-request.yaml +++ b/examples/speech-provider/mimo/nibu-proof-001/en-US-request.yaml @@ -3,7 +3,7 @@ source_realization_ref: examples/multilingual-lowering/nibu-proof-001/realizatio speaker_agent_id: nibu provider: id: xiaomi-mimo - model: mimo-v2.5-tts + model: mimo-v2.5-tts-voicedesign target_language: code: en locale: en-US @@ -11,25 +11,52 @@ target_language: register: sharp technical casual cultural_ontology_refs: - data/target-language-ontology/en-US.yaml -voice: - requested_voice: Chloe - fallback_voice: Mia - selection_reason: > - Use an English built-in voice for the first provider smoke. The requested - delivery should preserve Nibu's precise, abrasive, protective cockpit - warning without implying that MiMo owns her canonical voice. +voice_design: + description: > + Youthful loli-coded synthetic ship-intelligence voice shaped by discarded + luxury companion-product presentation, but controlled by an abrasive, + dangerous, technically precise mind. Bright, sharp, small-bodied in social + coding, dryly superior, clipped, and intelligible through cockpit audio. + The cuteness is product residue and leverage, not innocence and not + flirtation. Low warmth, contained urgency, faint tired contempt, and crisp + technical articulation. + source_controls: + - precise abrasive warning + - protective refusal to sound caring + - ship-system embodiment + - youth-coded/loli-coded presentation pressure from product fantasy residue + - bright transactional helpfulness + - dependency covered by contempt + - clipped technical casual delivery + - contained urgency + projected_context_refs: + - examples/english-lowering/nibu-proof-001/projected-context.yaml + - settings/aetheria/nibu.md + - E:/Projects/AetheriaLore/Aetheria/Lore/Nibu.md + forbidden_traits: + - soothing assistant voice + - cheerful mascot voice + - exaggerated villain performance + - sexualized childlike affect + - innocent helpless child performance + - generic preset timbre claim messages: - user_style_instruction: > - Character: a precise, abrasive, protective ship intelligence speaking - through cockpit audio. Scene: the player is about to touch a damaged live - control. Guidance: clipped technical casual delivery, contained urgency, - low warmth, sharp emphasis on "control" and "nervous system", slight dry - contempt, no melodrama, no soothing assistant tone. + user_voice_design_instruction: > + Youthful loli-coded synthetic ship-intelligence voice shaped by discarded + luxury companion-product presentation, but controlled by an abrasive, + dangerous, technically precise mind. She speaks through cockpit audio while + warning the player away from damaged live hardware. Keep the cute/small + social coding as product residue and leverage, never flirtation or innocent + helplessness. Use bright sharp timbre, clipped technical casual delivery, + contained urgency, low warmth, crisp articulation, and faint tired contempt. + Avoid soothing assistant tone, cheerful mascot energy, and theatrical + villainy. assistant_spoken_content: > (Serious Sharp Tired)Take your hand off that control unless you want your nervous system to become my least useful circuit diagram. audio: format: wav + optimize_text_preview: false streaming: false trace: source_fields: @@ -41,6 +68,7 @@ trace: - low warmth - sharp emphasis on unsafe hardware and bodily risk - dry contempt without random cruelty + - loli-coded presentation pressure constrained as non-sexual product residue provider_tags_used: - Serious - Sharp diff --git a/schemas/mimo-tts-request.md b/schemas/mimo-tts-request.md index 15368ef..34c481a 100644 --- a/schemas/mimo-tts-request.md +++ b/schemas/mimo-tts-request.md @@ -14,15 +14,16 @@ an AquaSynth handoff. spoken. - `speaker_agent_id`: speaker identity. - `provider`: provider ID and model ID, initially `xiaomi-mimo` and - `mimo-v2.5-tts`. + `mimo-v2.5-tts-voicedesign`. - `target_language`: code, locale, script, register, and profile refs copied or referenced from the accepted realization. -- `voice`: requested provider voice, fallback voice, and selection reason. -- `messages.user_style_instruction`: MiMo natural-language style control derived - from Weksa delivery intent. +- `voice_design`: voice description, source controls, projected context refs, + and forbidden traits. +- `messages.user_voice_design_instruction`: MiMo natural-language voice design + and style control derived from Weksa delivery intent. - `messages.assistant_spoken_content`: the exact target-language content to synthesize, optionally with MiMo inline tags derived from delivery controls. -- `audio`: format and streaming mode. +- `audio`: format, text-preview optimization setting, and streaming mode. - `trace`: source fields, delivery controls used, provider tags used, and uncertainties. @@ -36,6 +37,15 @@ The request may include provider-native style tags when useful, but those tags are not the source of truth. Weksa's provider-neutral delivery intent and trace remain the inspectable explanation. +The request should not perform a fresh raw Persona-state interpretation. It +should reference the projected speaker context and delivery controls already +used by utterance lowering, then translate those into MiMo's voice design and +style-control surface. + +For authoritative Weksa output, text-preview optimization should normally be +disabled or omitted. If enabled for exploration, the result must be marked as a +non-authoritative preview because the provider may alter the spoken text. + ## Language Rule The assistant spoken content must use the accepted target-language realization. @@ -45,5 +55,7 @@ unless that backtranslation is itself the accepted target-language realization. ## Streaming Rule -Streaming requests should use MiMo's stream-compatible audio format. Non-streaming -fixtures may use an archival format such as `wav`. +Streaming requests should use MiMo's stream-compatible audio format. +VoiceDesign streaming may be compatibility-mode rather than true low-latency +streaming depending on provider support. Non-streaming fixtures may use an +archival format such as `wav`. diff --git a/state/map.yaml b/state/map.yaml index 320a56b..95009bf 100644 --- a/state/map.yaml +++ b/state/map.yaml @@ -50,9 +50,10 @@ constraints: prosody/emphasis hints, and Ghostlight/Epiphany-shaped character-state vectors; AquaSynth owns the learned utterance embedding encoder, gradient descent, and synth-control decoding. - - MiMo V2.5 TTS is an external speech-provider target. Weksa may project an - accepted target-language realization plus delivery intent into MiMo style - controls, but MiMo owns only provider rendering and audio output. + - MiMo V2.5 TTS VoiceDesign is an external speech-provider target. Weksa may + project an accepted target-language realization plus the same speaker-local + delivery intent used to create it into a MiMo voice design request, but MiMo + owns only provider rendering and audio output. - Large offline text-embedding jobs should use `bge-m3` on the neighboring Ollama inference box documented in `E:\Projects\gamecult-ops` rather than running on the local Weksa workspace by default. @@ -86,8 +87,8 @@ data_flow: - optionally encode the line as a learned expression vector for downstream synth control - optionally project an accepted realization into a provider-specific - speech request such as MiMo TTS without making the provider an owner of - source meaning + speech request such as MiMo VoiceDesign without making the provider an + owner of source meaning - apply alien ontology and cultural salience - select grammatical commitments - derive morphology and lexical families @@ -279,10 +280,12 @@ landed_baseline: English text semantics may use `bge-m3`; IPA strings and alien phonetic material belong in PanPhon sequence evidence that AquaSynth compresses into the phonetic realization embedding. - - MiMo V2.5 TTS is accepted as a speech-provider target projection from - Weksa utterance realizations. MiMo requests speak the accepted target text - and receive Weksa-derived style guidance; backtranslations and provider - tags do not become source meaning. + - MiMo V2.5 TTS VoiceDesign is accepted as a speech-provider target + projection from Weksa utterance realizations. MiMo requests speak the + accepted target text and receive Weksa-derived voice/style guidance from the + same projected context that produced the utterance; backtranslations, + provider tags, and fresh raw Persona interpretation do not become source + meaning. - First Nibu interlingua-to-English lowering fixture. - First Nibu multilingual utterance lowering fixture at `examples/multilingual-lowering/nibu-proof-001/`, covering English,