Tract: coarticulation fixes for the tract-weak-spots-1 listening verdict (#52)
Merged as head maintainer after Soul passes 2 and 3 (aqua-body:verdict:cut-tract-coarticulation-pr52.s2, ...fix2.s3). Unheard: every clip is a measurement, not a listening. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
71cf9caeb6
commit
ca6af1d678
19 changed files with 981 additions and 71 deletions
76
docs/listening/tract-weak-spots-1-analysis-20261113.md
Normal file
76
docs/listening/tract-weak-spots-1-analysis-20261113.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# tract-weak-spots-1: why the predictions failed, and what was changed
|
||||
|
||||
The verdict is `tract-weak-spots-1-meta-20261113.md`. Every falsifiable prediction in the kit failed although the formant numbers
|
||||
behind them were right. The kit measured the steady part of each vowel (35-85 % of the clip) and the peak of each burst; the faults
|
||||
Meta heard sit in the first 100 ms of every clip, at the phone boundaries, and in how long the burst lasts. Nothing in the old
|
||||
measurement set looked there. Everything below was measured on the clips Meta heard (`weak-kit-out/clips`, WordRender at 140 Hz) before
|
||||
any change, and again after.
|
||||
|
||||
## Causes found (measured, not guessed)
|
||||
|
||||
| # | what she heard | cause | evidence (before) |
|
||||
|---|---|---|---|
|
||||
| 1 | nibu: "both consonants are just clicks"; leak-before: "reversed percussive hit" | The merged control curve of a surface holds the previous phone's value to the boundary and steps to the next phone's first value there (two points at one time). A step in a ladder cell's area is a one-sample transient. | Pitch-period window peaks: 0.399 at 95 ms and 0.487 at 275 ms, against vowel pulses of ~0.1-0.2; both instants are phone boundaries of the script (`start=0.095`, `0.275`). The whole-render peak the kit reported (0.487) was this click. Click ratio 3.54 (`ear_metrics.py`). The old report called 0.487 vs 0.399 a "leak" difference. |
|
||||
| 2 | every clip starts as a fade-in; the "kazoo" with "no audible closure" (ba, no burst); the /ɪ/ not like "bit" | `si.smoo` starts every control slider at 0. A word began with the tract shut, the vowel weights at 0 and the source off, and climbed for ~100 ms. The 140 ms "closure" of ba was largely this climb. | Isolated /ɪ/: level -66 dB at 0 ms, -45 at 40 ms, -30 at 100 ms, -28 from 140 ms; LPC bandwidth of F1 367 Hz at 0 ms, 232 at 60 ms, 103 at 100 ms, 90 from 140 ms. 100 of the clip's 309 ms were the climb; the steady-state window (35-85 %) started after it. |
|
||||
| 3 | ba with burst: "fah then bah" | The stop opened over 0.65 to 1.0 of its gesture: 33 ms of a 95 ms /b/. A burst of that length is a fricative. | Burst-notes: ba burst 0.0707-0.1039 s (33 ms). A recogniser reads the clip as `v ɑ` (with the burst), `b ɑ` without. |
|
||||
| 4 | nasals: "basketball buzzer, no audible articulation", /n/ and /m/ alike | The murmur had no entry and exit transitions (the step of cause 1) and began after the climb of cause 2; the machine ear read /n/ as /m/ (`v ɑ m ɑ`). | recogniser, before: ana -> `v ɑ m ɑ`. |
|
||||
| 5 | bun with burst: "a truck releasing its pneumatics" | cause 3 (long burst), same mechanism. | burst 0.0721-0.0998 s (28 ms). |
|
||||
|
||||
## What was changed (PR "phone transitions", this branch)
|
||||
|
||||
1. `PatchScript.WithPhoneTransition`: every authored gesture spline starts, when an earlier gesture left the surface at another value,
|
||||
with a 30 ms ramp from the previous value (at most 0.4 of the gesture, and before its second point). The first phone of a word and
|
||||
surfaces no earlier phone touched are unchanged. Test: `PhoneTransitionTests.NoSurfaceStepsAtAPhoneBoundary` (worst step in 1 ms
|
||||
around each boundary of nibu: below 0.03; before, 0.77: the vowel's 1 against the nasal's released 0.23).
|
||||
2. A stop holds all its articulators shut until 12 ms before its end and opens in those 12 ms (`StopReleaseSeconds`, a design value);
|
||||
it used to open over 0.65 to 1 of its gesture. Test: `AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture`.
|
||||
3. `smoo_init` replaces `si.smoo` on every control-surface and spline-point slider of the emitted DSP: the smoother starts at its input
|
||||
and keeps si.smoo's pole (a 22.7 ms time constant), so live control moves as fast as before. Patch parameters keep `si.smoo`. The
|
||||
source's own 10 ms pitch-ratio smoother starts at its input too. Tests: `ControlSmoothersStartAtTheirInputNotAtZero` (text) and
|
||||
`ASmootherStartsAtItsInputAndMovesAtSiSmoosPace` (the emitted smoother, rendered).
|
||||
4. The word scaffold's vocal envelope gets a 12 ms attack (it was 0, so the now-instant start was itself a click: first-sample peak 0.22).
|
||||
5. The ramp into a stop (item 1) sweeps the contact through every aperture between open and shut, and the exporter's burst noise is
|
||||
`4 * opening * (1 - opening)` times a gain that was held through the whole gesture: each closing ramp was a 15-20 ms hiss before the
|
||||
stop (>4 kHz share of the 6 ms frames at 188 ms of /ata/: -1 dB, against -57 dB with the closure stepped). The burst gain now exists
|
||||
only from the stop's release start (`PatchScript.StopReleaseStartSeconds`, the one owner of the release time), one ramp either side.
|
||||
A stop's lips are shut (aperture 0), not at 0.04: the word-initial /b/ voice bar read -19.1 dB re the vowel after it and /p/ -30.1
|
||||
against the design -34 to -26 and -60 to -38; shut they read -27.7 and -38.8. Spline handle times follow their segment (a third of
|
||||
it), so the C# timeline and the DSP agree on the 12 ms opening (the lowering ignores handle times). Tests:
|
||||
`ClosingIntoAStopMakesNoHighFrequencyNoise`, `TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt`, `PlosiveClosureCarriesTheVoiceBarOnlyWhenVoiced`.
|
||||
6. `tools/formant-check/ear_metrics.py` (click ratio, largest sample step, LPC bandwidths, HNR, energy above 4 kHz) and
|
||||
`tools/formant-check/phone_probe.py` (a speech recogniser as a machine ear) so these faults are measured, not heard first by Meta.
|
||||
|
||||
### After (same words, same 140 Hz, `fix-out-c`)
|
||||
|
||||
| word | before | after |
|
||||
|---|---|---|
|
||||
| nibu_flat | click ratio 3.54, peaks 0.399 / 0.487; recogniser `i b u` | peaks at the boundaries 0.077 / 0.089; click ratio 1.79 (the vowel's own pulses); median sample step in the /b/ closure 0.0007 (0.0005 before; 0.0161 with the ramp hiss); recogniser `n i l t uː` as read before the closure-noise and shut-lips fixes (not re-run: no torch on this host; the /n/ is back, the /b/ reads as `l t`: see below) |
|
||||
| ba (burst) | recogniser `v ɑ`; burst 33 ms | `b ɑ`; opening 12 ms |
|
||||
| ba (no burst) | `b ɑ` | `m ɑ` |
|
||||
| ana / ama | `v ɑ m ɑ` / `v ɑ m ɑ` | `ɑ n ɑ` / `ɑ m ɑ` |
|
||||
| isolated /i/ | `h iː`; level climb 100 ms | `i`; level within 3 dB of the steady value by 20 ms |
|
||||
| isolated /ɪ/ | `h i` | `ʔ i` |
|
||||
|
||||
## Not fixed, and why (open until she listens)
|
||||
|
||||
- **Voice quality: "kazoo with a two-pack-a-day smoking habit" (the two clips she kept), "reedy, like Pink Trombone" (/ʊ/), "basketball
|
||||
buzzer" (nasals).** Her comments on the two keeps say the vowel is right and the voice is wrong. Candidates, none tested by ear:
|
||||
(a) the glottal source is a band-limited saw through a one-pole lowpass with no closed phase (`FaustExport.PrimitiveSourceExpression`),
|
||||
where a Rosenberg/LF pulse has the open and closed phases real fold flow has; (b) harmonics-to-noise ratio of the steady part is 13-15 dB
|
||||
(healthy voices ~20+), from the source's `0.08 * noise` term and the saw's aliasing; (c) LPC bandwidths of the steady vowel are 90 Hz (F1),
|
||||
130 (F2), 100 (F3) for /ɪ/: plausible, so the wall-loss hypothesis is weak (the F2 of /ɑ/, 29 Hz, and the F1 of /i/, 14-24 Hz, are LPC
|
||||
artefacts of a pole sitting on a harmonic). Settling this needs a listening kit that varies one of them at a time; the next kit does.
|
||||
- **/ɪ/ as the "eu" of cordon bleu.** The recogniser cannot judge this tract's vowel quality (every isolated vowel came back /i/). The
|
||||
confirmed fault in that clip is cause 2: a third of it was a muffled climb through bandwidths of 300 Hz. Whether the steady /ɪ/ then
|
||||
reads as "bit" is for her ear; lax vowels in isolation and without vowel-inherent spectral change are known to be misheard
|
||||
(Hillenbrand et al. 1995, Strange et al. 1983), so the next kit also carries /ɪ/ in a word.
|
||||
- **The medial /b/ of nibu reads as `l t` to the recogniser after the fix, where it read `b` with the click and the long burst.** The closure
|
||||
voice bar sat 10 dB under the vowel (a murmur-like level; a stop's runs 20+ dB under; 16-19 dB with the lips shut), and a labial burst is injected at the glottal
|
||||
end in a word with no other plosive (`FaustExport` burst injection), so it is filtered by the whole tract like aspiration. Both are
|
||||
next, with the listening to decide.
|
||||
|
||||
## What the old measurements missed, as a rule
|
||||
|
||||
A steady-state window measures a steady state. Faults of onset, boundary and duration need metrics that look at onsets, boundaries
|
||||
and durations: click ratio and step size at each phone boundary, time-to-full-level, burst length, bandwidths, HNR. These now exist
|
||||
(`ear_metrics.py`) and the click and onset ones are pinned by tests.
|
||||
231
docs/listening/tract-weak-spots-1-meta-20261113.json
Normal file
231
docs/listening/tract-weak-spots-1-meta-20261113.json
Normal file
|
|
@ -0,0 +1,231 @@
|
|||
{
|
||||
"schema": "aqua.listen.verdict.v1",
|
||||
"kit": {
|
||||
"id": "tract-weak-spots-1",
|
||||
"title": "Articulatory vocal tract: stress, release, nasals, lax vowels"
|
||||
},
|
||||
"exportedAt": "2026-11-13T11:27:36.539Z",
|
||||
"seed": 1326320895,
|
||||
"consent": true,
|
||||
"listener": "Meta",
|
||||
"device": null,
|
||||
"notes": "",
|
||||
"summary": {
|
||||
"total": 14,
|
||||
"keep": 2,
|
||||
"fix": 12,
|
||||
"drop": 0,
|
||||
"unanswered": 0
|
||||
},
|
||||
"clips": [
|
||||
{
|
||||
"position": 1,
|
||||
"id": "lax-ih",
|
||||
"title": "isolated /ɪ/ 180 ms, 140 Hz. Measured: Burg F1/F2 375/2024 Hz; tube chain 346/1987 (+8.5/+1.8 %); Hillenbrand 1995 male 429/2034 (-12.5/-0.5 %); Peterson & Barney 392/1993 (-4.3/+1.5 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'bit' (or a slightly high, 'ee'-leaning version of it). FALSIFIED if heard as 'ee' of 'beet', as 'eh' of 'bet', or as a different phone.",
|
||||
"group": "lax",
|
||||
"feel": "the short vowel in 'bit'",
|
||||
"file": "v_ih.wav",
|
||||
"pair": null,
|
||||
"slot": null,
|
||||
"verdict": "fix",
|
||||
"reason": "It's more like the eu in cordon bleu than the i in bit ",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 2,
|
||||
"id": "burst-ba-on",
|
||||
"title": "ba with the plosive burst (gain 0.06, a design value). Measured: burst alone peaks 0.0142 at 85 ms = -24.0 dB re vowel peak, falls to a tenth in 17 ms, burst rms in the 30 ms after its peak -31.0 dB re vowel rms. PREDICTION: a slightly crisper, more 'b'-like onset than B/off; the burst is quiet and short so the difference is small, 'a little pop' at most. FALSIFIED if it sounds like a hiss, a click, or a separate noise after the vowel started, or if no difference is heard.",
|
||||
"group": "burst",
|
||||
"feel": "the syllable 'bah' with a clean, crisp 'b' at the start",
|
||||
"file": "ba.wav",
|
||||
"pair": "burst-ba-off",
|
||||
"slot": "A",
|
||||
"verdict": "fix",
|
||||
"reason": "More like fah then bah but you can hear something there",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 3,
|
||||
"id": "burst-ba-off",
|
||||
"title": "ba with the release_flow splines dropped (WordRender --no-burst). Measured: closure peak 0.0078 (voice bar), release (voicing breaking through)peak 0.1087 = -6.3 dB re vowel peak 0.2247, no burst noise. PREDICTION: a soft, voiced 'b' with the vowel starting smoothly; may sound slightly 'w'-like or lack the pop. FALSIFIED if listeners cannot hear any difference from the burst twin (then burst gain 0.06 is inaudible).",
|
||||
"group": "burst",
|
||||
"feel": "the syllable 'bah' with a soft start",
|
||||
"file": "ba_noburst.wav",
|
||||
"pair": "burst-ba-on",
|
||||
"slot": "B",
|
||||
"verdict": "fix",
|
||||
"reason": "There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 4,
|
||||
"id": "stress-nibu-flat",
|
||||
"title": "nibu, no stress marks. Measured: /i/ 150-250 ms rms vs the stressed render +0.21 dB; unstressed vowel 180 ms, f0 139.9 Hz. PREDICTION: reads asan even two-syllable word with no strong first syllable. FALSIFIED if listeners hear it as stressed on a syllable or cannot tell it from the stressed twin.",
|
||||
"group": "stress",
|
||||
"feel": "a made-up two-syllable word, 'nee-boo', both syllables even",
|
||||
"file": "nibu_flat.wav",
|
||||
"pair": "stress-nibu",
|
||||
"slot": "A",
|
||||
"verdict": "fix",
|
||||
"reason": "These both sound like 'ee'oo, with both consonants you're going for being just clicks",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 5,
|
||||
"id": "stress-nibu",
|
||||
"title": "nibu with primary stress on /i/ (NI-bu). Measured: /i/ 247 ms vs 180 ms (+75 ms; design +68), f0 148.1 vs 139.9 Hz (x1.059), amplitude factor 1.30; level of /i/ in the 150-250 ms window only +0.21 dB (five-vowel mean +1.93 dB, per vowel +1.2 to +3.0). PREDICTION: first syllable heard as the prominent one, mostly through length and pitch, little through loudness; B should be picked as 'stressed on the first syllable'. FALSIFIED if it sounds no more prominent than the flat twin, if it sounds like a question or a pitch accent rather than stress, or if the level is what is heard (then the f0-scatter question is wrong).",
|
||||
"group": "stress",
|
||||
"feel": "the same made-up word, with the first syllable stronger: 'NEE-boo'",
|
||||
"file": "nibu.wav",
|
||||
"pair": "stress-nibu-flat",
|
||||
"slot": "B",
|
||||
"verdict": "fix",
|
||||
"reason": "",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 6,
|
||||
"id": "stress-i-flat",
|
||||
"title": "isolated /i/ 180 ms at 140 Hz, no stress. Measured: level 100-170 ms is the reference. PREDICTION: plain short vowel. FALSIFIED if it already sounds emphasised.",
|
||||
"group": "stress",
|
||||
"feel": "a single short 'ee' vowel, plain",
|
||||
"file": "v_i.wav",
|
||||
"pair": "stress-i",
|
||||
"slot": "A",
|
||||
"verdict": "keep",
|
||||
"reason": "Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee",
|
||||
"landed": 4,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 7,
|
||||
"id": "stress-i",
|
||||
"title": "isolated /i/ with primary stress: 247 ms, f0 x1.06, amplitude 1.30. Measured against the flat /i/: +1.22 dB total (f0 rise alone -0.80 dB, the amplitude and the rest +2.02 dB). PREDICTION: heard as longer and a little higher; the 1.2 dB louder is under what a listener can judge, so 'longer and higher' is the answer, 'louder' is not. FALSIFIED if it sounds the same length and pitch, or clearly louder.",
|
||||
"group": "stress",
|
||||
"feel": "the same 'ee' vowel, held longer and said with emphasis",
|
||||
"file": "v_i_s.wav",
|
||||
"pair": "stress-i-flat",
|
||||
"slot": "B",
|
||||
"verdict": "keep",
|
||||
"reason": "",
|
||||
"landed": 4,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 8,
|
||||
"id": "lax-oo-short",
|
||||
"title": "isolated /ʊ/ 180 ms, 140 Hz. Measured: Burg F1/F2 486/894 Hz; tube chain 478/850 (+1.7/+5.2 %); Hillenbrand 469/1123 (+3.7/-20.4 %); Peterson &Barney 437/1023 (+11.3/-12.6 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'book', probably darker or rounder than a natural one (F2 20 % under the population mean, the back-vowel gap). FALSIFIED if heard as 'oo' of 'boot', 'o' of 'boat' or 'uh' of 'but'.",
|
||||
"group": "lax",
|
||||
"feel": "the short vowel in 'book'",
|
||||
"file": "v_oo_short.wav",
|
||||
"pair": null,
|
||||
"slot": null,
|
||||
"verdict": "fix",
|
||||
"reason": "It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared",
|
||||
"landed": 3,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 9,
|
||||
"id": "leak-nibu-flat-after",
|
||||
"title": "nibu (flat) after the fix: release_flow is zero outside its own plosive. Same audio as stress-nibu-flat, kept as its own clip so the before/after pair is one-to-one. Measured: leak 0.0 from 275 ms on, burst of the b itself unchanged (bin 0.0226 at 81 ms before and after). PREDICTION: clean hum into thevowel, no noisy onset; preferred over 'before' if the leak is audible. FALSIFIED if 'before' is preferred or no difference is heard.",
|
||||
"group": "release-leak",
|
||||
"feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'",
|
||||
"file": "nibu_flat_after.wav",
|
||||
"pair": "leak-nibu-flat-before",
|
||||
"slot": "A",
|
||||
"verdict": "fix",
|
||||
"reason": "Just slightly less reverse attack on this one",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 10,
|
||||
"id": "leak-nibu-flat-before",
|
||||
"title": "nibu (flat) rendered at main c0e6b98, before the release_flow fix. Measured: burst-gain leak vs the fixed render, peak 0.0688 and rms 0.0067 inthe leading /n/ (0-95 ms; the /n/ itself is rms 0.0058, so the hiss is louder than the murmur), peak 0.0504 over /i/ onset 95-150 ms, 0.0019 after; whole-render peak 0.487 vs 0.399. PREDICTION: an audible hiss or click as the /n/ opens into the vowel, the first sound of the word being a noisy 'sn'-like or 'tsh' onset instead of a clean nasal hum. FALSIFIED if the two are indistinguishable (then the leak is inaudible and the fix is cosmetic).",
|
||||
"group": "release-leak",
|
||||
"feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'",
|
||||
"file": "nibu_flat_before.wav",
|
||||
"pair": "leak-nibu-flat-after",
|
||||
"slot": "B",
|
||||
"verdict": "fix",
|
||||
"reason": "'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 11,
|
||||
"id": "nasal-n",
|
||||
"title": "/ɑnɑ/. Measured: murmur -9.64 dB re the flanking vowels (2.17 dB above /m/; the gap is 2.01/2.17/2.05/1.61 dB at f0 110/140/170/200 Hz, so not harmonic sampling), Burg N1 332 Hz, bands 38.4/28.2/16.8/0.9 dB. PREDICTION: heard as 'ah-nah', a tongue-tip hum, a shade stronger than the /m/ one; the 2 dB isnot expected to be noticed. FALSIFIED if heard as 'ama' or 'anga', if it sounds much louder than the /m/ clip, or if there is hiss.",
|
||||
"group": "nasal",
|
||||
"feel": "'ah-nah': the tongue touches behind the teeth for a soft hum between two open vowels",
|
||||
"file": "ana.wav",
|
||||
"pair": null,
|
||||
"slot": null,
|
||||
"verdict": "fix",
|
||||
"reason": "It just sounds like a basket ball buzzer, no audible articulation",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 12,
|
||||
"id": "burst-bun-off",
|
||||
"title": "bun, /bʊn/, no burst. Measured: closure peak 0.0043, release peak 0.1074 = -5.9 dB re vowel peak 0.2126. PREDICTION: soft 'b' then a short backvowel then a nasal; the final /n/ is a clean hum (no leak since the fix). FALSIFIED as for burst-ba-off.",
|
||||
"group": "burst",
|
||||
"feel": "a short word like 'bunn' or 'boon' said quickly, soft start",
|
||||
"file": "bun_noburst.wav",
|
||||
"pair": "burst-bun-on",
|
||||
"slot": "A",
|
||||
"verdict": "fix",
|
||||
"reason": "Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 13,
|
||||
"id": "burst-bun-on",
|
||||
"title": "bun, /bʊn/, with burst. Measured: burst peak 0.0130 at 81 ms = -24.3 dB re vowel peak, tenth in 15 ms; tail leak after the fix 0.0 (was 0.0126 at the end before). PREDICTION: slightly crisper onset than off; final /n/ clean. FALSIFIED as for burst-ba-on.",
|
||||
"group": "burst",
|
||||
"feel": "a short word like 'bunn' or 'boon' said quickly, crisp 'b' start",
|
||||
"file": "bun.wav",
|
||||
"pair": "burst-bun-off",
|
||||
"slot": "B",
|
||||
"verdict": "fix",
|
||||
"reason": "The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
},
|
||||
{
|
||||
"position": 14,
|
||||
"id": "nasal-m",
|
||||
"title": "/ɑmɑ/. Measured: murmur -11.81 dB re the flanking vowels (design band -16..-8), Burg N1 295 Hz, band levels 0-400/400-800/800-1600/1600-3200 Hzof the murmur 36.3/23.6/21.3/2.7 dB. PREDICTION: heard as 'ah-mah', a lip-closed hum between the vowels, clearly softer than the vowels. FALSIFIED if heard as 'abah' or 'awah', if the hum is as loud as the vowels, or if there is a gap or click.",
|
||||
"group": "nasal",
|
||||
"feel": "'ah-mah': lips close for a soft hum between two open vowels",
|
||||
"file": "ama.wav",
|
||||
"pair": null,
|
||||
"slot": null,
|
||||
"verdict": "fix",
|
||||
"reason": "That m is almost coming through, still more of a basketball buzzer",
|
||||
"landed": 2,
|
||||
"device": null
|
||||
}
|
||||
],
|
||||
"pairs": [
|
||||
{
|
||||
"a": "burst-ba-on",
|
||||
"b": "burst-ba-off",
|
||||
"preferred": "burst-ba-on"
|
||||
},
|
||||
{
|
||||
"a": "stress-nibu-flat",
|
||||
"b": "stress-nibu",
|
||||
"preferred": "no-difference"
|
||||
}
|
||||
],
|
||||
"_truncated": "the mail reader cut the body at 12000 characters inside the pairs list; pairs after the third are not recorded"
|
||||
}
|
||||
32
docs/listening/tract-weak-spots-1-meta-20261113.md
Normal file
32
docs/listening/tract-weak-spots-1-meta-20261113.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Listening verdict: tract-weak-spots-1 (page tract-listen-1b6d180)
|
||||
|
||||
Listener: Meta (operator), 2026-11-13. Exported 11:27Z by the page (`aqua.listen.verdict.v1`, seed 1326320895), pasted in
|
||||
#aquasynth (truncated after clip 5) and emailed to aqua@minds.gamecult.org at 11:30Z; the data here is the email,
|
||||
parsed whole by `mind-mail read` except the tail: the mail reader cut the body at 12000 characters inside `pairs`, so
|
||||
only the first two pair preferences survive. Raw data: `tract-weak-spots-1-meta-20261113.json`. Totals: 2 keep, 12 fix, 0 drop.
|
||||
`position` is the page's random order for this listener; ids are the kit's.
|
||||
|
||||
| # | clip | verdict | what she heard |
|
||||
|---|---|---|---|
|
||||
| 1 | `lax-ih` (`v_ih.wav`) | fix | It's more like the eu in cordon bleu than the i in bit |
|
||||
| 2 | `burst-ba-on` (`ba.wav`) | fix | More like fah then bah but you can hear something there |
|
||||
| 3 | `burst-ba-off` (`ba_noburst.wav`) | fix | There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo |
|
||||
| 4 | `stress-nibu-flat` (`nibu_flat.wav`) | fix | These both sound like 'ee'oo, with both consonants you're going for being just clicks |
|
||||
| 5 | `stress-nibu` (`nibu.wav`) | fix | (no comment) |
|
||||
| 6 | `stress-i-flat` (`v_i.wav`) | keep | Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee |
|
||||
| 7 | `stress-i` (`v_i_s.wav`) | keep | (no comment) |
|
||||
| 8 | `lax-oo-short` (`v_oo_short.wav`) | fix | It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared |
|
||||
| 9 | `leak-nibu-flat-after` (`nibu_flat_after.wav`) | fix | Just slightly less reverse attack on this one |
|
||||
| 10 | `leak-nibu-flat-before` (`nibu_flat_before.wav`) | fix | 'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant |
|
||||
| 11 | `nasal-n` (`ana.wav`) | fix | It just sounds like a basket ball buzzer, no audible articulation |
|
||||
| 12 | `burst-bun-off` (`bun_noburst.wav`) | fix | Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there |
|
||||
| 13 | `burst-bun-on` (`bun.wav`) | fix | The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human |
|
||||
| 14 | `nasal-m` (`ama.wav`) | fix | That m is almost coming through, still more of a basketball buzzer |
|
||||
|
||||
Pairs: `burst-ba-on` preferred over `burst-ba-off`; `stress-nibu-flat` vs `stress-nibu`: no difference.
|
||||
(`leak-nibu-flat-before`/`after` and the bun pair carry no recorded preference; she heard "just slightly less reverse
|
||||
attack" on the fixed one, the pneumatic-truck "air release" on the bun burst.)
|
||||
|
||||
Every prediction that was falsifiable failed, though the formant numbers behind them were measured correctly. The
|
||||
two keeps are the isolated /i/ pair, and even those were "a kazoo with a two pack a day smoking habit". Analysis and
|
||||
the fixes it led to: `tract-weak-spots-1-analysis-20261113.md`.
|
||||
|
|
@ -45,6 +45,15 @@ public static partial class FaustEmitter
|
|||
source.AppendLine("bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;");
|
||||
source.AppendLine("seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);");
|
||||
source.AppendLine("smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));");
|
||||
// si.smoo starts every control at 0 and climbs to it: a word began with the tract shut, the vowel weights at 0 and the source off,
|
||||
// and the first ~100 ms of every render were that climb (the level rose 40 dB over the first 100 ms of an isolated vowel, and
|
||||
// the bandwidths fell from ~300 Hz to 90). smoo_from starts at its input and smooths from there. smoo_init is si.smoo's own
|
||||
// pole (1 - 44.1 / SR, a 22.7 ms time constant), so live control moves exactly as fast as before; only the start differs.
|
||||
// Started at its input a constant control holds there to the bit; started at 0 (si.smoo, si.smooth) it stalls 1e-5 to 5e-5 short of it,
|
||||
// in either way of writing the step. The source's pitch-ratio smoother starts at its input for that reason: from 0 the 2744 Hz alias
|
||||
// source ran 0.12 Hz flat and leaked its harmonics into the alias measurement.
|
||||
source.AppendLine("smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };");
|
||||
source.AppendLine("smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);");
|
||||
source.AppendLine("seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));");
|
||||
source.AppendLine("seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));");
|
||||
source.AppendLine("seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));");
|
||||
|
|
@ -152,7 +161,7 @@ public static partial class FaustEmitter
|
|||
var baseId = $"{id}_base";
|
||||
var normalized = $"{id}_normalized";
|
||||
var physical = $"{id}_physical";
|
||||
var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : si.smoo";
|
||||
var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : smoo_init";
|
||||
var boundControl = parameters.TryBindingExpression(surface.FieldPath, out var bindingExpression)
|
||||
? $"clip01(({bindingExpression} - {F(surface.MinValue)}) / max(0.000001, {F(surface.MaxValue - surface.MinValue)}))"
|
||||
: publicControl;
|
||||
|
|
@ -172,7 +181,7 @@ public static partial class FaustEmitter
|
|||
var splineId = $"{id}_spline_{splineIndex}";
|
||||
EmitControlSpline(source, spline, splineId, surface.DefaultNormalized);
|
||||
var depth = $"{splineId}_depth";
|
||||
source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : si.smoo;");
|
||||
source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : smoo_init;");
|
||||
terms.Add($"(({splineId}_value - {F(surface.DefaultNormalized)}) * {depth})");
|
||||
}
|
||||
|
||||
|
|
@ -197,11 +206,11 @@ public static partial class FaustEmitter
|
|||
{
|
||||
var point = points[i];
|
||||
source.AppendLine($"{splineId}_p{i}_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/time\", {F(point.TimeSeconds)}, 0, 120, 0.001);");
|
||||
source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : si.smoo;");
|
||||
source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : smoo_init;");
|
||||
source.AppendLine($"{splineId}_p{i}_out_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_time\", {F(Math.Max(0, point.OutTimeOffsetSeconds))}, 0, 10, 0.001);");
|
||||
source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : si.smoo;");
|
||||
source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : smoo_init;");
|
||||
source.AppendLine($"{splineId}_p{i}_in_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_time\", {F(Math.Max(0, point.InTimeOffsetSeconds))}, 0, 10, 0.001);");
|
||||
source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : si.smoo;");
|
||||
source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : smoo_init;");
|
||||
}
|
||||
|
||||
if (points.Length == 1)
|
||||
|
|
@ -739,7 +748,9 @@ public static partial class FaustEmitter
|
|||
var impedance = parameters.Expression(OwnerField(owner, "impedance"), port.Impedance);
|
||||
var flowScale = parameters.Expression(OwnerField(owner, "flow_scale"), port.FlowScale);
|
||||
// f0 is the voice frequency times the source's ratio (the stress cue), smoothed over 10 ms so a step in the ratio is a glide.
|
||||
var pitchRatio = $"({parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)} : si.smooth(ba.tau2pole(0.01)))";
|
||||
// Its own 10 ms smoother also starts at its input: from 0 it glided the source's pitch up from nothing at the start of every word
|
||||
// (the control feeding it used to hide that under its own 100 ms climb) and rounded to a value up to ~4e-5 off its input.
|
||||
var pitchRatio = $"smoo_from(ba.tau2pole(0.01), {parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)})";
|
||||
var load = $"{voiceName}_primitive_source_{safe}_load_pressure";
|
||||
var voiced = $"{voiceName}_primitive_source_{safe}_voiced";
|
||||
var flow = $"{voiceName}_primitive_source_{safe}_flow";
|
||||
|
|
|
|||
|
|
@ -278,7 +278,7 @@ public static class IpaGestureExperiment
|
|||
probe_timeline name=flow network=voice blocks=12 block_size=64
|
||||
{{SideStubDeclarations(sideStub)}}vocal_network name=voice paths=oral_path,nasal_path{{(sideStub ? ",side_stub_path" : "")}} sources=folds contacts=contact branches=velopharynx{{(sideStub ? "," + PatchScript.SideStubBranch : "")}} radiation=mouth,nostril{{(sideStub ? ",side_stub_end" : "")}} probe=flow
|
||||
{{string.Join("\n", gestureLines)}}
|
||||
vocal network=voice freq={{Format(frequency)}} gain=.7 sustain={{Format(sustain)}}
|
||||
vocal network=voice freq={{Format(frequency)}} gain=.7 attack=.012 sustain={{Format(sustain)}}
|
||||
""";
|
||||
|
||||
private static IReadOnlyList<IpaGestureMetricSummary> SummarizeMetrics(IReadOnlyList<IpaGestureExperimentMetric> metrics) =>
|
||||
|
|
|
|||
|
|
@ -17,6 +17,22 @@ public static class PatchScript
|
|||
/// A script that gestures a lateral without declaring it is refused, not approximated.</summary>
|
||||
public const string SideStubBranch = "side_stub";
|
||||
|
||||
// Seconds an articulator takes to move from where the previous phone left it to this phone's first value. A tract does not step
|
||||
// between phones: the merged surface curve held the previous value up to the boundary and jumped, and a jump in a ladder cell's
|
||||
// area is a transient the ear hears as a click (nibu: peak 0.40 and 0.49 against a 0.2 vowel, at the 95 and 275 ms boundaries),
|
||||
// and it leaves the consonant without the formant transitions that tell /b/ from a fade-in. A DESIGN VALUE inside the 20-50 ms
|
||||
// of a CV formant transition; no measurement of this tract retrieved.
|
||||
public const float PhoneTransitionSeconds = 0.03f;
|
||||
|
||||
// Seconds a stop takes to open from full closure; its burst lasts about this long. A DESIGN VALUE inside the 5-15 ms of a
|
||||
// stop burst; no measurement of this tract retrieved. The opening's noise is still shaped by the exporter.
|
||||
private const float StopReleaseSeconds = 0.012f;
|
||||
|
||||
/// <summary>Seconds after the start of a stop gesture of the given duration at which its articulators begin to open (the stop is
|
||||
/// shut until then). The one owner of the release time: the contact, lip and constriction splines hold shut to here and the burst
|
||||
/// gain window opens here; a test that wants the release reads it from here.</summary>
|
||||
public static float StopReleaseStartSeconds(float durationSeconds) => durationSeconds - MathF.Min(StopReleaseSeconds, 0.3f * durationSeconds);
|
||||
|
||||
public static bool TryIpaDescriptor(string ipa, out string descriptor)
|
||||
{
|
||||
if (StoryVowelAreas.TryVowelForIpa(ipa, out var vowelDescriptor, out _))
|
||||
|
|
@ -1494,11 +1510,58 @@ public static class PatchScript
|
|||
duration * 0.25f,
|
||||
Math.Clamp(point.value, 0, 1)))
|
||||
.ToArray();
|
||||
splinePoints = FollowSegments(WithPhoneTransition(surfacePath, start, duration, splinePoints));
|
||||
var spline = new ControlSpline(splineName, surfacePath, splinePoints, ControlSplineInterpolation.Bezier);
|
||||
_controlSplines.Add(spline);
|
||||
_controlSplinesByName[spline.Name] = spline;
|
||||
}
|
||||
|
||||
|
||||
|
||||
// Prepends the boundary ramp: a point at the gesture start holding the value the surface has there, then the gesture's first
|
||||
// point PhoneTransitionSeconds later (at most 0.4 of the gesture, and before its second point). The first phone of a word, and
|
||||
// a surface the earlier phones never touched, keep their value from the start.
|
||||
private ControlSplinePoint[] WithPhoneTransition(string surfacePath, float start, float duration, ControlSplinePoint[] points)
|
||||
{
|
||||
ControlSplinePoint? previous = null;
|
||||
foreach (var spline in _controlSplines)
|
||||
{
|
||||
if (!spline.SurfacePath.Equals(surfacePath, StringComparison.OrdinalIgnoreCase)) continue;
|
||||
foreach (var point in spline.Points)
|
||||
{
|
||||
if (point.TimeSeconds <= start + 0.000001f && (previous is null || point.TimeSeconds >= previous.TimeSeconds)) previous = point;
|
||||
}
|
||||
}
|
||||
|
||||
if (previous is null || points.Length < 2 || MathF.Abs(previous.Value - points[0].Value) <= 0.0005f) return points;
|
||||
var ramp = MathF.Min(PhoneTransitionSeconds, 0.4f * duration);
|
||||
ramp = MathF.Min(ramp, points[1].TimeSeconds - points[0].TimeSeconds);
|
||||
if (ramp < 0.002f) return points;
|
||||
var handle = ramp / 3;
|
||||
var first = points[0];
|
||||
var lead = new ControlSplinePoint(first.TimeSeconds, previous.Value, handle, previous.Value, handle, previous.Value);
|
||||
var arrival = first with { TimeSeconds = first.TimeSeconds + ramp, InTimeOffsetSeconds = handle, InValue = first.Value };
|
||||
return [lead, arrival, .. points.Skip(1)];
|
||||
}
|
||||
|
||||
// Sets every handle's time to a third of the segment it belongs to. The Faust lowering evaluates a segment's Bezier on the
|
||||
// linear fraction of its time and ignores the handle times; a handle of a third makes the C# timeline (which solves the curve's
|
||||
// time for its parameter) agree with it exactly. A handle longer than the segment (0.25 of the gesture on a 12 ms opening) folds
|
||||
// the C# curve back on itself.
|
||||
private static ControlSplinePoint[] FollowSegments(ControlSplinePoint[] points)
|
||||
{
|
||||
if (points.Length < 2) return points;
|
||||
var shaped = new ControlSplinePoint[points.Length];
|
||||
for (var i = 0; i < points.Length; i++)
|
||||
{
|
||||
var after = (i + 1 < points.Length ? points[i + 1].TimeSeconds - points[i].TimeSeconds : points[i].TimeSeconds - points[i - 1].TimeSeconds) / 3;
|
||||
var before = (i > 0 ? points[i].TimeSeconds - points[i - 1].TimeSeconds : points[1].TimeSeconds - points[0].TimeSeconds) / 3;
|
||||
shaped[i] = points[i] with { InTimeOffsetSeconds = before, OutTimeOffsetSeconds = after };
|
||||
}
|
||||
|
||||
return shaped;
|
||||
}
|
||||
|
||||
private string UniqueSplineName(string name)
|
||||
{
|
||||
var safe = SafeName(name);
|
||||
|
|
@ -1561,18 +1624,25 @@ public static class PatchScript
|
|||
// closed-mouth sound, and an alveolar closure that began to open at 0.5 let the oral tract radiate again over the last third (/n/ read
|
||||
// -3.8 dB under its vowels against /m/ -13.2 dB, whose lips hold to 0.7).
|
||||
var stopsBehindTheLips = (tokens.Contains("plosive") || tokens.Contains("stop")) && !(tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial"));
|
||||
// A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (0.65): at the nasal's 0.03 cm
|
||||
// A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (StopReleaseStartSeconds): at the nasal's 0.03 cm
|
||||
// aperture a /t/ closure measured -10.4 dB re its vowels against /p/ -19.7 (the lips seal /p/ whatever the tongue does), and
|
||||
// rendered -21.5 dB with the aperture at 0 (area floor 1e-6 cm^2).
|
||||
var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : stopsBehindTheLips ? 0.65f : 0.5f;
|
||||
// Every stop holds all its articulators shut until StopReleaseSeconds before its end and opens in that time: the burst is the
|
||||
// opening, and an opening spread over a third of the gesture (33 ms of the 95 ms /b/) made the burst a fricative's length
|
||||
// (the phone recogniser heard /v/ before the vowel, Meta heard "fah then bah").
|
||||
var isStop = tokens.Contains("plosive") || tokens.Contains("stop");
|
||||
var stopHold = StopReleaseStartSeconds(duration) / duration;
|
||||
var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : isStop ? stopHold : 0.5f;
|
||||
var closed = stopsBehindTheLips ? 0f : constriction;
|
||||
AddAuthoredSpline($"{name}_constriction_diameter", "/vocal/areas/0/area/constriction_diameter", start, duration, (0, closed), (closureHold, closed), (1, Math.Min(0.65f, constriction + 0.2f)));
|
||||
|
||||
if (tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial") || tokens.Contains("rounded"))
|
||||
{
|
||||
var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : 0.04f;
|
||||
AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.45f));
|
||||
AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.75f));
|
||||
// A stop's lips are shut, not at a 0.04 leak: at 0.04 the closed-mouth voice bar of /b/ read -19.1 dB re the following vowel (the
|
||||
// test's design value is -34 to -26) and /p/ -30.1 (-60 to -38); shut, -27.7 and -38.8. A nasal keeps its murmur's small opening.
|
||||
var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : isStop ? 0f : 0.04f;
|
||||
AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.45f));
|
||||
AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.75f));
|
||||
}
|
||||
|
||||
if (tokens.Contains("nasal"))
|
||||
|
|
@ -1582,13 +1652,18 @@ public static class PatchScript
|
|||
}
|
||||
else if (tokens.Contains("plosive") || tokens.Contains("stop"))
|
||||
{
|
||||
AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (0.65f, 0.02f), (1, 0.9f));
|
||||
AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (0.65f, 0.75f), (1, 0.05f));
|
||||
AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (stopHold, 0.02f), (1, 0.9f));
|
||||
AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (stopHold, 0.75f), (1, 0.05f));
|
||||
// The release burst is turbulence at the opening constriction: its gain is held for the plosive and the exporter shapes
|
||||
// it by the stored pressure and the opening (see PrimitiveContactExpression). The gain exists only around this plosive:
|
||||
// a surface holds its last value after its last point (and its first before its first), so a gain left at its level
|
||||
// made noise at every later contact state, the final nasal's closure and the tail included.
|
||||
GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start, duration, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain);
|
||||
// The exporter's burst is noise times 4 * opening * (1 - opening): it sounds whenever the contact passes between shut and open
|
||||
// in either direction, so a gain held through the closure made the 30 ms closing ramp a 15-20 ms hiss before every stop (the
|
||||
// >4 kHz share of the frames before /t/ was -1 dB against -57 dB with the closure stepped). The gain exists only from the
|
||||
// stop's release start to its end, one ramp (0.15 of that window) either side.
|
||||
var releaseStart = StopReleaseStartSeconds(duration);
|
||||
GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start + releaseStart, duration - releaseStart, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain);
|
||||
PlaceBurst(name, tokens, place, start, duration);
|
||||
AddAuthoredSpline($"{name}_velum", "/vocal/branches/0/opening", start, duration, (0, VelumClosed), (1, VelumClosed));
|
||||
}
|
||||
|
|
@ -1670,15 +1745,14 @@ public static class PatchScript
|
|||
}
|
||||
|
||||
var level = Math.Clamp((gain - surface.MinValue) / (surface.MaxValue - surface.MinValue), 0, 1);
|
||||
var handle = duration * 0.25f;
|
||||
ControlSplinePoint At(float time, float value) => new(time, value, handle, value, handle, value);
|
||||
ControlSplinePoint At(float time, float value) => new(time, value, 0, value, 0, value);
|
||||
var ramp = duration * 0.15f;
|
||||
var points = new List<ControlSplinePoint>();
|
||||
if (start > 0) points.Add(At(Math.Max(0, start - ramp), 0));
|
||||
points.Add(At(start, level));
|
||||
points.Add(At(start + duration, level));
|
||||
points.Add(At(start + duration + ramp, 0));
|
||||
var spline = new ControlSpline(UniqueSplineName(name), surfacePath, points.ToArray(), ControlSplineInterpolation.Bezier);
|
||||
var spline = new ControlSpline(UniqueSplineName(name), surfacePath, FollowSegments(points.ToArray()), ControlSplineInterpolation.Bezier);
|
||||
_controlSplines.Add(spline);
|
||||
_controlSplinesByName[spline.Name] = spline;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ public sealed class NasalCouplingTests
|
|||
}
|
||||
|
||||
private static float Spline(SynthPatch patch, string namePrefix, string surfacePath) =>
|
||||
patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[0].Value;
|
||||
patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[^1].Value; // a held place: the last point is the gesture's value whether or not a boundary ramp precedes it
|
||||
|
||||
// Distance of a vowel_set consonant constriction from the lip plane in cm, from the constriction_index spline: the surface is a section
|
||||
// index in [0, sections] and the area model reads position = index / (sections - 1).
|
||||
|
|
|
|||
84
tests/AquaSynth.Dsl.Tests/PhoneTransitionTests.cs
Normal file
84
tests/AquaSynth.Dsl.Tests/PhoneTransitionTests.cs
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
using AquaSynth.Dsl;
|
||||
using AquaSynth.Faust;
|
||||
|
||||
namespace AquaSynth.Dsl.Tests;
|
||||
|
||||
// What Meta heard in tract-weak-spots-1 (2026-11-13) while every formant matched: clicks at the phone boundaries of nibu, a /b/ burst the
|
||||
// length of a fricative, and a first 100 ms that climbed from silence. These pin the three causes at the control-spline and DSP-source
|
||||
// level (docs/listening/tract-weak-spots-1-analysis-20261113.md); the renders are measured by tools/formant-check/ear_metrics.py.
|
||||
public sealed class PhoneTransitionTests
|
||||
{
|
||||
private static SynthPatch Nibu() => PatchScript.Parse(IpaGestureExperiment.BuildWordScript(
|
||||
[
|
||||
(new IpaGestureExperimentTarget("w_n", "n", "", .095f), 1f),
|
||||
(new IpaGestureExperimentTarget("w_i", "i", "", .18f), 1f),
|
||||
(new IpaGestureExperimentTarget("w_b", "b", "", .095f), 1f),
|
||||
(new IpaGestureExperimentTarget("w_u", "u", "", .18f), 1f)
|
||||
], frequency: 140));
|
||||
|
||||
[Fact]
|
||||
public void NoSurfaceStepsAtAPhoneBoundary()
|
||||
{
|
||||
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
|
||||
var worst = (Surface: "", Boundary: 0f, Step: 0f);
|
||||
foreach (var boundary in new[] { .095f, .275f, .37f })
|
||||
{
|
||||
foreach (var surface in timeline.SurfacePaths)
|
||||
{
|
||||
var step = MathF.Abs(timeline.ValueAt(surface, boundary + .0005f) - timeline.ValueAt(surface, boundary - .0005f));
|
||||
if (step > worst.Step) worst = (surface, boundary, step);
|
||||
}
|
||||
}
|
||||
|
||||
// 0.5 ms either side of a boundary; a 30 ms ramp across the full 0..1 range moves 0.017 in that time, a step moves it by the
|
||||
// whole difference (the pre-fix nibu stepped /vocal/areas/0/area/constriction_diameter by 0.77 at the 95 ms boundary).
|
||||
Assert.True(worst.Step < .03f, $"{worst.Surface} steps {worst.Step:0.000} at {worst.Boundary} s");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ThePhoneAfterABoundaryArrivesAtItsFirstValueWithinTheTransition()
|
||||
{
|
||||
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
|
||||
const string diameter = "/vocal/areas/0/area/constriction_diameter";
|
||||
var closedN = timeline.ValueAt(diameter, .05f);
|
||||
Assert.True(closedN < .1f, "the /n/ constriction is shut");
|
||||
Assert.True(timeline.ValueAt(diameter, .095f + .002f) < .5f, "still closing into the vowel two ms after the boundary");
|
||||
Assert.Equal(1f, timeline.ValueAt(diameter, .095f + .035f), 2);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void TheFirstPhoneOfAWordStartsAtItsOwnValue()
|
||||
{
|
||||
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
|
||||
Assert.Equal(timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", 0f), timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", .02f), 3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture()
|
||||
{
|
||||
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
|
||||
const string contact = "/vocal/contacts/0/opening";
|
||||
const float start = .275f, duration = .095f;
|
||||
Assert.True(timeline.ValueAt(contact, start + duration - .013f) < .1f, "closed until 12 ms before the end");
|
||||
Assert.True(timeline.ValueAt(contact, start + duration) > .8f, "open at the end");
|
||||
var opening = 0f;
|
||||
for (var t = start + .03f; t <= start + duration; t += .0005f)
|
||||
{
|
||||
if (timeline.ValueAt(contact, t) > .3f) { opening = t; break; }
|
||||
}
|
||||
|
||||
Assert.True(start + duration - opening < .012f, $"the opening crosses 0.3 only {start + duration - opening:0.000} s before the end");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ControlSmoothersStartAtTheirInputNotAtZero()
|
||||
{
|
||||
var source = FaustEmitter.Emit(Nibu(), new FaustExportOptions("transition_probe")).Source;
|
||||
Assert.Contains("smoo_init(x) =", source);
|
||||
Assert.Contains("smoo_init;", source);
|
||||
// Every control surface and spline-point slider uses it; patch parameters keep si.smoo.
|
||||
var remaining = source.Split('\n').Where(line => line.Contains("hslider", StringComparison.Ordinal) && line.Contains(": si.smoo;", StringComparison.Ordinal)
|
||||
&& (line.Contains("\"/vocal/", StringComparison.Ordinal) || line.Contains("\"/splines/", StringComparison.Ordinal))).ToArray();
|
||||
Assert.Empty(remaining);
|
||||
}
|
||||
}
|
||||
|
|
@ -16,9 +16,10 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
|
|||
|
||||
private static float ConstrictionIndex(string script, string name) =>
|
||||
PatchScript.Parse(script).ControlSplines
|
||||
.Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[0].Value;
|
||||
.Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[^1].Value; // the held place; a boundary ramp may precede it
|
||||
|
||||
// Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the
|
||||
// Re-pinned on the phone-transition cut (boundary ramps, 12 ms stop release, smoo_init, vocal attack): these ids moved on purpose; the invariant
|
||||
// that survives is that a word of labials writes no position surface (below), not these bytes. Faust ids re-pinned again on the PR 52 fix passes (smoo_init pole, pitch-ratio smoother, shut lips for stops, spline handles; the scripts are unchanged; the last move is the smoo_from step line only, which reverts to prev * pole + x * (1 - pole): the old ids come back when that line is put back). Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the
|
||||
// Faust source for /apa/ and /aba/, which this cut must not change by a byte; the ids are a SHA-256 prefix).
|
||||
private const float BIndex = 0.8333333f, NIndex = 0.7118056f;
|
||||
|
||||
|
|
@ -28,7 +29,7 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
|
|||
var b = ConstrictionIndex(Word("b"), "w_1_c");
|
||||
var n = ConstrictionIndex(Word("n"), "w_1_c");
|
||||
output.WriteLine($"b {b:R} n {n:R}");
|
||||
foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE") })
|
||||
foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C") })
|
||||
{
|
||||
var script = Word(ipa);
|
||||
var faust = FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("pin", SampleRate: 48000)).Source;
|
||||
|
|
@ -47,8 +48,8 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
|
|||
{
|
||||
foreach (var (ipa, scriptSha, faustSha) in new[]
|
||||
{
|
||||
("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE"),
|
||||
("n", "BDF24A66659D7BD6", "ABEB5D03DC9637DB"), ("t", "9DFBCDB5131F9776", "30E144D54F7E026F"),
|
||||
("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C"),
|
||||
("n", "5C3FA24E40D9DE26", "B2B9C5E1015B60A9"), ("t", "45941CCFD93561B3", "0AACE6D26F648FB0"),
|
||||
})
|
||||
{
|
||||
var script = Word(ipa);
|
||||
|
|
@ -109,8 +110,9 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
|
|||
var script = IpaGestureExperiment.BuildWordScript(
|
||||
[(Phone("0_b", "b", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (new IpaGestureExperimentTarget("w_2_t", "t", "voiceless_alveolar_plosive", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_p", "p", 0.095f), 1f)], 140);
|
||||
Assert.All(SitePoints(script, "w_0_b"), v => Assert.Equal(0f, v));
|
||||
Assert.All(SitePoints(script, "w_4_p"), v => Assert.Equal(0f, v));
|
||||
Assert.All(SitePoints(script, "w_2_t"), v => Assert.True(v > 0.8f));
|
||||
Assert.All(SitePoints(script, "w_4_p").Skip(1), v => Assert.Equal(0f, v));
|
||||
// after the 30 ms boundary ramp from the previous plosive's place (the first point is that place)
|
||||
Assert.All(SitePoints(script, "w_2_t").Skip(1), v => Assert.True(v > 0.8f));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
|
@ -120,11 +122,13 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
|
|||
.Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points.Select(p => p.Value).ToArray();
|
||||
var t = PatchScript.Parse(Word("t", "voiceless_alveolar_plosive")).ControlSplines
|
||||
.Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points;
|
||||
Assert.Equal(0f, t[0].Value);
|
||||
// lead-in (the vowel's open 1), arrival, closure hold, end: shut from the arrival to the hold, and the hold ends StopReleaseSeconds before the stop does
|
||||
Assert.Equal(1f, t[0].Value);
|
||||
Assert.Equal(0f, t[1].Value);
|
||||
Assert.Equal(0.65f, (t[1].TimeSeconds - 0.18f) / 0.095f, 3);
|
||||
Assert.Equal(0f, t[2].Value);
|
||||
Assert.Equal(1 - 0.012f / 0.095f, (t[2].TimeSeconds - 0.18f) / 0.095f, 3);
|
||||
var b = Diameter(Word("b"));
|
||||
Assert.True(b[0] > 0 && Math.Abs(b[0] - b[1]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)");
|
||||
Assert.True(b[1] > 0 && Math.Abs(b[1] - b[2]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
|
|
|||
183
tests/AquaSynth.Dsl.Tests/StopClosureAndControlRenderTests.cs
Normal file
183
tests/AquaSynth.Dsl.Tests/StopClosureAndControlRenderTests.cs
Normal file
|
|
@ -0,0 +1,183 @@
|
|||
using AquaSynth.Dsl;
|
||||
using AquaSynth.Faust;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace AquaSynth.Dsl.Tests;
|
||||
|
||||
// Renders behind the tract-coarticulation findings of the PR 52 Soul pass: what the closing ramp into a stop sounds like, what the DSP
|
||||
// does with a stop's 12 ms opening, and what the control smoothers and the word attack do at the start of a render. Each reads the
|
||||
// rendered signal (or the DSP's own probe), not the emitted text.
|
||||
[Collection("NativeFaust")]
|
||||
public sealed class StopClosureAndControlRenderTests(ITestOutputHelper output)
|
||||
{
|
||||
private const int Rate = 48000;
|
||||
|
||||
private static AquaSynthCompiledPatch? Compile(string source, float seconds)
|
||||
{
|
||||
using var compiler = new AquaSynthPatchCompiler(new AquaSynthNativeOptions(SampleRate: Rate));
|
||||
if (!compiler.TryCompileSource(new AquaSynthCompileIdentity("stop_closure", "stop_closure", ""), source, seconds, out var patch, out var error))
|
||||
{
|
||||
Assert.Fail($"compile failed: {error}");
|
||||
}
|
||||
|
||||
return patch;
|
||||
}
|
||||
|
||||
private static string Word(string ipa) => IpaGestureExperiment.BuildWordScript(
|
||||
[(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140);
|
||||
|
||||
// Power above 4 kHz (100 Hz grid to 12 kHz) of a Hann-windowed stretch.
|
||||
private static double HighPower(float[] x, double from, double to)
|
||||
{
|
||||
var a = (int)(from * Rate);
|
||||
var n = (int)((to - from) * Rate);
|
||||
var seg = Enumerable.Range(0, n).Select(i => x[a + i] * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
|
||||
double sum = 0;
|
||||
for (var hz = 4000; hz < 12000; hz += 100)
|
||||
{
|
||||
double re = 0, im = 0;
|
||||
for (var i = 0; i < n; i++) { var w = 2 * Math.PI * hz * i / Rate; re += seg[i] * Math.Cos(w); im -= seg[i] * Math.Sin(w); }
|
||||
sum += re * re + im * im;
|
||||
}
|
||||
|
||||
return sum;
|
||||
}
|
||||
|
||||
// The 30 ms boundary ramp into a stop moves the contact, constriction and lip surfaces through every aperture between open and shut. The
|
||||
// burst noise is 4 * opening * (1 - opening) times a gain, and the gain was held through the closure, so each of those passes was a 15-20 ms
|
||||
// hiss before the stop. The interval is the stop's start to its release, against the same length of the vowel before it, in the >4 kHz
|
||||
// band, with the glottal source silent and the burst gain kept: the voiced stops' own voicing runs through the ramp and its harmonics
|
||||
// fill that band (/d/ read -11.6 dB with the source on and -52.8 with it off; with the source on it swings between -13.5 and -9.3 dB
|
||||
// with f0 alone, 139.5 to 145 Hz, as the harmonics fall on and off the 100 Hz grid), so with the source on the measure cannot tell the
|
||||
// voicing from the hiss. Source off, the burst gain held through the closure read +5 dB (/p/) and +14 dB (/t/, /d/) on 3b62b9c; gated,
|
||||
// every stop reads -52 dB. -30 is the bound between the two. /g/ is not here: it is not in TryIpaDescriptor, so it would render the default approximant and could not fail.
|
||||
[FaustTheory]
|
||||
[InlineData("p")]
|
||||
[InlineData("b")]
|
||||
[InlineData("t")]
|
||||
[InlineData("d")]
|
||||
[InlineData("k")]
|
||||
public void ClosingIntoAStopMakesNoHighFrequencyNoise(string ipa)
|
||||
{
|
||||
var parsed = PatchScript.Parse(Word(ipa));
|
||||
var voiceless = parsed with
|
||||
{
|
||||
ControlSplines = parsed.ControlSplines.Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal)
|
||||
? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() }
|
||||
: s).ToArray()
|
||||
};
|
||||
using var patch = Compile(FaustEmitter.Emit(voiceless, new FaustExportOptions("stop_closure", SampleRate: Rate)).Source, 0.5f);
|
||||
var x = patch!.Render(0.5f);
|
||||
const double from = 0.18;
|
||||
var release = PatchScript.StopReleaseStartSeconds(0.095f);
|
||||
var closing = HighPower(x, from, from + release);
|
||||
var vowel = HighPower(x, from - release, from);
|
||||
var db = 10 * Math.Log10(closing / vowel);
|
||||
output.WriteLine($"/{ipa}/ >4 kHz energy of the closing interval {db:F1} dB re the vowel's before it, source silent");
|
||||
Assert.True(db < -30, $"/{ipa}/ closes with {db:F1} dB re the vowel's own >4 kHz energy");
|
||||
}
|
||||
|
||||
// The Faust lowering of a Bezier segment ignores the handle times and runs the curve on the linear fraction of the segment; the C# timeline
|
||||
// solves the curve's time, so a handle longer than the segment folds it. The stop's 12 ms opening had 23.75 ms handles: the contact opening
|
||||
// read 0.05 to 0.87 between 6 and 7 ms in C# and 0.46 to 0.57 in the DSP. This reads the opening through the DSP's own contact reservoir probe
|
||||
// (stored pressure + (1 - opening) * resistance, both at their defaults once the other splines are dropped) and compares it with the timeline.
|
||||
[FaustFact]
|
||||
public void TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt()
|
||||
{
|
||||
var script = Word("p");
|
||||
var parsed = PatchScript.Parse(script);
|
||||
var contact = parsed.ControlSplines.Where(s => s.SurfacePath == "/vocal/contacts/0/opening").ToArray();
|
||||
var alone = parsed with { ControlSplines = contact };
|
||||
var timeline = new ControlSplineTimeline(contact);
|
||||
var source = FaustEmitter.Emit(alone, new FaustExportOptions("stop_opening", SampleRate: Rate, DebugProbeUi: true)).Source;
|
||||
using var patch = Compile(source, 0.5f);
|
||||
using var stream = patch!.CreateStreamingPatch();
|
||||
var reservoir = stream.ProbePaths.Single(p => p.EndsWith("/contact/contact/reservoir", StringComparison.Ordinal));
|
||||
var release = 0.18f + PatchScript.StopReleaseStartSeconds(0.095f);
|
||||
var outputs = new[] { new float[48] };
|
||||
var dsp = new List<(float Time, float Reservoir)>();
|
||||
for (var block = 0; block < 400; block++)
|
||||
{
|
||||
stream.ProcessBlock([], outputs, 48);
|
||||
var t = (block + 1) * 48f / Rate;
|
||||
if (t >= release - 0.004f && t <= release + 0.016f) dsp.Add((t, stream.ReadProbe(reservoir)));
|
||||
}
|
||||
|
||||
// reservoir is linear in the opening; scale it by its own end points
|
||||
var shut = dsp.First().Reservoir;
|
||||
var open = dsp.Last().Reservoir;
|
||||
var worst = 0f;
|
||||
foreach (var (time, value) in dsp)
|
||||
{
|
||||
var dspOpening = (value - shut) / (open - shut);
|
||||
var curve = timeline.ValueAt("/vocal/contacts/0/opening", time);
|
||||
var shutValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.First().Time);
|
||||
var openValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.Last().Time);
|
||||
var timelineOpening = (curve - shutValue) / (openValue - shutValue);
|
||||
worst = MathF.Max(worst, MathF.Abs(dspOpening - timelineOpening));
|
||||
}
|
||||
|
||||
output.WriteLine($"worst difference between the DSP's opening and the timeline's over the release: {worst:F3}");
|
||||
Assert.True(worst < 0.08f, $"the DSP's stop opening departs from the timeline by {worst:F3}");
|
||||
}
|
||||
|
||||
// The same fact without the DSP: every handle of an authored spline is a third of its segment, so the C# curve is the curve the lowering runs.
|
||||
[Fact]
|
||||
public void EveryAuthoredSplineHandleIsAThirdOfItsSegment()
|
||||
{
|
||||
var splines = PatchScript.Parse(Word("p")).ControlSplines.Where(s => s.Points.Count > 1).ToArray();
|
||||
Assert.NotEmpty(splines);
|
||||
foreach (var spline in splines)
|
||||
{
|
||||
for (var i = 0; i + 1 < spline.Points.Count; i++)
|
||||
{
|
||||
var third = (spline.Points[i + 1].TimeSeconds - spline.Points[i].TimeSeconds) / 3;
|
||||
Assert.Equal(third, spline.Points[i].OutTimeOffsetSeconds, 5);
|
||||
Assert.Equal(third, spline.Points[i + 1].InTimeOffsetSeconds, 5);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Replaces the emitted patch's process with one control through the emitted smoo_init, so the smoother under test is the one the exporter
|
||||
// writes. Starts at its input at sample 0, and moves at si.smoo's 22.7 ms time constant when the control moves (5 ms gave 0.89 here).
|
||||
[FaustFact]
|
||||
public void ASmootherStartsAtItsInputAndMovesAtSiSmoosPace()
|
||||
{
|
||||
var emitted = FaustEmitter.Emit(PatchScript.Parse(Word("p")), new FaustExportOptions("smoo", SampleRate: Rate)).Source;
|
||||
var lines = emitted.Split('\n').Select(l => l.StartsWith("process =", StringComparison.Ordinal) ? "process = hslider(\"/vocal/sources/0/pressure\", 0.7, 0, 1, 0.001) : smoo_init;" : l);
|
||||
using var patch = Compile(string.Join('\n', lines), 0.2f);
|
||||
using var stream = patch!.CreateStreamingPatch();
|
||||
var first = new[] { new float[480] };
|
||||
stream.ProcessBlock([], first, 480);
|
||||
output.WriteLine($"sample 0 {first[0][0]:F4}, after 10 ms {first[0][479]:F4}");
|
||||
Assert.Equal(0.7f, first[0][0], 4);
|
||||
Assert.Equal(0.7f, first[0][479], 4);
|
||||
var after = new[] { new float[240] };
|
||||
stream.ProcessBlock([], after, 240, new Dictionary<string, float> { ["/vocal/sources/0/pressure"] = 1f });
|
||||
// 5 ms into a step from 0.7 to 1 with a 22.7 ms time constant
|
||||
var expected = 0.7 + 0.3 * (1 - Math.Pow(1 - 44.1 / Rate, 0.005 * Rate));
|
||||
output.WriteLine($"5 ms after a step to 1: {after[0][239]:F4}, si.smoo would be {expected:F4}");
|
||||
Assert.InRange(after[0][239], expected - 0.01, expected + 0.01);
|
||||
}
|
||||
|
||||
// A word starts with a 12 ms attack, not a step: the control smoothers no longer fade it in, and a step onset would be a click.
|
||||
[FaustFact]
|
||||
public void AWordBeginsWithATwelveMillisecondAttack()
|
||||
{
|
||||
double FirstMillisecondsPeak(string script)
|
||||
{
|
||||
using var patch = Compile(FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("attack", SampleRate: Rate)).Source, 0.4f);
|
||||
var x = patch!.Render(0.4f);
|
||||
var steady = x.Skip((int)(0.05 * Rate)).Take((int)(0.1 * Rate)).Max(Math.Abs);
|
||||
return x.Take((int)(0.002 * Rate)).Max(Math.Abs) / steady;
|
||||
}
|
||||
|
||||
var script = IpaGestureExperiment.BuildWordScript([(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.3f), 1f)], 140);
|
||||
Assert.Contains("attack=.012", script);
|
||||
var attacked = FirstMillisecondsPeak(script);
|
||||
var stepped = FirstMillisecondsPeak(script.Replace("attack=.012 ", "", StringComparison.Ordinal));
|
||||
output.WriteLine($"peak of the first 2 ms re the steady peak: with the attack {attacked:F3}, without {stepped:F3}");
|
||||
Assert.True(attacked < 0.25, $"the first 2 ms reach {attacked:F2} of the steady peak");
|
||||
Assert.True(stepped > 2 * attacked, "without the attack the onset is a step");
|
||||
}
|
||||
}
|
||||
|
|
@ -24,6 +24,11 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
using (patch) return patch!.Render(seconds);
|
||||
}
|
||||
|
||||
// The stop's release time and the start of its closure (every articulator at its shut value, after the boundary ramp into it), both from the
|
||||
// one owner in PatchScript rather than as fractions of the gesture; the consonant is [from, to).
|
||||
private static double ReleaseAt(double from, double to) => from + PatchScript.StopReleaseStartSeconds((float)(to - from));
|
||||
private static double ClosureFrom(double from) => from + PatchScript.PhoneTransitionSeconds;
|
||||
|
||||
private static double Rms(float[] x, double from, double to)
|
||||
{
|
||||
var a = (int)(from * Rate);
|
||||
|
|
@ -95,7 +100,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var withoutBurst = Render(script, 0.4f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() });
|
||||
// Everything but the burst's noise is deterministic and shared, so the difference of the two renders is the burst alone.
|
||||
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
|
||||
const double releaseFrom = 0.095 * 0.65, vowelFrom = 0.095;
|
||||
const double vowelFrom = 0.095;
|
||||
var releaseFrom = ReleaseAt(0, vowelFrom);
|
||||
var vowelPeak = Peak(withBurst, vowelFrom + 0.06, vowelFrom + 0.18);
|
||||
var burstPeak = Peak(burst, releaseFrom, vowelFrom + 0.01);
|
||||
var burstBeforeRelease = Peak(burst, 0, releaseFrom - 0.005);
|
||||
|
|
@ -121,7 +127,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var withoutBurst = Render(script, 0.77f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() });
|
||||
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
|
||||
const double bFrom = 0.275, bTo = 0.37;
|
||||
var atRelease = Peak(burst, bFrom + 0.095 * 0.6, bTo + 0.02);
|
||||
var atRelease = Peak(burst, ReleaseAt(bFrom, bTo) - 0.003, bTo + 0.02);
|
||||
var leading = Peak(burst, 0, bFrom);
|
||||
var trailing = Peak(burst, bTo + 0.12, 0.77);
|
||||
output.WriteLine($"burst peak {atRelease:F4} at the release, {leading:F4} before the plosive (n, a), {trailing:F4} after it (a, n, tail)");
|
||||
|
|
@ -130,17 +136,64 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
Assert.True(trailing < atRelease / 10, "no burst noise in the vowel, the nasal and the tail after the plosive");
|
||||
}
|
||||
|
||||
// The burst gain's window, read off the spline itself: it opens one ramp (0.15 of the gesture) before the plosive and has closed one ramp after it.
|
||||
// The rendered test above survived three mutants of the window (ramp 0, ramp 1.0 of the gesture, end moved to 3 ramps), so the bound is pinned here.
|
||||
// The burst gain's window, read off the spline itself: it exists only from the stop's release start, one ramp (0.15 of the release window)
|
||||
// before it and one after the stop's end. A gain held through the closure made the closing ramp a hiss (ClosingIntoAStopMakesNoHighFrequencyNoise).
|
||||
[Fact]
|
||||
public void PlosiveBurstGainWindowIsOneRampEitherSide()
|
||||
public void PlosiveBurstGainWindowIsOneRampEitherSideOfTheRelease()
|
||||
{
|
||||
IpaGestureExperimentTarget Phone(string label, string ipa, float seconds) => new($"w_{label}", ipa, "", seconds);
|
||||
var script = IpaGestureExperiment.BuildWordScript(
|
||||
[(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone("2_b", "b", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_n", "n", 0.095f), 1f)], 140);
|
||||
var window = PatchScript.Parse(script).ControlSplines.Single(s => s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal) && s.Points.Any(p => p.Value > 0));
|
||||
const double start = 0.275, duration = 0.095, ramp = 0.15 * duration;
|
||||
Assert.Equal((start - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4));
|
||||
const double start = 0.275, duration = 0.095;
|
||||
var release = start + PatchScript.StopReleaseStartSeconds((float)duration);
|
||||
var ramp = 0.15 * (start + duration - release);
|
||||
Assert.Equal((release - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4));
|
||||
}
|
||||
|
||||
// The contact opening, the tongue's constriction and the lips all leave their shut value at the instant the burst gain window opens: one
|
||||
// owner, StopReleaseStartSeconds. Read off the splines, so a stale constant in one of them (the contact opening 20 ms early while the
|
||||
// burst keeps the owner's time) fails here and not only through a closure level that happens to cross a bound.
|
||||
[Theory]
|
||||
[InlineData("b", "w_2_b")]
|
||||
[InlineData("t", "w_2_t")]
|
||||
[InlineData("k", "w_2_k")]
|
||||
public void AStopsArticulatorsOpenWhereItsBurstGainWindowOpens(string ipa, string name)
|
||||
{
|
||||
IpaGestureExperimentTarget Phone(string label, string symbol, float seconds) => new($"w_{label}", symbol, "", seconds);
|
||||
var script = IpaGestureExperiment.BuildWordScript(
|
||||
[(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone($"2_{ipa}", ipa, 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f)], 140);
|
||||
var splines = PatchScript.Parse(script).ControlSplines;
|
||||
const double start = 0.275, duration = 0.095;
|
||||
var release = start + PatchScript.StopReleaseStartSeconds((float)duration);
|
||||
var gain = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal));
|
||||
var ramp = 0.15 * (start + duration - release);
|
||||
Assert.Equal(release - ramp, gain.Points.Min(p => (double)p.TimeSeconds), 1e-4);
|
||||
var paths = ipa == "b"
|
||||
? new[] { "/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter", "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" }
|
||||
: ["/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter"];
|
||||
foreach (var path in paths)
|
||||
{
|
||||
var spline = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath == path);
|
||||
Assert.Equal(release, spline.Points.Where(p => p.TimeSeconds < start + duration - 1e-6).Max(p => (double)p.TimeSeconds), 1e-4);
|
||||
}
|
||||
}
|
||||
|
||||
// The nasal keeps its murmur's small lip opening where a stop shuts the lips (0.04 cm against 0): the shut-lip term is the stop's only.
|
||||
// Extending it to nasals moves /m/'s first nasal formant by 30 Hz and its murmur level by 0.4 dB and no render test notices, so the
|
||||
// value is read off the splines.
|
||||
[Theory]
|
||||
[InlineData("m", 0.04f)]
|
||||
[InlineData("b", 0f)]
|
||||
public void OnlyAStopShutsTheLipsCompletely(string ipa, float shut)
|
||||
{
|
||||
var script = IpaGestureExperiment.BuildWordScript(
|
||||
[(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140);
|
||||
foreach (var path in new[] { "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" })
|
||||
{
|
||||
var spline = PatchScript.Parse(script).ControlSplines.Single(s => s.Name.StartsWith("w_1_c", StringComparison.Ordinal) && s.SurfacePath == path);
|
||||
Assert.Equal(shut, spline.Points.Min(p => p.Value), 1e-4);
|
||||
}
|
||||
}
|
||||
|
||||
// /ɑ C ɑ/ at the scaffold's own durations (vowel 0.18 s, consonant 0.095 s). Returns the render and the consonant's [start, end).
|
||||
|
|
@ -241,7 +294,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
private static SynthPatch WithoutBurst(SynthPatch p) =>
|
||||
p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() };
|
||||
|
||||
// The closure of a stop is the part before its contact opens (0.65 of the gesture), 0.2 of the gesture in from its start, without the burst.
|
||||
// The closure of a stop is the part after the boundary ramp into it and before its release, without the burst (the ramp is the tract still
|
||||
// shutting on the vowel's decaying voicing; its noise is measured by ClosingIntoAStopMakesNoHighFrequencyNoise).
|
||||
// A tongue closure at the nasal's 0.03 cm aperture leaked: /t/ read -14.3 dB re the flanking vowels against /p/ -22.1 (the lips seal /p/);
|
||||
// completely closed it reads -24.3.
|
||||
[Fact]
|
||||
|
|
@ -252,7 +306,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst);
|
||||
if (x is null) return double.NaN;
|
||||
var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2);
|
||||
return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels);
|
||||
return 20 * Math.Log10(Rms(x, ClosureFrom(from), ReleaseAt(from, to)) / vowels);
|
||||
}
|
||||
|
||||
var p = Closure("p", "voiceless_bilabial_plosive");
|
||||
|
|
@ -269,7 +323,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var (nb, _, _) = VowelConsonantVowel("t", descriptor: "voiceless_alveolar_plosive", edit: WithoutBurst);
|
||||
if (x is null || nb is null) return;
|
||||
var burst = x.Zip(nb, (a, b) => a - b).ToArray();
|
||||
var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01);
|
||||
var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01);
|
||||
var vowel = Peak(x, to + 0.06, to + 0.17);
|
||||
output.WriteLine($"/t/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}");
|
||||
Assert.True(peak > 0.004, "audible");
|
||||
|
|
@ -288,7 +342,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
if (withBurst is null || withoutBurst is null) return;
|
||||
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
|
||||
const double tFrom = 0.275, tTo = 0.37;
|
||||
var atRelease = Peak(burst, tFrom + 0.095 * 0.6, tTo + 0.02);
|
||||
var atRelease = Peak(burst, ReleaseAt(tFrom, tTo) - 0.003, tTo + 0.02);
|
||||
var leading = Peak(burst, 0, tFrom);
|
||||
var trailing = Peak(burst, tTo + 0.12, 0.77);
|
||||
output.WriteLine($"/t/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");
|
||||
|
|
@ -305,7 +359,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var (x, from, to) = VowelConsonantVowel("t");
|
||||
var (nb, _, _) = VowelConsonantVowel("t", edit: WithoutBurst);
|
||||
if (x is null || nb is null) return;
|
||||
var start = (int)((from + 0.65 * (to - from)) * Rate);
|
||||
var start = (int)(ReleaseAt(from, to) * Rate);
|
||||
const int n = 960;
|
||||
var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
|
||||
double Power(double hz)
|
||||
|
|
@ -324,39 +378,44 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
}
|
||||
|
||||
// /k/ over the /t/ tests: its own place (the velar distance in the place table), closure sealed by the tongue, burst under half the vowel.
|
||||
// The spec asks /k/'s closure to be at or below /p/'s. It is not: with the voiceless source as /p/ and /t/ have it the closure reads
|
||||
// -14.9 dB re the vowels (/p/ -20.4, /t/ -24.9). The tract itself seals: with the glottal source's flow scale held at 0 over the gesture the
|
||||
// same closure reads -22.7 dB, under /p/; with the source on, the 150 Hz voicing is what reaches the output through a closure that leaves
|
||||
// only a short rear cavity (the glottal pressure that the tract wall radiates is higher the smaller that cavity is). Silencing voicing in
|
||||
// voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion is reported as unmet (fork), not met by a /k/ special case.
|
||||
// The second assertion is a regression bound on the measured leak, not the criterion.
|
||||
// The spec asks /k/'s closure to be at or below /p/'s. With the voiceless source as /p/ and /t/ have it, it is not, and was not on main
|
||||
// either (-14.9 dB re the vowels, /p/ -20.4): the 150 Hz voicing reaches the output through a closure that leaves only a short rear cavity.
|
||||
// Silencing voicing in voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion stays unmet (fork).
|
||||
// The measure is the source-on closure in its settled part (25 ms after the 30 ms closing ramp has finished, to 5 ms before the release),
|
||||
// because the source-off closure cannot see a leak: with the source off nothing drives the tract, and leaks of .02, .05 and .1 of the
|
||||
// diameter scale read -21.1, -21.2 and -21.2 dB over .21 to .263 s against -21.0 sealed, while the source-on closure reads 1.8, 1.8 and
|
||||
// 4.1 dB louder in the settled part. The closure
|
||||
// right after the ramp is no use either: the seal completes at the end of the ramp with the vowel's tail still in the rear cavity (the
|
||||
// source-off /k/ read -21.0 dB over .21 to .263 s with the ramp and -27.5 with the contact and the constriction stepped shut; main's
|
||||
// 30 ms-later window read -25.7), and that decays within the first 25 ms.
|
||||
// The test therefore proves its own measure first (a leak injected into the tongue's closed diameter must read 3 dB louder) and then bounds
|
||||
// the sealed closure (-16.5 measured; a .005, .03 or .035 leak reads -15.1, a .02 leak -14.7).
|
||||
[Fact]
|
||||
public void AVelarStopClosureSealsAndItsLeakIsTheGlottalSource()
|
||||
public void AVelarStopClosureSealsAndTheMeasureSeesALeak()
|
||||
{
|
||||
static SynthPatch WithoutSource(SynthPatch p) =>
|
||||
static SynthPatch Leaking(SynthPatch p) =>
|
||||
WithoutBurst(p) with
|
||||
{
|
||||
ControlSplines = WithoutBurst(p).ControlSplines
|
||||
.Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal)
|
||||
? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() }
|
||||
.Select(s => s.Name == "w_1_c_constriction_diameter"
|
||||
? s with { Points = s.Points.Select(q => q.Value == s.Points.Min(v => v.Value) ? q with { Value = 0.1f, OutValue = 0, InValue = 0 } : q).ToArray() }
|
||||
: s).ToArray()
|
||||
};
|
||||
|
||||
double Closure(string ipa, string descriptor, Func<SynthPatch, SynthPatch> edit)
|
||||
double Closure(Func<SynthPatch, SynthPatch> edit)
|
||||
{
|
||||
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: edit);
|
||||
var (x, from, to) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: edit);
|
||||
if (x is null) return double.NaN;
|
||||
var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2);
|
||||
return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels);
|
||||
return 20 * Math.Log10(Rms(x, ClosureFrom(from) + 0.025, ReleaseAt(from, to) - 0.005) / vowels);
|
||||
}
|
||||
|
||||
var p = Closure("p", "voiceless_bilabial_plosive", WithoutBurst);
|
||||
var k = Closure("k", "voiceless_velar_plosive", WithoutBurst);
|
||||
var kSilent = Closure("k", "voiceless_velar_plosive", WithoutSource);
|
||||
output.WriteLine($"closure re vowels: /p/ {p:F1} dB, /k/ {k:F1} dB, /k/ with the glottal source off {kSilent:F1} dB");
|
||||
if (double.IsNaN(p)) return;
|
||||
Assert.True(kSilent <= p, $"the /k/ closure seals: without the glottal source it is {kSilent:F1} dB, at or under /p/ ({p:F1} dB)");
|
||||
Assert.True(k <= -12, $"the /k/ closure with the source is {k:F1} dB (regression bound; the criterion, at or under /p/ {p:F1} dB, is not met)");
|
||||
var sealedClosure = Closure(WithoutBurst);
|
||||
var leaking = Closure(Leaking);
|
||||
output.WriteLine($"/k/ closure re vowels, source on, settled part: sealed {sealedClosure:F1} dB, with a 0.1 leak {leaking:F1} dB");
|
||||
if (double.IsNaN(sealedClosure)) return;
|
||||
Assert.True(leaking >= sealedClosure + 3, $"the measure sees a leak: sealed {sealedClosure:F1} dB, leaking {leaking:F1} dB");
|
||||
Assert.True(sealedClosure <= -15.5, $"the /k/ closure seals: {sealedClosure:F1} dB re the vowels (measured -16.5; leaks of .005 to .1 read -15.1 to -12.4, the measure is not monotonic in the leak, so the bound sits under the weakest of them)");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
|
@ -366,7 +425,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var (nb, _, _) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: WithoutBurst);
|
||||
if (x is null || nb is null) return;
|
||||
var burst = x.Zip(nb, (a, b) => a - b).ToArray();
|
||||
var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01);
|
||||
var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01);
|
||||
var vowel = Peak(x, to + 0.06, to + 0.17);
|
||||
output.WriteLine($"/k/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}");
|
||||
Assert.True(peak > 0.004, "audible");
|
||||
|
|
@ -384,7 +443,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
if (withBurst is null || withoutBurst is null) return;
|
||||
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
|
||||
const double kFrom = 0.275, kTo = 0.37;
|
||||
var atRelease = Peak(burst, kFrom + 0.095 * 0.6, kTo + 0.02);
|
||||
var atRelease = Peak(burst, ReleaseAt(kFrom, kTo) - 0.003, kTo + 0.02);
|
||||
var leading = Peak(burst, 0, kFrom);
|
||||
var trailing = Peak(burst, kTo + 0.12, 0.77);
|
||||
output.WriteLine($"/k/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");
|
||||
|
|
@ -402,7 +461,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor);
|
||||
var (nb, _, _) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst);
|
||||
if (x is null || nb is null) return (double.NaN, double.NaN);
|
||||
var start = (int)((from + 0.65 * (to - from)) * Rate);
|
||||
var start = (int)(ReleaseAt(from, to) * Rate);
|
||||
const int n = 960;
|
||||
var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
|
||||
double Power(double hz)
|
||||
|
|
@ -468,7 +527,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
if (withBurst is null || withoutBurst is null) return;
|
||||
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
|
||||
const double dFrom = 0.275, dTo = 0.37;
|
||||
var atRelease = Peak(burst, dFrom + 0.095 * 0.6, dTo + 0.02);
|
||||
var atRelease = Peak(burst, ReleaseAt(dFrom, dTo) - 0.003, dTo + 0.02);
|
||||
var leading = Peak(burst, 0, dFrom);
|
||||
var trailing = Peak(burst, dTo + 0.12, 0.77);
|
||||
output.WriteLine($"/d/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");
|
||||
|
|
|
|||
|
|
@ -148,10 +148,10 @@ public sealed class WeksaPhonesTests
|
|||
Assert.Equal([WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds * WeksaWordPlan.CueFor(WeksaStress.Primary).Duration, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds],
|
||||
plan.Phones.Select(p => p.Target.DurationSeconds));
|
||||
var patch = PatchScript.Parse(IpaGestureExperiment.BuildWordScript(plan.Phones.ToArray(), 140));
|
||||
var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value);
|
||||
var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value);
|
||||
Assert.Equal(1f, ih["/vocal/areas/0/area/vowel_ih"]);
|
||||
Assert.Equal(0f, ih["/vocal/areas/0/area/vowel_i"]);
|
||||
var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value);
|
||||
var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value);
|
||||
Assert.Equal(1f, oo["/vocal/areas/0/area/vowel_oo_short"]);
|
||||
Assert.Equal(0f, oo["/vocal/areas/0/area/vowel_u"]);
|
||||
}
|
||||
|
|
@ -190,7 +190,7 @@ public sealed class WeksaPhonesTests
|
|||
var none = patch(WeksaStress.None);
|
||||
var primary = patch(WeksaStress.Primary);
|
||||
float First(SynthPatch p, string prefix, string surface) =>
|
||||
p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[0].Value;
|
||||
p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[^1].Value;
|
||||
|
||||
// pitch_ratio surface range 0.5..2: normalized (1.06 - .5) / 1.5; unstressed holds 1 -> (1 - .5) / 1.5.
|
||||
Assert.Equal((1.06f - .5f) / 1.5f, First(primary, "w_1_i_source_pitch", "/vocal/sources/0/pitch_ratio"), 4);
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
|
|||
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
|
||||
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
|
||||
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
|
||||
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
|
||||
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
|
||||
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
|
||||
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
|
||||
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
|
|||
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
|
||||
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
|
||||
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
|
||||
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
|
||||
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
|
||||
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
|
||||
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
|
||||
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
|
|||
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
|
||||
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
|
||||
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
|
||||
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
|
||||
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
|
||||
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
|
||||
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
|
||||
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));
|
||||
|
|
|
|||
|
|
@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
|
|||
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
|
||||
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
|
||||
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
|
||||
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
|
||||
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
|
||||
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
|
||||
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
|
||||
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));
|
||||
|
|
|
|||
104
tools/formant-check/ear_metrics.py
Normal file
104
tools/formant-check/ear_metrics.py
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
#!/usr/bin/env python3
|
||||
"""What the formant numbers do not say. Measures a render for the faults Meta heard in tract-weak-spots-1 (2026-11-13) while every
|
||||
formant matched its target; it does not listen, and a pass here is not a verdict (the listening page is).
|
||||
ear_metrics.py FILE.wav [--f0 140] [--steady A B] [--json]
|
||||
|
||||
click_ratio largest pitch-period-window peak over the median window peak. A voiced word with no transient is near 1; the pre-fix
|
||||
nibu (a phone boundary stepping an area) was 3.54 with its window at the /b/ closure (0.271 s); after, 1.76 (a vowel's own pulses).
|
||||
step_db largest single-sample step, in dB over the median sample step of the voiced part. A cell area that jumps is one sample wide.
|
||||
bandwidths LPC pole bandwidths (Hz) of the steady part, F1..F3 as (Hz, bandwidth). Natural vowels run about 40-100 Hz at F1, 50-120 at
|
||||
F2, 80-200 at F3 (Hawks & Miller 1995, Fant 1972); a lossless tube rings narrower, which reads as reedy.
|
||||
hnr_db harmonics-to-noise ratio of the steady part by autocorrelation at the pitch lag.
|
||||
hf_db energy above 4 kHz re the total, dB.
|
||||
"""
|
||||
import sys, wave, json, argparse
|
||||
import numpy as np
|
||||
|
||||
|
||||
def wav(path):
|
||||
with wave.open(path) as w:
|
||||
return np.frombuffer(w.readframes(w.getnframes()), dtype="<i2").astype(float) / 32768.0, w.getframerate()
|
||||
|
||||
|
||||
def click_ratio(x, sr, f0):
|
||||
p = max(8, int(sr / f0))
|
||||
peaks = np.array([np.abs(x[i:i + p]).max() for i in range(0, len(x) - p, p // 2)])
|
||||
voiced = peaks[peaks > peaks.max() * 0.02]
|
||||
if len(voiced) < 4:
|
||||
return 0.0, 0.0
|
||||
at = int(np.argmax(peaks)) * (p // 2) / sr
|
||||
return float(peaks.max() / np.median(voiced)), at
|
||||
|
||||
|
||||
def step_db(x):
|
||||
d = np.abs(np.diff(x))
|
||||
live = d[d > d.max() * 1e-3]
|
||||
return float(20 * np.log10(d.max() / max(np.median(live), 1e-12))) if len(live) else 0.0
|
||||
|
||||
|
||||
def autocorr(s, n=None):
|
||||
"""Autocorrelation at lags 0..n-1 by FFT (np.correlate is quadratic)."""
|
||||
n = n or len(s)
|
||||
f = np.fft.rfft(s, 2 * len(s))
|
||||
return np.fft.irfft(f * np.conj(f))[:n]
|
||||
|
||||
|
||||
def lpc(s, o):
|
||||
s = s * np.hamming(len(s))
|
||||
r = autocorr(s, o + 1)
|
||||
R = np.array([[r[abs(i - j)] for j in range(o)] for i in range(o)])
|
||||
a = np.linalg.solve(R + 1e-9 * r[0] * np.eye(o), r[1:o + 1])
|
||||
return np.concatenate([[1], -a])
|
||||
|
||||
|
||||
def bandwidths(y, sr, order=14):
|
||||
dec = max(1, int(round(sr / 10000)))
|
||||
k = np.hanning(4 * dec + 1)
|
||||
k /= k.sum()
|
||||
z = np.convolve(y - y.mean(), k, "same")[::dec]
|
||||
fs = sr / dec
|
||||
z = np.append(z[0], z[1:] - 0.97 * z[:-1])
|
||||
rt = np.roots(lpc(z, order))
|
||||
rt = rt[np.imag(rt) > 0.001]
|
||||
fq = np.angle(rt) * fs / (2 * np.pi)
|
||||
bw = -np.log(np.abs(rt)) * fs / np.pi
|
||||
# a formant is a narrow pole; broad poles (> 400 Hz) are the spectral tilt the all-pole fit spends order on
|
||||
return [(int(f), int(b)) for f, b in sorted(zip(fq, bw)) if 150 < f < 4500 and b < 400][:3]
|
||||
|
||||
|
||||
def hnr_db(y, sr, f0):
|
||||
y = y - y.mean()
|
||||
ac = autocorr(y)
|
||||
ac /= ac[0]
|
||||
lo, hi = int(sr / (f0 * 1.3)), int(sr / (f0 * 0.75))
|
||||
m = float(ac[lo:hi].max())
|
||||
return 10 * np.log10(max(m, 1e-6) / max(1 - m, 1e-6))
|
||||
|
||||
|
||||
def measure(path, f0=140.0, steady=None):
|
||||
x, sr = wav(path)
|
||||
a, b = steady if steady else (len(x) / sr * 0.35, len(x) / sr * 0.85)
|
||||
y = x[int(a * sr):int(b * sr)]
|
||||
sp = np.abs(np.fft.rfft((y - y.mean()) * np.hanning(len(y))))
|
||||
fr = np.fft.rfftfreq(len(y), 1 / sr)
|
||||
cr, at = click_ratio(x, sr, f0)
|
||||
return {
|
||||
"file": path.split("/")[-1], "click_ratio": round(cr, 2), "click_at_s": round(at, 4), "step_db": round(step_db(x), 1),
|
||||
"bandwidths": bandwidths(y, sr), "hnr_db": round(hnr_db(y, sr, f0), 1),
|
||||
"hf_db": round(float(10 * np.log10(max((sp[fr > 4000] ** 2).sum(), 1e-20) / (sp ** 2).sum())), 1),
|
||||
}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("files", nargs="+")
|
||||
ap.add_argument("--f0", type=float, default=140.0)
|
||||
ap.add_argument("--steady", nargs=2, type=float)
|
||||
ap.add_argument("--json", action="store_true")
|
||||
a = ap.parse_args()
|
||||
rows = [measure(f, a.f0, a.steady) for f in a.files]
|
||||
if a.json:
|
||||
print(json.dumps(rows, indent=1))
|
||||
else:
|
||||
for r in rows:
|
||||
print(f"{r['file']:20s} click {r['click_ratio']:5.2f} at {r['click_at_s']:.3f}s step {r['step_db']:5.1f} dB hnr {r['hnr_db']:5.1f} hf {r['hf_db']:6.1f} F(Hz,bw) {r['bandwidths']}")
|
||||
39
tools/formant-check/phone_probe.py
Normal file
39
tools/formant-check/phone_probe.py
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
#!/usr/bin/env python3
|
||||
"""A machine ear for the tract renders: what phone string a speech recogniser reads out of a clip. A PROXY for 'what was heard', never a
|
||||
verdict: it is trained on natural speech and reads the vowels of this tract poorly (every isolated vowel came back as /i/), so use it
|
||||
for the events it separates well (a burst read as /v/ against /b/, a nasal present or missing, a consonant that is only a click).
|
||||
It reproduced three of Meta's four first listening reports on tract-weak-spots-1 before any fix: ba with burst -> 'v ɑ' (she heard
|
||||
'fah then bah'), ana -> 'v ɑ m ɑ' (a basketball buzzer, no articulation), nibu -> 'i b u' (both consonants clicks, the /n/ missing).
|
||||
phone_probe.py CLIP.wav...
|
||||
Needs torch, transformers, scipy and the HuggingFace model facebook/wav2vec2-lv-60-espeak-cv-ft (about 1.2 GB, downloaded once into
|
||||
HF_HOME). The CTC decode reads vocab.json directly, so the phonemizer library is not needed.
|
||||
"""
|
||||
import sys, wave, json
|
||||
import numpy as np
|
||||
import torch
|
||||
from scipy.signal import resample_poly
|
||||
from transformers import Wav2Vec2ForCTC
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
MODEL = "facebook/wav2vec2-lv-60-espeak-cv-ft"
|
||||
|
||||
|
||||
def phones(model, vocab, path):
|
||||
with wave.open(path) as w:
|
||||
sr = w.getframerate()
|
||||
x = np.frombuffer(w.readframes(w.getnframes()), dtype="<i2").astype(np.float32) / 32768
|
||||
x = resample_poly(x, 16000, sr).astype(np.float32)
|
||||
x = np.concatenate([np.zeros(4000, np.float32), x, np.zeros(4000, np.float32)]) # silence either side, as speech has
|
||||
x = (x - x.mean()) / (x.std() + 1e-7)
|
||||
with torch.no_grad():
|
||||
logits = model(torch.tensor(x)[None]).logits[0]
|
||||
ids = logits.argmax(-1).tolist()
|
||||
out = [vocab[i] for k, i in enumerate(ids) if (k == 0 or i != ids[k - 1]) and vocab[i] != "<pad>"]
|
||||
return " ".join(out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
model = Wav2Vec2ForCTC.from_pretrained(MODEL).eval()
|
||||
vocab = {v: k for k, v in json.load(open(hf_hub_download(MODEL, "vocab.json"))).items()}
|
||||
for p in sys.argv[1:]:
|
||||
print(f"{p.split('/')[-1]:24s} | {phones(model, vocab, p)}")
|
||||
|
|
@ -59,6 +59,10 @@ def stress(root):
|
|||
out["nibu_i_150_250ms_db"] = db(rms(xb, sr, .15, .25), rms(xa, sr, .15, .25))
|
||||
return out
|
||||
|
||||
def release_at(c0, c1):
|
||||
"""Where a stop's articulators begin to open: StopReleaseStartSeconds in PatchScript.cs (the end less min(12 ms, 0.3 of the gesture))."""
|
||||
return c1 - min(0.012, 0.3 * (c1 - c0))
|
||||
|
||||
def burst(root, name, c0, c1, sub="out"):
|
||||
"""c0..c1: the b gesture's start and end in seconds; v1: the vowel's steady part after it."""
|
||||
x, sr = load(root, sub, name); nb, _ = load(root, "outnb", name)
|
||||
|
|
@ -70,11 +74,12 @@ def burst(root, name, c0, c1, sub="out"):
|
|||
below = [k for k in range(int(tpk * 500), len(env)) if env[k] < pk * .1]
|
||||
decay_ms = round((below[0] / 500 - tpk) * 1000, 0) if below else None
|
||||
last = x[int((c1 + .06) * sr):]
|
||||
return {"closure_peak": round(peak(x, sr, c0, c0 + .6 * (c1 - c0)), 4), "release_peak_full": round(peak(x, sr, c0 + .6 * (c1 - c0), c1 + .005), 4),
|
||||
rel = release_at(c0, c1)
|
||||
return {"closure_peak": round(peak(x, sr, c0, rel), 4), "release_peak_full": round(peak(x, sr, rel, c1 + .005), 4),
|
||||
"burst_peak": round(pk, 4), "burst_peak_ms": round(tpk * 1000), "burst_to_tenth_ms": decay_ms, "vowel_peak": round(vp, 4),
|
||||
"burst_re_vowel_db": db(pk, vp), "release_peak_re_vowel_db": db(peak(x, sr, c0 + .6 * (c1 - c0), c1 + .005), vp),
|
||||
"burst_re_vowel_db": db(pk, vp), "release_peak_re_vowel_db": db(peak(x, sr, rel, c1 + .005), vp),
|
||||
"burst_rms_in_30ms_after_peak_re_vowel_rms_db": db(rms(d, sr, tpk, tpk + .03), rms(x, sr, c1 + .06, c1 + .17)),
|
||||
"burst_peak_from_120ms_after_release_to_end": round(peak(d, sr, c1 + .12, n / sr), 4), "burst_diff_samples_nonzero_before_closure_end": int(np.count_nonzero(np.abs(d[:int((c0 + .6 * (c1 - c0)) * sr)]) > 1e-9)),
|
||||
"burst_peak_from_120ms_after_release_to_end": round(peak(d, sr, c1 + .12, n / sr), 4), "burst_diff_samples_nonzero_before_closure_end": int(np.count_nonzero(np.abs(d[:int(rel * sr)]) > 1e-9)),
|
||||
"file_end_rms_last_100ms": round(rms(x, sr, len(x) / sr - .1, len(x) / sr), 5), "vowel_rms": round(rms(x, sr, c1 + .06, c1 + .17), 4)}
|
||||
|
||||
BURST_BANDS = ((0, 1000), (1000, 2000), (2000, 3500), (3500, 7000), (7000, 24000))
|
||||
|
|
|
|||
Loading…
Reference in a new issue