Tract: coarticulation fixes for the tract-weak-spots-1 listening verdict (#52)

Merged as head maintainer after Soul passes 2 and 3 (aqua-body:verdict:cut-tract-coarticulation-pr52.s2, ...fix2.s3). Unheard: every clip is a measurement, not a listening.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Aqua 2026-11-13 22:54:17 +00:00
commit ca6af1d678
19 changed files with 981 additions and 71 deletions

View file

@ -0,0 +1,76 @@
# tract-weak-spots-1: why the predictions failed, and what was changed
The verdict is `tract-weak-spots-1-meta-20261113.md`. Every falsifiable prediction in the kit failed although the formant numbers
behind them were right. The kit measured the steady part of each vowel (35-85 % of the clip) and the peak of each burst; the faults
Meta heard sit in the first 100 ms of every clip, at the phone boundaries, and in how long the burst lasts. Nothing in the old
measurement set looked there. Everything below was measured on the clips Meta heard (`weak-kit-out/clips`, WordRender at 140 Hz) before
any change, and again after.
## Causes found (measured, not guessed)
| # | what she heard | cause | evidence (before) |
|---|---|---|---|
| 1 | nibu: "both consonants are just clicks"; leak-before: "reversed percussive hit" | The merged control curve of a surface holds the previous phone's value to the boundary and steps to the next phone's first value there (two points at one time). A step in a ladder cell's area is a one-sample transient. | Pitch-period window peaks: 0.399 at 95 ms and 0.487 at 275 ms, against vowel pulses of ~0.1-0.2; both instants are phone boundaries of the script (`start=0.095`, `0.275`). The whole-render peak the kit reported (0.487) was this click. Click ratio 3.54 (`ear_metrics.py`). The old report called 0.487 vs 0.399 a "leak" difference. |
| 2 | every clip starts as a fade-in; the "kazoo" with "no audible closure" (ba, no burst); the /ɪ/ not like "bit" | `si.smoo` starts every control slider at 0. A word began with the tract shut, the vowel weights at 0 and the source off, and climbed for ~100 ms. The 140 ms "closure" of ba was largely this climb. | Isolated /ɪ/: level -66 dB at 0 ms, -45 at 40 ms, -30 at 100 ms, -28 from 140 ms; LPC bandwidth of F1 367 Hz at 0 ms, 232 at 60 ms, 103 at 100 ms, 90 from 140 ms. 100 of the clip's 309 ms were the climb; the steady-state window (35-85 %) started after it. |
| 3 | ba with burst: "fah then bah" | The stop opened over 0.65 to 1.0 of its gesture: 33 ms of a 95 ms /b/. A burst of that length is a fricative. | Burst-notes: ba burst 0.0707-0.1039 s (33 ms). A recogniser reads the clip as `v ɑ` (with the burst), `b ɑ` without. |
| 4 | nasals: "basketball buzzer, no audible articulation", /n/ and /m/ alike | The murmur had no entry and exit transitions (the step of cause 1) and began after the climb of cause 2; the machine ear read /n/ as /m/ (`v ɑ m ɑ`). | recogniser, before: ana -> `v ɑ m ɑ`. |
| 5 | bun with burst: "a truck releasing its pneumatics" | cause 3 (long burst), same mechanism. | burst 0.0721-0.0998 s (28 ms). |
## What was changed (PR "phone transitions", this branch)
1. `PatchScript.WithPhoneTransition`: every authored gesture spline starts, when an earlier gesture left the surface at another value,
with a 30 ms ramp from the previous value (at most 0.4 of the gesture, and before its second point). The first phone of a word and
surfaces no earlier phone touched are unchanged. Test: `PhoneTransitionTests.NoSurfaceStepsAtAPhoneBoundary` (worst step in 1 ms
around each boundary of nibu: below 0.03; before, 0.77: the vowel's 1 against the nasal's released 0.23).
2. A stop holds all its articulators shut until 12 ms before its end and opens in those 12 ms (`StopReleaseSeconds`, a design value);
it used to open over 0.65 to 1 of its gesture. Test: `AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture`.
3. `smoo_init` replaces `si.smoo` on every control-surface and spline-point slider of the emitted DSP: the smoother starts at its input
and keeps si.smoo's pole (a 22.7 ms time constant), so live control moves as fast as before. Patch parameters keep `si.smoo`. The
source's own 10 ms pitch-ratio smoother starts at its input too. Tests: `ControlSmoothersStartAtTheirInputNotAtZero` (text) and
`ASmootherStartsAtItsInputAndMovesAtSiSmoosPace` (the emitted smoother, rendered).
4. The word scaffold's vocal envelope gets a 12 ms attack (it was 0, so the now-instant start was itself a click: first-sample peak 0.22).
5. The ramp into a stop (item 1) sweeps the contact through every aperture between open and shut, and the exporter's burst noise is
`4 * opening * (1 - opening)` times a gain that was held through the whole gesture: each closing ramp was a 15-20 ms hiss before the
stop (>4 kHz share of the 6 ms frames at 188 ms of /ata/: -1 dB, against -57 dB with the closure stepped). The burst gain now exists
only from the stop's release start (`PatchScript.StopReleaseStartSeconds`, the one owner of the release time), one ramp either side.
A stop's lips are shut (aperture 0), not at 0.04: the word-initial /b/ voice bar read -19.1 dB re the vowel after it and /p/ -30.1
against the design -34 to -26 and -60 to -38; shut they read -27.7 and -38.8. Spline handle times follow their segment (a third of
it), so the C# timeline and the DSP agree on the 12 ms opening (the lowering ignores handle times). Tests:
`ClosingIntoAStopMakesNoHighFrequencyNoise`, `TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt`, `PlosiveClosureCarriesTheVoiceBarOnlyWhenVoiced`.
6. `tools/formant-check/ear_metrics.py` (click ratio, largest sample step, LPC bandwidths, HNR, energy above 4 kHz) and
`tools/formant-check/phone_probe.py` (a speech recogniser as a machine ear) so these faults are measured, not heard first by Meta.
### After (same words, same 140 Hz, `fix-out-c`)
| word | before | after |
|---|---|---|
| nibu_flat | click ratio 3.54, peaks 0.399 / 0.487; recogniser `i b u` | peaks at the boundaries 0.077 / 0.089; click ratio 1.79 (the vowel's own pulses); median sample step in the /b/ closure 0.0007 (0.0005 before; 0.0161 with the ramp hiss); recogniser `n i l t uː` as read before the closure-noise and shut-lips fixes (not re-run: no torch on this host; the /n/ is back, the /b/ reads as `l t`: see below) |
| ba (burst) | recogniser `v ɑ`; burst 33 ms | `b ɑ`; opening 12 ms |
| ba (no burst) | `b ɑ` | `m ɑ` |
| ana / ama | `v ɑ m ɑ` / `v ɑ m ɑ` | `ɑ n ɑ` / `ɑ m ɑ` |
| isolated /i/ | `h iː`; level climb 100 ms | `i`; level within 3 dB of the steady value by 20 ms |
| isolated /ɪ/ | `h i` | `ʔ i` |
## Not fixed, and why (open until she listens)
- **Voice quality: "kazoo with a two-pack-a-day smoking habit" (the two clips she kept), "reedy, like Pink Trombone" (/ʊ/), "basketball
buzzer" (nasals).** Her comments on the two keeps say the vowel is right and the voice is wrong. Candidates, none tested by ear:
(a) the glottal source is a band-limited saw through a one-pole lowpass with no closed phase (`FaustExport.PrimitiveSourceExpression`),
where a Rosenberg/LF pulse has the open and closed phases real fold flow has; (b) harmonics-to-noise ratio of the steady part is 13-15 dB
(healthy voices ~20+), from the source's `0.08 * noise` term and the saw's aliasing; (c) LPC bandwidths of the steady vowel are 90 Hz (F1),
130 (F2), 100 (F3) for /ɪ/: plausible, so the wall-loss hypothesis is weak (the F2 of /ɑ/, 29 Hz, and the F1 of /i/, 14-24 Hz, are LPC
artefacts of a pole sitting on a harmonic). Settling this needs a listening kit that varies one of them at a time; the next kit does.
- **/ɪ/ as the "eu" of cordon bleu.** The recogniser cannot judge this tract's vowel quality (every isolated vowel came back /i/). The
confirmed fault in that clip is cause 2: a third of it was a muffled climb through bandwidths of 300 Hz. Whether the steady /ɪ/ then
reads as "bit" is for her ear; lax vowels in isolation and without vowel-inherent spectral change are known to be misheard
(Hillenbrand et al. 1995, Strange et al. 1983), so the next kit also carries /ɪ/ in a word.
- **The medial /b/ of nibu reads as `l t` to the recogniser after the fix, where it read `b` with the click and the long burst.** The closure
voice bar sat 10 dB under the vowel (a murmur-like level; a stop's runs 20+ dB under; 16-19 dB with the lips shut), and a labial burst is injected at the glottal
end in a word with no other plosive (`FaustExport` burst injection), so it is filtered by the whole tract like aspiration. Both are
next, with the listening to decide.
## What the old measurements missed, as a rule
A steady-state window measures a steady state. Faults of onset, boundary and duration need metrics that look at onsets, boundaries
and durations: click ratio and step size at each phone boundary, time-to-full-level, burst length, bandwidths, HNR. These now exist
(`ear_metrics.py`) and the click and onset ones are pinned by tests.

View file

@ -0,0 +1,231 @@
{
"schema": "aqua.listen.verdict.v1",
"kit": {
"id": "tract-weak-spots-1",
"title": "Articulatory vocal tract: stress, release, nasals, lax vowels"
},
"exportedAt": "2026-11-13T11:27:36.539Z",
"seed": 1326320895,
"consent": true,
"listener": "Meta",
"device": null,
"notes": "",
"summary": {
"total": 14,
"keep": 2,
"fix": 12,
"drop": 0,
"unanswered": 0
},
"clips": [
{
"position": 1,
"id": "lax-ih",
"title": "isolated /ɪ/ 180 ms, 140 Hz. Measured: Burg F1/F2 375/2024 Hz; tube chain 346/1987 (+8.5/+1.8 %); Hillenbrand 1995 male 429/2034 (-12.5/-0.5 %); Peterson & Barney 392/1993 (-4.3/+1.5 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'bit' (or a slightly high, 'ee'-leaning version of it). FALSIFIED if heard as 'ee' of 'beet', as 'eh' of 'bet', or as a different phone.",
"group": "lax",
"feel": "the short vowel in 'bit'",
"file": "v_ih.wav",
"pair": null,
"slot": null,
"verdict": "fix",
"reason": "It's more like the eu in cordon bleu than the i in bit ",
"landed": 2,
"device": null
},
{
"position": 2,
"id": "burst-ba-on",
"title": "ba with the plosive burst (gain 0.06, a design value). Measured: burst alone peaks 0.0142 at 85 ms = -24.0 dB re vowel peak, falls to a tenth in 17 ms, burst rms in the 30 ms after its peak -31.0 dB re vowel rms. PREDICTION: a slightly crisper, more 'b'-like onset than B/off; the burst is quiet and short so the difference is small, 'a little pop' at most. FALSIFIED if it sounds like a hiss, a click, or a separate noise after the vowel started, or if no difference is heard.",
"group": "burst",
"feel": "the syllable 'bah' with a clean, crisp 'b' at the start",
"file": "ba.wav",
"pair": "burst-ba-off",
"slot": "A",
"verdict": "fix",
"reason": "More like fah then bah but you can hear something there",
"landed": 2,
"device": null
},
{
"position": 3,
"id": "burst-ba-off",
"title": "ba with the release_flow splines dropped (WordRender --no-burst). Measured: closure peak 0.0078 (voice bar), release (voicing breaking through)peak 0.1087 = -6.3 dB re vowel peak 0.2247, no burst noise. PREDICTION: a soft, voiced 'b' with the vowel starting smoothly; may sound slightly 'w'-like or lack the pop. FALSIFIED if listeners cannot hear any difference from the burst twin (then burst gain 0.06 is inaudible).",
"group": "burst",
"feel": "the syllable 'bah' with a soft start",
"file": "ba_noburst.wav",
"pair": "burst-ba-on",
"slot": "B",
"verdict": "fix",
"reason": "There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo",
"landed": 2,
"device": null
},
{
"position": 4,
"id": "stress-nibu-flat",
"title": "nibu, no stress marks. Measured: /i/ 150-250 ms rms vs the stressed render +0.21 dB; unstressed vowel 180 ms, f0 139.9 Hz. PREDICTION: reads asan even two-syllable word with no strong first syllable. FALSIFIED if listeners hear it as stressed on a syllable or cannot tell it from the stressed twin.",
"group": "stress",
"feel": "a made-up two-syllable word, 'nee-boo', both syllables even",
"file": "nibu_flat.wav",
"pair": "stress-nibu",
"slot": "A",
"verdict": "fix",
"reason": "These both sound like 'ee'oo, with both consonants you're going for being just clicks",
"landed": 2,
"device": null
},
{
"position": 5,
"id": "stress-nibu",
"title": "nibu with primary stress on /i/ (NI-bu). Measured: /i/ 247 ms vs 180 ms (+75 ms; design +68), f0 148.1 vs 139.9 Hz (x1.059), amplitude factor 1.30; level of /i/ in the 150-250 ms window only +0.21 dB (five-vowel mean +1.93 dB, per vowel +1.2 to +3.0). PREDICTION: first syllable heard as the prominent one, mostly through length and pitch, little through loudness; B should be picked as 'stressed on the first syllable'. FALSIFIED if it sounds no more prominent than the flat twin, if it sounds like a question or a pitch accent rather than stress, or if the level is what is heard (then the f0-scatter question is wrong).",
"group": "stress",
"feel": "the same made-up word, with the first syllable stronger: 'NEE-boo'",
"file": "nibu.wav",
"pair": "stress-nibu-flat",
"slot": "B",
"verdict": "fix",
"reason": "",
"landed": 2,
"device": null
},
{
"position": 6,
"id": "stress-i-flat",
"title": "isolated /i/ 180 ms at 140 Hz, no stress. Measured: level 100-170 ms is the reference. PREDICTION: plain short vowel. FALSIFIED if it already sounds emphasised.",
"group": "stress",
"feel": "a single short 'ee' vowel, plain",
"file": "v_i.wav",
"pair": "stress-i",
"slot": "A",
"verdict": "keep",
"reason": "Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee",
"landed": 4,
"device": null
},
{
"position": 7,
"id": "stress-i",
"title": "isolated /i/ with primary stress: 247 ms, f0 x1.06, amplitude 1.30. Measured against the flat /i/: +1.22 dB total (f0 rise alone -0.80 dB, the amplitude and the rest +2.02 dB). PREDICTION: heard as longer and a little higher; the 1.2 dB louder is under what a listener can judge, so 'longer and higher' is the answer, 'louder' is not. FALSIFIED if it sounds the same length and pitch, or clearly louder.",
"group": "stress",
"feel": "the same 'ee' vowel, held longer and said with emphasis",
"file": "v_i_s.wav",
"pair": "stress-i-flat",
"slot": "B",
"verdict": "keep",
"reason": "",
"landed": 4,
"device": null
},
{
"position": 8,
"id": "lax-oo-short",
"title": "isolated /ʊ/ 180 ms, 140 Hz. Measured: Burg F1/F2 486/894 Hz; tube chain 478/850 (+1.7/+5.2 %); Hillenbrand 469/1123 (+3.7/-20.4 %); Peterson &Barney 437/1023 (+11.3/-12.6 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'book', probably darker or rounder than a natural one (F2 20 % under the population mean, the back-vowel gap). FALSIFIED if heard as 'oo' of 'boot', 'o' of 'boat' or 'uh' of 'but'.",
"group": "lax",
"feel": "the short vowel in 'book'",
"file": "v_oo_short.wav",
"pair": null,
"slot": null,
"verdict": "fix",
"reason": "It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared",
"landed": 3,
"device": null
},
{
"position": 9,
"id": "leak-nibu-flat-after",
"title": "nibu (flat) after the fix: release_flow is zero outside its own plosive. Same audio as stress-nibu-flat, kept as its own clip so the before/after pair is one-to-one. Measured: leak 0.0 from 275 ms on, burst of the b itself unchanged (bin 0.0226 at 81 ms before and after). PREDICTION: clean hum into thevowel, no noisy onset; preferred over 'before' if the leak is audible. FALSIFIED if 'before' is preferred or no difference is heard.",
"group": "release-leak",
"feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'",
"file": "nibu_flat_after.wav",
"pair": "leak-nibu-flat-before",
"slot": "A",
"verdict": "fix",
"reason": "Just slightly less reverse attack on this one",
"landed": 2,
"device": null
},
{
"position": 10,
"id": "leak-nibu-flat-before",
"title": "nibu (flat) rendered at main c0e6b98, before the release_flow fix. Measured: burst-gain leak vs the fixed render, peak 0.0688 and rms 0.0067 inthe leading /n/ (0-95 ms; the /n/ itself is rms 0.0058, so the hiss is louder than the murmur), peak 0.0504 over /i/ onset 95-150 ms, 0.0019 after; whole-render peak 0.487 vs 0.399. PREDICTION: an audible hiss or click as the /n/ opens into the vowel, the first sound of the word being a noisy 'sn'-like or 'tsh' onset instead of a clean nasal hum. FALSIFIED if the two are indistinguishable (then the leak is inaudible and the fix is cosmetic).",
"group": "release-leak",
"feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'",
"file": "nibu_flat_before.wav",
"pair": "leak-nibu-flat-after",
"slot": "B",
"verdict": "fix",
"reason": "'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant",
"landed": 2,
"device": null
},
{
"position": 11,
"id": "nasal-n",
"title": "/ɑnɑ/. Measured: murmur -9.64 dB re the flanking vowels (2.17 dB above /m/; the gap is 2.01/2.17/2.05/1.61 dB at f0 110/140/170/200 Hz, so not harmonic sampling), Burg N1 332 Hz, bands 38.4/28.2/16.8/0.9 dB. PREDICTION: heard as 'ah-nah', a tongue-tip hum, a shade stronger than the /m/ one; the 2 dB isnot expected to be noticed. FALSIFIED if heard as 'ama' or 'anga', if it sounds much louder than the /m/ clip, or if there is hiss.",
"group": "nasal",
"feel": "'ah-nah': the tongue touches behind the teeth for a soft hum between two open vowels",
"file": "ana.wav",
"pair": null,
"slot": null,
"verdict": "fix",
"reason": "It just sounds like a basket ball buzzer, no audible articulation",
"landed": 2,
"device": null
},
{
"position": 12,
"id": "burst-bun-off",
"title": "bun, /bʊn/, no burst. Measured: closure peak 0.0043, release peak 0.1074 = -5.9 dB re vowel peak 0.2126. PREDICTION: soft 'b' then a short backvowel then a nasal; the final /n/ is a clean hum (no leak since the fix). FALSIFIED as for burst-ba-off.",
"group": "burst",
"feel": "a short word like 'bunn' or 'boon' said quickly, soft start",
"file": "bun_noburst.wav",
"pair": "burst-bun-on",
"slot": "A",
"verdict": "fix",
"reason": "Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there",
"landed": 2,
"device": null
},
{
"position": 13,
"id": "burst-bun-on",
"title": "bun, /bʊn/, with burst. Measured: burst peak 0.0130 at 81 ms = -24.3 dB re vowel peak, tenth in 15 ms; tail leak after the fix 0.0 (was 0.0126 at the end before). PREDICTION: slightly crisper onset than off; final /n/ clean. FALSIFIED as for burst-ba-on.",
"group": "burst",
"feel": "a short word like 'bunn' or 'boon' said quickly, crisp 'b' start",
"file": "bun.wav",
"pair": "burst-bun-off",
"slot": "B",
"verdict": "fix",
"reason": "The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human",
"landed": 2,
"device": null
},
{
"position": 14,
"id": "nasal-m",
"title": "/ɑmɑ/. Measured: murmur -11.81 dB re the flanking vowels (design band -16..-8), Burg N1 295 Hz, band levels 0-400/400-800/800-1600/1600-3200 Hzof the murmur 36.3/23.6/21.3/2.7 dB. PREDICTION: heard as 'ah-mah', a lip-closed hum between the vowels, clearly softer than the vowels. FALSIFIED if heard as 'abah' or 'awah', if the hum is as loud as the vowels, or if there is a gap or click.",
"group": "nasal",
"feel": "'ah-mah': lips close for a soft hum between two open vowels",
"file": "ama.wav",
"pair": null,
"slot": null,
"verdict": "fix",
"reason": "That m is almost coming through, still more of a basketball buzzer",
"landed": 2,
"device": null
}
],
"pairs": [
{
"a": "burst-ba-on",
"b": "burst-ba-off",
"preferred": "burst-ba-on"
},
{
"a": "stress-nibu-flat",
"b": "stress-nibu",
"preferred": "no-difference"
}
],
"_truncated": "the mail reader cut the body at 12000 characters inside the pairs list; pairs after the third are not recorded"
}

View file

@ -0,0 +1,32 @@
# Listening verdict: tract-weak-spots-1 (page tract-listen-1b6d180)
Listener: Meta (operator), 2026-11-13. Exported 11:27Z by the page (`aqua.listen.verdict.v1`, seed 1326320895), pasted in
#aquasynth (truncated after clip 5) and emailed to aqua@minds.gamecult.org at 11:30Z; the data here is the email,
parsed whole by `mind-mail read` except the tail: the mail reader cut the body at 12000 characters inside `pairs`, so
only the first two pair preferences survive. Raw data: `tract-weak-spots-1-meta-20261113.json`. Totals: 2 keep, 12 fix, 0 drop.
`position` is the page's random order for this listener; ids are the kit's.
| # | clip | verdict | what she heard |
|---|---|---|---|
| 1 | `lax-ih` (`v_ih.wav`) | fix | It's more like the eu in cordon bleu than the i in bit |
| 2 | `burst-ba-on` (`ba.wav`) | fix | More like fah then bah but you can hear something there |
| 3 | `burst-ba-off` (`ba_noburst.wav`) | fix | There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo |
| 4 | `stress-nibu-flat` (`nibu_flat.wav`) | fix | These both sound like 'ee'oo, with both consonants you're going for being just clicks |
| 5 | `stress-nibu` (`nibu.wav`) | fix | (no comment) |
| 6 | `stress-i-flat` (`v_i.wav`) | keep | Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee |
| 7 | `stress-i` (`v_i_s.wav`) | keep | (no comment) |
| 8 | `lax-oo-short` (`v_oo_short.wav`) | fix | It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared |
| 9 | `leak-nibu-flat-after` (`nibu_flat_after.wav`) | fix | Just slightly less reverse attack on this one |
| 10 | `leak-nibu-flat-before` (`nibu_flat_before.wav`) | fix | 'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant |
| 11 | `nasal-n` (`ana.wav`) | fix | It just sounds like a basket ball buzzer, no audible articulation |
| 12 | `burst-bun-off` (`bun_noburst.wav`) | fix | Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there |
| 13 | `burst-bun-on` (`bun.wav`) | fix | The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human |
| 14 | `nasal-m` (`ama.wav`) | fix | That m is almost coming through, still more of a basketball buzzer |
Pairs: `burst-ba-on` preferred over `burst-ba-off`; `stress-nibu-flat` vs `stress-nibu`: no difference.
(`leak-nibu-flat-before`/`after` and the bun pair carry no recorded preference; she heard "just slightly less reverse
attack" on the fixed one, the pneumatic-truck "air release" on the bun burst.)
Every prediction that was falsifiable failed, though the formant numbers behind them were measured correctly. The
two keeps are the isolated /i/ pair, and even those were "a kazoo with a two pack a day smoking habit". Analysis and
the fixes it led to: `tract-weak-spots-1-analysis-20261113.md`.

View file

@ -45,6 +45,15 @@ public static partial class FaustEmitter
source.AppendLine("bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;");
source.AppendLine("seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);");
source.AppendLine("smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));");
// si.smoo starts every control at 0 and climbs to it: a word began with the tract shut, the vowel weights at 0 and the source off,
// and the first ~100 ms of every render were that climb (the level rose 40 dB over the first 100 ms of an isolated vowel, and
// the bandwidths fell from ~300 Hz to 90). smoo_from starts at its input and smooths from there. smoo_init is si.smoo's own
// pole (1 - 44.1 / SR, a 22.7 ms time constant), so live control moves exactly as fast as before; only the start differs.
// Started at its input a constant control holds there to the bit; started at 0 (si.smoo, si.smooth) it stalls 1e-5 to 5e-5 short of it,
// in either way of writing the step. The source's pitch-ratio smoother starts at its input for that reason: from 0 the 2744 Hz alias
// source ran 0.12 Hz flat and leaked its harmonics into the alias measurement.
source.AppendLine("smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };");
source.AppendLine("smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);");
source.AppendLine("seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));");
source.AppendLine("seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));");
source.AppendLine("seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));");
@ -152,7 +161,7 @@ public static partial class FaustEmitter
var baseId = $"{id}_base";
var normalized = $"{id}_normalized";
var physical = $"{id}_physical";
var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : si.smoo";
var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : smoo_init";
var boundControl = parameters.TryBindingExpression(surface.FieldPath, out var bindingExpression)
? $"clip01(({bindingExpression} - {F(surface.MinValue)}) / max(0.000001, {F(surface.MaxValue - surface.MinValue)}))"
: publicControl;
@ -172,7 +181,7 @@ public static partial class FaustEmitter
var splineId = $"{id}_spline_{splineIndex}";
EmitControlSpline(source, spline, splineId, surface.DefaultNormalized);
var depth = $"{splineId}_depth";
source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : si.smoo;");
source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : smoo_init;");
terms.Add($"(({splineId}_value - {F(surface.DefaultNormalized)}) * {depth})");
}
@ -197,11 +206,11 @@ public static partial class FaustEmitter
{
var point = points[i];
source.AppendLine($"{splineId}_p{i}_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/time\", {F(point.TimeSeconds)}, 0, 120, 0.001);");
source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : si.smoo;");
source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : smoo_init;");
source.AppendLine($"{splineId}_p{i}_out_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_time\", {F(Math.Max(0, point.OutTimeOffsetSeconds))}, 0, 10, 0.001);");
source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : si.smoo;");
source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : smoo_init;");
source.AppendLine($"{splineId}_p{i}_in_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_time\", {F(Math.Max(0, point.InTimeOffsetSeconds))}, 0, 10, 0.001);");
source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : si.smoo;");
source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : smoo_init;");
}
if (points.Length == 1)
@ -739,7 +748,9 @@ public static partial class FaustEmitter
var impedance = parameters.Expression(OwnerField(owner, "impedance"), port.Impedance);
var flowScale = parameters.Expression(OwnerField(owner, "flow_scale"), port.FlowScale);
// f0 is the voice frequency times the source's ratio (the stress cue), smoothed over 10 ms so a step in the ratio is a glide.
var pitchRatio = $"({parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)} : si.smooth(ba.tau2pole(0.01)))";
// Its own 10 ms smoother also starts at its input: from 0 it glided the source's pitch up from nothing at the start of every word
// (the control feeding it used to hide that under its own 100 ms climb) and rounded to a value up to ~4e-5 off its input.
var pitchRatio = $"smoo_from(ba.tau2pole(0.01), {parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)})";
var load = $"{voiceName}_primitive_source_{safe}_load_pressure";
var voiced = $"{voiceName}_primitive_source_{safe}_voiced";
var flow = $"{voiceName}_primitive_source_{safe}_flow";

View file

@ -278,7 +278,7 @@ public static class IpaGestureExperiment
probe_timeline name=flow network=voice blocks=12 block_size=64
{{SideStubDeclarations(sideStub)}}vocal_network name=voice paths=oral_path,nasal_path{{(sideStub ? ",side_stub_path" : "")}} sources=folds contacts=contact branches=velopharynx{{(sideStub ? "," + PatchScript.SideStubBranch : "")}} radiation=mouth,nostril{{(sideStub ? ",side_stub_end" : "")}} probe=flow
{{string.Join("\n", gestureLines)}}
vocal network=voice freq={{Format(frequency)}} gain=.7 sustain={{Format(sustain)}}
vocal network=voice freq={{Format(frequency)}} gain=.7 attack=.012 sustain={{Format(sustain)}}
""";
private static IReadOnlyList<IpaGestureMetricSummary> SummarizeMetrics(IReadOnlyList<IpaGestureExperimentMetric> metrics) =>

View file

@ -17,6 +17,22 @@ public static class PatchScript
/// A script that gestures a lateral without declaring it is refused, not approximated.</summary>
public const string SideStubBranch = "side_stub";
// Seconds an articulator takes to move from where the previous phone left it to this phone's first value. A tract does not step
// between phones: the merged surface curve held the previous value up to the boundary and jumped, and a jump in a ladder cell's
// area is a transient the ear hears as a click (nibu: peak 0.40 and 0.49 against a 0.2 vowel, at the 95 and 275 ms boundaries),
// and it leaves the consonant without the formant transitions that tell /b/ from a fade-in. A DESIGN VALUE inside the 20-50 ms
// of a CV formant transition; no measurement of this tract retrieved.
public const float PhoneTransitionSeconds = 0.03f;
// Seconds a stop takes to open from full closure; its burst lasts about this long. A DESIGN VALUE inside the 5-15 ms of a
// stop burst; no measurement of this tract retrieved. The opening's noise is still shaped by the exporter.
private const float StopReleaseSeconds = 0.012f;
/// <summary>Seconds after the start of a stop gesture of the given duration at which its articulators begin to open (the stop is
/// shut until then). The one owner of the release time: the contact, lip and constriction splines hold shut to here and the burst
/// gain window opens here; a test that wants the release reads it from here.</summary>
public static float StopReleaseStartSeconds(float durationSeconds) => durationSeconds - MathF.Min(StopReleaseSeconds, 0.3f * durationSeconds);
public static bool TryIpaDescriptor(string ipa, out string descriptor)
{
if (StoryVowelAreas.TryVowelForIpa(ipa, out var vowelDescriptor, out _))
@ -1494,11 +1510,58 @@ public static class PatchScript
duration * 0.25f,
Math.Clamp(point.value, 0, 1)))
.ToArray();
splinePoints = FollowSegments(WithPhoneTransition(surfacePath, start, duration, splinePoints));
var spline = new ControlSpline(splineName, surfacePath, splinePoints, ControlSplineInterpolation.Bezier);
_controlSplines.Add(spline);
_controlSplinesByName[spline.Name] = spline;
}
// Prepends the boundary ramp: a point at the gesture start holding the value the surface has there, then the gesture's first
// point PhoneTransitionSeconds later (at most 0.4 of the gesture, and before its second point). The first phone of a word, and
// a surface the earlier phones never touched, keep their value from the start.
private ControlSplinePoint[] WithPhoneTransition(string surfacePath, float start, float duration, ControlSplinePoint[] points)
{
ControlSplinePoint? previous = null;
foreach (var spline in _controlSplines)
{
if (!spline.SurfacePath.Equals(surfacePath, StringComparison.OrdinalIgnoreCase)) continue;
foreach (var point in spline.Points)
{
if (point.TimeSeconds <= start + 0.000001f && (previous is null || point.TimeSeconds >= previous.TimeSeconds)) previous = point;
}
}
if (previous is null || points.Length < 2 || MathF.Abs(previous.Value - points[0].Value) <= 0.0005f) return points;
var ramp = MathF.Min(PhoneTransitionSeconds, 0.4f * duration);
ramp = MathF.Min(ramp, points[1].TimeSeconds - points[0].TimeSeconds);
if (ramp < 0.002f) return points;
var handle = ramp / 3;
var first = points[0];
var lead = new ControlSplinePoint(first.TimeSeconds, previous.Value, handle, previous.Value, handle, previous.Value);
var arrival = first with { TimeSeconds = first.TimeSeconds + ramp, InTimeOffsetSeconds = handle, InValue = first.Value };
return [lead, arrival, .. points.Skip(1)];
}
// Sets every handle's time to a third of the segment it belongs to. The Faust lowering evaluates a segment's Bezier on the
// linear fraction of its time and ignores the handle times; a handle of a third makes the C# timeline (which solves the curve's
// time for its parameter) agree with it exactly. A handle longer than the segment (0.25 of the gesture on a 12 ms opening) folds
// the C# curve back on itself.
private static ControlSplinePoint[] FollowSegments(ControlSplinePoint[] points)
{
if (points.Length < 2) return points;
var shaped = new ControlSplinePoint[points.Length];
for (var i = 0; i < points.Length; i++)
{
var after = (i + 1 < points.Length ? points[i + 1].TimeSeconds - points[i].TimeSeconds : points[i].TimeSeconds - points[i - 1].TimeSeconds) / 3;
var before = (i > 0 ? points[i].TimeSeconds - points[i - 1].TimeSeconds : points[1].TimeSeconds - points[0].TimeSeconds) / 3;
shaped[i] = points[i] with { InTimeOffsetSeconds = before, OutTimeOffsetSeconds = after };
}
return shaped;
}
private string UniqueSplineName(string name)
{
var safe = SafeName(name);
@ -1561,18 +1624,25 @@ public static class PatchScript
// closed-mouth sound, and an alveolar closure that began to open at 0.5 let the oral tract radiate again over the last third (/n/ read
// -3.8 dB under its vowels against /m/ -13.2 dB, whose lips hold to 0.7).
var stopsBehindTheLips = (tokens.Contains("plosive") || tokens.Contains("stop")) && !(tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial"));
// A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (0.65): at the nasal's 0.03 cm
// A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (StopReleaseStartSeconds): at the nasal's 0.03 cm
// aperture a /t/ closure measured -10.4 dB re its vowels against /p/ -19.7 (the lips seal /p/ whatever the tongue does), and
// rendered -21.5 dB with the aperture at 0 (area floor 1e-6 cm^2).
var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : stopsBehindTheLips ? 0.65f : 0.5f;
// Every stop holds all its articulators shut until StopReleaseSeconds before its end and opens in that time: the burst is the
// opening, and an opening spread over a third of the gesture (33 ms of the 95 ms /b/) made the burst a fricative's length
// (the phone recogniser heard /v/ before the vowel, Meta heard "fah then bah").
var isStop = tokens.Contains("plosive") || tokens.Contains("stop");
var stopHold = StopReleaseStartSeconds(duration) / duration;
var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : isStop ? stopHold : 0.5f;
var closed = stopsBehindTheLips ? 0f : constriction;
AddAuthoredSpline($"{name}_constriction_diameter", "/vocal/areas/0/area/constriction_diameter", start, duration, (0, closed), (closureHold, closed), (1, Math.Min(0.65f, constriction + 0.2f)));
if (tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial") || tokens.Contains("rounded"))
{
var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : 0.04f;
AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.45f));
AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.75f));
// A stop's lips are shut, not at a 0.04 leak: at 0.04 the closed-mouth voice bar of /b/ read -19.1 dB re the following vowel (the
// test's design value is -34 to -26) and /p/ -30.1 (-60 to -38); shut, -27.7 and -38.8. A nasal keeps its murmur's small opening.
var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : isStop ? 0f : 0.04f;
AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.45f));
AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.75f));
}
if (tokens.Contains("nasal"))
@ -1582,13 +1652,18 @@ public static class PatchScript
}
else if (tokens.Contains("plosive") || tokens.Contains("stop"))
{
AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (0.65f, 0.02f), (1, 0.9f));
AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (0.65f, 0.75f), (1, 0.05f));
AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (stopHold, 0.02f), (1, 0.9f));
AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (stopHold, 0.75f), (1, 0.05f));
// The release burst is turbulence at the opening constriction: its gain is held for the plosive and the exporter shapes
// it by the stored pressure and the opening (see PrimitiveContactExpression). The gain exists only around this plosive:
// a surface holds its last value after its last point (and its first before its first), so a gain left at its level
// made noise at every later contact state, the final nasal's closure and the tail included.
GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start, duration, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain);
// The exporter's burst is noise times 4 * opening * (1 - opening): it sounds whenever the contact passes between shut and open
// in either direction, so a gain held through the closure made the 30 ms closing ramp a 15-20 ms hiss before every stop (the
// >4 kHz share of the frames before /t/ was -1 dB against -57 dB with the closure stepped). The gain exists only from the
// stop's release start to its end, one ramp (0.15 of that window) either side.
var releaseStart = StopReleaseStartSeconds(duration);
GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start + releaseStart, duration - releaseStart, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain);
PlaceBurst(name, tokens, place, start, duration);
AddAuthoredSpline($"{name}_velum", "/vocal/branches/0/opening", start, duration, (0, VelumClosed), (1, VelumClosed));
}
@ -1670,15 +1745,14 @@ public static class PatchScript
}
var level = Math.Clamp((gain - surface.MinValue) / (surface.MaxValue - surface.MinValue), 0, 1);
var handle = duration * 0.25f;
ControlSplinePoint At(float time, float value) => new(time, value, handle, value, handle, value);
ControlSplinePoint At(float time, float value) => new(time, value, 0, value, 0, value);
var ramp = duration * 0.15f;
var points = new List<ControlSplinePoint>();
if (start > 0) points.Add(At(Math.Max(0, start - ramp), 0));
points.Add(At(start, level));
points.Add(At(start + duration, level));
points.Add(At(start + duration + ramp, 0));
var spline = new ControlSpline(UniqueSplineName(name), surfacePath, points.ToArray(), ControlSplineInterpolation.Bezier);
var spline = new ControlSpline(UniqueSplineName(name), surfacePath, FollowSegments(points.ToArray()), ControlSplineInterpolation.Bezier);
_controlSplines.Add(spline);
_controlSplinesByName[spline.Name] = spline;
}

View file

@ -68,7 +68,7 @@ public sealed class NasalCouplingTests
}
private static float Spline(SynthPatch patch, string namePrefix, string surfacePath) =>
patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[0].Value;
patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[^1].Value; // a held place: the last point is the gesture's value whether or not a boundary ramp precedes it
// Distance of a vowel_set consonant constriction from the lip plane in cm, from the constriction_index spline: the surface is a section
// index in [0, sections] and the area model reads position = index / (sections - 1).

View file

@ -0,0 +1,84 @@
using AquaSynth.Dsl;
using AquaSynth.Faust;
namespace AquaSynth.Dsl.Tests;
// What Meta heard in tract-weak-spots-1 (2026-11-13) while every formant matched: clicks at the phone boundaries of nibu, a /b/ burst the
// length of a fricative, and a first 100 ms that climbed from silence. These pin the three causes at the control-spline and DSP-source
// level (docs/listening/tract-weak-spots-1-analysis-20261113.md); the renders are measured by tools/formant-check/ear_metrics.py.
public sealed class PhoneTransitionTests
{
private static SynthPatch Nibu() => PatchScript.Parse(IpaGestureExperiment.BuildWordScript(
[
(new IpaGestureExperimentTarget("w_n", "n", "", .095f), 1f),
(new IpaGestureExperimentTarget("w_i", "i", "", .18f), 1f),
(new IpaGestureExperimentTarget("w_b", "b", "", .095f), 1f),
(new IpaGestureExperimentTarget("w_u", "u", "", .18f), 1f)
], frequency: 140));
[Fact]
public void NoSurfaceStepsAtAPhoneBoundary()
{
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
var worst = (Surface: "", Boundary: 0f, Step: 0f);
foreach (var boundary in new[] { .095f, .275f, .37f })
{
foreach (var surface in timeline.SurfacePaths)
{
var step = MathF.Abs(timeline.ValueAt(surface, boundary + .0005f) - timeline.ValueAt(surface, boundary - .0005f));
if (step > worst.Step) worst = (surface, boundary, step);
}
}
// 0.5 ms either side of a boundary; a 30 ms ramp across the full 0..1 range moves 0.017 in that time, a step moves it by the
// whole difference (the pre-fix nibu stepped /vocal/areas/0/area/constriction_diameter by 0.77 at the 95 ms boundary).
Assert.True(worst.Step < .03f, $"{worst.Surface} steps {worst.Step:0.000} at {worst.Boundary} s");
}
[Fact]
public void ThePhoneAfterABoundaryArrivesAtItsFirstValueWithinTheTransition()
{
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
const string diameter = "/vocal/areas/0/area/constriction_diameter";
var closedN = timeline.ValueAt(diameter, .05f);
Assert.True(closedN < .1f, "the /n/ constriction is shut");
Assert.True(timeline.ValueAt(diameter, .095f + .002f) < .5f, "still closing into the vowel two ms after the boundary");
Assert.Equal(1f, timeline.ValueAt(diameter, .095f + .035f), 2);
}
[Fact]
public void TheFirstPhoneOfAWordStartsAtItsOwnValue()
{
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
Assert.Equal(timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", 0f), timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", .02f), 3);
}
[Fact]
public void AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture()
{
var timeline = new ControlSplineTimeline(Nibu().ControlSplines);
const string contact = "/vocal/contacts/0/opening";
const float start = .275f, duration = .095f;
Assert.True(timeline.ValueAt(contact, start + duration - .013f) < .1f, "closed until 12 ms before the end");
Assert.True(timeline.ValueAt(contact, start + duration) > .8f, "open at the end");
var opening = 0f;
for (var t = start + .03f; t <= start + duration; t += .0005f)
{
if (timeline.ValueAt(contact, t) > .3f) { opening = t; break; }
}
Assert.True(start + duration - opening < .012f, $"the opening crosses 0.3 only {start + duration - opening:0.000} s before the end");
}
[Fact]
public void ControlSmoothersStartAtTheirInputNotAtZero()
{
var source = FaustEmitter.Emit(Nibu(), new FaustExportOptions("transition_probe")).Source;
Assert.Contains("smoo_init(x) =", source);
Assert.Contains("smoo_init;", source);
// Every control surface and spline-point slider uses it; patch parameters keep si.smoo.
var remaining = source.Split('\n').Where(line => line.Contains("hslider", StringComparison.Ordinal) && line.Contains(": si.smoo;", StringComparison.Ordinal)
&& (line.Contains("\"/vocal/", StringComparison.Ordinal) || line.Contains("\"/splines/", StringComparison.Ordinal))).ToArray();
Assert.Empty(remaining);
}
}

View file

@ -16,9 +16,10 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
private static float ConstrictionIndex(string script, string name) =>
PatchScript.Parse(script).ControlSplines
.Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[0].Value;
.Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[^1].Value; // the held place; a boundary ramp may precede it
// Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the
// Re-pinned on the phone-transition cut (boundary ramps, 12 ms stop release, smoo_init, vocal attack): these ids moved on purpose; the invariant
// that survives is that a word of labials writes no position surface (below), not these bytes. Faust ids re-pinned again on the PR 52 fix passes (smoo_init pole, pitch-ratio smoother, shut lips for stops, spline handles; the scripts are unchanged; the last move is the smoo_from step line only, which reverts to prev * pole + x * (1 - pole): the old ids come back when that line is put back). Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the
// Faust source for /apa/ and /aba/, which this cut must not change by a byte; the ids are a SHA-256 prefix).
private const float BIndex = 0.8333333f, NIndex = 0.7118056f;
@ -28,7 +29,7 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
var b = ConstrictionIndex(Word("b"), "w_1_c");
var n = ConstrictionIndex(Word("n"), "w_1_c");
output.WriteLine($"b {b:R} n {n:R}");
foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE") })
foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C") })
{
var script = Word(ipa);
var faust = FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("pin", SampleRate: 48000)).Source;
@ -47,8 +48,8 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
{
foreach (var (ipa, scriptSha, faustSha) in new[]
{
("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE"),
("n", "BDF24A66659D7BD6", "ABEB5D03DC9637DB"), ("t", "9DFBCDB5131F9776", "30E144D54F7E026F"),
("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C"),
("n", "5C3FA24E40D9DE26", "B2B9C5E1015B60A9"), ("t", "45941CCFD93561B3", "0AACE6D26F648FB0"),
})
{
var script = Word(ipa);
@ -109,8 +110,9 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
var script = IpaGestureExperiment.BuildWordScript(
[(Phone("0_b", "b", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (new IpaGestureExperimentTarget("w_2_t", "t", "voiceless_alveolar_plosive", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_p", "p", 0.095f), 1f)], 140);
Assert.All(SitePoints(script, "w_0_b"), v => Assert.Equal(0f, v));
Assert.All(SitePoints(script, "w_4_p"), v => Assert.Equal(0f, v));
Assert.All(SitePoints(script, "w_2_t"), v => Assert.True(v > 0.8f));
Assert.All(SitePoints(script, "w_4_p").Skip(1), v => Assert.Equal(0f, v));
// after the 30 ms boundary ramp from the previous plosive's place (the first point is that place)
Assert.All(SitePoints(script, "w_2_t").Skip(1), v => Assert.True(v > 0.8f));
}
[Fact]
@ -120,11 +122,13 @@ public sealed class PlaceTableTests(ITestOutputHelper output)
.Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points.Select(p => p.Value).ToArray();
var t = PatchScript.Parse(Word("t", "voiceless_alveolar_plosive")).ControlSplines
.Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points;
Assert.Equal(0f, t[0].Value);
// lead-in (the vowel's open 1), arrival, closure hold, end: shut from the arrival to the hold, and the hold ends StopReleaseSeconds before the stop does
Assert.Equal(1f, t[0].Value);
Assert.Equal(0f, t[1].Value);
Assert.Equal(0.65f, (t[1].TimeSeconds - 0.18f) / 0.095f, 3);
Assert.Equal(0f, t[2].Value);
Assert.Equal(1 - 0.012f / 0.095f, (t[2].TimeSeconds - 0.18f) / 0.095f, 3);
var b = Diameter(Word("b"));
Assert.True(b[0] > 0 && Math.Abs(b[0] - b[1]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)");
Assert.True(b[1] > 0 && Math.Abs(b[1] - b[2]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)");
}
[Fact]

View file

@ -0,0 +1,183 @@
using AquaSynth.Dsl;
using AquaSynth.Faust;
using Xunit.Abstractions;
namespace AquaSynth.Dsl.Tests;
// Renders behind the tract-coarticulation findings of the PR 52 Soul pass: what the closing ramp into a stop sounds like, what the DSP
// does with a stop's 12 ms opening, and what the control smoothers and the word attack do at the start of a render. Each reads the
// rendered signal (or the DSP's own probe), not the emitted text.
[Collection("NativeFaust")]
public sealed class StopClosureAndControlRenderTests(ITestOutputHelper output)
{
private const int Rate = 48000;
private static AquaSynthCompiledPatch? Compile(string source, float seconds)
{
using var compiler = new AquaSynthPatchCompiler(new AquaSynthNativeOptions(SampleRate: Rate));
if (!compiler.TryCompileSource(new AquaSynthCompileIdentity("stop_closure", "stop_closure", ""), source, seconds, out var patch, out var error))
{
Assert.Fail($"compile failed: {error}");
}
return patch;
}
private static string Word(string ipa) => IpaGestureExperiment.BuildWordScript(
[(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140);
// Power above 4 kHz (100 Hz grid to 12 kHz) of a Hann-windowed stretch.
private static double HighPower(float[] x, double from, double to)
{
var a = (int)(from * Rate);
var n = (int)((to - from) * Rate);
var seg = Enumerable.Range(0, n).Select(i => x[a + i] * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
double sum = 0;
for (var hz = 4000; hz < 12000; hz += 100)
{
double re = 0, im = 0;
for (var i = 0; i < n; i++) { var w = 2 * Math.PI * hz * i / Rate; re += seg[i] * Math.Cos(w); im -= seg[i] * Math.Sin(w); }
sum += re * re + im * im;
}
return sum;
}
// The 30 ms boundary ramp into a stop moves the contact, constriction and lip surfaces through every aperture between open and shut. The
// burst noise is 4 * opening * (1 - opening) times a gain, and the gain was held through the closure, so each of those passes was a 15-20 ms
// hiss before the stop. The interval is the stop's start to its release, against the same length of the vowel before it, in the >4 kHz
// band, with the glottal source silent and the burst gain kept: the voiced stops' own voicing runs through the ramp and its harmonics
// fill that band (/d/ read -11.6 dB with the source on and -52.8 with it off; with the source on it swings between -13.5 and -9.3 dB
// with f0 alone, 139.5 to 145 Hz, as the harmonics fall on and off the 100 Hz grid), so with the source on the measure cannot tell the
// voicing from the hiss. Source off, the burst gain held through the closure read +5 dB (/p/) and +14 dB (/t/, /d/) on 3b62b9c; gated,
// every stop reads -52 dB. -30 is the bound between the two. /g/ is not here: it is not in TryIpaDescriptor, so it would render the default approximant and could not fail.
[FaustTheory]
[InlineData("p")]
[InlineData("b")]
[InlineData("t")]
[InlineData("d")]
[InlineData("k")]
public void ClosingIntoAStopMakesNoHighFrequencyNoise(string ipa)
{
var parsed = PatchScript.Parse(Word(ipa));
var voiceless = parsed with
{
ControlSplines = parsed.ControlSplines.Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal)
? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() }
: s).ToArray()
};
using var patch = Compile(FaustEmitter.Emit(voiceless, new FaustExportOptions("stop_closure", SampleRate: Rate)).Source, 0.5f);
var x = patch!.Render(0.5f);
const double from = 0.18;
var release = PatchScript.StopReleaseStartSeconds(0.095f);
var closing = HighPower(x, from, from + release);
var vowel = HighPower(x, from - release, from);
var db = 10 * Math.Log10(closing / vowel);
output.WriteLine($"/{ipa}/ >4 kHz energy of the closing interval {db:F1} dB re the vowel's before it, source silent");
Assert.True(db < -30, $"/{ipa}/ closes with {db:F1} dB re the vowel's own >4 kHz energy");
}
// The Faust lowering of a Bezier segment ignores the handle times and runs the curve on the linear fraction of the segment; the C# timeline
// solves the curve's time, so a handle longer than the segment folds it. The stop's 12 ms opening had 23.75 ms handles: the contact opening
// read 0.05 to 0.87 between 6 and 7 ms in C# and 0.46 to 0.57 in the DSP. This reads the opening through the DSP's own contact reservoir probe
// (stored pressure + (1 - opening) * resistance, both at their defaults once the other splines are dropped) and compares it with the timeline.
[FaustFact]
public void TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt()
{
var script = Word("p");
var parsed = PatchScript.Parse(script);
var contact = parsed.ControlSplines.Where(s => s.SurfacePath == "/vocal/contacts/0/opening").ToArray();
var alone = parsed with { ControlSplines = contact };
var timeline = new ControlSplineTimeline(contact);
var source = FaustEmitter.Emit(alone, new FaustExportOptions("stop_opening", SampleRate: Rate, DebugProbeUi: true)).Source;
using var patch = Compile(source, 0.5f);
using var stream = patch!.CreateStreamingPatch();
var reservoir = stream.ProbePaths.Single(p => p.EndsWith("/contact/contact/reservoir", StringComparison.Ordinal));
var release = 0.18f + PatchScript.StopReleaseStartSeconds(0.095f);
var outputs = new[] { new float[48] };
var dsp = new List<(float Time, float Reservoir)>();
for (var block = 0; block < 400; block++)
{
stream.ProcessBlock([], outputs, 48);
var t = (block + 1) * 48f / Rate;
if (t >= release - 0.004f && t <= release + 0.016f) dsp.Add((t, stream.ReadProbe(reservoir)));
}
// reservoir is linear in the opening; scale it by its own end points
var shut = dsp.First().Reservoir;
var open = dsp.Last().Reservoir;
var worst = 0f;
foreach (var (time, value) in dsp)
{
var dspOpening = (value - shut) / (open - shut);
var curve = timeline.ValueAt("/vocal/contacts/0/opening", time);
var shutValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.First().Time);
var openValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.Last().Time);
var timelineOpening = (curve - shutValue) / (openValue - shutValue);
worst = MathF.Max(worst, MathF.Abs(dspOpening - timelineOpening));
}
output.WriteLine($"worst difference between the DSP's opening and the timeline's over the release: {worst:F3}");
Assert.True(worst < 0.08f, $"the DSP's stop opening departs from the timeline by {worst:F3}");
}
// The same fact without the DSP: every handle of an authored spline is a third of its segment, so the C# curve is the curve the lowering runs.
[Fact]
public void EveryAuthoredSplineHandleIsAThirdOfItsSegment()
{
var splines = PatchScript.Parse(Word("p")).ControlSplines.Where(s => s.Points.Count > 1).ToArray();
Assert.NotEmpty(splines);
foreach (var spline in splines)
{
for (var i = 0; i + 1 < spline.Points.Count; i++)
{
var third = (spline.Points[i + 1].TimeSeconds - spline.Points[i].TimeSeconds) / 3;
Assert.Equal(third, spline.Points[i].OutTimeOffsetSeconds, 5);
Assert.Equal(third, spline.Points[i + 1].InTimeOffsetSeconds, 5);
}
}
}
// Replaces the emitted patch's process with one control through the emitted smoo_init, so the smoother under test is the one the exporter
// writes. Starts at its input at sample 0, and moves at si.smoo's 22.7 ms time constant when the control moves (5 ms gave 0.89 here).
[FaustFact]
public void ASmootherStartsAtItsInputAndMovesAtSiSmoosPace()
{
var emitted = FaustEmitter.Emit(PatchScript.Parse(Word("p")), new FaustExportOptions("smoo", SampleRate: Rate)).Source;
var lines = emitted.Split('\n').Select(l => l.StartsWith("process =", StringComparison.Ordinal) ? "process = hslider(\"/vocal/sources/0/pressure\", 0.7, 0, 1, 0.001) : smoo_init;" : l);
using var patch = Compile(string.Join('\n', lines), 0.2f);
using var stream = patch!.CreateStreamingPatch();
var first = new[] { new float[480] };
stream.ProcessBlock([], first, 480);
output.WriteLine($"sample 0 {first[0][0]:F4}, after 10 ms {first[0][479]:F4}");
Assert.Equal(0.7f, first[0][0], 4);
Assert.Equal(0.7f, first[0][479], 4);
var after = new[] { new float[240] };
stream.ProcessBlock([], after, 240, new Dictionary<string, float> { ["/vocal/sources/0/pressure"] = 1f });
// 5 ms into a step from 0.7 to 1 with a 22.7 ms time constant
var expected = 0.7 + 0.3 * (1 - Math.Pow(1 - 44.1 / Rate, 0.005 * Rate));
output.WriteLine($"5 ms after a step to 1: {after[0][239]:F4}, si.smoo would be {expected:F4}");
Assert.InRange(after[0][239], expected - 0.01, expected + 0.01);
}
// A word starts with a 12 ms attack, not a step: the control smoothers no longer fade it in, and a step onset would be a click.
[FaustFact]
public void AWordBeginsWithATwelveMillisecondAttack()
{
double FirstMillisecondsPeak(string script)
{
using var patch = Compile(FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("attack", SampleRate: Rate)).Source, 0.4f);
var x = patch!.Render(0.4f);
var steady = x.Skip((int)(0.05 * Rate)).Take((int)(0.1 * Rate)).Max(Math.Abs);
return x.Take((int)(0.002 * Rate)).Max(Math.Abs) / steady;
}
var script = IpaGestureExperiment.BuildWordScript([(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.3f), 1f)], 140);
Assert.Contains("attack=.012", script);
var attacked = FirstMillisecondsPeak(script);
var stepped = FirstMillisecondsPeak(script.Replace("attack=.012 ", "", StringComparison.Ordinal));
output.WriteLine($"peak of the first 2 ms re the steady peak: with the attack {attacked:F3}, without {stepped:F3}");
Assert.True(attacked < 0.25, $"the first 2 ms reach {attacked:F2} of the steady peak");
Assert.True(stepped > 2 * attacked, "without the attack the onset is a step");
}
}

View file

@ -24,6 +24,11 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
using (patch) return patch!.Render(seconds);
}
// The stop's release time and the start of its closure (every articulator at its shut value, after the boundary ramp into it), both from the
// one owner in PatchScript rather than as fractions of the gesture; the consonant is [from, to).
private static double ReleaseAt(double from, double to) => from + PatchScript.StopReleaseStartSeconds((float)(to - from));
private static double ClosureFrom(double from) => from + PatchScript.PhoneTransitionSeconds;
private static double Rms(float[] x, double from, double to)
{
var a = (int)(from * Rate);
@ -95,7 +100,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var withoutBurst = Render(script, 0.4f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() });
// Everything but the burst's noise is deterministic and shared, so the difference of the two renders is the burst alone.
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
const double releaseFrom = 0.095 * 0.65, vowelFrom = 0.095;
const double vowelFrom = 0.095;
var releaseFrom = ReleaseAt(0, vowelFrom);
var vowelPeak = Peak(withBurst, vowelFrom + 0.06, vowelFrom + 0.18);
var burstPeak = Peak(burst, releaseFrom, vowelFrom + 0.01);
var burstBeforeRelease = Peak(burst, 0, releaseFrom - 0.005);
@ -121,7 +127,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var withoutBurst = Render(script, 0.77f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() });
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
const double bFrom = 0.275, bTo = 0.37;
var atRelease = Peak(burst, bFrom + 0.095 * 0.6, bTo + 0.02);
var atRelease = Peak(burst, ReleaseAt(bFrom, bTo) - 0.003, bTo + 0.02);
var leading = Peak(burst, 0, bFrom);
var trailing = Peak(burst, bTo + 0.12, 0.77);
output.WriteLine($"burst peak {atRelease:F4} at the release, {leading:F4} before the plosive (n, a), {trailing:F4} after it (a, n, tail)");
@ -130,17 +136,64 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
Assert.True(trailing < atRelease / 10, "no burst noise in the vowel, the nasal and the tail after the plosive");
}
// The burst gain's window, read off the spline itself: it opens one ramp (0.15 of the gesture) before the plosive and has closed one ramp after it.
// The rendered test above survived three mutants of the window (ramp 0, ramp 1.0 of the gesture, end moved to 3 ramps), so the bound is pinned here.
// The burst gain's window, read off the spline itself: it exists only from the stop's release start, one ramp (0.15 of the release window)
// before it and one after the stop's end. A gain held through the closure made the closing ramp a hiss (ClosingIntoAStopMakesNoHighFrequencyNoise).
[Fact]
public void PlosiveBurstGainWindowIsOneRampEitherSide()
public void PlosiveBurstGainWindowIsOneRampEitherSideOfTheRelease()
{
IpaGestureExperimentTarget Phone(string label, string ipa, float seconds) => new($"w_{label}", ipa, "", seconds);
var script = IpaGestureExperiment.BuildWordScript(
[(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone("2_b", "b", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_n", "n", 0.095f), 1f)], 140);
var window = PatchScript.Parse(script).ControlSplines.Single(s => s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal) && s.Points.Any(p => p.Value > 0));
const double start = 0.275, duration = 0.095, ramp = 0.15 * duration;
Assert.Equal((start - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4));
const double start = 0.275, duration = 0.095;
var release = start + PatchScript.StopReleaseStartSeconds((float)duration);
var ramp = 0.15 * (start + duration - release);
Assert.Equal((release - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4));
}
// The contact opening, the tongue's constriction and the lips all leave their shut value at the instant the burst gain window opens: one
// owner, StopReleaseStartSeconds. Read off the splines, so a stale constant in one of them (the contact opening 20 ms early while the
// burst keeps the owner's time) fails here and not only through a closure level that happens to cross a bound.
[Theory]
[InlineData("b", "w_2_b")]
[InlineData("t", "w_2_t")]
[InlineData("k", "w_2_k")]
public void AStopsArticulatorsOpenWhereItsBurstGainWindowOpens(string ipa, string name)
{
IpaGestureExperimentTarget Phone(string label, string symbol, float seconds) => new($"w_{label}", symbol, "", seconds);
var script = IpaGestureExperiment.BuildWordScript(
[(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone($"2_{ipa}", ipa, 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f)], 140);
var splines = PatchScript.Parse(script).ControlSplines;
const double start = 0.275, duration = 0.095;
var release = start + PatchScript.StopReleaseStartSeconds((float)duration);
var gain = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal));
var ramp = 0.15 * (start + duration - release);
Assert.Equal(release - ramp, gain.Points.Min(p => (double)p.TimeSeconds), 1e-4);
var paths = ipa == "b"
? new[] { "/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter", "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" }
: ["/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter"];
foreach (var path in paths)
{
var spline = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath == path);
Assert.Equal(release, spline.Points.Where(p => p.TimeSeconds < start + duration - 1e-6).Max(p => (double)p.TimeSeconds), 1e-4);
}
}
// The nasal keeps its murmur's small lip opening where a stop shuts the lips (0.04 cm against 0): the shut-lip term is the stop's only.
// Extending it to nasals moves /m/'s first nasal formant by 30 Hz and its murmur level by 0.4 dB and no render test notices, so the
// value is read off the splines.
[Theory]
[InlineData("m", 0.04f)]
[InlineData("b", 0f)]
public void OnlyAStopShutsTheLipsCompletely(string ipa, float shut)
{
var script = IpaGestureExperiment.BuildWordScript(
[(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140);
foreach (var path in new[] { "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" })
{
var spline = PatchScript.Parse(script).ControlSplines.Single(s => s.Name.StartsWith("w_1_c", StringComparison.Ordinal) && s.SurfacePath == path);
Assert.Equal(shut, spline.Points.Min(p => p.Value), 1e-4);
}
}
// /ɑ C ɑ/ at the scaffold's own durations (vowel 0.18 s, consonant 0.095 s). Returns the render and the consonant's [start, end).
@ -241,7 +294,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
private static SynthPatch WithoutBurst(SynthPatch p) =>
p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() };
// The closure of a stop is the part before its contact opens (0.65 of the gesture), 0.2 of the gesture in from its start, without the burst.
// The closure of a stop is the part after the boundary ramp into it and before its release, without the burst (the ramp is the tract still
// shutting on the vowel's decaying voicing; its noise is measured by ClosingIntoAStopMakesNoHighFrequencyNoise).
// A tongue closure at the nasal's 0.03 cm aperture leaked: /t/ read -14.3 dB re the flanking vowels against /p/ -22.1 (the lips seal /p/);
// completely closed it reads -24.3.
[Fact]
@ -252,7 +306,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst);
if (x is null) return double.NaN;
var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2);
return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels);
return 20 * Math.Log10(Rms(x, ClosureFrom(from), ReleaseAt(from, to)) / vowels);
}
var p = Closure("p", "voiceless_bilabial_plosive");
@ -269,7 +323,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var (nb, _, _) = VowelConsonantVowel("t", descriptor: "voiceless_alveolar_plosive", edit: WithoutBurst);
if (x is null || nb is null) return;
var burst = x.Zip(nb, (a, b) => a - b).ToArray();
var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01);
var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01);
var vowel = Peak(x, to + 0.06, to + 0.17);
output.WriteLine($"/t/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}");
Assert.True(peak > 0.004, "audible");
@ -288,7 +342,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
if (withBurst is null || withoutBurst is null) return;
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
const double tFrom = 0.275, tTo = 0.37;
var atRelease = Peak(burst, tFrom + 0.095 * 0.6, tTo + 0.02);
var atRelease = Peak(burst, ReleaseAt(tFrom, tTo) - 0.003, tTo + 0.02);
var leading = Peak(burst, 0, tFrom);
var trailing = Peak(burst, tTo + 0.12, 0.77);
output.WriteLine($"/t/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");
@ -305,7 +359,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var (x, from, to) = VowelConsonantVowel("t");
var (nb, _, _) = VowelConsonantVowel("t", edit: WithoutBurst);
if (x is null || nb is null) return;
var start = (int)((from + 0.65 * (to - from)) * Rate);
var start = (int)(ReleaseAt(from, to) * Rate);
const int n = 960;
var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
double Power(double hz)
@ -324,39 +378,44 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
}
// /k/ over the /t/ tests: its own place (the velar distance in the place table), closure sealed by the tongue, burst under half the vowel.
// The spec asks /k/'s closure to be at or below /p/'s. It is not: with the voiceless source as /p/ and /t/ have it the closure reads
// -14.9 dB re the vowels (/p/ -20.4, /t/ -24.9). The tract itself seals: with the glottal source's flow scale held at 0 over the gesture the
// same closure reads -22.7 dB, under /p/; with the source on, the 150 Hz voicing is what reaches the output through a closure that leaves
// only a short rear cavity (the glottal pressure that the tract wall radiates is higher the smaller that cavity is). Silencing voicing in
// voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion is reported as unmet (fork), not met by a /k/ special case.
// The second assertion is a regression bound on the measured leak, not the criterion.
// The spec asks /k/'s closure to be at or below /p/'s. With the voiceless source as /p/ and /t/ have it, it is not, and was not on main
// either (-14.9 dB re the vowels, /p/ -20.4): the 150 Hz voicing reaches the output through a closure that leaves only a short rear cavity.
// Silencing voicing in voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion stays unmet (fork).
// The measure is the source-on closure in its settled part (25 ms after the 30 ms closing ramp has finished, to 5 ms before the release),
// because the source-off closure cannot see a leak: with the source off nothing drives the tract, and leaks of .02, .05 and .1 of the
// diameter scale read -21.1, -21.2 and -21.2 dB over .21 to .263 s against -21.0 sealed, while the source-on closure reads 1.8, 1.8 and
// 4.1 dB louder in the settled part. The closure
// right after the ramp is no use either: the seal completes at the end of the ramp with the vowel's tail still in the rear cavity (the
// source-off /k/ read -21.0 dB over .21 to .263 s with the ramp and -27.5 with the contact and the constriction stepped shut; main's
// 30 ms-later window read -25.7), and that decays within the first 25 ms.
// The test therefore proves its own measure first (a leak injected into the tongue's closed diameter must read 3 dB louder) and then bounds
// the sealed closure (-16.5 measured; a .005, .03 or .035 leak reads -15.1, a .02 leak -14.7).
[Fact]
public void AVelarStopClosureSealsAndItsLeakIsTheGlottalSource()
public void AVelarStopClosureSealsAndTheMeasureSeesALeak()
{
static SynthPatch WithoutSource(SynthPatch p) =>
static SynthPatch Leaking(SynthPatch p) =>
WithoutBurst(p) with
{
ControlSplines = WithoutBurst(p).ControlSplines
.Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal)
? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() }
.Select(s => s.Name == "w_1_c_constriction_diameter"
? s with { Points = s.Points.Select(q => q.Value == s.Points.Min(v => v.Value) ? q with { Value = 0.1f, OutValue = 0, InValue = 0 } : q).ToArray() }
: s).ToArray()
};
double Closure(string ipa, string descriptor, Func<SynthPatch, SynthPatch> edit)
double Closure(Func<SynthPatch, SynthPatch> edit)
{
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: edit);
var (x, from, to) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: edit);
if (x is null) return double.NaN;
var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2);
return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels);
return 20 * Math.Log10(Rms(x, ClosureFrom(from) + 0.025, ReleaseAt(from, to) - 0.005) / vowels);
}
var p = Closure("p", "voiceless_bilabial_plosive", WithoutBurst);
var k = Closure("k", "voiceless_velar_plosive", WithoutBurst);
var kSilent = Closure("k", "voiceless_velar_plosive", WithoutSource);
output.WriteLine($"closure re vowels: /p/ {p:F1} dB, /k/ {k:F1} dB, /k/ with the glottal source off {kSilent:F1} dB");
if (double.IsNaN(p)) return;
Assert.True(kSilent <= p, $"the /k/ closure seals: without the glottal source it is {kSilent:F1} dB, at or under /p/ ({p:F1} dB)");
Assert.True(k <= -12, $"the /k/ closure with the source is {k:F1} dB (regression bound; the criterion, at or under /p/ {p:F1} dB, is not met)");
var sealedClosure = Closure(WithoutBurst);
var leaking = Closure(Leaking);
output.WriteLine($"/k/ closure re vowels, source on, settled part: sealed {sealedClosure:F1} dB, with a 0.1 leak {leaking:F1} dB");
if (double.IsNaN(sealedClosure)) return;
Assert.True(leaking >= sealedClosure + 3, $"the measure sees a leak: sealed {sealedClosure:F1} dB, leaking {leaking:F1} dB");
Assert.True(sealedClosure <= -15.5, $"the /k/ closure seals: {sealedClosure:F1} dB re the vowels (measured -16.5; leaks of .005 to .1 read -15.1 to -12.4, the measure is not monotonic in the leak, so the bound sits under the weakest of them)");
}
[Fact]
@ -366,7 +425,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var (nb, _, _) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: WithoutBurst);
if (x is null || nb is null) return;
var burst = x.Zip(nb, (a, b) => a - b).ToArray();
var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01);
var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01);
var vowel = Peak(x, to + 0.06, to + 0.17);
output.WriteLine($"/k/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}");
Assert.True(peak > 0.004, "audible");
@ -384,7 +443,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
if (withBurst is null || withoutBurst is null) return;
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
const double kFrom = 0.275, kTo = 0.37;
var atRelease = Peak(burst, kFrom + 0.095 * 0.6, kTo + 0.02);
var atRelease = Peak(burst, ReleaseAt(kFrom, kTo) - 0.003, kTo + 0.02);
var leading = Peak(burst, 0, kFrom);
var trailing = Peak(burst, kTo + 0.12, 0.77);
output.WriteLine($"/k/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");
@ -402,7 +461,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor);
var (nb, _, _) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst);
if (x is null || nb is null) return (double.NaN, double.NaN);
var start = (int)((from + 0.65 * (to - from)) * Rate);
var start = (int)(ReleaseAt(from, to) * Rate);
const int n = 960;
var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray();
double Power(double hz)
@ -468,7 +527,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
if (withBurst is null || withoutBurst is null) return;
var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray();
const double dFrom = 0.275, dTo = 0.37;
var atRelease = Peak(burst, dFrom + 0.095 * 0.6, dTo + 0.02);
var atRelease = Peak(burst, ReleaseAt(dFrom, dTo) - 0.003, dTo + 0.02);
var leading = Peak(burst, 0, dFrom);
var trailing = Peak(burst, dTo + 0.12, 0.77);
output.WriteLine($"/d/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it");

View file

@ -148,10 +148,10 @@ public sealed class WeksaPhonesTests
Assert.Equal([WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds * WeksaWordPlan.CueFor(WeksaStress.Primary).Duration, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds],
plan.Phones.Select(p => p.Target.DurationSeconds));
var patch = PatchScript.Parse(IpaGestureExperiment.BuildWordScript(plan.Phones.ToArray(), 140));
var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value);
var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value);
Assert.Equal(1f, ih["/vocal/areas/0/area/vowel_ih"]);
Assert.Equal(0f, ih["/vocal/areas/0/area/vowel_i"]);
var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value);
var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value);
Assert.Equal(1f, oo["/vocal/areas/0/area/vowel_oo_short"]);
Assert.Equal(0f, oo["/vocal/areas/0/area/vowel_u"]);
}
@ -190,7 +190,7 @@ public sealed class WeksaPhonesTests
var none = patch(WeksaStress.None);
var primary = patch(WeksaStress.Primary);
float First(SynthPatch p, string prefix, string surface) =>
p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[0].Value;
p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[^1].Value;
// pitch_ratio surface range 0.5..2: normalized (1.06 - .5) / 1.5; unstressed holds 1 -> (1 - .5) / 1.5.
Assert.Equal((1.06f - .5f) / 1.5f, First(primary, "w_1_i_source_pitch", "/vocal/sources/0/pitch_ratio"), 4);

View file

@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));

View file

@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));

View file

@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));

View file

@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d));
bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;
seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);
smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));
smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };
smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);
seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));
seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));
seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));

View file

@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""What the formant numbers do not say. Measures a render for the faults Meta heard in tract-weak-spots-1 (2026-11-13) while every
formant matched its target; it does not listen, and a pass here is not a verdict (the listening page is).
ear_metrics.py FILE.wav [--f0 140] [--steady A B] [--json]
click_ratio largest pitch-period-window peak over the median window peak. A voiced word with no transient is near 1; the pre-fix
nibu (a phone boundary stepping an area) was 3.54 with its window at the /b/ closure (0.271 s); after, 1.76 (a vowel's own pulses).
step_db largest single-sample step, in dB over the median sample step of the voiced part. A cell area that jumps is one sample wide.
bandwidths LPC pole bandwidths (Hz) of the steady part, F1..F3 as (Hz, bandwidth). Natural vowels run about 40-100 Hz at F1, 50-120 at
F2, 80-200 at F3 (Hawks & Miller 1995, Fant 1972); a lossless tube rings narrower, which reads as reedy.
hnr_db harmonics-to-noise ratio of the steady part by autocorrelation at the pitch lag.
hf_db energy above 4 kHz re the total, dB.
"""
import sys, wave, json, argparse
import numpy as np
def wav(path):
with wave.open(path) as w:
return np.frombuffer(w.readframes(w.getnframes()), dtype="<i2").astype(float) / 32768.0, w.getframerate()
def click_ratio(x, sr, f0):
p = max(8, int(sr / f0))
peaks = np.array([np.abs(x[i:i + p]).max() for i in range(0, len(x) - p, p // 2)])
voiced = peaks[peaks > peaks.max() * 0.02]
if len(voiced) < 4:
return 0.0, 0.0
at = int(np.argmax(peaks)) * (p // 2) / sr
return float(peaks.max() / np.median(voiced)), at
def step_db(x):
d = np.abs(np.diff(x))
live = d[d > d.max() * 1e-3]
return float(20 * np.log10(d.max() / max(np.median(live), 1e-12))) if len(live) else 0.0
def autocorr(s, n=None):
"""Autocorrelation at lags 0..n-1 by FFT (np.correlate is quadratic)."""
n = n or len(s)
f = np.fft.rfft(s, 2 * len(s))
return np.fft.irfft(f * np.conj(f))[:n]
def lpc(s, o):
s = s * np.hamming(len(s))
r = autocorr(s, o + 1)
R = np.array([[r[abs(i - j)] for j in range(o)] for i in range(o)])
a = np.linalg.solve(R + 1e-9 * r[0] * np.eye(o), r[1:o + 1])
return np.concatenate([[1], -a])
def bandwidths(y, sr, order=14):
dec = max(1, int(round(sr / 10000)))
k = np.hanning(4 * dec + 1)
k /= k.sum()
z = np.convolve(y - y.mean(), k, "same")[::dec]
fs = sr / dec
z = np.append(z[0], z[1:] - 0.97 * z[:-1])
rt = np.roots(lpc(z, order))
rt = rt[np.imag(rt) > 0.001]
fq = np.angle(rt) * fs / (2 * np.pi)
bw = -np.log(np.abs(rt)) * fs / np.pi
# a formant is a narrow pole; broad poles (> 400 Hz) are the spectral tilt the all-pole fit spends order on
return [(int(f), int(b)) for f, b in sorted(zip(fq, bw)) if 150 < f < 4500 and b < 400][:3]
def hnr_db(y, sr, f0):
y = y - y.mean()
ac = autocorr(y)
ac /= ac[0]
lo, hi = int(sr / (f0 * 1.3)), int(sr / (f0 * 0.75))
m = float(ac[lo:hi].max())
return 10 * np.log10(max(m, 1e-6) / max(1 - m, 1e-6))
def measure(path, f0=140.0, steady=None):
x, sr = wav(path)
a, b = steady if steady else (len(x) / sr * 0.35, len(x) / sr * 0.85)
y = x[int(a * sr):int(b * sr)]
sp = np.abs(np.fft.rfft((y - y.mean()) * np.hanning(len(y))))
fr = np.fft.rfftfreq(len(y), 1 / sr)
cr, at = click_ratio(x, sr, f0)
return {
"file": path.split("/")[-1], "click_ratio": round(cr, 2), "click_at_s": round(at, 4), "step_db": round(step_db(x), 1),
"bandwidths": bandwidths(y, sr), "hnr_db": round(hnr_db(y, sr, f0), 1),
"hf_db": round(float(10 * np.log10(max((sp[fr > 4000] ** 2).sum(), 1e-20) / (sp ** 2).sum())), 1),
}
if __name__ == "__main__":
ap = argparse.ArgumentParser()
ap.add_argument("files", nargs="+")
ap.add_argument("--f0", type=float, default=140.0)
ap.add_argument("--steady", nargs=2, type=float)
ap.add_argument("--json", action="store_true")
a = ap.parse_args()
rows = [measure(f, a.f0, a.steady) for f in a.files]
if a.json:
print(json.dumps(rows, indent=1))
else:
for r in rows:
print(f"{r['file']:20s} click {r['click_ratio']:5.2f} at {r['click_at_s']:.3f}s step {r['step_db']:5.1f} dB hnr {r['hnr_db']:5.1f} hf {r['hf_db']:6.1f} F(Hz,bw) {r['bandwidths']}")

View file

@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""A machine ear for the tract renders: what phone string a speech recogniser reads out of a clip. A PROXY for 'what was heard', never a
verdict: it is trained on natural speech and reads the vowels of this tract poorly (every isolated vowel came back as /i/), so use it
for the events it separates well (a burst read as /v/ against /b/, a nasal present or missing, a consonant that is only a click).
It reproduced three of Meta's four first listening reports on tract-weak-spots-1 before any fix: ba with burst -> 'v ɑ' (she heard
'fah then bah'), ana -> 'v ɑ m ɑ' (a basketball buzzer, no articulation), nibu -> 'i b u' (both consonants clicks, the /n/ missing).
phone_probe.py CLIP.wav...
Needs torch, transformers, scipy and the HuggingFace model facebook/wav2vec2-lv-60-espeak-cv-ft (about 1.2 GB, downloaded once into
HF_HOME). The CTC decode reads vocab.json directly, so the phonemizer library is not needed.
"""
import sys, wave, json
import numpy as np
import torch
from scipy.signal import resample_poly
from transformers import Wav2Vec2ForCTC
from huggingface_hub import hf_hub_download
MODEL = "facebook/wav2vec2-lv-60-espeak-cv-ft"
def phones(model, vocab, path):
with wave.open(path) as w:
sr = w.getframerate()
x = np.frombuffer(w.readframes(w.getnframes()), dtype="<i2").astype(np.float32) / 32768
x = resample_poly(x, 16000, sr).astype(np.float32)
x = np.concatenate([np.zeros(4000, np.float32), x, np.zeros(4000, np.float32)]) # silence either side, as speech has
x = (x - x.mean()) / (x.std() + 1e-7)
with torch.no_grad():
logits = model(torch.tensor(x)[None]).logits[0]
ids = logits.argmax(-1).tolist()
out = [vocab[i] for k, i in enumerate(ids) if (k == 0 or i != ids[k - 1]) and vocab[i] != "<pad>"]
return " ".join(out)
if __name__ == "__main__":
model = Wav2Vec2ForCTC.from_pretrained(MODEL).eval()
vocab = {v: k for k, v in json.load(open(hf_hub_download(MODEL, "vocab.json"))).items()}
for p in sys.argv[1:]:
print(f"{p.split('/')[-1]:24s} | {phones(model, vocab, p)}")

View file

@ -59,6 +59,10 @@ def stress(root):
out["nibu_i_150_250ms_db"] = db(rms(xb, sr, .15, .25), rms(xa, sr, .15, .25))
return out
def release_at(c0, c1):
"""Where a stop's articulators begin to open: StopReleaseStartSeconds in PatchScript.cs (the end less min(12 ms, 0.3 of the gesture))."""
return c1 - min(0.012, 0.3 * (c1 - c0))
def burst(root, name, c0, c1, sub="out"):
"""c0..c1: the b gesture's start and end in seconds; v1: the vowel's steady part after it."""
x, sr = load(root, sub, name); nb, _ = load(root, "outnb", name)
@ -70,11 +74,12 @@ def burst(root, name, c0, c1, sub="out"):
below = [k for k in range(int(tpk * 500), len(env)) if env[k] < pk * .1]
decay_ms = round((below[0] / 500 - tpk) * 1000, 0) if below else None
last = x[int((c1 + .06) * sr):]
return {"closure_peak": round(peak(x, sr, c0, c0 + .6 * (c1 - c0)), 4), "release_peak_full": round(peak(x, sr, c0 + .6 * (c1 - c0), c1 + .005), 4),
rel = release_at(c0, c1)
return {"closure_peak": round(peak(x, sr, c0, rel), 4), "release_peak_full": round(peak(x, sr, rel, c1 + .005), 4),
"burst_peak": round(pk, 4), "burst_peak_ms": round(tpk * 1000), "burst_to_tenth_ms": decay_ms, "vowel_peak": round(vp, 4),
"burst_re_vowel_db": db(pk, vp), "release_peak_re_vowel_db": db(peak(x, sr, c0 + .6 * (c1 - c0), c1 + .005), vp),
"burst_re_vowel_db": db(pk, vp), "release_peak_re_vowel_db": db(peak(x, sr, rel, c1 + .005), vp),
"burst_rms_in_30ms_after_peak_re_vowel_rms_db": db(rms(d, sr, tpk, tpk + .03), rms(x, sr, c1 + .06, c1 + .17)),
"burst_peak_from_120ms_after_release_to_end": round(peak(d, sr, c1 + .12, n / sr), 4), "burst_diff_samples_nonzero_before_closure_end": int(np.count_nonzero(np.abs(d[:int((c0 + .6 * (c1 - c0)) * sr)]) > 1e-9)),
"burst_peak_from_120ms_after_release_to_end": round(peak(d, sr, c1 + .12, n / sr), 4), "burst_diff_samples_nonzero_before_closure_end": int(np.count_nonzero(np.abs(d[:int(rel * sr)]) > 1e-9)),
"file_end_rms_last_100ms": round(rms(x, sr, len(x) / sr - .1, len(x) / sr), 5), "vowel_rms": round(rms(x, sr, c1 + .06, c1 + .17), 4)}
BURST_BANDS = ((0, 1000), (1000, 2000), (2000, 3500), (3500, 7000), (7000, 24000))