diff --git a/docs/listening/tract-weak-spots-1-analysis-20261113.md b/docs/listening/tract-weak-spots-1-analysis-20261113.md new file mode 100644 index 0000000..cf583c2 --- /dev/null +++ b/docs/listening/tract-weak-spots-1-analysis-20261113.md @@ -0,0 +1,76 @@ +# tract-weak-spots-1: why the predictions failed, and what was changed + +The verdict is `tract-weak-spots-1-meta-20261113.md`. Every falsifiable prediction in the kit failed although the formant numbers +behind them were right. The kit measured the steady part of each vowel (35-85 % of the clip) and the peak of each burst; the faults +Meta heard sit in the first 100 ms of every clip, at the phone boundaries, and in how long the burst lasts. Nothing in the old +measurement set looked there. Everything below was measured on the clips Meta heard (`weak-kit-out/clips`, WordRender at 140 Hz) before +any change, and again after. + +## Causes found (measured, not guessed) + +| # | what she heard | cause | evidence (before) | +|---|---|---|---| +| 1 | nibu: "both consonants are just clicks"; leak-before: "reversed percussive hit" | The merged control curve of a surface holds the previous phone's value to the boundary and steps to the next phone's first value there (two points at one time). A step in a ladder cell's area is a one-sample transient. | Pitch-period window peaks: 0.399 at 95 ms and 0.487 at 275 ms, against vowel pulses of ~0.1-0.2; both instants are phone boundaries of the script (`start=0.095`, `0.275`). The whole-render peak the kit reported (0.487) was this click. Click ratio 3.54 (`ear_metrics.py`). The old report called 0.487 vs 0.399 a "leak" difference. | +| 2 | every clip starts as a fade-in; the "kazoo" with "no audible closure" (ba, no burst); the /ɪ/ not like "bit" | `si.smoo` starts every control slider at 0. A word began with the tract shut, the vowel weights at 0 and the source off, and climbed for ~100 ms. The 140 ms "closure" of ba was largely this climb. | Isolated /ɪ/: level -66 dB at 0 ms, -45 at 40 ms, -30 at 100 ms, -28 from 140 ms; LPC bandwidth of F1 367 Hz at 0 ms, 232 at 60 ms, 103 at 100 ms, 90 from 140 ms. 100 of the clip's 309 ms were the climb; the steady-state window (35-85 %) started after it. | +| 3 | ba with burst: "fah then bah" | The stop opened over 0.65 to 1.0 of its gesture: 33 ms of a 95 ms /b/. A burst of that length is a fricative. | Burst-notes: ba burst 0.0707-0.1039 s (33 ms). A recogniser reads the clip as `v ɑ` (with the burst), `b ɑ` without. | +| 4 | nasals: "basketball buzzer, no audible articulation", /n/ and /m/ alike | The murmur had no entry and exit transitions (the step of cause 1) and began after the climb of cause 2; the machine ear read /n/ as /m/ (`v ɑ m ɑ`). | recogniser, before: ana -> `v ɑ m ɑ`. | +| 5 | bun with burst: "a truck releasing its pneumatics" | cause 3 (long burst), same mechanism. | burst 0.0721-0.0998 s (28 ms). | + +## What was changed (PR "phone transitions", this branch) + +1. `PatchScript.WithPhoneTransition`: every authored gesture spline starts, when an earlier gesture left the surface at another value, + with a 30 ms ramp from the previous value (at most 0.4 of the gesture, and before its second point). The first phone of a word and + surfaces no earlier phone touched are unchanged. Test: `PhoneTransitionTests.NoSurfaceStepsAtAPhoneBoundary` (worst step in 1 ms + around each boundary of nibu: below 0.03; before, 0.77: the vowel's 1 against the nasal's released 0.23). +2. A stop holds all its articulators shut until 12 ms before its end and opens in those 12 ms (`StopReleaseSeconds`, a design value); + it used to open over 0.65 to 1 of its gesture. Test: `AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture`. +3. `smoo_init` replaces `si.smoo` on every control-surface and spline-point slider of the emitted DSP: the smoother starts at its input + and keeps si.smoo's pole (a 22.7 ms time constant), so live control moves as fast as before. Patch parameters keep `si.smoo`. The + source's own 10 ms pitch-ratio smoother starts at its input too. Tests: `ControlSmoothersStartAtTheirInputNotAtZero` (text) and + `ASmootherStartsAtItsInputAndMovesAtSiSmoosPace` (the emitted smoother, rendered). +4. The word scaffold's vocal envelope gets a 12 ms attack (it was 0, so the now-instant start was itself a click: first-sample peak 0.22). +5. The ramp into a stop (item 1) sweeps the contact through every aperture between open and shut, and the exporter's burst noise is + `4 * opening * (1 - opening)` times a gain that was held through the whole gesture: each closing ramp was a 15-20 ms hiss before the + stop (>4 kHz share of the 6 ms frames at 188 ms of /ata/: -1 dB, against -57 dB with the closure stepped). The burst gain now exists + only from the stop's release start (`PatchScript.StopReleaseStartSeconds`, the one owner of the release time), one ramp either side. + A stop's lips are shut (aperture 0), not at 0.04: the word-initial /b/ voice bar read -19.1 dB re the vowel after it and /p/ -30.1 + against the design -34 to -26 and -60 to -38; shut they read -27.7 and -38.8. Spline handle times follow their segment (a third of + it), so the C# timeline and the DSP agree on the 12 ms opening (the lowering ignores handle times). Tests: + `ClosingIntoAStopMakesNoHighFrequencyNoise`, `TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt`, `PlosiveClosureCarriesTheVoiceBarOnlyWhenVoiced`. +6. `tools/formant-check/ear_metrics.py` (click ratio, largest sample step, LPC bandwidths, HNR, energy above 4 kHz) and + `tools/formant-check/phone_probe.py` (a speech recogniser as a machine ear) so these faults are measured, not heard first by Meta. + +### After (same words, same 140 Hz, `fix-out-c`) + +| word | before | after | +|---|---|---| +| nibu_flat | click ratio 3.54, peaks 0.399 / 0.487; recogniser `i b u` | peaks at the boundaries 0.077 / 0.089; click ratio 1.79 (the vowel's own pulses); median sample step in the /b/ closure 0.0007 (0.0005 before; 0.0161 with the ramp hiss); recogniser `n i l t uː` as read before the closure-noise and shut-lips fixes (not re-run: no torch on this host; the /n/ is back, the /b/ reads as `l t`: see below) | +| ba (burst) | recogniser `v ɑ`; burst 33 ms | `b ɑ`; opening 12 ms | +| ba (no burst) | `b ɑ` | `m ɑ` | +| ana / ama | `v ɑ m ɑ` / `v ɑ m ɑ` | `ɑ n ɑ` / `ɑ m ɑ` | +| isolated /i/ | `h iː`; level climb 100 ms | `i`; level within 3 dB of the steady value by 20 ms | +| isolated /ɪ/ | `h i` | `ʔ i` | + +## Not fixed, and why (open until she listens) + +- **Voice quality: "kazoo with a two-pack-a-day smoking habit" (the two clips she kept), "reedy, like Pink Trombone" (/ʊ/), "basketball + buzzer" (nasals).** Her comments on the two keeps say the vowel is right and the voice is wrong. Candidates, none tested by ear: + (a) the glottal source is a band-limited saw through a one-pole lowpass with no closed phase (`FaustExport.PrimitiveSourceExpression`), + where a Rosenberg/LF pulse has the open and closed phases real fold flow has; (b) harmonics-to-noise ratio of the steady part is 13-15 dB + (healthy voices ~20+), from the source's `0.08 * noise` term and the saw's aliasing; (c) LPC bandwidths of the steady vowel are 90 Hz (F1), + 130 (F2), 100 (F3) for /ɪ/: plausible, so the wall-loss hypothesis is weak (the F2 of /ɑ/, 29 Hz, and the F1 of /i/, 14-24 Hz, are LPC + artefacts of a pole sitting on a harmonic). Settling this needs a listening kit that varies one of them at a time; the next kit does. +- **/ɪ/ as the "eu" of cordon bleu.** The recogniser cannot judge this tract's vowel quality (every isolated vowel came back /i/). The + confirmed fault in that clip is cause 2: a third of it was a muffled climb through bandwidths of 300 Hz. Whether the steady /ɪ/ then + reads as "bit" is for her ear; lax vowels in isolation and without vowel-inherent spectral change are known to be misheard + (Hillenbrand et al. 1995, Strange et al. 1983), so the next kit also carries /ɪ/ in a word. +- **The medial /b/ of nibu reads as `l t` to the recogniser after the fix, where it read `b` with the click and the long burst.** The closure + voice bar sat 10 dB under the vowel (a murmur-like level; a stop's runs 20+ dB under; 16-19 dB with the lips shut), and a labial burst is injected at the glottal + end in a word with no other plosive (`FaustExport` burst injection), so it is filtered by the whole tract like aspiration. Both are + next, with the listening to decide. + +## What the old measurements missed, as a rule + +A steady-state window measures a steady state. Faults of onset, boundary and duration need metrics that look at onsets, boundaries +and durations: click ratio and step size at each phone boundary, time-to-full-level, burst length, bandwidths, HNR. These now exist +(`ear_metrics.py`) and the click and onset ones are pinned by tests. diff --git a/docs/listening/tract-weak-spots-1-meta-20261113.json b/docs/listening/tract-weak-spots-1-meta-20261113.json new file mode 100644 index 0000000..9504d9f --- /dev/null +++ b/docs/listening/tract-weak-spots-1-meta-20261113.json @@ -0,0 +1,231 @@ +{ + "schema": "aqua.listen.verdict.v1", + "kit": { + "id": "tract-weak-spots-1", + "title": "Articulatory vocal tract: stress, release, nasals, lax vowels" + }, + "exportedAt": "2026-11-13T11:27:36.539Z", + "seed": 1326320895, + "consent": true, + "listener": "Meta", + "device": null, + "notes": "", + "summary": { + "total": 14, + "keep": 2, + "fix": 12, + "drop": 0, + "unanswered": 0 + }, + "clips": [ + { + "position": 1, + "id": "lax-ih", + "title": "isolated /ɪ/ 180 ms, 140 Hz. Measured: Burg F1/F2 375/2024 Hz; tube chain 346/1987 (+8.5/+1.8 %); Hillenbrand 1995 male 429/2034 (-12.5/-0.5 %); Peterson & Barney 392/1993 (-4.3/+1.5 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'bit' (or a slightly high, 'ee'-leaning version of it). FALSIFIED if heard as 'ee' of 'beet', as 'eh' of 'bet', or as a different phone.", + "group": "lax", + "feel": "the short vowel in 'bit'", + "file": "v_ih.wav", + "pair": null, + "slot": null, + "verdict": "fix", + "reason": "It's more like the eu in cordon bleu than the i in bit ", + "landed": 2, + "device": null + }, + { + "position": 2, + "id": "burst-ba-on", + "title": "ba with the plosive burst (gain 0.06, a design value). Measured: burst alone peaks 0.0142 at 85 ms = -24.0 dB re vowel peak, falls to a tenth in 17 ms, burst rms in the 30 ms after its peak -31.0 dB re vowel rms. PREDICTION: a slightly crisper, more 'b'-like onset than B/off; the burst is quiet and short so the difference is small, 'a little pop' at most. FALSIFIED if it sounds like a hiss, a click, or a separate noise after the vowel started, or if no difference is heard.", + "group": "burst", + "feel": "the syllable 'bah' with a clean, crisp 'b' at the start", + "file": "ba.wav", + "pair": "burst-ba-off", + "slot": "A", + "verdict": "fix", + "reason": "More like fah then bah but you can hear something there", + "landed": 2, + "device": null + }, + { + "position": 3, + "id": "burst-ba-off", + "title": "ba with the release_flow splines dropped (WordRender --no-burst). Measured: closure peak 0.0078 (voice bar), release (voicing breaking through)peak 0.1087 = -6.3 dB re vowel peak 0.2247, no burst noise. PREDICTION: a soft, voiced 'b' with the vowel starting smoothly; may sound slightly 'w'-like or lack the pop. FALSIFIED if listeners cannot hear any difference from the burst twin (then burst gain 0.06 is inaudible).", + "group": "burst", + "feel": "the syllable 'bah' with a soft start", + "file": "ba_noburst.wav", + "pair": "burst-ba-on", + "slot": "B", + "verdict": "fix", + "reason": "There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo", + "landed": 2, + "device": null + }, + { + "position": 4, + "id": "stress-nibu-flat", + "title": "nibu, no stress marks. Measured: /i/ 150-250 ms rms vs the stressed render +0.21 dB; unstressed vowel 180 ms, f0 139.9 Hz. PREDICTION: reads asan even two-syllable word with no strong first syllable. FALSIFIED if listeners hear it as stressed on a syllable or cannot tell it from the stressed twin.", + "group": "stress", + "feel": "a made-up two-syllable word, 'nee-boo', both syllables even", + "file": "nibu_flat.wav", + "pair": "stress-nibu", + "slot": "A", + "verdict": "fix", + "reason": "These both sound like 'ee'oo, with both consonants you're going for being just clicks", + "landed": 2, + "device": null + }, + { + "position": 5, + "id": "stress-nibu", + "title": "nibu with primary stress on /i/ (NI-bu). Measured: /i/ 247 ms vs 180 ms (+75 ms; design +68), f0 148.1 vs 139.9 Hz (x1.059), amplitude factor 1.30; level of /i/ in the 150-250 ms window only +0.21 dB (five-vowel mean +1.93 dB, per vowel +1.2 to +3.0). PREDICTION: first syllable heard as the prominent one, mostly through length and pitch, little through loudness; B should be picked as 'stressed on the first syllable'. FALSIFIED if it sounds no more prominent than the flat twin, if it sounds like a question or a pitch accent rather than stress, or if the level is what is heard (then the f0-scatter question is wrong).", + "group": "stress", + "feel": "the same made-up word, with the first syllable stronger: 'NEE-boo'", + "file": "nibu.wav", + "pair": "stress-nibu-flat", + "slot": "B", + "verdict": "fix", + "reason": "", + "landed": 2, + "device": null + }, + { + "position": 6, + "id": "stress-i-flat", + "title": "isolated /i/ 180 ms at 140 Hz, no stress. Measured: level 100-170 ms is the reference. PREDICTION: plain short vowel. FALSIFIED if it already sounds emphasised.", + "group": "stress", + "feel": "a single short 'ee' vowel, plain", + "file": "v_i.wav", + "pair": "stress-i", + "slot": "A", + "verdict": "keep", + "reason": "Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee", + "landed": 4, + "device": null + }, + { + "position": 7, + "id": "stress-i", + "title": "isolated /i/ with primary stress: 247 ms, f0 x1.06, amplitude 1.30. Measured against the flat /i/: +1.22 dB total (f0 rise alone -0.80 dB, the amplitude and the rest +2.02 dB). PREDICTION: heard as longer and a little higher; the 1.2 dB louder is under what a listener can judge, so 'longer and higher' is the answer, 'louder' is not. FALSIFIED if it sounds the same length and pitch, or clearly louder.", + "group": "stress", + "feel": "the same 'ee' vowel, held longer and said with emphasis", + "file": "v_i_s.wav", + "pair": "stress-i-flat", + "slot": "B", + "verdict": "keep", + "reason": "", + "landed": 4, + "device": null + }, + { + "position": 8, + "id": "lax-oo-short", + "title": "isolated /ʊ/ 180 ms, 140 Hz. Measured: Burg F1/F2 486/894 Hz; tube chain 478/850 (+1.7/+5.2 %); Hillenbrand 469/1123 (+3.7/-20.4 %); Peterson &Barney 437/1023 (+11.3/-12.6 %). Present in the DSL table and renderable. PREDICTION: heard as the vowel of 'book', probably darker or rounder than a natural one (F2 20 % under the population mean, the back-vowel gap). FALSIFIED if heard as 'oo' of 'boot', 'o' of 'boat' or 'uh' of 'but'.", + "group": "lax", + "feel": "the short vowel in 'book'", + "file": "v_oo_short.wav", + "pair": null, + "slot": null, + "verdict": "fix", + "reason": "It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared", + "landed": 3, + "device": null + }, + { + "position": 9, + "id": "leak-nibu-flat-after", + "title": "nibu (flat) after the fix: release_flow is zero outside its own plosive. Same audio as stress-nibu-flat, kept as its own clip so the before/after pair is one-to-one. Measured: leak 0.0 from 275 ms on, burst of the b itself unchanged (bin 0.0226 at 81 ms before and after). PREDICTION: clean hum into thevowel, no noisy onset; preferred over 'before' if the leak is audible. FALSIFIED if 'before' is preferred or no difference is heard.", + "group": "release-leak", + "feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'", + "file": "nibu_flat_after.wav", + "pair": "leak-nibu-flat-before", + "slot": "A", + "verdict": "fix", + "reason": "Just slightly less reverse attack on this one", + "landed": 2, + "device": null + }, + { + "position": 10, + "id": "leak-nibu-flat-before", + "title": "nibu (flat) rendered at main c0e6b98, before the release_flow fix. Measured: burst-gain leak vs the fixed render, peak 0.0688 and rms 0.0067 inthe leading /n/ (0-95 ms; the /n/ itself is rms 0.0058, so the hiss is louder than the murmur), peak 0.0504 over /i/ onset 95-150 ms, 0.0019 after; whole-render peak 0.487 vs 0.399. PREDICTION: an audible hiss or click as the /n/ opens into the vowel, the first sound of the word being a noisy 'sn'-like or 'tsh' onset instead of a clean nasal hum. FALSIFIED if the two are indistinguishable (then the leak is inaudible and the fix is cosmetic).", + "group": "release-leak", + "feel": "a made-up word that should start with a clean, smooth hum into the vowel: 'nee-boo'", + "file": "nibu_flat_before.wav", + "pair": "leak-nibu-flat-after", + "slot": "B", + "verdict": "fix", + "reason": "'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant", + "landed": 2, + "device": null + }, + { + "position": 11, + "id": "nasal-n", + "title": "/ɑnɑ/. Measured: murmur -9.64 dB re the flanking vowels (2.17 dB above /m/; the gap is 2.01/2.17/2.05/1.61 dB at f0 110/140/170/200 Hz, so not harmonic sampling), Burg N1 332 Hz, bands 38.4/28.2/16.8/0.9 dB. PREDICTION: heard as 'ah-nah', a tongue-tip hum, a shade stronger than the /m/ one; the 2 dB isnot expected to be noticed. FALSIFIED if heard as 'ama' or 'anga', if it sounds much louder than the /m/ clip, or if there is hiss.", + "group": "nasal", + "feel": "'ah-nah': the tongue touches behind the teeth for a soft hum between two open vowels", + "file": "ana.wav", + "pair": null, + "slot": null, + "verdict": "fix", + "reason": "It just sounds like a basket ball buzzer, no audible articulation", + "landed": 2, + "device": null + }, + { + "position": 12, + "id": "burst-bun-off", + "title": "bun, /bʊn/, no burst. Measured: closure peak 0.0043, release peak 0.1074 = -5.9 dB re vowel peak 0.2126. PREDICTION: soft 'b' then a short backvowel then a nasal; the final /n/ is a clean hum (no leak since the fix). FALSIFIED as for burst-ba-off.", + "group": "burst", + "feel": "a short word like 'bunn' or 'boon' said quickly, soft start", + "file": "bun_noburst.wav", + "pair": "burst-bun-on", + "slot": "A", + "verdict": "fix", + "reason": "Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there", + "landed": 2, + "device": null + }, + { + "position": 13, + "id": "burst-bun-on", + "title": "bun, /bʊn/, with burst. Measured: burst peak 0.0130 at 81 ms = -24.3 dB re vowel peak, tenth in 15 ms; tail leak after the fix 0.0 (was 0.0126 at the end before). PREDICTION: slightly crisper onset than off; final /n/ clean. FALSIFIED as for burst-ba-on.", + "group": "burst", + "feel": "a short word like 'bunn' or 'boon' said quickly, crisp 'b' start", + "file": "bun.wav", + "pair": "burst-bun-off", + "slot": "B", + "verdict": "fix", + "reason": "The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human", + "landed": 2, + "device": null + }, + { + "position": 14, + "id": "nasal-m", + "title": "/ɑmɑ/. Measured: murmur -11.81 dB re the flanking vowels (design band -16..-8), Burg N1 295 Hz, band levels 0-400/400-800/800-1600/1600-3200 Hzof the murmur 36.3/23.6/21.3/2.7 dB. PREDICTION: heard as 'ah-mah', a lip-closed hum between the vowels, clearly softer than the vowels. FALSIFIED if heard as 'abah' or 'awah', if the hum is as loud as the vowels, or if there is a gap or click.", + "group": "nasal", + "feel": "'ah-mah': lips close for a soft hum between two open vowels", + "file": "ama.wav", + "pair": null, + "slot": null, + "verdict": "fix", + "reason": "That m is almost coming through, still more of a basketball buzzer", + "landed": 2, + "device": null + } + ], + "pairs": [ + { + "a": "burst-ba-on", + "b": "burst-ba-off", + "preferred": "burst-ba-on" + }, + { + "a": "stress-nibu-flat", + "b": "stress-nibu", + "preferred": "no-difference" + } + ], + "_truncated": "the mail reader cut the body at 12000 characters inside the pairs list; pairs after the third are not recorded" +} \ No newline at end of file diff --git a/docs/listening/tract-weak-spots-1-meta-20261113.md b/docs/listening/tract-weak-spots-1-meta-20261113.md new file mode 100644 index 0000000..5f063cc --- /dev/null +++ b/docs/listening/tract-weak-spots-1-meta-20261113.md @@ -0,0 +1,32 @@ +# Listening verdict: tract-weak-spots-1 (page tract-listen-1b6d180) + +Listener: Meta (operator), 2026-11-13. Exported 11:27Z by the page (`aqua.listen.verdict.v1`, seed 1326320895), pasted in +#aquasynth (truncated after clip 5) and emailed to aqua@minds.gamecult.org at 11:30Z; the data here is the email, +parsed whole by `mind-mail read` except the tail: the mail reader cut the body at 12000 characters inside `pairs`, so +only the first two pair preferences survive. Raw data: `tract-weak-spots-1-meta-20261113.json`. Totals: 2 keep, 12 fix, 0 drop. +`position` is the page's random order for this listener; ids are the kit's. + +| # | clip | verdict | what she heard | +|---|---|---|---| +| 1 | `lax-ih` (`v_ih.wav`) | fix | It's more like the eu in cordon bleu than the i in bit | +| 2 | `burst-ba-on` (`ba.wav`) | fix | More like fah then bah but you can hear something there | +| 3 | `burst-ba-off` (`ba_noburst.wav`) | fix | There is no audible constriction. While the vowel is recognizable, it still sounds like a kazoo | +| 4 | `stress-nibu-flat` (`nibu_flat.wav`) | fix | These both sound like 'ee'oo, with both consonants you're going for being just clicks | +| 5 | `stress-nibu` (`nibu.wav`) | fix | (no comment) | +| 6 | `stress-i-flat` (`v_i.wav`) | keep | Still spoken by a kazoo with a two pack a day smoking habit, but it's an ee | +| 7 | `stress-i` (`v_i_s.wav`) | keep | (no comment) | +| 8 | `lax-oo-short` (`v_oo_short.wav`) | fix | It's distinguishable, but it strongly has that reedy quality which I associate with pink trombone and that new one I shared | +| 9 | `leak-nibu-flat-after` (`nibu_flat_after.wav`) | fix | Just slightly less reverse attack on this one | +| 10 | `leak-nibu-flat-before` (`nibu_flat_before.wav`) | fix | 'Ee'oo with some odd clicks that sound more like a reversed percussive hit than a consonant | +| 11 | `nasal-n` (`ana.wav`) | fix | It just sounds like a basket ball buzzer, no audible articulation | +| 12 | `burst-bun-off` (`bun_noburst.wav`) | fix | Oou'mou is what this sounds closest to, with the apostrophe representing a clicking sound that's in there | +| 13 | `burst-bun-on` (`bun.wav`) | fix | The main difference seems to be a sort of air release added to the beginning which resembles more a truck releasing its pneumatics than a human | +| 14 | `nasal-m` (`ama.wav`) | fix | That m is almost coming through, still more of a basketball buzzer | + +Pairs: `burst-ba-on` preferred over `burst-ba-off`; `stress-nibu-flat` vs `stress-nibu`: no difference. +(`leak-nibu-flat-before`/`after` and the bun pair carry no recorded preference; she heard "just slightly less reverse +attack" on the fixed one, the pneumatic-truck "air release" on the bun burst.) + +Every prediction that was falsifiable failed, though the formant numbers behind them were measured correctly. The +two keeps are the isolated /i/ pair, and even those were "a kazoo with a two pack a day smoking habit". Analysis and +the fixes it led to: `tract-weak-spots-1-analysis-20261113.md`. diff --git a/src/AquaSynth.Core/FaustExport.cs b/src/AquaSynth.Core/FaustExport.cs index e006102..51ae72a 100644 --- a/src/AquaSynth.Core/FaustExport.cs +++ b/src/AquaSynth.Core/FaustExport.cs @@ -45,6 +45,15 @@ public static partial class FaustEmitter source.AppendLine("bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d;"); source.AppendLine("seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e);"); source.AppendLine("smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x));"); + // si.smoo starts every control at 0 and climbs to it: a word began with the tract shut, the vowel weights at 0 and the source off, + // and the first ~100 ms of every render were that climb (the level rose 40 dB over the first 100 ms of an isolated vowel, and + // the bandwidths fell from ~300 Hz to 90). smoo_from starts at its input and smooths from there. smoo_init is si.smoo's own + // pole (1 - 44.1 / SR, a 22.7 ms time constant), so live control moves exactly as fast as before; only the start differs. + // Started at its input a constant control holds there to the bit; started at 0 (si.smoo, si.smooth) it stalls 1e-5 to 5e-5 short of it, + // in either way of writing the step. The source's pitch-ratio smoother starts at its input for that reason: from 0 the 2744 Hz alias + // source ran 0.12 Hz flat and leaked its harmonics into the alias measurement. + source.AppendLine("smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); };"); + source.AppendLine("smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x);"); source.AppendLine("seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d));"); source.AppendLine("seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d)));"); source.AppendLine("seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b));"); @@ -152,7 +161,7 @@ public static partial class FaustEmitter var baseId = $"{id}_base"; var normalized = $"{id}_normalized"; var physical = $"{id}_physical"; - var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : si.smoo"; + var publicControl = $"hslider(\"{Escape(surface.Path)}\", {F(surface.DefaultNormalized)}, 0, 1, 0.001) : smoo_init"; var boundControl = parameters.TryBindingExpression(surface.FieldPath, out var bindingExpression) ? $"clip01(({bindingExpression} - {F(surface.MinValue)}) / max(0.000001, {F(surface.MaxValue - surface.MinValue)}))" : publicControl; @@ -172,7 +181,7 @@ public static partial class FaustEmitter var splineId = $"{id}_spline_{splineIndex}"; EmitControlSpline(source, spline, splineId, surface.DefaultNormalized); var depth = $"{splineId}_depth"; - source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : si.smoo;"); + source.AppendLine($"{depth} = hslider(\"/splines/{Escape(spline.Name)}/depth\", 1, 0, 1, 0.001) : smoo_init;"); terms.Add($"(({splineId}_value - {F(surface.DefaultNormalized)}) * {depth})"); } @@ -197,11 +206,11 @@ public static partial class FaustEmitter { var point = points[i]; source.AppendLine($"{splineId}_p{i}_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/time\", {F(point.TimeSeconds)}, 0, 120, 0.001);"); - source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : si.smoo;"); + source.AppendLine($"{splineId}_p{i}_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/value\", {F(Math.Clamp(point.Value, 0, 1))}, 0, 1, 0.001) : smoo_init;"); source.AppendLine($"{splineId}_p{i}_out_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_time\", {F(Math.Max(0, point.OutTimeOffsetSeconds))}, 0, 10, 0.001);"); - source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : si.smoo;"); + source.AppendLine($"{splineId}_p{i}_out_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/out_value\", {F(point.OutValue == 0 ? point.Value : Math.Clamp(point.OutValue, 0, 1))}, 0, 1, 0.001) : smoo_init;"); source.AppendLine($"{splineId}_p{i}_in_time = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_time\", {F(Math.Max(0, point.InTimeOffsetSeconds))}, 0, 10, 0.001);"); - source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : si.smoo;"); + source.AppendLine($"{splineId}_p{i}_in_value = hslider(\"/splines/{Escape(spline.Name)}/{i}/in_value\", {F(point.InValue == 0 ? point.Value : Math.Clamp(point.InValue, 0, 1))}, 0, 1, 0.001) : smoo_init;"); } if (points.Length == 1) @@ -739,7 +748,9 @@ public static partial class FaustEmitter var impedance = parameters.Expression(OwnerField(owner, "impedance"), port.Impedance); var flowScale = parameters.Expression(OwnerField(owner, "flow_scale"), port.FlowScale); // f0 is the voice frequency times the source's ratio (the stress cue), smoothed over 10 ms so a step in the ratio is a glide. - var pitchRatio = $"({parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)} : si.smooth(ba.tau2pole(0.01)))"; + // Its own 10 ms smoother also starts at its input: from 0 it glided the source's pitch up from nothing at the start of every word + // (the control feeding it used to hide that under its own 100 ms climb) and rounded to a value up to ~4e-5 off its input. + var pitchRatio = $"smoo_from(ba.tau2pole(0.01), {parameters.Expression(OwnerField(owner, "pitch_ratio"), port.PitchRatio)})"; var load = $"{voiceName}_primitive_source_{safe}_load_pressure"; var voiced = $"{voiceName}_primitive_source_{safe}_voiced"; var flow = $"{voiceName}_primitive_source_{safe}_flow"; diff --git a/src/AquaSynth.Core/IpaGestureExperiment.cs b/src/AquaSynth.Core/IpaGestureExperiment.cs index 84a2c69..867e399 100644 --- a/src/AquaSynth.Core/IpaGestureExperiment.cs +++ b/src/AquaSynth.Core/IpaGestureExperiment.cs @@ -278,7 +278,7 @@ public static class IpaGestureExperiment probe_timeline name=flow network=voice blocks=12 block_size=64 {{SideStubDeclarations(sideStub)}}vocal_network name=voice paths=oral_path,nasal_path{{(sideStub ? ",side_stub_path" : "")}} sources=folds contacts=contact branches=velopharynx{{(sideStub ? "," + PatchScript.SideStubBranch : "")}} radiation=mouth,nostril{{(sideStub ? ",side_stub_end" : "")}} probe=flow {{string.Join("\n", gestureLines)}} - vocal network=voice freq={{Format(frequency)}} gain=.7 sustain={{Format(sustain)}} + vocal network=voice freq={{Format(frequency)}} gain=.7 attack=.012 sustain={{Format(sustain)}} """; private static IReadOnlyList SummarizeMetrics(IReadOnlyList metrics) => diff --git a/src/AquaSynth.Core/PatchScript.cs b/src/AquaSynth.Core/PatchScript.cs index 38a619b..7b5a89e 100644 --- a/src/AquaSynth.Core/PatchScript.cs +++ b/src/AquaSynth.Core/PatchScript.cs @@ -17,6 +17,22 @@ public static class PatchScript /// A script that gestures a lateral without declaring it is refused, not approximated. public const string SideStubBranch = "side_stub"; + // Seconds an articulator takes to move from where the previous phone left it to this phone's first value. A tract does not step + // between phones: the merged surface curve held the previous value up to the boundary and jumped, and a jump in a ladder cell's + // area is a transient the ear hears as a click (nibu: peak 0.40 and 0.49 against a 0.2 vowel, at the 95 and 275 ms boundaries), + // and it leaves the consonant without the formant transitions that tell /b/ from a fade-in. A DESIGN VALUE inside the 20-50 ms + // of a CV formant transition; no measurement of this tract retrieved. + public const float PhoneTransitionSeconds = 0.03f; + + // Seconds a stop takes to open from full closure; its burst lasts about this long. A DESIGN VALUE inside the 5-15 ms of a + // stop burst; no measurement of this tract retrieved. The opening's noise is still shaped by the exporter. + private const float StopReleaseSeconds = 0.012f; + + /// Seconds after the start of a stop gesture of the given duration at which its articulators begin to open (the stop is + /// shut until then). The one owner of the release time: the contact, lip and constriction splines hold shut to here and the burst + /// gain window opens here; a test that wants the release reads it from here. + public static float StopReleaseStartSeconds(float durationSeconds) => durationSeconds - MathF.Min(StopReleaseSeconds, 0.3f * durationSeconds); + public static bool TryIpaDescriptor(string ipa, out string descriptor) { if (StoryVowelAreas.TryVowelForIpa(ipa, out var vowelDescriptor, out _)) @@ -1494,11 +1510,58 @@ public static class PatchScript duration * 0.25f, Math.Clamp(point.value, 0, 1))) .ToArray(); + splinePoints = FollowSegments(WithPhoneTransition(surfacePath, start, duration, splinePoints)); var spline = new ControlSpline(splineName, surfacePath, splinePoints, ControlSplineInterpolation.Bezier); _controlSplines.Add(spline); _controlSplinesByName[spline.Name] = spline; } + + + // Prepends the boundary ramp: a point at the gesture start holding the value the surface has there, then the gesture's first + // point PhoneTransitionSeconds later (at most 0.4 of the gesture, and before its second point). The first phone of a word, and + // a surface the earlier phones never touched, keep their value from the start. + private ControlSplinePoint[] WithPhoneTransition(string surfacePath, float start, float duration, ControlSplinePoint[] points) + { + ControlSplinePoint? previous = null; + foreach (var spline in _controlSplines) + { + if (!spline.SurfacePath.Equals(surfacePath, StringComparison.OrdinalIgnoreCase)) continue; + foreach (var point in spline.Points) + { + if (point.TimeSeconds <= start + 0.000001f && (previous is null || point.TimeSeconds >= previous.TimeSeconds)) previous = point; + } + } + + if (previous is null || points.Length < 2 || MathF.Abs(previous.Value - points[0].Value) <= 0.0005f) return points; + var ramp = MathF.Min(PhoneTransitionSeconds, 0.4f * duration); + ramp = MathF.Min(ramp, points[1].TimeSeconds - points[0].TimeSeconds); + if (ramp < 0.002f) return points; + var handle = ramp / 3; + var first = points[0]; + var lead = new ControlSplinePoint(first.TimeSeconds, previous.Value, handle, previous.Value, handle, previous.Value); + var arrival = first with { TimeSeconds = first.TimeSeconds + ramp, InTimeOffsetSeconds = handle, InValue = first.Value }; + return [lead, arrival, .. points.Skip(1)]; + } + + // Sets every handle's time to a third of the segment it belongs to. The Faust lowering evaluates a segment's Bezier on the + // linear fraction of its time and ignores the handle times; a handle of a third makes the C# timeline (which solves the curve's + // time for its parameter) agree with it exactly. A handle longer than the segment (0.25 of the gesture on a 12 ms opening) folds + // the C# curve back on itself. + private static ControlSplinePoint[] FollowSegments(ControlSplinePoint[] points) + { + if (points.Length < 2) return points; + var shaped = new ControlSplinePoint[points.Length]; + for (var i = 0; i < points.Length; i++) + { + var after = (i + 1 < points.Length ? points[i + 1].TimeSeconds - points[i].TimeSeconds : points[i].TimeSeconds - points[i - 1].TimeSeconds) / 3; + var before = (i > 0 ? points[i].TimeSeconds - points[i - 1].TimeSeconds : points[1].TimeSeconds - points[0].TimeSeconds) / 3; + shaped[i] = points[i] with { InTimeOffsetSeconds = before, OutTimeOffsetSeconds = after }; + } + + return shaped; + } + private string UniqueSplineName(string name) { var safe = SafeName(name); @@ -1561,18 +1624,25 @@ public static class PatchScript // closed-mouth sound, and an alveolar closure that began to open at 0.5 let the oral tract radiate again over the last third (/n/ read // -3.8 dB under its vowels against /m/ -13.2 dB, whose lips hold to 0.7). var stopsBehindTheLips = (tokens.Contains("plosive") || tokens.Contains("stop")) && !(tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial")); - // A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (0.65): at the nasal's 0.03 cm + // A stop whose seal is the tongue (not the lips) is closed completely and until the contact opens (StopReleaseStartSeconds): at the nasal's 0.03 cm // aperture a /t/ closure measured -10.4 dB re its vowels against /p/ -19.7 (the lips seal /p/ whatever the tongue does), and // rendered -21.5 dB with the aperture at 0 (area floor 1e-6 cm^2). - var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : stopsBehindTheLips ? 0.65f : 0.5f; + // Every stop holds all its articulators shut until StopReleaseSeconds before its end and opens in that time: the burst is the + // opening, and an opening spread over a third of the gesture (33 ms of the 95 ms /b/) made the burst a fricative's length + // (the phone recogniser heard /v/ before the vowel, Meta heard "fah then bah"). + var isStop = tokens.Contains("plosive") || tokens.Contains("stop"); + var stopHold = StopReleaseStartSeconds(duration) / duration; + var closureHold = tokens.Contains("nasal") || tokens.Contains("lateral") ? 0.8f : isStop ? stopHold : 0.5f; var closed = stopsBehindTheLips ? 0f : constriction; AddAuthoredSpline($"{name}_constriction_diameter", "/vocal/areas/0/area/constriction_diameter", start, duration, (0, closed), (closureHold, closed), (1, Math.Min(0.65f, constriction + 0.2f))); if (tokens.Contains("bilabial") || tokens.Contains("labiodental") || tokens.Contains("labial") || tokens.Contains("rounded")) { - var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : 0.04f; - AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.45f)); - AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (0.7f, lipTarget), (1, 0.75f)); + // A stop's lips are shut, not at a 0.04 leak: at 0.04 the closed-mouth voice bar of /b/ read -19.1 dB re the following vowel (the + // test's design value is -34 to -26) and /p/ -30.1 (-60 to -38); shut, -27.7 and -38.8. A nasal keeps its murmur's small opening. + var lipTarget = tokens.Contains("approximant") ? 0.38f : tokens.Contains("fricative") ? 0.18f : isStop ? 0f : 0.04f; + AddAuthoredSpline($"{name}_lip_area", "/vocal/areas/0/area/lip_opening", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.45f)); + AddAuthoredSpline($"{name}_lip_radiation", "/vocal/radiation/0/aperture", start, duration, (0, lipTarget), (isStop ? stopHold : 0.7f, lipTarget), (1, 0.75f)); } if (tokens.Contains("nasal")) @@ -1582,13 +1652,18 @@ public static class PatchScript } else if (tokens.Contains("plosive") || tokens.Contains("stop")) { - AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (0.65f, 0.02f), (1, 0.9f)); - AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (0.65f, 0.75f), (1, 0.05f)); + AddAuthoredSpline($"{name}_contact", "/vocal/contacts/0/opening", start, duration, (0, 0.02f), (stopHold, 0.02f), (1, 0.9f)); + AddAuthoredSpline($"{name}_stored_pressure", "/vocal/contacts/0/stored_pressure", start, duration, (0, 0.4f), (stopHold, 0.75f), (1, 0.05f)); // The release burst is turbulence at the opening constriction: its gain is held for the plosive and the exporter shapes // it by the stored pressure and the opening (see PrimitiveContactExpression). The gain exists only around this plosive: // a surface holds its last value after its last point (and its first before its first), so a gain left at its level // made noise at every later contact state, the final nasal's closure and the tail included. - GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start, duration, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain); + // The exporter's burst is noise times 4 * opening * (1 - opening): it sounds whenever the contact passes between shut and open + // in either direction, so a gain held through the closure made the 30 ms closing ramp a 15-20 ms hiss before every stop (the + // >4 kHz share of the frames before /t/ was -1 dB against -57 dB with the closure stepped). The gain exists only from the + // stop's release start to its end, one ramp (0.15 of that window) either side. + var releaseStart = StopReleaseStartSeconds(duration); + GainAroundGesture($"{name}_release_flow", "/vocal/contacts/0/release_flow", start + releaseStart, duration - releaseStart, stopsBehindTheLips ? TongueStopBurstGain : PlosiveBurstGain); PlaceBurst(name, tokens, place, start, duration); AddAuthoredSpline($"{name}_velum", "/vocal/branches/0/opening", start, duration, (0, VelumClosed), (1, VelumClosed)); } @@ -1670,15 +1745,14 @@ public static class PatchScript } var level = Math.Clamp((gain - surface.MinValue) / (surface.MaxValue - surface.MinValue), 0, 1); - var handle = duration * 0.25f; - ControlSplinePoint At(float time, float value) => new(time, value, handle, value, handle, value); + ControlSplinePoint At(float time, float value) => new(time, value, 0, value, 0, value); var ramp = duration * 0.15f; var points = new List(); if (start > 0) points.Add(At(Math.Max(0, start - ramp), 0)); points.Add(At(start, level)); points.Add(At(start + duration, level)); points.Add(At(start + duration + ramp, 0)); - var spline = new ControlSpline(UniqueSplineName(name), surfacePath, points.ToArray(), ControlSplineInterpolation.Bezier); + var spline = new ControlSpline(UniqueSplineName(name), surfacePath, FollowSegments(points.ToArray()), ControlSplineInterpolation.Bezier); _controlSplines.Add(spline); _controlSplinesByName[spline.Name] = spline; } diff --git a/tests/AquaSynth.Dsl.Tests/NasalCouplingTests.cs b/tests/AquaSynth.Dsl.Tests/NasalCouplingTests.cs index 638bf3f..58e3d9c 100644 --- a/tests/AquaSynth.Dsl.Tests/NasalCouplingTests.cs +++ b/tests/AquaSynth.Dsl.Tests/NasalCouplingTests.cs @@ -68,7 +68,7 @@ public sealed class NasalCouplingTests } private static float Spline(SynthPatch patch, string namePrefix, string surfacePath) => - patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[0].Value; + patch.ControlSplines.Single(spline => spline.Name.StartsWith(namePrefix, StringComparison.Ordinal) && spline.SurfacePath == surfacePath).Points[^1].Value; // a held place: the last point is the gesture's value whether or not a boundary ramp precedes it // Distance of a vowel_set consonant constriction from the lip plane in cm, from the constriction_index spline: the surface is a section // index in [0, sections] and the area model reads position = index / (sections - 1). diff --git a/tests/AquaSynth.Dsl.Tests/PhoneTransitionTests.cs b/tests/AquaSynth.Dsl.Tests/PhoneTransitionTests.cs new file mode 100644 index 0000000..43a3817 --- /dev/null +++ b/tests/AquaSynth.Dsl.Tests/PhoneTransitionTests.cs @@ -0,0 +1,84 @@ +using AquaSynth.Dsl; +using AquaSynth.Faust; + +namespace AquaSynth.Dsl.Tests; + +// What Meta heard in tract-weak-spots-1 (2026-11-13) while every formant matched: clicks at the phone boundaries of nibu, a /b/ burst the +// length of a fricative, and a first 100 ms that climbed from silence. These pin the three causes at the control-spline and DSP-source +// level (docs/listening/tract-weak-spots-1-analysis-20261113.md); the renders are measured by tools/formant-check/ear_metrics.py. +public sealed class PhoneTransitionTests +{ + private static SynthPatch Nibu() => PatchScript.Parse(IpaGestureExperiment.BuildWordScript( + [ + (new IpaGestureExperimentTarget("w_n", "n", "", .095f), 1f), + (new IpaGestureExperimentTarget("w_i", "i", "", .18f), 1f), + (new IpaGestureExperimentTarget("w_b", "b", "", .095f), 1f), + (new IpaGestureExperimentTarget("w_u", "u", "", .18f), 1f) + ], frequency: 140)); + + [Fact] + public void NoSurfaceStepsAtAPhoneBoundary() + { + var timeline = new ControlSplineTimeline(Nibu().ControlSplines); + var worst = (Surface: "", Boundary: 0f, Step: 0f); + foreach (var boundary in new[] { .095f, .275f, .37f }) + { + foreach (var surface in timeline.SurfacePaths) + { + var step = MathF.Abs(timeline.ValueAt(surface, boundary + .0005f) - timeline.ValueAt(surface, boundary - .0005f)); + if (step > worst.Step) worst = (surface, boundary, step); + } + } + + // 0.5 ms either side of a boundary; a 30 ms ramp across the full 0..1 range moves 0.017 in that time, a step moves it by the + // whole difference (the pre-fix nibu stepped /vocal/areas/0/area/constriction_diameter by 0.77 at the 95 ms boundary). + Assert.True(worst.Step < .03f, $"{worst.Surface} steps {worst.Step:0.000} at {worst.Boundary} s"); + } + + [Fact] + public void ThePhoneAfterABoundaryArrivesAtItsFirstValueWithinTheTransition() + { + var timeline = new ControlSplineTimeline(Nibu().ControlSplines); + const string diameter = "/vocal/areas/0/area/constriction_diameter"; + var closedN = timeline.ValueAt(diameter, .05f); + Assert.True(closedN < .1f, "the /n/ constriction is shut"); + Assert.True(timeline.ValueAt(diameter, .095f + .002f) < .5f, "still closing into the vowel two ms after the boundary"); + Assert.Equal(1f, timeline.ValueAt(diameter, .095f + .035f), 2); + } + + [Fact] + public void TheFirstPhoneOfAWordStartsAtItsOwnValue() + { + var timeline = new ControlSplineTimeline(Nibu().ControlSplines); + Assert.Equal(timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", 0f), timeline.ValueAt("/vocal/areas/0/area/constriction_diameter", .02f), 3); + } + + [Fact] + public void AStopOpensInTwelveMillisecondsNotAThirdOfItsGesture() + { + var timeline = new ControlSplineTimeline(Nibu().ControlSplines); + const string contact = "/vocal/contacts/0/opening"; + const float start = .275f, duration = .095f; + Assert.True(timeline.ValueAt(contact, start + duration - .013f) < .1f, "closed until 12 ms before the end"); + Assert.True(timeline.ValueAt(contact, start + duration) > .8f, "open at the end"); + var opening = 0f; + for (var t = start + .03f; t <= start + duration; t += .0005f) + { + if (timeline.ValueAt(contact, t) > .3f) { opening = t; break; } + } + + Assert.True(start + duration - opening < .012f, $"the opening crosses 0.3 only {start + duration - opening:0.000} s before the end"); + } + + [Fact] + public void ControlSmoothersStartAtTheirInputNotAtZero() + { + var source = FaustEmitter.Emit(Nibu(), new FaustExportOptions("transition_probe")).Source; + Assert.Contains("smoo_init(x) =", source); + Assert.Contains("smoo_init;", source); + // Every control surface and spline-point slider uses it; patch parameters keep si.smoo. + var remaining = source.Split('\n').Where(line => line.Contains("hslider", StringComparison.Ordinal) && line.Contains(": si.smoo;", StringComparison.Ordinal) + && (line.Contains("\"/vocal/", StringComparison.Ordinal) || line.Contains("\"/splines/", StringComparison.Ordinal))).ToArray(); + Assert.Empty(remaining); + } +} diff --git a/tests/AquaSynth.Dsl.Tests/PlaceTableTests.cs b/tests/AquaSynth.Dsl.Tests/PlaceTableTests.cs index 6813300..a19ea3f 100644 --- a/tests/AquaSynth.Dsl.Tests/PlaceTableTests.cs +++ b/tests/AquaSynth.Dsl.Tests/PlaceTableTests.cs @@ -16,9 +16,10 @@ public sealed class PlaceTableTests(ITestOutputHelper output) private static float ConstrictionIndex(string script, string name) => PatchScript.Parse(script).ControlSplines - .Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[0].Value; + .Single(s => s.Name.StartsWith($"{name}_constriction_index", StringComparison.Ordinal) && s.SurfacePath == "/vocal/areas/0/area/constriction_index").Points[^1].Value; // the held place; a boundary ramp may precede it - // Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the + // Re-pinned on the phone-transition cut (boundary ramps, 12 ms stop release, smoo_init, vocal attack): these ids moved on purpose; the invariant + // that survives is that a word of labials writes no position surface (below), not these bytes. Faust ids re-pinned again on the PR 52 fix passes (smoo_init pole, pitch-ratio smoother, shut lips for stops, spline handles; the scripts are unchanged; the last move is the smoo_from step line only, which reverts to prev * pole + x * (1 - pole): the old ids come back when that line is put back). Pre-cut values on af0e44f: the vowel_set constriction_index spline value of /b/ and /n/ (and the hash of the whole script and of the // Faust source for /apa/ and /aba/, which this cut must not change by a byte; the ids are a SHA-256 prefix). private const float BIndex = 0.8333333f, NIndex = 0.7118056f; @@ -28,7 +29,7 @@ public sealed class PlaceTableTests(ITestOutputHelper output) var b = ConstrictionIndex(Word("b"), "w_1_c"); var n = ConstrictionIndex(Word("n"), "w_1_c"); output.WriteLine($"b {b:R} n {n:R}"); - foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE") }) + foreach (var (ipa, scriptSha, faustSha) in new[] { ("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C") }) { var script = Word(ipa); var faust = FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("pin", SampleRate: 48000)).Source; @@ -47,8 +48,8 @@ public sealed class PlaceTableTests(ITestOutputHelper output) { foreach (var (ipa, scriptSha, faustSha) in new[] { - ("p", "61A96E0AE8C68F52", "28022E56A77DB5B1"), ("b", "5D5A6B653DC5D37D", "2FB84E792B878BFE"), - ("n", "BDF24A66659D7BD6", "ABEB5D03DC9637DB"), ("t", "9DFBCDB5131F9776", "30E144D54F7E026F"), + ("p", "0E86DB0F40C0F12A", "358106F493D1783D"), ("b", "A63403EBAAD016FB", "AE80CF31EF1DDC9C"), + ("n", "5C3FA24E40D9DE26", "B2B9C5E1015B60A9"), ("t", "45941CCFD93561B3", "0AACE6D26F648FB0"), }) { var script = Word(ipa); @@ -109,8 +110,9 @@ public sealed class PlaceTableTests(ITestOutputHelper output) var script = IpaGestureExperiment.BuildWordScript( [(Phone("0_b", "b", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (new IpaGestureExperimentTarget("w_2_t", "t", "voiceless_alveolar_plosive", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_p", "p", 0.095f), 1f)], 140); Assert.All(SitePoints(script, "w_0_b"), v => Assert.Equal(0f, v)); - Assert.All(SitePoints(script, "w_4_p"), v => Assert.Equal(0f, v)); - Assert.All(SitePoints(script, "w_2_t"), v => Assert.True(v > 0.8f)); + Assert.All(SitePoints(script, "w_4_p").Skip(1), v => Assert.Equal(0f, v)); + // after the 30 ms boundary ramp from the previous plosive's place (the first point is that place) + Assert.All(SitePoints(script, "w_2_t").Skip(1), v => Assert.True(v > 0.8f)); } [Fact] @@ -120,11 +122,13 @@ public sealed class PlaceTableTests(ITestOutputHelper output) .Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points.Select(p => p.Value).ToArray(); var t = PatchScript.Parse(Word("t", "voiceless_alveolar_plosive")).ControlSplines .Single(s => s.Name.StartsWith("w_1_c_constriction_diameter", StringComparison.Ordinal)).Points; - Assert.Equal(0f, t[0].Value); + // lead-in (the vowel's open 1), arrival, closure hold, end: shut from the arrival to the hold, and the hold ends StopReleaseSeconds before the stop does + Assert.Equal(1f, t[0].Value); Assert.Equal(0f, t[1].Value); - Assert.Equal(0.65f, (t[1].TimeSeconds - 0.18f) / 0.095f, 3); + Assert.Equal(0f, t[2].Value); + Assert.Equal(1 - 0.012f / 0.095f, (t[2].TimeSeconds - 0.18f) / 0.095f, 3); var b = Diameter(Word("b")); - Assert.True(b[0] > 0 && Math.Abs(b[0] - b[1]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)"); + Assert.True(b[1] > 0 && Math.Abs(b[1] - b[2]) < 1e-6f, "the lip-sealed stop keeps the 0.03 aperture spline (its source text is pinned)"); } [Fact] diff --git a/tests/AquaSynth.Dsl.Tests/StopClosureAndControlRenderTests.cs b/tests/AquaSynth.Dsl.Tests/StopClosureAndControlRenderTests.cs new file mode 100644 index 0000000..973d9e5 --- /dev/null +++ b/tests/AquaSynth.Dsl.Tests/StopClosureAndControlRenderTests.cs @@ -0,0 +1,183 @@ +using AquaSynth.Dsl; +using AquaSynth.Faust; +using Xunit.Abstractions; + +namespace AquaSynth.Dsl.Tests; + +// Renders behind the tract-coarticulation findings of the PR 52 Soul pass: what the closing ramp into a stop sounds like, what the DSP +// does with a stop's 12 ms opening, and what the control smoothers and the word attack do at the start of a render. Each reads the +// rendered signal (or the DSP's own probe), not the emitted text. +[Collection("NativeFaust")] +public sealed class StopClosureAndControlRenderTests(ITestOutputHelper output) +{ + private const int Rate = 48000; + + private static AquaSynthCompiledPatch? Compile(string source, float seconds) + { + using var compiler = new AquaSynthPatchCompiler(new AquaSynthNativeOptions(SampleRate: Rate)); + if (!compiler.TryCompileSource(new AquaSynthCompileIdentity("stop_closure", "stop_closure", ""), source, seconds, out var patch, out var error)) + { + Assert.Fail($"compile failed: {error}"); + } + + return patch; + } + + private static string Word(string ipa) => IpaGestureExperiment.BuildWordScript( + [(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140); + + // Power above 4 kHz (100 Hz grid to 12 kHz) of a Hann-windowed stretch. + private static double HighPower(float[] x, double from, double to) + { + var a = (int)(from * Rate); + var n = (int)((to - from) * Rate); + var seg = Enumerable.Range(0, n).Select(i => x[a + i] * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray(); + double sum = 0; + for (var hz = 4000; hz < 12000; hz += 100) + { + double re = 0, im = 0; + for (var i = 0; i < n; i++) { var w = 2 * Math.PI * hz * i / Rate; re += seg[i] * Math.Cos(w); im -= seg[i] * Math.Sin(w); } + sum += re * re + im * im; + } + + return sum; + } + + // The 30 ms boundary ramp into a stop moves the contact, constriction and lip surfaces through every aperture between open and shut. The + // burst noise is 4 * opening * (1 - opening) times a gain, and the gain was held through the closure, so each of those passes was a 15-20 ms + // hiss before the stop. The interval is the stop's start to its release, against the same length of the vowel before it, in the >4 kHz + // band, with the glottal source silent and the burst gain kept: the voiced stops' own voicing runs through the ramp and its harmonics + // fill that band (/d/ read -11.6 dB with the source on and -52.8 with it off; with the source on it swings between -13.5 and -9.3 dB + // with f0 alone, 139.5 to 145 Hz, as the harmonics fall on and off the 100 Hz grid), so with the source on the measure cannot tell the + // voicing from the hiss. Source off, the burst gain held through the closure read +5 dB (/p/) and +14 dB (/t/, /d/) on 3b62b9c; gated, + // every stop reads -52 dB. -30 is the bound between the two. /g/ is not here: it is not in TryIpaDescriptor, so it would render the default approximant and could not fail. + [FaustTheory] + [InlineData("p")] + [InlineData("b")] + [InlineData("t")] + [InlineData("d")] + [InlineData("k")] + public void ClosingIntoAStopMakesNoHighFrequencyNoise(string ipa) + { + var parsed = PatchScript.Parse(Word(ipa)); + var voiceless = parsed with + { + ControlSplines = parsed.ControlSplines.Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal) + ? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() } + : s).ToArray() + }; + using var patch = Compile(FaustEmitter.Emit(voiceless, new FaustExportOptions("stop_closure", SampleRate: Rate)).Source, 0.5f); + var x = patch!.Render(0.5f); + const double from = 0.18; + var release = PatchScript.StopReleaseStartSeconds(0.095f); + var closing = HighPower(x, from, from + release); + var vowel = HighPower(x, from - release, from); + var db = 10 * Math.Log10(closing / vowel); + output.WriteLine($"/{ipa}/ >4 kHz energy of the closing interval {db:F1} dB re the vowel's before it, source silent"); + Assert.True(db < -30, $"/{ipa}/ closes with {db:F1} dB re the vowel's own >4 kHz energy"); + } + + // The Faust lowering of a Bezier segment ignores the handle times and runs the curve on the linear fraction of the segment; the C# timeline + // solves the curve's time, so a handle longer than the segment folds it. The stop's 12 ms opening had 23.75 ms handles: the contact opening + // read 0.05 to 0.87 between 6 and 7 ms in C# and 0.46 to 0.57 in the DSP. This reads the opening through the DSP's own contact reservoir probe + // (stored pressure + (1 - opening) * resistance, both at their defaults once the other splines are dropped) and compares it with the timeline. + [FaustFact] + public void TheDspRendersTheStopOpeningAsTheTimelineEvaluatesIt() + { + var script = Word("p"); + var parsed = PatchScript.Parse(script); + var contact = parsed.ControlSplines.Where(s => s.SurfacePath == "/vocal/contacts/0/opening").ToArray(); + var alone = parsed with { ControlSplines = contact }; + var timeline = new ControlSplineTimeline(contact); + var source = FaustEmitter.Emit(alone, new FaustExportOptions("stop_opening", SampleRate: Rate, DebugProbeUi: true)).Source; + using var patch = Compile(source, 0.5f); + using var stream = patch!.CreateStreamingPatch(); + var reservoir = stream.ProbePaths.Single(p => p.EndsWith("/contact/contact/reservoir", StringComparison.Ordinal)); + var release = 0.18f + PatchScript.StopReleaseStartSeconds(0.095f); + var outputs = new[] { new float[48] }; + var dsp = new List<(float Time, float Reservoir)>(); + for (var block = 0; block < 400; block++) + { + stream.ProcessBlock([], outputs, 48); + var t = (block + 1) * 48f / Rate; + if (t >= release - 0.004f && t <= release + 0.016f) dsp.Add((t, stream.ReadProbe(reservoir))); + } + + // reservoir is linear in the opening; scale it by its own end points + var shut = dsp.First().Reservoir; + var open = dsp.Last().Reservoir; + var worst = 0f; + foreach (var (time, value) in dsp) + { + var dspOpening = (value - shut) / (open - shut); + var curve = timeline.ValueAt("/vocal/contacts/0/opening", time); + var shutValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.First().Time); + var openValue = timeline.ValueAt("/vocal/contacts/0/opening", dsp.Last().Time); + var timelineOpening = (curve - shutValue) / (openValue - shutValue); + worst = MathF.Max(worst, MathF.Abs(dspOpening - timelineOpening)); + } + + output.WriteLine($"worst difference between the DSP's opening and the timeline's over the release: {worst:F3}"); + Assert.True(worst < 0.08f, $"the DSP's stop opening departs from the timeline by {worst:F3}"); + } + + // The same fact without the DSP: every handle of an authored spline is a third of its segment, so the C# curve is the curve the lowering runs. + [Fact] + public void EveryAuthoredSplineHandleIsAThirdOfItsSegment() + { + var splines = PatchScript.Parse(Word("p")).ControlSplines.Where(s => s.Points.Count > 1).ToArray(); + Assert.NotEmpty(splines); + foreach (var spline in splines) + { + for (var i = 0; i + 1 < spline.Points.Count; i++) + { + var third = (spline.Points[i + 1].TimeSeconds - spline.Points[i].TimeSeconds) / 3; + Assert.Equal(third, spline.Points[i].OutTimeOffsetSeconds, 5); + Assert.Equal(third, spline.Points[i + 1].InTimeOffsetSeconds, 5); + } + } + } + + // Replaces the emitted patch's process with one control through the emitted smoo_init, so the smoother under test is the one the exporter + // writes. Starts at its input at sample 0, and moves at si.smoo's 22.7 ms time constant when the control moves (5 ms gave 0.89 here). + [FaustFact] + public void ASmootherStartsAtItsInputAndMovesAtSiSmoosPace() + { + var emitted = FaustEmitter.Emit(PatchScript.Parse(Word("p")), new FaustExportOptions("smoo", SampleRate: Rate)).Source; + var lines = emitted.Split('\n').Select(l => l.StartsWith("process =", StringComparison.Ordinal) ? "process = hslider(\"/vocal/sources/0/pressure\", 0.7, 0, 1, 0.001) : smoo_init;" : l); + using var patch = Compile(string.Join('\n', lines), 0.2f); + using var stream = patch!.CreateStreamingPatch(); + var first = new[] { new float[480] }; + stream.ProcessBlock([], first, 480); + output.WriteLine($"sample 0 {first[0][0]:F4}, after 10 ms {first[0][479]:F4}"); + Assert.Equal(0.7f, first[0][0], 4); + Assert.Equal(0.7f, first[0][479], 4); + var after = new[] { new float[240] }; + stream.ProcessBlock([], after, 240, new Dictionary { ["/vocal/sources/0/pressure"] = 1f }); + // 5 ms into a step from 0.7 to 1 with a 22.7 ms time constant + var expected = 0.7 + 0.3 * (1 - Math.Pow(1 - 44.1 / Rate, 0.005 * Rate)); + output.WriteLine($"5 ms after a step to 1: {after[0][239]:F4}, si.smoo would be {expected:F4}"); + Assert.InRange(after[0][239], expected - 0.01, expected + 0.01); + } + + // A word starts with a 12 ms attack, not a step: the control smoothers no longer fade it in, and a step onset would be a click. + [FaustFact] + public void AWordBeginsWithATwelveMillisecondAttack() + { + double FirstMillisecondsPeak(string script) + { + using var patch = Compile(FaustEmitter.Emit(PatchScript.Parse(script), new FaustExportOptions("attack", SampleRate: Rate)).Source, 0.4f); + var x = patch!.Render(0.4f); + var steady = x.Skip((int)(0.05 * Rate)).Take((int)(0.1 * Rate)).Max(Math.Abs); + return x.Take((int)(0.002 * Rate)).Max(Math.Abs) / steady; + } + + var script = IpaGestureExperiment.BuildWordScript([(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.3f), 1f)], 140); + Assert.Contains("attack=.012", script); + var attacked = FirstMillisecondsPeak(script); + var stepped = FirstMillisecondsPeak(script.Replace("attack=.012 ", "", StringComparison.Ordinal)); + output.WriteLine($"peak of the first 2 ms re the steady peak: with the attack {attacked:F3}, without {stepped:F3}"); + Assert.True(attacked < 0.25, $"the first 2 ms reach {attacked:F2} of the steady peak"); + Assert.True(stepped > 2 * attacked, "without the attack the onset is a step"); + } +} diff --git a/tests/AquaSynth.Dsl.Tests/StressAndReleaseRenderTests.cs b/tests/AquaSynth.Dsl.Tests/StressAndReleaseRenderTests.cs index 1873a27..74a3677 100644 --- a/tests/AquaSynth.Dsl.Tests/StressAndReleaseRenderTests.cs +++ b/tests/AquaSynth.Dsl.Tests/StressAndReleaseRenderTests.cs @@ -24,6 +24,11 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) using (patch) return patch!.Render(seconds); } + // The stop's release time and the start of its closure (every articulator at its shut value, after the boundary ramp into it), both from the + // one owner in PatchScript rather than as fractions of the gesture; the consonant is [from, to). + private static double ReleaseAt(double from, double to) => from + PatchScript.StopReleaseStartSeconds((float)(to - from)); + private static double ClosureFrom(double from) => from + PatchScript.PhoneTransitionSeconds; + private static double Rms(float[] x, double from, double to) { var a = (int)(from * Rate); @@ -95,7 +100,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var withoutBurst = Render(script, 0.4f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() }); // Everything but the burst's noise is deterministic and shared, so the difference of the two renders is the burst alone. var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray(); - const double releaseFrom = 0.095 * 0.65, vowelFrom = 0.095; + const double vowelFrom = 0.095; + var releaseFrom = ReleaseAt(0, vowelFrom); var vowelPeak = Peak(withBurst, vowelFrom + 0.06, vowelFrom + 0.18); var burstPeak = Peak(burst, releaseFrom, vowelFrom + 0.01); var burstBeforeRelease = Peak(burst, 0, releaseFrom - 0.005); @@ -121,7 +127,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var withoutBurst = Render(script, 0.77f, p => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() }); var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray(); const double bFrom = 0.275, bTo = 0.37; - var atRelease = Peak(burst, bFrom + 0.095 * 0.6, bTo + 0.02); + var atRelease = Peak(burst, ReleaseAt(bFrom, bTo) - 0.003, bTo + 0.02); var leading = Peak(burst, 0, bFrom); var trailing = Peak(burst, bTo + 0.12, 0.77); output.WriteLine($"burst peak {atRelease:F4} at the release, {leading:F4} before the plosive (n, a), {trailing:F4} after it (a, n, tail)"); @@ -130,17 +136,64 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) Assert.True(trailing < atRelease / 10, "no burst noise in the vowel, the nasal and the tail after the plosive"); } - // The burst gain's window, read off the spline itself: it opens one ramp (0.15 of the gesture) before the plosive and has closed one ramp after it. - // The rendered test above survived three mutants of the window (ramp 0, ramp 1.0 of the gesture, end moved to 3 ramps), so the bound is pinned here. + // The burst gain's window, read off the spline itself: it exists only from the stop's release start, one ramp (0.15 of the release window) + // before it and one after the stop's end. A gain held through the closure made the closing ramp a hiss (ClosingIntoAStopMakesNoHighFrequencyNoise). [Fact] - public void PlosiveBurstGainWindowIsOneRampEitherSide() + public void PlosiveBurstGainWindowIsOneRampEitherSideOfTheRelease() { IpaGestureExperimentTarget Phone(string label, string ipa, float seconds) => new($"w_{label}", ipa, "", seconds); var script = IpaGestureExperiment.BuildWordScript( [(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone("2_b", "b", 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f), (Phone("4_n", "n", 0.095f), 1f)], 140); var window = PatchScript.Parse(script).ControlSplines.Single(s => s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal) && s.Points.Any(p => p.Value > 0)); - const double start = 0.275, duration = 0.095, ramp = 0.15 * duration; - Assert.Equal((start - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4)); + const double start = 0.275, duration = 0.095; + var release = start + PatchScript.StopReleaseStartSeconds((float)duration); + var ramp = 0.15 * (start + duration - release); + Assert.Equal((release - ramp, start + duration + ramp), (window.Points.Min(p => (double)p.TimeSeconds), window.Points.Max(p => (double)p.TimeSeconds)), new TupleTolerance(1e-4)); + } + + // The contact opening, the tongue's constriction and the lips all leave their shut value at the instant the burst gain window opens: one + // owner, StopReleaseStartSeconds. Read off the splines, so a stale constant in one of them (the contact opening 20 ms early while the + // burst keeps the owner's time) fails here and not only through a closure level that happens to cross a bound. + [Theory] + [InlineData("b", "w_2_b")] + [InlineData("t", "w_2_t")] + [InlineData("k", "w_2_k")] + public void AStopsArticulatorsOpenWhereItsBurstGainWindowOpens(string ipa, string name) + { + IpaGestureExperimentTarget Phone(string label, string symbol, float seconds) => new($"w_{label}", symbol, "", seconds); + var script = IpaGestureExperiment.BuildWordScript( + [(Phone("0_n", "n", 0.095f), 1f), (Phone("1_a", "a", 0.18f), 1f), (Phone($"2_{ipa}", ipa, 0.095f), 1f), (Phone("3_a", "a", 0.18f), 1f)], 140); + var splines = PatchScript.Parse(script).ControlSplines; + const double start = 0.275, duration = 0.095; + var release = start + PatchScript.StopReleaseStartSeconds((float)duration); + var gain = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)); + var ramp = 0.15 * (start + duration - release); + Assert.Equal(release - ramp, gain.Points.Min(p => (double)p.TimeSeconds), 1e-4); + var paths = ipa == "b" + ? new[] { "/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter", "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" } + : ["/vocal/contacts/0/opening", "/vocal/areas/0/area/constriction_diameter"]; + foreach (var path in paths) + { + var spline = splines.Single(s => s.Name.StartsWith(name, StringComparison.Ordinal) && s.SurfacePath == path); + Assert.Equal(release, spline.Points.Where(p => p.TimeSeconds < start + duration - 1e-6).Max(p => (double)p.TimeSeconds), 1e-4); + } + } + + // The nasal keeps its murmur's small lip opening where a stop shuts the lips (0.04 cm against 0): the shut-lip term is the stop's only. + // Extending it to nasals moves /m/'s first nasal formant by 30 Hz and its murmur level by 0.4 dB and no render test notices, so the + // value is read off the splines. + [Theory] + [InlineData("m", 0.04f)] + [InlineData("b", 0f)] + public void OnlyAStopShutsTheLipsCompletely(string ipa, float shut) + { + var script = IpaGestureExperiment.BuildWordScript( + [(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.18f), 1f), (new IpaGestureExperimentTarget("w_1_c", ipa, "", 0.095f), 1f), (new IpaGestureExperimentTarget("w_2_a", "a", "", 0.18f), 1f)], 140); + foreach (var path in new[] { "/vocal/areas/0/area/lip_opening", "/vocal/radiation/0/aperture" }) + { + var spline = PatchScript.Parse(script).ControlSplines.Single(s => s.Name.StartsWith("w_1_c", StringComparison.Ordinal) && s.SurfacePath == path); + Assert.Equal(shut, spline.Points.Min(p => p.Value), 1e-4); + } } // /ɑ C ɑ/ at the scaffold's own durations (vowel 0.18 s, consonant 0.095 s). Returns the render and the consonant's [start, end). @@ -241,7 +294,8 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) private static SynthPatch WithoutBurst(SynthPatch p) => p with { ControlSplines = p.ControlSplines.Where(s => !s.SurfacePath.EndsWith("/release_flow", StringComparison.Ordinal)).ToArray() }; - // The closure of a stop is the part before its contact opens (0.65 of the gesture), 0.2 of the gesture in from its start, without the burst. + // The closure of a stop is the part after the boundary ramp into it and before its release, without the burst (the ramp is the tract still + // shutting on the vowel's decaying voicing; its noise is measured by ClosingIntoAStopMakesNoHighFrequencyNoise). // A tongue closure at the nasal's 0.03 cm aperture leaked: /t/ read -14.3 dB re the flanking vowels against /p/ -22.1 (the lips seal /p/); // completely closed it reads -24.3. [Fact] @@ -252,7 +306,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst); if (x is null) return double.NaN; var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2); - return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels); + return 20 * Math.Log10(Rms(x, ClosureFrom(from), ReleaseAt(from, to)) / vowels); } var p = Closure("p", "voiceless_bilabial_plosive"); @@ -269,7 +323,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var (nb, _, _) = VowelConsonantVowel("t", descriptor: "voiceless_alveolar_plosive", edit: WithoutBurst); if (x is null || nb is null) return; var burst = x.Zip(nb, (a, b) => a - b).ToArray(); - var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01); + var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01); var vowel = Peak(x, to + 0.06, to + 0.17); output.WriteLine($"/t/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}"); Assert.True(peak > 0.004, "audible"); @@ -288,7 +342,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) if (withBurst is null || withoutBurst is null) return; var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray(); const double tFrom = 0.275, tTo = 0.37; - var atRelease = Peak(burst, tFrom + 0.095 * 0.6, tTo + 0.02); + var atRelease = Peak(burst, ReleaseAt(tFrom, tTo) - 0.003, tTo + 0.02); var leading = Peak(burst, 0, tFrom); var trailing = Peak(burst, tTo + 0.12, 0.77); output.WriteLine($"/t/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it"); @@ -305,7 +359,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var (x, from, to) = VowelConsonantVowel("t"); var (nb, _, _) = VowelConsonantVowel("t", edit: WithoutBurst); if (x is null || nb is null) return; - var start = (int)((from + 0.65 * (to - from)) * Rate); + var start = (int)(ReleaseAt(from, to) * Rate); const int n = 960; var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray(); double Power(double hz) @@ -324,39 +378,44 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) } // /k/ over the /t/ tests: its own place (the velar distance in the place table), closure sealed by the tongue, burst under half the vowel. - // The spec asks /k/'s closure to be at or below /p/'s. It is not: with the voiceless source as /p/ and /t/ have it the closure reads - // -14.9 dB re the vowels (/p/ -20.4, /t/ -24.9). The tract itself seals: with the glottal source's flow scale held at 0 over the gesture the - // same closure reads -22.7 dB, under /p/; with the source on, the 150 Hz voicing is what reaches the output through a closure that leaves - // only a short rear cavity (the glottal pressure that the tract wall radiates is higher the smaller that cavity is). Silencing voicing in - // voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion is reported as unmet (fork), not met by a /k/ special case. - // The second assertion is a regression bound on the measured leak, not the criterion. + // The spec asks /k/'s closure to be at or below /p/'s. With the voiceless source as /p/ and /t/ have it, it is not, and was not on main + // either (-14.9 dB re the vowels, /p/ -20.4): the 150 Hz voicing reaches the output through a closure that leaves only a short rear cavity. + // Silencing voicing in voiceless stops would change /p/ and /t/ (pinned byte for byte), so the criterion stays unmet (fork). + // The measure is the source-on closure in its settled part (25 ms after the 30 ms closing ramp has finished, to 5 ms before the release), + // because the source-off closure cannot see a leak: with the source off nothing drives the tract, and leaks of .02, .05 and .1 of the + // diameter scale read -21.1, -21.2 and -21.2 dB over .21 to .263 s against -21.0 sealed, while the source-on closure reads 1.8, 1.8 and + // 4.1 dB louder in the settled part. The closure + // right after the ramp is no use either: the seal completes at the end of the ramp with the vowel's tail still in the rear cavity (the + // source-off /k/ read -21.0 dB over .21 to .263 s with the ramp and -27.5 with the contact and the constriction stepped shut; main's + // 30 ms-later window read -25.7), and that decays within the first 25 ms. + // The test therefore proves its own measure first (a leak injected into the tongue's closed diameter must read 3 dB louder) and then bounds + // the sealed closure (-16.5 measured; a .005, .03 or .035 leak reads -15.1, a .02 leak -14.7). [Fact] - public void AVelarStopClosureSealsAndItsLeakIsTheGlottalSource() + public void AVelarStopClosureSealsAndTheMeasureSeesALeak() { - static SynthPatch WithoutSource(SynthPatch p) => + static SynthPatch Leaking(SynthPatch p) => WithoutBurst(p) with { ControlSplines = WithoutBurst(p).ControlSplines - .Select(s => s.Name.StartsWith("w_1_c_source_level", StringComparison.Ordinal) - ? s with { Points = s.Points.Select(q => q with { Value = 0, OutValue = 0, InValue = 0 }).ToArray() } + .Select(s => s.Name == "w_1_c_constriction_diameter" + ? s with { Points = s.Points.Select(q => q.Value == s.Points.Min(v => v.Value) ? q with { Value = 0.1f, OutValue = 0, InValue = 0 } : q).ToArray() } : s).ToArray() }; - double Closure(string ipa, string descriptor, Func edit) + double Closure(Func edit) { - var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: edit); + var (x, from, to) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: edit); if (x is null) return double.NaN; var vowels = Math.Sqrt((Math.Pow(Rms(x, from - 0.08, from - 0.01), 2) + Math.Pow(Rms(x, to + 0.03, to + 0.12), 2)) / 2); - return 20 * Math.Log10(Rms(x, from + 0.2 * (to - from), from + 0.65 * (to - from)) / vowels); + return 20 * Math.Log10(Rms(x, ClosureFrom(from) + 0.025, ReleaseAt(from, to) - 0.005) / vowels); } - var p = Closure("p", "voiceless_bilabial_plosive", WithoutBurst); - var k = Closure("k", "voiceless_velar_plosive", WithoutBurst); - var kSilent = Closure("k", "voiceless_velar_plosive", WithoutSource); - output.WriteLine($"closure re vowels: /p/ {p:F1} dB, /k/ {k:F1} dB, /k/ with the glottal source off {kSilent:F1} dB"); - if (double.IsNaN(p)) return; - Assert.True(kSilent <= p, $"the /k/ closure seals: without the glottal source it is {kSilent:F1} dB, at or under /p/ ({p:F1} dB)"); - Assert.True(k <= -12, $"the /k/ closure with the source is {k:F1} dB (regression bound; the criterion, at or under /p/ {p:F1} dB, is not met)"); + var sealedClosure = Closure(WithoutBurst); + var leaking = Closure(Leaking); + output.WriteLine($"/k/ closure re vowels, source on, settled part: sealed {sealedClosure:F1} dB, with a 0.1 leak {leaking:F1} dB"); + if (double.IsNaN(sealedClosure)) return; + Assert.True(leaking >= sealedClosure + 3, $"the measure sees a leak: sealed {sealedClosure:F1} dB, leaking {leaking:F1} dB"); + Assert.True(sealedClosure <= -15.5, $"the /k/ closure seals: {sealedClosure:F1} dB re the vowels (measured -16.5; leaks of .005 to .1 read -15.1 to -12.4, the measure is not monotonic in the leak, so the bound sits under the weakest of them)"); } [Fact] @@ -366,7 +425,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var (nb, _, _) = VowelConsonantVowel("k", descriptor: "voiceless_velar_plosive", edit: WithoutBurst); if (x is null || nb is null) return; var burst = x.Zip(nb, (a, b) => a - b).ToArray(); - var peak = Peak(burst, from + 0.6 * (to - from), to + 0.01); + var peak = Peak(burst, ReleaseAt(from, to) - 0.003, to + 0.01); var vowel = Peak(x, to + 0.06, to + 0.17); output.WriteLine($"/k/ burst peak {peak:F4}, vowel peak {vowel:F4}, ratio {peak / vowel:F2}"); Assert.True(peak > 0.004, "audible"); @@ -384,7 +443,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) if (withBurst is null || withoutBurst is null) return; var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray(); const double kFrom = 0.275, kTo = 0.37; - var atRelease = Peak(burst, kFrom + 0.095 * 0.6, kTo + 0.02); + var atRelease = Peak(burst, ReleaseAt(kFrom, kTo) - 0.003, kTo + 0.02); var leading = Peak(burst, 0, kFrom); var trailing = Peak(burst, kTo + 0.12, 0.77); output.WriteLine($"/k/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it"); @@ -402,7 +461,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) var (x, from, to) = VowelConsonantVowel(ipa, descriptor: descriptor); var (nb, _, _) = VowelConsonantVowel(ipa, descriptor: descriptor, edit: WithoutBurst); if (x is null || nb is null) return (double.NaN, double.NaN); - var start = (int)((from + 0.65 * (to - from)) * Rate); + var start = (int)(ReleaseAt(from, to) * Rate); const int n = 960; var seg = Enumerable.Range(0, n).Select(i => (x[start + i] - nb[start + i]) * (0.5 - 0.5 * Math.Cos(2 * Math.PI * i / (n - 1)))).ToArray(); double Power(double hz) @@ -468,7 +527,7 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output) if (withBurst is null || withoutBurst is null) return; var burst = withBurst.Zip(withoutBurst, (x, y) => x - y).ToArray(); const double dFrom = 0.275, dTo = 0.37; - var atRelease = Peak(burst, dFrom + 0.095 * 0.6, dTo + 0.02); + var atRelease = Peak(burst, ReleaseAt(dFrom, dTo) - 0.003, dTo + 0.02); var leading = Peak(burst, 0, dFrom); var trailing = Peak(burst, dTo + 0.12, 0.77); output.WriteLine($"/d/ burst peak {atRelease:F4} at the release, {leading:F5} before the plosive, {trailing:F5} after it"); diff --git a/tests/AquaSynth.Dsl.Tests/WeksaPhonesTests.cs b/tests/AquaSynth.Dsl.Tests/WeksaPhonesTests.cs index 3f0994f..02e7ccb 100644 --- a/tests/AquaSynth.Dsl.Tests/WeksaPhonesTests.cs +++ b/tests/AquaSynth.Dsl.Tests/WeksaPhonesTests.cs @@ -148,10 +148,10 @@ public sealed class WeksaPhonesTests Assert.Equal([WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds * WeksaWordPlan.CueFor(WeksaStress.Primary).Duration, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.ConsonantSeconds, WeksaWordPlan.VowelSeconds], plan.Phones.Select(p => p.Target.DurationSeconds)); var patch = PatchScript.Parse(IpaGestureExperiment.BuildWordScript(plan.Phones.ToArray(), 140)); - var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value); + var ih = patch.ControlSplines.Where(s => s.Name.StartsWith("x_1_ɪ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value); Assert.Equal(1f, ih["/vocal/areas/0/area/vowel_ih"]); Assert.Equal(0f, ih["/vocal/areas/0/area/vowel_i"]); - var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[0].Value); + var oo = patch.ControlSplines.Where(s => s.Name.StartsWith("x_4_ʊ_vowel_", StringComparison.Ordinal)).ToDictionary(s => s.SurfacePath, s => s.Points[^1].Value); Assert.Equal(1f, oo["/vocal/areas/0/area/vowel_oo_short"]); Assert.Equal(0f, oo["/vocal/areas/0/area/vowel_u"]); } @@ -190,7 +190,7 @@ public sealed class WeksaPhonesTests var none = patch(WeksaStress.None); var primary = patch(WeksaStress.Primary); float First(SynthPatch p, string prefix, string surface) => - p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[0].Value; + p.ControlSplines.Single(sp => sp.Name.StartsWith(prefix, StringComparison.Ordinal) && sp.SurfacePath == surface).Points[^1].Value; // pitch_ratio surface range 0.5..2: normalized (1.06 - .5) / 1.5; unstressed holds 1 -> (1 - .5) / 1.5. Assert.Equal((1.06f - .5f) / 1.5f, First(primary, "w_1_i_source_pitch", "/vocal/sources/0/pitch_ratio"), 4); diff --git a/tools/aqua-feel/lowered-dings/glass-chime.dsp b/tools/aqua-feel/lowered-dings/glass-chime.dsp index 16d49a0..e77b4bd 100644 --- a/tools/aqua-feel/lowered-dings/glass-chime.dsp +++ b/tools/aqua-feel/lowered-dings/glass-chime.dsp @@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d)); bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d; seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e); smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x)); +smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); }; +smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x); seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d)); seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d))); seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b)); diff --git a/tools/aqua-feel/lowered-dings/muted-pluck.dsp b/tools/aqua-feel/lowered-dings/muted-pluck.dsp index 0fc3e02..c1205aa 100644 --- a/tools/aqua-feel/lowered-dings/muted-pluck.dsp +++ b/tools/aqua-feel/lowered-dings/muted-pluck.dsp @@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d)); bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d; seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e); smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x)); +smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); }; +smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x); seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d)); seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d))); seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b)); diff --git a/tools/aqua-feel/lowered-dings/soft-triangle.dsp b/tools/aqua-feel/lowered-dings/soft-triangle.dsp index 154b917..9b82afc 100644 --- a/tools/aqua-feel/lowered-dings/soft-triangle.dsp +++ b/tools/aqua-feel/lowered-dings/soft-triangle.dsp @@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d)); bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d; seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e); smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x)); +smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); }; +smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x); seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d)); seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d))); seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b)); diff --git a/tools/aqua-feel/lowered-dings/warm-bell.dsp b/tools/aqua-feel/lowered-dings/warm-bell.dsp index dd8fc08..8acf9bd 100644 --- a/tools/aqua-feel/lowered-dings/warm-bell.dsp +++ b/tools/aqua-feel/lowered-dings/warm-bell.dsp @@ -16,6 +16,8 @@ seg(t,t0,d,a,b) = a + (b - a) * clip01((t - t0) / max(0.0001, d)); bez(t,a,b,c,d) = (1.0 - t) * (1.0 - t) * (1.0 - t) * a + 3.0 * (1.0 - t) * (1.0 - t) * t * b + 3.0 * (1.0 - t) * t * t * c + t * t * t * d; seg_bez(t,t0,d,a,b,c,e) = bez(clip01((t - t0) / max(0.0001, d)), a, b, c, e); smooth01(x) = clip01(x) * clip01(x) * (3.0 - 2.0 * clip01(x)); +smoo_from(pole, x) = loop ~ _ with { loop(prev) = select2(1 - 1', prev * pole + x * (1.0 - pole), x); }; +smoo_init(x) = smoo_from(1.0 - 44.1 / ma.SR, x); seg_smooth(t,t0,d,a,b) = a + (b - a) * smooth01((t - t0) / max(0.0001, d)); seg_exp(t,t0,d,a,b) = exp(log(max(0.00001, a)) + (log(max(0.00001, b)) - log(max(0.00001, a))) * clip01((t - t0) / max(0.0001, d))); seg_curve(c,t,t0,d,a,b) = select2(c < 0.5, seg_exp(t,t0,d,a,b), seg(t,t0,d,a,b)); diff --git a/tools/formant-check/ear_metrics.py b/tools/formant-check/ear_metrics.py new file mode 100644 index 0000000..bbaa65f --- /dev/null +++ b/tools/formant-check/ear_metrics.py @@ -0,0 +1,104 @@ +#!/usr/bin/env python3 +"""What the formant numbers do not say. Measures a render for the faults Meta heard in tract-weak-spots-1 (2026-11-13) while every +formant matched its target; it does not listen, and a pass here is not a verdict (the listening page is). + ear_metrics.py FILE.wav [--f0 140] [--steady A B] [--json] + +click_ratio largest pitch-period-window peak over the median window peak. A voiced word with no transient is near 1; the pre-fix + nibu (a phone boundary stepping an area) was 3.54 with its window at the /b/ closure (0.271 s); after, 1.76 (a vowel's own pulses). +step_db largest single-sample step, in dB over the median sample step of the voiced part. A cell area that jumps is one sample wide. +bandwidths LPC pole bandwidths (Hz) of the steady part, F1..F3 as (Hz, bandwidth). Natural vowels run about 40-100 Hz at F1, 50-120 at + F2, 80-200 at F3 (Hawks & Miller 1995, Fant 1972); a lossless tube rings narrower, which reads as reedy. +hnr_db harmonics-to-noise ratio of the steady part by autocorrelation at the pitch lag. +hf_db energy above 4 kHz re the total, dB. +""" +import sys, wave, json, argparse +import numpy as np + + +def wav(path): + with wave.open(path) as w: + return np.frombuffer(w.readframes(w.getnframes()), dtype=" peaks.max() * 0.02] + if len(voiced) < 4: + return 0.0, 0.0 + at = int(np.argmax(peaks)) * (p // 2) / sr + return float(peaks.max() / np.median(voiced)), at + + +def step_db(x): + d = np.abs(np.diff(x)) + live = d[d > d.max() * 1e-3] + return float(20 * np.log10(d.max() / max(np.median(live), 1e-12))) if len(live) else 0.0 + + +def autocorr(s, n=None): + """Autocorrelation at lags 0..n-1 by FFT (np.correlate is quadratic).""" + n = n or len(s) + f = np.fft.rfft(s, 2 * len(s)) + return np.fft.irfft(f * np.conj(f))[:n] + + +def lpc(s, o): + s = s * np.hamming(len(s)) + r = autocorr(s, o + 1) + R = np.array([[r[abs(i - j)] for j in range(o)] for i in range(o)]) + a = np.linalg.solve(R + 1e-9 * r[0] * np.eye(o), r[1:o + 1]) + return np.concatenate([[1], -a]) + + +def bandwidths(y, sr, order=14): + dec = max(1, int(round(sr / 10000))) + k = np.hanning(4 * dec + 1) + k /= k.sum() + z = np.convolve(y - y.mean(), k, "same")[::dec] + fs = sr / dec + z = np.append(z[0], z[1:] - 0.97 * z[:-1]) + rt = np.roots(lpc(z, order)) + rt = rt[np.imag(rt) > 0.001] + fq = np.angle(rt) * fs / (2 * np.pi) + bw = -np.log(np.abs(rt)) * fs / np.pi + # a formant is a narrow pole; broad poles (> 400 Hz) are the spectral tilt the all-pole fit spends order on + return [(int(f), int(b)) for f, b in sorted(zip(fq, bw)) if 150 < f < 4500 and b < 400][:3] + + +def hnr_db(y, sr, f0): + y = y - y.mean() + ac = autocorr(y) + ac /= ac[0] + lo, hi = int(sr / (f0 * 1.3)), int(sr / (f0 * 0.75)) + m = float(ac[lo:hi].max()) + return 10 * np.log10(max(m, 1e-6) / max(1 - m, 1e-6)) + + +def measure(path, f0=140.0, steady=None): + x, sr = wav(path) + a, b = steady if steady else (len(x) / sr * 0.35, len(x) / sr * 0.85) + y = x[int(a * sr):int(b * sr)] + sp = np.abs(np.fft.rfft((y - y.mean()) * np.hanning(len(y)))) + fr = np.fft.rfftfreq(len(y), 1 / sr) + cr, at = click_ratio(x, sr, f0) + return { + "file": path.split("/")[-1], "click_ratio": round(cr, 2), "click_at_s": round(at, 4), "step_db": round(step_db(x), 1), + "bandwidths": bandwidths(y, sr), "hnr_db": round(hnr_db(y, sr, f0), 1), + "hf_db": round(float(10 * np.log10(max((sp[fr > 4000] ** 2).sum(), 1e-20) / (sp ** 2).sum())), 1), + } + + +if __name__ == "__main__": + ap = argparse.ArgumentParser() + ap.add_argument("files", nargs="+") + ap.add_argument("--f0", type=float, default=140.0) + ap.add_argument("--steady", nargs=2, type=float) + ap.add_argument("--json", action="store_true") + a = ap.parse_args() + rows = [measure(f, a.f0, a.steady) for f in a.files] + if a.json: + print(json.dumps(rows, indent=1)) + else: + for r in rows: + print(f"{r['file']:20s} click {r['click_ratio']:5.2f} at {r['click_at_s']:.3f}s step {r['step_db']:5.1f} dB hnr {r['hnr_db']:5.1f} hf {r['hf_db']:6.1f} F(Hz,bw) {r['bandwidths']}") diff --git a/tools/formant-check/phone_probe.py b/tools/formant-check/phone_probe.py new file mode 100644 index 0000000..ddd6a75 --- /dev/null +++ b/tools/formant-check/phone_probe.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +"""A machine ear for the tract renders: what phone string a speech recogniser reads out of a clip. A PROXY for 'what was heard', never a +verdict: it is trained on natural speech and reads the vowels of this tract poorly (every isolated vowel came back as /i/), so use it +for the events it separates well (a burst read as /v/ against /b/, a nasal present or missing, a consonant that is only a click). +It reproduced three of Meta's four first listening reports on tract-weak-spots-1 before any fix: ba with burst -> 'v ɑ' (she heard +'fah then bah'), ana -> 'v ɑ m ɑ' (a basketball buzzer, no articulation), nibu -> 'i b u' (both consonants clicks, the /n/ missing). + phone_probe.py CLIP.wav... +Needs torch, transformers, scipy and the HuggingFace model facebook/wav2vec2-lv-60-espeak-cv-ft (about 1.2 GB, downloaded once into +HF_HOME). The CTC decode reads vocab.json directly, so the phonemizer library is not needed. +""" +import sys, wave, json +import numpy as np +import torch +from scipy.signal import resample_poly +from transformers import Wav2Vec2ForCTC +from huggingface_hub import hf_hub_download + +MODEL = "facebook/wav2vec2-lv-60-espeak-cv-ft" + + +def phones(model, vocab, path): + with wave.open(path) as w: + sr = w.getframerate() + x = np.frombuffer(w.readframes(w.getnframes()), dtype=" 1e-9)), + "burst_peak_from_120ms_after_release_to_end": round(peak(d, sr, c1 + .12, n / sr), 4), "burst_diff_samples_nonzero_before_closure_end": int(np.count_nonzero(np.abs(d[:int(rel * sr)]) > 1e-9)), "file_end_rms_last_100ms": round(rms(x, sr, len(x) / sr - .1, len(x) / sr), 5), "vowel_rms": round(rms(x, sr, c1 + .06, c1 + .17), 4)} BURST_BANDS = ((0, 1000), (1000, 2000), (2000, 3500), (3500, 7000), (7000, 24000))