Render verification: independent Burg/tube-chain table, stress and burst measurements, amplitude-cue test
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Aqua
parent
563a6a8e1f
commit
5c32bf6453
3 changed files with 149 additions and 0 deletions
|
|
@ -72,3 +72,45 @@ Dang & Honda 1994 text; per-consonant nasal zero ranges; Fry and Lieberman magni
|
|||
/b/ voice-bar level; Liljencrants 1985; the JOS book tract chapters. The Story 2008 table came through a page summariser, not a PDF
|
||||
text extraction; every column was checked by computing its formants (all eleven give vowel-typical F1/F2 orderings), which
|
||||
catches a gross transcription error, not a small one.
|
||||
|
||||
## Render verification, stress cue and plosive burst (2026-11-08, unheard)
|
||||
|
||||
Renders: `WordRender` at f0 140 Hz, one 180 ms vowel per file (`v_<id>`), plus /ɑmɑ/, /ɑnɑ/, /ˈnibu/ with and without primary stress, /bɪn/, /bʊn/.
|
||||
Estimator: `tools/formant-check/verify_renders.py` over `burg_formants.py` (Burg LPC order 12 at 10 kHz, 30 ms frames, vowel window 70-150 ms), a separate
|
||||
Python process that shares no code with the C# tests or the ladder; references are Hillenbrand 1995 and Peterson & Barney 1952 male means (above) and the
|
||||
tube chain at 17.52 cm (`tube_formants.py`). Files: `/opt/commons/handoff/aqua-tract-articulatory-20261108` (wav/, scripts/, manifest.json, verify.txt, verify.json).
|
||||
|
||||
| vowel | Burg F1/F2 | tube chain F1/F2 | render vs tube F1/F2 % | Hillenbrand F1/F2 | render vs Hillenbrand % |
|
||||
|---|---|---|---|---|---|
|
||||
| i | 290/2111 | 251/2063 | +15.5/+2.3 | 343/2323 | -15.4/-9.1 |
|
||||
| ɪ | 374/2032 | 346/1987 | +8.2/+2.3 | 429/2034 | -12.7/-0.1 |
|
||||
| e | 439/1929 | 432/1827 | +1.6/+5.6 | 476/2090 | -7.8/-7.7 |
|
||||
| ɛ | 553/1602 | 527/1571 | +4.9/+2.0 | 588/1803 | -6.0/-11.1 |
|
||||
| æ | 718/1558 | 729/1523 | -1.5/+2.3 | 591/1930 | +21.5/-19.3 |
|
||||
| ʌ | 573/811 | 549/809 | +4.4/+0.3 | 621/1181 | -7.7/-31.3 |
|
||||
| ɑ | 646/949 | 650/913 | -0.6/+3.9 | 757/1309 | -14.7/-27.5 |
|
||||
| ɔ | 523/718 | 516/692 | +1.4/+3.8 | 656/1023 | -20.3/-29.8 |
|
||||
| o | 555/819 | 477/787 | +16.4/+4.1 | 498/910 | +11.4/-10.0 |
|
||||
| ʊ | 520/897 | 478/850 | +8.8/+5.5 | 469/1123 | +10.9/-20.1 |
|
||||
| u | 300/816 | 271/761 | +10.7/+7.2 | 380/992 | -21.1/-17.7 |
|
||||
|
||||
Against its own area function the render sits within 6 percent for e ɛ æ ʌ ɑ ɔ and F2 of all eleven within 7.2 percent; F1 of i, o, u and ʊ reads 9-16 percent high.
|
||||
The F1 excess is the ladder's known high bias (above) plus the LPC's resolution with a 140 Hz line spacing for F1 near 270-300 Hz: not separated. Against the population
|
||||
means the render is 8-30 percent off for the back vowels' F2, which is the one-speaker gap described above, not a defect test.
|
||||
|
||||
Nasal: /ɑmɑ/ murmur Burg N1 220 Hz (F2 989), /ɑnɑ/ murmur 285 Hz (F2 1041), inside Fujimura's low-N1 range; murmur rms re the second vowel -7.4 dB (m) and -2.7 dB (n). A real murmur
|
||||
is weaker than that (no source retrieved for the level); the /n/ is too loud. Open.
|
||||
|
||||
Stress (cue values in `WeksaWordPlan.CueFor`): primary stress renders longer by 75 ms for the vowel (design 68 ms), f0 148.1 against 139.9 Hz (ratio 1.059; design 1.06), level +0.93 dB at the end of the vowel
|
||||
(design +2 dB). The shortfall is not the amplitude path: `IntensityIsAnAmplitudeFactorOnTheRenderedLevel` renders /ɑ/ at intensity 1.26 against 1 and finds the 2 dB; in /i/ with F1 near 290 Hz,
|
||||
raising f0 by 6 percent moves the harmonic nearest F1 and costs about one dB of the gain.
|
||||
|
||||
Plosive: /bɪn/ closure peak 0.0000 (voice bar below 16-bit resolution here), release 60-95 ms peak 0.0625, vowel 0.126 (-6 dB). The burst itself (difference of renders with and without `release_flow`)
|
||||
peaks at about 0.0076, -24 dB re the vowel peak; the release peak in the table above is the voicing breaking through as the contact opens. Burst gain 0.06 is a DESIGN VALUE.
|
||||
|
||||
Weak spots found in the renders: (1) the vowel level is not steady, it rises for about 100 ms after onset (ɑ: rms 0.028 at 80 ms to 0.079 at 160 ms) so any window early in a vowel
|
||||
reads low; the cause is not isolated (surface smoothing and the pressure ramp are the suspects). (2) /n/ murmur louder than a nasal murmur should be. (3) Closure is silent, not a voice bar.
|
||||
(4) Only f0 and level of the first vowel are checked per cue; the cues' perceptual strength is unknown. Everything here is a measurement, nothing was heard.
|
||||
|
||||
Mutations (hand-applied, restored, each run against the named tests): pitch ratio ignored, burst removed, burst not shaped by opening, burst gain x20 (0.06 to 1.2), stress duration ignored,
|
||||
stress pitch ignored, stress amplitude ignored, secondary equal to primary, probe timeline sample rate re-hardcoded to 48000: nine of nine killed. Script: `work/mut-b3.sh` in the aqua workspace.
|
||||
|
|
|
|||
|
|
@ -74,6 +74,24 @@ public sealed class StressAndReleaseRenderTests(ITestOutputHelper output)
|
|||
Assert.InRange(up, 163, 167);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void IntensityIsAnAmplitudeFactorOnTheRenderedLevel()
|
||||
{
|
||||
double Level(float intensity)
|
||||
{
|
||||
var script = IpaGestureExperiment.BuildWordScript([(new IpaGestureExperimentTarget("w_0_a", "a", "", 0.5f), intensity)], 110);
|
||||
var x = Render(script, 0.6f);
|
||||
return x is null ? double.NaN : Rms(x, 0.3, 0.5);
|
||||
}
|
||||
|
||||
var unity = Level(1f);
|
||||
var stressed = Level(1.26f);
|
||||
if (double.IsNaN(unity)) return;
|
||||
var db = 20 * Math.Log10(stressed / unity);
|
||||
output.WriteLine($"level at intensity 1.26 against 1: {db:F2} dB (factor wanted {20 * Math.Log10(1.26):F2})");
|
||||
Assert.InRange(db, 1.7, 2.3);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void PlosiveReleaseBurstIsAboveTheClosureAndBelowTheFollowingVowel()
|
||||
{
|
||||
|
|
|
|||
89
tools/formant-check/verify_renders.py
Normal file
89
tools/formant-check/verify_renders.py
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Independent check of the WordRender outputs against published formants and the stress/burst cues. Shares no code with the C#
|
||||
tests or the ladder; formants come from burg_formants.py (Burg LPC), f0 from autocorrelation, levels from the WAV.
|
||||
verify_renders.py RENDER_DIR [--json OUT.json]
|
||||
RENDER_DIR holds one subdirectory per word (v_<id>, ama, ana, nibu, nibu_flat, bin, bun) as written by the WordRender runs of
|
||||
work/b3-render-all.sh. Prints a table: per vowel Burg F1/F2/F3 at the vowel's steady part against Hillenbrand 1995 male means and
|
||||
Peterson & Barney 1952 male means (docs/research/articulatory-vocal-tract-sources.md) and the Story tube-chain prediction
|
||||
(tube_formants.py, 17.52 cm), then the nasal murmur, the stress cue and the b burst. It measures; it does not listen.
|
||||
"""
|
||||
import sys, os, json, wave, subprocess, argparse
|
||||
import numpy as np
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
CSV = os.path.join(HERE, "..", "..", "docs", "research", "data", "story2008-area-functions.csv")
|
||||
# Male means, F1/F2/F3 Hz (docs/research/articulatory-vocal-tract-sources.md).
|
||||
HILL = {"i": (343, 2323, 3001), "ih": (429, 2034, 2687), "e": (476, 2090, 2692), "eh": (588, 1803, 2604), "ae": (591, 1930, 2595),
|
||||
"uh": (621, 1181, 2548), "ah": (757, 1309, 2535), "aw": (656, 1023, 2521), "o": (498, 910, 2459), "oo_short": (469, 1123, 2435), "u": (380, 992, 2355)}
|
||||
PB = {"i": (267, 2294, 2937), "ih": (392, 1993, 2569), "eh": (526, 1854, 2481), "ae": (664, 1727, 2420), "uh": (631, 1192, 2377),
|
||||
"ah": (718, 1091, 2442), "aw": (568, 836, 2403), "oo_short": (437, 1023, 2245), "u": (307, 876, 2239)}
|
||||
STORY_IDS = {"i": "i", "ih": "ɪ", "e": "e", "eh": "ɛ", "ae": "æ", "uh": "ʌ", "ah": "ɑ", "aw": "ɔ", "o": "o", "oo_short": "ʊ", "u": "u"}
|
||||
|
||||
def wav(path):
|
||||
with wave.open(path) as w:
|
||||
x = np.frombuffer(w.readframes(w.getnframes()), dtype="<i2").astype(float) / 32768.0
|
||||
return x, w.getframerate()
|
||||
|
||||
def burg(path, t0, t1):
|
||||
out = subprocess.run([sys.executable, os.path.join(HERE, "burg_formants.py"), path, str(t0), str(t1), "--json"], capture_output=True, text=True, check=True).stdout
|
||||
j = json.loads(out)
|
||||
return [j[f"F{i}"]["median"] if j[f"F{i}"] else None for i in (1, 2, 3)]
|
||||
|
||||
def tube(vowel):
|
||||
out = subprocess.run([sys.executable, os.path.join(HERE, "tube_formants.py"), CSV, "--vowel", STORY_IDS[vowel], "--length-cm", "17.52"], capture_output=True, text=True).stdout
|
||||
nums = [float(tok) for line in out.splitlines() for tok in line.replace("=", " ").replace("Hz", " ").split() if tok.replace(".", "", 1).isdigit()]
|
||||
return out.strip().splitlines()[-1] if out.strip() else "?"
|
||||
|
||||
def rms(x, sr, a, b):
|
||||
s = x[int(a * sr):int(b * sr)]
|
||||
return float(np.sqrt((s ** 2).mean())) if len(s) else 0.0
|
||||
|
||||
def f0(x, sr, a, b):
|
||||
s = x[int(a * sr):int(b * sr)]; s = s - s.mean()
|
||||
lo, hi = sr // 400, sr // 60
|
||||
c = [np.dot(s[:-l], s[l:]) for l in range(lo, hi)]
|
||||
return sr / (lo + int(np.argmax(c)))
|
||||
|
||||
def pct(v, ref): return None if v is None else round(100.0 * (v - ref) / ref, 1)
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(); ap.add_argument("dir"); ap.add_argument("--json"); a = ap.parse_args()
|
||||
res = {"vowels": {}}
|
||||
print("vowel Burg F1/F2/F3 (window) | Hillenbrand F1/F2 err% | P&B F1/F2 err% | tube chain")
|
||||
for v in HILL:
|
||||
p = os.path.join(a.dir, f"v_{v}", f"v_{v}.wav")
|
||||
if not os.path.exists(p): print(v, "MISSING"); continue
|
||||
f = burg(p, 0.07, 0.15) # 180 ms vowel; the onset ramp is excluded
|
||||
h = HILL[v]; pb = PB.get(v)
|
||||
res["vowels"][v] = {"burg": f, "hillenbrand": h, "pb": pb, "err_hill_pct": [pct(f[0], h[0]), pct(f[1], h[1])], "err_pb_pct": [pct(f[0], pb[0]), pct(f[1], pb[1])] if pb else None}
|
||||
print(f"{v:9s}{f} | {h[:2]} {res['vowels'][v]['err_hill_pct']} | {pb[:2] if pb else '-'} {res['vowels'][v]['err_pb_pct'] if pb else ''} | {tube(v)}")
|
||||
# nasal murmur: low first formant and weaker level than the flanking vowels
|
||||
for w, c in (("ama", "m"), ("ana", "n")):
|
||||
p = os.path.join(a.dir, w, f"{w}.wav")
|
||||
if not os.path.exists(p): continue
|
||||
x, sr = wav(p)
|
||||
seg = [(0, .18), (.18, .275), (.275, .455)]
|
||||
r = [rms(x, sr, *s) for s in seg]
|
||||
res[w] = {"rms": r, "murmur_db_re_vowel": round(20 * np.log10(r[1] / max(r[0], 1e-9)), 1), "burg_murmur": burg(p, .20, .26)}
|
||||
print(f"{w}: rms V/{c}/V {['%.4f' % q for q in r]} murmur {res[w]['murmur_db_re_vowel']} dB re first vowel; Burg of the murmur {res[w]['burg_murmur']}")
|
||||
# stress
|
||||
p0, p1 = os.path.join(a.dir, "nibu_flat", "nibu_flat.wav"), os.path.join(a.dir, "nibu", "nibu.wav")
|
||||
if os.path.exists(p0) and os.path.exists(p1):
|
||||
x0, sr = wav(p0); x1, _ = wav(p1)
|
||||
d = 0.18 * 220 / 160 - 0.18
|
||||
# vowel i spans 0.095-0.275 (flat) / 0.095-0.3225 (primary); compare steady parts
|
||||
rf, rp = rms(x0, sr, .15, .25), rms(x1, sr, .15, .25)
|
||||
ff, fp = f0(x0, sr, .15, .25), f0(x1, sr, .15, .25)
|
||||
res["stress"] = {"rms_flat": rf, "rms_primary": rp, "level_db": round(20 * np.log10(rp / rf), 2), "f0_flat": ff, "f0_primary": fp, "f0_ratio": round(fp / ff, 3),
|
||||
"length_flat_s": len(x0) / sr, "length_primary_s": len(x1) / sr, "length_diff_s": round(len(x1) / sr - len(x0) / sr, 3), "expected_diff_s": round(d, 3)}
|
||||
print("stress:", res["stress"])
|
||||
# b release in bin: closure/ burst / vowel
|
||||
p = os.path.join(a.dir, "bin", "bin.wav")
|
||||
if os.path.exists(p):
|
||||
x, sr = wav(p)
|
||||
step = int(sr * 0.002); env = [(i / sr, float(np.abs(x[i:i + step]).max())) for i in range(0, len(x) - step, step)]
|
||||
res["bin_env_peak_per_2ms"] = [(round(t * 1000), round(v, 4)) for t, v in env[:70]]
|
||||
pk = max(v for _, v in env)
|
||||
print("bin: peak", round(pk, 4), "b closure (0-60 ms) max", round(max(v for t, v in env if t < .06), 4), "release 60-95 ms max", round(max(v for t, v in env if .06 <= t < .1), 4), "vowel 110-260 ms max", round(max(v for t, v in env if .11 <= t < .26), 4))
|
||||
if a.json: json.dump(res, open(a.json, "w"), indent=1, default=float)
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
Loading…
Reference in a new issue