From 080f236ef25ddb2f444249f282b33d46f7555e65 Mon Sep 17 00:00:00 2001 From: Aqua Date: Sun, 8 Nov 2026 16:13:29 +0000 Subject: [PATCH] Articulatory tract sources, Story 2008 vowel area functions as data, independent formant estimators docs/research/articulatory-vocal-tract-sources.md cites what the tract design takes from Story, Pink Trombone, VocalTractLab, Maeda, Kelly-Lochbaum, nasal coupling and LF, with what could not be retrieved. The CSV is Story 2008 Table I (11 vowels, 44 sections). burg_formants.py (Burg LPC on a WAV) and tube_formants.py (frequency-domain tube chain on an area function) share no code with the ladder. Unheard. Co-Authored-By: Claude Sonnet 5.5 --- .../articulatory-vocal-tract-sources.md | 47 ++++++++++++ .../data/story2008-area-functions.csv | 49 ++++++++++++ tools/formant-check/burg_formants.py | 73 ++++++++++++++++++ tools/formant-check/tube_formants.py | 76 +++++++++++++++++++ 4 files changed, 245 insertions(+) create mode 100644 docs/research/articulatory-vocal-tract-sources.md create mode 100644 docs/research/data/story2008-area-functions.csv create mode 100644 tools/formant-check/burg_formants.py create mode 100644 tools/formant-check/tube_formants.py diff --git a/docs/research/articulatory-vocal-tract-sources.md b/docs/research/articulatory-vocal-tract-sources.md new file mode 100644 index 0000000..81068d6 --- /dev/null +++ b/docs/research/articulatory-vocal-tract-sources.md @@ -0,0 +1,47 @@ +# Articulatory vocal tract: sources and what AquaSynth takes from them + +Read 2026-11-08 for follow-up `aqua-body:follow_up:chat-s4456-4456`. The full fact sheet with quoted values and the list of +what could not be retrieved is the Eyes pass `tract-literature-20261108` (research notes in the aqua mind; +working copy `work/tract-literature-facts.md` in the aqua workspace). This page keeps only what the tract design rests on. +Nothing built from it has been heard by anyone; every number below is a measurement or a published value, not a listening result. + +## Sources and what each decides + +| Source | What we take | What we do not take | +|---|---|---| +| Kelly & Lochbaum 1962; Smith, *Physical Audio Signal Processing* (KL junctions); Liljencrants 1985 / Story 1995 (section length `c / 2Fs`, i.e. one sample per section); Mullen, Howard & Murphy 2006 (boundary reflections, wall loss 0.90-0.98 sets bandwidth) | The ladder: one cell per sample, junction `r = (A[i-1]-A[i])/(A[i-1]+A[i])`, glottal reflection about +0.75, lip reflection about -0.85, a per-pass loss factor for bandwidth | Fractional-delay cells (Välimäki & Karjalainen, Mathur & Story): needed for continuous length change, not built; the tract length is fixed per patch | +| Pink Trombone (Neil Thapen 2017, MIT; read in chdh/pink-trombone-mod `Tract.ts`) | Constants (glottal 0.75, lip -0.85, 0.999 per pass), the 3-port velum junction `rL=(2A_k-S)/S`, `rR`, `rN`, with `S = A_k + A_(k+1) + A_velum` | Its 44-cell tract at twice the sample rate and its Gaussian tongue model as the vowel source: the vowel shapes now come from measured area functions | +| Story, Titze & Hoffman 1996/1998 (MRI area functions; numeric tables paywalled) and **Story 2008, JASA 123(1):327-335, PMC2377017, Table I** (44-section area functions of 11 vowels incl. /ɪ/ /ʊ/, one adult male, 2002 scan) | The 11 vowel area functions, as data: `docs/research/data/story2008-area-functions.csv` | Story's 1994 set (plot only). The sizes are one speaker's, not a population mean (see the measured gap below) | +| Story 2005, JASA 117(5):3231-3254 (area function model: vowel substrate `(pi/4)[Omega + q1 phi1 + q2 phi2]^2`, consonant constriction `C(i) = 1 - exp(-ln16((i-lc)/rc)^2)`, nasal port at about section 22) | The principle: a vowel is a point in a low-dimensional space of whole area functions, blended in diameter space, with a consonant constriction laid on top of it. We blend the 11 measured shapes by weights instead of PCA modes (Omega, phi1, phi2 are in the paper's Table III; not transcribed) | Its 14-parameter control set | +| VocalTractLab (Birkholz; GPL; 2.4 manual, 2005 dissertation) | Nasal cavity area function (Dang & Honda subject, 19 sections x 0.6 cm = 11.4 cm, 0.76-4.24 cm^2); the observation that nasal antiresonances come from the oral cavity as a side branch of the open velum; c = 350 m/s; the wall-loss and radiation (R||L) constants as the reference for the independent check | Its 3D articulators and glottis models; the four paranasal sinus Helmholtz resonators (zeros at 552-1682 Hz) and piriform fossae are not modelled | +| Maeda 1979/1990 (seven jaw/tongue/lip/larynx parameters from cineradiographs; `A = alpha d^beta`) | Structure only: the argument for articulator-level parameters over formant knobs | Per-vowel parameter values (not retrievable); no Maeda parameterisation is built | +| Nasal coupling: Fujimura 1962 (nasal murmur: very low N1 near 250-400 Hz, zeros), Dang & Honda 1994/1995, Story 2005 tier IV | A velum that opens the 3-port junction; the murmur's low first formant and the oral side-branch zero as the measured behaviour to look for | Per-consonant zero ranges for /m n ŋ/: no readable primary source | +| LF glottal flow (Fant, Liljencrants & Lin 1985; Fant 1995 `Rd`; Gobl 2021; Wang & Gobl 2023 regressions `Rkp = 0.118 Rd + 0.224`, `Rap = 0.048 Rd - 0.01`) | Reference for the source spectrum: Rd is a tension axis (low = tense), tilt cutoff falls as Rd rises. The current source is a band-limited saw through a one-pole lowpass whose cutoff is the `tilt` control: it matches the LF pulse's spectral tilt only qualitatively | An LF pulse generator: not built. If the source is moved to LF, `Rd` replaces `tilt` | +| Hillenbrand et al. 1995, JASA 97(5):3099-3111 (male means computed from the public data of github.com/santiagobarreda/hillenbrand_et_al_1995, MIT) and Peterson & Barney 1952 (means from phonTools `pb52`) | The reference formants F1/F2/F3 per vowel for the verification, table below | Table V itself was not read; P&B vowel labels are inferred from the data codes | +| Stress: Sluijter & van Heuven 1996 (ICSLP table + JASA abstract); Fry 1955/58, Lieberman 1960 (secondary reports only) | Stressed syllables are longer (stressed 220 ms against 160 ms unstressed in the initial-stress pairs; a stressed/unstressed ratio of 1.68 in a native-speaker study), 1-3 dB louder overall, f0 higher by about 6 percent when unaccented; duration is the most reliable cue, f0 the weakest in unaccented words | Fry's and Lieberman's own magnitudes (not retrieved) | +| Plosives: Lisker & Abramson 1964 (VOT, secondary table), Chodroff & Wilson 2014 (burst spectra, /b/ COG 2833 Hz, peak 1643 Hz), Stevens & Blumstein (labial onset spectrum diffuse-falling) | A /b/ burst is a short low-frequency-weighted transient with a short voice-onset lag (0-5 ms mean 1 ms for English /b/) | A /b/ voice-bar level in dB and burst duration norms: no source retrieved, so the AquaSynth levels for them are design choices, labelled as such | + +## Male reference formants used for verification (Hz) + +Hillenbrand 1995 male means (45 men, steady state; computed from the public data): /i/ 343/2323/3001, /ɪ/ 429/2034/2687, +/e/ 476/2090/2692, /ɛ/ 588/1803/2604, /æ/ 591/1930/2595, /ɑ/ 757/1309/2535, /ɔ/ 656/1023/2521, /o/ 498/910/2459, +/ʊ/ 469/1123/2435, /u/ 380/992/2355, /ʌ/ 621/1181/2548. Peterson & Barney 1952 male means (33 men): /i/ 267/2294/2937, +/ɪ/ 392/1993/2569, /ɛ/ 526/1854/2481, /æ/ 664/1727/2420, /ʌ/ 631/1192/2377, /ɑ/ 718/1091/2442, /ɔ/ 568/836/2403, +/ʊ/ 437/1023/2245, /u/ 307/876/2239. + +## The gap the area functions must be read against + +Story's shapes belong to one speaker. Computed independently (frequency-domain tube chain, `tools/formant-check/tube_formants.py`, +no AquaSynth code), the tabulated area functions at their own lengths give for example /i/ 261/2139, /ʌ/ 561/827, /ɑ/ 666/933, +/u/ 243/684 Hz, against the population means above (/i/ 343/2323, /ʌ/ 621/1181, /ɑ/ 757/1309, /u/ 380/992). So a render that +follows Story's area functions can be correct and still sit 10-30 percent from a population mean. The verification therefore +reports two distances: render against the area function's own predicted formants (does the ladder implement the tube), and +render against the population means (is the speaker typical). Only the first is a defect test. + +## Not retrieved (from the fact sheet) + +Story-Titze-Hoffman 1996/1998 numeric tables; Fant 1960, Stevens-House, Wakita, Ladefoged tables; Maeda per-vowel values; +Dang & Honda 1994 text; per-consonant nasal zero ranges; Fry and Lieberman magnitudes; Klatt 1975; burst duration norms and the +/b/ voice-bar level; Liljencrants 1985; the JOS book tract chapters. The Story 2008 table came through a page summariser, not a PDF +text extraction; every column was checked by computing its formants (all eleven give vowel-typical F1/F2 orderings), which +catches a gross transcription error, not a small one. diff --git a/docs/research/data/story2008-area-functions.csv b/docs/research/data/story2008-area-functions.csv new file mode 100644 index 0000000..0a6e572 --- /dev/null +++ b/docs/research/data/story2008-area-functions.csv @@ -0,0 +1,49 @@ +# Story 2008, JASA 123(1):327-335, doi 10.1121/1.2805683, PMC2377017, Table I. Areas in cm^2, section 1 = glottis, section 44 = lips. +# Transcribed by an Eyes pass from the PMC page through a page summariser (not pdftotext). tools/formant-check/tube_formants.py checks every column for plausible formants. +section,i,ɪ,e,ɛ,æ,ʌ,ɑ,ɔ,o,ʊ,u +1,0.51,0.28,0.29,0.37,0.31,0.23,0.56,0.27,0.38,0.37,0.54 +2,0.59,0.21,0.26,0.30,0.21,0.34,0.62,0.43,0.45,0.38,0.61 +3,0.62,0.21,0.30,0.24,0.18,0.47,0.66,0.54,0.57,0.49,0.66 +4,0.72,0.30,0.40,0.23,0.23,0.60,0.78,0.67,0.77,0.62,0.75 +5,1.24,0.47,0.55,0.29,0.33,0.77,0.97,0.83,1.31,0.85,1.13 +6,2.30,0.71,0.74,0.41,0.50,1.06,1.16,0.92,1.92,1.28,1.99 +7,3.30,1.12,0.99,0.58,0.78,1.26,1.12,0.89,1.74,1.62,2.83 +8,3.59,1.48,1.09,0.82,0.96,1.09,0.82,0.73,1.11,1.47,2.90 +9,3.22,1.35,0.90,0.97,0.85,0.80,0.55,0.55,0.75,1.04,2.52 +10,2.86,1.05,0.69,0.82,0.63,0.65,0.45,0.44,0.59,0.81,2.40 +11,3.00,0.92,0.77,0.62,0.46,0.56,0.37,0.37,0.57,1.03,2.83 +12,3.61,0.92,1.31,0.54,0.36,0.47,0.29,0.28,0.68,1.44,3.56 +13,4.39,1.19,2.13,0.54,0.33,0.37,0.21,0.18,0.73,1.49,3.99 +14,4.95,1.94,2.74,0.68,0.46,0.24,0.15,0.14,0.67,1.28,3.89 +15,5.17,2.83,3.03,1.09,0.73,0.17,0.16,0.15,0.58,1.06,3.50 +16,5.16,3.31,3.23,1.62,1.00,0.18,0.25,0.14,0.49,0.85,3.04 +17,5.18,3.48,3.33,2.03,1.30,0.23,0.34,0.13,0.44,0.69,2.64 +18,5.26,3.60,3.27,2.35,1.66,0.27,0.43,0.14,0.42,0.56,2.44 +19,5.20,3.64,3.09,2.51,1.97,0.28,0.54,0.18,0.49,0.42,2.31 +20,5.02,3.49,2.84,2.39,2.06,0.29,0.61,0.20,0.53,0.27,2.07 +21,4.71,3.20,2.66,2.22,2.03,0.33,0.67,0.23,0.38,0.27,1.80 +22,4.13,2.90,2.46,2.13,2.01,0.52,0.98,0.49,0.30,0.41,1.52 +23,3.43,2.59,2.14,2.00,1.89,0.97,1.76,1.03,0.45,0.51,1.14 +24,2.83,2.21,1.79,1.78,1.66,1.50,2.75,1.58,0.61,0.49,0.74 +25,2.32,1.87,1.44,1.58,1.49,1.91,3.52,2.06,0.71,0.47,0.42 +26,1.83,1.54,1.17,1.43,1.42,2.23,4.08,2.61,0.79,0.50,0.22 +27,1.46,1.20,1.00,1.31,1.37,2.65,4.74,3.35,0.86,0.58,0.14 +28,1.23,0.92,0.88,1.23,1.34,3.29,5.61,4.34,1.01,0.80,0.20 +29,1.08,0.74,0.80,1.24,1.41,4.13,6.60,5.51,1.41,1.19,0.47 +30,0.94,0.59,0.81,1.38,1.58,5.00,7.61,6.70,2.09,1.62,0.89 +31,0.80,0.52,0.85,1.61,1.82,5.77,8.48,7.75,3.00,2.27,1.15 +32,0.67,0.54,0.84,1.82,2.19,6.33,9.06,8.63,4.10,3.24,1.42 +33,0.55,0.59,0.86,1.96,2.63,6.61,9.29,9.29,5.16,4.16,2.17 +34,0.46,0.65,0.96,2.01,2.97,6.63,9.26,9.59,6.22,5.00,3.04 +35,0.40,0.71,1.18,2.00,3.17,6.45,9.06,9.42,7.34,5.70,3.69 +36,0.36,0.67,1.35,1.95,3.40,6.04,8.64,8.78,8.15,6.11,4.70 +37,0.35,0.61,1.48,1.77,3.56,5.39,7.91,7.82,8.61,6.21,5.74 +38,0.35,0.57,1.62,1.48,3.57,4.42,6.98,6.50,8.37,6.29,5.41 +39,0.38,0.50,1.49,1.30,3.58,3.29,6.02,4.95,6.76,6.24,3.82 +40,0.51,0.48,1.29,1.21,3.44,2.37,5.13,3.47,4.37,4.91,2.34 +41,0.74,0.54,1.24,1.10,3.15,1.74,4.55,2.15,2.30,2.61,1.35 +42,0.92,0.73,1.17,0.99,3.38,1.36,4.52,1.38,1.06,1.09,0.65 +43,0.96,0.93,1.04,0.91,3.99,1.17,4.71,1.11,0.58,0.63,0.29 +44,0.91,0.82,0.95,0.78,4.17,0.99,4.72,0.90,0.47,0.59,0.16 +section_length_cm,0.384,0.376,0.386,0.393,0.366,0.390,0.388,0.395,0.417,0.440,0.445 +vtl_cm,16.90,16.55,16.98,17.30,16.11,17.14,17.09,17.40,18.33,19.34,19.59 diff --git a/tools/formant-check/burg_formants.py b/tools/formant-check/burg_formants.py new file mode 100644 index 0000000..9f72aed --- /dev/null +++ b/tools/formant-check/burg_formants.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python3 +"""Independent formant estimator for AquaSynth renders. Shares no code with the C# tests or with the ladder. + +Burg linear prediction (not autocorrelation LPC, not the tract's own impulse response): stable on short windows and +unbiased by the window edge. Usage: + burg_formants.py FILE.wav T0 T1 [--order N] [--frame-ms 30] [--json] +Resamples to 10 kHz (FFT-domain), pre-emphasises 0.97, Hamming 30 ms frames hop 10 ms in [T0,T1], Burg order 12 default, +poles with bandwidth < 400 Hz and 150 Hz < f < 4.5 kHz are formants. Prints the per-formant median over frames and the +interquartile spread. It measures; it does not listen. +""" +import sys, json, wave, argparse +import numpy as np + +def read_wav(path): + with wave.open(path) as w: + sr, n, ch, sw = w.getframerate(), w.getnframes(), w.getnchannels(), w.getsampwidth() + raw = w.readframes(n) + assert sw == 2, "16-bit PCM only" + x = np.frombuffer(raw, dtype=np.int16).astype(np.float64) / 32768.0 + return (x.reshape(-1, ch).mean(axis=1) if ch > 1 else x), sr + +def resample(x, sr, target): + if sr == target: + return x + n = int(round(len(x) * target / sr)) + X = np.fft.rfft(x) + m = n // 2 + 1 + Y = np.zeros(m, dtype=complex) + k = min(m, len(X)) + Y[:k] = X[:k] + return np.fft.irfft(Y, n) * (n / len(x)) + +def burg(x, order): + f = x.copy(); b = x.copy(); a = np.array([1.0]) + for _ in range(order): + fp, bp = f[1:], b[:-1] + k = -2.0 * np.dot(fp, bp) / (np.dot(fp, fp) + np.dot(bp, bp) + 1e-30) + a = np.concatenate([a, [0.0]]); a = a + k * a[::-1] + f, b = fp + k * bp, bp + k * fp + return a + +def formants(frame, fs, order): + a = burg(frame * np.hamming(len(frame)), order) + r = np.roots(a); r = r[np.imag(r) > 1e-4] + fq = np.angle(r) * fs / (2 * np.pi); bw = -np.log(np.abs(r)) * fs / np.pi + return sorted(float(f) for f, w in zip(fq, bw) if 150 < f < 4500 and w < 400) + +def main(): + p = argparse.ArgumentParser() + p.add_argument("wav"); p.add_argument("t0", type=float); p.add_argument("t1", type=float) + p.add_argument("--order", type=int, default=12); p.add_argument("--frame-ms", type=float, default=30) + p.add_argument("--json", action="store_true") + a = p.parse_args() + x, sr = read_wav(a.wav) + fs = 10000 + y = resample(x, sr, fs) + y = np.append(y[0], y[1:] - 0.97 * y[:-1]) + n = int(a.frame_ms * fs / 1000); hop = int(0.010 * fs) + rows = [] + for s in range(int(a.t0 * fs), int(a.t1 * fs) - n, hop): + fr = y[s:s + n] + if fr.std() < 1e-6: continue + rows.append(formants(fr, fs, a.order)[:4]) + out = {"file": a.wav, "t0": a.t0, "t1": a.t1, "order": a.order, "frames": len(rows)} + for i in range(4): + v = [r[i] for r in rows if len(r) > i] + out[f"F{i+1}"] = None if len(v) < max(3, len(rows) // 2) else {"median": round(float(np.median(v)), 1), "q1": round(float(np.percentile(v, 25)), 1), "q3": round(float(np.percentile(v, 75)), 1), "n": len(v)} + if a.json: print(json.dumps(out)) + else: + print(f"{a.wav} [{a.t0},{a.t1}] order {a.order} frames {len(rows)}: " + " ".join(f"F{i+1}={out[f'F{i+1}']['median']} ({out[f'F{i+1}']['q1']}-{out[f'F{i+1}']['q3']})" if out[f"F{i+1}"] else f"F{i+1}=-" for i in range(4))) + +if __name__ == "__main__": + main() diff --git a/tools/formant-check/tube_formants.py b/tools/formant-check/tube_formants.py new file mode 100644 index 0000000..9dda15a --- /dev/null +++ b/tools/formant-check/tube_formants.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +"""Frequency-domain formants of a published area function. Independent of the AquaSynth ladder (no time-domain scattering, +no Faust): a chain of lossy uniform tubes (ABCD matrices) between a high-impedance glottis and a piston-in-baffle radiation load. + + tube_formants.py CSV [--vowel V] [--length-cm L] [--c 350] [--fmax 5000] +CSV is docs/research/data/story2008-area-functions.csv. --length-cm rescales every vowel to one tract length (the ladder +runs one length for all vowels); default is each vowel's own VTL. Prints F1..F4 as peaks of |U_lips / U_glottis|. +""" +import sys, argparse +import numpy as np + +RHO = 1.14e-3 # g/cm^3 as in VocalTractLab's dissertation constants (SI 1.14 kg/m^3) + +def load(path): + rows = [l.strip() for l in open(path) if l.strip() and not l.startswith('#')] + head = rows[0].split(',')[1:] + areas = {v: [] for v in head} + extra = {} + for r in rows[1:]: + c = r.split(',') + if c[0].isdigit(): + for v, x in zip(head, c[1:]): areas[v].append(float(x)) + else: + extra[c[0]] = dict(zip(head, map(float, c[1:]))) + return head, {v: np.array(a) for v, a in areas.items()}, extra + +def transfer(areas, section_cm, c, freqs, wall=0.0): + """|U_lip/U_glottis| for tubes of the given areas (cm^2) and one section length (cm). cgs units.""" + out = np.zeros(len(freqs)) + for n, f in enumerate(freqs): + w = 2 * np.pi * f + # attenuation per cm: viscous/thermal ~ f^0.5 plus wall term, giving F1 bandwidths of order 50-100 Hz + alpha = 4e-5 * np.sqrt(f) + wall + 1.5e-3 + k = w / c * (1 - 1j * alpha * c / w) if w > 0 else 0 + M = np.eye(2, dtype=complex) + for A in areas: # glottis -> lips; state (P, U) + Z = RHO * c / A + kl = k * section_cm + M = M @ np.array([[np.cos(kl), 1j * Z * np.sin(kl)], [1j * np.sin(kl) / Z, np.cos(kl)]]) + A_l = areas[-1] + a = np.sqrt(A_l / np.pi) + ka = w / c * a + # Levine-Schwinger-ish low-ka piston radiation impedance (Flanagan: R||L) + Zr = (RHO * c / A_l) * (ka ** 2 / 2 + 1j * 8 * ka / (3 * np.pi)) if ka < 1.2 else (RHO * c / A_l) * (1 - 0j) + # U_glottis from open-circuit glottis (high Z): P_g free -> use U_g = 1, solve P_l = Zr*U_l; (P,U)_glottis = M (P,U)_lip + # lip: P_l = Zr*U_l. glottis U_g = M[1,0]*P_l + M[1,1]*U_l = U_l*(M10*Zr + M11) + out[n] = 1 / abs(M[1, 0] * Zr + M[1, 1]) + return out + +def peaks(freqs, mag, n=4): + idx = [i for i in range(1, len(mag) - 1) if mag[i] > mag[i - 1] and mag[i] >= mag[i + 1]] + res = [] + for i in idx: + y0, y1, y2 = np.log(mag[i - 1]), np.log(mag[i]), np.log(mag[i + 1]) + d = 0.5 * (y0 - y2) / (y0 - 2 * y1 + y2) # parabolic refinement + res.append(freqs[i] + d * (freqs[1] - freqs[0])) + return res[:n] + +def main(): + p = argparse.ArgumentParser() + p.add_argument('csv'); p.add_argument('--vowel'); p.add_argument('--length-cm', type=float) + p.add_argument('--c', type=float, default=350.0); p.add_argument('--fmax', type=float, default=5000) + a = p.parse_args() + head, areas, extra = load(a.csv) + freqs = np.arange(50, a.fmax, 5.0) + for v in head: + if a.vowel and v != a.vowel: continue + A = areas[v] + L = a.length_cm or extra['vtl_cm'][v] + sec = L * 100 / 100 / len(A) # cm per section + sec = L / len(A) + m = transfer(A, sec, a.c * 100, freqs) + print(f"{v} L={L:.2f}cm " + " ".join(f"F{i+1}={f:.0f}" for i, f in enumerate(peaks(freqs, m)))) + +if __name__ == '__main__': + main()