From d1f53ed51fc202c0997c8fd9c142a05f3d39586e Mon Sep 17 00:00:00 2001 From: Meta Vens Date: Fri, 16 Oct 2026 20:01:15 +0200 Subject: [PATCH] Interval ground map: Pass 9b, overloads stay native (build-only export wrappers, -Od, one SPIR-V edit); overload-renames cut deleted; wrapper probe scripts Co-Authored-By: Claude Opus 5.5 --- docs/cultmath-interval-ground-cut.md | 89 ++++++++++++++----- .../overload-wrappers/lava.sh | 29 ++++++ .../overload-wrappers/split.py | 39 ++++++++ .../overload-wrappers/spvedit.py | 71 +++++++++++++++ .../overload-wrappers/wrap.sh | 50 +++++++++++ .../overload-wrappers/wrapgen.py | 22 +++++ 6 files changed, 279 insertions(+), 21 deletions(-) create mode 100644 docs/research/native-shader-probes/overload-wrappers/lava.sh create mode 100644 docs/research/native-shader-probes/overload-wrappers/split.py create mode 100644 docs/research/native-shader-probes/overload-wrappers/spvedit.py create mode 100644 docs/research/native-shader-probes/overload-wrappers/wrap.sh create mode 100644 docs/research/native-shader-probes/overload-wrappers/wrapgen.py diff --git a/docs/cultmath-interval-ground-cut.md b/docs/cultmath-interval-ground-cut.md index 34fa08be..9ad6e0fe 100644 --- a/docs/cultmath-interval-ground-cut.md +++ b/docs/cultmath-interval-ground-cut.md @@ -2198,8 +2198,42 @@ and 16:50 UTC (18:30 to 18:50 CEST). Logs are in the Imagination scratchpad - **Overloads collapse in dxc, not in SPIRV-Cross.** With `export` on the 173 non-texture definitions, the library has 127 `OpFunction`s and 127 exports: one per name. Today's source has 22 overloaded names, 68 definitions and 46 lost. Trophic-nodes (branch `hands/cultmath-trophic-nodes`, unmerged) adds - `cultmath_segment_closest` and `cultmath_distance_to_segment` in 2D and 3D. After it merges there are - 24 names, 72 definitions and 48 to rename. + `cultmath_segment_closest` and `cultmath_distance_to_segment` in 2D and 3D, so 24 names and 72 definitions. + The loss comes from exporting overloaded names, and build-only wrappers avoid it (next bullet). The + source keeps its overloads (ruling `shader-overloads-stay-native`). +- **Build-only export wrappers keep every overload (Pass 9b).** Imagination `imagination-ag4-overload-wrappers`, + 2026-10-16, on 09940e20, ubuntu:24.04 on Yggdrasil, 17:40 to 17:55 UTC (19:40 to 19:55 CEST), log + `imagination-ag4-overload-wrappers/wrap-run2.log` in the session scratchpad. The probe generated one + wrapper per non-texture definition (173) in a build-only file. Each wrapper looks like + `export float3 cmw036_cultmath_lerp(float3 start, float3 end, float3 amount) { return cultmath_lerp(start, end, amount); }`. + The source was unchanged, with no `export` keyword anywhere, and it compiled with dxc `-T lib_6_3 -spirv`. + - At default optimisation dxc inlines every wrapper. The 173 wrappers hold the bodies, and no + overload survives as its own function. + - At `-Od` it keeps the call. That gives 346 functions: 173 wrappers, each with exactly one + `OpFunctionCall` to its own distinct callee. Each callee's `OpName` is already the plain source + name (`cultmath_lerp` four times), so no rename is needed. + - An instruction-level edit of the SPIR-V binary then makes the library. It keeps only the Shader + and Linkage capabilities, drops every `OpExtension`, deletes the wrapper functions with their + names and decorations, and moves each wrapper's `LinkageAttributes ... Export` onto its callee. + `spirv-val --target-env spv1.5` accepts the result. Deleting the export instead of moving it gives + a module SPIRV-Cross refuses ("no entry point"). + - SPIRV-Cross `vulkan-sdk-1.4.363.0 --es --version 310` emits 173 `cultmath_` definitions, as many + per name as the source has. There are no `_N` suffixes and no wrapper names: its own overload + handling renames only on a signature collision. The emitted text is 2621 lines, against 1497 for + today's three files, because `-Od` leaves temporaries. + - Call sites resolve. glslang compiles as a `#version 300 es` fragment shader that calls + `cultmath_lerp(float..)`, `cultmath_lerp(vec3..)`, `cultmath_saturate(vec4)` and + `cultmath_saturate(float)`. A call with `ivec2`, which has no overload, is refused ("no matching + overloaded function"). + - Same computation. All 105 families the probe generator covers ran on lavapipe through the + restored library, and every one is bit-identical to the committed GLSL route. 85 of the 105 are + also bit-identical to the HLSL route, the same 85 where the committed GLSL matches it. + - The rest of the pipeline carries over. Partitioned by origin file, the output gives 178 chunks + (125 core, 51 Cycles, 2 Phacelle). No origin is unknown and nothing calls across licences. + Every overload set lives in one file, and no signature repeats. glslang compiles all four + combinations as ES 3.00. The source has 13 `static const` lines, all of the form + `static const float CULTMATH_X = ;`. None of them reaches the output, and today's GLSL + holds 13. - **Capability strip.** `spirv-opt --trim-capabilities` (spirv-tools 2025.1~rc1) leaves all 14 capabilities and both extensions in place, and SPIRV-Cross still refuses ("RayQuery requires Vulkan GLSL 460"). Stripping every capability except Shader and Linkage and every OpExtension in the @@ -2221,7 +2255,8 @@ and 16:50 UTC (18:30 to 18:50 CEST). Logs are in the Imagination scratchpad probe's device), glslang-tools 15.1.0, spirv-tools 2025.1~rc1, libvulkan-dev 1.3.275 and gcc 13.2. `apt-get update --snapshot 20261016T000000Z` (apt 2.8.3) then installs the same versions. That pins the package set to a date and lets the C# judge run in the same container as the device. -- **Consumer audit for the renames.** It covers `origin` of Aetheria (master a8586d08 and all of its +- **Consumer audit.** No public name changes under the wrapper route, so it shows only who reads the + committed GLSL and HLSL. It covers `origin` of Aetheria (master a8586d08 and all of its remote branches), Asura (60fbd19), Odin (8e1de66) and gamecult-site (af6e206), and HEAD of every other git repo under F:\Projects except the CultLib worktrees, the archived AetheriaEve and the retired CultMath history repo. The search was for `cultmath_[a-z]` outside docs. Only one shader @@ -2243,30 +2278,42 @@ and 16:50 UTC (18:30 to 18:50 CEST). Logs are in the Imagination scratchpad the device's identity. lavapipe re-runs bit-identically, so the bound is a regression gate, and an identity change refuses until it is re-recorded. Exact and integer families must match bit for bit, and `check` families must pass their `Verify`. -- **The renames come before the switch.** dxc drops overloads, so the switch cannot run on today's - names. The renames are their own cut under today's `GlslLowering`, and the switch then replaces the - owner of a GLSL text whose names already match. +- **Unique names for dxc live only inside the build** (ruling `shader-overloads-stay-native`). The + build owns three things here. + - The wrapper list. It is generated from the source's own definitions, one wrapper per non-texture + `cultmath_` definition, and the build refuses a definition it cannot read, just as it refuses a + `static const` line of another form. The wrappers are text in the build's temp directory and are + never committed. The source gains no `export` keyword and no `CULTMATH_EXPORT` macro. + - The compile mode. The library compiles at `-Od`, because the default optimiser's inlining is + what erases the overloads. glslang and the drivers optimise downstream, and on lavapipe the + result is bit-identical to today's output. + - One SPIR-V edit. A single instruction-level pass over the binary, never a regex over text, + strips capabilities and extensions, deletes the wrappers and moves their exports onto the + callees. It replaces the disassembly grep that the capability-strip probe used. It refuses a + wrapper that is not exactly one call. SPIRV-Cross then emits the overloaded names as they are, + so the build has no rename step and no name map. ### Cut sequence (Pass 9; one line each, mapped when about to run) 1. `shader-conforming-run`: spec admitted. The lavapipe job with the C# device judge, run against - today's committed HLSL and GLSL, so the next two cuts are verified against a running check. -2. `overload-renames`: waits on question `overload-rename-scheme` and on `trophic-nodes` merging. - 48 overloads get unique names in both HLSL copies, the C# mirror's name maps, the fixture's family - names and the lavapipe bounds rows. The GLSL is regenerated under today's lowering. Muninn's - `cultmath_saturate(float)` keeps its name under the recommended scheme. Any other option makes Muninn a named follow-up. -3. `glsl-build-dxc`: the switch. `tools/build-glsl.sh` pins dxc v1.9.2607 (the sha above) and SPIRV-Cross - `vulkan-sdk-1.4.363.0`, and takes a `CULTMATH_EXPORT` macro (empty by default, `export` under the - build's `-D`). The build strips capabilities and extensions and runs `spirv-val`, emits ES 3.10 with - the guarded `#version` and no placeholder `main`, and partitions the output by origin file into the - three committed files with their licence lines and guards. It emits the 13 constants from their - source lines. `GlslLowering.cs` and its lowering contract tests are deleted. The lavapipe job also - checks that the committed files are the build's output, then re-records `lavapipe-glsl`. - `get-spirv-cross.ps1` is pinned to the same tag. Waits on 1 and 2. -4. `angle-capture` (stage-shaders-device part 2): the Starfire native-ANGLE tool on the shared work + today's committed HLSL and GLSL, so the next cut is verified against a running check. +2. `glsl-build-dxc`: the switch. `tools/build-glsl.sh` pins dxc v1.9.2607 (the sha above) and SPIRV-Cross + `vulkan-sdk-1.4.363.0`. Its first step generates the export wrappers, one per non-texture definition, + into the build's temp directory; they are never committed (Pass 9b). It compiles source plus + wrappers with `-T lib_6_3 -spirv -Od`. One SPIR-V edit then strips capabilities and extensions, + deletes the wrappers and moves their exports onto the callees, and `spirv-val` runs on the result. + The build emits ES 3.10 with the guarded `#version` and no placeholder `main`, and partitions the + output by origin file into the three committed files with their licence lines and guards. It emits + the 13 constants from their source lines. The HLSL source is not edited, and every overload keeps + its name in the HLSL and the GLSL. `GlslLowering.cs` and its lowering contract tests are deleted. + The lavapipe job checks that the committed files are the build's output, and that every family's + `lavapipe-glsl` result is bit-identical to the old lowering's. A miss is a defect in the cut, not + something to re-record. `get-spirv-cross.ps1` is pinned to the same tag. Waits on 1. If + `trophic-nodes` merges first, the build covers its four new overloads with nothing extra. +3. `angle-capture` (stage-shaders-device part 2): the Starfire native-ANGLE tool on the shared work list (ES 3.00 fragment flavour, RGBA32UI readback), loading a pinned ANGLE build rather than Docker Desktop's copy. Captures carry the renderer, ANGLE version and revision, the shader sha and per-family argument hashes and bits, and the ordinary test run judges them. A shader-sha change marks them stale. It settles `span-allowance-gpu-margin`'s six CPU survivors, `iv-exp-gpu-widening`'s measurement and `device-p2-pin-notes`, and closes `glsl-browser-parity-run` and `affine-device-runner-seed`. It waits - on 3, because a capture of the old lowering goes stale at the switch. + on 2, because a capture of the old lowering goes stale at the switch. diff --git a/docs/research/native-shader-probes/overload-wrappers/lava.sh b/docs/research/native-shader-probes/overload-wrappers/lava.sh new file mode 100644 index 00000000..e36f346b --- /dev/null +++ b/docs/research/native-shader-probes/overload-wrappers/lava.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# Imagination probe, lavapipe leg: every fixture family through three routes on lavapipe. +# h = HLSL by dxc cs_6_0, g = committed GLSL by glslang, r = restored overloaded library GLSL (lib.body.glsl) by glslang. +set -u; P=/tmp/p; W=/tmp/w; cd $W; SH=/src/packages/cultmath/shaders; FX=/src/packages/cultmath/tests/CultMath.Tests/fixtures/glsl-parity.json +echo; echo "=== 2b lavapipe: same computation" +mkdir -p fam; gcc -O2 -o vkrun $P/vkrun.c -lvulkan && python3 $P/fam.py gen $FX fam >/dev/null +cat $SH/CultMath.glsl $SH/CultMath.Cycles.glsl $SH/CultMath.Phacelle.glsl > all.glsl +ok=(0 0 0); first=1 +for b in fam/*.body.glsl; do t=${b%.body.glsl}; tag=$(basename $t) + n=$(python3 -c "import json;m={x['tag']:x for x in json.load(open('fam/manifest.json'))}['$tag'];print(m['n'],m['nout'])") + $DXC_PATH -T cs_6_0 -E main -HV 2021 -spirv -fspv-target-env=vulkan1.2 -I $SH $t.hlsl -Fo $t.h.spv >$t.h.log 2>&1 && ./vkrun $t.h.spv $t.in $t.h.out $n && ok[0]=$((ok[0]+1)) + for r in g r; do src=all.glsl; [ $r = r ] && src=lib.body.glsl + { printf '#version 310 es\nprecision highp float;\nprecision highp int;\n'; cat $src $b; } > $t.$r.comp + glslangValidator -V --target-env vulkan1.2 -o $t.$r.spv $t.$r.comp >$t.$r.log 2>&1 && ./vkrun $t.$r.spv $t.in $t.$r.out $n && { [ $r = g ] && ok[1]=$((ok[1]+1)) || ok[2]=$((ok[2]+1)); } + done +done +echo "ran on lavapipe: hlsl ${ok[0]}, committed glsl ${ok[1]}, restored glsl ${ok[2]} (of $(ls fam/*.body.glsl | wc -l))" +grep -h -m1 ERROR fam/*.r.log | sed -E 's/[0-9]+:/N:/g' | sort | uniq -c | sort -rn | head -5 +python3 - <<'PY' +import os,filecmp +d='/tmp/w/fam'; tags=sorted(f[:-10] for f in os.listdir(d) if f.endswith('.body.glsl')) +def same(a,b): return [t for t in tags if os.path.exists(f'{d}/{t}.{a}.out') and os.path.exists(f'{d}/{t}.{b}.out') and filecmp.cmp(f'{d}/{t}.{a}.out',f'{d}/{t}.{b}.out',shallow=False)] +rg=same('r','g'); rh=same('r','h'); gh=same('g','h') +print('bit-identical: restored vs committed glsl',len(rg),'| restored vs hlsl',len(rh),'| committed glsl vs hlsl',len(gh),'of',len(tags)) +print('restored differs from committed glsl in:',[t for t in tags if t not in rg][:12]) +print('restored differs from hlsl but committed glsl matches hlsl:',[t for t in gh if t not in rh][:12]) +PY + +CMP_TOP=10 python3 $P/fam.py cmp $FX fam r 2>&1 | head -16 diff --git a/docs/research/native-shader-probes/overload-wrappers/split.py b/docs/research/native-shader-probes/overload-wrappers/split.py new file mode 100644 index 00000000..d789d0da --- /dev/null +++ b/docs/research/native-shader-probes/overload-wrappers/split.py @@ -0,0 +1,39 @@ +import re,sys,os,collections +sh,g,out=sys.argv[1:4]; os.makedirs(out,exist_ok=True) +origin={}; owner={'CultMath.hlsl':'core','CultMath.Interval.hlsl':'core','CultMath.Affine.hlsl':'core','CultMath.Cycles.hlsl':'cycles','CultMath.Phacelle.hlsl':'phacelle'} +for fn,o in owner.items(): + t=open(os.path.join(sh,fn)).read() + for m in re.finditer(r'(?m)^(?:export\s+)?\w+\s+(cultmath_\w+)\s*\(',t): origin.setdefault(m.group(1),o) + for m in re.finditer(r'(?m)^struct\s+(\w+)',t): origin.setdefault(m.group(1),o) +text=open(g).read() +# top-level chunks: struct blocks and function definitions; everything else is prologue +lines=text.split('\n'); chunks=[]; i=0; pro=[] +while i1][:6]) +for o,b in files.items(): open(os.path.join(out,o+'.glsl'),'w').write('\n'.join(b)+'\n') +print('split sizes', {o:len('\n'.join(b).split('\n')) for o,b in files.items()}) diff --git a/docs/research/native-shader-probes/overload-wrappers/spvedit.py b/docs/research/native-shader-probes/overload-wrappers/spvedit.py new file mode 100644 index 00000000..1c2807d3 --- /dev/null +++ b/docs/research/native-shader-probes/overload-wrappers/spvedit.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Probe: instruction-level SPIR-V edit (binary, no text regex). +stats how each wrapper looks after dxc +edit mode drop: strip caps (keep Shader, Linkage) and extensions, delete + wrapper functions and their names/decorations, set each callee's OpName to the overloaded source name. + mode export: as drop, but each wrapper's Export decoration moves to its callee instead of being deleted.""" +import struct, sys, collections +def load(p): + b = open(p, 'rb').read(); w = list(struct.unpack('<%dI' % (len(b) // 4), b)); ins = []; i = 5 + while i < len(w): n = w[i] >> 16; ins.append(w[i:i + n]); i += n + return w[:5], ins +def s_dec(ws): + b = struct.pack('<%dI' % len(ws), *ws); return b[:b.index(0)].decode() +def s_enc(s): + b = s.encode() + b'\0'; b += b'\0' * (-len(b) % 4); return list(struct.unpack('<%dI' % (len(b) // 4), b)) +def op(x): return x[0] & 0xffff +def funcs(ins): + fs = {}; cur = None + for k, x in enumerate(ins): + if op(x) == 54: cur = x[2]; fs[cur] = [k, None, [], set()] + if cur is not None: + if op(x) == 57: fs[cur][2].append(x[3]) + o = op(x) + if o == 248: fs[cur][3].add(x[1]) + elif o not in (8, 317, 62, 246, 247, 249, 250, 253, 254, 252) and len(x) > 2: fs[cur][3].add(x[2]) + if op(x) == 56: fs[cur][1] = k; cur = None + return fs +def names(ins): return {x[1]: s_dec(x[2:]) for x in ins if op(x) == 5} +def wrappers(tsv): return [l.split('\t') for l in open(tsv).read().splitlines() if l] +def analyse(ins, tsv): + nm = names(ins); fs = funcs(ins); byname = {v: k for k, v in nm.items() if k in fs}; res = [] + for wn, name, fn, ps in wrappers(tsv): + fid = byname.get(wn); callees = fs[fid][2] if fid else None + res.append((wn, name, fid, callees)) + return nm, fs, res +if sys.argv[1] == 'stats': + h, ins = load(sys.argv[2]); nm, fs, res = analyse(ins, sys.argv[3]) + missing = [r[0] for r in res if r[2] is None]; inl = [r[0] for r in res if r[2] and not r[3]] + multi = [r[0] for r in res if r[3] and len(r[3]) != 1] + callee = collections.Counter(r[3][0] for r in res if r[3] and len(r[3]) == 1) + wrong = [(r[0], nm.get(r[3][0])) for r in res if r[3] and len(r[3]) == 1 and nm.get(r[3][0]) != r[1]] + print('functions %d, wrappers found %d of %d, missing %d, inlined (no call) %d, not-one-call %d' % (len(fs), len(res) - len(missing), len(res), len(missing), len(inl), len(multi))) + print('distinct callees %d, callee shared by >1 wrapper %d' % (len(callee), sum(1 for c in callee.values() if c > 1))) + print('callee OpName != source name: %d %s' % (len(wrong), wrong[:4])) + calls = sum(len(f[2]) for f in fs.values()); print('total OpFunctionCall %d; non-wrapper functions %d' % (calls, len(fs) - len(res) + len(missing))) + print('sample', [(r[0], nm.get(r[3][0]) if r[3] else None) for r in res[:3]], 'missing sample', missing[:3], 'inlined sample', inl[:3]) + sys.exit(0) +_, src, dst, tsv, mode = sys.argv[1:6] +h, ins = load(src); nm, fs, res = analyse(ins, tsv) +drop_k = set(); drop_ids = set(); setname = {}; retarget = {} +for wn, name, fid, callees in res: + if fid is None or not callees or len(callees) != 1: sys.exit('refuse: wrapper %s is not one call' % wn) + a, b, _, ids = fs[fid]; drop_k.update(range(a, b + 1)); drop_ids |= ids | {fid} + setname[callees[0]] = name; retarget[fid] = callees[0] +drop_ids |= set(retarget) +outi = []; named = set() +for k, x in enumerate(ins): + if k in drop_k: continue + o = op(x) + if o == 17 and x[1] not in (1, 5): continue + if o == 10: continue + if o == 5 and x[1] in setname: x = [0] + [x[1]] + s_enc(setname[x[1]]); x[0] = (len(x) << 16) | 5; named.add(x[1]) + elif o in (5, 6) and x[1] in drop_ids: continue + if o == 71 and x[1] in drop_ids: + if mode == 'export' and x[1] in retarget and x[2] == 41: x = [x[0], retarget[x[1]]] + x[2:] + else: continue + outi.append(x) +for fid, name in setname.items(): + if fid not in named: outi.insert(next(i for i, x in enumerate(outi) if op(x) == 5), [((2 + len(s_enc(name))) << 16) | 5, fid] + s_enc(name)) +w = h + [v for x in outi for v in x]; open(dst, 'wb').write(struct.pack('<%dI' % len(w), *w)) +print('edit %s: dropped %d wrapper functions, renamed %d callees, kept %d instructions' % (mode, len(res), len(setname), len(outi))) diff --git a/docs/research/native-shader-probes/overload-wrappers/wrap.sh b/docs/research/native-shader-probes/overload-wrappers/wrap.sh new file mode 100644 index 00000000..65d6360c --- /dev/null +++ b/docs/research/native-shader-probes/overload-wrappers/wrap.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# Imagination probe: build-only export wrappers -> dxc lib_6_3 -spirv -> SPIR-V edit -> SPIRV-Cross ES. /src = CultLib. +set -u; P=/tmp/p; W=/tmp/w; mkdir -p $W/tools; cd $W +step() { echo; echo "=== $*"; } +export DEBIAN_FRONTEND=noninteractive +apt-get update -qq >/dev/null && apt-get install -y -qq --no-install-recommends ca-certificates curl git cmake g++ make python3 \ + mesa-vulkan-drivers libvulkan-dev glslang-tools spirv-tools >/dev/null 2>&1; echo "apt exit $?" +curl -sL --fail -o dxc.tgz https://github.com/microsoft/DirectXShaderCompiler/releases/download/v1.9.2607/linux_dxc_2026_07_29.x86_x64.tar.gz +echo "dxc sha ok: $(sha256sum dxc.tgz | cut -c1-64 | grep -c 55665c87824051ed4774ff3280a79ccbbb7d39243b9736ca5e98222134112d54)" +mkdir -p tools/dxc && tar xzf dxc.tgz -C tools/dxc; DXC=$(find tools/dxc -name dxc -type f | head -1) +git clone -q --depth 1 --branch vulkan-sdk-1.4.363.0 https://github.com/KhronosGroup/SPIRV-Cross.git tools/sc +cmake -S tools/sc -B tools/scb -DCMAKE_BUILD_TYPE=Release -DSPIRV_CROSS_ENABLE_TESTS=OFF -DSPIRV_CROSS_CLI=ON >/dev/null && cmake --build tools/scb --target spirv-cross -j4 >/dev/null 2>&1; SC=tools/scb/spirv-cross; ls $SC >/dev/null && echo "spirv-cross built" +SH=/src/packages/cultmath/shaders +step "1 wrappers" +python3 $P/wrapgen.py $SH shw; head -3 shw/cm_wrappers.hlsl | cut -c1-160; grep -c '' shw/wrappers.tsv +echo "source defs per overloaded name: $(cut -f2 shw/wrappers.tsv | sort | uniq -c | awk '$1>1{n++; d+=$1} END{print n" names, "d" defs"}')" +step "2 dxc lib_6_3 on source plus wrappers" +for v in default Od; do f=""; [ $v = Od ] && f="-Od" + $DXC -T lib_6_3 -HV 2021 -spirv -fspv-target-env=universal1.5 $f -I shw shw/cm_wrappers.hlsl -Fo w.$v.spv 2>&1 | head -3; echo "[$v] dxc exit ${PIPESTATUS[0]}" + [ -s w.$v.spv ] && python3 $P/spvedit.py stats w.$v.spv shw/wrappers.tsv; done +V=default; python3 $P/spvedit.py stats w.default.spv shw/wrappers.tsv | grep -q 'inlined (no call) 0,' || V=Od; echo "using variant $V" +step "3 SPIR-V edit, validate, SPIRV-Cross ES 310" +for m in drop export; do python3 $P/spvedit.py edit w.$V.spv e.$m.spv shw/wrappers.tsv $m + spirv-val --target-env spv1.5 e.$m.spv 2>&1 | head -2; echo "[$m] val exit ${PIPESTATUS[0]}" + $SC e.$m.spv --es --version 310 --output e.$m.glsl 2>&1 | head -2; echo "[$m] spirv-cross exit ${PIPESTATUS[0]}, lines $(wc -l < e.$m.glsl 2>/dev/null)" + spirv-dis e.$m.spv | grep -E 'OpCapability|OpExtension' | tr -s ' ' | sort | uniq -c | tr '\n' ' '; echo; done +cmp -s e.drop.glsl e.export.glsl && echo "drop and export outputs identical" || echo "drop and export outputs DIFFER" +for m in drop export; do G=e.$m.glsl; [ -s $G ] || continue; python3 - $G shw/wrappers.tsv <<'PY' +import re,sys,collections +g=open(sys.argv[1]).read(); rows=[l.split('\t') for l in open(sys.argv[2]).read().splitlines() if l] +src=collections.Counter(r[1] for r in rows) +defs=re.findall(r'(?m)^(?:\w+ )*?(\w+) (\w+)\(([^)]*)\)\n\{',g); out=collections.Counter(d[1] for d in defs) +print(sys.argv[1],'defs',len(defs),'cultmath_ defs',sum(v for k,v in out.items() if k.startswith('cultmath_')),'source defs',sum(src.values())) +print(' names whose def count differs from source:',[(k,src[k],out.get(k,0)) for k in src if src[k]!=out.get(k,0)][:8]) +print(' non-source names emitted:',[k for k in out if k not in src][:10],' cmw in text:',g.count('cmw'),' suffixed _N cultmath names:',len(re.findall(r'\bcultmath_\w+_\d+\(',g))) +print(' prologue:',' | '.join(l for l in g.split('\n')[:6] if l.startswith('#'))[:200], ' main:', g.count('void main')) +PY +done +G=e.export.glsl; echo "--- cultmath_lerp and cultmath_saturate as emitted:"; grep -E '^\w+ cultmath_(lerp|saturate)\(' $G +step "3b glslang ES 3.00 fragment, call sites resolve per overload" +sed -E '/^#version/d; /^#ifdef SPIRV_CROSS/d; /^#endif/d; /^layout\(local_size/d' $G | awk 'BEGIN{s=0} /^void main\(\)/{s=1} s==0{print} s==1&&/^\}/{s=0}' > lib.body.glsl +{ printf '#version 300 es\nprecision highp float;\nprecision highp int;\n'; cat lib.body.glsl; printf 'out highp vec4 c;\nvoid main() { vec3 p = gl_FragCoord.xyz; float a = cultmath_lerp(1.0, 2.0, p.x); vec3 b = cultmath_lerp(p, p.yzx, vec3(0.5)); vec4 s = cultmath_saturate(vec4(p, a)); c = vec4(b, cultmath_saturate(a)) + s + vec4(cultmath_snoise(p)); }\n'; } > es300.frag +glslangValidator es300.frag > es300.txt 2>&1; echo "glslang ES 3.00 fragment: exit $? $(grep -m3 ERROR es300.txt | tr '\n' ' ' | cut -c1-300)" +sed -i 's/cultmath_saturate(a))/cultmath_saturate(ivec2(1)))/' es300.frag; glslangValidator es300.frag > neg.txt 2>&1; echo "negative (no ivec2 overload) refused: exit $? $(grep -m1 ERROR neg.txt | cut -c1-160)" +step "4 split, constants, capability strip" +python3 $P/split.py $SH $G $W/split 2>&1 | head -6 +for combo in "core" "core cycles" "core phacelle" "core cycles phacelle"; do f=$W/combo.frag; { printf '#version 300 es\nprecision highp float;\nprecision highp int;\n'; for c in $combo; do cat $W/split/$c.glsl; done; printf 'out highp vec4 c;\nvoid main() { c = vec4(cultmath_snoise(gl_FragCoord.xyz)) + vec4(cultmath_lerp(gl_FragCoord.xyz, vec3(1.0), vec3(0.5)), 1.0); }\n'; } > $f; glslangValidator $f > $W/c.txt 2>&1; echo "glslang [$combo]: exit $? $(grep -m2 ERROR $W/c.txt | tr '\n' ' ' | cut -c1-200)"; done +echo "static const lines: $(grep -hcE '^static const' $SH/*.hlsl | awk '{s+=$1}END{print s}'); of form 'static const float CULTMATH_X = lit;': $(grep -hcE '^static const float CULTMATH_\w+ = [-0-9.eEfx]+;' $SH/*.hlsl | awk '{s+=$1}END{print s}'); CULTMATH_ in output $(grep -c 'CULTMATH_' $G); in committed glsl $(cat $SH/*.glsl | grep -cE '^(const )?\w* ?float CULTMATH_')" +DXC_PATH=$DXC bash $P/lava.sh +echo PROBE-DONE diff --git a/docs/research/native-shader-probes/overload-wrappers/wrapgen.py b/docs/research/native-shader-probes/overload-wrappers/wrapgen.py new file mode 100644 index 00000000..e0a4cbf8 --- /dev/null +++ b/docs/research/native-shader-probes/overload-wrappers/wrapgen.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python3 +"""Probe: build-only export wrappers. One exported, uniquely named wrapper per top-level cultmath_ definition +(texture-taking ones skipped). Source is copied unchanged. usage: wrapgen.py +Writes /cm_wrappers.hlsl and /wrappers.tsv (wrapper, name, file, params).""" +import os, re, sys, shutil +sh, out = sys.argv[1:3]; shutil.copytree(sh, out, dirs_exist_ok=True) +SIG = re.compile(r'(?m)^(\w+)\s+(cultmath_\w+)\s*\(([^)]*)\)') +files = ['CultMath.hlsl', 'CultMath.Interval.hlsl', 'CultMath.Affine.hlsl', 'CultMath.Cycles.hlsl', 'CultMath.Phacelle.hlsl'] +w = ['#include "CultMath.hlsl"']; rows = []; k = 0 +for fn in files: + for m in SIG.finditer(open(os.path.join(sh, fn)).read()): + ret, name, ps = m.group(1), m.group(2), m.group(3) + if re.search(r'Texture|Sampler', ps): continue + params = [p.strip() for p in ps.split(',') if p.strip()] + args = [re.split(r'\s+', p)[-1] for p in params] + wn = 'cmw%03d_%s' % (k, name); k += 1 + call = '%s(%s)' % (name, ', '.join(args)) + w.append('export %s %s(%s) { %s%s; }' % (ret, wn, ', '.join(params), '' if ret == 'void' else 'return ', call)) + rows.append('\t'.join([wn, name, fn, ', '.join(params)])) +open(os.path.join(out, 'cm_wrappers.hlsl'), 'w').write('\n'.join(w) + '\n') +open(os.path.join(out, 'wrappers.tsv'), 'w').write('\n'.join(rows) + '\n') +print('wrappers', k)