map: recalibration wording after Soul s3 (ja effect, R, mild arms, noise stratum)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Weksa 2026-11-13 07:33:08 +00:00
commit 2b29efbc2d

View file

@ -450,55 +450,66 @@ en-US and ja-JP, round 1 recorded by briefed subagents; 6 prompted draws
including the recorded one, 2 mild degradations, an all-"Yes." control and 4
Train-guide draws by Claude sonnet-5-5; the same decision rule at FLOOR_Z 3 and
3-arm floors; 120 prompted trials from the 30 ordered pairs of 6 draws with the
other 4 as floor candidates, so trials share draws and are not independent):
other 4 as floor candidates, so trials share draws and are not independent).
Soul pass `cut-validation-growth.s3` re-derived every number below; its
findings shaped the wording.
- Live Validation items: 28 en-US and 32 ja-JP of 35 each; 10 are refused at
parse (en-US original:18, 23, 25, 27 and vgrow:4, 5, 6; ja-JP original-ja:17,
22 and vgrow-ja:2) and score 0 in every arm. Old 14: 10 en-US and 12 ja-JP
live; new 21: 18 and 20.
22 and vgrow-ja:2); `relower` skips an item with no packet, so every reading
here is over live items only. Old 14: 10 en-US and 12 ja-JP live; new 21: 18
and 20.
- Prompted repeat: NoGain 96.7%, Accepted 3.3%, Regressed 0% (14 items: 92 / 0 /
8). The false-accept bound is no better than before: 3.3% rests on 120 trials
over 6 draws.
8). All four false accepts are one pair, base p6 and candidate the recorded
lowering R: R comes from the round, the other five draws from `relower`, and
in en-US R scores +0.024 per item above them [+0.002, +0.052], so R is not
exchangeable with a repeat (a Plausible Soul finding, cause not isolated). No
false-accept bound can be drawn from 6 draws.
- Degraded arms read Regressed: all-"Yes." 100% (as before); mild-1 31.7%;
mild-2 0% (14 items: 100% and 75% on a subtler arm). Not comparable: these
mild arms were rewritten from the recorded lowering by blurring one detail
and move the mean composite by only 0.003 to 0.015, against 0.044 for the old
subtler arm; the original mild wording is not on disk, so this is a weaker
degradation, and the rule's sensitivity to a real small regression is
unchanged in kind but not re-measured at the old strength.
- Noise: the panel noise term of the sum rule is 0.045 on average over the
base and floor choices (0.031 to 0.064), against 0.079 at 14 items; by
1/sqrt(n) the extrapolation said near 0.05. That part of the extrapolation
held.
mild-2 0% (14 items: 100% and 75% on a subtler arm). The mild arms are weak
and not independent: they were rewritten from R with one detail blurred, the
original wording is not on disk, mild-2 equals R word for word in 5 of 60
items and mild-1 in 2, and mild-1 equals mild-2 in 27 of 60 although they
were asked to blur different details. Their mean composite is 0.003 to 0.015
below the prompted mean (the old subtler arm: 0.037 and 0.052). The
sensitivity to a small regression is not re-measured.
- Noise: the panel noise term of the sum rule is 0.045 on average over the base
and floor choices (0.031 to 0.064), against 0.079 at 14 items. The 1/sqrt(n)
forecast (about 0.05) assumed per-item variance stayed the same; it did not:
the per-item SD of the difference between two prompted draws is about 0.085
on the old items and 0.034 on the new ones, so the pooled term landed near
the forecast because less variable items were added. Not a confirmation of
the forecast.
- Power: the Train-guide arm (14 Train sentences as a register guide) is
Accepted in 21.2% of 240 trials, NoGain 78.8%, against 29% at 14 items, and
its mean gain over the prompted draws fell from 0.079 to 0.018 (gain over
noise 0.37). Split by item class (paired bootstrap over items, 95%): the
gain is on the old Validation items, composite +0.029 [+0.005, +0.052] (en-US
+0.049, ja-JP +0.013 with an interval through zero; 22 items), and nil on the
new ones, -0.002 [-0.011, +0.007] (en-US -0.017 [-0.027, -0.007], ja-JP
+0.011; 38 items). The old items share a writer, a vocabulary and a register
with Train; the new ones were written free of both. So the 14-item power
figure measured a gain that was in-sample register fit on text shaped like
the guide, and the "better arm" premise does not hold on independent
Validation. This is a measured difference; its cause is inferred, not
isolated. The power of the rule against a gain that exists is therefore not
re-measured: no better arm is known on the new items.
its mean composite gain over the prompted draws is 0.0185, 0.41 of the mean
noise term (0.37 on the first floor choice per base); with `relower`-only
bases (R removed) it is Accepted in 35% (28 of 80). Split by item class
(paired bootstrap over items, 95%): old items +0.029 [+0.005, +0.052] (en-US
+0.049 [+0.023, +0.084] on 10 items, ja-JP +0.013 [-0.021, +0.037] on 12);
new items -0.002 [-0.011, +0.007] pooled, but two opposite effects: en-US
-0.017 [-0.027, -0.007] on 18 items and ja-JP +0.011 [+0.001, +0.022] on 20.
So the old-versus-new contrast exists in en-US; in ja-JP old (+0.013) and new
(+0.011) do not differ. Measured: the 14-item gain does not carry to the new
en-US items. Inferred, not isolated: that it was in-sample register fit. The
new items differ from the old in several ways at once (written free of the
inventory, longer, a different scene convention, lower scores, a smaller
vocabulary overlap with Train in en-US: 0.29 against 0.52) and were briefed to
the same register as Train, by Claude as the old were. The 29% power figure
stays the stated baseline of the 14-item corpus; no better arm is known on
the new items, so the rule's power against a real gain is not re-measured.
- The two classes are different strata (Soul s2 `register-stratum-shift`): the
new items score lower (composite 0.41 against 0.64 pooled) and vary less per
item between draws (SD of the difference between two prompted draws 0.034
against 0.085), so a pooled 60-item mean mixes them; report old and new
separately.
new items score lower (composite 0.41 against 0.64 pooled) so a pooled
60-item mean mixes them; report old and new separately.
- The self-preference effect is not measured (ruling
`validation-growth-writer-claude-checked`). The new scenes use a different
`in_group` convention from the old ones (follow-up
`vgrow-scene-in-group-convention`), so the new-item scores are for scenes that
will be re-annotated; the corpus digest changes again then.
Measured: everything listed above. Extrapolated: nothing new; the 1/sqrt(n)
noise forecast is now a measurement, the old power figure is withdrawn as
in-sample. Evidence: `cal2/` and `runs/growth-base` in the Weksa Self
workspace, not in git.
Evidence lives in the Self workspace (`cal2/`, `runs/growth-base`), not in git,
and uses a DUMP patch kept out of the repo; no committed harness reruns these
numbers (Soul finding `evidence-outside-git`, follow-up owed).
### Smallest measurable distil slice (`distil-incontext`, not built)