docs: say what loop calibrate produces and what stayed in scratch scripts; restore the mild brief's diversity clause and warn

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Weksa 2026-11-13 08:49:23 +00:00
commit 541bfeec05
2 changed files with 31 additions and 18 deletions

View file

@ -5,10 +5,16 @@ Validation corpus: how often it accepts, rejects as no gain, or regresses a
candidate that should be none of those. It does not reimplement the rule. It
reads the answered draws through `RunStores::read_draws` (the scoring
`weksa loop relower` uses), decides every trial with `decide_panel` over
`paired_floor` (`crates/weksa/src/run.rs`), and prints counts. The numbers in
`docs/targets/interlingua-lowering-map.md` ("Recalibration on the grown
corpus") come from it. It is a diagnostic: it records nothing, writes no
prompts, and never touches Test or Train.
`paired_floor` (`crates/weksa/src/run.rs`), and prints, per class, the items
read and skipped per language, the trial totals with their Accepted, NoGain and
Regressed counts, and the mean panel noise. That is all it produces: the
Accepted/NoGain/Regressed counts and the noise mean in the map's "Recalibration
on the grown corpus" block are reproduced by it; the other figures there (the
35% with R removed, the noise range, the 0.0185 gain and its 0.41 ratio, the
bootstrap intervals, the per-item SDs, R's +0.024 and the mild-arm duplicate
counts) were computed by scratch scripts that are not in the repo. It is a
diagnostic: it records nothing, writes no prompts, and never touches Test or
Train.
## What it counts
@ -74,10 +80,14 @@ draw's answers. Only `mild` reads `round-N/*.answer`.
tense, polarity, modality or hedge, a referent, a quantity or a predicate's
concept (replace it with a near but different word). Keep language, register,
length and fluency; the result must read as a natural utterance and must not
mention the change. A second mild draw blurs a different detail than the first
and is written without reading the first's answers. The wording of the
distil-base original is lost, and the dispatch used for growth-base is not on
disk, so this is the wording from now on.
mention the change. Pick the detail that is not the most salient one. A
second mild draw blurs a different detail than the first and is written
without reading the first's answers. The wording of the distil-base original
is lost, and the dispatch used for growth-base is not on disk, so this is the
wording from now on. Warning: on growth-base two mild arms briefed this way
came out identical in 27 of 60 items, so independently briefed mild arms
are not independent degradations; read a mild draw's regress count with that
in mind.
- Train-guide: before answering, read the Train sentences of the corpus in the
batch's language as a register guide and let it steer word choice; the guide
gives phrasing only and each answer must still say exactly what its own packet

View file

@ -508,16 +508,19 @@ findings shaped the wording.
will be re-annotated; the speaker-state store changes then (a run pins its
digest), the corpus does not.
These numbers were first computed by scripts outside git (Soul finding
`evidence-outside-git`). `weksa loop calibrate` (`docs/calibration.md`)
computes them from `run.rs`'s own scoring and accept rule, and its rerun on
`runs/growth-base` reproduces them: prompted 116 NoGain, 4 Accepted, 0
Regressed of 120; control-yes 60 of 60 Regressed, mild-1 19 of 60, mild-2 0 of
60; the Train-guide arm 51 Accepted, 189 NoGain, 0 Regressed of 240; noise term
0.0451; items en-US n=28 skipped=7, ja-JP n=32 skipped=3. By class the harness
adds: old items (`item:original*`) prompted 111/9/0, noise 0.1069, Train-guide
71 Accepted of 240; new items (`item:vgrow*`) prompted 116/4/0, noise 0.0334,
Train-guide 0 Accepted and 74 Regressed of 240.
`weksa loop calibrate` (`docs/calibration.md`) reproduces from `run.rs`'s own
scoring and accept rule the counts per class, the trial totals and the mean
noise of this block: prompted 116 NoGain, 4 Accepted, 0 Regressed of 120;
control-yes 60 of 60 Regressed, mild-1 19 of 60, mild-2 0 of 60; the
Train-guide arm 51 Accepted, 189 NoGain, 0 Regressed of 240; noise 0.0451;
items en-US n=28 skipped=7, ja-JP n=32 skipped=3. By class it adds: old items
(`item:original*`) prompted 111/9/0, noise 0.1069, Train-guide 71 Accepted of
240; new items (`item:vgrow*`) prompted 116/4/0, noise 0.0334, Train-guide 0
Accepted and 74 Regressed of 240. The other figures of this block (35% with R
removed, the noise range, the 0.0185 gain and 0.41 ratio, the bootstrap
intervals, the per-item SDs, R's +0.024, the mild-arm duplicate counts) were
computed by scratch scripts outside git (Soul finding `evidence-outside-git`)
and are not reproduced by the harness.
### Smallest measurable distil slice (`distil-incontext`, not built)