ja-panel-corpus: 28 model-written Japanese passages and 8 Aozora Bunko Test sentences

weksa commented

2026-11-12 13:52:29 +00:00

Member

Cut 9b (spec weksa-lowering:cut_spec:cut-ja-panel-corpus.r1), rebased onto main after #49. data/original gains ja-JP: 14 Train and 14 Validation passages written natively in Japanese (ModelWritten, never Test, on the Dao r3 concept footing, none a rendering of an en-US item or a Dao line) and 8 one-sentence Test excerpts from Aozora Bunko works public domain in Japan and the US, each checked verbatim against its card and pinned in a test. Soul verdict s3 closes the cut; its two Low findings are follow-up ja-corpus-test-lows.

🤖 Generated with Claude Code

Cut 9b (spec weksa-lowering:cut_spec:cut-ja-panel-corpus.r1), rebased onto main after #49. data/original gains ja-JP: 14 Train and 14 Validation passages written natively in Japanese (ModelWritten, never Test, on the Dao r3 concept footing, none a rendering of an en-US item or a Dao line) and 8 one-sentence Test excerpts from Aozora Bunko works public domain in Japan and the US, each checked verbatim against its card and pinned in a test. Soul verdict s3 closes the cut; its two Low findings are follow-up `ja-corpus-test-lows`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

weksa added 3 commits

2026-11-12 13:52:29 +00:00

data/original: ja-JP slice, 28 model-written Japanese passages and 8 Aozora Bunko Test sentences 4dbaef9437

Train and Validation (14 each) are Japanese passages written in Japanese, ModelWritten
Anthropic, generator claude-sonnet-5-5; none sits in a Test group. Test is eight
one-sentence excerpts (Soseki, Kyoka, Akutagawa, Ogai, Arishima, Kenji, Kajii, Kunikida)
taken from the card ruby zips (cp932, ruby and the pipe marker dropped, NFC) and checked
verbatim against each card's XHTML file; licence strings keep 底本, 入力 and 校正 within
the store's 200-byte cap. Entities appended (keeper gains 番人; eight pd-* authors).
Items 1..28 and holdout:1..8 unchanged (digest pinned). Tests per language: counts,
provenance, Test public-domain only, no markup, panel round reads both languages, speaker
forms, recall 56, no Test text in any prompt.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

data/original: ja passages rewritten onto the Dao r3 concepts; ja Test sentences pinned verbatim e30ee3886a

The 28 Train/Validation ja passages are new Japanese sentences on the concepts the
pinned inventory holds (water, mind, belly, bone, sage, will, heaven, earth, fill, empty,
weaken, strengthen, compare, resemble ...), written in Japanese idiom, not translated.
The branch was unmerged, so corpus.cc was restored to 21eaa07 and rebuilt through
weksa corpus add; holdout-ja:1..8 and original:1..28 come out byte-identical.

Tests: the eight ja Test texts, groups, speakers, split and edition URLs are literal
expectations; every passage of both languages rests on a live inventory concept
(en 28/28, 18 concepts; ja 28/28, 21 concepts).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

data/original: ja passages replaced, none a rendering of an en-US original or a Dao line 20863615b3

All 28 Train/Validation ja passages are new Japanese sentences on the inventory's
concepts with their own imagery (paddies, charcoal, kites, the bath, the market); none
renders an en-US original, and none restates a data/daodejing line (ch3, 5, 7, 8, 78) or
other canon (the old ones on 'true words', 'long-enduring', 'bellows', 'water vs hard
things' are gone). corpus.cc restored from 21eaa07 and rebuilt through weksa corpus add:
numbering 1..28 unbroken, holdout-ja:1..8 and original:1..28 byte-identical.

The concept test now matches Japanese by multi-character surface and English by word
stem, strips false friends (天気, 心配, 腹が立つ, 水曜 ...), and has a negative test.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

weksa merged commit d9ebbcf889 into main

2026-11-12 13:52:37 +00:00

weksa referenced this pull request from a commit

2026-11-12 13:52:38 +00:00

Merge pull request 'ja-panel-corpus: 28 model-written Japanese passages and 8 Aozora Bunko Test sentences' (#50) from weksa-lowering/ja-panel-corpus into main

Rows
Columns

ja-panel-corpus: 28 model-written Japanese passages and 8 Aozora Bunko Test sentences #50