ontology-mint-probe: Train-only ontology mint call (loop mint-ontology) #63

Merged
weksa merged 3 commits from weksa/ontology-mint-probe into main 2026-11-12 22:20:44 +00:00
7 changed files with 595 additions and 4 deletions

View file

@ -14,6 +14,11 @@
//! entries that `ontology::select` chose for the view; the caller never names
//! entries.
//!
//! An ontology mint prompt is a second Train-only prompt: the [`TrainItems`] of
//! one language, the inventory's Stance, Discourse and Scene features (the only
//! ones a condition can read) and that language's existing entry lines. It reads
//! no packet, no score and no held-out text.
//!
//! Item text may hold line breaks; the renderer indents every continuation line
//! (`indented`), so item text cannot forge a `[key]` line or a paragraph.
//! Item blocks are data from the run's pinned corpus, reached only through
@ -28,7 +33,7 @@ use std::marker::PhantomData;
use serde::Serialize;
use crate::corpus::{CorpusItem, CorpusItemId};
use crate::inventory::{Inventory, InventoryEntry, OptionLabel, ValueKind};
use crate::inventory::{Inventory, InventoryEntry, OptionLabel, Slot, ValueKind};
use crate::lower::{EntityForms, LowerError, entity};
use crate::ontology::{Ontology, select};
use crate::packet::{
@ -82,6 +87,13 @@ pub const RECALL_INSTRUCTIONS: &str = "\
Below is the opening of a passage, with what is known of where it comes from. Continue it exactly as its source words it, from memory, until the passage ends. If you do not remember it, give your best guess in the same voice.
Answer with one JSON object and nothing else: {\"text\": \"...\"}, the continuation only, not the opening. No code fence, no commentary.";
/// The instructions of an ontology mint: drafting entries for one language from
/// its Train lines. Fixed per run by `PROMPT_SOURCE_DIGEST`.
pub const MINT_ONTOLOGY_INSTRUCTIONS: &str = "\
Below are lines written in one language (the language line), the features an ontology condition may test, and the entries that language's ontology already has. Draft new entries: choices this language's culture makes, for a speaker in a stance and scene, about word, grammar, discourse, sound or script, that the lines show.
Answer with one JSON object and nothing else: {\"entries\": [{\"layer\": \"...\", \"when\": [...], \"choice\": \"...\", \"examples\": [...]}]}. No code fence, no commentary.
Give 1 to 8 entries. layer is one of lexicon, grammar, discourse, phonology, orthography. when holds at most 4 conditions, each {\"slot\": \"Stance\" | \"Discourse\" | \"Scene\", \"feature\": \"<id>\", \"test\": {\"AtLeast\": <permille>} | {\"AtMost\": <permille>} | {\"AtMostOrUnset\": <permille>} | {\"Is\": <value>}}, over the listed feature ids only; when may be empty when a choice holds for every line of the language. choice states a pattern the lines show, in at most 200 bytes, without quoting a line. examples are original short phrases of the language, never copied from the lines below, at most 3.";
/// The live Train items of a corpus, in key order. The only input minting
/// reads; it cannot be built from a slice, and no accessor returns text or an
/// item.
@ -502,6 +514,56 @@ impl Prompt {
}
}
/// [`MINT_ONTOLOGY_INSTRUCTIONS`], the `language` line, the Stance,
/// Discourse and Scene features of `inventory` live at `rev` (grade, flag or
/// options: all a condition can read), the entries `ontology` already holds
/// for `language` as `<key> <choice>`, then the Train items of exactly
/// `language` in key order without prior turns. `None` when there is none.
pub(crate) fn mint_ontology(
train: &TrainItems,
language: &str,
inventory: &Inventory,
rev: u32,
ontology: &Ontology,
) -> Option<Prompt> {
let items: Vec<&CorpusItem> = train
.items
.iter()
.filter(|item| item.language == language)
.collect();
if items.is_empty() {
return None;
}
let features: String = inventory
.entries()
.filter(|e| {
e.live_at(rev)
&& e.value_kind.is_some()
&& e.applies_to
.iter()
.any(|s| matches!(s, Slot::Stance | Slot::Discourse | Slot::Scene))
})
.map(|e| format!("\n{}", entry_line(e)))
.collect();
let existing: String = ontology
.entries()
.iter()
.filter(|e| e.language == language)
.map(|e| format!("\n{} {}", e.key(), e.choice))
.collect();
let mut parts = vec![
MINT_ONTOLOGY_INSTRUCTIONS.to_string(),
format!("language: {language}"),
format!("features rev:{rev}{features}"),
format!("entries{existing}"),
];
parts.extend(items.iter().map(|item| block(item, None)));
Some(Prompt {
text: parts.join("\n\n"),
items: items.iter().map(|item| item.id.as_str().to_string()).collect(),
})
}
/// [`ENCODE_INSTRUCTIONS`], the one offered item with its prior turns resolved
/// against `train`, the entities a packet for it may cite, and the entries
/// of `inventory` live at `rev`.
@ -663,6 +725,18 @@ mod tests {
);
}
#[test]
fn an_ontology_mint_over_no_item_of_the_language_is_no_prompt() {
let inv = Inventory::default();
let train = TrainItems { items: vec![] };
assert!(Prompt::mint_ontology(&train, "en-US", &inv, 1, &Ontology::empty()).is_none());
let mut held = item("item:a:1", "x", &group_in(Split::Train), false);
held.language = "ja-JP".into();
let train = TrainItems { items: vec![held] };
assert!(Prompt::mint_ontology(&train, "en-US", &inv, 1, &Ontology::empty()).is_none());
assert!(Prompt::mint_ontology(&train, "ja-JP", &inv, 1, &Ontology::empty()).is_some());
}
#[test]
fn a_mint_prompt_over_no_train_items_is_the_brief_the_feedback_and_the_listing() {
let none = TrainItems { items: vec![] };

View file

@ -15,7 +15,7 @@ use weksa::packet::{FeatValue, FeatureValue, Grade};
use weksa::packet::proof_packet;
use weksa::model::RecordedAnswers;
use weksa::run::{
AblationOutcome, CycleOutcome, LowererReach, RecallOutcome, RoundOutcome, RunPaths, RunStores, cycle, round,
AblationOutcome, CycleOutcome, LowererReach, OntologyMintOutcome, RecallOutcome, RoundOutcome, RunPaths, RunStores, cycle, round,
};
use weksa::score::ScoreInput;
use weksa::{
@ -131,6 +131,15 @@ enum LoopCommand {
/// half). Prompts and answers live in `<ROOT>/recall`; the command exits 3
/// until every prompt has an answer. A diagnostic, never a gate.
Recall { root: PathBuf },
/// Draft ontology entries from the run's Train items of one language.
/// Prompts and answers live in `<ROOT>/ontology-mint`; the command exits 3
/// until the prompt has an answer. Admission is `weksa ontology admit`
/// into a copy, never into the run's pinned file.
MintOntology {
root: PathBuf,
#[arg(long)]
language: String,
},
/// Ask whether the lowerer uses a packet's stance: for each live Train and
/// Validation packet of round ROUND, ask again with the stance as it is,
/// emptied, and swapped for the next item's, and read how far the lowering
@ -1122,6 +1131,24 @@ fn main() -> Result<()> {
RecallOutcome::Read(reading) => println!("{}", reading.line()),
}
}
Command::Loop(LoopCommand::MintOntology { root, language }) => {
let run = RunStores::open(&root)?;
let mut files = AnswerFiles::new(run.ontology_mint_dir(), run.record().model.clone());
match run.mint_ontology(&mut files, &language)? {
OntologyMintOutcome::Unanswered { prompt } => {
println!("unanswered {prompt}");
std::process::exit(3);
}
OntologyMintOutcome::Drafted { prompt, entries } => println!(
"drafted {entries} entries {}",
run.ontology_mint_dir().join(format!("{prompt}.answer")).display()
),
OntologyMintOutcome::Refused { code } => {
println!("{code}");
std::process::exit(1);
}
}
}
Command::Loop(LoopCommand::Cycle { root, ollama_url }) => {
let unreadable = |_| anyhow::anyhow!("a store would not read");
let mut run = RunStores::open(&root)?;

View file

@ -40,7 +40,9 @@ use unicode_segmentation::UnicodeSegmentation;
use crate::context::{EditFeedback, EncoderOffer, Prompt, TrainItems};
use crate::corpus::{CorpusItem, CorpusItemId, ModelFamily, Source};
use crate::encode::{EditDraft, LoweringDraft, MAX_EDIT_CHANGES, PacketDraft, encoder_id};
use crate::encode::{
EditDraft, LoweringDraft, MAX_EDIT_CHANGES, OntologyDraft, PacketDraft, encoder_id,
};
use crate::form::{is_attestation, is_language_tag, is_slug};
use crate::model::{ModelPort, PortError};
use crate::packet::{Packet, Stance};
@ -1157,6 +1159,65 @@ impl RunStores {
}
}
/// What an ontology mint came to: the call's prompt digest, never an entry
/// admitted.
#[derive(Debug, PartialEq, Eq)]
pub enum OntologyMintOutcome {
/// The prompt is written; nobody has answered it.
Unanswered { prompt: String },
/// The answer reads as a draft of `entries` entries; the answer file is the
/// draft, for `weksa ontology admit`.
Drafted { prompt: String, entries: usize },
/// Nothing asked, or the answer is no draft: a closed code.
Refused { code: &'static str },
}
impl RunStores {
/// Asks `model` to draft ontology entries for `language` from the run's
/// Train items of that language. Writes the prompt and answer files of
/// `model` and nothing else: the draft is admitted, if ever, by
/// `weksa ontology admit` into a copy of the ontology, never here.
pub fn mint_ontology(
&self,
model: &mut dyn ModelPort,
language: &str,
) -> Result<OntologyMintOutcome, RunError> {
let store = |_| RunError::Store;
if !self.record.panel.iter().any(|tag| tag == language) {
return Ok(OntologyMintOutcome::Refused { code: "language-not-in-panel" });
}
let inventory = self.inventory.inventory().map_err(store)?;
let rev = inventory.latest().ok_or(RunError::NoInventory)?.n;
let ontology = self.ontology.ontology().map_err(store)?;
let train = self.train().map_err(store)?;
// A panel language has a live Train item (`create` requires it and the
// corpus is pinned), so no prompt here is a store that broke its pin.
let prompt = Prompt::mint_ontology(&train, language, &inventory, rev, &ontology)
.ok_or(RunError::Malformed("ontology-mint train"))?;
match model.answer(&prompt) {
Ok(answer) => Ok(match OntologyDraft::parse(&answer) {
Ok(draft) => OntologyMintOutcome::Drafted {
prompt: prompt.digest(),
entries: draft.entries.len(),
},
Err(e) => OntologyMintOutcome::Refused { code: e.class.code() },
}),
Err(PortError::Unanswered) => Ok(OntologyMintOutcome::Unanswered {
prompt: prompt.digest(),
}),
Err(PortError::Io(ErrorKind::InvalidData)) => {
Ok(OntologyMintOutcome::Refused { code: "invalid-data" })
}
Err(PortError::Io(kind)) => Err(RunError::Io(kind)),
}
}
/// `<root>/ontology-mint`, where a mint's prompts and answers live.
pub fn ontology_mint_dir(&self) -> PathBuf {
self.root.join("ontology-mint")
}
}
/// What the lowerer's answer did to each variant of a packet's stance, per
/// split. Means only; no key, text or stance value.
#[derive(Clone, Debug, PartialEq)]

View file

@ -1452,6 +1452,50 @@ fn open_dao_run(d: &std::path::Path) {
assert!(o.status.success(), "{}", out(&o));
}
/// `weksa loop mint-ontology`: exit 3 and a prompt file until answered, exit 0
/// and the answer path once it is a draft, a missing language or one outside
/// the panel refused, and nothing of the run written.
#[test]
fn loop_mint_ontology_protocol() {
let d = tempfile::tempdir().unwrap();
open_dao_run(d.path());
let root = d.path().join("canaryroot");
let held = |name: &str| std::fs::read(root.join(name)).unwrap();
let (run_cc, onto) = (held("run.cc"), std::fs::read(d.path().join("o.cc")).unwrap());
let mint = |args: &[&str]| weksa(d.path(), &[&["loop", "mint-ontology", "canaryroot"], args].concat());
let none = mint(&[]);
assert_eq!(none.status.code(), Some(2), "{}", out(&none));
assert!(!root.join("ontology-mint").exists());
let outside = mint(&["--language", "ja-JP"]);
assert_eq!(outside.status.code(), Some(1), "{}", out(&outside));
assert_eq!(out(&outside), "language-not-in-panel\n");
assert!(!root.join("ontology-mint").exists(), "a refused mint wrote a prompt");
let first = mint(&["--language", "en-US"]);
assert_eq!(first.status.code(), Some(3), "{}", out(&first));
let text = out(&first);
let digest = text.strip_prefix("unanswered ").unwrap().trim_end();
assert_eq!(digest.len(), 16, "{text}");
let dir = root.join("ontology-mint");
assert!(dir.join(format!("{digest}.prompt")).is_file());
let answer = dir.join(format!("{digest}.answer"));
std::fs::write(&answer, r#"{"entries":[{"layer":"Grammar","choice":"a pattern"}]}"#).unwrap();
let done = mint(&["--language", "en-US"]);
assert_eq!(done.status.code(), Some(0), "{}", out(&done));
assert_eq!(out(&done), format!("drafted 1 entries canaryroot/ontology-mint/{digest}.answer\n"));
std::fs::write(&answer, "not json").unwrap();
let bad = mint(&["--language", "en-US"]);
assert_eq!(bad.status.code(), Some(1), "{}", out(&bad));
assert_eq!(out(&bad), "syntax\n");
assert_eq!(held("run.cc"), run_cc, "the mint wrote the run record");
assert_eq!(std::fs::read(d.path().join("o.cc")).unwrap(), onto);
}
/// The family `loop open` pinned at `root`, read from the run record.
fn pinned_family(d: &std::path::Path, root: &str, flag: &[&str]) -> weksa::corpus::ModelFamily {
let mut args = vec![

View file

@ -692,6 +692,29 @@ mod lowering {
}
}
/// The mint cut added no line to what the lowerer sees: a prompt with a
/// selected ontology entry has the digest it had at af0a56d.
#[test]
fn the_lowering_prompt_of_a_fixed_packet_is_unchanged_by_the_ontology_mint() {
use weksa::encode::OntologyDraft;
use weksa::speaker::Annotator;
let (view, inv, entities, item) = dao(2);
let store = InventoryStore::open(&data("daodejing/inventory.cc")).unwrap();
let dir = tempfile::tempdir().unwrap();
let mut onto = weksa::OntologyStore::open(&dir.path().join("o.cc")).unwrap();
let annotator = Annotator::Model {
family: weksa::corpus::ModelFamily::Anthropic,
generator: "golden".into(),
};
let draft = r#"{"entries":[{"layer":"Grammar","choice":"golden choice","examples":["golden example"]}]}"#;
let entries = OntologyDraft::parse(draft).unwrap().into_entries("en-US", 3, 1, &annotator);
onto.admit(entries, &store).unwrap();
let prompt =
Prompt::lower(&view, &item, "en-US", &inv, &entities, &onto.ontology().unwrap()).unwrap();
assert!(prompt.text().contains("golden choice"));
assert_eq!(prompt.digest(), "4a40eb57282b059d");
}
#[test]
fn lower_prompt_moves_with_the_stance_alone() {
use weksa::packet::{FeatValue, FeatureValue, Grade, Stance};

View file

@ -0,0 +1,362 @@
//! The ontology mint (`weksa loop mint-ontology`): a Train-only call that drafts
//! entries for `weksa ontology admit`. It shows a model the Train lines of one
//! panel language, the features a condition can read and that language's entry
//! lines; it admits nothing and writes nothing of the run.
mod common;
use std::path::Path;
use common::{cast, entities_in, group_in_bucket, item_in};
use weksa::context::Prompt;
use weksa::corpus::{ModelFamily, Source};
use weksa::inventory::{Change, seed};
use weksa::model::{AnswerFiles, ModelPort, PortError};
use weksa::run::{LowererReach, OntologyMintOutcome, RunPaths, RunStores};
use weksa::speaker::Annotator;
use weksa::{CorpusStore, InventoryStore, OntologyStore, Slot, ValueKind};
/// Records every prompt shown; answers with `reply`.
struct Record {
prompts: Vec<(String, Vec<String>)>,
reply: String,
}
impl Record {
fn new(reply: &str) -> Self {
Self { prompts: vec![], reply: reply.to_string() }
}
}
impl ModelPort for Record {
fn model(&self) -> &str {
"record"
}
fn answer(&mut self, p: &Prompt) -> Result<String, PortError> {
self.prompts.push((p.text().to_string(), p.item_keys().to_vec()));
Ok(self.reply.clone())
}
}
fn public() -> Source {
Source::PublicDomain {
work: "A Work".into(),
author: "An Author".into(),
edition_url: "https://example.org/e".into(),
licence: "CC0".into(),
jurisdiction: "US".into(),
translator: "A Translator".into(),
edition_year: 1900,
}
}
const STANCE_G: &str = "feature:60";
const DISC_FLAG: &str = "feature:61";
const REF_ONLY: &str = "feature:62";
const PRED_ONLY: &str = "feature:63";
const STANCE_RETIRED: &str = "feature:64";
const SCENE_G: &str = "feature:65";
/// A fixture: items in en-US and ja-JP across every split and a withdrawal, an
/// inventory with condition-readable and unreadable features, an ontology with
/// en-US and ja-JP entries, and a run over them with `panel`.
fn fixture(dir: &Path, panel: &[&str]) -> RunStores {
let entities = entities_in(dir);
let mut corpus = CorpusStore::open(&dir.join("c.cc")).unwrap();
let rows = [
("item:en:1", "en-US", "entrain mark one", 10, cast(), false),
("item:en:2", "en-US", "entrain mark two", 11, cast(), false),
("item:en:3", "en-US", "envalid mark three", 50, cast(), false),
("item:en:4", "en-US", "engone mark four", 12, cast(), true),
("item:en:5", "en-US", "entest mark five", 90, public(), false),
("item:ja:1", "ja-JP", "jatrain mark one", 13, cast(), false),
("item:ja:2", "ja-JP", "javalid mark two", 51, cast(), false),
("item:ja:3", "ja-JP", "jatest mark three", 91, public(), false),
];
for (id, language, text, bucket, source, withdrawn) in rows {
let mut item = item_in(id, text, &group_in_bucket(bucket), source);
item.language = language.into();
corpus.add(item, &entities).unwrap();
if withdrawn {
corpus.withdraw(id).unwrap();
}
}
drop(corpus);
let mut inv = InventoryStore::open(&dir.join("i.cc")).unwrap();
let (rev1, entries) = seed();
inv.admit_revision(rev1, entries).unwrap();
let f = |id, kind, slots: &[Slot]| common::feature(id, kind, slots);
let features = vec![
f(STANCE_G, ValueKind::Grade, &[Slot::Stance]),
f(DISC_FLAG, ValueKind::Flag, &[Slot::Discourse]),
f(REF_ONLY, ValueKind::Grade, &[Slot::Referent]),
f(PRED_ONLY, ValueKind::Grade, &[Slot::Predication]),
f(STANCE_RETIRED, ValueKind::Grade, &[Slot::Stance]),
f(SCENE_G, ValueKind::Grade, &[Slot::Scene]),
];
let changes = features.iter().map(|e| Change::Add(e.id)).collect();
let mut features = features;
for e in &mut features {
e.introduced_rev = 2;
}
inv.admit_revision(common::rev_with(changes, 2, Some(1)), features).unwrap();
inv.admit_revision(
common::rev_with(vec![Change::Retire(common::iv(STANCE_RETIRED))], 3, Some(2)),
vec![],
)
.unwrap();
let mut onto = OntologyStore::open(&dir.join("ontology.cc")).unwrap();
let annotator = Annotator::Model { family: ModelFamily::Anthropic, generator: "seed".into() };
let entry = |language: &str, choice: &str, examples: &[&str]| {
weksa::encode::OntologyDraft::parse(&format!(
r#"{{"entries":[{{"layer":"Grammar","choice":"{choice}","examples":[{}]}}]}}"#,
examples.iter().map(|x| format!("\"{x}\"")).collect::<Vec<_>>().join(",")
))
.unwrap()
.into_entries(language, 3, 1, &annotator)
};
onto.admit(entry("en-US", "enchoice marker", &["enexample marker"]), &inv).unwrap();
onto.admit(entry("ja-JP", "jachoice marker", &["jaexample marker"]), &inv).unwrap();
drop(onto);
drop(inv);
std::fs::write(dir.join("speaker-state.cc"), [0x90]).unwrap();
let panel: Vec<String> = panel.iter().map(|s| s.to_string()).collect();
RunStores::create(
&dir.join("run"),
"t",
RunPaths {
corpus: &dir.join("c.cc"),
inventory: &dir.join("i.cc"),
entities: &dir.join("entities.cc"),
ontology: &dir.join("ontology.cc"),
speaker_state: &dir.join("speaker-state.cc"),
},
"fake-model",
"bag",
LowererReach::ToolLessRelay,
ModelFamily::Unrecorded,
&panel,
)
.unwrap()
}
const DRAFT: &str =
r#"{"entries":[{"layer":"Discourse","when":[],"choice":"a pattern","examples":[]}]}"#;
#[test]
fn mint_ontology_prompt_shows_only_train_items_of_the_language() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US", "ja-JP"]);
let mut m = Record::new(DRAFT);
run.mint_ontology(&mut m, "en-US").unwrap();
let (text, keys) = &m.prompts[0];
assert!(text.contains("language: en-US"));
assert_eq!(keys, &["item:en:1", "item:en:2"]);
for shown in ["entrain mark one", "entrain mark two"] {
assert!(text.contains(shown), "{shown}");
}
for hidden in [
"envalid", "engone", "entest", "jatrain", "javalid", "jatest", "item:en:3", "item:en:5",
"item:ja",
] {
assert!(!text.contains(hidden), "the en-US mint showed {hidden}");
}
let mut m = Record::new(DRAFT);
run.mint_ontology(&mut m, "ja-JP").unwrap();
let (text, keys) = &m.prompts[0];
assert_eq!(keys, &["item:ja:1"]);
assert!(text.contains("jatrain mark one") && !text.contains("entrain"));
}
#[test]
fn mint_ontology_refuses_a_language_outside_the_panel() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US"]);
let mut m = Record::new(DRAFT);
// ja-JP has Train items in the corpus but is not in the panel.
for (language, code) in [("ja-JP", "language-not-in-panel"), ("fr-FR", "language-not-in-panel")] {
assert_eq!(
run.mint_ontology(&mut m, language).unwrap(),
OntologyMintOutcome::Refused { code }
);
}
assert!(m.prompts.is_empty(), "a refused mint asked the model");
}
#[test]
fn mint_ontology_prompt_lists_only_condition_readable_features() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US"]);
let mut m = Record::new(DRAFT);
run.mint_ontology(&mut m, "en-US").unwrap();
let text = &m.prompts[0].0;
let listed = |id: &str| text.contains(&format!("\n{id}: "));
for id in [STANCE_G, DISC_FLAG, SCENE_G] {
assert!(listed(id), "{id} is readable by a condition");
}
for id in [REF_ONLY, PRED_ONLY, STANCE_RETIRED] {
assert!(!listed(id), "{id} is not");
}
// Nothing that is not a feature: no concept or role line.
assert!(!text.contains("\nconcept:") && !text.contains("\nrole:"));
assert!(text.contains("features rev:3"));
}
#[test]
fn mint_ontology_prompt_lists_only_the_languages_existing_entries() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US", "ja-JP"]);
let mut m = Record::new(DRAFT);
run.mint_ontology(&mut m, "en-US").unwrap();
let text = &m.prompts[0].0;
assert!(text.contains("onto:en-US:1 enchoice marker"));
for hidden in ["jachoice", "onto:ja-JP", "enexample", "jaexample"] {
assert!(!text.contains(hidden), "{hidden}");
}
}
fn digests(dir: &Path) -> Vec<(String, Vec<u8>)> {
let mut out = vec![];
let mut stack = vec![dir.to_path_buf()];
while let Some(d) = stack.pop() {
for e in std::fs::read_dir(&d).unwrap() {
let p = e.unwrap().path();
if p.is_dir() {
stack.push(p);
} else if !p.starts_with(dir.join("run").join("ontology-mint")) {
out.push((p.display().to_string(), std::fs::read(&p).unwrap()));
}
}
}
out.sort();
out
}
#[test]
fn mint_ontology_writes_nothing_but_its_prompt_and_answer_files() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US", "ja-JP"]);
let before = digests(dir.path());
let mut files = AnswerFiles::new(run.ontology_mint_dir(), "m".into());
let OntologyMintOutcome::Unanswered { prompt } =
run.mint_ontology(&mut files, "en-US").unwrap()
else {
panic!("unanswered");
};
std::fs::write(run.ontology_mint_dir().join(format!("{prompt}.answer")), DRAFT).unwrap();
assert!(matches!(
run.mint_ontology(&mut files, "en-US").unwrap(),
OntologyMintOutcome::Drafted { entries: 1, .. }
));
assert_eq!(digests(dir.path()), before, "the mint wrote outside ontology-mint");
let mut names: Vec<String> = std::fs::read_dir(run.ontology_mint_dir())
.unwrap()
.map(|e| e.unwrap().file_name().into_string().unwrap())
.collect();
names.sort();
assert_eq!(names, [format!("{prompt}.answer"), format!("{prompt}.prompt")]);
drop(run);
RunStores::open(&dir.path().join("run")).expect("no Drift after a mint");
}
#[test]
fn mint_ontology_unanswered_then_drafted_then_refused() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US"]);
let mut files = AnswerFiles::new(run.ontology_mint_dir(), "m".into());
let OntologyMintOutcome::Unanswered { prompt } =
run.mint_ontology(&mut files, "en-US").unwrap()
else {
panic!("unanswered");
};
let mint = run.ontology_mint_dir();
assert!(mint.join(format!("{prompt}.prompt")).exists());
let answer = mint.join(format!("{prompt}.answer"));
std::fs::write(&answer, DRAFT).unwrap();
assert_eq!(
run.mint_ontology(&mut files, "en-US").unwrap(),
OntologyMintOutcome::Drafted { prompt: prompt.clone(), entries: 1 }
);
let two = r#"{"entries":[{"layer":"Grammar","choice":"a"},{"layer":"Lexicon","choice":"b"}]}"#;
std::fs::write(&answer, two).unwrap();
assert!(matches!(
run.mint_ontology(&mut files, "en-US").unwrap(),
OntologyMintOutcome::Drafted { entries: 2, .. }
));
let unknown = r#"{"entries":[{"layer":"Grammar","choice":"a","key":"x"}]}"#;
std::fs::write(&answer, unknown).unwrap();
assert_eq!(
run.mint_ontology(&mut files, "en-US").unwrap(),
OntologyMintOutcome::Refused { code: "unknown-field" }
);
std::fs::write(&answer, "not json").unwrap();
assert_eq!(
run.mint_ontology(&mut files, "en-US").unwrap(),
OntologyMintOutcome::Refused { code: "syntax" }
);
}
#[test]
fn a_mint_answer_admits_into_a_copy_and_a_new_run_pins_the_copy() {
let dir = tempfile::tempdir().unwrap();
let run = fixture(dir.path(), &["en-US"]);
let pinned = std::fs::read(dir.path().join("ontology.cc")).unwrap();
let pin = run.record().ontology_digest.clone();
let draft = format!(
r#"{{"entries":[{{"layer":"Discourse","when":[{{"slot":"Stance","feature":"{STANCE_G}","test":{{"AtLeast":500}}}}],"choice":"a grown choice","examples":[]}}]}}"#
);
let mut files = AnswerFiles::new(run.ontology_mint_dir(), "m".into());
let OntologyMintOutcome::Unanswered { prompt } =
run.mint_ontology(&mut files, "en-US").unwrap()
else {
panic!("unanswered");
};
let answer = run.ontology_mint_dir().join(format!("{prompt}.answer"));
std::fs::write(&answer, &draft).unwrap();
let copy = dir.path().join("grown.cc");
std::fs::copy(dir.path().join("ontology.cc"), &copy).unwrap();
let inv = InventoryStore::open(&dir.path().join("i.cc")).unwrap();
let mut grown = OntologyStore::open(&copy).unwrap();
let annotator = Annotator::Model { family: ModelFamily::Anthropic, generator: "minter".into() };
let first_n = grown.ontology().unwrap().entries().iter().filter(|e| e.language == "en-US").count() as u32 + 1;
let entries = weksa::encode::OntologyDraft::parse(&std::fs::read_to_string(&answer).unwrap())
.unwrap()
.into_entries("en-US", 3, first_n, &annotator);
grown.admit(entries, &inv).unwrap();
drop(grown);
assert_eq!(std::fs::read(dir.path().join("ontology.cc")).unwrap(), pinned);
let again = RunStores::open(&dir.path().join("run")).unwrap();
assert_eq!(again.record().ontology_digest, pin);
let second = RunStores::create(
&dir.path().join("run-grown"),
"t",
RunPaths {
corpus: &dir.path().join("c.cc"),
inventory: &dir.path().join("i.cc"),
entities: &dir.path().join("entities.cc"),
ontology: &copy,
speaker_state: &dir.path().join("speaker-state.cc"),
},
"fake-model",
"bag",
LowererReach::ToolLessRelay,
ModelFamily::Unrecorded,
&["en-US".to_string()],
)
.unwrap();
assert_ne!(second.record().ontology_digest, pin);
// A condition on a feature outside the inventory is the store's to refuse.
let outside = draft.replace(STANCE_G, "feature:99");
let entries = weksa::encode::OntologyDraft::parse(&outside)
.unwrap()
.into_entries("en-US", 3, 9, &annotator);
let mut grown = OntologyStore::open(&copy).unwrap();
assert!(grown.admit(entries, &inv).is_err());
}

View file

@ -34,7 +34,7 @@ means no store holds it and nothing writes it.
| Inventory entry | Key `<family>:<n>`, `n` canonical decimal >= 1 (families: concept, frame, role, relation, speech_act, feature, referent_kind, extension); type `weksa.inventory_entry`, with a bounded gloss, value kind (option labels bounded), the slots a feature may sit in (`Discourse`, `Referent`, `Predication`, `Relation`; `Stance` from cut `speaker-state`, `Speaker` and `Scene` from cut `speaker-scene`), `introduced_rev` and optional `retired_rev`. Speaker axes, scene relations and affects are features of those slots, so the loop grows them like any closed-class family and convergence covers them | Added or retired by an inventory revision; never edited in place (a changed meaning is a retire plus an add); a retired id still resolves for packets of earlier revisions | The loop's accepted rounds, whose minting call the context builder assembles from Train items only; `weksa inventory admit` for one model-drafted `EditDraft` outside a run (cap 256, reviewed whole in its cut; same numbering and admission check as the loop); Self on merge to main. `data/original` revs 5 (233 concepts and frames) and 6 (five concepts closing named gaps) were drafted by claude-sonnet-5-5 (Weksa Hands, model-drafted original-bed lexicon from Train and Validation passages); Validation-informed: numbers on this bed measure lowering, never generalization or convergence. Residue after rev 6, not amended (each fresh coverage reader names new gaps; the loop's growth path owns them): simultaneity (ながら), distal deixis (あの), 'first' (まず), counterfactual mood, 'at least', emphatic question; heaven versus sky for the visible 天 stays weak. Admission checks form, not text |
| Inventory revision | Key `rev:<n>`, type `weksa.inventory_revision`, with parent, change list and `frozen` | Appended per accepted round; exactly one revision is ever marked frozen, by the convergence test | The loop's accept rule appends; the convergence test freezes; Self records the freeze as a ruling |
| Target profile | Locale or language-project id (`en-US`, `ja-JP`, a conlang's slug). Until entries grow in rounds there is no profile revision document: a run names the profile it lowered with by the ontology store's path and digest in its `RunRecord` (cut `language-ontology`); `profile:<language>:<n>` revision documents arrive with follow-up `language-ontology-growth` | The ontology store is revised by commit; a run pins it and refuses drift; it likewise pins the speaker-state store (`RunRecord` keys 17 and 18, `Drift(SpeakerState)`, cut `language-ontology-scene`) | Profile authors and accepted rounds; never the lowerer's defaults (`persona-drives-register`) |
| Language ontology entry (cut `language-ontology`) | Key `onto:<language>:<n>`, `n` from 1 per language; type `weksa.ontology_entry` in `data/ontology/ontology.cc`: the language, the revision of the inventory it was admitted through (provenance: beds number revisions independently and a run reads each condition's feature at its own head), a layer (lexicon, grammar, discourse, phonology, orthography), at most four conditions (a feature id of slot `Stance`, `Discourse` or `Scene` with `AtLeast`/`AtMost`/`AtMostOrUnset` a grade or `Is` an option or flag; a `Scene` condition reads the relation of the item's speaker to its addressee that `with_scene` selected; slot `Speaker` and community ids wait on follow-up `language-ontology-communities`), a choice of 1 to 200 bytes, at most three examples of 1 to 120 bytes, and an `Annotator`. Whether an entry's text keeps politeness on the scene is a model reader's judgment, owed per change (a mint owes it under `language-ontology-growth`), not a test | Admitted in all-or-nothing batches through `OntologyStore::admit`, immutable, never edited; retire arrives with `language-ontology-growth`. The run pins the store like the entity store (path and digest, `Drift(Ontology)`), never copies it, because no loop step writes it yet. Cut `language-ontology-seed` converts by drafting, not by parse: a fresh model drafts each language's seed from that language's YAML and the inventory's feature lines with no corpus text, then deletes `data/target-language-ontology/` and `.weksa/cultural-ontology/` in the same cut | Seed: a labelled model (`Annotator::Model`), no operator gate, never Test evidence (`ja-ontology-owner-answer`); attestation (operator, native reader) appends later. Accepted rounds later, through a mint call from Train items only. Selection is `ontology::select(ontology, language, view)`, called only inside `Prompt::lower`, so no caller picks entries; the prompt prints each selected entry's id, layer, choice and examples. Its text is minted text for the leak diagnostic, which today has no runtime caller (follow-up `leak-diagnostic-in-round`). The store was rebuilt without ja-JP `:14` and `:15` (above-status Scene entries; a paired measure found them net-negative on register, follow-up `ja-scene-entries-lower-register-agreement`), so ja-JP ends at `:13`: an above-status out-group addressee, and an above-status in-group addressee below intimacy 700, select no ja-JP Scene entry (`:12` still selects in-group at intimacy 700 or more regardless of status) until a native reader attests a replacement (`ja-native-reader-scenes`). ja-JP `:5`, `:7` and `:8` were revised by certainty and restraint `AtMostOrUnset` conditions (holds when the feature is unrated or at most the grade; `AtMost` still fails on an unrated feature) in a store rebuild (the store has no retire, so a conditions change is a rebuild), which makes ja-JP's hedging, certainty, affection, restraint and sadness entries disjoint on `:5`/`:6`, `:4`/`:7`, `:6`/`:7`, `:6`/`:8`, and ja-JP `:3` (anger) carries `AtMostOrUnset` hedge and affection bounds, as en-US `:3`, which makes `:3`/`:5` and `:3`/`:7` disjoint; en-US `:3` carries `AtMostOrUnset` hedge and affection bounds, `:5` an `AtMostOrUnset` certainty bound (the clause it held in prose) and `:8` one too (as ja-JP `:7`), which makes `:3`/`:5`, `:3`/`:8`, `:5`/`:6` and `:6`/`:8` disjoint, while `:6`/`:10` stay co-selectable and are kept apart by scoping in `:6`'s text; ja-JP `:9` was reworded in a rebuild to state its question ending as available, so `:6`/`:9`, `:5`/`:9` and `:3`/`:9` stay co-selectable and are reconciled by wording, not by condition |
| Language ontology entry (cut `language-ontology`) | Key `onto:<language>:<n>`, `n` from 1 per language; type `weksa.ontology_entry` in `data/ontology/ontology.cc`: the language, the revision of the inventory it was admitted through (provenance: beds number revisions independently and a run reads each condition's feature at its own head), a layer (lexicon, grammar, discourse, phonology, orthography), at most four conditions (a feature id of slot `Stance`, `Discourse` or `Scene` with `AtLeast`/`AtMost`/`AtMostOrUnset` a grade or `Is` an option or flag; a `Scene` condition reads the relation of the item's speaker to its addressee that `with_scene` selected; slot `Speaker` and community ids wait on follow-up `language-ontology-communities`), a choice of 1 to 200 bytes, at most three examples of 1 to 120 bytes, and an `Annotator`. Whether an entry's text keeps politeness on the scene is a model reader's judgment, owed per change (a mint owes it under `language-ontology-growth`), not a test | Admitted in all-or-nothing batches through `OntologyStore::admit`, immutable, never edited; retire arrives with `language-ontology-growth`. The run pins the store like the entity store (path and digest, `Drift(Ontology)`) and never writes it; slice 1 of growth is outside any run: `weksa loop mint-ontology` drafts entries from the run's Train items of one language into `<root>/ontology-mint/<digest>.answer`, `weksa ontology admit --ontology <copy>` admits them into a file copy, and a new run pins the copy (a run-owned copy waits on follow-up `language-ontology-growth`). Cut `language-ontology-seed` converts by drafting, not by parse: a fresh model drafts each language's seed from that language's YAML and the inventory's feature lines with no corpus text, then deletes `data/target-language-ontology/` and `.weksa/cultural-ontology/` in the same cut | Seed: a labelled model (`Annotator::Model`), no operator gate, never Test evidence (`ja-ontology-owner-answer`); attestation (operator, native reader) appends later. Accepted rounds later, through a mint call from Train items only. Selection is `ontology::select(ontology, language, view)`, called only inside `Prompt::lower`, so no caller picks entries; the prompt prints each selected entry's id, layer, choice and examples. Its text is minted text for the leak diagnostic, which today has no runtime caller (follow-up `leak-diagnostic-in-round`). The store was rebuilt without ja-JP `:14` and `:15` (above-status Scene entries; a paired measure found them net-negative on register, follow-up `ja-scene-entries-lower-register-agreement`), so ja-JP ends at `:13`: an above-status out-group addressee, and an above-status in-group addressee below intimacy 700, select no ja-JP Scene entry (`:12` still selects in-group at intimacy 700 or more regardless of status) until a native reader attests a replacement (`ja-native-reader-scenes`). ja-JP `:5`, `:7` and `:8` were revised by certainty and restraint `AtMostOrUnset` conditions (holds when the feature is unrated or at most the grade; `AtMost` still fails on an unrated feature) in a store rebuild (the store has no retire, so a conditions change is a rebuild), which makes ja-JP's hedging, certainty, affection, restraint and sadness entries disjoint on `:5`/`:6`, `:4`/`:7`, `:6`/`:7`, `:6`/`:8`, and ja-JP `:3` (anger) carries `AtMostOrUnset` hedge and affection bounds, as en-US `:3`, which makes `:3`/`:5` and `:3`/`:7` disjoint; en-US `:3` carries `AtMostOrUnset` hedge and affection bounds, `:5` an `AtMostOrUnset` certainty bound (the clause it held in prose) and `:8` one too (as ja-JP `:7`), which makes `:3`/`:5`, `:3`/`:8`, `:5`/`:6` and `:6`/`:8` disjoint, while `:6`/`:10` stay co-selectable and are kept apart by scoping in `:6`'s text; ja-JP `:9` was reworded in a rebuild to state its question ending as available, so `:6`/`:9`, `:5`/`:9` and `:3`/`:9` stay co-selectable and are reconciled by wording, not by condition |
| Community (cut `language-ontology`, follow-up `language-ontology-communities`) | Key `community:<slug>`; type `weksa.community`, converted from `data/cultural-ontology/cultures/` and `referents/` (YAML deleted in the same cut) | Revised by commit; a speaker names the communities it belongs to by id | The cultural ontology's author (owner-attested evidence only; Discord-only content stays deleted, `discord-only-content-deleted`) |
| Speaker (cut `speaker-scene`) | Key `speaker:<slug>@<n>`; type `weksa.speaker`: stable axes as feature values of slot `Speaker` (origin, socioeconomic standing, age cohort, education and whatever the loop adds), at an inventory revision, and an `Annotator` | A new revision appends, never edits; a corpus item and every prior turn name a speaker slug, and a round records the speaker revisions it read | The annotator it names: `PersonaSnapshot` (a projection of a `gamecult.persona_state.v0` document, naming its source document and `updatedAt`), `Operator`, `WorkMetadata` (a public-domain work's own cast list and narration), or `Model { family, generator }` (the 2b `ModelFamily`) |
| Scene (cut `speaker-scene`) | Key `scene:<group>@<n>`, the item's split group; type `weksa.scene`: participants (speaker slugs) and, per ordered pair, relation features of slot `Scene` (relative status, intimacy, in-group, audience and whatever the loop adds), at an inventory revision (provenance; a run reads the features at its own head), and an `Annotator` | As Speaker. Keyed by the group, so a scene never straddles the split. `data/original/speaker-state.cc` holds Model-annotated keeper scenes, one per Train or Validation group of both languages (`weksa scene put|dump`); a Test group has none and reads `NoScene` | As Speaker; a model-annotated scene is never Test evidence |