GameCult/Weksa
Watch
9
0
Fork
You've already forked Weksa
 0
Code Issues Pull requests 2 Projects Releases Packages Wiki Activity Actions

language-ontology-scene: scene features, one scene door, Scene-conditioned ja/en entries, runs read entries at their own head #53

Merged
weksa merged 5 commits from weksa-lowering/language-ontology-scene into main 2026-11-12 17:22:10 +00:00
Conversation 0 Commits 5 Files changed 29 +1046 -99
29 changed files with 1046 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

9
README.md
View file

@ -5,7 +5,8 @@ culture, and constraint, and the planned lowerer that turns an agent's
meaning packet into the reader's display language.
**What is valuable here.** The interlingua standard and the lowering design: a
meaning packet owns what is meant, speaker and scene state (planned; not yet live) own register,
meaning packet owns what is meant, the scene state (live: `weksa scene put|dump`; speaker
documents are planned) owns the speaker's relation to the addressee, which selects register,
address and politeness, and a lowerer owns surface text and leaves a trace.
**The problem it solves.** An agent's outward speech today is one English
@ -84,9 +85,9 @@ target-specific English receipt.
The language ontology is `data/ontology/ontology.cc` (read it with
`weksa ontology dump`): per language, the lexicon, grammar and discourse choices a lowerer
reads for a stance and discourse packet. No entry fixes an address or politeness level; those
belong to the speaker and scene state. Its en-US and ja-JP entries are
model-drafted seeds, never Test evidence. Nothing yet grows them from accepted
reads for a stance, a discourse and a scene relation. Only an entry with a Scene condition names an
address or politeness level; the relation, from `data/original/speaker-state.cc`, picks it. Its
en-US and ja-JP entries and the keeper scenes are model-drafted or model-annotated, never Test evidence. Nothing yet grows them from accepted
rounds; until that mint exists they change only by a new `weksa ontology admit`.
For the first AquaSynth speech-training handoff, see

1
crates/weksa/examples/show_working.rs
View file

@ -259,6 +259,7 @@ fn lower_error(e: &LowerError) -> Value {
LowerError::ConsentMissing { predication } => ("ConsentMissing", format!("{predication} is a request the packet does not carry the addressee's consent to")),
LowerError::AskerNotAddressee { predication } => ("AskerNotAddressee", format!("the agent of {predication} is not the addressee")),
LowerError::EntityRead { entity, .. } => ("EntityRead", format!("the entity store failed reading {entity}")),
LowerError::SceneUnread => ("SceneUnread", "the view's scene was never read".to_string()),
};
json!({ "variant": variant, "reason": reason })
}

27
crates/weksa/src/context.rs
View file

@ -37,7 +37,7 @@ use crate::packet::{
use crate::run::LADDER_STEP;
use crate::split::{Checkpoint, Split, SplitScheme, fnv1a64, hex16};
use crate::store::{CorpusStore, normalized};
use crate::validate::LoweringInput;
use crate::validate::{LoweringInput, SceneRead};
/// The instructions a mint prompt starts with: the editor's whole brief. Fixed
/// per run by `PROMPT_SOURCE_DIGEST`.
@ -71,7 +71,7 @@ A feature is {feature, value}; value is {\"Grade\": 0 to 1000}, {\"Option\": lab
/// Fixed per run by `PROMPT_SOURCE_DIGEST`.
pub const LOWER_INSTRUCTIONS: &str = "\
Say the packet below in the language on the language line, as one utterance of its speaker: all of its meaning and nothing it does not carry.
Every id is the inventory's listed below and every feature value is as listed; a grade is permille, 0 to 1000. The stance is the speaker's felt affect and restraint: it shapes how the utterance is said, not what it says. The ontology lines are choices this language makes for this speaker and stance: realize each where the packet allows; they add no meaning.
Every id is the inventory's listed below and every feature value is as listed; a grade is permille, 0 to 1000. The stance is the speaker's felt affect and restraint: it shapes how the utterance is said, not what it says. The ontology lines are choices this language makes for this speaker, stance and scene: realize each where the packet allows; they add no meaning.
A referent with an entity is named by one of that entity's forms on the entities line.
Answer with one JSON object and nothing else: {\"text\": \"...\"}. No code fence, no commentary.";
@ -89,7 +89,7 @@ Answer with one JSON object and nothing else: {\"text\": \"...\"}, the continuat
/// ```no_run
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let panel = ["en-US".to_string()];
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &panel).unwrap();
/// let t = run.train().unwrap();
@ -104,7 +104,7 @@ Answer with one JSON object and nothing else: {\"text\": \"...\"}, the continuat
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # for n in ["c.cc", "i.cc", "e.cc", "o.cc"] { std::fs::write(p(n), [0x90]).unwrap(); }
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &["en-US".to_string()]).unwrap();
/// let t = run.train().unwrap();
/// let _u = t.clone();
@ -115,7 +115,7 @@ Answer with one JSON object and nothing else: {\"text\": \"...\"}, the continuat
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # for n in ["c.cc", "i.cc", "e.cc", "o.cc"] { std::fs::write(p(n), [0x90]).unwrap(); }
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &["en-US".to_string()]).unwrap();
/// let t = run.train().unwrap();
/// s(&t);
@ -337,7 +337,7 @@ impl EditFeedback {
/// ```no_run
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let panel = ["en-US".to_string()];
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &panel).unwrap();
/// let t = run.train().unwrap();
@ -356,7 +356,7 @@ impl EditFeedback {
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # for n in ["c.cc", "i.cc", "e.cc", "o.cc"] { std::fs::write(p(n), [0x90]).unwrap(); }
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &["en-US".to_string()]).unwrap();
/// let t = run.train().unwrap();
/// let inv = run.inventory().inventory().unwrap();
@ -369,7 +369,7 @@ impl EditFeedback {
/// # let dir = tempfile::tempdir().unwrap();
/// # let p = |n: &str| dir.path().join(n);
/// # for n in ["c.cc", "i.cc", "e.cc", "o.cc"] { std::fs::write(p(n), [0x90]).unwrap(); }
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc") };
/// # let paths = weksa::run::RunPaths { corpus: &p("c.cc"), inventory: &p("i.cc"), entities: &p("e.cc"), ontology: &p("o.cc"), speaker_state: &p("s.cc") };
/// # let run = weksa::run::RunStores::create(&p("run"), "t", paths, "m", "e", weksa::run::LowererReach::ToolLessRelay, weksa::corpus::ModelFamily::Unrecorded, &["en-US".to_string()]).unwrap();
/// let t = run.train().unwrap();
/// let inv = run.inventory().inventory().unwrap();
@ -528,9 +528,11 @@ impl Prompt {
/// `input` (no kind, constraints, scene, speaker or group), the forms in
/// `language` of every entity its referents cite, the `ontology:` section
/// and the entries of `inventory` live at the view's revision. The section
/// holds the entries `ontology::select` chooses from the view's stance and
/// discourse features and `language`, never the caller; the view's speaker
/// and group are still never read. `item`
/// holds the entries `ontology::select` chooses from the view's stance,
/// discourse and the scene relation `with_scene` read, and `language`,
/// never the caller; the scene's values are never printed, only the entries
/// they select. A view whose scene was never read is refused
/// ([`LowerError::SceneUnread`]). `item`
/// goes only to [`Prompt::item_keys`], never into the text. A speaker's
/// entity id appears
/// only when the encoder cites that entity on a referent, which is naming a
@ -545,6 +547,9 @@ impl Prompt {
entities: &dyn EntityForms,
ontology: &Ontology,
) -> Result<Prompt, LowerError> {
if input.scene_read() == &SceneRead::Unread {
return Err(LowerError::SceneUnread);
}
let mut forms: BTreeMap<&str, Vec<String>> = BTreeMap::new();
for id in input.referents().iter().filter_map(|r| r.entity.as_ref()) {
let found = entity(entities, id)?;

10
crates/weksa/src/inventory.rs
View file

@ -541,7 +541,7 @@ pub fn proof_revision() -> (InventoryRevision, Vec<InventoryEntry>) {
)
}
/// Revision 3: the speaker-state vocabulary. First the Scene features, the
/// Revision `n`: the speaker-state vocabulary. First the Scene features, the
/// speaker's relation to one party of a scene (slot `Scene`): `relative_status`
/// is the speaker's standing against the addressee's, not a register word, what
/// it selects is each language's ontology's business. Then the Speaker features
@ -551,7 +551,7 @@ pub fn proof_revision() -> (InventoryRevision, Vec<InventoryEntry>) {
/// arrive as community ids with cut `language-ontology`. Hand-seeded like
/// [`proof_revision`] until the loop's minting round owns the vocabulary; the
/// features take the ids after the seed's.
pub fn speaker_scene_revision() -> (InventoryRevision, Vec<InventoryEntry>) {
pub fn speaker_scene_revision(n: u32) -> (InventoryRevision, Vec<InventoryEntry>) {
let taken = seed()
.1
.iter()
@ -593,15 +593,15 @@ pub fn speaker_scene_revision() -> (InventoryRevision, Vec<InventoryEntry>) {
gloss: gloss.to_string(),
value_kind: Some(kind),
applies_to: vec![slot],
introduced_rev: 3,
introduced_rev: n,
retired_rev: None,
});
}
let changes = entries.iter().map(|e| Change::Add(e.id)).collect();
(
InventoryRevision {
n: 3,
parent: Some(2),
n,
parent: Some(n - 1),
changes,
frozen: false,
},

3
crates/weksa/src/lower.rs
View file

@ -182,6 +182,8 @@ pub enum LowerError {
AskerNotAddressee { predication: NodeId },
/// The entity port failed to read.
EntityRead { entity: EntityId, why: String },
/// A lowering prompt was asked of a view whose scene was never read.
SceneUnread,
}
impl fmt::Display for LowerError {
@ -203,6 +205,7 @@ impl fmt::Display for LowerError {
LowerError::ConsentMissing { .. } => "request lacks the addressee's consent",
LowerError::AskerNotAddressee { .. } => "request agent is not the addressee",
LowerError::EntityRead { .. } => "entity port failed to read",
LowerError::SceneUnread => "the view's scene was never read",
})
}
}

184
crates/weksa/src/main.rs
View file

@ -10,7 +10,8 @@ use unicode_normalization::UnicodeNormalization;
use weksa::corpus::{CorpusItem, ModelFamily, NameSpan, Source, Turn};
use weksa::encode::OntologyDraft;
use weksa::entity::{Entity, EntityOrigin, LanguageForms, Pronunciation};
use weksa::inventory::{proof_revision, seed};
use weksa::inventory::{proof_revision, seed, speaker_scene_revision};
use weksa::packet::{FeatValue, FeatureValue, Grade};
use weksa::packet::proof_packet;
use weksa::model::RecordedAnswers;
use weksa::run::{
@ -18,8 +19,8 @@ use weksa::run::{
};
use weksa::score::ScoreInput;
use weksa::{
Annotator, AnswerFiles, CorpusStore, EntityId, EntityStore, InventoryStore, OllamaEmbedder, OntologyStore, PacketStore,
Scorer, Split, SplitScheme, StoreError, lower, pronounce, validate,
Annotator, AnswerFiles, CorpusStore, EntityId, EntityStore, Family, InventoryStore, OllamaEmbedder, OntologyStore, OptionLabel,
PacketStore, PairRelation, Party, Scene, Slot, SpeakerStateStore, ValueKind, Scorer, Split, SplitScheme, StoreError, lower, pronounce, validate,
};
#[derive(Parser)]
@ -40,6 +41,9 @@ struct Cli {
/// The language ontology store.
#[arg(long, global = true, default_value = "data/ontology/ontology.cc")]
ontology: PathBuf,
/// The speaker-state store (scenes).
#[arg(long, global = true, default_value = "data/corpus/speaker-state.cc")]
speaker_state: PathBuf,
#[command(subcommand)]
command: Command,
}
@ -57,6 +61,8 @@ enum Command {
#[command(subcommand)]
Ontology(OntologyCommand),
#[command(subcommand)]
Scene(SceneCommand),
#[command(subcommand)]
Packet(PacketCommand),
/// Lower a stored packet to English text.
Lower {
@ -87,7 +93,7 @@ enum Command {
#[derive(Subcommand)]
enum LoopCommand {
/// Pin the global --corpus, --inventory, --entities and --ontology as a run at ROOT.
/// Pin the global --corpus, --inventory, --entities, --ontology and --speaker-state as a run at ROOT.
Open {
root: PathBuf,
#[arg(long)]
@ -191,6 +197,38 @@ enum InventoryCommand {
Proof,
/// Admit revision 3 of the Dao De Jing test bed stores (after seed and proof).
Dao,
/// Admit the speaker-state revision (Scene and Speaker features) as the
/// inventory's latest plus one.
SpeakerScene,
}
#[derive(Subcommand)]
enum SceneCommand {
/// Append a scene revision for one corpus group: its one live speaker's
/// relation to the addressee, as Scene-slot feature values.
Put {
#[arg(long)]
group: String,
/// `entity:<slug>` of a named addressee, or `unnamed`.
#[arg(long)]
to: String,
/// The inventory revision the glosses resolve at.
#[arg(long)]
inventory_rev: u32,
/// `gloss=value`, repeated: a grade 0-1000, an option label or true/false.
#[arg(long = "set", value_name = "GLOSS=VALUE")]
set: Vec<String>,
#[arg(long, requires = "generator", conflicts_with = "operator")]
model_family: Option<ModelFamilyArg>,
/// The attestation string naming the annotating model.
#[arg(long, requires = "model_family")]
generator: Option<String>,
/// Annotated by the operator, not a model.
#[arg(long)]
operator: bool,
},
/// Text projection of the scenes for review; not a store.
Dump,
}
#[derive(Subcommand)]
@ -734,6 +772,143 @@ fn main() -> Result<()> {
.map_err(refused)?;
println!("admitted rev:2 with {count} entries");
}
Command::Inventory(InventoryCommand::SpeakerScene) => {
let store = InventoryStore::open(&cli.inventory)?;
let latest = store
.inventory()?
.latest()
.map(|r| r.n)
.context("the inventory holds no revision")?;
let (rev, entries) = speaker_scene_revision(latest + 1);
let (n, count) = (rev.n, entries.len());
let mut store = store;
store.admit_revision(rev, entries).map_err(refused)?;
println!("admitted rev:{n} with {count} entries");
}
Command::Scene(SceneCommand::Put {
group,
to,
inventory_rev,
set,
model_family,
generator,
operator,
}) => {
let annotator = match (model_family, generator, operator) {
(Some(family), Some(generator), false) => Annotator::Model {
family: family.into(),
generator,
},
(None, None, true) => Annotator::Operator,
_ => bail!("name the annotator: --model-family with --generator, or --operator"),
};
let inventory_store = InventoryStore::open(&cli.inventory)?;
let inventory = inventory_store.inventory()?;
let corpus = CorpusStore::open(&cli.corpus)?;
let live: Vec<CorpusItem> = corpus
.all()?
.into_iter()
.filter(|i| i.group == group && !i.withdrawn)
.collect();
let mut speakers: Vec<EntityId> = live.iter().map(|i| i.speaker.clone()).collect();
speakers.sort();
speakers.dedup();
let [from] = speakers.as_slice() else {
bail!("a scene group holds one live speaker; multi-party scenes wait");
};
let to = match to.as_str() {
"unnamed" => Party::Unnamed,
named => Party::Named(
EntityId::parse(named).map_err(|_| anyhow::anyhow!("the addressee is unnamed or an entity id"))?,
),
};
let mut features = Vec::new();
for pair in &set {
let bad = || anyhow::anyhow!("a --set is gloss=value of a live Scene feature");
let (gloss, value) = pair.split_once('=').ok_or_else(bad)?;
let entry = inventory
.entries()
.find(|e| {
e.family() == Family::Feature
&& e.gloss == gloss
&& e.live_at(inventory_rev)
&& e.applies_to.contains(&Slot::Scene)
})
.ok_or_else(bad)?;
let value = match entry.value_kind.as_ref().ok_or_else(bad)? {
ValueKind::Grade => FeatValue::Grade(
Grade::new(value.parse().map_err(|_| bad())?).map_err(|_| bad())?,
),
ValueKind::Options(_) => {
FeatValue::Option(OptionLabel::parse(value).map_err(|_| bad())?)
}
ValueKind::Flag => FeatValue::Flag(value.parse().map_err(|_| bad())?),
};
features.push(FeatureValue { feature: entry.id, value });
}
let mut participants = vec![from.clone()];
if let Party::Named(a) = &to
&& a != from
{
participants.push(a.clone());
}
let mut state = SpeakerStateStore::open(&cli.speaker_state)?;
let latest = state.scene(&group)?;
let mut scene = Scene {
group,
n: latest.as_ref().map_or(1, |l| l.n + 1),
inventory_rev,
participants,
relations: vec![PairRelation { from: from.clone(), to, features }],
annotator,
};
if let Some(l) = &latest {
let same = Scene { n: l.n, ..scene.clone() };
if &same == l {
println!("unchanged");
return Ok(());
}
}
scene.n = latest.map_or(1, |l| l.n + 1);
let key = weksa::speaker::scene_key(&scene.group, scene.n);
let outcome = state
.put_scene(scene, &inventory_store, &corpus, &EntityStore::open(&cli.entities)?)
.map_err(refused)?;
println!("{outcome:?} {key}");
}
Command::Scene(SceneCommand::Dump) => {
let inventory = InventoryStore::open(&cli.inventory)?.inventory()?;
let gloss = |f: &FeatureValue| {
let name = inventory.entry(&f.feature).map_or("?", |e| e.gloss.as_str());
match &f.value {
FeatValue::Grade(g) => format!("{name}={}", g.get()),
FeatValue::Option(l) => format!("{name}={l}"),
FeatValue::Flag(b) => format!("{name}={b}"),
}
};
for s in SpeakerStateStore::open(&cli.speaker_state)?.all_scenes()? {
let who = match &s.annotator {
Annotator::Operator => "Operator".to_string(),
Annotator::Model { family, generator } => format!("Model {family:?} {generator}"),
};
let people: Vec<&str> = s.participants.iter().map(|p| p.as_str()).collect();
println!(
"scene:{}@{} rev:{} {who} [{}]",
s.group,
s.n,
s.inventory_rev,
people.join(",")
);
for r in &s.relations {
let to = match &r.to {
Party::Named(e) => e.as_str().to_string(),
Party::Unnamed => "unnamed".to_string(),
};
let feats: Vec<String> = r.features.iter().map(&gloss).collect();
println!(" {}->{to} {}", r.from, feats.join(" "));
}
}
}
Command::Inventory(InventoryCommand::Dao) => {
let (rev, entries) = weksa::daodejing::revision();
let count = entries.len();
@ -850,6 +1025,7 @@ fn main() -> Result<()> {
inventory: &cli.inventory,
entities: &cli.entities,
ontology: &cli.ontology,
speaker_state: &cli.speaker_state,
},
&answerer,
&embed_model,

16
crates/weksa/src/ontology.rs
View file

@ -1,7 +1,7 @@
//! The language ontology: per target language, the choices its culture makes
//! about word, grammar, discourse, sound and script for a speaker in a stance
//! (ruling `persona-drives-ontology`). An entry says when it applies (feature
//! conditions over the lowering view's stance and discourse) and what it
//! conditions over the lowering view's stance, discourse and scene relation) and what it
//! chooses. `select` is the only reader that decides which entries a lowering
//! prompt carries, and it runs only inside `Prompt::lower`;
//! `OntologyStore::admit` is the only writer. Entries are immutable.
@ -51,8 +51,8 @@ pub enum Test {
Is(FeatValue),
}
/// One feature of one slot of the view, tested. Only `Stance` and `Discourse`
/// reach a lowering view today.
/// One feature of one slot of the view, tested. `Stance`, `Discourse` and
/// `Scene` reach a lowering view; `Speaker` does not yet.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
pub struct Condition {
pub slot: Slot,
@ -100,7 +100,9 @@ pub struct OntologyEntry {
/// The entry's number within its language, from 1.
#[cultcache(key = 1)]
pub n: u32,
/// The inventory revision the conditions read against.
/// The revision, of the inventory the entry was admitted through, at which the
/// store read its conditions: provenance only; a run reads the conditions'
/// features at its own head.
#[cultcache(key = 2)]
pub inventory_rev: u32,
#[cultcache(key = 3)]
@ -149,8 +151,9 @@ impl Ontology {
}
/// The entries of exactly `language` whose every condition holds against the
/// view's stance and discourse features, in `n` order. An absent feature meets
/// no condition. Nothing else of the view is read.
/// view's stance, discourse and scene-relation features (the scene's only when
/// `with_scene` selected a relation), in `n` order. An absent feature meets no
/// condition. Nothing else of the view is read.
pub fn select<'a>(
o: &'a Ontology,
language: &str,
@ -163,6 +166,7 @@ pub fn select<'a>(
e.when.iter().all(|c| match c.slot {
Slot::Stance => c.holds(&input.stance().features),
Slot::Discourse => c.holds(&input.discourse().features),
Slot::Scene => c.holds(input.scene()),
_ => false,
})
})

105
crates/weksa/src/run.rs
View file

@ -43,15 +43,16 @@ use crate::corpus::{CorpusItem, CorpusItemId, ModelFamily, Source};
use crate::encode::{EditDraft, LoweringDraft, PacketDraft, encoder_id};
use crate::form::{is_attestation, is_language_tag, is_slug};
use crate::model::{ModelPort, PortError};
use crate::ontology::OntologyEntry;
use crate::packet::{Packet, Stance};
use crate::score::{CHRF_VERSION, chrf, Embedder, ItemScore, ScoreInput, Scorer, SplitMean, aggregate};
use crate::split::{Split, SplitScheme, fnv1a64, fnv1a64_of, hex16};
use crate::store::{
CorpusStore, EntityStore, InventoryStore, OntologyStore, PacketStore, StoreError,
CorpusStore, EntityStore, InventoryStore, OntologyStore, PacketStore, SpeakerStateStore,
StoreError,
check_ontology_entry, open_cache_over,
};
use crate::validate::LoweringInput;
use crate::inventory::Slot;
use crate::validate::{LoweringInput, feature_refusals};
/// The digest of the prompt builder's source: `context.rs`, then `split.rs`,
/// whose scheme and digest function decide which lines the prompt holds as
@ -137,6 +138,13 @@ pub struct RunRecord {
/// 16 hex of the ontology file's bytes.
#[cultcache(key = 16)]
pub ontology_digest: String,
/// Canonical absolute path of the speaker-state store (scenes): read-only
/// for the run, pinned by `speaker_state_digest`.
#[cultcache(key = 17)]
pub speaker_state: String,
/// 16 hex of the speaker-state file's bytes.
#[cultcache(key = 18)]
pub speaker_state_digest: String,
}
/// How a run's lowering prompts reach the model that answers them (rulings
@ -442,6 +450,7 @@ pub enum DriftOf {
Corpus,
Entities,
Ontology,
SpeakerState,
Prompts,
Model,
Embedder,
@ -480,6 +489,8 @@ pub enum RunError {
/// An ontology entry of a panel language does not read against the run's
/// head inventory at its revision.
Ontology,
/// A scene does not read against the run's inventory.
SpeakerState,
}
impl fmt::Display for RunError {
@ -504,6 +515,9 @@ impl fmt::Display for RunError {
RunError::Ontology => f.write_str(
"an ontology entry of a panel language does not read against the run's inventory",
),
RunError::SpeakerState => {
f.write_str("a scene does not read against the run's inventory")
}
RunError::Panel => f.write_str(
"the panel is empty, repeats or misspells a language, or names one with no live Train or Validation item",
),
@ -519,13 +533,14 @@ impl fmt::Debug for RunError {
impl std::error::Error for RunError {}
/// The four stores a run pins.
/// The five stores a run pins.
#[derive(Clone, Copy)]
pub struct RunPaths<'a> {
pub corpus: &'a Path,
pub inventory: &'a Path,
pub entities: &'a Path,
pub ontology: &'a Path,
pub speaker_state: &'a Path,
}
/// A run's open stores, reopened only from its record. The one public producer
@ -538,6 +553,7 @@ pub struct RunStores {
inventory: InventoryStore,
entities: EntityStore,
ontology: OntologyStore,
speaker_state: SpeakerStateStore,
}
/// The run's own copy of the inventory, made at `create`.
@ -554,6 +570,9 @@ fn check_pins(cache: &CultCache, root: &Path, record: &RunRecord) -> Result<(),
if file_digest(&record.ontology).as_ref() != Some(&record.ontology_digest) {
return Err(RunError::Drift(DriftOf::Ontology));
}
if file_digest(&record.speaker_state).as_ref() != Some(&record.speaker_state_digest) {
return Err(RunError::Drift(DriftOf::SpeakerState));
}
let (path, digest, _) = head_of(cache, root, record)?;
if file_digest_at(&path).as_ref() != Some(&digest) {
return Err(RunError::Drift(DriftOf::Inventory));
@ -692,6 +711,8 @@ impl RunStores {
entities: canonical(paths.entities)?,
ontology: canonical(paths.ontology)?,
ontology_digest: String::new(),
speaker_state: canonical(paths.speaker_state)?,
speaker_state_digest: String::new(),
corpus_digest: String::new(),
entity_digest: String::new(),
prompt_digest: hex16(PROMPT_SOURCE_DIGEST),
@ -714,12 +735,14 @@ impl RunStores {
entity_digest: file_digest(&record.entities).ok_or(RunError::Store)?,
inventory_digest: file_digest_at(&head).ok_or(RunError::Store)?,
ontology_digest: file_digest(&record.ontology).ok_or(RunError::Store)?,
speaker_state_digest: file_digest(&record.speaker_state).ok_or(RunError::Store)?,
..record
};
between();
let mut run = Self::with(root, cache, record)?;
run.check_panel()?;
run.check_ontology()?;
run.check_scenes()?;
// The stores were read between the digests: unless they still match
// the pins, what was opened is not what was pinned. Nothing is
// recorded for a run that did not pin.
@ -771,6 +794,8 @@ impl RunStores {
inventory: InventoryStore::open(&head).map_err(store)?,
entities: EntityStore::open(Path::new(&record.entities)).map_err(store)?,
ontology: OntologyStore::open(Path::new(&record.ontology)).map_err(store)?,
speaker_state: SpeakerStateStore::open(Path::new(&record.speaker_state))
.map_err(store)?,
cache,
record,
})
@ -781,28 +806,50 @@ impl RunStores {
}
/// Every ontology entry of a panel language reads against the run's head
/// inventory at its own revision: the check the store applied at admission,
/// applied again to this inventory.
/// inventory: every condition's feature is live at the head in the
/// condition's slot and fits its test. An entry's `inventory_rev` names a
/// revision of the inventory it was admitted through, and beds number
/// revisions independently, so a run never reads it.
fn check_ontology(&self) -> Result<(), RunError> {
let inventory = self.inventory.inventory().map_err(|_| RunError::Store)?;
let entries = self.ontology.all().map_err(|_| RunError::Store)?;
let head = inventory.latest().map(|r| r.n);
entries
.iter()
.filter(|e| self.record.panel.contains(&e.language))
.try_for_each(|e| {
// At its own revision, and again at the head: an entry on a
// feature the head retired could never select.
let at_head = OntologyEntry {
inventory_rev: head.ok_or(RunError::Ontology)?,
..e.clone()
};
check_ontology_entry(e, &inventory)
.and_then(|()| check_ontology_entry(&at_head, &inventory))
.map_err(|_| RunError::Ontology)
let head = inventory.latest().map(|r| r.n).ok_or(RunError::Ontology)?;
check_ontology_entry(e, &inventory, head).map_err(|_| RunError::Ontology)
})
}
/// Every scene's relation features read against the run's head inventory
/// in slot `Scene`, as [`check_ontology`](Self::check_ontology) reads
/// entries: the scene's `inventory_rev` is admission provenance a run never reads.
fn check_scenes(&self) -> Result<(), RunError> {
let inventory = self.inventory.inventory().map_err(|_| RunError::Store)?;
let scenes = self.speaker_state.all_scenes().map_err(|_| RunError::Store)?;
if scenes.is_empty() {
return Ok(());
}
let head = inventory.latest().map(|r| r.n).ok_or(RunError::SpeakerState)?;
for r in scenes.iter().flat_map(|s| &s.relations) {
if !feature_refusals(&inventory, head, &r.features, Slot::Scene).is_empty() {
return Err(RunError::SpeakerState);
}
}
Ok(())
}
/// The one door by which a loop view gets its scene: `with_scene` over the
/// run's pinned speaker-state and corpus stores.
fn assembled(
&self,
view: LoweringInput,
inventory: &InventoryStore,
) -> Result<LoweringInput, StoreError> {
view.with_scene(&inventory.inventory()?, &self.speaker_state, &self.corpus)
}
/// Every panel language holds a live Train and a live Validation item.
fn check_panel(&self) -> Result<(), RunError> {
let items = self.corpus.all().map_err(|_| RunError::Store)?;
@ -1300,6 +1347,9 @@ impl RunStores {
let view = stores[slot]
.admit(packet, &self.inventory, &self.corpus, &self.entities)
.map_err(|_| RunError::Store)?;
let view = self
.assembled(view, &self.inventory)
.map_err(|_| RunError::Store)?;
Prompt::lower(
&view,
&packet_item,
@ -1586,19 +1636,23 @@ fn score_round<M: ModelPort, E: Embedder>(
// Phase 2: one lowering prompt per admitted view, answered through the
// port. Nothing is kept unless every one is answered.
let mut unanswered = Vec::new();
for (at, id, packet_key, view) in &admitted {
for (at, id, packet_key, view) in admitted {
// Each item is lowered in its own language: the only reference the
// scorer has is the item's own text.
let language = &items[*at].language;
let Ok(prompt) = Prompt::lower(view, id, language, &inventory, &run.entities, &ontology) else {
fates[*at] = Fate::Refused(Stage::Lower, Some(packet_key.clone()));
let language = &items[at].language;
let Ok(view) = run.assembled(view, inventory_store) else {
fates[at] = Fate::Refused(Stage::Lower, Some(packet_key.clone()));
continue;
};
let Ok(prompt) = Prompt::lower(&view, &id, language, &inventory, &run.entities, &ontology) else {
fates[at] = Fate::Refused(Stage::Lower, Some(packet_key.clone()));
continue;
};
calls.push(Call {
prompt: prompt.digest(),
items: prompt.item_keys().to_vec(),
});
fates[*at] = match model.answer(&prompt) {
fates[at] = match model.answer(&prompt) {
Ok(answer) => match LoweringDraft::parse(&answer) {
Ok(draft) if draft.usable() => {
Fate::Lowered(draft.text, packet_key.clone())
@ -2793,6 +2847,8 @@ mod tests {
panel: vec!["en-US".into()],
ontology: String::new(),
ontology_digest: String::new(),
speaker_state: String::new(),
speaker_state_digest: String::new(),
}
}
@ -2884,7 +2940,7 @@ mod tests {
}
/// Answers the Dao encode prompts from `answers` by item, the candidate
/// inventory's (rev 4) from the good drafts, and the editor with one concept.
/// inventory's (rev 5) from the good drafts, and the editor with one concept.
struct Fake {
degraded: HashMap<String, String>,
good: HashMap<String, String>,
@ -2911,7 +2967,7 @@ mod tests {
};
return Ok(format!(r#"{{"text":"{text}"}}"#));
}
let answers = if prompt.text().contains("\n\ninventory rev:4\n") {
let answers = if prompt.text().contains("\n\ninventory rev:5\n") {
&self.good
} else {
&self.degraded
@ -2985,6 +3041,8 @@ mod tests {
);
let ontology = dir.path().join("o.cc");
fs::write(&ontology, [0x90]).unwrap();
let speaker_state = dir.path().join("s.cc");
fs::write(&speaker_state, [0x90]).unwrap();
let root = dir.path().join("run");
let mut run = RunStores::create(
&root,
@ -2994,6 +3052,7 @@ mod tests {
inventory: &inventory,
entities: &entities,
ontology: &ontology,
speaker_state: &speaker_state,
},
"fake-model",
"bag",

2
crates/weksa/src/speaker.rs
View file

@ -71,6 +71,8 @@ pub struct Scene {
/// The revision, from 1.
#[cultcache(key = 1)]
pub n: u32,
/// The revision, of the inventory the scene was admitted through, at which
/// `put_scene` read its features: provenance only; a run never reads it.
#[cultcache(key = 2)]
pub inventory_rev: u32,
#[cultcache(key = 3)]

24
crates/weksa/src/store.rs
View file

@ -1273,7 +1273,7 @@ impl OntologyStore {
return Err(malformed(&key, "a language holds at most 64 entries"));
}
counted.insert(e.language.as_str(), e.n);
check_ontology_entry(e, &inv)?;
check_ontology_entry(e, &inv, e.inventory_rev)?;
}
let mut staged = Vec::new();
for e in &batch {
@ -1286,13 +1286,14 @@ impl OntologyStore {
/// The rules of one ontology entry against an inventory: the annotator, the
/// bounded and normalized text of the choice and examples, the condition count
/// and each condition's feature (slot `Stance` or `Discourse`, a grade feature
/// for `AtLeast` and `AtMost`, an option or flag value for `Is`) read at the
/// entry's inventory revision. The store applies it on admission and a run
/// applies it again against its own head inventory.
/// and each condition's feature (slot `Stance`, `Discourse` or `Scene`, a grade
/// feature for `AtLeast` and `AtMost`, an option or flag value for `Is`) read at
/// `rev`. The store passes the entry's own `inventory_rev` on admission; a run
/// passes its inventory's head, because beds number revisions independently.
pub(crate) fn check_ontology_entry(
e: &OntologyEntry,
inventory: &Inventory,
rev: u32,
) -> Result<(), StoreError> {
let key = e.key();
check_annotator(&key, &e.annotator)?;
@ -1320,21 +1321,24 @@ pub(crate) fn check_ontology_entry(
}
let mut out = Vec::new();
for c in &e.when {
if !matches!(c.slot, Slot::Stance | Slot::Discourse) {
return Err(malformed(&key, "a condition reads slot Stance or Discourse"));
if !matches!(c.slot, Slot::Stance | Slot::Discourse | Slot::Scene) {
return Err(malformed(
&key,
"a condition reads slot Stance, Discourse or Scene",
));
}
if matches!(c.test, Test::Is(FeatValue::Grade(_))) {
return Err(malformed(&key, "Is names an option or a flag"));
}
out.extend(feature_refusals(
inventory,
e.inventory_rev,
rev,
&[c.as_value()],
c.slot,
));
}
if e.when.is_empty() && !inventory.has_revision(e.inventory_rev) {
out.extend(feature_refusals(inventory, e.inventory_rev, &[], Slot::Stance));
if e.when.is_empty() && !inventory.has_revision(rev) {
out.extend(feature_refusals(inventory, rev, &[], Slot::Stance));
}
if !out.is_empty() {
return Err(StoreError::Invalid(out));

68
crates/weksa/tests/cli.rs
View file

@ -14,6 +14,8 @@ fn weksa(dir: &std::path::Path, args: &[&str]) -> Output {
"./e.cc",
"--ontology",
"./o.cc",
"--speaker-state",
"./s.cc",
])
.args(args)
.output()
@ -1341,6 +1343,7 @@ fn dao_stores(dir: &std::path::Path) {
std::fs::copy(src.join(from), dir.join(to)).unwrap();
}
std::fs::write(dir.join("o.cc"), [0x90]).unwrap();
std::fs::write(dir.join("s.cc"), [0x90]).unwrap();
}
#[test]
@ -1413,7 +1416,7 @@ fn loop_open_and_unanswered_round() {
assert!(second.status.success(), "{}", out(&second));
assert_eq!(
out(&second),
"round 1 rev 3 en-US train n=1 scored=0 chrf=0.00 meaning=0.0000 refused=1 validation n=2 scored=0 chrf=0.00 meaning=0.0000 refused parse=2 admit=0 lower=0 parse-reasons=syntaxx3 lowering=tool-less-relay\n"
"round 1 rev 4 en-US train n=1 scored=0 chrf=0.00 meaning=0.0000 refused=1 validation n=2 scored=0 chrf=0.00 meaning=0.0000 refused parse=2 admit=0 lower=0 parse-reasons=syntaxx3 lowering=tool-less-relay\n"
);
// The global store flags are not read by loop round: a drifted store is refused.
@ -1546,7 +1549,7 @@ fn loop_cycle_asks_then_decides() {
assert!(done.status.success(), "{}", out(&done));
assert_eq!(
out(&done),
"cycle 1 rev 3->none en-US base=0.0000 candidate=none floor=0.0000 threshold=0.0050 rejected edit-parse lowering=tool-less-relay\n"
"cycle 1 rev 4->none en-US base=0.0000 candidate=none floor=0.0000 threshold=0.0050 rejected edit-parse lowering=tool-less-relay\n"
);
// The next cycle asks afresh in its own directory.
@ -1811,3 +1814,64 @@ fn ontology_admit_and_dump() {
assert!(!out(&o).contains("LEAKYCHOICE"), "{}", out(&o));
assert_eq!(out(&weksa(d.path(), &["ontology", "dump"])), before);
}
#[test]
fn scene_put_and_dump() {
let d = tempfile::tempdir().unwrap();
cast_entities(d.path(), &["nibu", "brokkr"]);
for step in ["seed", "proof", "speaker-scene"] {
let o = weksa(d.path(), &["inventory", step]);
assert!(o.status.success(), "{}", out(&o));
}
let mut add = ADD.to_vec();
add.extend(["--persona", "nibu", "--fixture", "f.yaml", "--author", "Ada Writer"]);
let o = weksa(d.path(), &add);
assert!(o.status.success(), "{}", out(&o));
let put = |group: &str, to: &str, sets: &[&str]| {
let mut args = vec![
"scene", "put", "--group", group, "--to", to, "--inventory-rev", "3",
"--model-family", "anthropic", "--generator", "a scene annotator",
];
for s in sets {
args.extend(["--set", s]);
}
weksa(d.path(), &args)
};
let sets = ["relative_status=above", "intimacy=200", "in_group=false"];
let o = put("scene-1", "unnamed", &sets);
assert!(o.status.success(), "{}", out(&o));
let dump = out(&weksa(d.path(), &["scene", "dump"]));
for want in ["scene:scene-1@1", "rev:3", "Anthropic", "a scene annotator", "entity:nibu->unnamed", "relative_status=above", "intimacy=200", "in_group=false"] {
assert!(dump.contains(want), "{want}: {dump}");
}
// A repeat is unchanged and writes nothing.
let o = put("scene-1", "unnamed", &sets);
assert!(out(&o).contains("unchanged"), "{}", out(&o));
assert_eq!(out(&weksa(d.path(), &["scene", "dump"])), dump);
// A changed value is the next revision.
let o = put("scene-1", "unnamed", &["relative_status=level", "intimacy=200", "in_group=false"]);
assert!(o.status.success(), "{}", out(&o));
assert!(out(&weksa(d.path(), &["scene", "dump"])).contains("scene:scene-1@2"));
// An unknown gloss and an unknown group are refused and not echoed.
for (group, set) in [("scene-1", "canarygloss=1"), ("canarygroup", "intimacy=1")] {
let before = out(&weksa(d.path(), &["scene", "dump"]));
let o = put(group, "unnamed", &[set]);
assert!(!o.status.success());
assert!(!out(&o).contains("canary"), "{}", out(&o));
assert_eq!(out(&weksa(d.path(), &["scene", "dump"])), before);
}
// A group with two live speakers is refused: multi-party scenes wait.
let mut two = add.clone();
let at = two.iter().position(|a| *a == "entity:nibu").unwrap();
two[at] = "entity:brokkr";
let at = two.iter().position(|a| *a == "nibu").unwrap();
two[at] = "brokkr";
two.retain(|a| *a != "Mind the hatch.");
let at = two.iter().position(|a| *a == "--text").unwrap() + 1;
two.insert(at, "Brokkr speaks.");
let o = weksa(d.path(), &two);
assert!(o.status.success(), "{}", out(&o));
let o = put("scene-1", "unnamed", &sets);
assert!(!o.status.success());
assert!(!out(&o).contains("scene-1"), "{}", out(&o));
}

30
crates/weksa/tests/common/mod.rs
View file

@ -123,6 +123,35 @@ pub fn ontology_in(dir: &std::path::Path) -> std::path::PathBuf {
path
}
/// A new inventory store at `path` holding the Dao inventory as it stood at
/// rev 3 (seed, proof, the Dao revision): the loop tests' fixed head, whose
/// round and candidate revisions they pin. The committed
/// `data/daodejing/inventory.cc` is rev 4 and is read by the corpus tests.
pub fn dao_rev3_at(path: &std::path::Path) {
let mut store = weksa::InventoryStore::open(path).unwrap();
for (rev, entries) in [
weksa::inventory::seed(),
weksa::inventory::proof_revision(),
weksa::daodejing::revision(),
] {
store.admit_revision(rev, entries).unwrap();
}
}
/// An empty speaker-state store beside `ontology` (same directory), made if absent.
pub fn speaker_state_beside(ontology: &std::path::Path) -> std::path::PathBuf {
let path = ontology.with_file_name("speaker-state.cc");
if !path.exists() {
std::fs::write(&path, [0x90]).unwrap();
}
path
}
/// An empty speaker-state store at `<dir>/speaker-state.cc`, made if absent.
pub fn speaker_state_in(dir: &std::path::Path) -> std::path::PathBuf {
speaker_state_beside(&dir.join("ontology.cc"))
}
/// A run over the stores `world()` and `corpus()` keep in `dir`: `c.cc`,
/// `i.cc` and `entities.cc`. Read at the call, so create it after the items.
/// The run's en-US panel needs a live Train and a live Validation item: a
@ -158,6 +187,7 @@ pub fn run_over(dir: &std::path::Path) -> weksa::run::RunStores {
inventory: &dir.join("i.cc"),
entities: &dir.join("entities.cc"),
ontology: &dir.join("ontology.cc"),
speaker_state: &speaker_state_beside(&dir.join("ontology.cc")),
},
"fake-model",
"bag",

34
crates/weksa/tests/context.rs
View file

@ -547,7 +547,8 @@ mod lowering {
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data").join(sub)
}
fn admitted(
/// The admitted view, its scene never read.
fn admitted_unread(
packet: Packet,
inv: &InventoryStore,
corpus: &str,
@ -561,6 +562,37 @@ mod lowering {
.unwrap()
}
/// The admitted view with its scene read over an empty speaker-state store.
fn admitted(
packet: Packet,
inv: &InventoryStore,
corpus: &str,
entities: &EntityStore,
) -> LoweringInput {
let view = admitted_unread(packet, inv, corpus, entities);
let dir = tempfile::tempdir().unwrap();
let state = weksa::SpeakerStateStore::open(&dir.path().join("s.cc")).unwrap();
let corpus = CorpusStore::open(&data(corpus)).unwrap();
view.with_scene(&inv.inventory().unwrap(), &state, &corpus).unwrap()
}
#[test]
fn lowering_prompt_refuses_an_unread_scene() {
let cast = InventoryStore::open(&data("interlingua/inventory.cc")).unwrap();
let entities = EntityStore::open(&data("entity/entity.cc")).unwrap();
let inv = cast.inventory().unwrap();
let item = CorpusItemId::parse("item:nibu:1").unwrap();
let held = Held(vec![nibu(&[("en-US", &["Nibu"])])]);
let unread = admitted_unread(proof_packet(), &cast, "corpus/corpus.cc", &entities);
let empty = weksa::Ontology::empty();
assert_eq!(
Prompt::lower(&unread, &item, "en-US", &inv, &held, &empty).err(),
Some(LowerError::SceneUnread)
);
let read = admitted(proof_packet(), &cast, "corpus/corpus.cc", &entities);
assert!(Prompt::lower(&read, &item, "en-US", &inv, &held, &empty).is_ok());
}
fn dao(index: usize) -> (LoweringInput, Inventory, EntityStore, CorpusItemId) {
let inv = InventoryStore::open(&data("daodejing/inventory.cc")).unwrap();
let entities = EntityStore::open(&data("daodejing/entity.cc")).unwrap();

307
crates/weksa/tests/golf_loop.rs
View file

@ -24,12 +24,13 @@ fn rewrite_run(root: &Path, change: impl FnOnce(&mut RunRecord)) {
cache.put(format!("run:{}", record.label), &record).unwrap();
}
fn paths(dir: &Path) -> (PathBuf, PathBuf, PathBuf, PathBuf) {
fn paths(dir: &Path) -> (PathBuf, PathBuf, PathBuf, PathBuf, PathBuf) {
(
dir.join("c.cc"),
dir.join("i.cc"),
dir.join("entities.cc"),
common::ontology_in(dir),
common::speaker_state_in(dir),
)
}
@ -42,7 +43,7 @@ fn a_run_pins_its_stores_and_refuses_drift() {
let mut w = world();
let dir = w.dir.path().to_path_buf();
let root = dir.join("run");
let (c, i, e, o) = paths(&dir);
let (c, i, e, o, s) = paths(&dir);
let at = |label: &str, model: &str| {
RunStores::create(
&root,
@ -52,6 +53,7 @@ fn a_run_pins_its_stores_and_refuses_drift() {
inventory: &i,
entities: &e,
ontology: &o,
speaker_state: &s,
},
model,
"bag",
@ -147,7 +149,7 @@ fn a_run_works_on_its_own_copy_of_the_inventory() {
let w = world();
let dir = w.dir.path().to_path_buf();
let root = dir.join("run");
let (c, i, e, o) = paths(&dir);
let (c, i, e, o, s) = paths(&dir);
let source_bytes = std::fs::read(&i).unwrap();
let mut run = RunStores::create(
&root,
@ -157,6 +159,7 @@ fn a_run_works_on_its_own_copy_of_the_inventory() {
inventory: &i,
entities: &e,
ontology: &o,
speaker_state: &s,
},
"fake-model",
"bag",
@ -236,6 +239,7 @@ fn create_refuses_a_root_that_holds_an_inventory_and_never_truncates_it() {
inventory,
entities: &dao.entities,
ontology: &dao.ontology,
speaker_state: &common::speaker_state_beside(&dao.ontology),
},
"fake-model",
"bag",
@ -285,6 +289,7 @@ fn a_create_that_fails_leaves_no_inventory_and_the_retry_succeeds() {
inventory,
entities,
ontology: &dao.ontology,
speaker_state: &common::speaker_state_beside(&dao.ontology),
},
"fake-model",
"bag",
@ -398,11 +403,13 @@ fn a_store_changed_between_its_digest_and_its_open_is_drift() {
}
let (c, i, e) = (dir.join("c.cc"), dir.join("i.cc"), dir.join("entities.cc"));
let o = dir.join("ontology.cc");
let s = common::speaker_state_beside(&o);
let paths = || weksa::run::RunPaths {
corpus: &c,
inventory: &i,
entities: &e,
ontology: &o,
speaker_state: &s,
};
assert_eq!(
RunStores::create_between(
@ -457,6 +464,7 @@ fn window(
inventory: &i,
entities: &e,
ontology: &o,
speaker_state: &common::speaker_state_beside(&o),
};
let created = RunStores::create_between(
&root,
@ -888,9 +896,11 @@ fn dao_build(
std::fs::copy(src.join(from), dir.path().join(to)).unwrap();
dir.path().join(to)
};
// The Dao inventory at rev 3: the pins below are written against it.
common::dao_rev3_at(&dir.path().join("i.cc"));
let (corpus, inventory, entities) = (
copy("corpus.cc", "c.cc"),
copy("inventory.cc", "i.cc"),
dir.path().join("i.cc"),
copy("entity.cc", "e.cc"),
);
edit(&corpus, &entities);
@ -905,6 +915,7 @@ fn dao_build(
inventory: &inventory,
entities: &entities,
ontology: &ontology,
speaker_state: &common::speaker_state_beside(&ontology),
},
"fake-model",
"bag",
@ -1087,6 +1098,8 @@ fn lowering_prompts(dao: &Dao, inventory: &InventoryStore) -> Vec<(String, Promp
let view = store
.admit(packet, inventory, &corpus, dao.run.entities())
.unwrap();
let state = weksa::SpeakerStateStore::open(&dir.path().join("s.cc")).unwrap();
let view = view.with_scene(&inv, &state, &corpus).unwrap();
let prompt = Prompt::lower(&view, &key, "en-US", &inv, dao.run.entities(), &weksa::Ontology::empty()).unwrap();
(key.to_string(), prompt)
})
@ -1560,6 +1573,7 @@ fn errors_echo_no_input() {
inventory: &missing,
entities: &missing,
ontology: &missing,
speaker_state: &common::speaker_state_beside(&missing),
};
let e = RunStores::create(
&root,
@ -1629,6 +1643,7 @@ fn errors_echo_no_input() {
inventory: &dao.inventory,
entities: &dao.entities,
ontology: &dao.ontology,
speaker_state: &common::speaker_state_beside(&dao.ontology),
},
"fake-model",
"bag",
@ -2551,6 +2566,7 @@ fn a_changed_head_is_drift() {
inventory: &dao.inventory,
entities: &dao.entities,
ontology: &dao.ontology,
speaker_state: &common::speaker_state_beside(&dao.ontology),
},
"fake-model",
"bag",
@ -2891,6 +2907,7 @@ fn a_summary_line_says_how_lowerings_were_answered() {
inventory: &dao.inventory,
entities: &dao.entities,
ontology: &dao.ontology,
speaker_state: &common::speaker_state_beside(&dao.ontology),
};
RunStores::create(
&root,
@ -3615,6 +3632,7 @@ fn mixed_family_round(family: ModelFamily) -> u32 {
inventory: &dir.join("i.cc"),
entities: &dir.join("entities.cc"),
ontology: &dir.join("ontology.cc"),
speaker_state: &common::speaker_state_beside(&dir.join("ontology.cc")),
},
"not-a-draft",
"bag",
@ -4402,7 +4420,7 @@ fn open_refuses_a_bad_panel() {
let created = RunStores::create(
&root,
"p",
RunPaths { corpus, inventory, entities, ontology: &dao.ontology },
RunPaths { corpus, inventory, entities, ontology: &dao.ontology, speaker_state: &common::speaker_state_beside(&dao.ontology) },
"fake-model",
"bag",
weksa::run::LowererReach::ToolLessRelay,
@ -4762,6 +4780,7 @@ fn create_refuses_an_entry_its_inventory_cannot_read() {
inventory: &dir.join("i.cc"),
entities: &dir.join("e.cc"),
ontology,
speaker_state: &common::speaker_state_beside(ontology),
},
"fake-model",
"bag",
@ -4817,3 +4836,281 @@ fn round_lowering_prompts_carry_selected_entries() {
}
assert_eq!(seen.len(), 6, "three items in each language");
}
/// Admits the speaker-state revision (rev 4) onto the rev-3 Dao inventory at
/// `inventory`, and the scene for the group of `item:daodejing:3`: Laozi's
/// relation to an unnamed party, above, intimacy 200.
fn put_dao_scene(speaker_state: &Path, inventory: &Path, corpus: &Path, entities: &Path) {
use weksa::packet::{FeatValue, FeatureValue};
let mut inv_store = InventoryStore::open(inventory).unwrap();
let (rev, entries) = weksa::inventory::speaker_scene_revision(4);
inv_store.admit_revision(rev, entries).unwrap();
let inv = inv_store.inventory().unwrap();
let id = |gloss: &str| inv.entries().find(|e| e.gloss == gloss).unwrap().id;
let corpus = CorpusStore::open(corpus).unwrap();
let item = corpus.get("item:daodejing:3").unwrap().unwrap();
let scene = weksa::Scene {
group: item.group.clone(),
n: 1,
inventory_rev: 4,
participants: vec![item.speaker.clone()],
relations: vec![weksa::PairRelation {
from: item.speaker.clone(),
to: weksa::Party::Unnamed,
features: vec![
FeatureValue {
feature: id("relative_status"),
value: FeatValue::Option(common::lab("above")),
},
FeatureValue {
feature: id("intimacy"),
value: FeatValue::Grade(common::grade(200)),
},
],
}],
annotator: weksa::Annotator::Operator,
};
weksa::SpeakerStateStore::open(speaker_state)
.unwrap()
.put_scene(scene, &inv_store, &corpus, &weksa::EntityStore::open(entities).unwrap())
.unwrap();
}
/// A Dao run whose store holds a scene for item 3's group and whose ontology
/// holds en-US:1, selected when the speaker is above the addressee.
fn scene_dao(extra: impl FnOnce(&Path, &Path)) -> Dao {
dao_build(
|_, _| {},
&["en-US"],
|ontology, inventory| {
use weksa::{Condition, Layer, OntologyEntry, OntologyStore, Slot, Test};
let dir = ontology.parent().unwrap();
put_dao_scene(
&common::speaker_state_beside(ontology),
inventory,
&dir.join("c.cc"),
&dir.join("e.cc"),
);
let inv_store = InventoryStore::open(inventory).unwrap();
let status = inv_store
.inventory()
.unwrap()
.entries()
.find(|e| e.gloss == "relative_status")
.unwrap()
.id;
OntologyStore::open(ontology)
.unwrap()
.admit(
vec![OntologyEntry {
language: "en-US".into(),
n: 1,
inventory_rev: 4,
layer: Layer::Discourse,
when: vec![Condition {
slot: Slot::Scene,
feature: status,
test: Test::Is(weksa::packet::FeatValue::Option(common::lab("above"))),
}],
choice: "ENABOVE".into(),
examples: vec![],
annotator: weksa::Annotator::Operator,
}],
&inv_store,
)
.unwrap();
extra(&dir.join("c.cc"), &dir.join("e.cc"));
},
)
}
/// Answers for items 1 to 3 where item 3 names referent n2 (no entity) as its
/// addressee, or nobody.
fn addressing(addressee: Option<&str>) -> Vec<(usize, String)> {
(1..=3)
.map(|n| {
let mut v = serde_json::to_value(draft_of(n)).unwrap();
if n == 3 {
v["discourse"]["addressee"] = serde_json::json!(addressee);
}
(n, v.to_string())
})
.collect()
}
fn lowering_of(model: &FakeModel, marker: &str) -> Vec<String> {
lowering_texts(model)
.into_iter()
.filter(|t| t.contains(marker))
.map(String::from)
.collect()
}
#[test]
fn round_prompts_carry_scene_selected_entries() {
// Above an unnamed addressee: item 3's prompt holds the entry, the other
// two (no scene for their groups) read NoScene and print without it.
let mut dao = scene_dao(|_, _| {});
let mut model = fake(&addressing(Some("n2")));
recorded(round(&mut dao.run, &mut model, &Scorer::new(Bag)).unwrap());
assert_eq!(lowering_texts(&model).len(), 3);
let with = lowering_of(&model, "\nonto:en-US:1 [discourse] ENABOVE");
assert_eq!(with.len(), 1, "only item 3 has a scene and an addressee");
assert_eq!(
lowering_texts(&model).iter().filter(|t| t.contains("ontology: none")).count(),
2
);
// The same item with no addressee selects no Scene entry.
let mut dao = scene_dao(|_, _| {});
let mut model = fake(&addressing(None));
recorded(round(&mut dao.run, &mut model, &Scorer::new(Bag)).unwrap());
assert!(lowering_of(&model, "ENABOVE").is_empty());
assert_eq!(lowering_texts(&model).len(), 3);
}
#[test]
fn a_scene_read_failure_is_a_lower_refusal() {
// A live item of item 3's group, in a language outside the panel, spoken by
// someone the scene does not cover: V16 for that group.
let mut dao = scene_dao(|corpus, entities| {
let mut entity_store = weksa::EntityStore::open(entities).unwrap();
entity_store.put(ent("stranger", "en-US", &["Stranger"])).unwrap();
let mut store = CorpusStore::open(corpus).unwrap();
let mut twin = store.get("item:daodejing:3").unwrap().unwrap();
twin.id = "item:daodejing:93".into();
twin.language = "pt-BR".into();
twin.text = "PORTUGUESEMARK 93 o sabio".into();
twin.speaker = common::eid("stranger");
twin.names = vec![];
twin.prior_turns = vec![];
store.add(twin, &entity_store).unwrap();
});
let mut model = fake(&addressing(Some("n2")));
let r = recorded(round(&mut dao.run, &mut model, &Scorer::new(Bag)).unwrap());
// Item 3 never reached the model; items 1 and 2 did, and the round recorded.
assert_eq!(lowering_texts(&model).len(), 2);
assert_eq!(r.n(), 1);
}
#[test]
fn a_run_pins_its_speaker_state_and_refuses_drift() {
let dao = scene_dao(|_, _| {});
let r = dao.run.record();
let path = dao.ontology.with_file_name("speaker-state.cc");
assert_eq!(r.speaker_state, std::fs::canonicalize(&path).unwrap().to_str().unwrap());
assert_eq!(r.speaker_state_digest, hex(&path));
let root = dao.root.clone();
assert_eq!(RunStores::open(&root).unwrap().record(), dao.run.record());
let mut bytes = std::fs::read(&path).unwrap();
bytes.push(0xc0);
std::fs::write(&path, bytes).unwrap();
assert_eq!(
RunStores::open(&root).err(),
Some(RunError::Drift(DriftOf::SpeakerState))
);
}
#[test]
fn create_refuses_a_scene_its_inventory_cannot_read() {
// The scene is admitted against a copy that gained rev 4; the run's head
// inventory stops at rev 3.
let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/daodejing");
let dir = tempfile::tempdir().unwrap();
for (from, to) in [("corpus.cc", "c.cc"), ("entity.cc", "e.cc")] {
std::fs::copy(src.join(from), dir.path().join(to)).unwrap();
}
common::dao_rev3_at(&dir.path().join("i.cc"));
common::dao_rev3_at(&dir.path().join("rich.cc"));
let state = common::speaker_state_in(dir.path());
put_dao_scene(&state, &dir.path().join("rich.cc"), &dir.path().join("c.cc"), &dir.path().join("e.cc"));
let ontology = common::ontology_in(dir.path());
let root = dir.path().join("run");
let create = || {
RunStores::create(
&root,
"x",
RunPaths {
corpus: &dir.path().join("c.cc"),
inventory: &dir.path().join("i.cc"),
entities: &dir.path().join("e.cc"),
ontology: &ontology,
speaker_state: &state,
},
"fake-model",
"bag",
weksa::run::LowererReach::ToolLessRelay,
ModelFamily::Unrecorded,
&["en-US".to_string()],
)
};
assert_eq!(create().err(), Some(RunError::SpeakerState));
assert!(!root.join("run.cc").exists());
assert!(!root.join("inventory.cc").exists());
}
#[test]
fn a_run_reads_entries_and_scenes_at_its_own_head() {
use weksa::{Condition, Layer, OntologyEntry, OntologyStore, Slot, Test};
// An entry and a scene admitted at rev 4 through a Dao-shaped inventory open
// a run whose own inventory is seed, proof and the scene revision as rev 3:
// beds number revisions independently, so a run reads features at its head.
let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/daodejing");
let dir = tempfile::tempdir().unwrap();
for (from, to) in [("corpus.cc", "c.cc"), ("entity.cc", "e.cc")] {
std::fs::copy(src.join(from), dir.path().join(to)).unwrap();
}
let rich = dir.path().join("rich.cc");
common::dao_rev3_at(&rich);
let state = common::speaker_state_in(dir.path());
put_dao_scene(&state, &rich, &dir.path().join("c.cc"), &dir.path().join("e.cc"));
let rich_store = InventoryStore::open(&rich).unwrap();
let status = rich_store.inventory().unwrap().entries().find(|e| e.gloss == "relative_status").unwrap().id;
let ontology = common::ontology_in(dir.path());
OntologyStore::open(&ontology)
.unwrap()
.admit(
vec![OntologyEntry {
language: "en-US".into(),
n: 1,
inventory_rev: 4,
layer: Layer::Discourse,
when: vec![Condition {
slot: Slot::Scene,
feature: status,
test: Test::Is(weksa::packet::FeatValue::Option(common::lab("above"))),
}],
choice: "ENABOVE".into(),
examples: vec![],
annotator: weksa::Annotator::Operator,
}],
&rich_store,
)
.unwrap();
let head = dir.path().join("i.cc");
let mut own = InventoryStore::open(&head).unwrap();
for (rev, entries) in [
weksa::inventory::seed(),
weksa::inventory::proof_revision(),
weksa::inventory::speaker_scene_revision(3),
] {
own.admit_revision(rev, entries).unwrap();
}
drop(own);
let run = RunStores::create(
&dir.path().join("run"),
"x",
RunPaths {
corpus: &dir.path().join("c.cc"),
inventory: &head,
entities: &dir.path().join("e.cc"),
ontology: &ontology,
speaker_state: &state,
},
"fake-model",
"bag",
weksa::run::LowererReach::ToolLessRelay,
ModelFamily::Unrecorded,
&["en-US".to_string()],
);
assert!(run.is_ok(), "{:?}", run.err());
}

186
crates/weksa/tests/original_corpus.rs
View file

@ -18,7 +18,7 @@ use weksa::{CorpusStore, EntityStore, InventoryStore};
fn stores() -> (CorpusStore, EntityStore) {
let dir = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/original");
let tmp = tempfile::tempdir().unwrap();
for f in ["corpus.cc", "entity.cc", "inventory.cc"] {
for f in ["corpus.cc", "entity.cc", "inventory.cc", "speaker-state.cc"] {
std::fs::copy(dir.join(f), tmp.path().join(f)).unwrap();
}
let keep = tmp.keep();
@ -241,14 +241,22 @@ fn the_first_six_original_items_are_unchanged() {
}
#[test]
fn the_original_inventory_is_the_dao_revision_three() {
fn the_original_inventory_is_the_dao_inventory() {
let dao = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/daodejing/inventory.cc");
let own = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/original/inventory.cc");
assert_eq!(std::fs::read(dao).unwrap(), std::fs::read(&own).unwrap());
let tmp = tempfile::tempdir().unwrap();
let copy = tmp.path().join("inventory.cc");
std::fs::copy(&own, &copy).unwrap();
assert!(InventoryStore::open(&copy).is_ok());
let inventory = InventoryStore::open(&copy).unwrap().inventory().unwrap();
// Revision 4 is the speaker-state vocabulary: the same six features the
// generator admits, ids and all.
assert_eq!(inventory.latest().unwrap().n, 4);
let (rev, entries) = weksa::inventory::speaker_scene_revision(4);
assert_eq!(inventory.latest().unwrap().changes, rev.changes);
for e in &entries {
assert_eq!(inventory.entry(&e.id), Some(e), "{}", e.gloss);
}
}
/// The committed seed ontology: every entry model-drafted, exactly the two
@ -270,8 +278,13 @@ fn committed_ontology_is_model_drafted_and_reads_every_inventory() {
match &e.annotator {
Annotator::Model { family, generator } => {
assert_eq!(*family, ModelFamily::Anthropic, "{}", e.key());
assert_eq!(
generator, "claude-sonnet-5-5 (Weksa Hands, model-drafted seed ontology)",
assert!(
[
"claude-sonnet-5-5 (Weksa Hands, model-drafted seed ontology)",
"claude-sonnet-5-5 (Weksa Hands, model-drafted scene ontology)",
"claude-sonnet-5-5 (Weksa Hands, reworded seed entry)",
]
.contains(&generator.as_str()),
"{}",
e.key()
);
@ -283,6 +296,15 @@ fn committed_ontology_is_model_drafted_and_reads_every_inventory() {
langs.sort();
langs.dedup();
assert_eq!(langs, PANEL);
// Every entry from number 10 on is conditioned on the scene relation, and
// only an entry with a Scene condition may name a politeness level.
for e in entries.iter().filter(|e| e.n >= 10) {
assert!(
e.when.iter().any(|c| c.slot == weksa::inventory::Slot::Scene),
"{} has no Scene condition",
e.key()
);
}
for lang in PANEL {
let stance = |c: &Condition| c.slot == weksa::inventory::Slot::Stance;
assert!(
@ -292,21 +314,35 @@ fn committed_ontology_is_model_drafted_and_reads_every_inventory() {
"{lang} has no Stance-conditioned entry"
);
}
for (corpus, panel) in [("original", &PANEL[..]), ("daodejing", &PANEL[..1])] {
// A run opens over every committed inventory bed with its panel. The default
// bed (data/interlingua) numbers the scene revision 3 and holds no Dao
// revision; the Dao beds number it 4. Entries and scenes admitted at 4 open
// over both because a run reads features at its own head. The default bed is
// read with the original corpus, entities and scenes, as `loop open` with its
// defaults and `--corpus data/original` does.
for (corpus, bed, panel) in [
("original", "original", &PANEL[..]),
("daodejing", "daodejing", &PANEL[..1]),
("original", "interlingua", &PANEL[..]),
] {
let dir = data.join(corpus);
let copy = |f: &str| {
std::fs::copy(dir.join(f), tmp.path().join(format!("{corpus}-{f}"))).unwrap();
tmp.path().join(format!("{corpus}-{f}"))
let name = format!("{corpus}-{bed}");
let copy = |from: &Path, f: &str| {
let to = tmp.path().join(format!("{name}-{f}"));
std::fs::copy(from.join(f), &to).unwrap();
to
};
let (c, i, e) = (copy("corpus.cc"), copy("inventory.cc"), copy("entity.cc"));
let (c, i, e) = (copy(&dir, "corpus.cc"), copy(&data.join(bed), "inventory.cc"), copy(&dir, "entity.cc"));
let s = copy(&dir, "speaker-state.cc");
RunStores::create(
&tmp.path().join(format!("run-{corpus}")),
&tmp.path().join(format!("run-{name}")),
corpus,
RunPaths {
corpus: &c,
inventory: &i,
entities: &e,
ontology: &onto,
speaker_state: &s,
},
"m",
"bag",
@ -314,7 +350,34 @@ fn committed_ontology_is_model_drafted_and_reads_every_inventory() {
ModelFamily::Unrecorded,
&panel.iter().map(|l| l.to_string()).collect::<Vec<_>>(),
)
.unwrap_or_else(|e| panic!("{corpus}: {e:?}"));
.unwrap_or_else(|e| panic!("{name}: {e:?}"));
}
// Revision numbers are local to a bed, so agreement is proven on the beds:
// every feature an entry condition or a committed scene relation names means
// the same in all three (gloss, value kind, slots, retirement).
let mut named: std::collections::BTreeSet<weksa::InvId> = entries
.iter()
.flat_map(|e| e.when.iter().map(|c| c.feature))
.collect();
let scenes = weksa::SpeakerStateStore::open(&data.join("original/speaker-state.cc")).unwrap();
for s in scenes.all_scenes().unwrap() {
named.extend(s.relations.iter().flat_map(|r| r.features.iter().map(|f| f.feature)));
}
let beds: Vec<_> = ["interlingua", "original", "daodejing"]
.iter()
.map(|b| {
let copy = tmp.path().join(format!("bed-{b}.cc"));
std::fs::copy(data.join(b).join("inventory.cc"), &copy).unwrap();
InventoryStore::open(&copy).unwrap().inventory().unwrap()
})
.collect();
for id in named {
let shape = |inv: &weksa::Inventory| {
let e = inv.entry(&id).unwrap_or_else(|| panic!("{id} is missing from a bed"));
(e.gloss.clone(), e.value_kind.clone(), e.applies_to.clone(), e.retired_rev)
};
assert_eq!(shape(&beds[0]), shape(&beds[1]), "{id}");
assert_eq!(shape(&beds[1]), shape(&beds[2]), "{id}");
}
}
@ -351,6 +414,7 @@ fn the_recall_probe_covers_every_live_original_item_and_names_no_work() {
inventory: &i,
entities: &e,
ontology: &o,
speaker_state: &common::speaker_state_beside(&o),
},
"m",
"bag",
@ -417,7 +481,7 @@ fn original_round(family: ModelFamily) -> (weksa::run::RunStores, weksa::run::Ro
use weksa::run::{LowererReach, RoundOutcome, RunPaths, RunStores, round};
let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/original");
let tmp = tempfile::tempdir().unwrap().keep();
for f in ["corpus.cc", "entity.cc", "inventory.cc"] {
for f in ["corpus.cc", "entity.cc", "inventory.cc", "speaker-state.cc"] {
std::fs::copy(src.join(f), tmp.join(f)).unwrap();
}
std::fs::write(tmp.join("ontology.cc"), [0x90]).unwrap();
@ -429,6 +493,7 @@ fn original_round(family: ModelFamily) -> (weksa::run::RunStores, weksa::run::Ro
inventory: &tmp.join("inventory.cc"),
entities: &tmp.join("entity.cc"),
ontology: &tmp.join("ontology.cc"),
speaker_state: &common::speaker_state_beside(&tmp.join("ontology.cc")),
},
"nope",
"bag",
@ -575,7 +640,7 @@ fn no_test_item_is_offered_to_a_round_or_recall_or_recorded() {
use weksa::run::{LowererReach, RoundOutcome, RunPaths, RunStores, round};
let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/original");
let tmp = tempfile::tempdir().unwrap();
for f in ["corpus.cc", "entity.cc", "inventory.cc"] {
for f in ["corpus.cc", "entity.cc", "inventory.cc", "speaker-state.cc"] {
std::fs::copy(src.join(f), tmp.path().join(f)).unwrap();
}
std::fs::write(tmp.path().join("ontology.cc"), [0x90]).unwrap();
@ -588,6 +653,7 @@ fn no_test_item_is_offered_to_a_round_or_recall_or_recorded() {
inventory: &p("inventory.cc"),
entities: &p("entity.cc"),
ontology: &p("ontology.cc"),
speaker_state: &common::speaker_state_beside(&p("ontology.cc")),
},
"nope",
"bag",
@ -951,3 +1017,95 @@ fn every_original_passage_rests_on_a_live_inventory_concept() {
assert!(hit.len() >= 12, "{lang}: {} distinct concepts", hit.len());
}
}
/// The committed scenes: one per live Train or Validation group of
/// `data/original`, none for any other group, every one model-annotated and
/// never Test evidence (a Test group gets no scene and reads `NoScene`).
#[test]
fn committed_scenes_cover_original_groups() {
use std::collections::BTreeSet;
use weksa::speaker::Annotator;
use weksa::{CorpusStore, Split, SplitScheme, SpeakerStateStore};
let data = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data/original");
let tmp = tempfile::tempdir().unwrap();
let state = tmp.path().join("speaker-state.cc");
std::fs::copy(data.join("speaker-state.cc"), &state).unwrap();
let scenes = SpeakerStateStore::open(&state).unwrap().all_scenes().unwrap();
let live: BTreeSet<String> = CorpusStore::open(&data.join("corpus.cc"))
.unwrap()
.all()
.unwrap()
.into_iter()
.filter(|i| matches!(SplitScheme::V1.live(i), Some(Split::Train | Split::Validation)))
.map(|i| i.group)
.collect();
assert_eq!(live.len(), 56);
let held: BTreeSet<String> = scenes.iter().map(|s| s.group.clone()).collect();
assert_eq!(held, live, "a scene for each live Train or Validation group, none other");
// Revisions append: a group's later revisions re-derived its relation.
assert!(scenes.iter().all(|s| s.n <= 2));
for s in &scenes {
match &s.annotator {
Annotator::Model { family, generator } => {
assert_eq!(*family, ModelFamily::Anthropic, "{}", s.group);
assert_eq!(
generator, "claude-sonnet-5-5 (Weksa Hands, model-annotated original scenes)",
"{}",
s.group
);
}
other => panic!("{} is {other:?}", s.group),
}
}
}
/// The committed ontology's Scene-conditioned entries, over every scene relation
/// (relative_status x in_group x intimacy in steps of 50): two entries of one
/// language and layer are never selected by the same relation, so the ontology
/// never gives two instructions for one relation in one layer. Whether an entry's
/// text keeps a politeness level on the scene is a model reading owed per change,
/// not a test.
#[test]
fn scene_entries_never_co_select_in_a_layer() {
use weksa::inventory::Slot;
use weksa::ontology::{Condition, Test};
use weksa::packet::FeatValue;
use weksa::OntologyStore;
let data = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../data");
let tmp = tempfile::tempdir().unwrap();
let onto = tmp.path().join("ontology.cc");
std::fs::copy(data.join("ontology/ontology.cc"), &onto).unwrap();
let inv = InventoryStore::open(&data.join("original/inventory.cc")).unwrap().inventory().unwrap();
let id = |gloss: &str| inv.entries().find(|e| e.gloss == gloss).unwrap().id;
let (status, intimacy, in_group) = (id("relative_status"), id("intimacy"), id("in_group"));
let entries = OntologyStore::open(&onto).unwrap().all().unwrap();
let holds = |c: &Condition, rel: (&str, u16, bool)| match (c.feature, &c.test) {
(f, Test::Is(FeatValue::Option(l))) if f == status => l.as_str() == rel.0,
(f, Test::AtLeast(g)) if f == intimacy => rel.1 >= g.get(),
(f, Test::AtMost(g)) if f == intimacy => rel.1 <= g.get(),
(f, Test::Is(FeatValue::Flag(b))) if f == in_group => *b == rel.2,
_ => panic!("a Scene condition this test does not model"),
};
let selects = |e: &weksa::OntologyEntry, rel| {
e.when.iter().all(|c| c.slot != Slot::Scene || holds(c, rel))
&& e.when.iter().all(|c| c.slot == Slot::Scene)
};
for st in ["below", "level", "above"] {
for ig in [false, true] {
for it in (0..=1000).step_by(50) {
let rel = (st, it, ig);
let mut seen = std::collections::BTreeSet::new();
for e in entries.iter().filter(|e| e.when.iter().any(|c| c.slot == Slot::Scene)) {
if selects(e, rel) {
assert!(
seen.insert((e.language.clone(), e.layer.name())),
"{} co-selects with another {} entry at {rel:?}",
e.key(),
e.layer.name()
);
}
}
}
}
}
}

61
crates/weksa/tests/scene.rs
View file

@ -27,7 +27,7 @@ struct W {
fn world() -> W {
let dir = tempfile::tempdir().unwrap();
let mut inventory = InventoryStore::open(&dir.path().join("i.cc")).unwrap();
for (rev, entries) in [seed(), proof_revision(), speaker_scene_revision()] {
for (rev, entries) in [seed(), proof_revision(), speaker_scene_revision(3)] {
inventory.admit_revision(rev, entries).unwrap();
}
let entities = entities_in(dir.path());
@ -60,7 +60,7 @@ fn feat(gloss: &str, value: FeatValue) -> FeatureValue {
let feature = seed()
.1
.into_iter()
.chain(speaker_scene_revision().1)
.chain(speaker_scene_revision(3).1)
.find(|e| e.gloss == gloss)
.unwrap_or_else(|| panic!("no feature {gloss}"))
.id;
@ -408,3 +408,60 @@ fn only_live_speakers_of_the_views_group_and_the_views_own_speaker_must_be_cover
["V16"]
);
}
mod select {
use super::*;
use weksa::speaker::Annotator;
use weksa::{Condition, Layer, OntologyEntry, OntologyStore, Slot, Test, select};
fn scene_feature(gloss: &str) -> weksa::InvId {
speaker_scene_revision(3).1.into_iter().find(|e| e.gloss == gloss).unwrap().id
}
fn entry(n: u32, slot: Slot, gloss: &str, test: Test) -> OntologyEntry {
OntologyEntry {
language: "ja-JP".into(),
n,
inventory_rev: 3,
layer: Layer::Grammar,
when: vec![Condition { slot, feature: scene_feature(gloss), test }],
choice: "a choice".into(),
examples: vec![],
annotator: Annotator::Operator,
}
}
#[test]
fn select_reads_the_scene_relation() {
let mut w = world();
put(&mut w, standard("g", 1));
let mut store = OntologyStore::open(&w.dir.path().join("o.cc")).unwrap();
store
.admit(
vec![
entry(1, Slot::Scene, "relative_status", Test::Is(FeatValue::Option(lab("above")))),
entry(2, Slot::Scene, "intimacy", Test::AtLeast(grade(150))),
],
&w.inventory,
)
.unwrap();
let ontology = store.ontology().unwrap();
let keys = |read: SceneRead, w: &W| {
let view = view_of(w, 3, Some("n3")).with_scene(&inv(w), &w.state, &w.corpus).unwrap();
assert_eq!(view.scene_read(), &read);
select(&ontology, "ja-JP", &view).iter().map(|e| e.n).collect::<Vec<_>>()
};
// Brokkr is above-status at intimacy 200: both Scene entries select.
assert_eq!(keys(selected(1, status()), &w), [1, 2]);
// No scene for the group: neither selects.
let empty = world();
assert_eq!(keys(SceneRead::NoScene, &empty), Vec::<u32>::new());
// No addressee, and no relation to the addressee: neither selects.
let view = view_of(&w, 3, None).with_scene(&inv(&w), &w.state, &w.corpus).unwrap();
assert_eq!(view.scene_read(), &SceneRead::NoAddressee { rev: 1 });
assert!(select(&ontology, "ja-JP", &view).is_empty());
let mut lone = world();
put(&mut lone, scene("g", 1, &["nibu", "brokkr"], vec![relation("brokkr", Party::Named(eid("nibu")), intimacy(99))]));
assert_eq!(keys(SceneRead::NoRelation { rev: 1 }, &lone), Vec::<u32>::new());
}
}

4
crates/weksa/tests/speaker_state.rs
View file

@ -26,7 +26,7 @@ struct W {
fn world() -> W {
let dir = tempfile::tempdir().unwrap();
let mut inventory = InventoryStore::open(&dir.path().join("i.cc")).unwrap();
for (rev, entries) in [seed(), proof_revision(), speaker_scene_revision()] {
for (rev, entries) in [seed(), proof_revision(), speaker_scene_revision(3)] {
inventory.admit_revision(rev, entries).unwrap();
}
let entities = entities_in(dir.path());
@ -60,7 +60,7 @@ fn feat(gloss: &str, value: FeatValue) -> FeatureValue {
let feature = seed()
.1
.into_iter()
.chain(speaker_scene_revision().1)
.chain(speaker_scene_revision(3).1)
.find(|e| e.gloss == gloss)
.unwrap_or_else(|| panic!("no feature {gloss}"))
.id;

47
crates/weksa/tests/stores.rs
View file

@ -1584,7 +1584,9 @@ fn seed_reseeded() {
.unwrap();
assert_eq!(items.len(), 3);
// The committed inventory is the seed, reseeded (stance features in, the
// displayed-affect features out), then the nibu-proof revision.
// displayed-affect features out), the nibu-proof revision and the
// speaker-scene revision as rev 3: never the Dao revision (ruling
// `dao-round-seed-inventory`).
let committed = InventoryStore::open(&root.join("data/interlingua/inventory.cc"))
.unwrap()
.inventory()
@ -1593,6 +1595,8 @@ fn seed_reseeded() {
let (proof, proof_entries) = weksa::inventory::proof_revision();
assert!(committed.has_revision(rev.n) && committed.has_revision(proof.n));
entries.extend(proof_entries);
entries.extend(weksa::inventory::speaker_scene_revision(3).1);
assert_eq!(committed.latest().unwrap().n, 3);
let mut have: Vec<_> = committed.entries().cloned().collect();
let mut want = entries;
have.sort_by_key(|e| e.id);
@ -1740,9 +1744,9 @@ mod ontology {
"five conditions",
vec![OntologyEntry { when: (0..5).map(|_| restraint()).collect(), ..entry("ja-JP", 1) }],
));
for slot in [Slot::Speaker, Slot::Scene, Slot::Predication] {
for slot in [Slot::Speaker, Slot::Predication] {
cases.push((
"slot not Stance or Discourse",
"slot not Stance, Discourse or Scene",
vec![OntologyEntry {
when: vec![when(slot, "restraint", Test::AtLeast(Grade::new(1).unwrap()))],
..entry("ja-JP", 1)
@ -1818,6 +1822,43 @@ mod ontology {
assert_eq!(s.all().unwrap().len(), 64);
}
#[test]
fn ontology_admit_takes_scene_refuses_speaker() {
use weksa::inventory::{proof_revision, speaker_scene_revision};
let d = dir();
let mut inventory = InventoryStore::open(&d.path().join("i.cc")).unwrap();
for (rev, entries) in [seed(), proof_revision(), speaker_scene_revision(3)] {
inventory.admit_revision(rev, entries).unwrap();
}
let scene_feature = |gloss: &str| {
speaker_scene_revision(3).1.into_iter().find(|e| e.gloss == gloss).unwrap().id
};
let at = |slot, feature, test| OntologyEntry {
inventory_rev: 3,
when: vec![Condition { slot, feature, test }],
..entry("ja-JP", 1)
};
let above = Test::Is(FeatValue::Option(lab("above")));
let mut s = store(&d);
// A Scene condition on a Scene feature admits.
assert_eq!(
s.admit(vec![at(Slot::Scene, scene_feature("relative_status"), above.clone())], &inventory)
.unwrap(),
AddOutcome::Added
);
// Speaker and Predication slots refuse, and a Scene condition naming a
// Stance feature refuses.
for bad in [
at(Slot::Speaker, scene_feature("education"), Test::AtLeast(Grade::new(1).unwrap())),
at(Slot::Predication, feature("modality"), Test::Is(FeatValue::Option(lab("actual")))),
at(Slot::Scene, feature("restraint"), Test::AtLeast(Grade::new(1).unwrap())),
] {
let bad = OntologyEntry { n: 2, ..bad };
assert!(s.admit(vec![bad], &inventory).is_err());
}
assert_eq!(s.all().unwrap().len(), 1);
}
#[test]
fn ontology_admit_is_all_or_nothing() {
let d = dir();

1
data/corpus/speaker-state.cc Normal file
View file

@ -0,0 +1 @@
<EFBFBD>

BIN
data/daodejing/inventory.cc
View file

Binary file not shown.

1
data/daodejing/speaker-state.cc Normal file
View file

@ -0,0 +1 @@
<EFBFBD>

BIN
data/interlingua/inventory.cc
View file

Binary file not shown.

BIN
data/ontology/ontology.cc
View file

Binary file not shown.

BIN
data/original/inventory.cc
View file

Binary file not shown.

BIN
data/original/speaker-state.cc Normal file
View file

Binary file not shown.

8
docs/targets/interlingua-lowering-map.md
View file

@ -33,11 +33,11 @@ means no store holds it and nothing writes it.
| Schema version | `weksa.interlingua_packet.v<N>` constant in `crates/weksa`; published JSON Schema at `schemas/cultnet/` | Bumped by a commit that changes the Rust types; the published schema is regenerated in the same commit and a test fails on drift | A cut (Hands, under a spec); never the loop |
| Inventory entry | Key `<family>:<n>`, `n` canonical decimal >= 1 (families: concept, frame, role, relation, speech_act, feature, referent_kind, extension); type `weksa.inventory_entry`, with a bounded gloss, value kind (option labels bounded), the slots a feature may sit in (`Discourse`, `Referent`, `Predication`, `Relation`; `Stance` from cut `speaker-state`, `Speaker` and `Scene` from cut `speaker-scene`), `introduced_rev` and optional `retired_rev`. Speaker axes, scene relations and affects are features of those slots, so the loop grows them like any closed-class family and convergence covers them | Added or retired by an inventory revision; never edited in place (a changed meaning is a retire plus an add); a retired id still resolves for packets of earlier revisions | The loop's accepted rounds, whose minting call the context builder assembles from Train items only; Self on merge to main. Admission checks form, not text |
| Inventory revision | Key `rev:<n>`, type `weksa.inventory_revision`, with parent, change list and `frozen` | Appended per accepted round; exactly one revision is ever marked frozen, by the convergence test | The loop's accept rule appends; the convergence test freezes; Self records the freeze as a ruling |
| Target profile | Locale or language-project id (`en-US`, `ja-JP`, a conlang's slug). Until entries grow in rounds there is no profile revision document: a run names the profile it lowered with by the ontology store's path and digest in its `RunRecord` (cut `language-ontology`); `profile:<language>:<n>` revision documents arrive with follow-up `language-ontology-growth` | The ontology store is revised by commit; a run pins it and refuses drift | Profile authors and accepted rounds; never the lowerer's defaults (`persona-drives-register`) |
| Language ontology entry (cut `language-ontology`) | Key `onto:<language>:<n>`, `n` from 1 per language; type `weksa.ontology_entry` in `data/ontology/ontology.cc`: the language, the inventory revision its conditions are read at, a layer (lexicon, grammar, discourse, phonology, orthography), at most four conditions (a feature id of slot `Stance` or `Discourse` with `AtLeast`/`AtMost` a grade or `Is` an option or flag; slots `Speaker` and `Scene` and community ids join with follow-up `language-ontology-view-assembly`), a choice of 1 to 200 bytes, at most three examples of 1 to 120 bytes, and an `Annotator` | Admitted in all-or-nothing batches through `OntologyStore::admit`, immutable, never edited; retire arrives with `language-ontology-growth`. The run pins the store like the entity store (path and digest, `Drift(Ontology)`), never copies it, because no loop step writes it yet. Cut `language-ontology-seed` converts by drafting, not by parse: a fresh model drafts each language's seed from that language's YAML and the inventory's feature lines with no corpus text, then deletes `data/target-language-ontology/` and `.weksa/cultural-ontology/` in the same cut | Seed: a labelled model (`Annotator::Model`), no operator gate, never Test evidence (`ja-ontology-owner-answer`); attestation (operator, native reader) appends later. Accepted rounds later, through a mint call from Train items only. Selection is `ontology::select(ontology, language, view)`, called only inside `Prompt::lower`, so no caller picks entries; the prompt prints each selected entry's id, layer, choice and examples. Its text is minted text for the leak diagnostic, which today has no runtime caller (follow-up `leak-diagnostic-in-round`) |
| Target profile | Locale or language-project id (`en-US`, `ja-JP`, a conlang's slug). Until entries grow in rounds there is no profile revision document: a run names the profile it lowered with by the ontology store's path and digest in its `RunRecord` (cut `language-ontology`); `profile:<language>:<n>` revision documents arrive with follow-up `language-ontology-growth` | The ontology store is revised by commit; a run pins it and refuses drift; it likewise pins the speaker-state store (`RunRecord` keys 17 and 18, `Drift(SpeakerState)`, cut `language-ontology-scene`) | Profile authors and accepted rounds; never the lowerer's defaults (`persona-drives-register`) |
| Language ontology entry (cut `language-ontology`) | Key `onto:<language>:<n>`, `n` from 1 per language; type `weksa.ontology_entry` in `data/ontology/ontology.cc`: the language, the revision of the inventory it was admitted through (provenance: beds number revisions independently and a run reads each condition's feature at its own head), a layer (lexicon, grammar, discourse, phonology, orthography), at most four conditions (a feature id of slot `Stance`, `Discourse` or `Scene` with `AtLeast`/`AtMost` a grade or `Is` an option or flag; a `Scene` condition reads the relation of the item's speaker to its addressee that `with_scene` selected; slot `Speaker` and community ids wait on follow-up `language-ontology-communities`), a choice of 1 to 200 bytes, at most three examples of 1 to 120 bytes, and an `Annotator`. Whether an entry's text keeps politeness on the scene is a model reader's judgment, owed per change (a mint owes it under `language-ontology-growth`), not a test | Admitted in all-or-nothing batches through `OntologyStore::admit`, immutable, never edited; retire arrives with `language-ontology-growth`. The run pins the store like the entity store (path and digest, `Drift(Ontology)`), never copies it, because no loop step writes it yet. Cut `language-ontology-seed` converts by drafting, not by parse: a fresh model drafts each language's seed from that language's YAML and the inventory's feature lines with no corpus text, then deletes `data/target-language-ontology/` and `.weksa/cultural-ontology/` in the same cut | Seed: a labelled model (`Annotator::Model`), no operator gate, never Test evidence (`ja-ontology-owner-answer`); attestation (operator, native reader) appends later. Accepted rounds later, through a mint call from Train items only. Selection is `ontology::select(ontology, language, view)`, called only inside `Prompt::lower`, so no caller picks entries; the prompt prints each selected entry's id, layer, choice and examples. Its text is minted text for the leak diagnostic, which today has no runtime caller (follow-up `leak-diagnostic-in-round`) |
| Community (cut `language-ontology`, follow-up `language-ontology-communities`) | Key `community:<slug>`; type `weksa.community`, converted from `data/cultural-ontology/cultures/` and `referents/` (YAML deleted in the same cut) | Revised by commit; a speaker names the communities it belongs to by id | The cultural ontology's author (owner-attested evidence only; Discord-only content stays deleted, `discord-only-content-deleted`) |
| Speaker (cut `speaker-scene`) | Key `speaker:<slug>@<n>`; type `weksa.speaker`: stable axes as feature values of slot `Speaker` (origin, socioeconomic standing, age cohort, education and whatever the loop adds), at an inventory revision, and an `Annotator` | A new revision appends, never edits; a corpus item and every prior turn name a speaker slug, and a round records the speaker revisions it read | The annotator it names: `PersonaSnapshot` (a projection of a `gamecult.persona_state.v0` document, naming its source document and `updatedAt`), `Operator`, `WorkMetadata` (a public-domain work's own cast list and narration), or `Model { family, generator }` (the 2b `ModelFamily`) |
| Scene (cut `speaker-scene`) | Key `scene:<group>@<n>`, the item's split group; type `weksa.scene`: participants (speaker slugs) and, per ordered pair, relation features of slot `Scene` (relative status, intimacy, in-group, audience and whatever the loop adds), at an inventory revision, and an `Annotator` | As Speaker. Keyed by the group, so a scene never straddles the split | As Speaker |
| Scene (cut `speaker-scene`) | Key `scene:<group>@<n>`, the item's split group; type `weksa.scene`: participants (speaker slugs) and, per ordered pair, relation features of slot `Scene` (relative status, intimacy, in-group, audience and whatever the loop adds), at an inventory revision (provenance; a run reads the features at its own head), and an `Annotator` | As Speaker. Keyed by the group, so a scene never straddles the split. `data/original/speaker-state.cc` holds Model-annotated keeper scenes, one per Train or Validation group of both languages (`weksa scene put|dump`); a Test group has none and reads `NoScene` | As Speaker; a model-annotated scene is never Test evidence |
| Corpus item | Key `item:<source-slug>:<n>`, `n` canonical decimal >= 1 (no sign, no leading zero, from cut 2b); type `weksa.corpus_item` with text, language, speaker, split group, prior turns, source. Source `ModelWritten` carries a `ModelFamily` enum, the generator string as the fixture wrote it, and a prompt digest (cut 2b). The item carries no speaker state of its own: its speaker slug and group name the Speaker and Scene documents | Admitted once; the whole item is immutable (re-adding an identical item is a no-op, any difference is refused; a correction is a new item that `supersedes` the old); text already held under another key is refused unless the new item supersedes that key (cut 2b); withdrawn, never deleted, so old scores stay explainable | Importers and `weksa corpus add`, under `corpus-cast-plus-public-domain`; the source enum has no variant for Discord or any training-forbidden source (`corpus-clean`) |
| Split assignment | Derived: `split(scheme, item.group)` | Fixed for a scheme; a new scheme is a new constant, never a reassignment. A model-written item whose group derives test is in no split; a superseding item keeps its predecessor's group | The hash function. Nobody assigns by hand, so there is no writer to forbid |
| Loop round | Key `run:<label>:round:<n>`; type `weksa.loop_round` with spec version, profile revisions, model ids and prompt digests, sampled train items, Validation means and refusal counts (from cut `ja-panel-loop`: one set per language of the run's panel, no pooled mean), the digest of the inventory file it was scored under and, for a cycle's candidate round, its cycle `k` | Scored and recorded whole; immutable. A candidate round is put in one batch with its Decision. Lives in the run store on Yggdrasil | `run::round` and `run::cycle` (both through `score_round`) |
@ -424,7 +424,7 @@ One line per cut; specs exist for the cuts about to run (2d and 4, siblings on `
4. `speaker-state`: packet `stance` (felt affect and restraint, slot `Stance`, packet schema v4); formality, warmth and intensity out of the seed and the standard; only `PacketStore::admit` yields a `LoweringInput`, and it refuses packets for withdrawn items (follow-up `lowering-input-from-packet-store`). (spec r1)
4a. `speaker-relation-input`: slot `Scene` and its first seed features (`relative_status`, `intimacy`, `in_group`; inventory revision 3, now `speaker_scene_revision`); `LoweringInput::with_scene` carries the speaker's relation to the addressee as typed `Scene` feature values. Since 4b it takes the `SpeakerStateStore` and the `CorpusStore`, reads the scene of the view's own item group for the item's speaker, and selects the relation by the addressee referent's entity (V16 when the scene does not list the item's speaker or the speaker of any live item of the group, the rule `put_scene` applies too; V17 refuses an addressee that names a predication; V13 is retired). The English lowerer does not render it. (PR for follow-up `chat-s3352-3352`)
4b. `speaker-scene` (spec r1; `speaker-scene-seeds` is its cut 4c: CLI, inventory revision 3, cast speakers and scenes; `Party::Unnamed` covers an addressee with no entity; Speaker and Scene share one `SpeakerStateStore`): slots `Speaker` and `Scene` with their seed features; `weksa.speaker` (`speaker:<entity-slug>@<n>`) and `weksa.scene` (`scene:<group>@<n>`, participants and ordered-pair relations) documents with `Annotator` `Operator` and `Model { family, generator }`, validated by the packet's shared feature check, append-only per key; their stores, `weksa speaker` and `weksa scene` commands, published schemas; Model-annotated cast speakers and scenes for the committed items. Waits on 2d and 4; `pd-speaker-annotation` blocks only the public-domain annotator. Part 1 landed (branch `weksa-lowering/speaker-scene`): `speaker.rs`, `SpeakerStateStore`, slots `Speaker` and `Scene` exclusive to their own feature lists, `speaker_scene_revision()` (Speaker features 22-24), `with_scene` on the store, schemas. The CLI, the committed inventory revision 3 and the seeds stay with 4c.
5. `language-ontology`, in sequential cuts on `d9ebbcf`. 5a `language-ontology` (spec r1): `weksa.ontology_entry`, `OntologyStore`, `ontology::select` over the view's `Stance` and `Discourse` features, `Prompt::lower` printing the selected entries of the item's language, `RunRecord` pinning the ontology store (keys 15 and 16), `weksa ontology admit|dump`; `ontology.rs` joins `PROMPT_SOURCE_DIGEST`. 5b `language-ontology-seed` (spec r1, after 5a): Model-annotated en-US and ja-JP seed entries drafted by an isolated model, `data/target-language-ontology/`, `.weksa/cultural-ontology/` and `schemas/target-language-cultural-ontology.md` deleted. Lost until 5d: keigo, hierarchy and uchi-soto hang on scene relations, so ja-JP politeness is still the lowerer's guess. 5c `language-ontology-view-assembly` (follow-up): the run pins the `SpeakerStateStore`, the round reads the item's Speaker and Scene through the stores, `select` widens to slots `Speaker` and `Scene`; waits on `speaker-scene-seeds`. 5d `language-ontology-communities` (follow-up): `weksa.community` from `data/cultural-ontology/` (YAML deleted, owns `edda-referent-yaml-parse`), community ids on `weksa.speaker`, community conditions; waits on 5c and question `persona-speech-community-home`.
5. `language-ontology`, in sequential cuts on `d9ebbcf`. 5a `language-ontology` (spec r1): `weksa.ontology_entry`, `OntologyStore`, `ontology::select` over the view's `Stance` and `Discourse` features, `Prompt::lower` printing the selected entries of the item's language, `RunRecord` pinning the ontology store (keys 15 and 16), `weksa ontology admit|dump`; `ontology.rs` joins `PROMPT_SOURCE_DIGEST`. 5b `language-ontology-seed` (spec r1, after 5a): Model-annotated en-US and ja-JP seed entries drafted by an isolated model, `data/target-language-ontology/`, `.weksa/cultural-ontology/` and `schemas/target-language-cultural-ontology.md` deleted. Lost until 5c': keigo, hierarchy and uchi-soto hang on the scene relation from 5c', so ja-JP politeness is still the lowerer's guess. 5c' `language-ontology-scene` (spec r1, after 5b): the run pins the `SpeakerStateStore` (keys 17 and 18, `Drift(SpeakerState)`), `RunStores::assembled` is the one door that reads the item's scene into a loop view, `select` reads slot `Scene`, `Prompt::lower` refuses a view whose scene was never read, `weksa scene put|dump` and `weksa inventory speaker-scene` (rev 4 in data/original and data/daodejing, rev 3 in data/interlingua, which holds no Dao revision), Model-annotated keeper scenes and Scene-conditioned entries for `data/original`. 5c `language-ontology-view-assembly` (follow-up, now only the Speaker read): `select` widens to slot `Speaker`; waits on `speaker-scene-seeds`. 5d `language-ontology-communities` (follow-up): `weksa.community` from `data/cultural-ontology/` (YAML deleted, owns `edda-referent-yaml-parse`), community ids on `weksa.speaker`, community conditions; waits on 5c and question `persona-speech-community-home`.
6. `pd-importer`: public-domain play and fiction dialogue into the corpus with speaker, scene group, edition provenance, and Speaker and Scene annotations by the ruled annotator. Waits on 4, follow-up `gutenberg-terms` and question `pd-speaker-annotation`.
7a. `loop-context-builder`: `context.rs` (`TrainItems`, `EncoderOffer`, `Prompt`: corpus text reaches a model call only through `Prompt::mint` over live Train items or `Prompt::encode` over one offered item; instructions are the consts `MINT_INSTRUCTIONS` and `ENCODE_INSTRUCTIONS` in `context.rs`; `Checkpoint` has no public constructor, offers each item once, and its offers borrow it (the gate is typed `TestReadings` and Test offers on the round path; operator tools such as `weksa corpus dump` and `weksa score` still read corpus text of any split); prior turns print the held Train text), `SplitScheme::live`, `CorpusItem::cited_entities` (one owner of the id set `PacketStore::admit` and the offer share), and checkpoint-only Test readings (`Checkpoint`, `TestReadings`; `Aggregate` and `leak_diagnostic` carry no Test). Library only; since 7b `TrainItems::from_corpus` and `EncoderOffer::for_round` are crate-private and `RunStores` is their one public producer. Merges follow-ups `loop-context-builder-2` and `scorer-test-readings-type`. (spec r1)
7b. `golf-loop-round` (spec r1; branch `weksa-lowering/golf-loop-round`): the smallest loop that scores one round. `run.rs`: `RunStores` pins the corpus and entity stores of a run by canonical path and FNV-1a 64 digest at `create`, reopens only from `<root>/run.cc` and refuses drift (corpus, entities, `PROMPT_SOURCE_DIGEST` of `context.rs`, a second pin on `lower.rs` retired in 7d, model, embedder); `round` encodes every live Train and Validation item through a `ModelPort` (`model.rs`; the one adapter `AnswerFiles` writes `<digest>.prompt` and reads `<digest>.answer`, so the model is the Claude session), reads each answer into a `PacketDraft` (`encode.rs`, the one place a model's JSON is read), admits through `PacketStore::admit` into `<root>/round-<n>/packets.cc`, lowers, scores once, and keeps a `RoundRecord` (`weksa.loop_round`, no text, no Test; Train rows, Validation only as means and refusal counts). `weksa loop open|round`. Not here: the spec-edit step, accept rule and composite, noise floor, Score and lowering documents, a subprocess adapter, the checkpoint door, a model lowerer (follow-ups `golf-loop-spec-edit`, `golf-loop-accept-rule`, `golf-loop-score-documents`, `golf-loop-subprocess-adapter`, `golf-loop-checkpoint-door-bound`, `golf-loop-model-lowerer`). `packet-store-inventory-binding` is met for loop packets only (`RunStores` admits with the run's inventory); `validate`, `packet proof|dao` and `lower` still take caller stores. Waits on 7a, `checkpoint-gate`, 4b.

12
schemas/cultnet/weksa.loop_run.schema.json
View file

@ -66,6 +66,14 @@
},
"prompt_digest": {
"type": "string"
},
"speaker_state": {
"description": "Canonical absolute path of the speaker-state store (scenes): read-only\nfor the run, pinned by `speaker_state_digest`.",
"type": "string"
},
"speaker_state_digest": {
"description": "16 hex of the speaker-state file's bytes.",
"type": "string"
}
},
"required": [
@ -83,7 +91,9 @@
"lowerer_reach",
"panel",
"ontology",
"ontology_digest"
"ontology_digest",
"speaker_state",
"speaker_state_digest"
],
"$defs": {
"LowererReach": {

4
schemas/cultnet/weksa.ontology_entry.schema.json
View file

@ -17,7 +17,7 @@
}
},
"inventory_rev": {
"description": "The inventory revision the conditions read against.",
"description": "The revision, of the inventory the entry was admitted through, at which the\nstore read its conditions: provenance only; a run reads the conditions'\nfeatures at its own head.",
"type": "integer",
"format": "uint32",
"minimum": 0
@ -89,7 +89,7 @@
]
},
"Condition": {
"description": "One feature of one slot of the view, tested. Only `Stance` and `Discourse`\nreach a lowering view today.",
"description": "One feature of one slot of the view, tested. `Stance`, `Discourse` and\n`Scene` reach a lowering view; `Speaker` does not yet.",
"type": "object",
"properties": {
"feature": {

1
schemas/cultnet/weksa.scene.schema.json
View file

@ -12,6 +12,7 @@
"type": "string"
},
"inventory_rev": {
"description": "The revision, of the inventory the scene was admitted through, at which\n`put_scene` read its features: provenance only; a run never reads it.",
"type": "integer",
"format": "uint32",
"minimum": 0