diff --git a/.gitignore b/.gitignore index e00572b..1428a57 100644 --- a/.gitignore +++ b/.gitignore @@ -22,4 +22,3 @@ build/ .venv/ venv/ target/ -tools/show-working/out/ diff --git a/.weksa/cultural-ontology/README.md b/.weksa/cultural-ontology/README.md new file mode 100644 index 0000000..a424e82 --- /dev/null +++ b/.weksa/cultural-ontology/README.md @@ -0,0 +1,27 @@ +# Cultural Ontology CultCache Seeds + +This directory stores Weksa cultural ontology as CultCache `.cc` witnesses. + +## Starter Database + +- `starter-targets.cc` + +This store contains one catalog entry and three draft guarded profile entries: + +- `weksa.cultural_ontology.pt-BR.vanilla_brazilian.v0` +- `weksa.cultural_ontology.en.transatlantic.v0` +- `weksa.cultural_ontology.ja-JP.kanto.contemporary.v0` + +The catalog entry is: + +- `weksa.cultural_ontology.catalog.seed_targets.v0` + +The `.cc` file is the typed state witness. This README is only an inspection +guide; it does not own cultural ontology truth. + +The profiles follow `schemas/cultural-ontology-profile.md`: salience axes and +linguistic affordances are nested typed state with feature IDs, activation +gates, constraints, indexicality, evidence, uncertainty, and trace labels. + +Qdrant retrieval may later propose evidence bundles for these profiles. It does +not own accepted profile state. diff --git a/.weksa/cultural-ontology/starter-targets.cc b/.weksa/cultural-ontology/starter-targets.cc new file mode 100644 index 0000000..680b06c Binary files /dev/null and b/.weksa/cultural-ontology/starter-targets.cc differ diff --git a/README.md b/README.md index 2b5d0d3..d327e48 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,7 @@ culture, and constraint, and the planned lowerer that turns an agent's meaning packet into the reader's display language. **What is valuable here.** The interlingua standard and the lowering design: a -meaning packet owns what is meant, the scene state (live: `weksa scene put|dump`, labelled by the rule in -`docs/scene-annotation.md`; speaker documents are planned) owns the speaker's relation to the addressee, which selects register, +meaning packet owns what is meant, a target-language profile owns register, address and politeness, and a lowerer owns surface text and leaves a trace. **The problem it solves.** An agent's outward speech today is one English @@ -25,21 +24,16 @@ measured. published schemas in `schemas/cultnet/`. - `data/corpus/corpus.cc`, `data/entity/entity.cc` and `data/interlingua/inventory.cc`: the English corpus, its entities and the inventory, read with `weksa corpus dump`, - `weksa entity dump` and `weksa inventory dump`. `data/original/inventory.cc` - carries its own Validation-informed lexicon revisions (5 and 6) for lowering - measurements; scored runs use `data/interlingua`. + `weksa entity dump` and `weksa inventory dump`. - [Lowering target](./docs/targets/interlingua-lowering.md) and [its map](./docs/targets/interlingua-lowering-map.md): the design, scoring and cut sequence. -- `data/`: the committed stores (corpora, entities, inventory, language ontology) and research data; there are no target-language profiles. +- `data/`: the cultural ontology and target-language profile data. - `settings/`: setting-specific proving grounds for language projects. **Live versus planned.** Live: docs, fixtures, examples, ontology data and the -packet crate `crates/weksa`, including `weksa lower` (the template lowerer for -proof fixtures) and `weksa score`, and the golf loop's round and edit-accept -cycle (`weksa loop open|round|cycle`), which encodes and lowers through the -model (a prompted Claude subagent, per staged-distill). -Planned: Score and lowering documents and the other panel languages. No +packet crate `crates/weksa`, including the scoring library. Planned: lowering, +the `weksa score` subcommand, and the golf loop that scores lowered English against an expressive corpus. No daemon runs; the earlier Node daemon is parked at tag `parked/weksa-node-body`. ## Ownership Model @@ -85,17 +79,9 @@ English, Brazilian Portuguese, Japanese, and later target-language dialogue. [Flavored English Lowering](./docs/flavored-english-lowering.md) remains the target-specific English receipt. -The language ontology is `data/ontology/ontology.cc` (read it with -`weksa ontology dump`): per language, the lexicon, grammar and discourse choices a lowerer -reads for a stance, a discourse and a scene relation. Only an entry with a Scene condition names an -address or politeness level; the relation, from `data/original/speaker-state.cc`, picks it. Its -en-US and ja-JP entries and the keeper scenes are model-drafted or model-annotated, never Test evidence. Nothing yet grows them from accepted -rounds; until that mint exists they change only by a new `weksa ontology admit`. ja-JP holds Scene entries `onto:ja-JP:10` to `:13`. It holds none for an above-status out-group -addressee, and none for an above-status in-group addressee below intimacy 700 (`:12` still selects -an in-group addressee at intimacy 700 or more, whatever the status). Two entries for the above-status -cases were withdrawn: a paired measure found they made the lowerer match the reference register less -often, and they stay out until a native reader attests them. -ja-JP `:5` (hedging), `:7` (affection) and `:8` (restrained sadness) carry certainty and restraint conditions, through `AtMostOrUnset` bounds (an unrated certainty or restraint does not defeat them), so the hedging, certainty, affection, restraint and sadness entries never co-select on the pairs `:5`/`:6`, `:4`/`:7`, `:6`/`:7` and `:6`/`:8`; ja-JP `:3` (anger) carries `AtMostOrUnset` hedge and affection bounds, as en-US `:3`, so `:3`/`:5` and `:3`/`:7` never co-select; the store has no retire, so that was a rebuild of `data/ontology/ontology.cc` with the ja-JP text unchanged. en-US `:3` (blunt anger) carries `AtMostOrUnset` hedge and affection bounds, and `:5` (hedging) and `:8` (affection) each an `AtMostOrUnset` certainty bound (as ja-JP `:7`), so `:3`/`:5`, `:3`/`:8`, `:5`/`:6` and `:6`/`:8` never co-select; `:5`'s certainty clause is that condition and no longer prose; `:6` and the Scene entry `:10` still co-select and are kept apart by scoping in the text. ja-JP `:9` (surprise) states its question ending as available, not prescribed, so `:6`/`:9`, `:5`/`:9` and `:3`/`:9` stay co-selectable and are reconciled by wording, not by condition. +See [Target-Language Cultural Ontology Profiles](./data/target-language-ontology/README.md) +for the reviewable address, register, politeness, idiom, metaphor, and script +distinctions that target lowerers must inspect before producing output. For the first AquaSynth speech-training handoff, see [Tiny PanPhon v0.1 Speech Curriculum](./examples/speech-training/tiny-panphon-v0.1/README.md). diff --git a/crates/weksa/examples/show_working.rs b/crates/weksa/examples/show_working.rs deleted file mode 100644 index 6a31b26..0000000 --- a/crates/weksa/examples/show_working.rs +++ /dev/null @@ -1,398 +0,0 @@ -//! Build-time generator for the "Show her working" page. Runs the real -//! admission door, lowerer and IPA reader over the committed proof packet and -//! variants of it, and prints one JSON document on stdout: the typed slots, the -//! inventory entries the packet chose, and the outcome, which is lowered text or -//! a named refusal. Scoring is not here: `tools/show-working/build.py` calls the -//! `weksa score` CLI on the text this emits. -//! -//! Run from the repository root: `cargo run --release --example show_working`. -//! Every variant is a mutation of `proof_packet()`, spelled in its `source`. - -use std::path::Path; - -use serde_json::{Value, json}; -use weksa::packet::{ - EntityId, FeatValue, FeatureValue, Grade, NodeId, ProvenanceSource, RoleFill, proof_packet, -}; -use weksa::{ - CorpusStore, EntityStore, InvId, Inventory, InventoryStore, LowerError, Packet, PacketStore, - StoreError, lower, pronounce, -}; - -struct Fixture { - id: &'static str, - title: &'static str, - note: &'static str, - source: &'static str, - packet: Packet, -} - -fn id(s: &str) -> InvId { - s.parse().expect("inventory id") -} - -fn node(s: &str) -> NodeId { - NodeId::parse(s).expect("node id") -} - -fn grade(feature: &str, v: u16) -> FeatureValue { - FeatureValue { - feature: id(feature), - value: FeatValue::Grade(Grade::new(v).expect("grade")), - } -} - -fn fixtures() -> Vec { - let proof = proof_packet(); - let mut out = vec![Fixture { - id: "proof", - title: "Nibu warns the player off the control", - note: "The committed proof packet, unchanged.", - source: "proof_packet() in crates/weksa/src/packet.rs", - packet: proof.clone(), - }]; - let mut add = |id, title, note, source, f: &dyn Fn(&mut Packet)| { - let mut packet = proof.clone(); - f(&mut packet); - out.push(Fixture { id, title, note, source, packet }); - }; - add( - "no-addressee", - "Same meaning, nobody addressed", - "With the addressee unset the player is a third party: no \"you\", no reflexive.", - "proof_packet() with discourse.addressee = None", - &|p| p.discourse.addressee = None, - ); - add( - "stance-held-back", - "Anger held back, restraint high", - "Stance is the speaker's felt affect. The lowerer has no row that reads it, so the text is the proof's text.", - "proof_packet() with stance anger=300, restraint=700", - &|p| p.stance.features = vec![grade("feature:9", 300), grade("feature:18", 700)], - ); - add( - "hedged", - "Hedged and uncertain warning", - "Hedge and certainty are meaning the lowerer cannot yet say. It refuses rather than drop them.", - "proof_packet() with discourse hedge=600, certainty=400", - &|p| p.discourse.features = vec![grade("feature:6", 600), grade("feature:7", 400)], - ); - add( - "negated", - "Negated intervention (polarity: negative)", - "Dropping the negation would say the opposite, so the lowerer refuses. This is the shape of a negated Dao line awaiting polarity.", - "proof_packet() with predication n5 polarity=negative", - &|p| { - p.predications[1].features = vec![FeatureValue { - feature: id("feature:1"), - value: FeatValue::Option("negative".parse().expect("label")), - }] - }, - ); - add( - "consent-authored", - "Addressee consent, authored source", - "Someone who saw the request says the addressee asked for it. No lowering speaks for that request yet.", - "proof_packet() with discourse.addressee_consent = n4 (source Authored)", - &|p| p.discourse.addressee_consent = Some(node("n4")), - ); - add( - "consent-imported", - "Addressee consent, imported-corpus source", - "A reader of wording cannot have seen a request, so the packet is refused at admission (V15), before any lowering.", - "proof_packet() with addressee_consent = n4 and provenance.source = ImportedCorpus", - &|p| { - p.discourse.addressee_consent = Some(node("n4")); - p.provenance.source = ProvenanceSource::ImportedCorpus; - }, - ); - add( - "relation", - "A causal relation between the predications", - "Relations are not rendered; the lowerer refuses instead of asserting a different meaning.", - "proof_packet() with relation:8 (causes) n4 -> n5", - &|p| { - p.relations = vec![weksa::packet::Relation { - relation: id("relation:8"), - from: node("n4"), - to: node("n5"), - features: vec![], - }] - }, - ); - add( - "dangling-filler", - "A role filled by a node that does not exist", - "Refused at admission (V4): the packet is not a packet.", - "proof_packet() with predication n5 agent filler n9", - &|p| { - p.predications[1].roles[0] = RoleFill { role: id("role:1"), filler: node("n9") } - }, - ); - add( - "unknown-entity", - "A referent naming an entity nobody holds", - "The entity store is the only owner of names; a packet cannot invent one.", - "proof_packet() with referent n3 entity = entity:nobody", - &|p| p.referents[2].entity = Some(EntityId::parse("entity:nobody").expect("entity")), - ); - add( - "common-noun-agent", - "The ship mind as a common noun", - "No entity: the referent lowers as \"the ship mind\", and there is no name to pronounce.", - "proof_packet() with referent n3 entity = None", - &|p| p.referents[2].entity = None, - ); - out -} - -fn gloss(inv: &Inventory, rev: u32, i: &InvId) -> Value { - match inv.resolve(i, rev) { - Some(e) => json!({ "id": i.to_string(), "gloss": e.gloss }), - None => json!({ "id": i.to_string(), "gloss": null }), - } -} - -fn feature_json(inv: &Inventory, rev: u32, slot: &str, at: &str, f: &FeatureValue) -> Value { - let mut g = gloss(inv, rev, &f.feature); - g["slot"] = json!(slot); - g["at"] = json!(at); - g["value"] = serde_json::to_value(&f.value).expect("value"); - g -} - -/// Every feature the packet carries, with where it sits. -fn features(p: &Packet, inv: &Inventory) -> Vec { - let rev = p.inventory_rev; - let mut v = Vec::new(); - for f in &p.discourse.features { - v.push(feature_json(inv, rev, "discourse", "discourse", f)); - } - for f in &p.stance.features { - v.push(feature_json(inv, rev, "stance", "stance", f)); - } - for r in &p.referents { - for f in &r.features { - v.push(feature_json(inv, rev, "referent", r.id.as_str(), f)); - } - } - for pr in &p.predications { - for f in &pr.features { - v.push(feature_json(inv, rev, "predication", pr.id.as_str(), f)); - } - } - v -} - -/// Ids the packet chose, once each, with the inventory's gloss at its revision. -fn chosen(p: &Packet, inv: &Inventory) -> Vec { - let rev = p.inventory_rev; - let mut ids: Vec = Vec::new(); - let mut push = |i: &InvId| { - if !ids.contains(i) { - ids.push(*i) - } - }; - p.context.setting.iter().for_each(&mut push); - p.context.medium.iter().for_each(&mut push); - p.discourse.speech_act.iter().for_each(&mut push); - for r in &p.referents { - push(&r.kind); - push(&r.concept); - } - for pr in &p.predications { - push(&pr.concept); - push(&pr.frame); - pr.roles.iter().for_each(|f| push(&f.role)); - } - p.relations.iter().for_each(|r| push(&r.relation)); - features(p, inv) - .iter() - .filter_map(|f| f["id"].as_str().and_then(|s| s.parse().ok())) - .for_each(|i: InvId| push(&i)); - ids.iter().map(|i| gloss(inv, rev, i)).collect() -} - -fn slots(p: &Packet) -> Value { - let j = |v: &dyn erased::Ser| v.to_value(); - json!({ - "meaning": { - "referents": p.referents.iter().map(|r| j(r)).collect::>(), - "predications": p.predications.iter().map(|r| j(r)).collect::>(), - "relations": p.relations.iter().map(|r| j(r)).collect::>(), - }, - "stance": j(&p.stance), - "discourse": j(&p.discourse), - "context": j(&p.context), - "extensions": p.extensions.iter().map(|r| j(r)).collect::>(), - "constraints": j(&p.constraints), - "provenance": { "source": j(&p.provenance.source), "corpus_item": p.provenance.corpus_item.to_string(), "confidence": p.provenance.confidence.get() }, - "kind": j(&p.kind), - "inventory_rev": p.inventory_rev, - }) -} - -mod erased { - pub trait Ser { - fn to_value(&self) -> serde_json::Value; - } - impl Ser for T { - fn to_value(&self) -> serde_json::Value { - serde_json::to_value(self).expect("serialize") - } - } -} - -fn lower_error(e: &LowerError) -> Value { - let (variant, reason) = match e { - LowerError::Unrendered { what } => ("Unrendered", format!("the packet carries {what}, which the lowerer does not render")), - LowerError::NoRow { id } => ("NoRow", format!("no table row for {id}")), - LowerError::GlossMismatch { id, row } => ("GlossMismatch", format!("{id} is not the {row} the row speaks for")), - LowerError::UnknownRole { frame, role } => ("UnknownRole", format!("{frame} names role {role}, which the inventory lacks")), - LowerError::MissingRole { predication, role } => ("MissingRole", format!("{predication} fills no {role} role")), - LowerError::UnknownEntity { entity } => ("UnknownEntity", format!("the entity store does not hold {entity}")), - LowerError::NoForm { entity } => ("NoForm", format!("{entity} has no en-US form")), - LowerError::NoPronunciation { entity } => ("NoPronunciation", format!("{entity} has no ruled en-US pronunciation")), - LowerError::NoEntityReferent => ("NoEntityReferent", "the packet cites no entity".to_string()), - LowerError::UnknownPhone { entity, at, .. } => ("UnknownPhone", format!("the ruled pronunciation of {entity} holds a symbol at {at} with no listed phone")), - LowerError::DanglingStress { entity, at } => ("DanglingStress", format!("the ruled pronunciation of {entity} has a stress mark at {at} with no vowel after it")), - LowerError::ConsentMissing { predication } => ("ConsentMissing", format!("{predication} is a request the packet does not carry the addressee's consent to")), - LowerError::AskerNotAddressee { predication } => ("AskerNotAddressee", format!("the agent of {predication} is not the addressee")), - LowerError::EntityRead { entity, .. } => ("EntityRead", format!("the entity store failed reading {entity}")), - LowerError::SceneUnread => ("SceneUnread", "the view's scene was never read".to_string()), - LowerError::SpeakerTokenTaken => ("SpeakerTokenTaken", "a referent cites the entity id the speaker token uses".to_string()), - }; - json!({ "variant": variant, "reason": reason }) -} - -struct World { - inventory: InventoryStore, - corpus: CorpusStore, - entities: EntityStore, - inv: Inventory, -} - -/// Admit `packet` through a fresh packet store and lower it. `Ok` carries the -/// text and the IPA outcome; `Err` is the refusal, as JSON. -fn run(w: &World, dir: &Path, packet: &Packet) -> Result<(String, Value), Value> { - let mut store = PacketStore::open(&dir.join("p.cc")).expect("packet store"); - let view = match store.admit(packet.clone(), &w.inventory, &w.corpus, &w.entities) { - Ok(v) => v, - Err(StoreError::Invalid(rs)) => { - return Err(json!({ - "kind": "admission_refused", - "refusals": rs.iter().map(|r| json!({ "rule": r.rule(), "detail": r.to_string() })).collect::>(), - })); - } - Err(e) => { - return Err(json!({ "kind": "admission_refused", "refusals": [{ "rule": "store", "detail": e.to_string() }] })); - } - }; - let text = lower(&view, &w.inv, &w.entities).map_err(|e| { - json!({ "kind": "lower_error", "error": lower_error(&e) }) - })?; - let ipa = match pronounce(&view, &w.entities) { - Ok(v) => json!({ "ok": v }), - Err(e) => json!({ "error": lower_error(&e) }), - }; - Ok((text, ipa)) -} - -fn main() { - let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); - let inventory = InventoryStore::open(&root.join("data/interlingua/inventory.cc")).expect("inventory"); - let inv = inventory.inventory().expect("inventory view"); - let w = World { - corpus: CorpusStore::open(&root.join("data/corpus/corpus.cc")).expect("corpus"), - entities: EntityStore::open(&root.join("data/entity/entity.cc")).expect("entities"), - inventory, - inv, - }; - let dir = std::env::temp_dir().join(format!("show-working-{}", std::process::id())); - std::fs::create_dir_all(&dir).expect("scratch dir"); - let mut out = Vec::new(); - for fx in fixtures() { - let p = &fx.packet; - let first = run(&w, &dir, p); - let outcome = match first { - Err(e) if e["kind"] == "admission_refused" => e, - first => { - // Ablation: run again with each feature removed, against the - // baseline outcome. It is how the page says whether the lowerer - // read a feature, instead of the page asserting it. - let base = first.as_ref().ok().map(|(t, _)| t.clone()); - let n_disc = p.discourse.features.len(); - let n_stance = p.stance.features.len(); - let total = features(p, &w.inv).len(); - let mut effects = Vec::new(); - for k in 0..total { - let mut q = p.clone(); - if k < n_disc { - q.discourse.features.remove(k); - } else if k < n_disc + n_stance { - q.stance.features.remove(k - n_disc); - } else { - let mut at = k - n_disc - n_stance; - 'o: { - for r in &mut q.referents { - if at < r.features.len() { r.features.remove(at); break 'o; } - at -= r.features.len(); - } - for pr in &mut q.predications { - if at < pr.features.len() { pr.features.remove(at); break 'o; } - at -= pr.features.len(); - } - } - } - let _ = std::fs::remove_file(dir.join("p.cc")); - effects.push(match (&base, run(&w, &dir, &q)) { - (Some(t), Ok((u, _))) if *t == u => "Without it the text is identical: the lowerer did not read it.", - (Some(_), Ok(_)) => "Without it the text changes: the lowerer read it.", - (Some(_), Err(_)) => "Without it the packet is refused.", - (None, Ok(_)) => "Without it the packet lowers: this feature is what the lowerer refuses.", - (None, Err(_)) => "Still refused without this feature alone.", - }); - } - match first { - Ok((text, ipa)) => json!({ "kind": "lowered", "text": text, "ipa": ipa, "feature_effects": effects }), - Err(e) => { let mut e = e; e["feature_effects"] = json!(effects); e } - } - } - }; - let _ = std::fs::remove_file(dir.join("p.cc")); - let mut slots = slots(p); - slots["features"] = json!(features(p, &w.inv)); - out.push(json!({ - "id": fx.id, - "title": fx.title, - "note": fx.note, - "source": fx.source, - "packet_id": p.id.to_string(), - "corpus_item": p.provenance.corpus_item.to_string(), - "slots": slots, - "chosen": chosen(p, &w.inv), - "outcome": outcome, - })); - } - let _ = std::fs::remove_dir_all(&dir); - println!("{}", serde_json::to_string_pretty(&json!({ "fixtures": out })).expect("json")); -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn refusal_carries_the_closed_reason_never_a_debug_of_the_error() { - // UnknownPhone holds the ruled IPA text; no field of the output may. - let e = LowerError::UnknownPhone { - entity: EntityId::parse("entity:nibu").expect("entity id"), - at: 3, - text: "CANARY-IPA".into(), - }; - let v = lower_error(&e); - assert!(!v.to_string().contains("CANARY")); - let keys: Vec<_> = v.as_object().expect("object").keys().cloned().collect(); - assert_eq!(keys, ["reason", "variant"]); - } -} diff --git a/crates/weksa/src/context.rs b/crates/weksa/src/context.rs index 0ecd067..4dd5e30 100644 --- a/crates/weksa/src/context.rs +++ b/crates/weksa/src/context.rs @@ -1,61 +1,28 @@ -//! The one place prompt text is built. Corpus text reaches a model call only as -//! a [`Prompt`], and a `Prompt` is made only by minting from [`TrainItems`], by -//! encoding one item an [`EncoderOffer`] holds, or by lowering one -//! [`LoweringInput`], which holds no corpus text, item key, group or speaker: -//! Train-only minting, per-item encoding and text-free lowering -//! (leak-owned-by-flow-and-scorer-3, loop-context-builder-2). Instruction text -//! is [`MINT_INSTRUCTIONS`], [`ENCODE_INSTRUCTIONS`] and [`LOWER_INSTRUCTIONS`], -//! consts in this module; every prompt also lists the inventory at the run's -//! revision. A mint prompt is the editor's: it reads [`TrainItems`], an -//! [`Inventory`] and an [`EditFeedback`] and nothing else, so no held-out text, -//! key or per-item score can reach it. A lowering prompt names the language it -//! is said in on a language line; every language is named by its tag, none by a -//! default. A lowering prompt also prints the target language's ontology -//! entries that `ontology::select` chose for the view; the caller never names -//! entries. +//! The one place prompt text is built from corpus text. Corpus text reaches a +//! model call only as a [`Prompt`], and a `Prompt` is made only by minting from +//! [`TrainItems`] or by encoding one item an [`EncoderOffer`] holds: Train-only +//! minting and per-item encoding (leak-owned-by-flow-and-scorer-3, +//! loop-context-builder-2). Instruction text is [`MINT_INSTRUCTIONS`] and +//! [`ENCODE_INSTRUCTIONS`], consts in this module; an encode prompt also lists +//! the inventory at the run's revision. //! -//! An ontology mint prompt is a second Train-only prompt: the [`TrainItems`] of -//! one language, the inventory's Stance, Discourse and Scene features (the only -//! ones a condition can read) and that language's existing entry lines. It reads -//! no packet, no score and no held-out text. -//! -//! Item text may hold line breaks; the renderer indents every continuation line -//! (`indented`), so item text cannot forge a `[key]` line or a paragraph. //! Item blocks are data from the run's pinned corpus, reached only through //! `RunStores`, the one public producer of `TrainItems` and `EncoderOffer` //! (forged-item-text-in-prompt). A forged item in a scratch store is a different //! run with a different corpus digest, visible in its record; nothing inside a //! process its forger controls can stop it. -use std::collections::BTreeMap; use std::marker::PhantomData; -use serde::Serialize; - -use crate::corpus::{CorpusItem, CorpusItemId}; -use crate::inventory::{Inventory, InventoryEntry, OptionLabel, Slot, ValueKind}; -use crate::lower::{EntityForms, LowerError, entity}; -use crate::encode::{EntryOntologyDraft, OntologyDraft}; -use crate::ontology::{Condition, Layer, Ontology, Test, select}; -use crate::packet::{ - Context, Discourse, EntityId, Extension, FeatValue, Grade, Predication, Referent, Relation, Stance, -}; -use crate::run::LADDER_STEP; +use crate::corpus::CorpusItem; +use crate::inventory::{Inventory, InventoryEntry, OptionLabel, ValueKind}; +use crate::packet::EntityId; use crate::split::{Checkpoint, Split, SplitScheme, fnv1a64, hex16}; use crate::store::{CorpusStore, normalized}; -use crate::validate::{LoweringInput, SceneRead}; -/// The instructions a mint prompt starts with: the editor's whole brief. Fixed -/// per run by `PROMPT_SOURCE_DIGEST`. -pub const MINT_INSTRUCTIONS: &str = "\ -Propose one edit to the inventory below, so that the lines below can be encoded and lowered, each in its own language, more faithfully. -Answer with one JSON object and nothing else: no code fence, no commentary. -Shape: {\"add\": [{\"family\", \"gloss\", \"value_kind\", \"applies_to\"}], \"retire\": [ids]}; 1 to 8 changes in all, adds and retires together. -- family: one of Concept, Frame, Role, Relation, SpeechAct, Feature, ReferentKind, Extension. -- gloss: 1 to 120 bytes naming a general meaning, never a quoted line. -- value_kind and applies_to only on a Feature; value_kind only on an Extension. value_kind is \"Grade\", \"Flag\" or {\"Options\": [labels]}; applies_to is a list of slots. -- ids and revision numbers are the store's: write an id only to retire a live entry, by the id the inventory lists. -The line after this brief says whether your last edit was kept and the held-out composite score at the current inventory."; +/// The instructions a mint prompt starts with. golf-loop-en edits the wording +/// here. +pub const MINT_INSTRUCTIONS: &str = "Mint glosses from the lines below."; /// The instructions an encode prompt starts with: the encoder's whole brief. /// Fixed per run by `PROMPT_SOURCE_DIGEST`. @@ -70,43 +37,21 @@ Fields: - predications: a list of {id, concept, frame, roles, features}; roles is a list of {role, filler}, filler a node id. - relations: a list of {relation, from, to, features}. - stance: {features}; the speaker's felt affect and restraint, never the affect the line displays. -Node ids are n1, n2, and so on, one per referent and predication. Every other id comes from the inventory listed below and must be live at its revision. Write each id in full, exactly as the inventory lines spell it, family prefix included: concept:7, never 7. An entity comes only from the entities line. A referent's concept is never null, even when it carries an entity: give the nearest concept id. Participants and audience list the node ids of referents, never an entity id: an entity speaker is a referent whose entity field holds it, and participants lists that referent's node id. A predication holds each role at most once, and a feature list each feature at most once: two or more fillers of one role are one referent of kind group that fills the role, and each member is related to it by a member_of relation from the member to the group. +Node ids are n1, n2, and so on, one per referent and predication. Every other id comes from the inventory listed below and must be live at its revision. An entity comes only from the entities line. A feature is {feature, value}; value is {\"Grade\": 0 to 1000}, {\"Option\": label} or {\"Flag\": true or false}, as the inventory lists the feature's kind."; -/// The instructions a lowering prompt starts with: the lowerer's whole brief. -/// Fixed per run by `PROMPT_SOURCE_DIGEST`. -pub const LOWER_INSTRUCTIONS: &str = "\ -Say the packet below in the language on the language line, as one utterance of its speaker: all of its meaning and nothing it does not carry. -Every id is the inventory's listed below and every feature value is as listed; a grade is permille, 0 to 1000. The stance is the speaker's felt affect and restraint: it shapes how the utterance is said, not what it says. The ontology lines are choices this language makes for this speaker, stance and scene: realize each where the packet allows; they add no meaning. -A referent with an entity is named by one of that entity's forms on the entities line. -Answer with one JSON object and nothing else: {\"text\": \"...\"}. No code fence, no commentary."; - -/// The instructions of a recall probe (`lowering-recall-diagnostic`): a -/// completion of a passage from its opening, asked of a model that has no -/// packet and no inventory. It measures recitation, not lowering. -pub const RECALL_INSTRUCTIONS: &str = "\ -Below is the opening of a passage, with what is known of where it comes from. Continue it exactly as its source words it, from memory, until the passage ends. If you do not remember it, give your best guess in the same voice. -Answer with one JSON object and nothing else: {\"text\": \"...\"}, the continuation only, not the opening. No code fence, no commentary."; - -/// The instructions of an ontology mint: drafting entries for one language from -/// its Train lines. Fixed per run by `PROMPT_SOURCE_DIGEST`. -pub const MINT_ONTOLOGY_INSTRUCTIONS: &str = "\ -Below are lines written in one language (the language line), the features an ontology condition may test, and the entries that language's ontology already has. Draft new entries: choices this language's culture makes, for a speaker in a stance and scene, about word, grammar, discourse, sound or script, that the lines show. -Answer with one JSON object and nothing else: {\"entries\": [{\"layer\": \"...\", \"when\": [...], \"choice\": \"...\", \"examples\": [...]}]}. No code fence, no commentary. -Give only entries the lines show, as many as they support (a language holds at most 64 in all). layer is exactly one of Lexicon, Grammar, Discourse, Phonology, Orthography. when holds at most 4 conditions, each {\"slot\": \"Stance\" | \"Discourse\" | \"Scene\", \"feature\": \"\", \"test\": ...}, where is the id printed before the colon of a listed feature (\"feature:10\", never its name) and its slot is one that feature lists. test is {\"AtLeast\": }, {\"AtMost\": } or {\"AtMostOrUnset\": } on a grade feature, or {\"Is\": {\"Option\": \"