diff --git a/crates/eureka-pipeline/README.md b/crates/eureka-pipeline/README.md index d6097b8..5d9c633 100644 --- a/crates/eureka-pipeline/README.md +++ b/crates/eureka-pipeline/README.md @@ -15,7 +15,8 @@ like the rest of the workspace. Committed (runs in `cargo test -p eureka-pipeline`): `encoded_envelopes_match_the_committed_golden` asserts key, cultcache type id, schema name and payload bytes of every kind and every encoding-relevant variant against `golden/envelopes.txt`. A diff -there is a wire change; do not regenerate it to make the test pass. +there is a wire change; do not regenerate it to make the test pass. An additive +field adds golden lines and never changes an existing one. Opt-in, against a real store: `stored_documents_read_back` decodes, validates and keys every stored pipeline document and asserts that it re-encodes to the diff --git a/crates/eureka-pipeline/golden/envelopes.txt b/crates/eureka-pipeline/golden/envelopes.txt index 1519dcf..a5e6fc8 100644 --- a/crates/eureka-pipeline/golden/envelopes.txt +++ b/crates/eureka-pipeline/golden/envelopes.txt @@ -9,6 +9,7 @@ # live mind store (eureka-body:verdict:cut-leaf-into-huginn.s1) and to # Epiphany ef956865 modulo that rename (verdict ...s2). Never regenerate to make # a failing test pass: a diff here is a wire change and needs its own ruling. +# cut_spec.reads-and-symbol and finding.location-symbol were appended after 44c010c added CodeLocation.symbol and PipelineCutSpec.reads (ruling context-pack-read-anchors); no earlier line changed. sample.campaign eureka-state:campaign:self epiphany.pipeline.campaign.v2 EpiphanyPipelineCampaignDocument 9185a4736c7567ac657572656b612d7374617465a57469746c65b5457572656b6120706970656c696e65207374617465a57265706f7391b147616d6543756c742f4570697068616e79ae776f726b696e675f6272616e6368bb636f6465782f657572656b612d706970656c696e652d7374617465aa7461726765745f646f6384a470617468d9256e6f7465732f657572656b612d706970656c696e652d73746174652d7461726765742e6d64aa73746172745f6c696e6501a8656e645f6c696e6509a6636f6d6d6974a83566393832323864 sample.target eureka-state:target:r2 epiphany.pipeline.target.v2 EpiphanyPipelineTargetDocument 9186a863616d706169676eac657572656b612d7374617465a87265766973696f6e02aa696e76617269616e74739182a56c6162656cab6d696e642d61646d697473a973746174656d656e74b64f6e6c792061646d697373696f6e207772697465732eac6e6f745f696e5f73636f706591ac4576652062726f7773696e67b963616e6f6e6963616c5f696d706c656d656e746174696f6e7391a743756c744c6962a3646f6384a470617468d9256e6f7465732f657572656b612d706970656c696e652d73746174652d7461726765742e6d64aa73746172745f6c696e6501a8656e645f6c696e6509a6636f6d6d6974a83566393832323864 sample.question eureka-state:question:Q1 epiphany.pipeline.question.v2 EpiphanyPipelineQuestionDocument 9189a863616d706169676eac657572656b612d7374617465a56c6162656ca25131a57469746c65b357686f206f776e73207468652073746174653fa87175657374696f6eb357686f206f776e73207468652073746174653fa76f7074696f6e739282a56c6162656ca141a474657874ab616e20696e7374616e636582a56c6162656ca142a474657874a661207265706fab7265636f6d6d656e646564a141a7646570656e647391a6437574203361a97261697365645f696e82a46b696e64a743757453706563a26964bf657572656b612d73746174653a6375745f737065633a6375742d33612e7231a861736b65645f6f6eaa323032362d30392d3135 @@ -40,3 +41,5 @@ finding.plausible-blocker-preexisting-kinds eureka-state:finding:cut-3a.s2.F4 ep finding.severity-medium eureka-state:finding:cut-3a.s2.F4 epiphany.pipeline.finding.v2 EpiphanyPipelineFindingDocument 918da863616d706169676eac657572656b612d7374617465a776657264696374be657572656b612d73746174653a766572646963743a6375742d33612e7332a56c6162656ca24634a572616e676582a462617365a83566393832323864a468656164a83566393832323864aa636f6e666964656e6365a9436f6e6669726d6564a87365766572697479a64d656469756da5636c61696dd9214120646f74746564206c6162656c20636f6d706f7365732074776f206b6579732eaa696e76617269616e747391ab6d696e642d61646d697473a96c6f636174696f6e739183a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e6501a8656e645f6c696e6509b06661696c7572655f7363656e6172696fbc54776f20646f63756d656e747320636c61696d206f6e65206b65792ea865766964656e63659183a46b696e64a454657374a76c6f6361746f72aa636172676f2074657374a6726573756c74a26f6baa707265636564656e747390a66f726967696eaa496e74726f6475636564 finding.severity-low eureka-state:finding:cut-3a.s2.F4 epiphany.pipeline.finding.v2 EpiphanyPipelineFindingDocument 918da863616d706169676eac657572656b612d7374617465a776657264696374be657572656b612d73746174653a766572646963743a6375742d33612e7332a56c6162656ca24634a572616e676582a462617365a83566393832323864a468656164a83566393832323864aa636f6e666964656e6365a9436f6e6669726d6564a87365766572697479a34c6f77a5636c61696dd9214120646f74746564206c6162656c20636f6d706f7365732074776f206b6579732eaa696e76617269616e747391ab6d696e642d61646d697473a96c6f636174696f6e739183a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e6501a8656e645f6c696e6509b06661696c7572655f7363656e6172696fbc54776f20646f63756d656e747320636c61696d206f6e65206b65792ea865766964656e63659183a46b696e64a454657374a76c6f6361746f72aa636172676f2074657374a6726573756c74a26f6baa707265636564656e747390a66f726967696eaa496e74726f6475636564 ruling.authority-mind eureka-state:ruling:R8 epiphany.pipeline.ruling.v2 EpiphanyPipelineRulingDocument 918aa863616d706169676eac657572656b612d7374617465a56c6162656ca25238a57469746c65b9416e20696e7374616e6365206f776e7320697473206d696e64a7616e7377657273c0a663686f696365c0a672756c696e67ba416e20696e7374616e6365206f776e7320697473206d696e642eae6f70657261746f725f71756f7465c0a872756c65645f6f6eaa323032362d30392d3135aa707265636564656e74739185a47265706fb147616d6543756c742f4165746865726961a6636f6d6d6974d92861616161616161616161616161616161616161616161616161616161616161616161616161616161a46b696e64a652756c696e67a26964b363756c7463616368653a72756c696e673a5231ae7061796c6f61645f736861323536d94062626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262626262a9617574686f72697479a44d696e64 +cut_spec.reads-and-symbol eureka-state:cut_spec:cut-3a.r1 epiphany.pipeline.cut_spec.v2 EpiphanyPipelineCutSpecDocument 91de0013a863616d706169676eac657572656b612d7374617465a3637574a23361a87265766973696f6e01a57469746c65b2506970656c696e6520646f63756d656e7473a47265706fb147616d6543756c742f4570697068616e79a66272616e6368bb636f6465782f657572656b612d706970656c696e652d7374617465a462617365a83566393832323864aa646570656e64735f6f6e91a132a5666972737491ae526561642074686520737065632ea764656c657465739183a470617468a66f6c642e7273a56c696e657303a46e6f7465a464656164ab6b656570735f6d6f76657391ac636f6d6d6974206f776e6572a46164647391b5706970656c696e655f646f63756d656e74732e7273ac66696c655f6368616e6765739182a86c6f636174696f6e84a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e6501a8656e645f6c696e6509a673796d626f6cab76616c75655f7479706573a66368616e6765a3616464a572656164739282a86c6f636174696f6e84a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e650aa8656e645f6c696e6514a673796d626f6caf506970656c696e6543757453706563a3776879b274686520646f63756d656e7420736861706582a86c6f636174696f6e83a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e651ea8656e645f6c696e65c0a3776879a86f6e65206c696e65ad617574686f726974795f6d617087a56f776e6572a4636f7265a6696e7075747391af747970656420646f63756d656e7473a76f75747075747391a9656e76656c6f706573ad646572697665645f737461746591ac64657269766564206b657973b1666f7262696464656e5f7772697465727391a34d4350ac7368617265645f706174687391a961646d697373696f6ead64656c6574696f6e5f6c696e65a36e2f61ac766572696669636174696f6e84a66275696c647391ab636172676f20636865636ba574657374739182a46e616d65a46b657973a470696e73af6f6e652064657269766564206b6579a86e656761746976659182a77061747465726ea75665633c75383ea573636f7065a9646f63756d656e7473a86f70657261746f7291a46e6f6e65a8657374696d61746588ab6c696e65735f6164646564cd0384ad6c696e65735f72656d6f76656400b2646570656e64656e636965735f616464656491a8736368656d617273b4646570656e64656e636965735f72656d6f76656490ad666f726d6174735f616464656491b66570697068616e792e706970656c696e652e2a2e7631af666f726d6174735f72656d6f76656490ad746172676574735f616464656490af746172676574735f72656d6f76656490a772756c696e677391b6657572656b612d73746174653a72756c696e673a5238a97175657374696f6e7391b8657572656b612d73746174653a7175657374696f6e3a5131 +finding.location-symbol eureka-state:finding:cut-3a.s2.F4 epiphany.pipeline.finding.v2 EpiphanyPipelineFindingDocument 918da863616d706169676eac657572656b612d7374617465a776657264696374be657572656b612d73746174653a766572646963743a6375742d33612e7332a56c6162656ca24634a572616e676582a462617365a83566393832323864a468656164a83566393832323864aa636f6e666964656e6365a9436f6e6669726d6564a87365766572697479a448696768a5636c61696dd9214120646f74746564206c6162656c20636f6d706f7365732074776f206b6579732eaa696e76617269616e747391ab6d696e642d61646d697473a96c6f636174696f6e739184a470617468d9216372617465732f657572656b612d706970656c696e652f7372632f6c69622e7273a46c696e6501a8656e645f6c696e6509a673796d626f6cab2631313533303930343938b06661696c7572655f7363656e6172696fbc54776f20646f63756d656e747320636c61696d206f6e65206b65792ea865766964656e63659183a46b696e64a454657374a76c6f6361746f72aa636172676f2074657374a6726573756c74a26f6baa707265636564656e747390a66f726967696eaa496e74726f6475636564 diff --git a/crates/eureka-pipeline/src/lib.rs b/crates/eureka-pipeline/src/lib.rs index abced06..07c6857 100644 --- a/crates/eureka-pipeline/src/lib.rs +++ b/crates/eureka-pipeline/src/lib.rs @@ -27,7 +27,9 @@ //! cannot be declared without being validated. Format rules live in the field's //! type (`Label`, `Slug`, `OrgRepo`, `Sha`, `Sha256Hex`, `Date`), never in a //! hand-written impl that a later field can slip past. A `Vec` field must carry -//! a maximum, because `Vec` has no `Bounded` impl of its own. +//! a maximum, because `Vec` has no `Bounded` impl of its own. A field added +//! at the epoch carries the `= absent` marker, which defaults it on read and +//! omits it on write when empty. //! //! The wrappers are crate-private: outside code registers, prepares and //! decodes them through `register_pipeline_document_types`, @@ -82,6 +84,24 @@ impl Bounded for Option { } } +/// A value with an empty spelling. The `= absent` marker on a `value_types!` +/// field requires it, and the field is then omitted on write when it is absent. +pub(crate) trait Absent { + fn absent(&self) -> bool; +} + +impl Absent for Option { + fn absent(&self) -> bool { + self.is_none() + } +} + +impl Absent for Vec { + fn absent(&self) -> bool { + self.is_empty() + } +} + macro_rules! unbounded_scalars { ($($ty:ty),* $(,)?) => {$( impl Bounded for $ty { @@ -238,6 +258,14 @@ fn title_text(field: &str, value: &str) -> Result<(), PipelineRefusal> { within(200)(field, value) } +/// A symbol is 1 to 128 bytes with no whitespace and no control character. +fn symbol_text(field: &str, value: &str) -> Result<(), PipelineRefusal> { + if value.is_empty() || value.chars().any(|character| character.is_whitespace() || character.is_control()) { + return Err(format_error(field, value)); + } + within(128)(field, value) +} + bounded_text! { /// Text of at most 200 UTF-8 bytes. Short = 200, |field, value| within(200)(field, value); @@ -249,6 +277,11 @@ bounded_text! { Title = 200, title_text; /// Text of at most 1,000 UTF-8 bytes. Line = 1000, |field, value| within(1000)(field, value); + /// A name the location's lines must contain as written there (an + /// identifier or a YAML `&fileID`); not an identity. 1 to 128 bytes, no + /// whitespace and no control character. + #[schemars(extend("minLength" = 1))] + Symbol = 128, symbol_text; /// Text of at most 4,000 UTF-8 bytes; longer narrative is cited by `DocRef`. Para = 4000, |field, value| within(4000)(field, value); /// A key label: `[A-Za-z0-9_-]{1,64}`. @@ -415,19 +448,25 @@ macro_rules! bounded_field { /// The single field list: one invocation emits the value struct and its /// `Bounded` impl, so a field can never be left out of validation. macro_rules! value_types { - ($($(#[$attr:meta])* pub struct $name:ident { $($field:ident: $ty:ty $([$max:literal])?),* $(,)? } $(=> $extra:path)?)*) => {$( + ($($(#[$attr:meta])* pub struct $name:ident { $($field:ident: $ty:ty $([$max:literal])? $(= $marker:ident)?),* $(,)? } $(=> $extra:path)?)*) => {$( $(#[$attr])* #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] pub struct $name { $( $(#[schemars(extend("maxItems" = $max))])? + // `= absent`: defaulted on read, omitted on write when absent. + // The `cfg_attr(any(), ..)` is a no-op that binds `$marker` to the group. + $(#[serde(default, skip_serializing_if = "Absent::absent")] #[cfg_attr(any(), $marker)])? pub $field: $ty ),* } impl Bounded for $name { fn validate(&self, at: &str) -> Result<(), PipelineRefusal> { - $(bounded_field!(self.$field, &format!("{at}.{}", stringify!($field)) $(, $max)?);)* + $( + $(let _: fn(&$ty) -> bool = <$ty as Absent>::$marker;)? + bounded_field!(self.$field, &format!("{at}.{}", stringify!($field)) $(, $max)?); + )* $($extra(self, at)?;)? Ok(()) } @@ -472,6 +511,33 @@ fn campaign_repos_are_distinct(campaign: &PipelineCampaign, at: &str) -> Result< Ok(()) } +/// The most lines one read anchor may span. +const READ_WINDOW_MAX_LINES: u32 = 100; + +/// A location starts at line 1 or later, ends no earlier than it starts, and a +/// read anchor spans at most `READ_WINDOW_MAX_LINES`. The bounds are on +/// `ReadAnchor` alone: `CodeLocation` elsewhere keeps its stored shape. +fn read_window_is_bounded(anchor: &ReadAnchor, at: &str) -> Result<(), PipelineRefusal> { + let location = &anchor.location; + if location.line < 1 { + return Err(format_error(&format!("{at}.location.line"), &location.line.to_string())); + } + if let Some(end_line) = location.end_line { + if end_line < location.line { + return Err(format_error(&format!("{at}.location.end_line"), &end_line.to_string())); + } + let span = end_line - location.line + 1; + if span > READ_WINDOW_MAX_LINES { + return Err(PipelineRefusal::FieldBound { + field: format!("{at}.location.end_line"), + limit: READ_WINDOW_MAX_LINES, + actual: span, + }); + } + } + Ok(()) +} + /// A run claims work, never runs, rulings or campaigns: its claims are cut /// specs, findings and follow-ups, the three kinds a Self takes on. And its /// budget is a canonical decimal (`0`, `12`, `7.5`: no sign, no leading or @@ -509,7 +575,7 @@ unit_enums! { } value_types! { - pub struct CodeLocation { path: Short, line: u32, end_line: Option } + pub struct CodeLocation { path: Short, line: u32, end_line: Option, symbol: Option = absent } pub struct CommitRange { base: Sha, head: Sha } pub struct Evidence { kind: EvidenceKind, locator: Line, result: Line } /// A citation of a document in another repo's pipeline store (D6). @@ -519,6 +585,8 @@ value_types! { pub struct QuestionOption { label: Label, text: Line } pub struct CutDelete { path: Short, lines: u32, note: Line } pub struct FileChange { location: CodeLocation, change: Line } + /// Code Hands must see but not change, in the spec's repo at the spec's base. + pub struct ReadAnchor { location: CodeLocation, why: Short } => read_window_is_bounded pub struct AuthorityMap { owner: Line, inputs: Vec[16], outputs: Vec[16], derived_state: Vec[16], forbidden_writers: Vec[16], shared_paths: Vec[16], deletion_line: Line, @@ -572,7 +640,7 @@ value_types! { pub struct PipelineCutSpec { campaign: Slug, cut: Label, revision: u32, title: Title, repo: OrgRepo, branch: Short, base: Sha, depends_on: Vec[8], first: Vec[16], deletes: Vec[64], keeps_moves: Vec[64], - adds: Vec[64], file_changes: Vec[256], authority_map: Option, + adds: Vec[64], file_changes: Vec[256], reads: Vec[16] = absent, authority_map: Option, verification: CutVerification, estimate: StructuralDelta, rulings: Vec[32], questions: Vec[16], } @@ -737,6 +805,7 @@ macro_rules! pipeline_kinds { /// shape, not a document's. #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, JsonSchema)] #[serde(tag = "kind", content = "value", rename_all = "snake_case")] + #[allow(clippy::large_enum_variant, reason = "one value per request or per batch of at most 64; boxing a variant would buy no memory and change every construction site")] pub enum PipelineDocument { $($variant($value)),* } impl PipelineKind { @@ -967,7 +1036,8 @@ fn local_max(kind: PipelineKind) -> usize { /// The schema epoch every pipeline store is written at, owned here with the /// schemas it names. Evolution is additive and keeps it: a new named field -/// with a serde default, or a widened `PipelineKind`, since each reader ships +/// with the `value_types!` `= absent` marker (a replayed batch re-encodes byte +/// for byte, because an absent field is not written), or a widened `PipelineKind`, since each reader ships /// with the variants it knows and refuses an unknown kind on the kind, not on /// the epoch. A breaking change bumps it, so that a store written at the old /// one can be refused by whoever opens it; this crate owns no store and @@ -1105,7 +1175,7 @@ pub fn validate_pipeline_write_envelope(envelope: &CultCacheEnvelope) -> Result< #[cfg(test)] mod tests { use super::*; - use std::collections::BTreeSet; + use std::collections::{BTreeMap, BTreeSet}; use std::path::Path; const CAMPAIGN: &str = "eureka-state"; @@ -1144,7 +1214,7 @@ mod tests { } fn location() -> CodeLocation { - CodeLocation { path: s("crates/eureka-pipeline/src/lib.rs"), line: 1, end_line: Some(9) } + CodeLocation { path: s("crates/eureka-pipeline/src/lib.rs"), line: 1, end_line: Some(9), symbol: None } } fn evidence() -> Evidence { @@ -1196,6 +1266,7 @@ mod tests { deletes: vec![CutDelete { path: s("old.rs"), lines: 3, note: "dead".into() }], keeps_moves: vec!["commit owner".into()], adds: vec!["pipeline_documents.rs".into()], file_changes: vec![FileChange { location: location(), change: "add".into() }], + reads: vec![], authority_map: Some(AuthorityMap { owner: "core".into(), inputs: vec!["typed documents".into()], outputs: vec!["envelopes".into()], derived_state: vec!["derived keys".into()], forbidden_writers: vec!["MCP".into()], @@ -1425,6 +1496,16 @@ mod tests { out.push((format!("finding.severity-{name}"), D::Finding(f))); } out.push(("ruling.authority-mind".into(), D::Ruling(mind_ruling()))); + let PipelineDocument::CutSpec(mut spec) = samples().remove(4).0 else { unreachable!() }; + spec.file_changes[0].location.symbol = Some("value_types".into()); + spec.reads = vec![ + ReadAnchor { location: CodeLocation { path: s("crates/eureka-pipeline/src/lib.rs"), line: 10, end_line: Some(20), symbol: Some("PipelineCutSpec".into()) }, why: "the document shape".into() }, + ReadAnchor { location: CodeLocation { path: s("crates/eureka-pipeline/src/lib.rs"), line: 30, end_line: None, symbol: None }, why: "one line".into() }, + ]; + out.push(("cut_spec.reads-and-symbol".into(), D::CutSpec(spec))); + let mut f = finding_sample(); + f.locations[0].symbol = Some("&1153090498".into()); + out.push(("finding.location-symbol".into(), D::Finding(f))); out } @@ -1827,7 +1908,7 @@ mod tests { // refuses this one: the digits rule is satisfied either way. refused(&format!("{CAMPAIGN}:cut_spec:cut-3a.1")); refused(&format!("{CAMPAIGN}:CUT_SPEC:cut-3a.r1")); - refused(&format!("EUREKA-STATE:cut_spec:cut-3a.r1")); + refused("EUREKA-STATE:cut_spec:cut-3a.r1"); refused(&format!("{CAMPAIGN}:cut_spec:cut-3a.r1:")); refused(&format!("{CAMPAIGN}::cut-3a.r1")); refused("cut-3a.r1"); @@ -2964,6 +3045,133 @@ mod tests { ); } + fn read_anchor(line: u32, end_line: Option, symbol: Option<&str>) -> ReadAnchor { + ReadAnchor { + location: CodeLocation { path: s("crates/eureka-pipeline/src/lib.rs"), line, end_line, symbol: symbol.map(Symbol::from) }, + why: "read".into(), + } + } + + fn cut_spec_sample() -> PipelineCutSpec { + let PipelineDocument::CutSpec(spec) = samples().remove(4).0 else { unreachable!() }; + spec + } + + /// Every committed golden line decodes, from its committed payload bytes, + /// to the corpus document it was derived from. The pre-field lines carry no + /// `reads` or `symbol` key, so they decode to empty and `None` only + /// because both fields default on read. + #[test] + fn golden_payloads_decode_to_their_documents() -> Result<()> { + let cache = schema_cache()?; + let golden = include_str!("../golden/envelopes.txt"); + let lines: BTreeMap<&str, &str> = golden + .lines() + .filter(|line| !line.starts_with('#') && !line.is_empty()) + .filter_map(|line| Some((line.split(' ').next()?, line.rsplit(' ').next()?))) + .collect(); + let corpus = golden_documents(); + assert_eq!(lines.len(), corpus.len()); + for (label, document) in corpus { + let hex = lines[label.as_str()]; + let mut envelope = document.prepare(&cache)?; + envelope.payload = (0..hex.len()).step_by(2).map(|at| u8::from_str_radix(&hex[at..at + 2], 16).unwrap()).collect(); + assert_eq!(PipelineDocument::decode(&envelope)?, document, "{label} decodes to its document"); + } + Ok(()) + } + + /// The MCP door: JSON with no `reads` key and locations with no `symbol` + /// key is the pre-field shape, and it decodes. + #[test] + fn pre_field_json_decodes() -> Result<()> { + let json = serde_json::to_value(PipelineDocument::CutSpec(cut_spec_sample()))?; + assert!(json["value"].get("reads").is_none(), "an empty reads is not written"); + assert!(json["value"]["file_changes"][0]["location"].get("symbol").is_none(), "an absent symbol is not written"); + let PipelineDocument::CutSpec(spec) = serde_json::from_value::(json)? else { unreachable!() }; + assert!(spec.reads.is_empty()); + assert_eq!(spec.file_changes[0].location.symbol, None); + Ok(()) + } + + #[test] + fn symbol_and_reads_round_trip() -> Result<()> { + let mut spec = cut_spec_sample(); + spec.reads = (0..16).map(|index| read_anchor(index + 1, Some(index + 100), Some("value_types"))).collect(); + let mut finding = finding_sample(); + finding.locations[0].symbol = Some("&1153090498".into()); + for document in [PipelineDocument::CutSpec(spec), PipelineDocument::Finding(finding)] { + document.validate()?; + let json = serde_json::to_value(&document)?; + assert_eq!(serde_json::from_value::(json.clone())?, document); + let packed = rmp_serde::to_vec_named(&document)?; + assert_eq!(rmp_serde::from_slice::(&packed)?, document); + assert!(packed.windows(6).any(|window| window == b"symbol"), "the symbol key is written"); + assert_eq!(packed.windows(5).any(|window| window == b"reads"), matches!(document, PipelineDocument::CutSpec(_))); + assert!(json.to_string().contains("\"symbol\"")); + } + Ok(()) + } + + #[test] + fn read_and_symbol_bounds() { + let with_reads = |reads: Vec| { + let mut spec = cut_spec_sample(); + spec.reads = reads; + PipelineDocument::CutSpec(spec).validate() + }; + let with_symbol = |symbol: &str| { + let mut finding = finding_sample(); + finding.locations[0].symbol = Some(symbol.into()); + PipelineDocument::Finding(finding).validate() + }; + assert_eq!(with_reads((0..16).map(|_| read_anchor(1, Some(100), None)).collect()), Ok(())); + assert_eq!( + with_reads((0..17).map(|_| read_anchor(1, Some(100), None)).collect()), + Err(PipelineRefusal::FieldBound { field: "cut_spec.reads".into(), limit: 16, actual: 17 }) + ); + assert_eq!( + with_reads(vec![read_anchor(5, Some(4), None)]), + Err(PipelineRefusal::InvalidFormat { field: "cut_spec.reads[0].location.end_line".into(), value: "4".into() }) + ); + assert_eq!( + with_reads(vec![read_anchor(0, None, None)]), + Err(PipelineRefusal::InvalidFormat { field: "cut_spec.reads[0].location.line".into(), value: "0".into() }) + ); + assert_eq!(with_reads(vec![read_anchor(7, Some(7), None)]), Ok(()), "a one-line span"); + assert_eq!(with_reads(vec![read_anchor(10, Some(109), None)]), Ok(()), "a span of 100"); + assert_eq!( + with_reads(vec![read_anchor(10, Some(110), None)]), + Err(PipelineRefusal::FieldBound { field: "cut_spec.reads[0].location.end_line".into(), limit: 100, actual: 101 }) + ); + assert_eq!(with_symbol(&"s".repeat(128)), Ok(())); + assert_eq!( + with_symbol(&"s".repeat(129)), + Err(PipelineRefusal::FieldBound { field: "finding.locations[0].symbol".into(), limit: 128, actual: 129 }) + ); + for bad in ["a b", "a\tb", "", "a\u{0}b", "a\u{3000}b"] { + assert_eq!( + with_symbol(bad), + Err(PipelineRefusal::InvalidFormat { field: "finding.locations[0].symbol".into(), value: bad.into() }), + "{bad:?}" + ); + } + // A location outside a read anchor keeps its stored shape. + let mut finding = finding_sample(); + finding.locations[0].line = 0; + finding.locations[0].end_line = Some(0); + assert_eq!(PipelineDocument::Finding(finding).validate(), Ok(())); + } + + /// The marker needs an `Absent` spelling; the two impls say what absent is. + #[test] + fn absent_is_none_and_empty() { + assert!(Absent::absent(&None::)); + assert!(!Absent::absent(&Some(0u32))); + assert!(Absent::absent(&Vec::::new())); + assert!(!Absent::absent(&vec![0u32])); + } + /// Cut 8: the leaf owns the serialisation of the shapes it owns, so a wire /// carrying a document or a refusal reads back the same value rather than /// a mirror definition kept in step by hand in the reader's repo. Both diff --git a/crates/eureka-state/tests/client.rs b/crates/eureka-state/tests/client.rs index ce9decb..123c508 100644 --- a/crates/eureka-state/tests/client.rs +++ b/crates/eureka-state/tests/client.rs @@ -56,10 +56,11 @@ fn cut_spec(cut: &str, file_changes: usize) -> PipelineDocument { adds: lines(64), file_changes: (0..file_changes) .map(|n| FileChange { - location: CodeLocation { path: short(n), line: 1, end_line: Some(9) }, + location: CodeLocation { path: short(n), line: 1, end_line: Some(9), symbol: None }, change: line(n), }) .collect(), + reads: vec![], authority_map: Some(AuthorityMap { owner: line(0), inputs: lines(16), diff --git a/crates/huginn-daemon/src/daemon.rs b/crates/huginn-daemon/src/daemon.rs index 8e54d34..c5453ff 100644 --- a/crates/huginn-daemon/src/daemon.rs +++ b/crates/huginn-daemon/src/daemon.rs @@ -150,7 +150,7 @@ impl> Daemon { HuginnMindResponse::Admit(outcome) } HuginnMindRequest::View { id, .. } => match self.mind.view(&id) { - Ok(view) => HuginnMindResponse::View(view), + Ok(view) => HuginnMindResponse::View(view.map(Box::new)), Err(refusal) => HuginnMindResponse::Refused(refusal), }, HuginnMindRequest::Query { selection, semantic: None, .. } => match self.mind.query(&selection) { @@ -353,11 +353,12 @@ pub(crate) mod tests { /// roughly a megabyte of field content, which is what a real-sized answer /// looks like at the leaf's bounds rather than at a fixture's convenience. /// `depends_on`, `rulings` and `questions` stay empty because each is a - /// reference admission resolves. + /// reference admission resolves. `reads` and location symbols stay absent: + /// `FITTING_CHANGES` is calibrated to `MAX_RESPONSE_BYTES` without them. pub(crate) fn cut_spec(cut: &str, file_changes: usize) -> PipelineDocument { let lines = |count: usize| (0..count).map(line).collect::>(); let shorts = |count: usize| (0..count).map(short).collect::>(); - let location = |n: usize| CodeLocation { path: short(n), line: 1, end_line: Some(9) }; + let location = |n: usize| CodeLocation { path: short(n), line: 1, end_line: Some(9), symbol: None }; PipelineDocument::CutSpec(PipelineCutSpec { campaign: slug(CAMPAIGN), cut: cut.into(), @@ -372,6 +373,7 @@ pub(crate) mod tests { keeps_moves: lines(64), adds: lines(64), file_changes: (0..file_changes).map(|n| FileChange { location: location(n), change: line(n) }).collect(), + reads: vec![], authority_map: Some(AuthorityMap { owner: line(0), inputs: lines(16), @@ -493,9 +495,11 @@ pub(crate) mod tests { Ok(Daemon::new(Mind::open(state_root, instance)?, NoIndex)) } + type Seen = Vec<(Vec, Vec)>; + #[derive(Clone, Default)] struct RecordingIndex { - seen: Arc, Vec)>>>, + seen: Arc>, } impl IndexSink for RecordingIndex { @@ -605,7 +609,7 @@ pub(crate) mod tests { let PipelinePageItems::Documents(views) = &page.items else { panic!("the document projection is the view") }; assert_eq!(views[0].status, PipelineStatus::InForce); let view = daemon.handle(HuginnMindRequest::View { instance: slug(INSTANCE), id: id.clone() }, now()).answered(); - assert_eq!(view, HuginnMindResponse::View(Some(views[0].clone()))); + assert_eq!(view, HuginnMindResponse::View(Some(Box::new(views[0].clone())))); } /// Ruling 14 across the transport, on both sides: the mind refuses a read diff --git a/crates/huginn-daemon/src/index/mod.rs b/crates/huginn-daemon/src/index/mod.rs index fef9ebb..d52f5b6 100644 --- a/crates/huginn-daemon/src/index/mod.rs +++ b/crates/huginn-daemon/src/index/mod.rs @@ -596,7 +596,7 @@ impl Projector { let found = self.index.search(&self.collection, &vector, limit, budget.left()?)?; let mut hits = Vec::with_capacity(found.len()); for hit in found { - match PipelineKind::ALL.into_iter().find(|kind| kind.name() == hit.kind) { + match PipelineKind::ALL.iter().find(|kind| kind.name() == hit.kind) { Some(kind) => hits.push((PipelineRef { kind: *kind, id: Short(hit.doc_id) }, hit.score)), None => eprintln!("huginn: the index returned {} under the unknown kind {:?}; it is left out", hit.doc_id, hit.kind), } diff --git a/crates/huginn-daemon/src/index/tests.rs b/crates/huginn-daemon/src/index/tests.rs index d18f25b..2e53a16 100644 --- a/crates/huginn-daemon/src/index/tests.rs +++ b/crates/huginn-daemon/src/index/tests.rs @@ -19,10 +19,13 @@ use super::*; use crate::daemon::Daemon; use crate::daemon::tests::{INSTANCE, OTHER, batch, campaign_seed, now, question_and_ruling, slug}; +type AfterAbsorb = Box; +type Edit = (&'static str, Box); + thread_local! { /// Runs once, in the worker's own turn, between the inbox being absorbed /// and a search being taken, holding the guard `serve_search` holds. - static AFTER_ABSORB: std::cell::RefCell>> = const { std::cell::RefCell::new(None) }; + static AFTER_ABSORB: std::cell::RefCell> = const { std::cell::RefCell::new(None) }; } /// The hook `serve_search` calls at the point where a second lock once left a @@ -167,7 +170,7 @@ fn reconcile_indexes_exactly_the_indexable_documents_the_collection_lacks() { fn a_model_digest_change_rebuilds_the_collection() { let (_root, mind) = mind_with_documents(); let entries = mind.index_entries(None).unwrap(); - let changes: [(&str, Box); 3] = [ + let changes: [Edit; 3] = [ ("digest", Box::new(|identity| identity.digest = "d2".into())), ("name", Box::new(|identity| identity.name = "other-model".into())), ("dimensions", Box::new(|identity| identity.dimensions = 8)), @@ -1352,7 +1355,7 @@ fn a_model_changed_refusal_names_the_state_and_the_pending_count() { fn a_search_re_reads_the_collections_label() { let (_root, mind) = mind_with_documents(); let entries = mind.index_entries(None).unwrap(); - let cases: [(&str, Box); 3] = [ + let cases: [Edit; 3] = [ ( "another mind's label", Box::new(|stored| { diff --git a/crates/huginn-daemon/src/serve.rs b/crates/huginn-daemon/src/serve.rs index 561ad1b..60755d5 100644 --- a/crates/huginn-daemon/src/serve.rs +++ b/crates/huginn-daemon/src/serve.rs @@ -1024,7 +1024,7 @@ mod tests { let message_id = format!("{stem}{}", "0".repeat(zeros)); let at_probe = encoded_len(&reply_with_payload(&message_id, operation, runtime_id, "A".repeat(probe))); let payload_len = (probe as i64 + (target as i64 - at_probe as i64)) as usize; - if payload_len % 4 == 0 { + if payload_len.is_multiple_of(4) { let message = reply_with_payload(&message_id, operation, runtime_id, "A".repeat(payload_len)); assert_eq!(encoded_len(&message), target); return (message_id, message); diff --git a/crates/huginn-mind/src/admission.rs b/crates/huginn-mind/src/admission.rs index ec815f6..cf51c3c 100644 --- a/crates/huginn-mind/src/admission.rs +++ b/crates/huginn-mind/src/admission.rs @@ -608,7 +608,7 @@ fn matrix(subject: PipelineKind, outcome: &ResolutionOutcome) -> bool { use PipelineKind as K; use ResolutionOutcome as O; let all = |by: &[PipelineRef], kind: K| by.iter().all(|supersessor| supersessor.kind == kind); - let fits = match (subject, outcome) { + match (subject, outcome) { (K::Target, O::Superseded { by }) => all(by, K::Target), (K::Question, O::Answered { by }) => by.kind == K::Ruling, (K::Question, O::Withdrawn { .. }) => true, @@ -626,8 +626,7 @@ fn matrix(subject: PipelineKind, outcome: &ResolutionOutcome) -> bool { (K::Run, O::Recorded { .. } | O::Withdrawn { .. }) => true, (K::Resolution, O::Withdrawn { .. }) => true, _ => false, - }; - fits + } } /// The resolution row: the sequence, the in-force subject, the matrix, the @@ -869,6 +868,42 @@ mod tests { assert_eq!(receipts[0].provenance.tool, s("admit tool")); } + /// The read bounds are the leaf's: admission applies the leaf's validation + /// and adds none. Sixteen anchors of a hundred lines admit and read back; + /// a seventeenth refuses at `cut_spec.reads` and nothing is written. + #[test] + fn reads_admit_at_the_bound_and_refuse_over_it() { + let anchors = |count: usize| { + (0..count) + .map(|n| eureka_pipeline::ReadAnchor { + location: eureka_pipeline::CodeLocation { + path: s("crates/eureka-pipeline/src/lib.rs"), + line: 1, + end_line: Some(100), + symbol: Some(format!("symbol_{n}").as_str().into()), + }, + why: s("read"), + }) + .collect::>() + }; + let mut mind = seeded(); + let mut spec = cut_spec("1", 1); + spec.reads = anchors(16); + committed(admit(&mut mind, vec![D::CutSpec(spec.clone())])); + let held = mind.view(&r(K::CutSpec, &id("cut_spec", "cut-1.r1"))).unwrap().expect("the spec is held"); + assert_eq!(held.document, D::CutSpec(spec)); + + let mut over = cut_spec("2", 1); + over.reads = anchors(17); + let receipts = mind.receipts().unwrap().len(); + assert_eq!( + refusal(admit(&mut mind, vec![D::CutSpec(over)])), + MindRefusal::Document(PipelineRefusal::FieldBound { field: "cut_spec.reads".into(), limit: 16, actual: 17 }) + ); + assert_eq!(mind.receipts().unwrap().len(), receipts); + assert_eq!(mind.view(&r(K::CutSpec, &id("cut_spec", "cut-2.r1"))).unwrap(), None, "nothing is written"); + } + /// The leaf's `Title` rule is private, so this holds the provenance rule to /// it: the two accept and refuse the same text. #[test] @@ -1640,7 +1675,7 @@ mod tests { let hand_off_ref = r(K::HandOff, &format!("{INSTANCE}:hand_off:{OTHER_INSTANCE}.gamecult_-huginn.2026-09-16")); // Accepted, one per resolvable kind. - committed(admit(&mut world(), vec![target(2, &[INVARIANT]), resolution(target_ref.clone(), superseded(&[target_r2.clone()]))])); + committed(admit(&mut world(), vec![target(2, &[INVARIANT]), resolution(target_ref.clone(), superseded(std::slice::from_ref(&target_r2)))])); committed(admit(&mut world(), vec![resolution(question_ref.clone(), withdrawn())])); committed(admit(&mut world(), vec![D::Ruling(ruling("R2")), resolution(ruling_ref.clone(), superseded(&[r(K::Ruling, &id("ruling", "R2"))]))])); committed(admit(&mut world(), vec![resolution(spec_ref.clone(), withdrawn())])); @@ -1683,7 +1718,7 @@ mod tests { incompatible(K::CutSpec, "Superseded") ); assert_eq!( - refusal(admit(&mut world(), vec![resolution(finding_ref.clone(), superseded(&[finding_ref.clone()]))])), + refusal(admit(&mut world(), vec![resolution(finding_ref.clone(), superseded(std::slice::from_ref(&finding_ref)))])), incompatible(K::Finding, "Superseded") ); assert_eq!( @@ -2302,7 +2337,7 @@ mod tests { committed(admit(&mut open(), vec![resolution(subject(), withdrawn())])); let other_run = r(K::Run, &run_key("z")); let refused = [ - (superseded(&[other_run.clone()]), "Superseded"), + (superseded(std::slice::from_ref(&other_run)), "Superseded"), (ResolutionOutcome::Answered { by: r(K::Ruling, &id("ruling", "R1")) }, "Answered"), (ResolutionOutcome::Fixed { commit: sha(), by: None }, "Fixed"), (ResolutionOutcome::Deferred { to: other_run }, "Deferred"), diff --git a/crates/huginn-mind/src/fixtures.rs b/crates/huginn-mind/src/fixtures.rs index 100f32c..86cb827 100644 --- a/crates/huginn-mind/src/fixtures.rs +++ b/crates/huginn-mind/src/fixtures.rs @@ -85,7 +85,7 @@ fn range() -> CommitRange { } fn location() -> CodeLocation { - CodeLocation { path: s("crates/huginn-mind/src/admission.rs"), line: 1, end_line: None } + CodeLocation { path: s("crates/huginn-mind/src/admission.rs"), line: 1, end_line: None, symbol: None } } pub(crate) fn evidence() -> Evidence { @@ -203,6 +203,7 @@ pub(crate) fn cut_spec(cut: &str, revision: u32) -> PipelineCutSpec { keeps_moves: vec![], adds: vec![], file_changes: vec![], + reads: vec![], authority_map: None::, verification: CutVerification { builds: vec![], tests: vec![], negative: vec![], operator: vec![] }, estimate: delta(), diff --git a/crates/huginn-mind/src/receipt.rs b/crates/huginn-mind/src/receipt.rs index 37b042a..35705d9 100644 --- a/crates/huginn-mind/src/receipt.rs +++ b/crates/huginn-mind/src/receipt.rs @@ -155,6 +155,7 @@ impl HuginnCommitReceipt { /// How a commit ended. An exact replay is decided by admission before the /// commit is attempted, so it is not an exit of the primitive. #[derive(Clone, Debug, PartialEq, Eq)] +#[allow(clippy::large_enum_variant, reason = "one outcome per commit, returned once and matched at once")] pub(crate) enum CommitOutcome { /// The batch and its receipt landed whole. Committed(HuginnCommitReceipt), diff --git a/crates/huginn-mind/src/wire.rs b/crates/huginn-mind/src/wire.rs index a8df61f..6a4cd09 100644 --- a/crates/huginn-mind/src/wire.rs +++ b/crates/huginn-mind/src/wire.rs @@ -105,7 +105,7 @@ impl HuginnMindRequest { pub enum HuginnMindResponse { Whoami(MindStatus), Admit(PipelineAdmissionOutcome), - View(Option), + View(Option>), Query(PipelineSelectionPage), Refused(MindRefusal), /// The answer did not fit one send and is being held for the client to diff --git a/crates/mind-body/src/testkit.rs b/crates/mind-body/src/testkit.rs index 8c3580b..6628ab6 100644 --- a/crates/mind-body/src/testkit.rs +++ b/crates/mind-body/src/testkit.rs @@ -244,6 +244,7 @@ pub(crate) fn spec(cut: &str) -> PipelineDocument { keeps_moves: vec![], adds: vec![], file_changes: vec![], + reads: vec![], authority_map: None, verification: CutVerification { builds: vec![], tests: vec![], negative: vec![], operator: vec![] }, estimate: delta(), diff --git a/crates/mind-body/tests/control.rs b/crates/mind-body/tests/control.rs index bfb66fd..005420b 100644 --- a/crates/mind-body/tests/control.rs +++ b/crates/mind-body/tests/control.rs @@ -491,13 +491,15 @@ mod unsafe_installs { error.downcast::().expect("the refusal is the typed UnsafeStore error") } + type Case = (&'static str, u32, Option, Option, &'static str); + #[test] fn a_group_or_world_writable_directory_lock_or_store_is_refused_and_nothing_is_written() { if !running_as_root() { eprintln!("SKIPPED: needs root to own the fixtures"); return; } - let cases: [(&str, u32, Option, Option, &str); 5] = [ + let cases: [Case; 5] = [ ("world-writable directory", 0o777, None, None, ""), ("group-writable directory", 0o775, None, None, ""), ("world-writable lock", 0o755, Some(0o666), None, "control.cc.lock"), diff --git a/docs/eureka-body-map.md b/docs/eureka-body-map.md index 7188de3..b943400 100644 --- a/docs/eureka-body-map.md +++ b/docs/eureka-body-map.md @@ -21,6 +21,21 @@ mind-control-store and run-kind had merged. New facts are F18-F20, which name th owner of the brake stop. The Brake, Units and run rows changed, and a rationale section was added. +Revised again on 2026-10-09 by Imagination (`imagination-launch-r3`, session +`self-2026-10-09-eureka`) at the mind-launch head `8dee875`, after Soul's verdict +`eureka-body:verdict:cut-mind-launch.s1` falsified that cut with nine findings. New fact +F21. The run, Repetition breaker, Brake and Units rows changed, and the rationale gained +"Who decides one live run, and who ends a dead one" and "What the breaker counts". +Specs: `cut-run-live-admission.r1`, `cut-mind-launch.r3`, `cut-mind-wake.r3`, +`cut-brake-stop.r2`. + +Revised again on 2026-10-10 by Imagination (`imagination-mind-units`, session +`self-2026-10-10-eureka`) at Huginn `0c6f6cb` and gamecult-ops `6b4a874`, for cut +`mind-units-install`. New fact F22. New section "The mind units: identity, lifecycle and +authority", written before the cut was mapped. Specs: `cut-mind-unit-gate.r1` (Huginn) and +`cut-mind-units-install.r1` (gamecult-ops). Questions: `mind-unit-names` and +`mind-unit-start-grant`. + ## Pinned heads (fetched 2026-10-08 ~12:10 UTC) | Repo | Ref | Commit | Note | @@ -179,6 +194,69 @@ Each line: what was run, where, result. was not acted on in time, and the process was killed at the host (prior-art file lines 323-325, secondary). Cut mind-wake r2 has the waker return `Held` before any query, admission or unit call, and lists "the waker ... ending a live run" among its forbidden writers. +- **F21 What the mind can attribute a document to, at the mind-launch head `8dee875`.** Read + from a throwaway worktree on 2026-10-09 (Imagination `imagination-launch-r3`). + - A receipt's provenance is `{ faculty, agent, session, tool }`, all declared by the writer. + No selection alias reads `session`, so attributing by session means scanning headers. + `open_and_launch` admits a run with session equal to its run label + (`crates/mind-body/src/launch.rs:205`). Nothing makes a Self run's later admissions carry + that label: the Self organ is not built. + - Citations are typed content. A `cites` selection with no role over every schema is + supported and tested (`crates/huginn-mind/src/query.rs:1499`). A run's header facts carry + `turn`, `operated_by` and `claims` (query.rs:139). + - The breaker at 8dee875 scans every header in the mind, newest first, back to the oldest + counted run (`crates/mind-body/src/queue.rs:210`). Its cost grows with the whole mind. + - `Mind::admit` takes `&mut self` (`crates/huginn-mind/src/admission.rs:81`), so + admissions are serialized. The run rule is `run_claims_rule` (admission.rs:537-547). It is + applied when a run opens (admission.rs:526-529) and when a withdrawal reinstates a closed + run (admission.rs:685-694). It checks claims only. + - `MindRefusal` is huginn-mind's (`crates/huginn-mind/src/refusal.rs`), published in + `schemas/cultnet/huginn.mind_response.v1.schema.json` and pinned by + `published_wire_schemas_match_derivation` (wire.rs:276). The leaf's `PipelineRefusal` has + five variants (`crates/eureka-pipeline/src/lib.rs:59-65`) and is not where admission's rules + refuse. + - `Effective::Released { cadence, run_cap_usd }` is a public struct variant + (`crates/mind-body/src/control.rs:117-120`), so any caller can build one without reading the + store. `ControlState::effective` is called only by `read_effective` (control.rs:192-197). + - huginn-mind's `fixtures::run` builds an in-force SelfRun with `operated_by: Mind` + (fixtures.rs:331-342). The claim tests open several of them at once + (admission.rs:1987-2015). + - `Slug::validate_slug` and `Label::validate_label` are public + (crates/eureka-pipeline/src/lib.rs:287, 386). `RunTurn` and `RunOperator` spell their + variants through `Debug`, which is also their `NAMES` (lib.rs:436-452). +- **F22 What the mind units would land on, 2026-10-10.** Read-only `ssh yggdrasil` as + `gamecultadmin` and an Eyes pass over Idunn `6667f0f` (Imagination `imagination-mind-units`). + - Still no unit, user or store of hers: `systemctl list-unit-files 'mind-*'` lists 0; + `getent passwd` has no `mind-*` user; `/etc/gamecult/minds` is absent; no `mind-control` + on PATH. `/opt/gamecult/huginn/current` is `releases/107552dc...` and holds only + `huginn-daemon` (0755 root). The Release step builds only `-p huginn-daemon` + (gamecult-ops `runbooks/huginn-yggdrasil.md:57`). + - systemd 257 (257.13-1~deb13u1): `ExecCondition=` (since 243) exists. Per + systemd.service(5), an ExecCondition exit of 1-254 skips the unit (inactive, not failed) + and 255 or a signal fails it; the `+` prefix runs that command with full privileges. + - **polkit is not installed:** `polkitd` "Installed: (none)", candidate 126-2; + `/etc/polkit-1/rules.d` absent (`/usr/share/polkit-1/actions` exists, from systemd). + Without polkitd, systemd refuses an unprivileged caller's `StartUnit`, so no user of hers + can start a unit today. `/etc/sudoers.d` holds only `90-gamecultadmin` and README. + - systemd cannot derive a per-instance user from a `: