diff --git a/docs/document-variants-cut.md b/docs/document-variants-cut.md index 51136a78..680c8440 100644 --- a/docs/document-variants-cut.md +++ b/docs/document-variants-cut.md @@ -314,6 +314,35 @@ not closeable, and a fix batch is in Hands: **Open for the operator:** plain records with duplicate unique-index values are silently overwritten today. That predates variants. Fixing it could refuse existing stores. +**Operator rulings, 2026-09-30:** +- **Duplicate unique-index values do not load** ("Duplicate indices should not load"). This covers + plain records too, so R6-for-indexes becomes one general rule: + - a commit that would leave two records holding one indexed value is refused; + - a store holding duplicates refuses to load, and the refusal names both keys and the index. + - The variant-specific rule `RefuseVariantIndexSharing` collapses into this rule. + - Before it ships, every real store is scanned, so no store stops loading by surprise. + - **Scan result (Eyes, 2026-09-30):** `[CultIndex]` had been recycled as a grouping and + selection-filter declaration: `Category`, `Kind`, `HardpointType`, `ShardId`, and others in + CultMesh, Geometry and docs. AetheriaEve's `aetheria-world.cc` (taxidermy) is the only store + holding duplicates. The only single-result lookups in real code are unique: + `PlayerData.PlayerId` and `Email` in `Server.cs`. The operator: "Pretty sure indexes are + supposed to be unique, it was not my decision to recycle that mechanism for grouping + purposes" and "Grouping was a desirable feature from my old DatabaseListView that didn't + quite make it into the initial CultCache Studio port". + - **Resolution (Self, from those answers):** `[CultIndex]` means unique. Grouping belongs to + the Studio record-grouping campaign's own attribute (`docs/studio-grouping-cut.md` on + `codex/studio-grouping`, attribute-based per the 2026-09-17 ruling). CultNet selection + filters any declared member by its catalog name, so no index is needed for filtering. The + grouping-only `[CultIndex]` declarations are deleted. This is one CultLib cut, queued + behind C2a because both edit `CultCache.cs`. + - The TS/Python last-writer-wins parity gap closes by the same rule when those runtimes + adopt it. +- **CultNet invisibility is a defect** ("looks like a defect to me"): `cache.Commit` and + `cache.UpsertAsync` on a database's cache must publish to subscribers and to the mutation + log (`CultCache.Publish`'s `if (!loaded) continue;`). +- **Hop subscriptions keep refusing variants** until CultNet carries deltas (C5): "keep refusing, + I guess, CultNet will have deltas by the time we're done here anyway". + ### C2. C# element identity and nested paths (Set, Insert, Remove) Adds: diff --git a/docs/document-variants-target.md b/docs/document-variants-target.md index 42b07bc4..0e62ece1 100644 --- a/docs/document-variants-target.md +++ b/docs/document-variants-target.md @@ -97,6 +97,11 @@ edited in CultCache Studio: observe the change atomically. - Compare-exchange (`Expect`) tests the stored delta. The resolved document is derived. - **3, cross-store bases: no.** A variant and its base live in the same store. + *Future direction (operator, 2026-09-30, from Aetheria moddable ships):* "we'll want cross-store variant + support eventually; modders should be able to add variants of existing items without modifying the + original". Ruling 3 holds for this campaign. A mod store layering variants over a shipped catalog is the + named future consumer that would reopen it. Until then, no consumer builds its own cross-store resolver: + Aetheria's compose step stays out of resolution. - **4, CultNet: deltas.** - **5, runtimes: C# first.** TypeScript, Python, Rust and Kotlin refuse a store or snapshot containing variants, loudly, until each implements resolution. diff --git a/docs/studio-grouping-cut.md b/docs/studio-grouping-cut.md new file mode 100644 index 00000000..770d0698 --- /dev/null +++ b/docs/studio-grouping-cut.md @@ -0,0 +1,432 @@ +# Studio Record Grouping And Reference Drag-Drop Cut + +First mapped 2026-09-17 (Imagination) against CultLib `122c217`. **Refreshed 2026-09-29 +(Imagination) against CultLib `main` `feeced8` and Aetheria `codex/fire-control-12` `4b594e11`.** +Every `file:line` below is against those revisions unless marked. Hands re-reads an anchor +before editing it; `main` keeps moving. + +**Rulings.** Q1 **B** (a class attribute on the model; argument order is nesting order; +2026-09-17). Q2 **A** (occupied values only). Q3 **A** (list only, not the ref picker). +Q4 **A** (no record search). Q5 **overtaken**: one release from `main`, together with +document-variants C3 (`docs/document-variants-cut.md`), taking whatever native-plugin drift +`main` carries. Q6 **A** (flat row by resolved value, C3's variant marker; 2026-09-30). Q7 **B** (legacy parity +minus manufacturer, plus `FactionProductData` by `Manufacturer` then `Design`, plus +`WeaponItemData` by `WeaponType`; 2026-09-30). + +## Target + +### Ends + +1. **Declared record grouping.** `[CultInspectorGroupBy(nameof(A), nameof(B))]` on a document + class makes Studio's record list for that type a foldout tree, nested in argument order. A + declaration on a base type groups every derived type listed under it. +2. **Create in a group.** Every group node offers `Create`. The new record gets every grouped + member on the path to that node set to that node's value, so it lands in that node. +3. **Reference drag and drop.** A record row dragged from Studio's list onto any + `CultRecordRef` value the inspector draws (member, list element, dictionary key or value) + is accepted exactly when the ref's picker would offer that record. + +Recovered from the legacy "Database Tools" window (Aetheria `d3db1730`, +`Assets/Scripts/CultCache/Editor/DatabaseView.cs`, `AetheriaDatabaseView.cs`, +`Inspectors/DatabaseLinkInspector.cs`). Selector lambdas are rejected (operator, 2026-09-17). + +**The attribute is not `[CultIndex]`.** `[CultIndex]` means unique (operator, 2026-09-30; +`CultDocumentContracts.cs:34`). Grouping is `CultInspectorGroupByAttribute`, in +`CultInspectorAttributes.cs`, read only by `CultInspectorModel`. It changes no schema id, +catalog entry, index, or store byte. The index cut's deletion of grouping-only `[CultIndex]` +declarations (CultMesh, Geometry) adds no `[CultInspectorGroupBy]` in their place: those are +service stores, not Studio-authored catalogs. + +### Invariants + +- **I1. The model owns grouping.** `CultInspectorModel` alone decides which members group a + listed type and in what order, which declarations are invalid (and the notice), the tree's + partition, node identity, order and labels, and the object create-in-group makes. Studio + lowers the tree; it never reads the attribute, calls `SetValue` for a preset, or computes a + group label. +- **I2. The leaves partition the candidates.** Across all leaves the tree holds exactly + `RecordCandidates(CultRecordRef, records)`, each record once, in candidate order + inside a leaf. Grouping never hides a record, including one whose grouped value is unset, + dangling, or an undefined enum value. +- **I3. The listed type owns the tree's shape.** Grouping comes from the type selected in the + type pane, never from each record's runtime type. +- **I4. Node identity is value identity, not the label.** A ref groups by key, an enum by its + underlying value, other kinds by the value's invariant string. Foldout state keys on node id. +- **I5. Create-in-group round-trips.** Upsert what `CreateInGroup(listed, node)` returns, + regroup: the new record lies in a node with the same id. +- **I6. One candidate rule.** `IsRecordCandidate(refType, record)` is the only test for "this + record may be this ref's value". `RecordCandidates` filters through it; drop acceptance calls it. +- **I7. One ref label.** The model owns `None` and `Missing `; the picker and group labels + both use it. +- **I8. Invalid declarations never break the window.** They give a flat list plus the model's + notice, never a throw in `OnGUI`. +- **I9. A drop honours the picker's gates.** Refused in a disabled (read-only) scope, when the + payload came from another model, and when the key equals the current key. A dictionary-key + drop still passes `ReplaceKey` (`CultCacheStudioDrawers.cs:303-309`). +- **I10 (new). Grouping reads what readers read.** A variant record groups, labels, and is a + drop candidate by its resolved `Document` (`CultCache.cs:369-371`), never its delta. + +### Non-goals + +No sibling-runtime port (TS `cult-cache-inspector.ts` is a byte dumper; Rust and Python have +no inspector). No runtime CultUI panel. No search (Q4), no picker grouping (Q3), no +drop-to-append on list headers, no legacy `New ` naming. + +## Body findings + +Kept from 2026-09-17, re-anchored: + +- **F3.** Legacy grouping matched the exact type; grouping derived records under a base + declaration is new. +- **F4.** Legacy create-in-group was dead (`DatabaseView.cs:70-75` throws); legacy foldouts + keyed on the label hash. +- **F5.** `member.GetCustomAttributes(true)` ignores `inherit` for a `PropertyInfo`, so a base + `[M] abstract P` is not seen on the override. Sites: `CultInspectorModel.cs:36` (now the only + metadata site: `MetadataOf` caches it per member, `:341-351`) and `:209` + (`CultInspectorDrawerClaims.Resolve`). Fix: `Attribute.GetCustomAttributes(member, true)`. +- **F6.** Studio `Add` creates only the listed type (`CultCacheStudioWindow.cs:340-346`, + `CreateElement(type, type)`), disabled when the model cannot create it (`:141`). So + create-in-group never makes a derived type, and an abstract listed type offers no `Create`. +- **F7.** Studio search filters the type pane only (`:112-120`). +- **F8.** Foldouts are in-memory (`CultCacheStudioDrawers.cs:33`); `OnDisable` closes the store + (`:48-51`), domain reload included. The only `EditorPrefs` key is `LastPathKey` (`:13`). +- **F9.** No drawer claims `CultRecordRef<>`, so every ref reaches `DrawRecordRef` + (`CultCacheStudioDrawers.cs:254-280`). +- **F10.** The Studio editor sources compile outside Unity (netstandard2.1, C# 9, against + Unity 6000.3.24f1 `Editor\Data\Managed\UnityEngine\*.dll` without the `UnityEditor.dll` + facade, plus the built `GameCult.Caching*` and tracked `MessagePack*.dll`). The 09-17 + scratch project is gone; Hands rebuilds it in its own scratch from this recipe. +- **F11.** The record row reads its click in `GUILayout.Toggle` (`:184`); a drag start must be + handled on a reserved rect before the toggle draws (IMGUI convention, not probed). + +New since the map: + +- **F12. Variants (C1, `6ffe17a`).** `CultInspectorModel(registry, CultCodec)` + (`CultInspectorModel.cs:302-306`); Studio still calls `CultCacheMessagePack.CreateInspectorModel(registry)` + (`CultCacheMessagePack.cs:33-34`, unchanged). `CultStoredDocument.Variant` is the delta + (`CultCache.cs:374`); `Document` is the resolved view. A top-level override is a + `Set` whose path is one step at the member's slot (`CultVariants.cs:72-73`), so "is this + grouped member overridden on this variant" is computable from the member's `Slot`. +- **F13. Every Studio commit to a variant key is refused until C3.** `CultInspectorEdit.Commit` + is a plain `UpsertAsync` (`CultInspectorModel.cs:266-282`); Q1a refuses a plain write at a + variant key (`CultCache.cs:2494-2496`). A drop into a variant's inspector therefore reports + that refusal until C3 lands. The release waits for C3, so no shipped Studio shows it, but + the operator click-through must run after C3. +- **F14. Studio `Duplicate` of a variant makes a flattened plain copy** + (`CultCacheStudioWindow.cs:349-353` clones the resolved `Document`). Not this campaign's + defect; C3's `CreateVariant` sits beside it. +- **F15. Element ids (C2a, `hands/variants-c2a`, WIP `df74f8c`).** No grouping interaction: + grouped members are top-level scalars and refs; ids live on list elements. Drops write a ref + into the edit document; ids are untouched and minted on write for new elements; C3's diff + keys overrides on them. Two consequences: + - After C2a, registration refuses a document type whose object-list element lacks a + `[CultElementId]` (`CultCache.cs` descriptor build, WIP). Cut 1's new fixtures carry no + object lists. + - C2a does not hide or lock the id member, so Studio shows it as an editable string. + Hand-editing one retargets overrides. That belongs to C3 (make `[CultElementId]` members + read-only in the model), not here; flagged for Self. +- **F16. The 1.0.60/1.4.0 release shipped** (`0155ba8`, tags on `45c2f40`) and + `codex/asset-guid-attribute` is merged. It already carried the 302,080-byte native bridge, + so the 09-17 Q5 drift is spent. Native source has moved since (`git diff --stat 45c2f40 main + -- native/`: 5 files, +840/-118), so this release rebuilds the bridge again. The v1 exports + the managed side imports (`src/GameCult.Mesh.Quic.Native/*.cs:263-276`) remain at + `cultmesh_quic_native.cpp:1405-1465`. +- **F17. Aetheria's schema moved.** `ItemData.Manufacturer` is gone (slot 3 retired, + `ItemData.cs:281-282`). The manufacturer relation now lives on `FactionProductData` + (`FactionProduct.cs:27`, `CultRecordRef Manufacturer`; `:24`, + `CultRecordRef Design`). Aetheria pins `cultlib-unity-v1.0.60` / + `caching-unity-v1.4.0` / `cultmath-unity-v0.2.4` (`Packages/manifest.json:54-56`), + `CultLibRevision` `45c2f40` and a separate `CultMathRevision` `6d5e209` + (`Directory.Build.props:6,10`). +- **F18. After C2a, Aetheria cannot take the new CultLib without its element-id sweep** + (`List`, `List`, `List`, `List`, + `List`, `List`, among others). So the Aetheria re-pin belongs to + variants C4, and this campaign's Cut 4 shrinks to annotations and tests on top of it. + +## Identity and authority + +| Thing | Owner | Lifecycle | Persisted | +|---|---|---|---| +| `[CultInspectorGroupBy]` | consumer source | compile time; nearest declaration wins; no arguments opts out | source only | +| `CultInspectorGrouping` (resolved) | model | cached per listed type like `_shapes` (`CultInspectorModel.cs:296`) | no | +| `CultInspectorRecordGroup` tree, node ids, labels | model | rebuilt each `OnGUI`, as `RecordCandidates` is (`CultCacheStudioWindow.cs:167`) | no | +| Group foldouts (node id → bool) | window | window memory; cleared by `CloseStore` (`:318-327`) | no | +| Drag payload `{ Model, Key }` under `"GameCult.CultCacheStudio.Record"` | Studio list (source) | one drag | no | +| Drop acceptance | `IsRecordCandidate` plus I9 gates | per `DragUpdated`/`DragPerform` | no | +| Created-in-group object | model makes; `CultCache` admits via `UpsertAsync`, as `Add` | new plain record | store, on `Save` | + +## Forks + +Q1-Q5 are ruled (header). Two remain. + +### Q6. RULED A (2026-09-30). How does the grouped list show a variant? (blocks nothing in Cuts 1-2; shapes C3's row lowering) + +Mechanical defaults, not asked: a variant groups by its resolved values (I10), so a variant +that inherits its base's `HullType` sits in the base's node and moves when the base changes; +create-in-group always makes a plain record; C3's `CreateVariant` needs no preset because the +variant inherits its base's group; a variant is a valid drag payload. + +- **A. Flat by resolved value.** A variant is an ordinary row in its resolved group. C3 adds a + row marker ("variant of Longinus"). Which of its grouped values are overridden shows in the + inspector through C3's per-member marker, not in the tree. +- **B. Nested under the base.** Inside a leaf, a variant row sits indented under its base's row + when both share the leaf; otherwise it stands alone with the marker. +- **C. A's rows plus an inherited/overridden glyph on the row** for the grouped members. +- **Recommendation: A.** I2 stays one rule, the tree has one shape, and the base link already + lives in C3's inspector. B makes the tree's shape depend on two relations, and splits + whenever a variant overrides a grouped value. C duplicates C3's inspector marker. + +### Q7. RULED B (2026-09-30). Which Aetheria types group, and by what? + +Candidates on `codex/fire-control-12` (enums unless noted; `[Flags]` enums excluded): + +| Type | Member | Anchor | Note | +|---|---|---|---| +| `SimpleCommodityData` | `Category` (`SimpleCommodityCategory`) | `ItemData.cs:310` | legacy parity | +| `CompoundCommodityData` | `Category` (`CompoundCommodityCategory`) | `ItemData.cs:338` | legacy parity | +| `GearData` | `Hardpoint` (`HardpointType`) | `ItemData.cs:454` | legacy parity; `WeaponItemData` inherits when listed under `GearData` | +| `WeaponItemData` | `WeaponType`; or `WeaponCaliber`, `WeaponRange` | `ItemData.cs:485`, `:482`, `:479` | re-declaration, applies when listing weapons | +| `HullData` | `HullType` | `ItemData.cs:519` | legacy parity | +| `FactionProductData` | `Manufacturer` (ref `Faction`), then `Design` (ref `CraftedItemData`) | `FactionProduct.cs:27`, `:24` | where "by manufacturer" now lives | +| `Loadout` | `Hull` (ref `HullData`) | `Loadout.cs:17` | optional | + +`ConsumableItemData`, `CargoBayData`, `DockingBayData`, `Faction`, `PersonalityAttribute` +have no groupable member worth a level. + +- **A. Legacy parity minus manufacturer:** commodities by `Category`, `GearData` by + `Hardpoint`, `HullData` by `HullType`. +- **B. A, plus `FactionProductData` by `Manufacturer` then `Design`, plus `WeaponItemData` by + `WeaponType`.** +- **C. B with `FactionProductData` by `Design` then `Manufacturer`** (the market-segment view: + who makes this design). +- **D. B plus `Loadout` by `Hull`.** +- **Recommendation: B.** It recovers everything the legacy window grouped, puts the + manufacturer view where the manufacturer now is, and gives the weapon list its own axis. + C is the better view if the operator authors products per design rather than per faction. + Product record counts are unprobed; Cut 4's Hands reports them. + +## Defaults (recorded, not asked) + +- **Labels:** enum `Enum.GetName`, else the number; ref `RecordLabel` of the candidate with that + key (for `Faction`, `Name`, not legacy `ShortName`); unset ref `None`; dangling `Missing `; + string its value or `(empty)`; integer invariant string; bool `False`/`True`. +- **Order within a level:** enums by underlying value; refs `None` first, then label + (`OrdinalIgnoreCase`, then key), `Missing` last; strings `OrdinalIgnoreCase` then ordinal; + integers numeric; `false` before `true`. +- **Groupable kinds:** String, Integer, Bool, Enum (not `[Flags]`), RecordRef. Refused with a + notice and a flat list: any other kind (Float included), a `[Flags]` enum, `Hidden`, + `IsReadOnly`, a name that is not an inspector member of the listed type, a name given twice. +- **Global types are never grouped.** +- **Foldouts:** window memory, collapsed by default, cleared by `CloseStore`. Create-in-group, + `Add` and `Duplicate` expand the path to the new selection. No `EditorPrefs`/`SessionState`. +- **Create into a `Missing ` node** presets the dangling key (keeps I5). +- **A drop of the current key is rejected** (no no-op commit). +- **Variants:** see Q6's defaults and I10. + +## Sequence with document variants + +| Cut | Files | Overlaps grouping? | +|---|---|---| +| Variants C2a (element ids, in flight) | `CultCache.cs`, new `CultElementIds.cs`, Mesh/Geometry documents, `ElementIdTests.cs` | no | +| Index cut (`[CultIndex]` means unique; queued behind C2a) | `CultCache.cs`, Mesh/Geometry | no | +| Variants C2b (nested paths; inferred from the brief, not yet written in the variants map) | `CultCache.cs`, `CultVariants.cs` | no | +| **Grouping Cuts 1-2** | `CultInspectorModel.cs`, `CultInspectorAttributes.cs`, `CultInspectorModelTests.cs`, `CultCacheStudioWindow.cs`, `CultCacheStudioDrawers.cs`, Studio README | — | +| Variants C3 (Studio edits the delta) | `CultInspectorModel.cs` (edit, commit, diff, `CreateVariant`, `Rebase`), window (variant controls beside `Duplicate`, row marker), drawers (per-member marker and revert), `CultInspectorModelTests.cs` | **yes: all five grouping files** | + +**Order:** grouping Cuts 1-2 now, in parallel with C2a, the index cut and C2b (no shared +files). Grouping merges to `main` before C3's Hands starts; C3 needs C2b anyway. C3 then +rebases onto grouping and lowers its row marker inside the grouped row drawing that Cut 2 +writes. Then **one release (Cut 3)**, then one Aetheria pass (variants C4 plus Cut 4). + +Where they touch: `DrawRecords` rows (`CultCacheStudioWindow.cs:180-185`; grouping rewrites, +C3 marks); the toolbar (`:142-157`; C3 adds variant buttons beside `Duplicate`, grouping +leaves it alone); `DrawRecordRef` (grouping adds the drop target; C3 marks at the member +level in `DrawValue`, not inside it); `CultInspectorModel.cs` (disjoint methods); +`CultInspectorModelTests.cs` (disjoint fixtures; name grouping fixtures `Group*`). + +**One release commit:** Cut 3 below is the single release for both campaigns. The variants +map's C3 line "Then tag `caching-unity-v1.5.0`" should point here instead of owning a second +release step (proposal for Self). + +## Cuts + +Hands for Cuts 1-2: branch `hands/studio-grouping` in a fresh worktree off `origin/main` +(Self picks the path, e.g. `C:/wsNN-grouping`). `F:\Projects\CultLib-studio-grouping` holds the +superseded docs-only `codex/studio-grouping`; do not reuse it. Do not touch the main checkout. +Mutation testing is Stryker.NET on the cut's diff; there are no hand-applied mutations. + +### Cut 1. CultLib model: declaration, tree, candidate and label authority (~150k tokens) + +- **Delete and collapse first:** + - `CultInspectorModel.cs:519`: the inline `target.IsInstanceOfType(record.Document) && + record.Key.Value.Length > 0` in `RecordCandidates` (`:514-522`) moves into + `IsRecordCandidate`; `RecordCandidates` filters through it (I6). + - `:36` and `:209`: `member.GetCustomAttributes(true)` → `Attribute.GetCustomAttributes(member, true)` (F5). +- **Add in `CultInspectorAttributes.cs`**, before `CultInspectorDrawerAttribute` (`:81-86`): + `CultInspectorGroupByAttribute(params string[] members)`, `AttributeUsage(Class, + Inherited = true, AllowMultiple = false)`, property `Members`, with a comment in the file's + style stating nesting, nearest-wins and opt-out. +- **Add in `CultInspectorModel.cs`** (or a sibling `CultInspectorGrouping.cs`, same namespace; + one file while it stays under ~800 lines; it is 645 now): + - `CultInspectorGrouping { Type ListedType; IReadOnlyList Members; string? Notice }` + (members empty when `Notice` is set). + - `CultInspectorRecordGroup { string Id; string Label; int Depth; IReadOnlyList Values; + IReadOnlyList Children; IReadOnlyList Records; int Count }`. + - `GroupingOf(Type listed)`, cached like `ShapeOf` (`:324-333`): names resolve against + `MembersOf(listed)` (`:335-339`); validation per Defaults. + - `GroupRecords(Type listed, IEnumerable records)`: partitions + `RecordCandidates(CultRecordRef)` by value identity, reading each record's + `Document` (resolved, I10). + - `CreateInGroup(Type listed, CultInspectorRecordGroup group, out string? notice)`: + `CreateElement(listed, listed, out notice)` (`:387-398`), then `SetValue` each level; refs + via `CreateRecordRef` (`:524-529`). `ArgumentException` for a node not of `listed`'s current + grouping. + - `IsRecordCandidate(Type refType, CultStoredDocument record)`. + - `RecordRefLabel(Type refType, object? value, IEnumerable records)`: `None`, + `RecordLabel` (`:315-319`), or `Missing `. +- **Tests** in `tests/GameCult.Caching.Tests/CultInspectorModelTests.cs` (476 lines; fixtures + `:299-476`). New fixtures `GroupBase` (abstract) / `GroupLeaf : GroupBase` `[CultDocument]`, + a re-declaring type and an opting-out type; members: enum declared out of alphabetical + order, `CultRecordRef`, string, float, `[Flags]` enum, hidden int, read-only int. + No object lists (F15). Behaviours, one test each: + 1. levels nest in declared order, not slot order; + 2. a `GroupLeaf` listed under `GroupBase` groups by the base declaration (I3); + 3. nearest declaration wins; an empty declaration opts out; + 4. leaves partition the candidates over unset refs, dangling refs and undefined enum values (I2); + 5. two refs sharing one `[CultName]` give two nodes; `None` first; `Missing k` last (I4, I7); + 6. enum nodes order by value; + 7. create at depth 2, upsert into a temp single-file `CultCache`, regroup: same node id (I5); + 8. each invalid declaration (unknown, float, `[Flags]`, hidden, read-only, duplicate) gives a + notice and a flat tree, no throw (I8); + 9. `IsRecordCandidate` accepts a subtype, refuses a sibling and an empty key, and + `RecordCandidates` equals the records it accepts (I6); + 10. a base abstract property's `[CultInspectorLabel]` reaches the override's metadata (F5); + 11. **a variant groups by its resolved value**: base with `Enum=A`, variant overriding it to + `B` via `UpsertVariantAsync` and `Override`, regroup: the variant is in `B`; change the + base to `C` with the variant not overriding it, regroup: it follows to `C` (I10). +- **Verify (Yggdrasil):** + `ygg-verify.sh /f/Projects/CultLib dotnet 'dotnet test tests/GameCult.Caching.Tests -c Release'`, + then + `ygg-verify.sh /f/Projects/CultLib dotnet 'dotnet tool install -g dotnet-stryker && export PATH="$PATH:$HOME/.dotnet/tools" && cd tests/GameCult.Caching.Tests && dotnet stryker --project GameCult.Caching.csproj --since: > /tmp/s.log 2>&1; rc=$?; tail -60 /tmp/s.log; exit $rc'`. + Survivors triaged by name in the report. +- **Ledger:** model ~+170, attributes ~+15, tests ~+230; one predicate collapsed. + +### Cut 2. CultLib Studio: lower the tree, create in group, drag source, drop target (~150k tokens) + +- **Delete first:** + - `CultCacheStudioDrawers.cs:266` inline `"Missing " + key : "None"` → + `Model.RecordRefLabel(type, value, Records)` for `names[0]` when `index == 0`. + - `CultCacheStudioWindow.cs:180-185` flat `foreach` → tree drawing. +- **Window** (`CultCacheStudioWindow.cs`): + - fields (`:13-30`): `Dictionary _groupFoldouts` (ordinal), `const string RecordDragKey`; + `CloseStore` (`:318-327`) clears them. + - `DrawRecords` (`:135-189`): `GroupingOf(_selectedType)`; a `Notice` shows a warning + `HelpBox`. Draw `GroupRecords(_selectedType, _records)` recursively: foldout `Label (Count)` + indented by `Depth`; an expanded node draws children, leaf rows, then a `Create` mini + button under `Add`'s gate (`:141-146`). The root is not a foldout; `Add` stays root create. + - row: reserve the rect with `GUILayoutUtility.GetRect`, handle `MouseDrag` there first + (`PrepareStartDrag`, `SetGenericData(RecordDragKey, new CultCacheStudioRecordDrag(_model, key))`, + empty `objectReferences`, `StartDrag(label)`, `Use()`), then `GUI.Toggle(rect, ...)` (F11). + Keep the existing `` suffix logic (`:182-183`) in one place for C3 to extend. + - `AddInGroup(node)` mirrors `Add` (`:340-346`) through `Run` (`:369`); selects the key and + expands the node's id prefixes. `Add`/`Duplicate` reveal the new record's path (a + `_revealKey` resolved next frame is acceptable). +- **Drawers** (`CultCacheStudioDrawers.cs`): `internal sealed class CultCacheStudioRecordDrag + { Model; Key }`. In `DrawRecordRef` (`:254-280`), capture the `HorizontalScope` rect; on + `DragUpdated`/`DragPerform` inside it with our payload: `Copy` when `GUI.enabled`, same + `Model`, key differs, record in `Records`, and `IsRecordCandidate`; else `Rejected`. Leave + foreign drags alone. On perform: `AcceptDrag`, `next = key`, `GUI.changed = true`, `Use()`; + return through the existing `:279`. +- **README** (`src/GameCult.Unity/Assets/Caching/README.md`): grouping, create in group and + drag-to-reference after the paragraph at `:30-34`; the grouping API, `IsRecordCandidate`, + `RecordRefLabel` under `## Inspection Model` (`:39`). +- **Verify:** + - Cut 1's Yggdrasil test command still passes. + - Negative greps (each empty): + `rg -n "IsInstanceOfType" src/GameCult.Unity/Assets/Caching/Editor | rg -v "assetType.IsInstanceOfType"` (I6; `:215` is the asset-GUID sub-asset check, not candidacy); + `rg -n "\"Missing \"" src/GameCult.Unity/Assets/Caching/Editor` (I7; the union drawer's + `"None"` at `:393` is not a ref label, so grep `Missing` only); + `rg -n "SetValue|GetCustomAttribute|CultInspectorGroupBy" src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioWindow.cs` (I1); + `rg -n "EditorPrefs" src/GameCult.Unity/Assets/Caching/Editor` shows only `LastPathKey`. + - **Starfire, one job, editor closed:** the F10 compile gate, `dotnet build` of a scratch + netstandard2.1 project over the worktree's `Assets/Caching/Editor/*.cs`. 0 errors. +- **Ledger:** window ~+80, drawers ~+30, README ~+20; two inline decisions deleted. + +### Cut 3. Joint release: cultlib Unity 1.0.61, Studio 1.5.0 (after grouping and variants C3 merge) + +- **Base:** `main` with grouping Cuts 1-2 and variants C1-C3 merged; a clean worktree. +- **Files:** `unity/org.gamecult.cultlib/package.json:4` `1.0.60` → `1.0.61`; + `src/GameCult.Unity/Assets/Caching/package.json:4` `1.4.0` → `1.5.0` and `:13` dependency + `1.0.61`; `CHANGELOG.md` entries in both packages (grouping, create in group, drag to + reference; variant editing from C3); plugins via + `powershell -File scripts\build-unity-package.ps1 -UpdateTemplate` (Windows: MSVC and the + pinned MsQuic digests; its version check is at `:168-170`). +- **Tags:** `cultlib-unity-v1.0.61`, `caching-unity-v1.5.0`; no `cultmath-unity` tag. +- **Verify (Starfire, one job at a time):** + - byte check: rerun the script; `git status --porcelain unity/` is empty; + - expected diff: `GameCult.Caching.dll/.pdb` changes, and `gamecult_mesh_quic_native.dll` + changes (F16); `msquic.dll` should not (same pinned digest). Anything else stops the release; + - API present: reflect over the tracked `GameCult.Caching.dll` for + `CultInspectorGroupByAttribute`, `GroupRecords`, `CreateInGroup`, `IsRecordCandidate`, + `RecordRefLabel`, and C3's `CreateVariant`; + - native ABI: `dumpbin /exports` lists `cultmesh_quic_open`, `_state`, `_poll`, `_error`, `_close`; + - `git ls-remote --tags origin` lists both tags. + +### Cut 4. Aetheria: grouping annotations (inside variants C4's pass; blocked on Q7) + +- **Where:** the Aetheria branch and worktree Self opens for variants C4. Never + `F:\Projects\Aetheria`'s working tree. C4 owns the pin bump (`Directory.Build.props:6`, and + `Packages/manifest.json:54-55` to `#caching-unity-v1.5.0` / `#cultlib-unity-v1.0.61`; + `:56` cultmath unchanged; `packages-lock.json` follows) and the element-id sweep (F18). This + cut adds only: + - Q7's `[CultInspectorGroupBy(...)]` class attributes (recommendation B: five declarations); + - `tests/Aetheria.Shared.Tests/StudioGroupingTests.cs` (xUnit): `GroupingOf` over + `CultCacheMessagePack.CreateInspectorModel(registry)` names exactly the declared members + per annotated type with no notice; `GroupingOf(typeof(WeaponItemData))` follows Q7; every + registered document type's `GroupingOf` has a null notice. +- **Verify:** + - headless (Starfire, one job; the revision check needs `CultLibRoot` clean at the release + SHA and `CultMathRoot` clean at `6d5e209`): + `dotnet test tests\Aetheria.Shared.Tests -p:CultLibRoot= -p:CultMathRoot=`; + - Unity 6000.3.24f1 batchmode compile, editor closed: + `-batchmode -nographics -quit -projectPath -logFile `; 0 errors, + both new tags resolved. +- **Operator only, in Aetheria's Studio against a copy of `GameData/Aetheria.cc`:** + - each annotated type shows the declared levels; `GearData` includes `` rows; + - create in a node: the record lands there and is selected; + - drag a `Faction` onto a `FactionProductData.Manufacturer`: accepted; drag a gear record onto + it: rejected; + - drag a `PersonalityAttribute` onto a `DemandProfile` key that already exists: refused + through the notice; + - read-only open: every drop rejected; + - a variant (once C4 authors one, e.g. `LonginusX`) sits in its resolved `HullType` node, + and a drop into its inspector writes an override (C3), not a refusal; + - close without edits: store bytes unchanged. + +## Subtraction ledger + +Added: one attribute, two model types, five model methods, ~11 CultLib and ~3 Aetheria tests, +~110 Studio lines, five annotations. Collapsed: the candidate predicate (now one method, reused +by drop), the ref label strings, the property-inheritance miss (F5). Not added: selectors, +registries, `EditorPrefs` keys, runtime ports, packages, targets, daemons, a second release. +Build footprint: `GameCult.Caching` and its tests (Yggdrasil); the F10 compile gate, the +package script (all tracked plugins including the native bridge) and Aetheria's headless and +batchmode checks (Starfire; Windows is the target for the editor plugin and win-x64 bridge). + +## What the 2026-09-17 map had wrong as of `feeced8` + +- Cut 4 grouped `GearData` by `Manufacturer`; `ItemData.Manufacturer` no longer exists (F17). +- Q5's native-plugin drift was spent by the 1.0.60 release (F16); the Cut 3 "rebase onto + `codex/asset-guid-attribute`" note is obsolete (merged). +- Cut 4 re-pinned Aetheria on its own; after C2a it cannot, without the id sweep (F18). +- Aetheria branch `codex/cultcache-cutover`, props `:5`, manifest `:53-55`, cultmath `v0.2.3`: now + `codex/fire-control-12`, `:6`, `:54-56`, `v0.2.4` with a separate CultMath pin. +- Negative greps for `IsInstanceOfType` and `"None"` would fail on unrelated lines (`:215`, `:393`). +- Hand-applied mutations per test are retired; Stryker.NET on the diff replaces them. +- Moved anchors: `RecordCandidates` `:512-520`→`:514-522`; `_shapes` `:298`→`:296`; `RecordLabel` + `:313-317`→`:315-319`; `CreateElement` `:385-396`→`:387-398`; drawer attribute `:80`→`:81-86`; + `DrawRecordRef` `:195-221`→`:254-280`, label `:207`→`:266`; `ReplaceKey` routing + `:243-251`→`:303-309`; test fixtures `:380-459`→`:299-476`; README `:36-39`→`:30-39`; build-script + version check `:150-155`→`:168-170`. +- Worktree `F:\Projects\CultLib-studio-grouping` is now the superseded docs branch; Hands uses a + fresh one. diff --git a/packages/cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/THIRD-PARTY-NOTICES.md index 06ebb6c3..1de744a7 100644 --- a/packages/cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/THIRD-PARTY-NOTICES.md @@ -6,6 +6,16 @@ simplex-noise functions by Ian McEwan and Ashima Arts, as distributed by Stefan Gustavson's `webgl-noise` project and Unity Mathematics. +`math.snoise(float3)` and `cultmath_snoise(float3)` follow `src/noise3D.glsl` and +`math.snoise(float2)` follows `src/noise2D.glsl` of stegu/webgl-noise at commit +`22434e04d7753f7e949e8d724ab3da2864c17a0f` (the simplex-boundary fix is +`21d9fe23d7`, scale `ff3b5d34ea`, permutation `a3e6d57095`). + +`math.snoise_grad`/`cultmath_snoise_grad` (and the `fbm_grad`/`ridged_grad` +octave sums built on them) follow the analytic-gradient differentiation from +the same project's `src/noise3Dgrad.glsl`, with the same kernel radius, +permutation and scale as `snoise(float3)`. + - Copyright 2011 Ashima Arts. - Upstream: - License: MIT. diff --git a/packages/cultmath/docs/design.md b/packages/cultmath/docs/design.md index 30cef06c..e14927fe 100644 --- a/packages/cultmath/docs/design.md +++ b/packages/cultmath/docs/design.md @@ -233,6 +233,38 @@ together, laid out as `float4(gradient.xyz, value.w)`, with no value-only twin paths are proven bit-identical (`PrunedSearchIsBitIdenticalToTheUnprunedFiveCubedSearch`, which samples `[-50, 50]^3`, is comfortably inside that domain). +- `snoise(float3)` follows webgl-noise `src/noise3D.glsl` as of + stegu/webgl-noise `22434e04d7`: kernel radius squared `0.5`, output scale + `105`, permutation `mod289((34x+10)x)`. Upstream's `21d9fe23d7` (2020-10-14, + "an age-old bug that caused slight discontinuities along simplex + boundaries") moved the radius from `0.6` to `0.5`; at `0.6` a lattice vertex + outside the four summed corners can lie inside the kernel, leaving a value + jump at the cell boundary (3.1e-3 over the committed probe's 400 lines). `ff3b5d34ea` set the scale + to `105` and `a3e6d57095` (2021-06-30) the permutation; the 2D `snoise` shares + that permutation helper, so its values changed too, while its own kernel + (`0.5`) and scale (`130`) already matched upstream. + `NoiseGradTests.SnoiseAndItsGradientAreContinuousAcrossSimplexCellBoundaries` + pins the continuity. +- `snoise_grad(float3 p)` carries `snoise(float3)` to value-and-gradient form, + following the differentiation in webgl-noise `src/noise3Dgrad.glsl` (Ashima + Arts / Ian McEwan, MIT), which uses the same radius, permutation and scale as + `snoise(float3)`: per corner, `m0 = max(0.5 - dot(x,x), 0)`, so + `d(m0^4 * dot(p,x))/dx = -8*m0^3*dot(p,x)*x + m0^4*p`, summed over the four + corners and scaled by the same constant as the value, so `.w` is exactly + `snoise(float3)`. `fbm_grad` and + `ridged_grad` are octave sums of `snoise_grad`: each octave samples at + `p * frequency`, so by the chain rule its gradient scales by that same + frequency, and both amplitude (`gain`) and frequency (`lacunarity`) compound + per octave rather than applying once to the whole sum. `ridged_grad` folds + each octave about zero (`Σ aᵢ(1 − |nᵢ|)`, gradient `−Σ aᵢfᵢ sign(nᵢ)∇nᵢ`, + Musgrave's ridged multifractal, *Texturing and Modeling* ch. 16); the crease + at `nᵢ = 0` is deliberate (the operator's dune term, 2026-09-25), not a bug. + Both clamp `octaves` to `[0, 16]`: `HlslSourceCompatibilityTests`'s bit-parity + oracle drives every mirrored `int` parameter across the full int32 range, + which is the right domain for `pcg3d`/`pcg4d`'s O(1) hash inputs but would + turn an unclamped octave count into a multi-billion-iteration loop for these + two mirrors. + Integer hashing uses the PCG hashes from Jarzynski and Olano, "Hash Functions for GPU Rendering" (JCGT 9(3), 2020): `pcg(uint)` is O'Neill's RXS-M-XS 32/32 permutation over one LCG step, and `pcg3d`/`pcg4d` are the paper's (3 → 3) and diff --git a/packages/cultmath/shaders/CultMath.hlsl b/packages/cultmath/shaders/CultMath.hlsl index 5c00bda0..50c5a746 100644 --- a/packages/cultmath/shaders/CultMath.hlsl +++ b/packages/cultmath/shaders/CultMath.hlsl @@ -76,8 +76,8 @@ float cultmath_smootherstep(float value) float2 cultmath_snoise_mod289(float2 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } float3 cultmath_snoise_mod289(float3 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } float4 cultmath_snoise_mod289(float4 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } -float3 cultmath_snoise_permute(float3 value) { return cultmath_snoise_mod289(((value * 34.0) + 1.0) * value); } -float4 cultmath_snoise_permute(float4 value) { return cultmath_snoise_mod289(((value * 34.0) + 1.0) * value); } +float3 cultmath_snoise_permute(float3 value) { return cultmath_snoise_mod289(((value * 34.0) + 10.0) * value); } +float4 cultmath_snoise_permute(float4 value) { return cultmath_snoise_mod289(((value * 34.0) + 10.0) * value); } // Ashima Arts / Ian McEwan 3D simplex noise (MIT), same float32 evaluation // order as the C# math.snoise(float3) mirror. @@ -125,9 +125,9 @@ float cultmath_snoise(float3 value) p2 *= norm.z; p3 *= norm.w; - float4 m = max(0.6 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); + float4 m = max(0.5 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); m = m * m; - return 42.0 * dot(m * m, float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); + return 105.0 * dot(m * m, float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); } float cultmath_snoise(float2 value) @@ -162,6 +162,114 @@ float cultmath_snoise(float2 value) return 130.0 * dot(m, g); } +// Analytic gradient of cultmath_snoise(float3) (invariant 8), following webgl-noise +// src/noise3Dgrad.glsl (Ashima Arts / McEwan, MIT); same kernel, permutation and scale as +// cultmath_snoise(float3) (stegu/webgl-noise 22434e04d7). See the C# math.snoise_grad comment for the +// derivation. Every local up through p0..p3 and x0..x3 is cultmath_snoise(float3)'s own derivation +// verbatim. +float4 cultmath_snoise_grad(float3 value) +{ + float2 c = float2(1.0 / 6.0, 1.0 / 3.0); + float3 i = floor(value + dot(value, c.yyy)); + float3 x0 = value - i + dot(i, c.xxx); + float3 g = step(x0.yzx, x0.xyz); + float3 l = 1.0 - g; + float3 i1 = min(g.xyz, l.zxy); + float3 i2 = max(g.xyz, l.zxy); + float3 x1 = x0 - i1 + c.xxx; + float3 x2 = x0 - i2 + c.yyy; + float3 x3 = x0 - 0.5; + + i = cultmath_snoise_mod289(i); + float4 p = cultmath_snoise_permute(cultmath_snoise_permute(cultmath_snoise_permute( + i.z + float4(0.0, i1.z, i2.z, 1.0)) + i.y + float4(0.0, i1.y, i2.y, 1.0)) + i.x + float4(0.0, i1.x, i2.x, 1.0)); + + const float n_ = 0.142857142857; + float3 ns = float3(2.0 * n_, 0.5 * n_ - 1.0, n_); + float4 j = p - 49.0 * floor(p * ns.z * ns.z); + float4 x_ = floor(j * ns.z); + float4 y_ = floor(j - 7.0 * x_); + float4 x = x_ * ns.x + ns.yyyy; + float4 y = y_ * ns.x + ns.yyyy; + float4 h = 1.0 - abs(x) - abs(y); + + float4 b0 = float4(x.xy, y.xy); + float4 b1 = float4(x.zw, y.zw); + float4 s0 = floor(b0) * 2.0 + 1.0; + float4 s1 = floor(b1) * 2.0 + 1.0; + float4 sh = -step(h, 0.0); + float4 a0 = b0.xzyw + s0.xzyw * sh.xxyy; + float4 a1 = b1.xzyw + s1.xzyw * sh.zzww; + + float3 p0 = float3(a0.xy, h.x); + float3 p1 = float3(a0.zw, h.y); + float3 p2 = float3(a1.xy, h.z); + float3 p3 = float3(a1.zw, h.w); + float4 norm = 1.79284291400159 - 0.85373472095314 * float4(dot(p0, p0), dot(p1, p1), dot(p2, p2), dot(p3, p3)); + p0 *= norm.x; + p1 *= norm.y; + p2 *= norm.z; + p3 *= norm.w; + + float4 m0 = max(0.5 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); + float4 m2 = m0 * m0; + float4 m3 = m2 * m0; + float4 m4 = m2 * m2; + + float4 px = float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3)); + float value2 = 105.0 * dot(m4, px); + float3 grad = -8.0 * m3.x * x0 * px.x + m4.x * p0 + + -8.0 * m3.y * x1 * px.y + m4.y * p1 + + -8.0 * m3.z * x2 * px.z + m4.z * p2 + + -8.0 * m3.w * x3 * px.w + m4.w * p3; + grad *= 105.0; + + return float4(grad, value2); +} + +// Octave sum of cultmath_snoise_grad (invariant 8); see the C# math.fbm_grad comment for the +// per-octave frequency/amplitude derivation and why octaves is clamped. +float4 cultmath_fbm_grad(float3 p, int octaves, float lacunarity, float gain) +{ + octaves = clamp(octaves, 0, 16); + float amplitude = 1.0; + float frequency = 1.0; + float value = 0.0; + float3 gradient = float3(0.0, 0.0, 0.0); + for (int i = 0; i < octaves; i++) + { + float4 n = cultmath_snoise_grad(p * frequency); + value += amplitude * n.w; + gradient += amplitude * frequency * n.xyz; + frequency *= lacunarity; + amplitude *= gain; + } + + return float4(gradient, value); +} + +// Ridged multifractal noise (Musgrave, Texturing and Modeling, ch. 16); see the C# math.ridged_grad +// comment for the fold/sign derivation, the deliberate crease at n = 0, and why octaves is clamped. +float4 cultmath_ridged_grad(float3 p, int octaves, float lacunarity, float gain) +{ + octaves = clamp(octaves, 0, 16); + float amplitude = 1.0; + float frequency = 1.0; + float value = 0.0; + float3 gradient = float3(0.0, 0.0, 0.0); + for (int i = 0; i < octaves; i++) + { + float4 n = cultmath_snoise_grad(p * frequency); + float s = sign(n.w); + value += amplitude * (1.0 - abs(n.w)); + gradient += -amplitude * frequency * s * n.xyz; + frequency *= lacunarity; + amplitude *= gain; + } + + return float4(gradient, value); +} + float cultmath_lengthsq(float2 value) { return dot(value, value); } float cultmath_lengthsq(float3 value) { return dot(value, value); } float cultmath_lengthsq(float4 value) { return dot(value, value); } diff --git a/packages/cultmath/src/CultMath/BoundedLeastSquares.cs b/packages/cultmath/src/CultMath/BoundedLeastSquares.cs new file mode 100644 index 00000000..4cfbad74 --- /dev/null +++ b/packages/cultmath/src/CultMath/BoundedLeastSquares.cs @@ -0,0 +1,347 @@ +using System.Runtime.InteropServices; + +namespace CultMath; + +/// Outcome of . +public enum BoundedLeastSquaresStatus +{ + /// The KKT conditions hold: every free variable has zero gradient and every bound has a gradient pointing into it. + Converged, + + /// The iteration cap was reached. x holds the last feasible iterate, which is never worse than the clamped start. + IterationLimit, + + /// The input broke the contract; nothing was solved and x is exactly as passed. + InvalidInput, +} + +/// +/// Dense bounded least squares: min ||A x - b||^2 subject to lo <= x <= hi, for small problems +/// (tens of rows and columns). C#-only, like the intercept helpers in ; there is no HLSL mirror. +/// +/// +/// Primal active set. Each iteration solves the free variables by Cholesky on the normal equations, takes the +/// step to the first bound it would cross, and, once a full step lands, releases the bound with the most +/// violated gradient sign. Free columns that are linearly dependent on earlier free columns (a zero column, a +/// duplicate) get a zero step instead of a failed pivot, so a singular free set never throws. +/// +/// Precision. A, b and x are float at the API; everything else, including the working iterate, the +/// gradient and the KKT test, is double, carved from the caller's workspace. x is converted in at the start +/// and back out at the end, so a returned float x is the nearest float to the double solution and always +/// inside the box. +/// +/// +/// Stopping. KKT termination uses a tolerance of 1e-6 times the problem's gradient scale (the largest gradient +/// component at the box point nearest the origin), so scaling A, b or the bounds, and the choice of warm start, do +/// not change it. The tolerance is only a resolution: once a stationarity miss or a +/// bound release has been followed by a pass that lowers the cost by no more than rounding, the solver reports +/// , so an optimal point whose double gradient sits above the +/// tolerance costs one confirming iteration rather than the iteration cap. +/// +/// +/// Near-singular problems. The normal equations square the condition number, so the cost gap grows with cond(A) +/// (measured: below 1e-6 of |b|^2 through cond(A) 3e5, about 2e-5 at 1e6; the gap is relative to |b|^2), independent of the absolute scale of A. +/// The solver owns optimality up to that range. Columns whose pivot falls below 1e-12 of their own diagonal (cond(A) +/// beyond roughly 1e6) are treated as dependent and not moved, so the answer there is one optimum among many, not the +/// minimum-norm one. Callers beyond that range, or that need the minimum-norm answer, add a small ridge row to +/// A and b; that stays the caller's policy. +/// +/// The solver allocates nothing and is deterministic. +/// +/// Input contract: a and b must be finite; lo and hi must not be NaN and need +/// lo <= hi. Infinite bounds are allowed and mean unbounded on that side (lo = -Inf, +/// hi = +Inf); lo = +Inf or hi = -Inf is invalid. Any violation returns +/// without throwing, with iterations 0 and +/// x exactly as passed. A NaN or infinite warm start is legal: it is clamped into the box (an infinite +/// result on an unbounded side becomes 0 clamped to the box). Spans shorter than the stated dimensions are a +/// caller bug and throw . +/// +/// +public static class BoundedLeastSquares +{ + /// Default iteration cap for . + public const int DefaultMaxIterations = 100; + + private const double PivotRelativeTolerance = 1e-12; + private const double KktRelativeTolerance = 1e-6; + private const double StallRelativeTolerance = 1e-15; + + /// Floats of caller-supplied workspace needed for columns. + public static int WorkspaceLength(int n) => 2 * (2 * n * n + 6 * n) + 3 * n; + + /// + /// Minimises ||A x - b||^2 subject to lo <= x <= hi. is the warm start + /// on entry (clamped to the box first) and the result on exit. + /// + /// Rows of . + /// Columns of , and length of , and . + /// Row-major m * n matrix. + /// Length m target. + /// Lower bounds; each must be <= hi; -Inf means unbounded below. + /// Upper bounds; +Inf means unbounded above. + /// Warm start in, solution out. + /// At least floats; contents on entry are ignored. + /// Active-set iterations used. + /// Iteration cap. + public static BoundedLeastSquaresStatus Solve( + int m, int n, + ReadOnlySpan a, ReadOnlySpan b, + ReadOnlySpan lo, ReadOnlySpan hi, + Span x, Span workspace, + out int iterations, + int maxIterations = DefaultMaxIterations) + { + if (m < 0 || n < 0 || a.Length < m * n || b.Length < m || lo.Length < n || hi.Length < n + || x.Length < n || workspace.Length < WorkspaceLength(n)) + throw new ArgumentException("BoundedLeastSquares: span shorter than the stated dimensions."); + + // Double-precision vectors and matrices first, then the integer flags, all carved from the caller's floats. + var doubleCount = 2 * n * n + 6 * n; + var dbl = MemoryMarshal.Cast(workspace.Slice(0, 2 * doubleCount)); + var ata = Take(ref dbl, n * n); + var chol = Take(ref dbl, n * n); + var atb = Take(ref dbl, n); + var g = Take(ref dbl, n); + var p = Take(ref dbl, n); + var y = Take(ref dbl, n); + var rhs = Take(ref dbl, n); + var xd = Take(ref dbl, n); + // One flag per column: 0 free, -1 at lo, +1 at hi. Free-list and dependency flags follow. + var ints = MemoryMarshal.Cast(workspace.Slice(2 * doubleCount, 3 * n)); + var state = Take(ref ints, n); + var freeIndex = Take(ref ints, n); + var dependent = Take(ref ints, n); + + iterations = 0; + for (var j = 0; j < n; j++) + if (!(lo[j] <= hi[j]) || float.IsPositiveInfinity(lo[j]) || float.IsNegativeInfinity(hi[j])) + return BoundedLeastSquaresStatus.InvalidInput; + + // A float is never large enough to overflow a double product or a sum of tens of them, so a non-finite + // A^T b entry is exactly a non-finite entry of A or b (NaN and infinity never cancel to a finite value). + for (var j = 0; j < n; j++) + { + var sb = 0.0; + for (var r = 0; r < m; r++) + sb += (double)a[r * n + j] * b[r]; + if (!double.IsFinite(sb)) + return BoundedLeastSquaresStatus.InvalidInput; + atb[j] = sb; + for (var k = 0; k <= j; k++) + { + var sa = 0.0; + for (var r = 0; r < m; r++) + sa += (double)a[r * n + j] * a[r * n + k]; + ata[j * n + k] = sa; + ata[k * n + j] = sa; + } + } + + // From here x is written. A warm start that is NaN is moved to the lower bound, and an infinite result + // (an unbounded side) restarts from 0 clamped to the box. The working iterate lives in double; x is only + // the API's float view of it. + for (var j = 0; j < n; j++) + { + var l = lo[j]; + var h = hi[j]; + var v = float.IsNaN(x[j]) ? l : Math.Clamp(x[j], l, h); + if (float.IsInfinity(v)) + v = Math.Clamp(0f, l, h); + x[j] = v; + xd[j] = v; + state[j] = v == l ? -1 : (v == h ? 1 : 0); + } + + // The tolerance's scale is the gradient magnitude at the box point nearest the origin (the origin itself + // when it is inside the box, where the gradient is -A^T b). It depends on A, b and the bounds, never on the + // warm start, so a far start cannot be judged converged by its own steepness, and b = 0 with a box that + // excludes the origin still gets a real scale. It is zero only when that point is already stationary, and + // then the progress rule below does the stopping. + var gScale = 0.0; + for (var j = 0; j < n; j++) + { + var s = -atb[j]; + for (var k = 0; k < n; k++) + s += ata[j * n + k] * Math.Clamp(0.0, lo[k], hi[k]); + gScale = Math.Max(gScale, Math.Abs(s)); + } + var kktTol = KktRelativeTolerance * gScale; + + // Cost at the last point from which the solver chose to continue past a stationarity miss or a + // release. A pass that ends without lowering it has nothing left to gain: the tolerance is a + // resolution, not the stopping rule, and a noise floor above it must not spin to the cap. + var fMark = double.PositiveInfinity; + var status = BoundedLeastSquaresStatus.IterationLimit; + while (true) + { + Gradient(ata, atb, xd, g, n); + + var freeCount = 0; + for (var j = 0; j < n; j++) + if (state[j] == 0) + freeIndex[freeCount++] = j; + SolveFreeSet(ata, chol, g, p, y, rhs, freeIndex.Slice(0, freeCount), dependent, n); + + var t = 1.0; + var block = -1; + for (var i = 0; i < freeCount; i++) + { + var j = freeIndex[i]; + var pj = p[j]; + if (pj == 0.0) continue; + var tj = pj > 0.0 ? (hi[j] - xd[j]) / pj : (lo[j] - xd[j]) / pj; + if (tj < t) + { + t = tj; + block = j; + } + } + + if (block >= 0) + { + if (iterations >= maxIterations) { status = BoundedLeastSquaresStatus.IterationLimit; break; } + iterations++; + for (var i = 0; i < freeCount; i++) + { + var j = freeIndex[i]; + xd[j] = Math.Clamp(xd[j] + t * p[j], lo[j], hi[j]); + } + if (p[block] > 0.0) { xd[block] = hi[block]; state[block] = 1; } + else { xd[block] = lo[block]; state[block] = -1; } + continue; + } + + for (var i = 0; i < freeCount; i++) + { + var j = freeIndex[i]; + xd[j] = Math.Clamp(xd[j] + p[j], lo[j], hi[j]); + } + Gradient(ata, atb, xd, g, n); + + // Stationarity of the free set is part of KKT: a Newton step on a near-singular free set can leave a residual gradient. + var freeViolated = false; + for (var i = 0; i < freeCount; i++) + if (Math.Abs(g[freeIndex[i]]) > kktTol) + freeViolated = true; + + var release = -1; + var worst = kktTol; + for (var j = 0; j < n; j++) + { + if (lo[j] == hi[j]) continue; + // A bound at lo wants a positive gradient and one at hi a negative one; a free column has state 0. + var violation = state[j] * g[j]; + if (violation > worst) + { + worst = violation; + release = j; + } + } + if (release < 0 && !freeViolated) { status = BoundedLeastSquaresStatus.Converged; break; } + + var f = Objective(xd, g, atb, n, out var fScale); + if (f >= fMark - StallRelativeTolerance * fScale) { status = BoundedLeastSquaresStatus.Converged; break; } + fMark = f; + if (iterations >= maxIterations) { status = BoundedLeastSquaresStatus.IterationLimit; break; } + iterations++; + // A stationarity miss re-solves the same free set from the improved point. + if (release >= 0) state[release] = 0; + } + + for (var j = 0; j < n; j++) + x[j] = (float)xd[j]; + return status; + } + + private static Span Take(ref Span pool, int length) + { + var head = pool.Slice(0, length); + pool = pool.Slice(length); + return head; + } + + // 0.5 x^T A^T A x - x^T A^T b (the cost up to a constant), from the gradient at x; scale is the magnitude of + // the terms summed, which is what the rounding error of the sum is proportional to. + private static double Objective(ReadOnlySpan x, ReadOnlySpan g, ReadOnlySpan atb, int n, out double scale) + { + var f = 0.0; + scale = 0.0; + for (var j = 0; j < n; j++) + { + f += 0.5 * x[j] * (g[j] - atb[j]); + scale += 0.5 * Math.Abs(x[j]) * (Math.Abs(g[j]) + Math.Abs(atb[j])); + } + return f; + } + + private static void Gradient(ReadOnlySpan ata, ReadOnlySpan atb, ReadOnlySpan x, Span g, int n) + { + for (var j = 0; j < n; j++) + { + var s = -atb[j]; + for (var k = 0; k < n; k++) + s += ata[j * n + k] * x[k]; + g[j] = s; + } + } + + // Newton step over the free set: solves (A^T A)_FF p_F = -g_F by Cholesky. A free column whose pivot collapses + // (dependent on earlier free columns) is dropped from the factor and gets p = 0. No refinement step: Cholesky is + // backward stable, so the residual, which is the gradient the KKT test reads, is already at rounding level, and + // refining in the same precision does not improve the forward error. + private static void SolveFreeSet( + ReadOnlySpan ata, Span l, ReadOnlySpan g, Span p, Span y, + Span rhs, ReadOnlySpan free, Span dependent, int n) + { + var k = free.Length; + for (var i = 0; i < k; i++) + { + var fi = free[i]; + var d = ata[fi * n + fi]; + for (var c = 0; c < i; c++) + d -= l[i * n + c] * l[i * n + c]; + if (d <= PivotRelativeTolerance * ata[fi * n + fi]) + { + dependent[i] = 1; + for (var c = 0; c < i; c++) l[i * n + c] = 0.0; + l[i * n + i] = 1.0; + for (var r = i + 1; r < k; r++) l[r * n + i] = 0.0; + continue; + } + dependent[i] = 0; + var lii = Math.Sqrt(d); + l[i * n + i] = lii; + for (var r = i + 1; r < k; r++) + { + var s = ata[free[r] * n + fi]; + for (var c = 0; c < i; c++) + s -= l[r * n + c] * l[i * n + c]; + l[r * n + i] = s / lii; + } + } + + for (var i = 0; i < k; i++) + rhs[i] = dependent[i] != 0 ? 0.0 : -g[free[i]]; + Substitute(l, rhs, y, p, free, n); + } + + // Solves L L^T out = rhs; rhs is indexed by free-list position, out by column. + private static void Substitute(ReadOnlySpan l, ReadOnlySpan rhs, Span y, Span outByColumn, ReadOnlySpan free, int n) + { + var k = free.Length; + for (var i = 0; i < k; i++) + { + var s = rhs[i]; + for (var c = 0; c < i; c++) + s -= l[i * n + c] * y[c]; + y[i] = s / l[i * n + i]; + } + for (var i = k - 1; i >= 0; i--) + { + var s = y[i]; + for (var r = i + 1; r < k; r++) + s -= l[r * n + i] * outByColumn[free[r]]; + outByColumn[free[i]] = s / l[i * n + i]; + } + } +} diff --git a/packages/cultmath/src/CultMath/BoundedLeastSquares.cs.meta b/packages/cultmath/src/CultMath/BoundedLeastSquares.cs.meta new file mode 100644 index 00000000..fb5bd819 --- /dev/null +++ b/packages/cultmath/src/CultMath/BoundedLeastSquares.cs.meta @@ -0,0 +1,2 @@ +fileFormatVersion: 2 +guid: 4ba991e1b6c54e099bc7ea16541734c5 diff --git a/packages/cultmath/src/CultMath/math.cs b/packages/cultmath/src/CultMath/math.cs index 1b03f4ed..23916b6b 100644 --- a/packages/cultmath/src/CultMath/math.cs +++ b/packages/cultmath/src/CultMath/math.cs @@ -595,7 +595,10 @@ public static partial class math } // Ashima Arts / Ian McEwan 3D simplex noise (MIT), mirrored by cultmath_snoise(float3) - // in shaders/CultMath.hlsl with the same float32 evaluation order. + // in shaders/CultMath.hlsl with the same float32 evaluation order. Follows stegu/webgl-noise + // src/noise3D.glsl at 22434e04d7 (kernel radius^2 0.5, scale 105, permute (34x+10)x); see + // THIRD-PARTY-NOTICES.md. Continuous across simplex-cell boundaries + // (NoiseGradTests.SnoiseAndItsGradientAreContinuousAcrossSimplexCellBoundaries). public static float snoise(float3 value) { const float cx = 1.0f / 6.0f; @@ -642,9 +645,9 @@ public static partial class math p2 *= norm.z; p3 *= norm.w; - var m = max(0.6f - new float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0f); + var m = max(0.5f - new float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0f); m = m * m; - return 42.0f * dot(m * m, new float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); + return 105.0f * dot(m * m, new float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); } // Ashima Arts / Ian McEwan 2D simplex noise, kept component-explicit so @@ -690,8 +693,134 @@ public static partial class math private static float2 snoise_mod289(float2 value) => value - floor(value * (1.0f / 289.0f)) * 289.0f; private static float3 snoise_mod289(float3 value) => value - floor(value * (1.0f / 289.0f)) * 289.0f; private static float4 snoise_mod289(float4 value) => value - floor(value * (1.0f / 289.0f)) * 289.0f; - private static float3 snoise_permute(float3 value) => snoise_mod289(((value * 34.0f) + 1.0f) * value); - private static float4 snoise_permute(float4 value) => snoise_mod289(((value * 34.0f) + 1.0f) * value); + private static float3 snoise_permute(float3 value) => snoise_mod289(((value * 34.0f) + 10.0f) * value); + private static float4 snoise_permute(float4 value) => snoise_mod289(((value * 34.0f) + 10.0f) * value); + + // Analytic gradient of snoise(float3) (invariant 8), following the differentiation in + // webgl-noise src/noise3Dgrad.glsl (Ashima Arts / McEwan, MIT), which uses the same kernel + // radius^2 0.5, permutation (34x+10)x and scale 105 as snoise(float3) (stegu/webgl-noise + // 22434e04d7). Every local up through p0..p3 and x0..x3 below is snoise(float3)'s own derivation + // verbatim, so .w is that same field (NoiseGradTests.SnoiseGradValueIsBitEqualToSnoise pins the + // two bit-equal over a random sweep). Per corner, m0 = max(0.5 - dot(x,x), 0), so dm0/dx = -2x, + // and d(m0^4 * dot(p,x))/dx = 4*m0^3*(-2x)*dot(p,x) + m0^4*p = -8*m0^3*dot(p,x)*x + m0^4*p (p is + // that corner's own gradient constant, independent of x); summed over the four corners and + // scaled by the same 105 as the value. + public static float4 snoise_grad(float3 value) + { + const float cx = 1.0f / 6.0f; + const float cy = 1.0f / 3.0f; + var i = floor(value + dot(value, new float3(cy, cy, cy))); + var x0 = value - i + dot(i, new float3(cx, cx, cx)); + var g = step(new float3(x0.y, x0.z, x0.x), x0); + var l = 1.0f - g; + var lzxy = new float3(l.z, l.x, l.y); + var i1 = min(g, lzxy); + var i2 = max(g, lzxy); + var x1 = x0 - i1 + cx; + var x2 = x0 - i2 + cy; + var x3 = x0 - 0.5f; + + i = snoise_mod289(i); + var p = snoise_permute(snoise_permute(snoise_permute( + i.z + new float4(0.0f, i1.z, i2.z, 1.0f)) + i.y + new float4(0.0f, i1.y, i2.y, 1.0f)) + i.x + new float4(0.0f, i1.x, i2.x, 1.0f)); + + const float n = 0.142857142857f; + var ns = new float3(2.0f * n, 0.5f * n - 1.0f, n); + var j = p - 49.0f * floor(p * ns.z * ns.z); + var xs = floor(j * ns.z); + var ys = floor(j - 7.0f * xs); + var x = xs * ns.x + ns.y; + var y = ys * ns.x + ns.y; + var h = 1.0f - abs(x) - abs(y); + + var b0 = new float4(x.x, x.y, y.x, y.y); + var b1 = new float4(x.z, x.w, y.z, y.w); + var s0 = floor(b0) * 2.0f + 1.0f; + var s1 = floor(b1) * 2.0f + 1.0f; + var sh = -step(h, 0.0f); + var a0 = new float4(b0.x, b0.z, b0.y, b0.w) + new float4(s0.x, s0.z, s0.y, s0.w) * new float4(sh.x, sh.x, sh.y, sh.y); + var a1 = new float4(b1.x, b1.z, b1.y, b1.w) + new float4(s1.x, s1.z, s1.y, s1.w) * new float4(sh.z, sh.z, sh.w, sh.w); + + var p0 = new float3(a0.x, a0.y, h.x); + var p1 = new float3(a0.z, a0.w, h.y); + var p2 = new float3(a1.x, a1.y, h.z); + var p3 = new float3(a1.z, a1.w, h.w); + var norm = 1.79284291400159f - 0.85373472095314f * new float4(dot(p0, p0), dot(p1, p1), dot(p2, p2), dot(p3, p3)); + p0 *= norm.x; + p1 *= norm.y; + p2 *= norm.z; + p3 *= norm.w; + + var m0 = max(0.5f - new float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0f); + var m2 = m0 * m0; + var m3 = m2 * m0; + var m4 = m2 * m2; + + var px = new float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3)); + var value2 = 105.0f * dot(m4, px); + var grad = -8.0f * m3.x * x0 * px.x + m4.x * p0 + + -8.0f * m3.y * x1 * px.y + m4.y * p1 + + -8.0f * m3.z * x2 * px.z + m4.z * p2 + + -8.0f * m3.w * x3 * px.w + m4.w * p3; + grad *= 105.0f; + + return new float4(grad, value2); + } + + // Octave sum of snoise_grad (invariant 8). Each octave samples snoise_grad at p scaled by the + // running frequency, so by the chain rule its gradient scales by that same frequency + // (d/dp[n(f*p)] = f * grad_n(f*p)); amplitude and frequency both compound per octave (gain, + // lacunarity respectively), not applied once for the whole sum, so with 1 octave this is exactly + // snoise_grad (NoiseGradTests.FbmGradWithOneOctaveEqualsSnoiseGrad). octaves is clamped to + // [0, 16]: HlslSourceCompatibilityTests' bit-parity oracle drives every mirrored int parameter + // across the full int32 range, which is the right domain for pcg3d/pcg4d's O(1) hash inputs but + // would turn an unclamped octave count into a multi-billion-iteration loop for this mirror. + public static float4 fbm_grad(float3 p, int octaves, float lacunarity, float gain) + { + octaves = clamp(octaves, 0, 16); + var amplitude = 1.0f; + var frequency = 1.0f; + var value = 0.0f; + var gradient = new float3(0.0f, 0.0f, 0.0f); + for (var i = 0; i < octaves; i++) + { + var n = snoise_grad(p * frequency); + value += amplitude * n.w; + gradient += amplitude * frequency * new float3(n.x, n.y, n.z); + frequency *= lacunarity; + amplitude *= gain; + } + + return new float4(gradient, value); + } + + // Ridged multifractal noise (Musgrave, Texturing and Modeling, ch. 16; the operator's dune term, + // 2026-09-25): each octave folds snoise_grad about zero, value = Sum a_i*(1 - |n_i|), gradient = + // -Sum a_i*f_i*sign(n_i)*grad(n_i), the same per-octave frequency scaling as fbm_grad's chain + // rule plus the fold's sign flip (d|n|/dp = sign(n)*dn/dp). The crease at n_i = 0 is real and + // deliberate, not a bug: NoiseGradTests excludes a band around it, the same way cellular excludes + // its F1 = F2 seam. sign follows this codebase's HLSL-matching convention (design.md: sign + // returns int, 0 at exactly 0), so the crease itself contributes no gradient rather than an + // arbitrary one. Same octave clamp as fbm_grad, for the same bit-parity-oracle reason. + public static float4 ridged_grad(float3 p, int octaves, float lacunarity, float gain) + { + octaves = clamp(octaves, 0, 16); + var amplitude = 1.0f; + var frequency = 1.0f; + var value = 0.0f; + var gradient = new float3(0.0f, 0.0f, 0.0f); + for (var i = 0; i < octaves; i++) + { + var n = snoise_grad(p * frequency); + var s = (float)sign(n.w); + value += amplitude * (1.0f - abs(n.w)); + gradient += -amplitude * frequency * s * new float3(n.x, n.y, n.z); + frequency *= lacunarity; + amplitude *= gain; + } + + return new float4(gradient, value); + } // Inigo Quilez, "smooth minimum" (https://iquilezles.org/articles/smin/): the quadratic-polynomial // smin, carried to value-and-gradient form (invariant 8). h is affine in (b.w - a.w) inside the diff --git a/packages/cultmath/tests/CultMath.Tests/BoundedLeastSquaresTests.cs b/packages/cultmath/tests/CultMath.Tests/BoundedLeastSquaresTests.cs new file mode 100644 index 00000000..a9a950be --- /dev/null +++ b/packages/cultmath/tests/CultMath.Tests/BoundedLeastSquaresTests.cs @@ -0,0 +1,1141 @@ +using System; +using CultMath; +using Xunit; + +namespace CultMath.Tests; + +// Fixtures are deliberately non-degenerate: A is non-square and non-identity, and b is outside A's range, +// so the unconstrained optimum has a real residual and the box actually bites. +public sealed class BoundedLeastSquaresTests +{ + private static double Cost(int m, int n, float[] a, float[] b, float[] x) + { + var c = 0.0; + for (var r = 0; r < m; r++) + { + var s = -(double)b[r]; + for (var j = 0; j < n; j++) s += (double)a[r * n + j] * x[j]; + c += s * s; + } + return c; + } + + // Gradient of 0.5*||Ax-b||^2, in double. + private static double[] Gradient(int m, int n, float[] a, float[] b, float[] x) + { + var g = new double[n]; + for (var r = 0; r < m; r++) + { + var s = -(double)b[r]; + for (var j = 0; j < n; j++) s += (double)a[r * n + j] * x[j]; + for (var j = 0; j < n; j++) g[j] += a[r * n + j] * s; + } + return g; + } + + private static double AtbScale(int m, int n, float[] a, float[] b) + { + var mx = 0.0; + for (var j = 0; j < n; j++) + { + var s = 0.0; + for (var r = 0; r < m; r++) s += (double)a[r * n + j] * b[r]; + mx = Math.Max(mx, Math.Abs(s)); + } + return 1 + mx; + } + + // Asserts feasibility and the KKT sign rule: free columns have ~zero gradient, columns at lo have + // gradient >= 0, columns at hi have gradient <= 0. + private static void AssertKkt(int m, int n, float[] a, float[] b, float[] lo, float[] hi, float[] x) + { + var g = Gradient(m, n, a, b, x); + var tol = 1e-3 * AtbScale(m, n, a, b); + for (var j = 0; j < n; j++) + { + Assert.True(x[j] >= lo[j] && x[j] <= hi[j], $"column {j} outside its box: {x[j]}"); + if (lo[j] == hi[j]) continue; + if (x[j] == lo[j]) Assert.True(g[j] >= -tol, $"column {j} at lo with gradient {g[j]}"); + else if (x[j] == hi[j]) Assert.True(g[j] <= tol, $"column {j} at hi with gradient {g[j]}"); + else Assert.True(Math.Abs(g[j]) <= tol, $"column {j} free with gradient {g[j]}"); + } + } + + // Accelerated projected gradient (FISTA) with step 1/L, L bounded by the Frobenius norm squared of A. + // Independent of the solver's active-set logic; run to a fixed point. + private static double ReferenceCost(int m, int n, float[] a, float[] b, float[] lo, float[] hi) + { + var frob = 0.0; + foreach (var v in a) frob += (double)v * v; + var step = 1.0 / Math.Max(frob, 1e-12); + var x = new double[n]; + var y = new double[n]; + var xf = new float[n]; + for (var j = 0; j < n; j++) x[j] = y[j] = Math.Clamp(0.0, lo[j], hi[j]); + var t = 1.0; + for (var it = 0; it < 100000; it++) + { + for (var j = 0; j < n; j++) xf[j] = (float)y[j]; + var g = Gradient(m, n, a, b, xf); + var moved = 0.0; + var tNext = (1.0 + Math.Sqrt(1.0 + 4.0 * t * t)) / 2.0; + for (var j = 0; j < n; j++) + { + var next = Math.Clamp(y[j] - step * g[j], lo[j], hi[j]); + moved = Math.Max(moved, Math.Abs(next - x[j])); + y[j] = next + (t - 1.0) / tNext * (next - x[j]); + x[j] = next; + } + t = tNext; + if (moved < 1e-10) break; + } + for (var j = 0; j < n; j++) xf[j] = (float)x[j]; + return Cost(m, n, a, b, xf); + } + + private static BoundedLeastSquaresStatus Solve(int m, int n, float[] a, float[] b, float[] lo, float[] hi, float[] x, + out int iterations, int maxIterations = BoundedLeastSquares.DefaultMaxIterations) + => BoundedLeastSquares.Solve(m, n, a, b, lo, hi, x, new float[BoundedLeastSquares.WorkspaceLength(n)], out iterations, maxIterations); + + private static float[] Fill(float value, int n) + { + var v = new float[n]; + Array.Fill(v, value); + return v; + } + + private static double[] NormalEquationSolution(int m, int n, float[] a, float[] b) + { + var mat = new double[n, n + 1]; + for (var i = 0; i < n; i++) + { + for (var j = 0; j < n; j++) + for (var r = 0; r < m; r++) mat[i, j] += (double)a[r * n + i] * a[r * n + j]; + for (var r = 0; r < m; r++) mat[i, n] += (double)a[r * n + i] * b[r]; + } + for (var c = 0; c < n; c++) + { + var piv = mat[c, c]; + for (var j = 0; j <= n; j++) mat[c, j] /= piv; + for (var i = 0; i < n; i++) + { + if (i == c) continue; + var f = mat[i, c]; + for (var j = 0; j <= n; j++) mat[i, j] -= f * mat[c, j]; + } + } + var x = new double[n]; + for (var i = 0; i < n; i++) x[i] = mat[i, n]; + return x; + } + + private static (int m, int n, float[] a, float[] b, float[] lo, float[] hi) RandomProblem(uint seed) + { + var rng = new CultMath.Random(seed * 7919u); + var n = rng.NextInt(2, 17); + var m = rng.NextInt(n, 25); + var a = new float[m * n]; + var b = new float[m]; + for (var i = 0; i < a.Length; i++) a[i] = rng.NextFloat(-1f, 1f); + for (var i = 0; i < m; i++) b[i] = rng.NextFloat(-3f, 3f); + var lo = new float[n]; + var hi = new float[n]; + for (var j = 0; j < n; j++) + { + switch (rng.NextInt(3)) + { + case 0: lo[j] = -1f; hi[j] = 1f; break; + case 1: lo[j] = 0f; hi[j] = 1f; break; + default: lo[j] = -0.25f; hi[j] = 0.1f; break; + } + } + return (m, n, a, b, lo, hi); + } + + private static readonly float[] A54 = + { + 1.0f, 0.3f, -0.2f, + 0.2f, 1.1f, 0.4f, + -0.5f, 0.6f, 0.9f, + 0.7f, -0.4f, 0.5f, + 0.1f, 0.8f, -0.6f, + }; + + private static readonly float[] B5 = { 0.9f, -0.4f, 1.3f, 0.2f, -0.7f }; + + [Fact] + public void InteriorOptimumEqualsNormalEquations() + { + const int m = 5, n = 3; + var x = new float[n]; + var status = Solve(m, n, A54, B5, Fill(-100f, n), Fill(100f, n), x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + + var expected = NormalEquationSolution(m, n, A54, B5); + for (var i = 0; i < n; i++) Assert.Equal(expected[i], x[i], 1e-4); + } + + [Fact] + public void KktHoldsOnRandomBoxedProblems() + { + var anyActive = false; + for (uint seed = 1; seed <= 150; seed++) + { + var (m, n, a, b, lo, hi) = RandomProblem(seed); + var x = new float[n]; + var status = Solve(m, n, a, b, lo, hi, x, out _); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"seed {seed} ({m}x{n}) hit the cap"); + AssertKkt(m, n, a, b, lo, hi, x); + for (var j = 0; j < n; j++) anyActive |= x[j] == lo[j] || x[j] == hi[j]; + + var cs = Cost(m, n, a, b, x); + var cr = ReferenceCost(m, n, a, b, lo, hi); + Assert.True(Math.Abs(cs - cr) <= 1e-4 * cr + 1e-6, $"seed {seed} ({m}x{n}): solver cost {cs}, reference {cr}"); + } + Assert.True(anyActive); + } + + [Fact] + public void AllBoundsActive() + { + const int m = 4, n = 3; + var a = new[] { 1.0f, 0.2f, 0.1f, 0.1f, 1.0f, 0.3f, -0.2f, 0.2f, 1.0f, 0.5f, 0.5f, 0.4f }; + var b = new[] { 10f, -10f, 10f, 1f }; + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + var x = new float[n]; + var status = Solve(m, n, a, b, lo, hi, x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(new[] { 1f, -1f, 1f }, x); + Assert.Equal(3, iterations); + AssertKkt(m, n, a, b, lo, hi, x); + } + + [Fact] + public void ZeroColumn() + { + const int m = 5, n = 3; + var a = (float[])A54.Clone(); + for (var r = 0; r < m; r++) a[r * n + 1] = 0f; + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + var x = new float[n]; + var status = Solve(m, n, a, B5, lo, hi, x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + AssertKkt(m, n, a, B5, lo, hi, x); + var cs = Cost(m, n, a, B5, x); + var cr = ReferenceCost(m, n, a, B5, lo, hi); + Assert.True(Math.Abs(cs - cr) <= 1e-4 * cr + 1e-6); + } + + [Fact] + public void DuplicateColumns() + { + const int m = 5, n = 3; + var a = (float[])A54.Clone(); + for (var r = 0; r < m; r++) a[r * n + 2] = a[r * n + 0]; + var lo = Fill(-1f, n); + var hi = new[] { 0.3f, 1f, 0.3f }; + var x = new float[n]; + var status = Solve(m, n, a, B5, lo, hi, x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + AssertKkt(m, n, a, B5, lo, hi, x); + var cs = Cost(m, n, a, B5, x); + var cr = ReferenceCost(m, n, a, B5, lo, hi); + Assert.True(Math.Abs(cs - cr) <= 1e-4 * cr + 1e-6); + } + + [Fact] + public void WarmStartMatchesColdStart() + { + const int m = 5, n = 3; + var lo = new[] { -0.2f, -1f, 0f }; + var hi = new[] { 0.5f, 0.3f, 1f }; + var cold = new float[n]; + Solve(m, n, A54, B5, lo, hi, cold, out _); + AssertKkt(m, n, A54, B5, lo, hi, cold); + + var starts = new[] { new[] { 0.5f, -1f, 1f }, new[] { 9f, 9f, -9f }, new[] { 0.1f, 0.1f, 0.5f }, (float[])cold.Clone() }; + foreach (var start in starts) + { + var status = Solve(m, n, A54, B5, lo, hi, start, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + for (var j = 0; j < n; j++) Assert.Equal(cold[j], start[j], 1e-4); + if (ReferenceEquals(start, starts[^1])) Assert.Equal(0, iterations); + } + } + + [Fact] + public void AsymmetricBounds() + { + const int m = 5, n = 3; + var lo = new[] { -1f, 0f, -1f }; + var hi = new[] { 1f, 1f, 1f }; + // Column 1 wants to go negative (b has a strong component against it) but its floor is 0. + var b = new[] { 0.2f, -2.5f, 0.4f, -1.5f, -2.0f }; + var x = new float[n]; + var status = Solve(m, n, A54, b, lo, hi, x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + AssertKkt(m, n, A54, b, lo, hi, x); + Assert.Equal(0f, x[1]); + Assert.True(x[0] < 0f || x[2] < 0f, "a symmetric-bounds column should use its negative side"); + var cs = Cost(m, n, A54, b, x); + var cr = ReferenceCost(m, n, A54, b, lo, hi); + Assert.True(Math.Abs(cs - cr) <= 1e-4 * cr + 1e-6); + } + + [Fact] + public void IterationCapReturnsFeasibleIterateAndLimitStatus() + { + const int m = 4, n = 3; + var a = new[] { 1.0f, 0.2f, 0.1f, 0.1f, 1.0f, 0.3f, -0.2f, 0.2f, 1.0f, 0.5f, 0.5f, 0.4f }; + var b = new[] { 10f, -10f, 10f, 1f }; + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + var x = new float[n]; + var start = Cost(m, n, a, b, x); + var status = Solve(m, n, a, b, lo, hi, x, out var iterations, maxIterations: 1); + Assert.Equal(BoundedLeastSquaresStatus.IterationLimit, status); + Assert.Equal(1, iterations); + for (var j = 0; j < n; j++) Assert.InRange(x[j], lo[j], hi[j]); + Assert.True(Cost(m, n, a, b, x) < start); + + var zero = new float[n]; + Assert.Equal(BoundedLeastSquaresStatus.IterationLimit, Solve(m, n, a, b, lo, hi, zero, out iterations, maxIterations: 0)); + Assert.Equal(0, iterations); + Assert.Equal(new float[n], zero); + } + + [Fact] + public void StartOutsideTheBoxIsClampedFirst() + { + const int m = 5, n = 3; + var lo = Fill(-0.1f, n); + var hi = Fill(0.1f, n); + var x = new[] { 50f, -50f, float.NaN }; + Solve(m, n, A54, B5, lo, hi, x, out _, maxIterations: 0); + Assert.Equal(new[] { 0.1f, -0.1f, -0.1f }, x); + } + + [Fact] + public void FixedColumnStaysPutEvenWhenItsGradientWantsOut() + { + const int m = 5, n = 3; + var lo = new[] { -100f, -1f, -100f }; + var hi = new[] { 100f, -1f, 100f }; + var x = new float[n]; + var status = Solve(m, n, A54, B5, lo, hi, x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(-1f, x[1]); + Assert.True(Gradient(m, n, A54, B5, x)[1] < -1e-3, "fixture must pull the fixed column inward"); + Assert.Equal(0, iterations); + AssertKkt(m, n, A54, B5, lo, hi, x); + } + + [Fact] + public void WarmCallAllocatesNothing() + { + const int m = 5, n = 3; + var lo = new[] { -0.2f, -1f, 0f }; + var hi = new[] { 0.5f, 0.3f, 1f }; + var x = new float[n]; + var workspace = new float[BoundedLeastSquares.WorkspaceLength(n)]; + BoundedLeastSquares.Solve(m, n, A54, B5, lo, hi, x, workspace, out _); + + var before = GC.GetAllocatedBytesForCurrentThread(); + var status = BoundedLeastSquares.Solve(m, n, A54, B5, lo, hi, x, workspace, out _); + var after = GC.GetAllocatedBytesForCurrentThread(); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(0L, after - before); + } + + [Fact] + public void LargeInteriorOptimumEqualsNormalEquations() + { + const int m = 24, n = 16; + var rng = new CultMath.Random(4242u); + var a = new float[m * n]; + var b = new float[m]; + for (var i = 0; i < a.Length; i++) a[i] = rng.NextFloat(-1f, 1f); + for (var i = 0; i < m; i++) b[i] = rng.NextFloat(-3f, 3f); + var x = new float[n]; + var status = Solve(m, n, a, b, Fill(-1000f, n), Fill(1000f, n), x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(0, iterations); + var expected = NormalEquationSolution(m, n, a, b); + for (var i = 0; i < n; i++) Assert.Equal(expected[i], x[i], 1e-3); + } + + [Fact] + public void ScaledAndNearDuplicateColumnsAreOptimal() + { + const int m = 5, n = 3; + foreach (var noise in new[] { 0f, 1e-5f }) + { + var a = (float[])A54.Clone(); + for (var r = 0; r < m; r++) a[r * n + 2] = 0.37f * a[r * n + 0] + noise * (r - 2); + var lo = Fill(-1f, n); + var hi = new[] { 0.6f, 1f, 0.6f }; + var x = new float[n]; + var status = Solve(m, n, a, B5, lo, hi, x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + AssertKkt(m, n, a, B5, lo, hi, x); + var cs = Cost(m, n, a, B5, x); + var cr = ReferenceCost(m, n, a, B5, lo, hi); + Assert.True(Math.Abs(cs - cr) <= 1e-4 * cr + 1e-6, $"noise {noise}: solver cost {cs}, reference {cr}"); + } + } + + [Fact] + public void WorkspaceContentsOnEntryAreIgnored() + { + const int m = 5, n = 3; + var zeroCol = (float[])A54.Clone(); + for (var r = 0; r < m; r++) zeroCol[r * n + 1] = 0f; + var dupCol = (float[])A54.Clone(); + for (var r = 0; r < m; r++) dupCol[r * n + 2] = dupCol[r * n + 0]; + foreach (var a in new[] { A54, zeroCol, dupCol }) + { + var lo = new[] { -1f, -1f, -1f }; + var hi = new[] { 0.3f, 1f, 0.3f }; + var clean = new float[n]; + Solve(m, n, a, B5, lo, hi, clean, out var cleanIterations); + foreach (var junk in new[] { float.NaN, 7f, -1e30f }) + { + var workspace = new float[BoundedLeastSquares.WorkspaceLength(n)]; + Array.Fill(workspace, junk); + var x = new float[n]; + BoundedLeastSquares.Solve(m, n, a, B5, lo, hi, x, workspace, out var iterations); + Assert.Equal(clean, x); + Assert.Equal(cleanIterations, iterations); + } + } + } + + // Junk in the workspace, including NaN doubles, must not reach any decision: random boxed problems (which release + // bounds and use the tolerance) give bit-identical x and the same iteration count. + [Fact] + public void WorkspaceJunkNeverReachesTheSolveOnRandomProblems() + { + for (uint seed = 1; seed <= 25; seed++) + { + var (m, n, a, b, lo, hi) = RandomProblem(seed); + var clean = new float[n]; + Solve(m, n, a, b, lo, hi, clean, out var cleanIterations); + foreach (var junk in new[] { float.NaN, 7f, -1e30f }) + { + var workspace = new float[BoundedLeastSquares.WorkspaceLength(n)]; + Array.Fill(workspace, junk); + var x = new float[n]; + BoundedLeastSquares.Solve(m, n, a, b, lo, hi, x, workspace, out var iterations); + Assert.Equal(cleanIterations, iterations); + for (var j = 0; j < n; j++) + Assert.Equal(BitConverter.SingleToInt32Bits(clean[j]), BitConverter.SingleToInt32Bits(x[j])); + } + } + } + + [Fact] + public void WarmStartAtAVertexTakesNoIterations() + { + const int m = 4, n = 3; + var a = new[] { 1.0f, 0.2f, 0.1f, 0.1f, 1.0f, 0.3f, -0.2f, 0.2f, 1.0f, 0.5f, 0.5f, 0.4f }; + var b = new[] { 10f, -10f, 10f, 1f }; + var x = new[] { 1f, -1f, 1f }; + var status = Solve(m, n, a, b, Fill(-1f, n), Fill(1f, n), x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(0, iterations); + Assert.Equal(new[] { 1f, -1f, 1f }, x); + } + + [Fact] + public void IterationCountFollowsTheCapExactly() + { + var sawRelease = false; + for (uint seed = 1; seed <= 40; seed++) + { + var (m, n, a, b, lo, hi) = RandomProblem(seed); + var full = new float[n]; + Solve(m, n, a, b, lo, hi, full, out var needed); + var previousCost = double.MaxValue; + for (var cap = 0; cap <= needed; cap++) + { + var x = new float[n]; + var status = Solve(m, n, a, b, lo, hi, x, out var iterations, cap); + Assert.Equal(cap, iterations); + Assert.Equal(cap < needed ? BoundedLeastSquaresStatus.IterationLimit : BoundedLeastSquaresStatus.Converged, status); + for (var j = 0; j < n; j++) Assert.InRange(x[j], lo[j], hi[j]); + var cost = Cost(m, n, a, b, x); + Assert.True(cost <= previousCost * (1 + 1e-6) + 1e-9, $"seed {seed} cap {cap}: cost rose from {previousCost} to {cost}"); + previousCost = cost; + } + sawRelease |= needed > n; + } + Assert.True(sawRelease, "no random problem needed more iterations than columns, so no bound was ever released"); + } + + [Fact] + public void EachShortSpanIsRejected() + { + const int m = 5, n = 3; + var ws = new float[BoundedLeastSquares.WorkspaceLength(n)]; + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + void Expect(Action call) => Assert.Contains("BoundedLeastSquares", Assert.Throws(call).Message); + Expect(() => BoundedLeastSquares.Solve(m, n, new float[m * n - 1], B5, lo, hi, new float[n], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, n, A54, new float[m - 1], lo, hi, new float[n], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, n, A54, B5, new float[n - 1], hi, new float[n], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, n, A54, B5, lo, new float[n - 1], new float[n], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, n, A54, B5, lo, hi, new float[n - 1], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, n, A54, B5, lo, hi, new float[n], new float[ws.Length - 1], out _)); + Expect(() => BoundedLeastSquares.Solve(-1, n, A54, B5, lo, hi, new float[n], ws, out _)); + Expect(() => BoundedLeastSquares.Solve(m, -1, A54, B5, lo, hi, new float[n], ws, out _)); + // Exactly sized spans, and empty problems, are accepted. + BoundedLeastSquares.Solve(m, n, A54, B5, lo, hi, new float[n], ws, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, BoundedLeastSquares.Solve(0, 0, default, default, default, default, default, default, out _)); + Assert.Equal(BoundedLeastSquaresStatus.Converged, BoundedLeastSquares.Solve(0, n, default, default, lo, hi, new float[n], ws, out _)); + } + + // Well-conditioned enough (m = n + 6) that x is comparable across scales. + private static (int m, int n, float[] a, float[] b, float[] lo, float[] hi) ScaleProblem(uint seed) + { + var rng = new CultMath.Random(seed * 104729u); + var n = rng.NextInt(3, 11); + var m = n + 6; + var a = new float[m * n]; + var b = new float[m]; + for (var i = 0; i < a.Length; i++) a[i] = rng.NextFloat(-1f, 1f); + for (var i = 0; i < m; i++) b[i] = rng.NextFloat(-3f, 3f); + var lo = new float[n]; + var hi = new float[n]; + for (var j = 0; j < n; j++) + { + lo[j] = rng.NextFloat(-0.6f, -0.05f); + hi[j] = rng.NextFloat(0.05f, 0.6f); + } + return (m, n, a, b, lo, hi); + } + + private static float[] Scaled(float[] v, float k) + { + var r = new float[v.Length]; + for (var i = 0; i < v.Length; i++) r[i] = v[i] * k; + return r; + } + + [Fact] + public void SolutionIsScaleFree() + { + var sawActive = false; + for (uint seed = 1; seed <= 40; seed++) + { + var (m, n, a, b, lo, hi) = ScaleProblem(seed); + var x = (float[])lo.Clone(); + Assert.Equal(BoundedLeastSquaresStatus.Converged, Solve(m, n, a, b, lo, hi, x, out _)); + AssertKkt(m, n, a, b, lo, hi, x); + for (var j = 0; j < n; j++) sawActive |= x[j] == lo[j] || x[j] == hi[j]; + + foreach (var k in new[] { 1e-6f, 1e3f, 1e6f }) + { + // b and the bounds scaled: x scales with them. + var xb = Scaled(lo, k); + var status = Solve(m, n, a, Scaled(b, k), Scaled(lo, k), Scaled(hi, k), xb, out _); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"seed {seed} k {k}: {status}"); + for (var j = 0; j < n; j++) + Assert.True(Math.Abs(xb[j] - x[j] * k) <= 1e-3 * Math.Abs(k) * (1 + Math.Abs(x[j])), $"seed {seed} b*{k} col {j}: {xb[j]} vs {x[j] * k}"); + + // A scaled: x scales inversely, so the bounds stay put and only b is scaled with A to keep the same box. + var xa = (float[])lo.Clone(); + var lo2 = Scaled(lo, 1f / k); + var hi2 = Scaled(hi, 1f / k); + for (var j = 0; j < n; j++) xa[j] = lo2[j]; + status = Solve(m, n, Scaled(a, k), b, lo2, hi2, xa, out _); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"seed {seed} A*{k}: {status}"); + for (var j = 0; j < n; j++) + Assert.True(Math.Abs(xa[j] - x[j] / k) <= 1e-3 * Math.Abs(1 / k) * (1 + Math.Abs(x[j])), $"seed {seed} A*{k} col {j}: {xa[j]} vs {x[j] / k}"); + } + } + Assert.True(sawActive); + } + + [Fact] + public void NonFiniteOrInvertedInputIsRejectedAndLeavesXUntouched() + { + const int m = 5, n = 3; + var nan = float.NaN; + var inf = float.PositiveInfinity; + var cases = new (string name, Action corrupt)[] + { + ("A NaN", (a, b, lo, hi) => a[4] = nan), + ("A +Inf", (a, b, lo, hi) => a[7] = inf), + ("A -Inf", (a, b, lo, hi) => a[1] = -inf), + ("b NaN", (a, b, lo, hi) => b[2] = nan), + ("b +Inf", (a, b, lo, hi) => b[0] = inf), + ("lo NaN", (a, b, lo, hi) => lo[1] = nan), + ("hi NaN", (a, b, lo, hi) => hi[2] = nan), + ("lo > hi", (a, b, lo, hi) => { lo[1] = 0.5f; hi[1] = 0.25f; }), + ("lo = +Inf", (a, b, lo, hi) => { lo[0] = inf; hi[0] = inf; }), + ("hi = -Inf", (a, b, lo, hi) => { lo[0] = -inf; hi[0] = -inf; }), + }; + foreach (var (name, corrupt) in cases) + { + var a = (float[])A54.Clone(); + var b = (float[])B5.Clone(); + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + corrupt(a, b, lo, hi); + foreach (var start in new[] { new[] { 0.5f, -3f, 9f }, new[] { nan, 0f, 0f } }) + { + var x = (float[])start.Clone(); + var status = Solve(m, n, a, b, lo, hi, x, out var iterations); + Assert.True(status == BoundedLeastSquaresStatus.InvalidInput, $"{name}: {status}"); + Assert.Equal(0, iterations); + for (var j = 0; j < n; j++) + Assert.Equal(BitConverter.SingleToInt32Bits(start[j]), BitConverter.SingleToInt32Bits(x[j])); + } + } + } + + [Fact] + public void InfiniteBoundsMeanUnbounded() + { + const int m = 5, n = 3; + var inf = float.PositiveInfinity; + var x = new float[n]; + Assert.Equal(BoundedLeastSquaresStatus.Converged, Solve(m, n, A54, B5, Fill(-inf, n), Fill(inf, n), x, out _)); + var expected = NormalEquationSolution(m, n, A54, B5); + for (var j = 0; j < n; j++) Assert.Equal(expected[j], x[j], 1e-4); + + // One-sided, with an infinite warm start: the start is clamped to a finite point first. + var y = new[] { -inf, inf, float.NaN }; + var lo = new[] { -inf, 0.2f, -inf }; + var hi = new[] { inf, inf, 0.1f }; + Assert.Equal(BoundedLeastSquaresStatus.Converged, Solve(m, n, A54, B5, lo, hi, y, out _)); + AssertKkt(m, n, A54, B5, lo, hi, y); + Assert.All(y, v => Assert.True(float.IsFinite(v))); + } + + // Exact reference: every assignment of each column to {lo, hi, free}, free set solved in double. + private static double ExactBoxedCost(int m, int n, float[] a, float[] b, float[] lo, float[] hi, double pivotTol = 1e-9, double ridge = 0.0) + { + var maxDiag = 0.0; + for (var j = 0; j < n; j++) + { + var d = 0.0; + for (var r = 0; r < m; r++) d += (double)a[r * n + j] * a[r * n + j]; + maxDiag = Math.Max(maxDiag, d); + } + var best = double.MaxValue; + var total = 1; + for (var j = 0; j < n; j++) total *= 3; + var x = new double[n]; + for (var code = 0; code < total; code++) + { + var c = code; + var free = new System.Collections.Generic.List(); + for (var j = 0; j < n; j++) + { + var d = c % 3; c /= 3; + if (d == 0) x[j] = lo[j]; else if (d == 1) x[j] = hi[j]; else free.Add(j); + } + var k = free.Count; + if (k > 0) + { + var mat = new double[k, k + 1]; + for (var i = 0; i < k; i++) + { + for (var l = 0; l < k; l++) + for (var r = 0; r < m; r++) mat[i, l] += (double)a[r * n + free[i]] * a[r * n + free[l]]; + for (var r = 0; r < m; r++) + { + var fixedPart = 0.0; + for (var j = 0; j < n; j++) if (!free.Contains(j)) fixedPart += (double)a[r * n + j] * x[j]; + mat[i, k] += (double)a[r * n + free[i]] * (b[r] - fixedPart); + } + } + for (var i = 0; i < k; i++) mat[i, i] += ridge * maxDiag; + var ok = true; + for (var col = 0; col < k && ok; col++) + { + var piv = col; + for (var r = col + 1; r < k; r++) if (Math.Abs(mat[r, col]) > Math.Abs(mat[piv, col])) piv = r; + if (Math.Abs(mat[piv, col]) < pivotTol) { ok = false; break; } + for (var l = 0; l <= k; l++) (mat[col, l], mat[piv, l]) = (mat[piv, l], mat[col, l]); + for (var r = 0; r < k; r++) + { + if (r == col) continue; + var f = mat[r, col] / mat[col, col]; + for (var l = col; l <= k; l++) mat[r, l] -= f * mat[col, l]; + } + } + if (!ok) continue; + for (var i = 0; i < k; i++) + { + x[free[i]] = mat[i, k] / mat[i, i]; + if (x[free[i]] < lo[free[i]] - 1e-12 || x[free[i]] > hi[free[i]] + 1e-12) ok = false; + } + if (!ok) continue; + } + var cost = 0.0; + for (var r = 0; r < m; r++) + { + var s = -(double)b[r]; + for (var j = 0; j < n; j++) s += (double)a[r * n + j] * x[j]; + cost += s * s; + } + best = Math.Min(best, cost); + } + return best; + } + + // m < n with columns that are nearly parallel (condition number of A around 1e2), b outside the reachable set. + // The absolute scale of A must not matter: the KKT tolerance is relative, and the working iterate is double, so a + // 1e6 A with untouched bounds reaches the same optimum to the same allowance. + [Theory] + [InlineData(1f)] + [InlineData(1e6f)] + public void NearCollinearColumnsReachTheExactOptimum(float scale) + { + const int m = 2, n = 7; + for (var family = 0; family < 4; family++) + { + for (uint seed = 1; seed <= 120; seed++) + { + var rng = new CultMath.Random(seed * 15485863u + (uint)family * 977u); + var a = new float[m * n]; + for (var j = 0; j < n; j++) + { + switch (family) + { + case 0: a[j] = rng.NextFloat(0.5f, 1f); a[n + j] = 0.01f * rng.NextFloat(-1f, 1f); break; + case 1: a[j] = 1f + 0.02f * rng.NextFloat(-1f, 1f); a[n + j] = 0.01f * rng.NextFloat(-1f, 1f); break; + case 2: a[j] = rng.NextFloat(-1f, 1f); a[n + j] = 0.01f * a[j] + 0.01f * rng.NextFloat(-1f, 1f); break; + default: a[j] = rng.NextFloat(0.5f, 1f); a[n + j] = a[j] * 0.9f + 0.01f * rng.NextFloat(-1f, 1f); break; + } + } + for (var i = 0; i < a.Length; i++) a[i] *= scale; + var b = new[] { rng.NextFloat(-8f, 8f), rng.NextFloat(-8f, 8f) }; + var lo = new float[n]; + var hi = new float[n]; + for (var j = 0; j < n; j++) { lo[j] = rng.NextFloat() < 0.5f ? -1f : 0f; hi[j] = 1f; } + var x = new float[n]; + var status = Solve(m, n, a, b, lo, hi, x, out _); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"family {family} seed {seed}: {status}"); + var gap = Cost(m, n, a, b, x) - ExactBoxedCost(m, n, a, b, lo, hi); + var allowed = 1e-5 * (b[0] * b[0] + b[1] * b[1]); + Assert.True(gap <= allowed, $"family {family} seed {seed}: cost {gap} above the exact optimum (allowed {allowed})"); + } + } + } + + // A = U diag(s) V^T with s geometric from 1 down to 1/cond, so cond(A) is the requested value (float rounding of A + // perturbs it slightly). m x n with m >= n; b is generic, so it has a real residual when m > n. + private static (float[] a, float[] b) ConditionedProblem(int m, int n, double cond, uint seed, double aScale = 1.0, bool bAlongSmallestSingularVector = false) + { + var rng = new CultMath.Random(seed * 2654435761u + 17u); + double[][] Orthonormal(int dim, int count) + { + var q = new double[count][]; + for (var k = 0; k < count; k++) + { + var v = new double[dim]; + for (var i = 0; i < dim; i++) v[i] = rng.NextFloat(-1f, 1f); + for (var pass = 0; pass < 2; pass++) + for (var l = 0; l < k; l++) + { + var dot = 0.0; + for (var i = 0; i < dim; i++) dot += v[i] * q[l][i]; + for (var i = 0; i < dim; i++) v[i] -= dot * q[l][i]; + } + var norm = 0.0; + for (var i = 0; i < dim; i++) norm += v[i] * v[i]; + norm = Math.Sqrt(norm); + for (var i = 0; i < dim; i++) v[i] /= norm; + q[k] = v; + } + return q; + } + var u = Orthonormal(m, n); + var w = Orthonormal(n, n); + var a = new float[m * n]; + for (var k = 0; k < n; k++) + { + var s = aScale * Math.Pow(cond, -(double)k / Math.Max(1, n - 1)); + for (var r = 0; r < m; r++) + for (var c = 0; c < n; c++) + a[r * n + c] += (float)(u[k][r] * s * w[k][c]); + } + var b = new float[m]; + for (var i = 0; i < m; i++) b[i] = bAlongSmallestSingularVector ? (float)u[n - 1][i] : rng.NextFloat(-3f, 3f); + return (a, b); + } + + // A pass can end with the KKT test still failing and the cost exactly where it was: a dependent free column keeps a + // gradient above the tolerance, or a bound release lands on a column the factor drops. The solver must call that + // converged; if the stall rule mis-reads the cost scale or the progress, it spins to the iteration cap instead. + // Each kind is a fixed-seed batch of small boxed problems with warm starts on and off the bounds and pinned columns: + // rank-deficient (duplicate and zero columns), nearly collinear (1e-2 to 1e-5), integer-tied, wide, cond(A) 1e5 to + // 3e5, and "weak", where b lies along the weakest singular direction of a cond(A) 1e6 to 1e7 matrix so the gradient + // tolerance sits below the rounding noise with interior columns nonzero. Every kind must converge; all but "weak" + // (past the documented condition range) must also land within the documented cost gap, relative to |b|^2, of an + // exact enumeration of the active sets (a 1e-15 ridge keeps singular free sets solvable). + [Theory] + [InlineData("plain", 300)] + [InlineData("rankdef", 300)] + [InlineData("collinear", 300)] + [InlineData("integer-tie", 300)] + [InlineData("wide", 300)] + [InlineData("cond", 300)] + [InlineData("weak", 6000)] + public void StalledPassesConvergeInsteadOfSpinningToTheCap(string kind, int seeds) + { + var failures = new System.Collections.Generic.List(); + for (uint seed = 1; seed <= seeds; seed++) + { + var rng = new CultMath.Random(seed * 2246822519u + (uint)kind.Length * 3266489917u); + double Gauss() => Math.Sqrt(-2.0 * Math.Log(1.0 - rng.NextFloat() * 0.999999)) * Math.Cos(2.0 * Math.PI * rng.NextFloat()); + var n = rng.NextInt(2, 8); + var m = rng.NextInt(2, 12); + if (kind == "wide") m = Math.Max(1, n - rng.NextInt(1, 4)); + if (kind == "cond" || kind == "weak") m = n + rng.NextInt(0, 5); + float[] a; + float[] bWeak = null; + if (kind == "weak") + { + (a, bWeak) = ConditionedProblem(m, n, Math.Pow(10.0, 6.0 + rng.NextFloat()), seed, 1.0, true); + } + else if (kind == "cond") + { + a = ConditionedProblem(m, n, Math.Pow(10.0, 5.0 + 0.5 * rng.NextFloat()), seed).a; + } + else + { + a = new float[m * n]; + for (var i = 0; i < m; i++) + { + var row = new double[n]; + for (var j = 0; j < n; j++) row[j] = kind == "integer-tie" ? rng.NextInt(-2, 3) : Gauss(); + for (var j = 1; j < n; j++) + { + if (kind == "rankdef" && j % 3 == 1) row[j] = row[j - 1]; + if (kind == "rankdef" && j % 3 == 2) row[j] = 0.0; + if (kind == "collinear" && j % 2 == 1) row[j] = row[j - 1] + Math.Pow(10.0, -rng.NextInt(2, 6)) * Gauss(); + } + for (var j = 0; j < n; j++) a[i * n + j] = (float)row[j]; + } + } + var lo = new float[n]; + var hi = new float[n]; + var x = new float[n]; + var xt = new double[n]; + for (var j = 0; j < n; j++) + { + var w = Math.Pow(10.0, rng.NextInt(-2, 2)); + lo[j] = (float)(-w * rng.NextFloat()); + hi[j] = (float)(w * rng.NextFloat()); + if (rng.NextInt(6) == 0) hi[j] = lo[j]; + else if (rng.NextInt(8) == 0) lo[j] = hi[j] = 0f; + else if (rng.NextInt(8) == 0) { lo[j] = (float)(0.5 + rng.NextFloat()); hi[j] = lo[j] + 0.5f; } + if (kind == "weak") { var c = (float)Math.Pow(10.0, rng.NextInt(3, 9)); lo[j] = rng.NextInt(2) == 0 ? -c : 0f; hi[j] = c; } + if (kind == "integer-tie") { lo[j] = -1f; hi[j] = rng.NextInt(4) == 0 ? -1f : 1f; } + x[j] = rng.NextInt(3) == 0 ? lo[j] : rng.NextInt(2) == 0 ? hi[j] : lo[j] + (hi[j] - lo[j]) * rng.NextFloat(); + xt[j] = (lo[j] + hi[j]) / 2.0 + (hi[j] - lo[j]) * 3.0 * (rng.NextFloat() - 0.5); + } + var noise = Math.Pow(10.0, rng.NextInt(-2, 4)); + var mode = rng.NextInt(4); + var b = new float[m]; + for (var i = 0; i < m; i++) + { + var s = 0.0; + for (var j = 0; j < n; j++) s += (double)a[i * n + j] * xt[j]; + b[i] = bWeak != null ? bWeak[i] : mode == 0 ? 0f : (float)(s + noise * Gauss() * (mode == 3 ? 1.0 : 0.01)); + } + var status = Solve(m, n, a, b, lo, hi, x, out var iterations); + if (status != BoundedLeastSquaresStatus.Converged) + { + failures.Add($"seed {seed}: {status} after {iterations}"); + continue; + } + if (kind == "weak") continue; + var bb = 0.0; + foreach (var v in b) bb += (double)v * v; + if (bb == 0.0) bb = 1.0; + var gap = Cost(m, n, a, b, x) - ExactBoxedCost(m, n, a, b, lo, hi, 1e-300, 1e-15); + if (gap > 1e-5 * bb) failures.Add($"seed {seed}: cost gap {gap / bb:E2} of |b|^2"); + } + Assert.True(failures.Count == 0, $"{kind}: {failures.Count} of {seeds} failed; first: {string.Join("; ", failures.GetRange(0, Math.Min(3, failures.Count)))}"); + } + + // Unsaturated allocator-shaped ticks: 24 columns, every column free at the optimum. The solve is one Newton step; + // at most one more may confirm it. Before the working iterate moved to double, the float gradient's noise floor + // exceeded the tolerance from cond ~1e3 and the solver spun to the 100-iteration cap on optimal points. + [Theory] + [InlineData(1e2)] + [InlineData(1e3)] + [InlineData(1e4)] + public void UnsaturatedIllConditionedProblemsConvergeInAtMostTwoIterations(double cond) + { + const int m = 28, n = 24; + for (uint seed = 1; seed <= 30; seed++) + { + var (a, b) = ConditionedProblem(m, n, cond, seed); + var x = new float[n]; + var status = Solve(m, n, a, b, Fill(-1e9f, n), Fill(1e9f, n), x, out var iterations); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"cond {cond} seed {seed}: {status} after {iterations}"); + Assert.True(iterations <= 2, $"cond {cond} seed {seed}: {iterations} iterations"); + var bb = 0.0; + foreach (var v in b) bb += (double)v * v; + var expected = NormalEquationSolution(m, n, a, b); + var xs = new float[n]; + for (var j = 0; j < n; j++) xs[j] = (float)expected[j]; + Assert.True(Cost(m, n, a, b, x) - Cost(m, n, a, b, xs) <= 1e-6 * bb, $"cond {cond} seed {seed}: cost above the exact optimum"); + } + } + + // Near the pivot cutoff (cond(A) ~ 3e7, cond(A^T A) ~ 1e15) a Newton step leaves a free gradient above the + // tolerance on some problems. Those take one confirming iteration and stop because it lowers nothing; the rest + // are stationary at once. Never the cap, and never spinning. + [Fact] + public void NearSingularFreeSetsConfirmOnceAndStop() + { + const int m = 28, n = 24; + var confirmed = 0; + for (uint seed = 1; seed <= 60; seed++) + { + var (a, b) = ConditionedProblem(m, n, 3e7, seed); + var x = new float[n]; + var status = Solve(m, n, a, b, Fill(-1e9f, n), Fill(1e9f, n), x, out var iterations); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"seed {seed}: {status}"); + Assert.True(iterations <= 3, $"seed {seed}: {iterations} iterations"); + if (iterations > 0) confirmed++; + } + Assert.InRange(confirmed, 1, 20); + } + + // b along the smallest left singular vector makes the optimum lie along the weakest direction: |A^T b| is tiny, so + // the tolerance (1e-6 of it) sits below the rounding noise of the gradient (~1e-16 |A^T A| |x|) for cond(A) >= 1e6. + // The tolerance is then unreachable, and the solver must stop on the progress rule, after the handful of Newton + // passes that still lower the cost, never at the 100-iteration cap. + [Theory] + [InlineData(1e6)] + [InlineData(1e7)] + public void ToleranceBelowTheNoiseFloorStopsOnProgressNotOnTheCap(double cond) + { + const int m = 28, n = 24; + var confirmed = 0; + for (uint seed = 1; seed <= 30; seed++) + { + var (a, b) = ConditionedProblem(m, n, cond, seed, 1.0, bAlongSmallestSingularVector: true); + var x = new float[n]; + var status = Solve(m, n, a, b, Fill(-1e9f, n), Fill(1e9f, n), x, out var iterations); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"cond {cond} seed {seed}: {status} after {iterations}"); + Assert.True(iterations <= 6, $"cond {cond} seed {seed}: {iterations} iterations"); + if (iterations >= 1) confirmed++; + } + Assert.True(confirmed >= 15, $"cond {cond}: only {confirmed} of 30 reached the tolerance floor"); + } + + // Scale-freeness was only claimed for scaling A, b and the bounds together; A alone at 1e6 with the bounds left + // alone must also converge, and as fast. + [Theory] + [InlineData(1e2)] + [InlineData(1e4)] + public void LargeAWithUnscaledBoundsConvergesInAtMostTwoIterations(double cond) + { + const int m = 28, n = 24; + for (uint seed = 1; seed <= 30; seed++) + { + var (a, b) = ConditionedProblem(m, n, cond, seed, 1e6); + var x = new float[n]; + var status = Solve(m, n, a, b, Fill(-1e3f, n), Fill(1e3f, n), x, out var iterations); + Assert.True(status == BoundedLeastSquaresStatus.Converged, $"cond {cond} seed {seed}: {status} after {iterations}"); + Assert.True(iterations <= 2, $"cond {cond} seed {seed}: {iterations} iterations"); + } + } + + // b = 0 must not borrow its tolerance from the start point: a far start's own gradient is enormous, so that + // scale judged it converged after one iteration at cost 5001. The optimum here is ~1e-9. + [Fact] + public void ZeroTargetFromAFarStartIsNotJudgedConvergedByItsOwnSteepness() + { + var a = new[] { 1f, 1f, 1f, 1.0001f }; + var b = new float[2]; + var lo = new[] { 0.5f, -1e6f }; + var hi = new[] { 1e6f, 1e6f }; + var x = new[] { 1e6f, 1e6f }; + var status = Solve(2, 2, a, b, lo, hi, x, out _); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + AssertKkt(2, 2, a, b, lo, hi, x); + Assert.True(Cost(2, 2, a, b, x) < 1e-6, $"cost {Cost(2, 2, a, b, x)}"); + Assert.True(x[0] >= 0.5f); + + // The origin inside the box: the answer is the origin, from any start. + var lo2 = new[] { -1e6f, -1e6f }; + var y = new[] { 1e6f, -1e6f }; + Assert.Equal(BoundedLeastSquaresStatus.Converged, Solve(2, 2, a, b, lo2, hi, y, out _)); + Assert.True(Cost(2, 2, a, b, y) < 1e-9, $"cost {Cost(2, 2, a, b, y)}"); + } + + // The tolerance scale is the gradient at the box point nearest the origin. Here that point is 1000 and the + // wanted release is a gradient of about 6e-5 against a 1000-scale problem: only a scale taken from the box point + // (about 6e-5, tolerance 6e-11) releases it; a scale taken from A^T b (about 1000, tolerance 1e-3) would not. + // Run with the excluded column in each position so the scale must come from the right entry. + [Fact] + public void ToleranceScaleIsTheGradientAtTheBoxPointNearestTheOrigin() + { + var a = new[] { 1f, 0f, 0f, 1f }; + foreach (var excluded in new[] { 0, 1 }) + { + var b = new float[2]; + var lo = new[] { -1e6f, -1e6f }; + var hi = new[] { 1e6f, 1e6f }; + b[excluded] = 1000.0001f; + lo[excluded] = 1000f; + hi[excluded] = 2000f; + var x = new float[2]; + x[excluded] = 1000f; + var status = Solve(2, 2, a, b, lo, hi, x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.True(iterations >= 1, $"column {excluded}: no release"); + Assert.Equal(b[excluded], x[excluded]); + Assert.Equal(0f, x[1 - excluded]); + } + } + + // b = 0 with the origin in the box and the start at the origin: the gradient is exactly zero and so is the + // tolerance. Nothing is violated, so the solve is over without an iteration. + [Fact] + public void ZeroTargetAtTheOriginTakesNoIterations() + { + const int m = 5, n = 3; + var x = new float[n]; + var status = Solve(m, n, A54, new float[m], Fill(-1f, n), Fill(1f, n), x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(0, iterations); + Assert.Equal(new float[n], x); + } + + // One column, A = [1], with b beyond the box: the first Newton step would cross a bound, so with a cap of 0 the + // solver returns the clamped start untouched. That makes the clamp itself the thing under test, for every + // combination of a finite, infinite or NaN start with finite and infinite bounds. + [Fact] + public void WarmStartClampIsExact() + { + var inf = float.PositiveInfinity; + var cases = new (float lo, float hi, float start, float expected)[] + { + (-1f, 1f, 0.25f, 0.25f), + (-1f, 1f, 5f, 1f), + (-1f, 1f, -5f, -1f), + (-1f, 1f, float.NaN, -1f), + (0.5f, 2f, inf, 2f), + (0.5f, 2f, -inf, 0.5f), + (0.2f, inf, inf, 0.2f), + (0.2f, inf, -inf, 0.2f), + (0.2f, inf, float.NaN, 0.2f), + (-inf, -2f, -inf, -2f), + (-inf, -2f, 3f, -2f), + (-inf, 3f, float.NaN, 0f), + (-inf, 3f, -inf, 0f), + (-2f, inf, float.NaN, -2f), + (-inf, -2f, float.NaN, -2f), + (-inf, 3f, 1f, 1f), + }; + foreach (var (l, h, start, expected) in cases) + { + // b beyond the box on the side the start is not on the bound of, so the step crosses a bound (or the + // start already sits on one). + var b = new[] { float.IsInfinity(h) ? l - 10f : h + 10f }; + if (float.IsInfinity(l) && float.IsInfinity(h)) b[0] = 0f; + var x = new[] { start }; + Solve(1, 1, new[] { 1f }, b, new[] { l }, new[] { h }, x, out var iterations, maxIterations: 0); + Assert.Equal(0, iterations); + Assert.True(x[0] == expected, $"box [{l}, {h}] start {start}: {x[0]}, expected {expected}"); + } + } + + // A duplicate column that is free and interior gets no step: the columns share one gradient, and moving both + // along it would overshoot. + [Fact] + public void FreeDuplicateColumnGetsNoStep() + { + const int m = 5, n = 3; + var a = (float[])A54.Clone(); + for (var r = 0; r < m; r++) a[r * n + 2] = a[r * n + 0]; + var x = new float[n]; + var status = Solve(m, n, a, B5, Fill(-100f, n), Fill(100f, n), x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.Equal(0, iterations); + Assert.Equal(0f, x[2]); + var reduced = new float[m * 2]; + for (var r = 0; r < m; r++) { reduced[r * 2] = a[r * n]; reduced[r * 2 + 1] = a[r * n + 1]; } + var xr = NormalEquationSolution(m, 2, reduced, B5); + Assert.Equal(xr[0], x[0], 1e-4); + Assert.Equal(xr[1], x[1], 1e-4); + } + + // Columns 1000x apart in scale and exactly parallel: the pivot tolerance is relative to the column's own + // diagonal, so it catches the duplicate at any absolute scale. + [Theory] + [InlineData(1e3f)] + [InlineData(1e-3f)] + public void ParallelColumnsAreDependentAtAnyAbsoluteScale(float scale) + { + const int m = 5, n = 3; + var a = (float[])A54.Clone(); + for (var r = 0; r < m; r++) a[r * n + 2] = 0.37f * a[r * n + 0]; + for (var i = 0; i < a.Length; i++) a[i] *= scale; + var lo = Fill(-1000f / scale, n); + var hi = Fill(1000f / scale, n); + var x = new float[n]; + var status = Solve(m, n, a, B5, lo, hi, x, out var iterations); + Assert.Equal(BoundedLeastSquaresStatus.Converged, status); + Assert.True(iterations <= 2, $"{iterations} iterations"); + var reduced = new float[m * 2]; + for (var r = 0; r < m; r++) { reduced[r * 2] = a[r * n]; reduced[r * 2 + 1] = a[r * n + 1]; } + var xr = NormalEquationSolution(m, 2, reduced, B5); + var xs = new float[n]; + xs[0] = (float)xr[0]; + xs[1] = (float)xr[1]; + var bb = 0.0; + foreach (var v in B5) bb += (double)v * v; + Assert.True(Cost(m, n, a, B5, x) - Cost(m, n, a, B5, xs) <= 1e-6 * bb, "cost above the reduced optimum"); + } + + // No path allocates: not the first InvalidInput, not a cap hit, not a solve that releases bounds. Each path is + // measured on its first call and again after, so a one-time runtime cost shows up as a difference between the two + // rather than as a solver allocation. + [Fact] + public void EveryPathAllocatesNothing() + { + const int m = 4, n = 3; + var a = new[] { 1.0f, 0.2f, 0.1f, 0.1f, 1.0f, 0.3f, -0.2f, 0.2f, 1.0f, 0.5f, 0.5f, 0.4f }; + var b = new[] { 10f, -10f, 10f, 1f }; + var lo = Fill(-1f, n); + var hi = Fill(1f, n); + var workspace = new float[BoundedLeastSquares.WorkspaceLength(n)]; + var x = new float[n]; + var bad = (float[])a.Clone(); + bad[4] = float.NaN; + + var paths = new (string name, float[] a, int cap, BoundedLeastSquaresStatus expected)[] + { + ("invalid", bad, 100, BoundedLeastSquaresStatus.InvalidInput), + ("converged", a, 100, BoundedLeastSquaresStatus.Converged), + ("limit", a, 1, BoundedLeastSquaresStatus.IterationLimit), + }; + foreach (var (name, matrix, cap, expected) in paths) + { + var bytes = new long[2]; + for (var call = 0; call < 2; call++) + { + Array.Clear(x); + var before = GC.GetAllocatedBytesForCurrentThread(); + var status = BoundedLeastSquares.Solve(m, n, matrix, b, lo, hi, x, workspace, out _, cap); + bytes[call] = GC.GetAllocatedBytesForCurrentThread() - before; + Assert.Equal(expected, status); + } + Assert.True(bytes[0] == 0 && bytes[1] == 0, $"{name}: first call {bytes[0]} B, second {bytes[1]} B"); + } + } +} diff --git a/packages/cultmath/tests/CultMath.Tests/HlslSemanticsTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslSemanticsTests.cs index 954c4375..c6bac266 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslSemanticsTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslSemanticsTests.cs @@ -444,11 +444,11 @@ public sealed class HlslSemanticsTests } [Theory] - [InlineData(0.0f, 0.0f, 0.0f, -0.41219875f)] - [InlineData(0.25f, -0.5f, 1.75f, 0.5250269f)] - [InlineData(12.25f, -4.5f, 3.125f, 0.008141451f)] - [InlineData(-7.3f, 2.9f, 101.4f, 0.578015f)] - public void Snoise3MatchesUnityMathematicsReference(float x, float y, float z, float expected) + [InlineData(0.0f, 0.0f, 0.0f, -0.435872972f)] + [InlineData(0.25f, -0.5f, 1.75f, 0.0492948629f)] + [InlineData(12.25f, -4.5f, 3.125f, 0.443114728f)] + [InlineData(-7.3f, 2.9f, 101.4f, -0.10221792f)] + public void Snoise3MatchesGoldenValues(float x, float y, float z, float expected) { Assert.Equal(expected, math.snoise(new float3(x, y, z)), precision: 5); } diff --git a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs index fb269ee2..be8a3df4 100644 --- a/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/HlslSourceCompatibilityTests.cs @@ -27,6 +27,9 @@ public sealed class HlslSourceCompatibilityTests Assert.NotNull(ShaderFunction(shader, "cultmath_value_noise_bicubic", typeof(float2))); Assert.NotNull(ShaderFunction(shader, "cultmath_smin_grad", typeof(float4), typeof(float4), typeof(float))); Assert.NotNull(ShaderFunction(shader, "cultmath_cellular", typeof(float3))); + Assert.NotNull(ShaderFunction(shader, "cultmath_snoise_grad", typeof(float3))); + Assert.NotNull(ShaderFunction(shader, "cultmath_fbm_grad", typeof(float3), typeof(int), typeof(float), typeof(float))); + Assert.NotNull(ShaderFunction(shader, "cultmath_ridged_grad", typeof(float3), typeof(int), typeof(float), typeof(float))); } // HLSL functions carry no access modifier, so they compile as private instance methods. @@ -127,7 +130,7 @@ public sealed class HlslSourceCompatibilityTests } Assert.True(mismatches.Count == 0, $"{mismatches.Count} mismatches:{Environment.NewLine}{string.Join(Environment.NewLine, mismatches.Take(12))}"); - Assert.Equal(30, compared.Count); + Assert.Equal(33, compared.Count); } /// diff --git a/packages/cultmath/tests/CultMath.Tests/MathTests.cs b/packages/cultmath/tests/CultMath.Tests/MathTests.cs index b47a389e..3dc374bf 100644 --- a/packages/cultmath/tests/CultMath.Tests/MathTests.cs +++ b/packages/cultmath/tests/CultMath.Tests/MathTests.cs @@ -307,10 +307,10 @@ public sealed class MathTests [Theory] [InlineData(0.0f, 0.0f, 0.0f)] - [InlineData(0.25f, -0.5f, -0.425866783f)] - [InlineData(12.25f, -4.5f, 0.258312f)] - [InlineData(536.5106f, 536.5106f, -0.6901103f)] - public void SimplexNoiseMatchesUnityMathematics(float x, float y, float expected) + [InlineData(0.25f, -0.5f, 0.225074813f)] + [InlineData(12.25f, -4.5f, -0.388599217f)] + [InlineData(536.5106f, 536.5106f, 0.538390875f)] + public void SimplexNoiseMatchesGoldenValues(float x, float y, float expected) { Assert.Equal(expected, math.snoise(new float2(x, y)), precision: 6); } diff --git a/packages/cultmath/tests/CultMath.Tests/NoiseGradTests.cs b/packages/cultmath/tests/CultMath.Tests/NoiseGradTests.cs new file mode 100644 index 00000000..77ec4819 --- /dev/null +++ b/packages/cultmath/tests/CultMath.Tests/NoiseGradTests.cs @@ -0,0 +1,380 @@ +using System; +using CultMath; +using Xunit; + +namespace CultMath.Tests; + +public sealed class NoiseGradTests +{ + // ---- Invariant-8 property harness (shared by snoise_grad, fbm_grad, ridged_grad) ---- + // + // Each test compares a function's analytic gradient with central differences of its own .w at + // random points in [-3, 3]^3. eps trades curvature error (eps too large) against float32 + // rounding noise (eps too small). snoise is continuous across simplex-cell boundaries (the + // continuity test below), so no point needs excluding for a seam; ridged_grad excludes a band + // around its own deliberate crease. Each tolerance is at least 5x the largest error the harness + // itself produces at its seed. + private const float PropertyEps = 0.0001f; + private const float SnoiseGradTolerance = 0.03f; + private const float FbmGradTolerance = 0.05f; + private const float RidgedGradTolerance = 0.05f; + + [Fact] + public void SnoiseGradGradientMatchesCentralDifferencesOfItsOwnValue() + { + var random = new System.Random(0xA11CE); + var tested = 0; + var attempts = 0; + while (tested < 200 && attempts < 20000) + { + attempts++; + var p = RandomPoint(random); + var g = math.snoise_grad(p); + float F(float3 q) => math.snoise_grad(q).w; + var fine = CentralDifference(F, p, PropertyEps); + + tested++; + AssertGradientWithinTolerance(fine, g, SnoiseGradTolerance); + } + + Assert.True(tested >= 150, $"only {tested} of {attempts} points were usable"); + } + + [Fact] + public void FbmGradGradientMatchesCentralDifferencesOfItsOwnValue() + { + const int octaves = 4; + const float lacunarity = 2.0f; + const float gain = 0.5f; + var random = new System.Random(0xA11CF); + var tested = 0; + var attempts = 0; + while (tested < 200 && attempts < 40000) + { + attempts++; + var p = RandomPoint(random); + var g = math.fbm_grad(p, octaves, lacunarity, gain); + float F(float3 q) => math.fbm_grad(q, octaves, lacunarity, gain).w; + var fine = CentralDifference(F, p, PropertyEps); + + tested++; + AssertGradientWithinTolerance(fine, g, FbmGradTolerance); + } + + Assert.True(tested >= 150, $"only {tested} of {attempts} points were usable"); + } + + [Fact] + public void RidgedGradGradientMatchesCentralDifferencesExcludingAnyOctavesZeroBand() + { + // ridged_grad's own crease (any octave's n_i = 0) is a real, deliberate discontinuity + // (design.md, "the operator's dune term"), not a numerical artifact: exclude a band around it + // for every octave, the same way cellular excludes its F1 = F2 seam. + const float zeroBand = 0.1f; + const int octaves = 3; + const float lacunarity = 2.0f; + const float gain = 0.5f; + var random = new System.Random(0xA11D0); + var tested = 0; + var attempts = 0; + while (tested < 200 && attempts < 80000) + { + attempts++; + var p = RandomPoint(random); + if (AnyOctaveNearZero(p, octaves, lacunarity, zeroBand)) + continue; + + var g = math.ridged_grad(p, octaves, lacunarity, gain); + float F(float3 q) => math.ridged_grad(q, octaves, lacunarity, gain).w; + var fine = CentralDifference(F, p, PropertyEps); + + tested++; + AssertGradientWithinTolerance(fine, g, RidgedGradTolerance); + } + + Assert.True(tested >= 150, $"only {tested} of {attempts} points were outside the zero band"); + } + + private static bool AnyOctaveNearZero(float3 p, int octaves, float lacunarity, float band) + { + var frequency = 1.0f; + for (var i = 0; i < octaves; i++) + { + if (MathF.Abs(math.snoise_grad(p * frequency).w) < band) + return true; + frequency *= lacunarity; + } + + return false; + } + + private static float3 RandomPoint(System.Random random) => new( + random.NextSingle() * 6.0f - 3.0f, random.NextSingle() * 6.0f - 3.0f, random.NextSingle() * 6.0f - 3.0f); + + private static float3 CentralDifference(Func f, float3 p, float eps) => new( + (f(p + new float3(eps, 0.0f, 0.0f)) - f(p - new float3(eps, 0.0f, 0.0f))) / (2.0f * eps), + (f(p + new float3(0.0f, eps, 0.0f)) - f(p - new float3(0.0f, eps, 0.0f))) / (2.0f * eps), + (f(p + new float3(0.0f, 0.0f, eps)) - f(p - new float3(0.0f, 0.0f, eps))) / (2.0f * eps)); + + private static void AssertGradientWithinTolerance(float3 numeric, float4 analytic, float tolerance) + { + AssertWithinTolerance(numeric.x, analytic.x, tolerance); + AssertWithinTolerance(numeric.y, analytic.y, tolerance); + AssertWithinTolerance(numeric.z, analytic.z, tolerance); + } + + private static void AssertWithinTolerance(float expected, float actual, float tolerance) => + Assert.True(MathF.Abs(expected - actual) <= tolerance, $"expected {expected}, actual {actual}, tolerance {tolerance}"); + + // ---- snoise is continuous across simplex-cell boundaries ---- + + // The simplex corner set (i, i1, i2) summed by snoise(float3), derived the same way snoise does. + private static (float3 i, float3 i1, float3 i2) SimplexCell(float3 p) + { + var i = math.floor(p + math.dot(p, new float3(1.0f / 3.0f, 1.0f / 3.0f, 1.0f / 3.0f))); + var x0 = p - i + math.dot(i, new float3(1.0f / 6.0f, 1.0f / 6.0f, 1.0f / 6.0f)); + var g = math.step(new float3(x0.y, x0.z, x0.x), x0); + var l = 1.0f - g; + var lzxy = new float3(l.z, l.x, l.y); + return (i, math.min(g, lzxy), math.max(g, lzxy)); + } + + [Fact] + public void SnoiseAndItsGradientAreContinuousAcrossSimplexCellBoundaries() + { + // March random lines in steps of 1e-3 until the corner set changes, bisect the crossing to + // float precision, and compare the two sides. With a kernel radius^2 above 0.5 a lattice + // vertex outside the four summed corners can lie inside the kernel, which leaves a value + // jump at the boundary (stegu/webgl-noise fixed it in 21d9fe23d7); at 0.5 the jump is float + // noise. A value jump below 1e-5 and a gradient jump below 1e-4 is float noise on both sides. + const int lines = 400; + const int maxSteps = 20000; + const float dt = 1.0e-3f; + var random = new System.Random(0x5EA4); + var crossings = 0; + var maxValueJump = 0.0f; + var maxGradientJump = 0.0f; + for (var line = 0; line < lines; line++) + { + var origin = RandomPoint(random); + var direction = math.normalize(new float3( + random.NextSingle() - 0.5f, random.NextSingle() - 0.5f, random.NextSingle() - 0.5f)); + var start = SimplexCell(origin); + var t = 0.0f; + var found = false; + for (var step = 0; step < maxSteps && !found; step++) + { + t += dt; + found = !SimplexCell(origin + direction * t).Equals(start); + } + + Assert.True(found, $"line {line} never left its simplex cell in {maxSteps} steps"); + + double lo = t - dt; + double hi = t; + for (var bisect = 0; bisect < 60; bisect++) + { + var mid = 0.5 * (lo + hi); + if (SimplexCell(origin + direction * (float)mid).Equals(start)) lo = mid; else hi = mid; + } + + var a = math.snoise_grad(origin + direction * (float)lo); + var b = math.snoise_grad(origin + direction * (float)hi); + crossings++; + maxValueJump = MathF.Max(maxValueJump, MathF.Abs(a.w - b.w)); + maxGradientJump = MathF.Max(maxGradientJump, MathF.Max(MathF.Abs(a.x - b.x), MathF.Max(MathF.Abs(a.y - b.y), MathF.Abs(a.z - b.z)))); + } + + Assert.Equal(lines, crossings); + Assert.True(maxValueJump < 1.0e-5f, $"value jump {maxValueJump} across a simplex boundary"); + Assert.True(maxGradientJump < 1.0e-4f, $"gradient jump {maxGradientJump} across a simplex boundary"); + } + + // ---- snoise_grad.w vs snoise ---- + + [Fact] + public void SnoiseGradValueIsBitEqualToSnoise() + { + // Both compute 105 * dot(m4, px) via the exact same operation order (math.cs comment on + // snoise_grad), so this asserts bit equality over 5,000 random points in [-100, 100]^3. + var random = new System.Random(0x50F2); + for (var i = 0; i < 5000; i++) + { + var p = new float3( + random.NextSingle() * 200.0f - 100.0f, random.NextSingle() * 200.0f - 100.0f, random.NextSingle() * 200.0f - 100.0f); + var expected = math.snoise(p); + var actual = math.snoise_grad(p).w; + Assert.Equal(BitConverter.SingleToInt32Bits(expected), BitConverter.SingleToInt32Bits(actual)); + } + } + + // ---- fbm_grad / ridged_grad: per-octave gain and lacunarity ---- + + [Fact] + public void FbmGradWithOneOctaveEqualsSnoiseGrad() + { + var random = new System.Random(0xF6B10); + for (var i = 0; i < 64; i++) + { + var p = RandomPoint(random); + var lacunarity = 1.0f + random.NextSingle() * 3.0f; + var gain = 0.1f + random.NextSingle() * 0.8f; + + var expected = math.snoise_grad(p); + var actual = math.fbm_grad(p, 1, lacunarity, gain); + + Assert.Equal(expected.x, actual.x); + Assert.Equal(expected.y, actual.y); + Assert.Equal(expected.z, actual.z); + Assert.Equal(expected.w, actual.w); + } + } + + [Fact] + public void FbmGradCompoundsGainAndLacunarityPerOctaveNotOnce() + { + // Falsifies two mutant shapes at once: applying gain/lacunarity once for the whole sum + // instead of compounding per octave, and swapping which argument drives amplitude versus + // frequency. lacunarity != gain and both != 1 so a swap changes the result, and 3 octaves + // gives amplitude/frequency three distinct compounded values (1, gain, gain^2 and + // 1, lacunarity, lacunarity^2) so "applied once" mutants cannot coincidentally match. + // The expected value is built by the same incremental multiply chain fbm_grad itself uses + // (frequency *= lacunarity, amplitude *= gain, one step at a time, not lacunarity*lacunarity + // or MathF.Pow), so this asserts bit-exact equality rather than needing a tolerance. + const float lacunarity = 2.5f; + const float gain = 0.6f; + var random = new System.Random(0xF6B11); + for (var i = 0; i < 32; i++) + { + var p = RandomPoint(random); + + var frequency = 1.0f; + var amplitude = 1.0f; + var n0 = math.snoise_grad(p * frequency); + var expectedValue = amplitude * n0.w; + var expectedGradient = amplitude * frequency * new float3(n0.x, n0.y, n0.z); + frequency *= lacunarity; + amplitude *= gain; + + var n1 = math.snoise_grad(p * frequency); + expectedValue += amplitude * n1.w; + expectedGradient += amplitude * frequency * new float3(n1.x, n1.y, n1.z); + frequency *= lacunarity; + amplitude *= gain; + + var n2 = math.snoise_grad(p * frequency); + expectedValue += amplitude * n2.w; + expectedGradient += amplitude * frequency * new float3(n2.x, n2.y, n2.z); + + var actual = math.fbm_grad(p, 3, lacunarity, gain); + + Assert.Equal(expectedValue, actual.w); + Assert.Equal(expectedGradient.x, actual.x); + Assert.Equal(expectedGradient.y, actual.y); + Assert.Equal(expectedGradient.z, actual.z); + } + } + + [Fact] + public void RidgedGradCompoundsGainAndLacunarityPerOctaveNotOnce() + { + // Same construction as FbmGradCompoundsGainAndLacunarityPerOctaveNotOnce, folded: value is + // Sum a_i*(1-|n_i|), gradient is -Sum a_i*f_i*sign(n_i)*grad(n_i) (math.cs comment on + // ridged_grad). p is offset away from the origin and re-tried on a near-zero octave so no + // octave's n_i lands in the fold's own zero band, which would make the closed form's sign() + // ambiguous between runs. + const float lacunarity = 2.5f; + const float gain = 0.6f; + var random = new System.Random(0xF6B12); + var tested = 0; + var attempts = 0; + while (tested < 32 && attempts < 4000) + { + attempts++; + var p = RandomPoint(random); + if (AnyOctaveNearZero(p, 3, lacunarity, 0.1f)) + continue; + tested++; + + var frequency = 1.0f; + var amplitude = 1.0f; + var n0 = math.snoise_grad(p * frequency); + var s0 = (float)MathF.Sign(n0.w); + var expectedValue = amplitude * (1.0f - MathF.Abs(n0.w)); + var expectedGradient = -amplitude * frequency * s0 * new float3(n0.x, n0.y, n0.z); + frequency *= lacunarity; + amplitude *= gain; + + var n1 = math.snoise_grad(p * frequency); + var s1 = (float)MathF.Sign(n1.w); + expectedValue += amplitude * (1.0f - MathF.Abs(n1.w)); + expectedGradient += -amplitude * frequency * s1 * new float3(n1.x, n1.y, n1.z); + frequency *= lacunarity; + amplitude *= gain; + + var n2 = math.snoise_grad(p * frequency); + var s2 = (float)MathF.Sign(n2.w); + expectedValue += amplitude * (1.0f - MathF.Abs(n2.w)); + expectedGradient += -amplitude * frequency * s2 * new float3(n2.x, n2.y, n2.z); + + var actual = math.ridged_grad(p, 3, lacunarity, gain); + + Assert.Equal(expectedValue, actual.w); + Assert.Equal(expectedGradient.x, actual.x); + Assert.Equal(expectedGradient.y, actual.y); + Assert.Equal(expectedGradient.z, actual.z); + } + + Assert.True(tested == 32, $"only {tested} of {attempts} points were outside the zero band"); + } + + // ---- ridged_grad's deliberate crease at n = 0 ---- + + [Fact] + public void RidgedGradOneSidedGradientsAcrossAZeroCrossingDifferInSign() + { + // 1 octave: ridged_grad(p) = 1 - |snoise(p)|, gradient = -sign(snoise(p)) * grad(snoise)(p). + // Bisect along a fixed direction from a random start point to find a tight straddle of a + // snoise zero crossing, then check the two ridged_grad gradients just either side of it point + // in near-opposite directions (their dot product is negative): the crease is real, not a + // numerical accident (design.md, "the operator's dune term"). + var random = new System.Random(0xC2E55); + for (var attempt = 0; attempt < 200; attempt++) + { + var origin = RandomPoint(random); + var direction = new float3(random.NextSingle() - 0.5f, random.NextSingle() - 0.5f, random.NextSingle() - 0.5f); + if (math.length(direction) < 1.0e-3f) + continue; + + float NoiseAt(float t) => math.snoise(origin + direction * t); + + var lo = -3.0f; + var hi = 3.0f; + if (Math.Sign(NoiseAt(lo)) == Math.Sign(NoiseAt(hi))) + continue; + + for (var step = 0; step < 40; step++) + { + var mid = (lo + hi) * 0.5f; + if (Math.Sign(NoiseAt(lo)) == Math.Sign(NoiseAt(mid))) lo = mid; else hi = mid; + } + + // lo and hi now straddle the crossing within about 3 * 2^-40 of parameter t; step off by + // a small margin either side so each sample is cleanly on one side, not exactly on it. + const float margin = 1.0e-4f; + var minusSide = origin + direction * (lo - margin); + var plusSide = origin + direction * (hi + margin); + if (MathF.Sign(NoiseAt(lo - margin)) == MathF.Sign(NoiseAt(hi + margin))) + continue; // margin stepped back across the crossing; try another direction. + + var gradMinus = math.ridged_grad(minusSide, 1, 2.0f, 0.5f); + var gradPlus = math.ridged_grad(plusSide, 1, 2.0f, 0.5f); + var dot = gradMinus.x * gradPlus.x + gradMinus.y * gradPlus.y + gradMinus.z * gradPlus.z; + + Assert.True(dot < 0.0f, $"one-sided gradients did not differ in sign: minus=({gradMinus.x},{gradMinus.y},{gradMinus.z}) plus=({gradPlus.x},{gradPlus.y},{gradPlus.z}) dot={dot}"); + return; + } + + Assert.Fail("did not find a snoise zero crossing to straddle"); + } +} diff --git a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl index 5c00bda0..50c5a746 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl +++ b/packages/cultmath/unity/org.gamecult.cultmath/Shaders/CultMath.hlsl @@ -76,8 +76,8 @@ float cultmath_smootherstep(float value) float2 cultmath_snoise_mod289(float2 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } float3 cultmath_snoise_mod289(float3 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } float4 cultmath_snoise_mod289(float4 value) { return value - floor(value * (1.0 / 289.0)) * 289.0; } -float3 cultmath_snoise_permute(float3 value) { return cultmath_snoise_mod289(((value * 34.0) + 1.0) * value); } -float4 cultmath_snoise_permute(float4 value) { return cultmath_snoise_mod289(((value * 34.0) + 1.0) * value); } +float3 cultmath_snoise_permute(float3 value) { return cultmath_snoise_mod289(((value * 34.0) + 10.0) * value); } +float4 cultmath_snoise_permute(float4 value) { return cultmath_snoise_mod289(((value * 34.0) + 10.0) * value); } // Ashima Arts / Ian McEwan 3D simplex noise (MIT), same float32 evaluation // order as the C# math.snoise(float3) mirror. @@ -125,9 +125,9 @@ float cultmath_snoise(float3 value) p2 *= norm.z; p3 *= norm.w; - float4 m = max(0.6 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); + float4 m = max(0.5 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); m = m * m; - return 42.0 * dot(m * m, float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); + return 105.0 * dot(m * m, float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3))); } float cultmath_snoise(float2 value) @@ -162,6 +162,114 @@ float cultmath_snoise(float2 value) return 130.0 * dot(m, g); } +// Analytic gradient of cultmath_snoise(float3) (invariant 8), following webgl-noise +// src/noise3Dgrad.glsl (Ashima Arts / McEwan, MIT); same kernel, permutation and scale as +// cultmath_snoise(float3) (stegu/webgl-noise 22434e04d7). See the C# math.snoise_grad comment for the +// derivation. Every local up through p0..p3 and x0..x3 is cultmath_snoise(float3)'s own derivation +// verbatim. +float4 cultmath_snoise_grad(float3 value) +{ + float2 c = float2(1.0 / 6.0, 1.0 / 3.0); + float3 i = floor(value + dot(value, c.yyy)); + float3 x0 = value - i + dot(i, c.xxx); + float3 g = step(x0.yzx, x0.xyz); + float3 l = 1.0 - g; + float3 i1 = min(g.xyz, l.zxy); + float3 i2 = max(g.xyz, l.zxy); + float3 x1 = x0 - i1 + c.xxx; + float3 x2 = x0 - i2 + c.yyy; + float3 x3 = x0 - 0.5; + + i = cultmath_snoise_mod289(i); + float4 p = cultmath_snoise_permute(cultmath_snoise_permute(cultmath_snoise_permute( + i.z + float4(0.0, i1.z, i2.z, 1.0)) + i.y + float4(0.0, i1.y, i2.y, 1.0)) + i.x + float4(0.0, i1.x, i2.x, 1.0)); + + const float n_ = 0.142857142857; + float3 ns = float3(2.0 * n_, 0.5 * n_ - 1.0, n_); + float4 j = p - 49.0 * floor(p * ns.z * ns.z); + float4 x_ = floor(j * ns.z); + float4 y_ = floor(j - 7.0 * x_); + float4 x = x_ * ns.x + ns.yyyy; + float4 y = y_ * ns.x + ns.yyyy; + float4 h = 1.0 - abs(x) - abs(y); + + float4 b0 = float4(x.xy, y.xy); + float4 b1 = float4(x.zw, y.zw); + float4 s0 = floor(b0) * 2.0 + 1.0; + float4 s1 = floor(b1) * 2.0 + 1.0; + float4 sh = -step(h, 0.0); + float4 a0 = b0.xzyw + s0.xzyw * sh.xxyy; + float4 a1 = b1.xzyw + s1.xzyw * sh.zzww; + + float3 p0 = float3(a0.xy, h.x); + float3 p1 = float3(a0.zw, h.y); + float3 p2 = float3(a1.xy, h.z); + float3 p3 = float3(a1.zw, h.w); + float4 norm = 1.79284291400159 - 0.85373472095314 * float4(dot(p0, p0), dot(p1, p1), dot(p2, p2), dot(p3, p3)); + p0 *= norm.x; + p1 *= norm.y; + p2 *= norm.z; + p3 *= norm.w; + + float4 m0 = max(0.5 - float4(dot(x0, x0), dot(x1, x1), dot(x2, x2), dot(x3, x3)), 0.0); + float4 m2 = m0 * m0; + float4 m3 = m2 * m0; + float4 m4 = m2 * m2; + + float4 px = float4(dot(p0, x0), dot(p1, x1), dot(p2, x2), dot(p3, x3)); + float value2 = 105.0 * dot(m4, px); + float3 grad = -8.0 * m3.x * x0 * px.x + m4.x * p0 + + -8.0 * m3.y * x1 * px.y + m4.y * p1 + + -8.0 * m3.z * x2 * px.z + m4.z * p2 + + -8.0 * m3.w * x3 * px.w + m4.w * p3; + grad *= 105.0; + + return float4(grad, value2); +} + +// Octave sum of cultmath_snoise_grad (invariant 8); see the C# math.fbm_grad comment for the +// per-octave frequency/amplitude derivation and why octaves is clamped. +float4 cultmath_fbm_grad(float3 p, int octaves, float lacunarity, float gain) +{ + octaves = clamp(octaves, 0, 16); + float amplitude = 1.0; + float frequency = 1.0; + float value = 0.0; + float3 gradient = float3(0.0, 0.0, 0.0); + for (int i = 0; i < octaves; i++) + { + float4 n = cultmath_snoise_grad(p * frequency); + value += amplitude * n.w; + gradient += amplitude * frequency * n.xyz; + frequency *= lacunarity; + amplitude *= gain; + } + + return float4(gradient, value); +} + +// Ridged multifractal noise (Musgrave, Texturing and Modeling, ch. 16); see the C# math.ridged_grad +// comment for the fold/sign derivation, the deliberate crease at n = 0, and why octaves is clamped. +float4 cultmath_ridged_grad(float3 p, int octaves, float lacunarity, float gain) +{ + octaves = clamp(octaves, 0, 16); + float amplitude = 1.0; + float frequency = 1.0; + float value = 0.0; + float3 gradient = float3(0.0, 0.0, 0.0); + for (int i = 0; i < octaves; i++) + { + float4 n = cultmath_snoise_grad(p * frequency); + float s = sign(n.w); + value += amplitude * (1.0 - abs(n.w)); + gradient += -amplitude * frequency * s * n.xyz; + frequency *= lacunarity; + amplitude *= gain; + } + + return float4(gradient, value); +} + float cultmath_lengthsq(float2 value) { return dot(value, value); } float cultmath_lengthsq(float3 value) { return dot(value, value); } float cultmath_lengthsq(float4 value) { return dot(value, value); } diff --git a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md index 06ebb6c3..1de744a7 100644 --- a/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md +++ b/packages/cultmath/unity/org.gamecult.cultmath/THIRD-PARTY-NOTICES.md @@ -6,6 +6,16 @@ simplex-noise functions by Ian McEwan and Ashima Arts, as distributed by Stefan Gustavson's `webgl-noise` project and Unity Mathematics. +`math.snoise(float3)` and `cultmath_snoise(float3)` follow `src/noise3D.glsl` and +`math.snoise(float2)` follows `src/noise2D.glsl` of stegu/webgl-noise at commit +`22434e04d7753f7e949e8d724ab3da2864c17a0f` (the simplex-boundary fix is +`21d9fe23d7`, scale `ff3b5d34ea`, permutation `a3e6d57095`). + +`math.snoise_grad`/`cultmath_snoise_grad` (and the `fbm_grad`/`ridged_grad` +octave sums built on them) follow the analytic-gradient differentiation from +the same project's `src/noise3Dgrad.glsl`, with the same kernel radius, +permutation and scale as `snoise(float3)`. + - Copyright 2011 Ashima Arts. - Upstream: - License: MIT. diff --git a/src/GameCult.Caching/CultInspectorAttributes.cs b/src/GameCult.Caching/CultInspectorAttributes.cs index 2ae523e0..2a4e1839 100644 --- a/src/GameCult.Caching/CultInspectorAttributes.cs +++ b/src/GameCult.Caching/CultInspectorAttributes.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Generic; namespace GameCult.Caching { @@ -78,6 +79,21 @@ namespace GameCult.Caching public Type? AssetType { get; } } + // Groups a document type's record list by the named members, outermost first: the argument order is the nesting order. + // The nearest declaration up the inheritance chain wins, so a derived type may re-declare, and an empty declaration + // opts out of a base's. A member must be a visible, editable string, integer, bool, non-[Flags] enum or CultRecordRef; + // anything else gives a flat list and a notice. Presentation only: no schema, index or stored byte changes. + [AttributeUsage(AttributeTargets.Class, Inherited = true, AllowMultiple = false)] + public sealed class CultInspectorGroupByAttribute : Attribute + { + public CultInspectorGroupByAttribute(params string[] members) + { + Members = members ?? throw new ArgumentNullException(nameof(members)); + } + + public IReadOnlyList Members { get; } + } + // Marks a lowering's drawer class as the drawer for a claimed type. Claimed is either an Attribute subclass (the // drawer draws every member carrying that attribute) or a value type (every value of that type, or of any closed // form when Claimed is an open generic definition). Attribute claims win over type claims, which win over the diff --git a/src/GameCult.Caching/CultInspectorGrouping.cs b/src/GameCult.Caching/CultInspectorGrouping.cs new file mode 100644 index 00000000..07ef42a6 --- /dev/null +++ b/src/GameCult.Caching/CultInspectorGrouping.cs @@ -0,0 +1,60 @@ +using System; +using System.Collections.Generic; + +namespace GameCult.Caching +{ + // Which members group a listed document type's records, outermost first. + public sealed class CultInspectorGrouping + { + internal CultInspectorGrouping(Type listedType, IReadOnlyList members, string? notice) + { + ListedType = listedType; + Members = members; + Notice = notice; + } + + public Type ListedType { get; } + + // Empty when nothing groups: no declaration, an opt-out, or a refused declaration (then Notice says why). + public IReadOnlyList Members { get; } + + public string? Notice { get; } + } + + // One node of a grouped record list. Interior nodes hold Children; leaves hold Records. + public sealed class CultInspectorRecordGroup + { + internal CultInspectorRecordGroup( + string id, + string label, + int depth, + IReadOnlyList values, + IReadOnlyList children, + IReadOnlyList records, + int count) + { + Id = id; + Label = label; + Depth = depth; + Values = values; + Children = children; + Records = records; + Count = count; + } + + // Value identity along the path, not the label: stable across renames, so a lowering keys its foldout state on it. + public string Id { get; } + public string Label { get; } + + // 0 for the outermost level. + public int Depth { get; } + + // The grouped member values on the path to this node, outermost first. + public IReadOnlyList Values { get; } + public IReadOnlyList Children { get; } + public IReadOnlyList Records { get; } + + // The records under this node, at any depth. + public int Count { get; } + } +} diff --git a/src/GameCult.Caching/CultInspectorModel.cs b/src/GameCult.Caching/CultInspectorModel.cs index 99d82205..7aa6ab74 100644 --- a/src/GameCult.Caching/CultInspectorModel.cs +++ b/src/GameCult.Caching/CultInspectorModel.cs @@ -3,6 +3,7 @@ using System.Collections; using System.Collections.Generic; using System.Linq; using System.Reflection; +using System.Text; using MessagePack; namespace GameCult.Caching @@ -33,7 +34,7 @@ namespace GameCult.Caching internal CultInspectorMetadata(MemberInfo? member) { - Attributes = member == null ? Array.Empty() : member.GetCustomAttributes(true).OfType().ToArray(); + Attributes = member == null ? Array.Empty() : Attribute.GetCustomAttributes(member, true); var label = Find()?.Label; Label = string.IsNullOrWhiteSpace(label) ? null : label; Hidden = Find() != null; @@ -206,7 +207,7 @@ namespace GameCult.Caching private CultInspectorClaim ResolveCore(Type valueType, MemberInfo? member) { Type? drawer = null; - foreach (var attribute in member?.GetCustomAttributes(true).OfType() ?? Enumerable.Empty()) + foreach (var attribute in member == null ? Array.Empty() : Attribute.GetCustomAttributes(member, true)) { for (var type = attribute.GetType(); type != null && type != typeof(Attribute); type = type.BaseType) { @@ -294,6 +295,7 @@ namespace GameCult.Caching private readonly CultCodec _codec; private readonly Dictionary _shapes = new Dictionary(); + private readonly Dictionary _groupings = new Dictionary(); private readonly Dictionary _metadata = new Dictionary(); // The codec is the store's (CultCacheMessagePack.CreateInspectorModel for .cc stores), the one a CultCache resolves @@ -513,14 +515,24 @@ namespace GameCult.Caching // The records a CultRecordRef may point at: every record whose document is a T, by label. public IReadOnlyList RecordCandidates(Type recordRefType, IEnumerable records) { - var target = ShapeOf(recordRefType).RecordTarget - ?? throw new ArgumentException($"{recordRefType.Name} is not a CultRecordRef.", nameof(recordRefType)); + var target = RecordTargetOf(recordRefType); return records - .Where(record => target.IsInstanceOfType(record.Document) && record.Key.Value.Length > 0) + .Where(record => IsCandidateFor(target, record)) .OrderBy(RecordLabel, StringComparer.OrdinalIgnoreCase) .ToArray(); } + // Whether record may be the value of a recordRefType: the one rule the picker's candidates and a drop share. + public bool IsRecordCandidate(Type recordRefType, CultStoredDocument record) => + IsCandidateFor(RecordTargetOf(recordRefType), record ?? throw new ArgumentNullException(nameof(record))); + + private Type RecordTargetOf(Type recordRefType) => + ShapeOf(recordRefType).RecordTarget + ?? throw new ArgumentException($"{recordRefType.Name} is not a CultRecordRef.", nameof(recordRefType)); + + private static bool IsCandidateFor(Type target, CultStoredDocument record) => + target.IsInstanceOfType(record.Document) && record.Key.Value.Length > 0; + public object CreateRecordRef(Type recordRefType, string key) { if (ShapeOf(recordRefType).Kind != CultInspectorValueKind.RecordRef) @@ -528,6 +540,203 @@ namespace GameCult.Caching return Activator.CreateInstance(recordRefType, new CultRecordKey(key))!; } + // How a listed document type's records group, from the nearest [CultInspectorGroupBy] up its inheritance chain. Members + // is empty when nothing groups, with a Notice when a declaration was refused. + public CultInspectorGrouping GroupingOf(Type listed) + { + if (listed == null) throw new ArgumentNullException(nameof(listed)); + lock (_groupings) + { + if (!_groupings.TryGetValue(listed, out var grouping)) + _groupings[listed] = grouping = ResolveGrouping(listed); + return grouping; + } + } + + private CultInspectorGrouping ResolveGrouping(Type listed) + { + var declared = (CultInspectorGroupByAttribute?)Attribute.GetCustomAttribute(listed, typeof(CultInspectorGroupByAttribute), true); + if (declared == null || declared.Members.Count == 0) + return new CultInspectorGrouping(listed, Array.Empty(), null); + + CultInspectorGrouping Refused(string reason) => + new CultInspectorGrouping(listed, Array.Empty(), $"[CultInspectorGroupBy] on {listed.Name}: {reason} The list is not grouped."); + + if (listed.GetCustomAttribute() != null) + return Refused("a global document is never grouped."); + var members = new List(); + foreach (var name in declared.Members) + { + var member = MembersOf(listed).FirstOrDefault(candidate => candidate.Name == name); + if (member == null) return Refused($"{name} is not an inspector member."); + if (members.Contains(member)) return Refused($"{name} is named twice."); + if (member.Metadata.Hidden) return Refused($"{name} is hidden."); + if (member.IsReadOnly) return Refused($"{name} is read-only."); + var kind = ShapeOf(member.ValueType).Kind; + var groupable = kind is CultInspectorValueKind.String or CultInspectorValueKind.Integer or CultInspectorValueKind.Bool or CultInspectorValueKind.RecordRef + || kind == CultInspectorValueKind.Enum && !member.ValueType.IsDefined(typeof(FlagsAttribute), false); + if (!groupable) return Refused($"{name} is a {member.ValueType.Name}, which does not group."); + members.Add(member); + } + + return new CultInspectorGrouping(listed, members, null); + } + + // The tree of listed's candidate records: one level per grouped member, occupied values only. Leaves hold the records, + // in candidate order, and together exactly RecordCandidates(CultRecordRef, records); a variant groups by its + // resolved values. Empty when GroupingOf(listed) groups nothing: the lowering lists the candidates flat. + public IReadOnlyList GroupRecords(Type listed, IEnumerable records) + { + var grouping = GroupingOf(listed); + if (grouping.Members.Count == 0) + return Array.Empty(); + var all = records.ToArray(); + var targets = grouping.Members + .Where(member => ShapeOf(member.ValueType).Kind == CultInspectorValueKind.RecordRef) + .ToDictionary(member => member, member => RefTargets(member.ValueType, all)); + var candidates = RecordCandidates(typeof(CultRecordRef<>).MakeGenericType(listed), all); + return Partition(grouping, candidates, 0, Array.Empty(), targets); + } + + private IReadOnlyList Partition( + CultInspectorGrouping grouping, + IReadOnlyList records, + int depth, + IReadOnlyList path, + IReadOnlyDictionary> targets) + { + var member = grouping.Members[depth]; + targets.TryGetValue(member, out var refTargets); + return records + .Select(record => (Record: record, Key: LevelOf(member, member.GetValue(record.Document), refTargets))) + .GroupBy(entry => entry.Key.Identity) + .Select(bucket => (Key: bucket.First().Key, Records: bucket.Select(entry => entry.Record).ToArray())) + .OrderBy(bucket => bucket.Key, GroupLevel.Order) + .Select(bucket => + { + var values = path.Append(bucket.Key.Value).ToArray(); + var children = depth + 1 < grouping.Members.Count + ? Partition(grouping, bucket.Records, depth + 1, values, targets) + : Array.Empty(); + return new CultInspectorRecordGroup( + GroupId(grouping.Members, values), bucket.Key.Label, depth, values, children, + children.Count == 0 ? bucket.Records : Array.Empty(), bucket.Records.Length); + }) + .ToArray(); + } + + // A new listed document set to every grouped value on group's path, so it lands in group. Null with a notice when + // listed cannot be created. The group must come from GroupRecords for this listed type. + public object? CreateInGroup(Type listed, CultInspectorRecordGroup group, out string? notice) + { + if (group == null) throw new ArgumentNullException(nameof(group)); + var members = GroupingOf(listed).Members; + if (group.Values.Count == 0 || group.Values.Count > members.Count || group.Id != GroupId(members, group.Values)) + throw new ArgumentException($"Group {group.Id} is not a node of {listed.Name}'s grouping.", nameof(group)); + var created = CreateElement(listed, listed, out notice); + if (created == null) + return null; + for (var i = 0; i < group.Values.Count; i++) + members[i].SetValue(created, group.Values[i]); + return created; + } + + // What a CultRecordRef value shows: None, the target's label, or Missing when the key names no candidate. + public string RecordRefLabel(Type recordRefType, object? value, IEnumerable records) => + RefLabel(RecordKey(value), RefTargets(recordRefType, records)); + + private Dictionary RefTargets(Type recordRefType, IEnumerable records) => + RecordCandidates(recordRefType, records).ToDictionary(record => record.Key.Value, StringComparer.Ordinal); + + private static string RefLabel(string key, IReadOnlyDictionary targets) => + key.Length == 0 ? "None" : targets.TryGetValue(key, out var target) ? RecordLabel(target) : "Missing " + key; + + // A node's id is its members' value identities, so it survives relabelling: a ref by key, an enum by its underlying + // value, anything else by the value's invariant string. + private static string GroupId(IReadOnlyList members, IReadOnlyList values) => + string.Join("/", values.Select((value, i) => members[i].Name + "=" + EscapeIdentity(IdentityOf(value)))); + + // Injective: text is spelled as Uri.EscapeDataString spells it, which turns '/' and '%' into %XX; an unpaired + // surrogate, which that method would flatten to U+FFFD, becomes %uXXXX, a form no escaped run contains. + private static string EscapeIdentity(string identity) + { + var escaped = new StringBuilder(); + var run = 0; + for (var i = 0; i < identity.Length; i++) + { + if (char.IsHighSurrogate(identity[i]) && i + 1 < identity.Length && char.IsLowSurrogate(identity[i + 1])) + i++; + else if (char.IsSurrogate(identity[i])) + { + escaped.Append(Uri.EscapeDataString(identity.Substring(run, i - run))).Append("%u").Append(((int)identity[i]).ToString("X4")); + run = i + 1; + } + } + + return escaped.Append(Uri.EscapeDataString(identity.Substring(run))).ToString(); + } + + private static string IdentityOf(object? value) => + value switch + { + null => string.Empty, + ICultRecordRef reference => reference.Key.Value, + Enum enumeration => Convert.ToString(Convert.ChangeType(enumeration, Enum.GetUnderlyingType(enumeration.GetType())), System.Globalization.CultureInfo.InvariantCulture)!, + _ => Convert.ToString(value, System.Globalization.CultureInfo.InvariantCulture)! + }; + + private GroupLevel LevelOf(CultInspectorMember member, object? value, IReadOnlyDictionary? refTargets) + { + var identity = IdentityOf(value); + switch (ShapeOf(member.ValueType).Kind) + { + case CultInspectorValueKind.RecordRef: + var label = RefLabel(identity, refTargets!); + var rank = identity.Length == 0 ? 0 : refTargets!.ContainsKey(identity) ? 1 : 2; + return new GroupLevel(identity, label, CreateRecordRef(member.ValueType, identity), rank, 0m, label, identity); + case CultInspectorValueKind.Enum: + var number = Convert.ToDecimal(Convert.ChangeType(value, Enum.GetUnderlyingType(member.ValueType)), System.Globalization.CultureInfo.InvariantCulture); + return new GroupLevel(identity, Enum.GetName(member.ValueType, value!) ?? identity, value, 1, number); + case CultInspectorValueKind.Integer: + return new GroupLevel(identity, identity, value, 1, Convert.ToDecimal(value, System.Globalization.CultureInfo.InvariantCulture)); + case CultInspectorValueKind.Bool: + return new GroupLevel(identity, identity, value, 1, (bool)value! ? 1m : 0m); + default: + return new GroupLevel(identity, identity.Length == 0 ? "(empty)" : identity, identity, 1, 0m, identity, identity); + } + } + + private sealed class GroupLevel + { + public GroupLevel(string identity, string label, object? value, int rank, decimal number, string text = "", string tie = "") + { + Identity = identity; + Label = label; + Value = value; + Rank = rank; + Number = number; + Text = text; + Tie = tie; + } + + public string Identity { get; } + public string Label { get; } + public object? Value { get; } + private int Rank { get; } + private decimal Number { get; } + private string Text { get; } + private string Tie { get; } + + // None first and Missing last among refs; then the value, the text ignoring case, then the text exactly. + public static readonly IComparer Order = Comparer.Create((a, b) => + { + var order = a.Rank.CompareTo(b.Rank); + if (order == 0) order = a.Number.CompareTo(b.Number); + if (order == 0) order = StringComparer.OrdinalIgnoreCase.Compare(a.Text, b.Text); + return order != 0 ? order : StringComparer.Ordinal.Compare(a.Tie, b.Tie); + }); + } + private CultInspectorShape Classify(Type type) { if (type == typeof(string)) return new CultInspectorShape(type, CultInspectorValueKind.String); diff --git a/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioDrawers.cs b/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioDrawers.cs index 8f008f24..14bb10e4 100644 --- a/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioDrawers.cs +++ b/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioDrawers.cs @@ -263,22 +263,45 @@ namespace GameCult.Unity.Caching.Editor if (candidates[i].Key.Value == key) index = i + 1; } - names[0] = index == 0 && key.Length > 0 ? "Missing " + key : "None"; + names[0] = index == 0 ? Model.RecordRefLabel(type, value, Records) : "None"; var next = key; - using (new EditorGUILayout.HorizontalScope()) + using (var row = new EditorGUILayout.HorizontalScope()) { + var dropped = AcceptRecordDrop(row.rect, type, key); var picked = EditorGUILayout.Popup(label, index, names); var indent = EditorGUI.indentLevel; EditorGUI.indentLevel = 0; var typed = EditorGUILayout.DelayedTextField(key, GUILayout.Width(120)); EditorGUI.indentLevel = indent; - if (picked != index) next = picked == 0 ? string.Empty : candidates[picked - 1].Key.Value; + if (dropped != null) next = dropped; + else if (picked != index) next = picked == 0 ? string.Empty : candidates[picked - 1].Key.Value; else if (typed != key) next = typed; } return next == key ? value : Model.CreateRecordRef(type, next); } + // A record row dragged from the Studio list onto a CultRecordRef value. Returns the dropped key when the drop is + // performed and the ref's picker would offer that record, or null. The payload is the Studio's own; a foreign drag is + // left alone. Refused in a disabled scope, from another model, and for the key already held. + private string AcceptRecordDrop(Rect rect, Type type, string key) + { + var current = Event.current; + if (current.type != EventType.DragUpdated && current.type != EventType.DragPerform) return null; + if (!rect.Contains(current.mousePosition) || !(DragAndDrop.GetGenericData(CultCacheStudioRecordDrag.DragKey) is CultCacheStudioRecordDrag drag)) return null; + + var perform = current.type == EventType.DragPerform; + var record = ReferenceEquals(drag.Model, Model) && drag.Key != key ? Records.FirstOrDefault(candidate => candidate.Key.Value == drag.Key) : null; + var accepted = GUI.enabled && record != null && Model.IsRecordCandidate(type, record); + DragAndDrop.visualMode = accepted ? DragAndDropVisualMode.Copy : DragAndDropVisualMode.Rejected; + current.Use(); + if (!accepted || !perform) return null; + + DragAndDrop.AcceptDrag(); + GUI.changed = true; + return drag.Key; + } + private object DrawDictionary(string label, CultInspectorShape shape, IDictionary dictionary, MemberInfo member) { var path = _path; @@ -503,4 +526,19 @@ namespace GameCult.Unity.Caching.Editor private static string LabelOf(CultInspectorMember member) => member.Metadata.Label ?? ObjectNames.NicifyVariableName(member.Name); } + + // The payload of a Studio record drag: the record's key, and the model it came from so a drop refuses another window's. + internal sealed class CultCacheStudioRecordDrag + { + internal const string DragKey = "GameCult.CultCacheStudio.Record"; + + internal CultCacheStudioRecordDrag(CultInspectorModel model, string key) + { + Model = model; + Key = key; + } + + internal CultInspectorModel Model { get; } + internal string Key { get; } + } } diff --git a/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioWindow.cs b/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioWindow.cs index 6c9fedf8..20199d60 100644 --- a/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioWindow.cs +++ b/src/GameCult.Unity/Assets/Caching/Editor/CultCacheStudioWindow.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Generic; using System.IO; using System.Linq; using GameCult.Caching; @@ -11,6 +12,7 @@ namespace GameCult.Unity.Caching.Editor public sealed class CultCacheStudioWindow : EditorWindow { private const string LastPathKey = "GameCult.CultCacheStudio.LastPath"; + private const float RowIndent = 14f; private CultCache _cache; private CultInspectorModel _model; @@ -23,6 +25,10 @@ namespace GameCult.Unity.Caching.Editor private string _search = string.Empty; private Type _selectedType; private string _selectedKey; + private string _pressedKey; + private string _revealKey; + // Group foldouts by listed type and node id: two types grouped by the same values share node ids, not fold state. + private readonly Dictionary<(Type, string), bool> _groupFoldouts = new Dictionary<(Type, string), bool>(); private Vector2 _typeScroll; private Vector2 _recordScroll; private Vector2 _inspectorScroll; @@ -143,7 +149,7 @@ namespace GameCult.Unity.Caching.Editor { using (new EditorGUI.DisabledScope(cannotAdd || descriptor.IsGlobal)) { - if (GUILayout.Button("Add", EditorStyles.miniButtonLeft)) Add(); + if (GUILayout.Button("Add", EditorStyles.miniButtonLeft)) Create(null); } using (new EditorGUI.DisabledScope(selected == null || selected.Descriptor.IsGlobal || ReadOnly)) @@ -165,6 +171,8 @@ namespace GameCult.Unity.Caching.Editor else { var records = _model.RecordCandidates(typeof(CultRecordRef<>).MakeGenericType(_selectedType), _records); + var grouping = _model.GroupingOf(_selectedType); + if (grouping.Notice != null) EditorGUILayout.HelpBox(grouping.Notice, MessageType.Warning); if (descriptor.IsGlobal && records.Count == 0) { using (new EditorGUILayout.HorizontalScope()) @@ -172,23 +180,100 @@ namespace GameCult.Unity.Caching.Editor EditorGUILayout.LabelField("Global is absent."); using (new EditorGUI.DisabledScope(cannotAdd)) { - if (GUILayout.Button("Create", GUILayout.Width(56))) Add(); + if (GUILayout.Button("Create", GUILayout.Width(56))) Create(null); } } } - foreach (var record in records) + // Nothing groups: the candidates are the list. Otherwise the model's tree is. + if (grouping.Members.Count == 0) { - var label = CultInspectorModel.RecordLabel(record); - var text = record.Descriptor.DocumentType == _selectedType ? label : label + " <" + record.Descriptor.DocumentType.Name + ">"; - if (GUILayout.Toggle(_selectedKey == record.Key.Value, text, EditorStyles.miniButton)) _selectedKey = record.Key.Value; + foreach (var record in records) DrawRecordRow(record, 0); } + else + { + var groups = _model.GroupRecords(_selectedType, _records); + if (_revealKey != null) Reveal(groups, _revealKey); + DrawGroups(groups, cannotAdd); + } + + _revealKey = null; } EditorGUILayout.EndScrollView(); } } + private void DrawGroups(IReadOnlyList groups, bool cannotAdd) + { + foreach (var group in groups) + { + var id = (_selectedType, group.Id); + _groupFoldouts.TryGetValue(id, out var open); + EditorGUI.indentLevel = group.Depth; + open = EditorGUILayout.Foldout(open, group.Label + " (" + group.Count + ")", true); + EditorGUI.indentLevel = 0; + _groupFoldouts[id] = open; + if (!open) continue; + + DrawGroups(group.Children, cannotAdd); + foreach (var record in group.Records) DrawRecordRow(record, group.Depth + 1); + using (new EditorGUILayout.HorizontalScope()) + { + GUILayout.Space((group.Depth + 1) * RowIndent); + using (new EditorGUI.DisabledScope(cannotAdd)) + { + if (GUILayout.Button("Create", EditorStyles.miniButton, GUILayout.Width(56))) Create(group); + } + } + } + } + + // Expands every node on the path to the record, so a record just made or duplicated is on screen. + private bool Reveal(IReadOnlyList groups, string key) + { + var found = false; + foreach (var group in groups) + { + if (!group.Records.Any(record => record.Key.Value == key) && !Reveal(group.Children, key)) continue; + _groupFoldouts[(_selectedType, group.Id)] = true; + found = true; + } + + return found; + } + + // A record row selects on click and starts a record drag. The drag starts from the reserved rect on the first drag + // event after a press inside it; the toggle still reads the click. + private void DrawRecordRow(CultStoredDocument record, int depth) + { + var label = CultInspectorModel.RecordLabel(record); + var text = record.Descriptor.DocumentType == _selectedType ? label : label + " <" + record.Descriptor.DocumentType.Name + ">"; + var key = record.Key.Value; + var rect = GUILayoutUtility.GetRect(new GUIContent(text), EditorStyles.miniButton); + rect.xMin += depth * RowIndent; + var current = Event.current; + if (current.type == EventType.MouseDown && current.button == 0 && rect.Contains(current.mousePosition)) + { + _pressedKey = key; + } + else if (current.type == EventType.MouseDrag && _pressedKey == key) + { + _pressedKey = null; + DragAndDrop.PrepareStartDrag(); + DragAndDrop.SetGenericData(CultCacheStudioRecordDrag.DragKey, new CultCacheStudioRecordDrag(_model, key)); + DragAndDrop.objectReferences = Array.Empty(); + DragAndDrop.StartDrag(label); + current.Use(); + } + else if (current.type == EventType.MouseUp) + { + _pressedKey = null; + } + + if (GUI.Toggle(rect, _selectedKey == key, text, EditorStyles.miniButton)) _selectedKey = key; + } + private void DrawInspector() { using (new EditorGUILayout.VerticalScope(GUILayout.ExpandWidth(true))) @@ -324,6 +409,9 @@ namespace GameCult.Unity.Caching.Editor _edit = null; _records = Array.Empty(); _selectedKey = null; + _pressedKey = null; + _revealKey = null; + _groupFoldouts.Clear(); } private bool ConfirmDiscard() @@ -337,20 +425,31 @@ namespace GameCult.Unity.Caching.Editor Run("Saved " + _path + ".", () => _cache.FlushAsync().GetAwaiter().GetResult()); } - private void Add() + // A new record of the selected type: at the root, or preset by the group it is created in. + private void Create(CultInspectorRecordGroup group) { var type = _selectedType; - // The model decides whether the type can be made; its notice is the refusal. - Run("Added " + type.Name + ".", () => _selectedKey = _cache.UpsertAsync(type, - _model.CreateElement(type, type, out var notice) ?? throw new InvalidOperationException(notice)).GetAwaiter().GetResult().Value); + Run("Added " + type.Name + ".", () => + { + // The model decides whether the type can be made; its notice is the refusal. + var created = group == null ? _model.CreateElement(type, type, out var notice) : _model.CreateInGroup(type, group, out notice); + Focus(_cache.UpsertAsync(type, created ?? throw new InvalidOperationException(notice)).GetAwaiter().GetResult().Value); + }); GUIUtility.ExitGUI(); } + // Selects the record and opens the groups it sits in on the next frame. + private void Focus(string key) + { + _selectedKey = key; + _revealKey = key; + } + private void Duplicate(CultStoredDocument record) { var type = record.Descriptor.DocumentType; Run("Duplicated " + CultInspectorModel.RecordLabel(record) + ".", () => - _selectedKey = _cache.UpsertAsync(type, _model.Clone(record.Document, type)).GetAwaiter().GetResult().Value); + Focus(_cache.UpsertAsync(type, _model.Clone(record.Document, type)).GetAwaiter().GetResult().Value)); GUIUtility.ExitGUI(); } diff --git a/src/GameCult.Unity/Assets/Caching/README.md b/src/GameCult.Unity/Assets/Caching/README.md index 7f1d7ab6..32a1fa69 100644 --- a/src/GameCult.Unity/Assets/Caching/README.md +++ b/src/GameCult.Unity/Assets/Caching/README.md @@ -33,6 +33,21 @@ fallback. `Add`, `Duplicate` and `Delete` act on the selected type and record. A `[CultGlobal]` type with no record shows as absent with a `Create` action; the Studio never creates one on its own. +A document class marked `[CultInspectorGroupBy(nameof(Kind), nameof(Owner))]` +lists its records as foldouts nested in that order, one per value in use; a +derived type listed under a marked base groups the same way. Each foldout has +a `Create` button that makes a new record with the grouped members preset to +that foldout's values, so it lands there. A declaration the model refuses (an +unknown, hidden, read-only, repeated or ungroupable member, or a global type) +lists the records flat under a warning that says why. A variant sits in the +foldout its resolved values name. + +Drag a record row onto any record-reference value in the inspector (a member, +a list element, a dictionary key or value) to set it. The drop is accepted +exactly when the reference's picker would offer that record; it is refused +in a read-only store, for the record the reference already holds, and for a +record from another Studio window. The reference picker itself is not grouped. + Opening a store never writes it, directory stores included. Closing or reloading scripts drops unsaved edits. @@ -57,6 +72,14 @@ leaves the cached object unchanged. The Studio is one lowering of that model and owns only widgets and layout; a runtime CultUI panel needs only its own renderer and drawers to show and edit the same documents under the same rules. +Grouping is the model's too: `GroupingOf` resolves a listed type's +`[CultInspectorGroupBy]` members (or the notice for a refused declaration), +`GroupRecords` returns the tree of nodes (id, label, depth, values, children, +records) that a lowering draws, and `CreateInGroup` makes the record for a +node. `IsRecordCandidate` is the one test for whether a record may be a +reference's value, shared by the picker's candidates and a drop, and +`RecordRefLabel` is the one spelling of `None`, a target's label and `Missing`. + ## Drawers Built in: every model shape above. A struct without `[Key]` members is edited diff --git a/tests/GameCult.Caching.Tests/CultInspectorModelTests.cs b/tests/GameCult.Caching.Tests/CultInspectorModelTests.cs index 5c2c171b..4ab5c680 100644 --- a/tests/GameCult.Caching.Tests/CultInspectorModelTests.cs +++ b/tests/GameCult.Caching.Tests/CultInspectorModelTests.cs @@ -14,7 +14,7 @@ namespace GameCult.Caching.Tests public class CultInspectorModelTests { private static readonly CultDocumentRegistry Registry = - CultDocumentRegistry.ForTypes(new[] { typeof(InspectItem), typeof(InspectOther), typeof(InspectFixed) }); + CultDocumentRegistry.ForTypes(new[] { typeof(InspectItem), typeof(InspectOther), typeof(InspectFixed), typeof(InspectLabeled) }); private static CultInspectorModel Model() => CultCacheMessagePack.CreateInspectorModel(Registry); @@ -128,6 +128,35 @@ namespace GameCult.Caching.Tests Assert.That(CultInspectorModel.RecordKey(new CultRecordRef(a)), Is.EqualTo(a.Value)); } + [Test] + public void OneRuleDecidesWhichRecordsMayBeARefsValue() + { + var model = Model(); + CultStoredDocument Record(string key, object document) => + new(new CultRecordKey(key), "test", Registry.GetRequired(document.GetType()), document); + var leaf = Record("leaf", new InspectLabeled { Name = "a" }); + var sibling = Record("other", new InspectOther { Name = "b" }); + var unkeyed = Record("", new InspectLabeled { Name = "c" }); + var records = new[] { sibling, leaf, unkeyed }; + var baseRef = typeof(CultRecordRef); + + Assert.That(model.IsRecordCandidate(baseRef, leaf), Is.True, "a subtype of the target"); + Assert.That(model.IsRecordCandidate(baseRef, sibling), Is.False, "a sibling type"); + Assert.That(model.IsRecordCandidate(baseRef, unkeyed), Is.False, "an empty key"); + Assert.That(model.RecordCandidates(baseRef, records), Is.EqualTo(records.Where(record => model.IsRecordCandidate(baseRef, record)))); + Assert.That(() => model.IsRecordCandidate(typeof(string), leaf), Throws.ArgumentException); + } + + [Test] + public void ABaseMembersLabelReachesItsOverride() + { + var member = Model().MembersOf(typeof(InspectLabeled)).Single(candidate => candidate.Name == nameof(InspectLabeled.Tag)); + + Assert.That(member.Metadata.Label, Is.EqualTo("Base Label")); + var claims = new CultInspectorDrawerClaims(new[] { typeof(LabelClaimDrawer) }, typeof(IFakeDrawer)); + Assert.That(claims.Resolve(typeof(string), member.Member).Drawer, Is.EqualTo(typeof(LabelClaimDrawer)), "attribute claims see the base attribute too"); + } + [Test] public async Task DictionaryKeysRefuseNullEmptyReferencesAndDuplicates() { @@ -296,6 +325,395 @@ namespace GameCult.Caching.Tests Is.EqualTo(new Dictionary { ["a"] = 1 })); } + // ---- grouping ---- + + private static readonly CultDocumentRegistry GroupRegistry = CultDocumentRegistry.ForTypes(new[] + { + typeof(GroupLeaf), typeof(GroupRedeclared), typeof(GroupOptOut), typeof(GroupMixed), typeof(GroupGlobal), typeof(GroupNoCtor), typeof(GroupTagged), typeof(InspectOther) + }); + + private string _groupPath = string.Empty; + + [SetUp] + public void PickGroupPath() => _groupPath = Path.Combine(Path.GetTempPath(), $"cultlib-group-{Guid.NewGuid():N}.cc"); + + [TearDown] + public void DeleteGroupPath() + { + if (File.Exists(_groupPath)) + File.Delete(_groupPath); + } + + private static CultInspectorModel GroupModel() => CultCacheMessagePack.CreateInspectorModel(GroupRegistry); + + private CultCache OpenGroups() => CultCacheMessagePack.Create(_groupPath, new CultCacheOpenOptions { Registry = GroupRegistry }); + + private static CultRecordRef OwnerRef(string key) => new(new CultRecordKey(key)); + + private static Task PutOwner(CultCache cache, string key, string name) => + cache.UpsertAsync(typeof(InspectOther), new InspectOther { Name = name }, new CultRecordKey(key)); + + private static Task PutLeaf(CultCache cache, string key, string name, string? owner, GroupHull hull) => + cache.UpsertAsync(typeof(GroupLeaf), + new GroupLeaf { Name = name, Owner = owner == null ? default : OwnerRef(owner), Hull = hull }, new CultRecordKey(key)); + + private static IEnumerable Dump(IEnumerable groups) => + groups.SelectMany(group => new[] + { + new string(' ', group.Depth * 2) + group.Label + " (" + group.Count + ")" + + (group.Records.Count > 0 ? ": " + string.Join(",", group.Records.Select(record => record.Key.Value)) : string.Empty) + }.Concat(Dump(group.Children))); + + private static IEnumerable Leaves(IEnumerable groups) => + groups.SelectMany(group => group.Children.Count == 0 ? new[] { group } : Leaves(group.Children)); + + private static bool Holds(CultInspectorRecordGroup group, string key) => + group.Records.Any(record => record.Key.Value == key) || group.Children.Any(child => Holds(child, key)); + + private static IEnumerable Containing(IEnumerable groups, string key) => + groups.Where(group => Holds(group, key)).SelectMany(group => new[] { group }.Concat(Containing(group.Children, key))); + + private async Task SeededGroups() + { + var cache = OpenGroups(); + await PutOwner(cache, "o-a", "Alpha"); + await PutOwner(cache, "o-b", "Beta"); + await PutLeaf(cache, "r1", "zeta", "o-b", GroupHull.Cruiser); + await PutLeaf(cache, "r2", "alpha", "o-a", GroupHull.Frigate); + await PutLeaf(cache, "r3", "mid", "o-a", GroupHull.Cruiser); + await PutLeaf(cache, "r4", "beta", "o-a", GroupHull.Cruiser); + return cache; + } + + [Test] + public async Task GroupLevelsNestInDeclaredOrderNotSlotOrder() + { + using var cache = await SeededGroups(); + var model = GroupModel(); + + Assert.That(model.GroupingOf(typeof(GroupLeaf)).Members.Select(member => member.Name), Is.EqualTo(new[] { "Owner", "Hull" })); + Assert.That(Dump(model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments)), Is.EqualTo(new[] + { + "Alpha (3)", + " Cruiser (2): r4,r3", + " Frigate (1): r2", + "Beta (1)", + " Cruiser (1): r1" + }), "an interior node holds no records; a leaf holds its records in candidate order"); + } + + [Test] + public void GroupADerivedListedTypeGroupsByItsBasesDeclarationAndTheNearestDeclarationWins() + { + var model = GroupModel(); + + Assert.That(model.GroupingOf(typeof(GroupLeaf)).Members.Select(member => member.Name), Is.EqualTo(new[] { "Owner", "Hull" }), "inherited from GroupBase"); + Assert.That(model.GroupingOf(typeof(GroupRedeclared)).Members.Select(member => member.Name), Is.EqualTo(new[] { "Kind" })); + + var optOut = model.GroupingOf(typeof(GroupOptOut)); + Assert.That(optOut.Members, Is.Empty, "an empty declaration opts out of the base's"); + Assert.That(optOut.Notice, Is.Null); + Assert.That(model.GroupingOf(typeof(GroupLeaf)), Is.SameAs(model.GroupingOf(typeof(GroupLeaf))), "cached"); + } + + [Test] + public async Task GroupLeavesPartitionTheCandidatesWhateverTheirGroupedValues() + { + using var cache = OpenGroups(); + await PutOwner(cache, "o-a", "Alpha"); + await PutLeaf(cache, "r1", "n1", null, GroupHull.Cruiser); + await PutLeaf(cache, "r2", "n2", "ghost", (GroupHull)99); + await PutLeaf(cache, "r3", "n3", "o-a", GroupHull.Frigate); + var model = GroupModel(); + var records = cache.AllStoredDocuments.ToArray(); + + var groups = model.GroupRecords(typeof(GroupLeaf), records); + + Assert.That(Dump(groups), Is.EqualTo(new[] + { + "None (1)", " Cruiser (1): r1", + "Alpha (1)", " Frigate (1): r3", + "Missing ghost (1)", " 99 (1): r2" + })); + var candidates = model.RecordCandidates(typeof(CultRecordRef), records); + Assert.That(Leaves(groups).SelectMany(leaf => leaf.Records), Is.EquivalentTo(candidates), "each candidate once, the owner record is no candidate"); + } + + [Test] + public async Task GroupRefsGroupByKeyNotLabelWithNoneFirstAndMissingLast() + { + using var cache = OpenGroups(); + await PutOwner(cache, "o1", "Same"); + await PutOwner(cache, "o2", "Same"); + await PutOwner(cache, "o3", "Aaa"); + await PutOwner(cache, "o4", "Zzz"); + foreach (var (key, owner) in new[] { ("r1", "o2"), ("r2", "o1"), ("r3", (string?)null), ("r4", "ghost-b"), ("r5", "ghost-a"), ("r6", "o3"), ("r7", "o4") }) + await PutLeaf(cache, key, key, owner, GroupHull.Cruiser); + var model = GroupModel(); + + var roots = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments); + + Assert.That(roots.Select(group => group.Label), + Is.EqualTo(new[] { "None", "Aaa", "Same", "Same", "Zzz", "Missing ghost-a", "Missing ghost-b" })); + Assert.That(roots.Select(group => CultInspectorModel.RecordKey(group.Values[0])), Is.EqualTo(new[] { "", "o3", "o1", "o2", "o4", "ghost-a", "ghost-b" }), + "equal labels order by key"); + Assert.That(roots.Select(group => group.Id).Distinct().Count(), Is.EqualTo(7)); + + var sameId = roots[2].Id; + await PutOwner(cache, "o1", "Renamed"); + var regrouped = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments); + Assert.That(regrouped.Select(group => group.Id), Does.Contain(sameId), "identity is the key, not the label"); + Assert.That(regrouped.Select(group => group.Label), Does.Contain("Renamed")); + } + + [Test] + public async Task GroupRefLabelsAreOwnedByTheModel() + { + using var cache = OpenGroups(); + await PutOwner(cache, "o1", "Alpha"); + await PutOwner(cache, "blank", " "); + await PutLeaf(cache, "x", "x", null, GroupHull.Cruiser); + var model = GroupModel(); + var records = cache.AllStoredDocuments.ToArray(); + var refType = typeof(CultRecordRef); + + Assert.That(model.RecordRefLabel(refType, default(CultRecordRef), records), Is.EqualTo("None")); + Assert.That(model.RecordRefLabel(refType, OwnerRef("o1"), records), Is.EqualTo("Alpha")); + Assert.That(model.RecordRefLabel(refType, OwnerRef("blank"), records), Is.EqualTo("blank"), "an unnamed record shows its key"); + Assert.That(model.RecordRefLabel(refType, OwnerRef("gone"), records), Is.EqualTo("Missing gone")); + Assert.That(model.RecordRefLabel(refType, OwnerRef("x"), records), Is.EqualTo("Missing x"), "a record that is no candidate is missing"); + Assert.That(model.RecordRefLabel(refType, null, records), Is.EqualTo("None")); + Assert.That(() => model.RecordRefLabel(typeof(string), null, records), Throws.ArgumentException); + } + + [Test] + public async Task GroupEnumNodesOrderByValueAndScalarLevelsByTheirDefaults() + { + using var cache = OpenGroups(); + foreach (var (key, hull) in new[] { ("h1", GroupHull.Frigate), ("h2", GroupHull.Cruiser), ("h3", GroupHull.Carrier), ("h4", (GroupHull)99) }) + await PutLeaf(cache, key, key, null, hull); + var model = GroupModel(); + + Assert.That(model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments).Single().Children.Select(group => group.Label), + Is.EqualTo(new[] { "Cruiser", "Carrier", "Frigate", "99" }), "by underlying value (10, 20, 30, 99), not by name"); + + foreach (var (key, kind, count, flag) in new[] + { + ("m1", "b", 10, true), ("m2", "A", 9, false), ("m3", "a", 9, true), ("m4", "", -1, false), + ("m5", "a", 10, true), ("m6", "a", 10, false), ("m7", "a", -1, true), ("m8", "B", 10, true) + }) + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = kind, Count = count, Flag = flag }, new CultRecordKey(key)); + Assert.That(Dump(model.GroupRecords(typeof(GroupMixed), cache.AllStoredDocuments)), Is.EqualTo(new[] + { + "(empty) (1)", " -1 (1)", " False (1): m4", + "A (1)", " 9 (1)", " False (1): m2", + "a (4)", " -1 (1)", " True (1): m7", " 9 (1)", " True (1): m3", " 10 (2)", " False (1): m6", " True (1): m5", + "B (1)", " 10 (1)", " True (1): m8", + "b (1)", " 10 (1)", " True (1): m1" + }), "strings ignore case then compare exactly, integers order numerically, false before true"); + } + + [Test] + public async Task GroupNodeIdsStayDistinctWhateverTheValuesContain() + { + using var cache = OpenGroups(); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = "x/Count=1/Flag=True" }, new CultRecordKey("a")); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = "x", Count = 1, Flag = true }, new CultRecordKey("b")); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = "aCount0" }, new CultRecordKey("c")); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = "a" }, new CultRecordKey("d")); + + var ids = new List(); + void Collect(IEnumerable groups) + { + foreach (var group in groups) + { + ids.Add(group.Id); + Collect(group.Children); + } + } + + Collect(GroupModel().GroupRecords(typeof(GroupMixed), cache.AllStoredDocuments)); + + Assert.That(ids, Has.Count.EqualTo(12)); + Assert.That(ids.Distinct().Count(), Is.EqualTo(12), "a path never spells another node's id"); + } + + [Test] + public async Task GroupNodeIdsSeparateAnEscapeFromTheTextItSpells() + { + // "a/b" and "a%2Fb" collide under an id that escapes '/' alone; the unpaired surrogates, U+FFFD and "a%uD800" + // are the values a replacement character or a sloppy %u form would merge; the valid pair is one node, not two halves. + var kinds = new[] { "a/b", "a%2Fb", "a%2fb", "a%252Fb", "a", "A", "\uD800", "\uD801", "\uDC00", "�", "a%uD800", "a\uD800", "😀", "�\uDE00", "\uDE00\uD83D", "a\uD800b\uD801c" }; + using var cache = OpenGroups(); + for (var i = 0; i < kinds.Length; i++) + await cache.UpsertAsync(typeof(GroupRedeclared), new GroupRedeclared { Kind = kinds[i] }, new CultRecordKey("k" + i)); + + var roots = GroupModel().GroupRecords(typeof(GroupRedeclared), cache.AllStoredDocuments); + + Assert.That(roots, Has.Count.EqualTo(kinds.Length), "every distinct value is its own node, 'a' and 'A' included"); + Assert.That(roots.Select(group => group.Id).Distinct().Count(), Is.EqualTo(kinds.Length), "and none spells another's id"); + Assert.That(roots.Select(group => group.Label).Where(label => label == "a" || label == "A"), Is.EqualTo(new[] { "A", "a" }), + "labels equal ignoring case fall back to ordinal order"); + } + + [Test] + public async Task GroupRefsWhoseKeysDifferOnlyByCaseAreTwoNodes() + { + using var cache = OpenGroups(); + await cache.UpsertAsync(typeof(InspectOther), new InspectOther { Name = "lower" }, new CultRecordKey("o-a")); + await cache.UpsertAsync(typeof(InspectOther), new InspectOther { Name = "upper" }, new CultRecordKey("O-A")); + await PutLeaf(cache, "r1", "one", "o-a", GroupHull.Cruiser); + await PutLeaf(cache, "r2", "two", "O-A", GroupHull.Cruiser); + var model = GroupModel(); + + var roots = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments); + + Assert.That(roots.Select(group => group.Label), Is.EqualTo(new[] { "lower", "upper" }), "each key finds its own target"); + Assert.That(roots.Select(group => group.Id).Distinct().Count(), Is.EqualTo(2)); + Assert.That(model.RecordRefLabel(typeof(CultRecordRef), OwnerRef("O-A"), cache.AllStoredDocuments), Is.EqualTo("upper")); + } + + [Test] + public async Task GroupAnUngroupedListedTypeHasNoTreeAndItsCandidatesStayFlat() + { + using var cache = OpenGroups(); + await cache.UpsertAsync(typeof(GroupGlobal), new GroupGlobal { Kind = "k" }, new CultRecordKey("global")); + var model = GroupModel(); + + var grouping = model.GroupingOf(typeof(GroupGlobal)); + + Assert.That(grouping.Members, Is.Empty); + Assert.That(grouping.Notice, Is.Not.Null, "a refused declaration still says why"); + Assert.That(model.GroupRecords(typeof(GroupGlobal), cache.AllStoredDocuments), Is.Empty, "no tree: the lowering lists candidates flat"); + Assert.That(model.RecordCandidates(typeof(CultRecordRef), cache.AllStoredDocuments).Select(record => record.Key.Value), + Is.EqualTo(new[] { "global" })); + } + + [Test] + public async Task GroupCreateInGroupRoundTripsIntoTheSameNode() + { + using var cache = await SeededGroups(); + await PutLeaf(cache, "r5", "ghosted", "ghost", GroupHull.Carrier); + var model = GroupModel(); + var groups = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments); + var alpha = groups.Single(group => group.Label == "Alpha"); + var alphaCruiser = alpha.Children.Single(group => group.Label == "Cruiser"); + var ghost = groups.Single(group => group.Label == "Missing ghost"); + + var deep = (GroupLeaf)model.CreateInGroup(typeof(GroupLeaf), alphaCruiser, out var notice)!; + Assert.That(notice, Is.Null); + Assert.That(CultInspectorModel.RecordKey(deep.Owner), Is.EqualTo("o-a")); + Assert.That(deep.Hull, Is.EqualTo(GroupHull.Cruiser)); + var shallow = (GroupLeaf)model.CreateInGroup(typeof(GroupLeaf), alpha, out _)!; + Assert.That(CultInspectorModel.RecordKey(shallow.Owner), Is.EqualTo("o-a")); + var dangling = (GroupLeaf)model.CreateInGroup(typeof(GroupLeaf), ghost, out _)!; + Assert.That(CultInspectorModel.RecordKey(dangling.Owner), Is.EqualTo("ghost"), "a missing node presets the dangling key"); + + await cache.UpsertAsync(typeof(GroupLeaf), deep, new CultRecordKey("deep")); + await cache.UpsertAsync(typeof(GroupLeaf), shallow, new CultRecordKey("shallow")); + await cache.UpsertAsync(typeof(GroupLeaf), dangling, new CultRecordKey("dangling")); + var after = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments); + + Assert.That(Containing(after, "deep").Select(group => group.Id), Is.EqualTo(new[] { alpha.Id, alphaCruiser.Id })); + Assert.That(Containing(after, "shallow").Select(group => group.Id), Does.Contain(alpha.Id)); + Assert.That(Containing(after, "dangling").Select(group => group.Id), Does.Contain(ghost.Id)); + } + + [Test] + public async Task GroupCreateInGroupRefusesANodeOfAnotherGrouping() + { + using var cache = await SeededGroups(); + var model = GroupModel(); + var alphaCruiser = model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments)[0].Children[0]; + + Assert.That(() => model.CreateInGroup(typeof(GroupRedeclared), alphaCruiser, out _), Throws.ArgumentException, "a deeper path than the grouping has"); + Assert.That(() => model.CreateInGroup(typeof(GroupLeaf), null!, out _), Throws.ArgumentNullException); + + await cache.UpsertAsync(typeof(GroupRedeclared), new GroupRedeclared { Kind = "k" }, new CultRecordKey("g1")); + var kindNode = model.GroupRecords(typeof(GroupRedeclared), cache.AllStoredDocuments).Single(); + Assert.That(() => model.CreateInGroup(typeof(GroupTagged), kindNode, out _), Throws.ArgumentException, "a value that would fit another grouping's member"); + } + + [Test] + public async Task GroupCreateInGroupReportsATypeItCannotCreate() + { + using var cache = OpenGroups(); + await cache.UpsertAsync(typeof(GroupNoCtor), new GroupNoCtor("n") { Kind = "k" }, new CultRecordKey("c1")); + var model = GroupModel(); + var node = model.GroupRecords(typeof(GroupNoCtor), cache.AllStoredDocuments).Single(); + + Assert.That(model.CreateInGroup(typeof(GroupNoCtor), node, out var notice), Is.Null); + Assert.That(notice, Does.Contain("parameterless constructor")); + } + + [Test] + public void GroupATypeWithoutADeclarationIsNotGrouped() + { + var model = GroupModel(); + + var grouping = model.GroupingOf(typeof(InspectOther)); + + Assert.That(grouping.ListedType, Is.EqualTo(typeof(InspectOther))); + Assert.That(grouping.Members, Is.Empty); + Assert.That(grouping.Notice, Is.Null); + Assert.That(model.GroupRecords(typeof(InspectOther), Array.Empty()), Is.Empty); + Assert.That(() => model.GroupingOf(null!), Throws.ArgumentNullException); + } + + [Test] + public async Task GroupANullStringSharesTheEmptyStringsNode() + { + using var cache = OpenGroups(); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = null! }, new CultRecordKey("n")); + await cache.UpsertAsync(typeof(GroupMixed), new GroupMixed { Kind = "" }, new CultRecordKey("e")); + + var roots = GroupModel().GroupRecords(typeof(GroupMixed), cache.AllStoredDocuments); + + Assert.That(roots.Select(group => group.Label), Is.EqualTo(new[] { "(empty)" })); + Assert.That(roots.Single().Count, Is.EqualTo(2)); + } + + [TestCase(typeof(GroupUnknown), "Nope")] + [TestCase(typeof(GroupFloat), "Mass")] + [TestCase(typeof(GroupFlagsEnum), "Traits")] + [TestCase(typeof(GroupHiddenMember), "Secret")] + [TestCase(typeof(GroupReadOnlyMember), "Locked")] + [TestCase(typeof(GroupDuplicate), "Kind")] + [TestCase(typeof(GroupValidThenInvalid), "Mass")] + [TestCase(typeof(GroupGlobal), "global")] + public void GroupInvalidDeclarationsGiveANoticeAndAFlatList(Type listed, string mentions) + { + var model = GroupModel(); + + var grouping = model.GroupingOf(listed); + + Assert.That(grouping.Members, Is.Empty); + Assert.That(grouping.Notice, Does.Contain(mentions).And.Contain("CultInspectorGroupBy").And.Contain("not grouped")); + Assert.That(model.GroupRecords(listed, Array.Empty()), Is.Empty); + } + + [Test] + public async Task GroupAVariantGroupsByItsResolvedValues() + { + using var cache = OpenGroups(); + var baseKey = new CultRecordKey("base"); + await PutLeaf(cache, "base", "base", null, GroupHull.Cruiser); + await cache.UpsertVariantAsync(new CultRecordKey("over"), baseKey, new[] { cache.Override(nameof(GroupLeaf.Name), "over-name"), cache.Override(nameof(GroupLeaf.Hull), GroupHull.Frigate) }); + await cache.UpsertVariantAsync(new CultRecordKey("inherit"), baseKey, new[] { cache.Override(nameof(GroupLeaf.Name), "inheritor") }); + var model = GroupModel(); + + Assert.That(Dump(model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments)), Is.EqualTo(new[] + { + "None (3)", " Cruiser (2): base,inherit", " Frigate (1): over" + })); + + await PutLeaf(cache, "base", "base", null, GroupHull.Carrier); + Assert.That(Dump(model.GroupRecords(typeof(GroupLeaf), cache.AllStoredDocuments)), Is.EqualTo(new[] + { + "None (3)", " Carrier (2): base,inherit", " Frigate (1): over" + }), "an inheriting variant follows its base; an overriding one stays"); + } + [CultDocument("tests.inspect_item", "tests.inspect_item.v1")] [MessagePackObject] public sealed class InspectItem @@ -330,6 +748,28 @@ namespace GameCult.Caching.Tests public string Name = string.Empty; } + public abstract class InspectLabeledBase + { + [Key(1)] [CultInspectorLabel("Base Label")] + public abstract string Tag { get; set; } + } + + [CultDocument("tests.inspect_labeled", "tests.inspect_labeled.v1")] + [MessagePackObject] + public sealed class InspectLabeled : InspectLabeledBase + { + [Key(0)] [CultName] + public string Name = string.Empty; + + [Key(1)] + public override string Tag { get; set; } = string.Empty; + } + + [CultInspectorDrawer(typeof(CultInspectorLabelAttribute))] + public sealed class LabelClaimDrawer : IFakeDrawer + { + } + [CultDocument("tests.inspect_asset", "tests.inspect_asset.v1")] [MessagePackObject] public sealed class InspectAsset @@ -428,6 +868,163 @@ namespace GameCult.Caching.Tests public override string ToString() => "key"; } + public enum GroupHull + { + Frigate = 30, + Cruiser = 10, + Carrier = 20 + } + + [Flags] + public enum GroupTraits + { + Fast = 1, + Armored = 2 + } + + [CultInspectorGroupBy(nameof(GroupLeaf.Owner), nameof(GroupLeaf.Hull))] + public abstract class GroupBase + { + } + + [CultDocument("tests.group_leaf", "tests.group_leaf.v1")] + [MessagePackObject] + public sealed class GroupLeaf : GroupBase + { + [Key(0)] [CultName] + public string Name = string.Empty; + + [Key(1)] + public GroupHull Hull; + + [Key(2)] + public CultRecordRef Owner; + } + + [CultInspectorGroupBy("Kind")] + [CultDocument("tests.group_redeclared", "tests.group_redeclared.v1")] + [MessagePackObject] + public sealed class GroupRedeclared : GroupBase + { + [Key(0)] + public string Kind = string.Empty; + } + + [CultInspectorGroupBy] + [CultDocument("tests.group_opt_out", "tests.group_opt_out.v1")] + [MessagePackObject] + public sealed class GroupOptOut : GroupBase + { + [Key(0)] + public string Kind = string.Empty; + } + + [CultInspectorGroupBy("Kind", "Count", "Flag")] + [CultDocument("tests.group_mixed", "tests.group_mixed.v1")] + [MessagePackObject] + public sealed class GroupMixed + { + [Key(0)] + public string Kind = string.Empty; + + [Key(1)] + public int Count; + + [Key(2)] + public bool Flag; + } + + [CultInspectorGroupBy("Kind")] + [CultGlobal] + [CultDocument("tests.group_global", "tests.group_global.v1")] + [MessagePackObject] + public sealed class GroupGlobal + { + [Key(0)] + public string Kind = string.Empty; + } + + [CultInspectorGroupBy("Tag")] + [CultDocument("tests.group_tagged", "tests.group_tagged.v1")] + [MessagePackObject] + public sealed class GroupTagged + { + [Key(0)] + public string Tag = string.Empty; + } + + [CultInspectorGroupBy("Kind")] + [CultDocument("tests.group_no_ctor", "tests.group_no_ctor.v1")] + [MessagePackObject] + public sealed class GroupNoCtor + { + [SerializationConstructor] + public GroupNoCtor(string name) + { + Name = name; + } + + [Key(0)] + public string Name { get; } + + [Key(1)] + public string Kind = string.Empty; + } + + [MessagePackObject] + public class GroupShape + { + [Key(0)] + public string Kind = string.Empty; + + [Key(1)] + public float Mass; + + [Key(2)] + public GroupTraits Traits; + + [Key(3)] [CultInspectorHidden] + public int Secret; + + [Key(4)] [CultInspectorReadOnly] + public int Locked; + } + + [CultInspectorGroupBy("Nope")] + public sealed class GroupUnknown : GroupShape + { + } + + [CultInspectorGroupBy("Mass")] + public sealed class GroupFloat : GroupShape + { + } + + [CultInspectorGroupBy("Traits")] + public sealed class GroupFlagsEnum : GroupShape + { + } + + [CultInspectorGroupBy("Secret")] + public sealed class GroupHiddenMember : GroupShape + { + } + + [CultInspectorGroupBy("Locked")] + public sealed class GroupReadOnlyMember : GroupShape + { + } + + [CultInspectorGroupBy("Kind", "Kind")] + public sealed class GroupDuplicate : GroupShape + { + } + + [CultInspectorGroupBy("Kind", "Mass")] + public sealed class GroupValidThenInvalid : GroupShape + { + } + public interface IFakeDrawer { }