Add lean optimizer replacement map
This commit is contained in:
Meta Vens
parent
863b518e1e
commit
8aee563d4a
4 changed files with 397 additions and 0 deletions
|
|
@ -65,6 +65,7 @@ Current tile-graph status:
|
|||
- the next relax cut also paid off: removing the old relax-mode bonus while keeping the core relax stage improved the same pinned badge smoke again from `0.07485` to `0.07451`
|
||||
- the optimizer now emits stage displacement diagnostics too, so `run.json` can show what snap, relax handoff, relax mode, and final refine actually did to the per-cell source-pixel displacement field instead of just reporting a single fidelity score
|
||||
- those new diagnostics also settled the design argument about relax: on the pinned badge check under `artifacts/optimizer-relax-purpose-check/`, relax improves source-fidelity a bit (`0.07549` without relax to `0.07451` with it), but it does not act like a strong broad-swatch phase smoother; the displacement field stays roughly as jittery, so the current relax stage is better described as a soft consensus / handoff stabilizer over fixed local candidates
|
||||
- that diagnosis finally forced the obvious architectural verdict: the current optimizer is still a tray-based discrete chooser in a fake moustache, so the repo now also carries `docs/lean-optimizer-algorithm-map.md`, a replacement target for a real displacement-field optimizer with one field, one objective, and one final sample
|
||||
- after the first pruning pass, tile-graph no longer participates in pipeline phase-rerank probes and no longer carries hybrid geometry priors through its unary cost; the path is now one lattice-conditioned candidate generator plus one local discrete solver
|
||||
- this fixes the core design mismatch that had allowed repeated distant labels to create big same-color patches and opaque black background blocks
|
||||
- on the current `24x24` emblem smoke case, an end-to-end `tile-graph` run dropped from about `2.57s` on CPU to `0.61s` on CUDA on this machine
|
||||
|
|
@ -146,6 +147,7 @@ That keeps benchmark assets, attribution exports, and tuning runs from polluting
|
|||
- `docs/implementation-plan.md`: working roadmap
|
||||
- `docs/tile-graph-algorithm-map.md`: detailed tile-graph dataflow and failure-point map
|
||||
- `docs/optimizer-algorithm-map.md`: detailed optimizer dataflow and contradiction map
|
||||
- `docs/lean-optimizer-algorithm-map.md`: replacement target for the stripped-down displacement-field optimizer
|
||||
- `examples/corpus/README.md`: local corpus layout and attribution workflow
|
||||
|
||||
Core modules:
|
||||
|
|
|
|||
|
|
@ -170,6 +170,45 @@ That means the next optimizer cuts should treat `relax` as a measurable subsolve
|
|||
|
||||
The next optimizer simplification pass should use that map as its cutting checklist instead of guessing from scattered helper names.
|
||||
|
||||
### Lean optimizer restart target
|
||||
|
||||
The next real optimizer move is not another incremental trim on the old tray-based solver. It is a restart toward the smaller machine now mapped in `docs/lean-optimizer-algorithm-map.md`.
|
||||
|
||||
What that new map says plainly:
|
||||
|
||||
- keep lattice inference and edge scouting
|
||||
- keep one fixed grid of cell centers
|
||||
- attach one displacement vector `(dx, dy)` to each output cell, initialized to zero
|
||||
- optimize that displacement field directly
|
||||
- use a tiny objective with only:
|
||||
- local solidity / coherence
|
||||
- smoothness
|
||||
- edge-aware smoothness
|
||||
- anti-collapse / anti-fold constraints
|
||||
- displacement magnitude regularization
|
||||
- sample the final output once from the optimized field
|
||||
|
||||
What that restart explicitly rejects:
|
||||
|
||||
- `representative_t`
|
||||
- `source_lattice_reference`
|
||||
- `source_reliability_t`
|
||||
- tray-based snap / relax / refine as the core optimizer state
|
||||
- repeated adjacency / motif / line dialects trying to compensate for each other
|
||||
|
||||
Why this is the right next cut:
|
||||
|
||||
- the current optimizer map now makes the contradiction impossible to ignore: the code is not really optimizing phase directly anymore
|
||||
- the relax diagnostics proved that one of the old justifications was overstated; the machine gets a tiny score win from relax, but not the promised broad-swatch phase drift
|
||||
- the user's target description is cleaner than the current code: find cell size, initialize a phase field at zero, wiggle it into solid areas coherently, sample, done
|
||||
|
||||
Next implementation steps:
|
||||
|
||||
1. create a brand-new optimizer module beside `continuous.py`
|
||||
2. wire it up behind an experimental reconstruction mode or internal switch
|
||||
3. start with only the five objective terms listed above
|
||||
4. compare it against pinned badge and emblem runs before porting any old heuristics
|
||||
|
||||
## High-value regression case
|
||||
|
||||
### AI badge emblem with baked checkerboard background
|
||||
|
|
|
|||
352
docs/lean-optimizer-algorithm-map.md
Normal file
352
docs/lean-optimizer-algorithm-map.md
Normal file
|
|
@ -0,0 +1,352 @@
|
|||
# Lean Optimizer Replacement Map
|
||||
|
||||
## Why this map exists
|
||||
|
||||
The current optimizer is useful, but it is also a fussy little parliament.
|
||||
|
||||
Too many portraits. Too many local trays. Too many side negotiations about adjacency, motif, line, handoff, guardrails, and whether the machine still trusts its own decisions.
|
||||
|
||||
This document is not a map of the code we have.
|
||||
|
||||
It is a map of the optimizer we should replace it with:
|
||||
|
||||
`infer one lattice -> attach one displacement vector to each output cell -> nudge those vectors toward solid source regions while keeping the field coherent -> sample once`
|
||||
|
||||
That is the whole dream. No watercolor portrait. No accountant's ledger. No snap religion and refine religion living in the same cathedral. Just one field, one objective, one final sample.
|
||||
|
||||
## One-sentence machine
|
||||
|
||||
The lean optimizer should be:
|
||||
|
||||
`source image -> edge scout -> fixed lattice centers -> zero displacement field -> direct phase-field optimization -> nearest source sampling -> done`
|
||||
|
||||
Or, in pictures:
|
||||
|
||||
- the source image is the mural
|
||||
- the inferred lattice is the graph paper
|
||||
- the displacement field is each graph-paper cell deciding how far to slide before it bites into real paint
|
||||
- the objective is trying to make those slides land inside solid fake-pixel regions without the whole field folding into a heap
|
||||
|
||||
## How to read this machine
|
||||
|
||||
The replacement optimizer should carry only four important pieces of cargo:
|
||||
|
||||
- `uv0`: the fixed lattice centers chosen by inference
|
||||
- `d`: one displacement vector `(dx, dy)` per output cell, initialized to zero
|
||||
- `edge_map`: one edge scout report for the source image
|
||||
- `sampled_rgba`: the current source color under `uv0 + d`
|
||||
|
||||
Everything else is a term in the objective, not a new ontology.
|
||||
|
||||
If a future version introduces more than those four core state variables, it should have to explain itself like a defendant.
|
||||
|
||||
## What survives from the current code
|
||||
|
||||
These existing pieces are still worth keeping:
|
||||
|
||||
- lattice inference in `src/repixelizer/inference.py`
|
||||
- edge analysis in `src/repixelizer/analysis.py`
|
||||
- source-lattice consistency metrics in `src/repixelizer/metrics.py`
|
||||
- diagnostics and compare-mode output in `src/repixelizer/pipeline.py` and `src/repixelizer/diagnostics.py`
|
||||
- the current optimizer in `src/repixelizer/continuous.py` as a benchmark and fallback during the replacement
|
||||
|
||||
These pieces should **not** be part of the new core:
|
||||
|
||||
- `representative_t`
|
||||
- `source_lattice_reference`
|
||||
- `source_reliability_t`
|
||||
- snap candidate trays
|
||||
- relax candidate probabilities
|
||||
- greedy refine candidate trays
|
||||
- the current layered boundary / motif / line dialect stack
|
||||
|
||||
The new machine should optimize one thing directly: the displacement field.
|
||||
|
||||
## Stage 0: The pipeline picks the ruler
|
||||
|
||||
Source that survives:
|
||||
|
||||
- `run_pipeline(...)` in `src/repixelizer/pipeline.py`
|
||||
- `infer_lattice(...)` and `infer_fixed_lattice(...)` in `src/repixelizer/inference.py`
|
||||
|
||||
Meaning:
|
||||
|
||||
The new optimizer does **not** get to argue about target size or global phase after the pipeline hands it the ruler.
|
||||
|
||||
The ruler still decides:
|
||||
|
||||
- output width and height
|
||||
- the initial phase offset
|
||||
- the initial cell spacing in source pixels
|
||||
|
||||
That part of the machine is already a separate concern and should stay that way.
|
||||
|
||||
### Metaphor
|
||||
|
||||
The survey crew hammers the stakes in once.
|
||||
After that, the new optimizer is not a politician. It is a field crew. It works inside the stakes or it goes home.
|
||||
|
||||
## Stage 1: Lay down the fixed lattice centers
|
||||
|
||||
Current source to reuse:
|
||||
|
||||
- `_make_regular_uv(...)` in `src/repixelizer/continuous.py`
|
||||
|
||||
New state:
|
||||
|
||||
- `uv0[y, x]`: the base source-space center for each output cell
|
||||
|
||||
Meaning:
|
||||
|
||||
This is the unmoving graph paper.
|
||||
|
||||
Unlike the current optimizer, the new machine should be honest about what is fixed and what is variable:
|
||||
|
||||
- `uv0` is fixed
|
||||
- `d` is the thing being optimized
|
||||
|
||||
There should be no fake "UV optimizer" language once that distinction exists.
|
||||
|
||||
### Metaphor
|
||||
|
||||
This is the empty skeleton of the machine.
|
||||
The graph paper is nailed to the mural. Nothing clever has happened yet. Each cell is just a little pinned square waiting to decide how much to slide before it commits.
|
||||
|
||||
## Stage 2: Initialize one displacement vector per cell
|
||||
|
||||
New state:
|
||||
|
||||
- `d[y, x] = (0, 0)` initially
|
||||
|
||||
Meaning:
|
||||
|
||||
Every output cell starts by targeting the exact center of its inferred source cell.
|
||||
|
||||
That is the null hypothesis:
|
||||
|
||||
- maybe the inferred phase is already fine
|
||||
- maybe some regions need to drift
|
||||
|
||||
The machine starts from stillness and earns every movement.
|
||||
|
||||
### Metaphor
|
||||
|
||||
Every cell begins with its feet planted under its shoulders.
|
||||
No lunging. No mystical premonitions about where it "really wants" to go. Just a row of workers standing still on the scaffold until the foreman starts shouting.
|
||||
|
||||
## Stage 3: Sample the source at the displaced centers
|
||||
|
||||
Derived state:
|
||||
|
||||
- `uv = uv0 + d`
|
||||
- `sampled_rgba[y, x]`
|
||||
|
||||
Meaning:
|
||||
|
||||
At any moment, each output cell points at one source location:
|
||||
|
||||
- start from the fixed center
|
||||
- add the current displacement
|
||||
- sample the source there
|
||||
|
||||
This is the only color the cell currently "believes in."
|
||||
|
||||
There should be no alternate portrait layer describing what the cell vaguely feels like. The sample is the truth of the current hypothesis.
|
||||
|
||||
### Metaphor
|
||||
|
||||
Each worker shines a laser pointer from the scaffold down onto the mural.
|
||||
Where the dot lands, that is the color they are claiming. No watercolor sketch. No emotional support ledger. Just a dot on paint.
|
||||
|
||||
## Stage 4: Compute the local evidence term
|
||||
|
||||
New objective family:
|
||||
|
||||
- prefer displacement targets that land inside solid or internally coherent local regions
|
||||
|
||||
Candidate implementation options worth testing:
|
||||
|
||||
- low local variance in a tiny neighborhood around `uv`
|
||||
- low edge energy in a tiny neighborhood around `uv`
|
||||
- high agreement between the center sample and nearby samples inside a fractional cell window
|
||||
|
||||
Meaning:
|
||||
|
||||
This is the term that tries to make each cell land inside a fake-pixel interior instead of on a messy boundary.
|
||||
|
||||
The purpose is simple:
|
||||
|
||||
- solid colored areas are safe
|
||||
- high-conflict mixed areas are dangerous
|
||||
|
||||
This term is the main answer to "why move at all?"
|
||||
|
||||
### Metaphor
|
||||
|
||||
The worker is probing the mural with a boot before putting full weight down.
|
||||
Some spots are firm stone. Some are crumbling ledges. This term tells the field to step onto the stable bits instead of balancing on the edge of a crack like an idiot.
|
||||
|
||||
## Stage 5: Compute coherence and topology terms
|
||||
|
||||
New objective family:
|
||||
|
||||
- neighboring cells should have similar displacements unless a real edge justifies a break
|
||||
- nearby output cells should not collapse onto the same source pixel
|
||||
- the field should not fold, cross, or invert local order
|
||||
- displacement magnitude should stay modest unless the data term really wants motion
|
||||
|
||||
Meaning:
|
||||
|
||||
This is where the optimizer stops being a mob of independent little boot-probes.
|
||||
|
||||
These terms are what make the displacement field a field:
|
||||
|
||||
- smoothness keeps neighboring cells moving together
|
||||
- edge-aware gating prevents smoothness from smearing through sharp source boundaries
|
||||
- anti-collapse / anti-fold constraints stop the grid from crumpling into duplicated samples
|
||||
- magnitude regularization keeps the field from wandering off because one local patch looked briefly tempting
|
||||
|
||||
This is the actual home for the design goal that `relax` only half fulfilled.
|
||||
|
||||
### Metaphor
|
||||
|
||||
Now imagine the workers are connected by springs, but the springs are smart enough to loosen at real cracks in the wall.
|
||||
They can drift together like a sheet, but they are not allowed to pile into the same footprint or climb over each other like rats in a drain.
|
||||
|
||||
## Stage 6: Update the displacement field directly
|
||||
|
||||
New core operation:
|
||||
|
||||
- optimize `d` itself, not a tray of discrete local candidate picks
|
||||
|
||||
Candidate solver shapes worth trying:
|
||||
|
||||
- gradient descent or Adam on a differentiable approximation
|
||||
- projected gradient steps with explicit clamp / anti-fold projection
|
||||
- alternating local smoothing plus data-term descent
|
||||
|
||||
Hard invariants:
|
||||
|
||||
- `d` remains bounded
|
||||
- local topology constraints are enforced every iteration
|
||||
- edge-aware coherence remains part of the update, not a decorative afterthought
|
||||
|
||||
Meaning:
|
||||
|
||||
This is the heart of the replacement.
|
||||
|
||||
The field should move directly under one unified objective, not by:
|
||||
|
||||
- inventing a portrait
|
||||
- turning it into candidate trays
|
||||
- softening the trays
|
||||
- hardening the trays
|
||||
- comparing against a previous tray-based religion
|
||||
|
||||
If the optimizer exists, it should optimize.
|
||||
|
||||
### Metaphor
|
||||
|
||||
This is finally the graph paper itself breathing and sliding over the mural.
|
||||
Not stones being swapped in and out of trays. Not committees debating motifs. The sheet moves, the springs pull, the cracks resist, the dots settle.
|
||||
|
||||
## Stage 7: Quantize to final source pixels
|
||||
|
||||
Final sampling step:
|
||||
|
||||
- convert `uv0 + d` to source sampling positions
|
||||
- round or nearest-sample to the final source pixel for each output cell
|
||||
- emit `target_rgba`
|
||||
|
||||
Meaning:
|
||||
|
||||
Once the field has settled, the image comes from one final honest sampling step.
|
||||
|
||||
This is where the current machine keeps cheating by spending most of its life as a discrete chooser. The replacement should do the opposite:
|
||||
|
||||
- optimize a field first
|
||||
- discretize once at the end
|
||||
|
||||
That keeps the algorithm aligned with its own story.
|
||||
|
||||
### Metaphor
|
||||
|
||||
After all the sliding and settling, each worker plants the flag once.
|
||||
No more hedging. No more "best of snap and refine." Just one final stamp where the field says the cell belongs.
|
||||
|
||||
## Stage 8: Evaluate against reality
|
||||
|
||||
Source to keep:
|
||||
|
||||
- `source_lattice_consistency_breakdown(...)` in `src/repixelizer/metrics.py`
|
||||
- compare-mode diagnostics in `src/repixelizer/pipeline.py`
|
||||
|
||||
Meaning:
|
||||
|
||||
The new machine still needs adult supervision.
|
||||
|
||||
But the supervision should judge one coherent pipeline, not referee a knife fight between three subsolvers. The important diagnostics for the replacement should be:
|
||||
|
||||
- source-lattice fidelity
|
||||
- displacement magnitude statistics
|
||||
- displacement smoothness / jitter
|
||||
- topology violations prevented or projected away
|
||||
- maybe edge-crossing counts if we make that measurable
|
||||
|
||||
If the replacement loses badly to the old optimizer, we keep the old optimizer. No romance.
|
||||
|
||||
### Metaphor
|
||||
|
||||
This is the inspector with the clipboard, but now the questions are cleaner.
|
||||
Did the field settle into real cell interiors? Did the sheet drift coherently? Did it fold? Did it cheat? Did the result actually look less stupid?
|
||||
|
||||
## What fits this machine
|
||||
|
||||
These things belong:
|
||||
|
||||
- one fixed inferred ruler
|
||||
- one displacement field
|
||||
- one direct objective
|
||||
- edge-aware smoothness
|
||||
- anti-collapse / anti-fold constraints
|
||||
- one final discrete sample
|
||||
|
||||
Those all speak the same language.
|
||||
|
||||
## What does not fit this machine
|
||||
|
||||
These things should be treated as suspicious imports from the old religion:
|
||||
|
||||
- multiple competing portraits of the same cell
|
||||
- separate snap / relax / refine solver identities
|
||||
- repeated adjacency / motif / line stacks in different stages
|
||||
- candidate trays as the main state of the optimizer
|
||||
- any scoring term that exists mainly to compensate for another scoring term
|
||||
|
||||
If one of those comes back, it should have to win a public trial.
|
||||
|
||||
## The current replacement plan
|
||||
|
||||
The lean optimizer should be built beside the current one, not by mutating the old whale in place.
|
||||
|
||||
Recommended build order:
|
||||
|
||||
1. Reuse lattice inference and edge scouting.
|
||||
2. Create a new displacement-field optimizer module.
|
||||
3. Implement the smallest viable objective:
|
||||
- local solidity
|
||||
- smoothness
|
||||
- edge-aware smoothness
|
||||
- anti-collapse
|
||||
- displacement magnitude prior
|
||||
4. Add diagnostics for displacement magnitude, smoothness, and topology health.
|
||||
5. Compare against the current optimizer on pinned badge and emblem cases.
|
||||
6. Only port any old "fancy" idea if it clearly improves the real outputs.
|
||||
|
||||
## The diamond test
|
||||
|
||||
The replacement is worth keeping only if it passes this crude, necessary test:
|
||||
|
||||
Can a new reader describe the whole machine without sounding like they are defending a tax code?
|
||||
|
||||
If not, there is still more to cut.
|
||||
|
|
@ -12,6 +12,10 @@ When it fails, this map is here to answer:
|
|||
|
||||
Where did the machine start negotiating with itself too much?
|
||||
|
||||
This is a map of the **current** optimizer in `src/repixelizer/continuous.py`.
|
||||
|
||||
If you want the replacement target instead of the autopsy, read `docs/lean-optimizer-algorithm-map.md` too. That companion map describes the smaller displacement-field machine this repo should grow next.
|
||||
|
||||
## One-sentence machine
|
||||
|
||||
The current optimizer machine is:
|
||||
|
|
|
|||
Loading…
Reference in a new issue