GameCult/repixelizer
Watch
9
0
Fork
You've already forked repixelizer
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Make optimizer refine source-first

Browse source
This commit is contained in:
Meta Vens 2026-04-23 17:54:46 +01:00
commit 69cc05e6c2
6 changed files with 29 additions and 44 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -61,6 +61,7 @@ Current tile-graph status:
- that speed pass is a real throughput win but a small quality regression: the same pinned badge case moves from `0.1814` to `0.2036` source-fidelity
- the optimizer now has its own deep-dive map in `docs/optimizer-algorithm-map.md`, and the first cut from that map removed the old UV-optimizer name plus the continuous k-means cluster-boundary scout; the continuous path is now a fixed-lattice, source-first discrete chooser guided by real edge evidence
- optimizer preparation is now split into a private `_OptimizerPrep` bundle, so the continuous path has a visible seam between "build the map" and "choose the pixels"
- refine is now source-first too: the representative portrait still stabilizes snap, but refine and final structure choice no longer consult it, and the pinned badge smoke improved slightly from `0.07535` to `0.07485`
- after the first pruning pass, tile-graph no longer participates in pipeline phase-rerank probes and no longer carries hybrid geometry priors through its unary cost; the path is now one lattice-conditioned candidate generator plus one local discrete solver
- this fixes the core design mismatch that had allowed repeated distant labels to create big same-color patches and opaque black background blocks
- on the current `24x24` emblem smoke case, an end-to-end `tile-graph` run dropped from about `2.57s` on CPU to `0.61s` on CUDA on this machine

9
docs/implementation-plan.md
View file

@ -130,9 +130,16 @@ The second optimizer cut is structural:
Next after that:
- put the representative portrait on trial in refine
- add a focused ablation runner for pinned badge/emblem cases before cutting more behavior
- collapse duplicated adjacency / motif / line voices only after ablation shows which stage owns each idea
- put relax on trial now that refine itself is source-first
The third optimizer cut is behavioral:
- refine no longer scores candidates against the representative portrait
- `_structure_score(...)` no longer includes representative-match as a final tiebreaker
- the dead refine/structure representative knobs were removed from `SolverHyperParams` and the tuning harness
- the pinned cleaned-badge continuous smoke run under `artifacts/optimizer-cut-v3-source-refine-badge-126/` improved slightly: snap stayed `0.08100`, final moved from `0.07535` to `0.07485`
The next optimizer simplification pass should use that map as its cutting checklist instead of guessing from scattered helper names.

20
docs/optimizer-algorithm-map.md
View file

@ -16,7 +16,7 @@ Where did the machine start negotiating with itself too much?
The current optimizer machine is:
`source image -> edge scout -> regular UV grid -> soft representative portrait + hard source lattice portrait -> source-first snap -> relaxed candidate field -> greedy discrete refine -> keep snap if refine made things worse`
`source image -> edge scout -> regular UV grid -> soft representative portrait + hard source lattice portrait -> source-first snap -> source-first refine -> keep snap if refine made things worse`
Or, in pictures:
@ -320,7 +320,7 @@ So the refine stage can reach sharper alternatives than the original snap neighb
Each candidate now accumulates several costs:
- anchor match: how far it strays from the snapped output
- representative/source match: how well it fits the blended portrait story
- source match: how well it fits the hard source portrait
- alpha match
- distance from the UV center
@ -413,7 +413,6 @@ The structure score in `_structure_score(...)` mixes:
- adjacency agreement with the hard source portrait
- motif agreement with the hard source portrait
- line agreement with the hard source portrait
- a tiny representative-match tiebreaker
Natural-language picture:
@ -465,6 +464,7 @@ These parts fit the optimizer's core story:
- building a hard source-assignment portrait from the actual lattice ownership
- using per-cell reliability to blend those two portraits instead of always trusting one
- forcing snap and refine to choose real source pixels, not synthesized colors
- letting the soft representative portrait stabilize snap, then making refine answer only to the snapped anchor and the hard source portrait
- keeping snap if refine makes source-lattice fidelity worse
Those are coherent. They all serve the same broad goal: make one global lattice without drifting too far from real local source evidence.
@ -473,10 +473,10 @@ Those are coherent. They all serve the same broad goal: make one global lattice
These are the places where the optimizer still speaks in an overly complicated or self-contradictory voice:
- the machine still maintains two overlapping portraits of the same lattice and spends much of its complexity mediating the argument between them
- the machine still maintains two overlapping portraits of the same lattice, even though refine is now source-first and only snap still arbitrates between them
- snap, relax, and refine all carry slightly different versions of adjacency, motif, line, and delta agreement; the same idea is being said three times with different accents
- the candidate generator is local and the solver is discrete, but the vocabulary around it still sounds like a continuous deformation engine
- the main entry point is now honestly named `optimize_lattice_pixels(...)`, the k-means boundary scout is gone, and prep is split from decision-making; those cuts removed three contradictions from the previous map
- the main entry point is now honestly named `optimize_lattice_pixels(...)`, the k-means boundary scout is gone, prep is split from decision-making, and refine no longer consults the representative portrait; those cuts removed four contradictions from the previous map
This is probably the real cutting checklist for the next optimizer pass.
@ -484,7 +484,7 @@ This is probably the real cutting checklist for the next optimizer pass.
If we strip the optimizer down to what it is really doing today, the machine is:
`source image -> edge scout -> fixed lattice centers -> soft portrait + hard portrait -> source-first snap -> soft neighborhood relax -> greedy discrete refine -> keep snap if refine lied`
`source image -> edge scout -> fixed lattice centers -> soft portrait + hard portrait -> source-first snap -> soft neighborhood relax -> source-first greedy refine -> keep snap if refine lied`
That is much simpler than the surrounding names make it sound.
@ -492,11 +492,11 @@ That is much simpler than the surrounding names make it sound.
If we keep pruning, the highest-value cuts look like this:
1. Decide whether the representative portrait still earns its keep everywhere.
If the source-first path is now strong enough, some of that arbitration machinery may be removable.
1. Put relax on trial.
The soft relaxation stage may be a useful bridge, or it may be an expensive second solver that mostly repeats refine.
2. Collapse duplicate structure voices.
Adjacency, motif, and line are important, but they should not need three near-parallel dialects unless they are truly doing different work.
3. Put relax on trial.
The soft relaxation stage may be a useful bridge, or it may be an expensive second solver that mostly repeats refine.
3. Decide whether snap still needs both portraits everywhere.
Refine is already source-first now, so the next big question is whether snap still needs all of its portrait arbitration machinery or only the parts that genuinely stabilize hard cases.
That is the next place to swing the axe.

30
src/repixelizer/continuous.py
View file

@ -1115,7 +1115,6 @@ def _structure_score(
source_t,
uv_t,
candidate_t,
representative_t,
source_reference_t,
anchor_t,
solver_params: SolverHyperParams,
@ -1133,7 +1132,6 @@ def _structure_score(
source_adj = _adjacency_pattern_loss(candidate_t, source_reference_t)
source_motif = _motif_pattern_loss(candidate_t, source_reference_t)
source_line = _line_pattern_loss(torch, candidate_t, source_reference_t)
representative_match = (candidate_t - representative_t).abs().mean()
return (
boundary * solver_params.structure_boundary_weight
+ anchor_adj * solver_params.structure_anchor_adjacency_weight
@ -1142,7 +1140,6 @@ def _structure_score(
+ source_adj * solver_params.structure_source_adjacency_weight
+ source_motif * solver_params.structure_source_motif_weight
+ source_line * solver_params.structure_source_line_weight
+ representative_match * solver_params.structure_representative_weight
)
@ -1151,9 +1148,7 @@ def _discrete_refine_output(
F,
source_t,
uv_t,
representative_t,
source_reference_t,
source_reliability_t,
source_reference_np,
anchor_t,
source_delta_x_t,
@ -1170,12 +1165,10 @@ def _discrete_refine_output(
height = source_hw.shape[0]
width = source_hw.shape[1]
uv = uv_t[0]
representative = representative_t[0]
source_reference = source_reference_t[0]
source_reliability = source_reliability_t[0]
anchor = anchor_t[0].permute(1, 2, 0)
output_height = representative.shape[0]
output_width = representative.shape[1]
output_height = anchor.shape[0]
output_width = anchor.shape[1]
candidate_levels = max(3, int(solver_params.refine_candidate_levels))
if candidate_levels % 2 == 0:
@ -1195,26 +1188,18 @@ def _discrete_refine_output(
)
candidate_colors = source_hw[candidate_y, candidate_x]
anchor_energy = (candidate_colors - anchor[..., None, :]).abs().mean(dim=-1)
rep_energy = (candidate_colors - representative[..., None, :]).abs().mean(dim=-1)
source_reference_energy = (candidate_colors - source_reference[..., None, :]).abs().mean(dim=-1)
alpha_energy = (candidate_colors[..., 3] - anchor[..., None, 3]).abs()
distance_energy = (candidate_offset_x / max(cell_x, 1e-4)).square() + (candidate_offset_y / max(cell_y, 1e-4)).square()
match_energy = _reference_match_energy(
rep_energy,
source_reference_energy,
source_reliability,
representative_weight=solver_params.refine_representative_match_weight,
source_weight=solver_params.refine_source_match_weight,
)
base_energy = (
anchor_energy * solver_params.refine_anchor_weight
+ match_energy * solver_params.refine_representative_weight
+ source_reference_energy * solver_params.refine_source_weight
+ alpha_energy * solver_params.refine_alpha_weight
+ distance_energy * solver_params.refine_distance_weight
)
relax_base_energy = (
anchor_energy * solver_params.refine_anchor_weight * solver_params.relax_anchor_scale
+ match_energy * solver_params.refine_representative_weight
+ source_reference_energy * solver_params.refine_source_weight
+ alpha_energy * solver_params.refine_alpha_weight
+ distance_energy * solver_params.refine_distance_weight
)
@ -1321,7 +1306,6 @@ def _discrete_refine_output(
source_t,
uv_t,
start_colors[None, ...],
representative_t,
source_reference_t,
anchor_hw,
solver_params,
@ -1344,7 +1328,6 @@ def _discrete_refine_output(
source_t,
uv_t,
candidate_t,
representative_t,
source_reference_t,
anchor_hw,
solver_params,
@ -1375,7 +1358,6 @@ def _discrete_refine_output(
source_t,
uv_t,
candidate_colors_selected[None, ...],
representative_t,
source_reference_t,
anchor_hw,
solver_params,
@ -1392,7 +1374,7 @@ def _discrete_refine_output(
candidate_colors,
final_selected,
final_ranking_energy,
representative[..., 3],
anchor[..., 3],
solver_params=solver_params,
)
best_colors = _select_colors(candidate_colors, hardened_selected)
@ -1523,9 +1505,7 @@ def optimize_lattice_pixels(
F,
prep.source_t,
prep.uv_t,
prep.representative_t,
prep.source_reference_t,
prep.source_reliability_t,
prep.source_lattice_reference,
snap_t,
prep.source_delta_x_t,

5
src/repixelizer/params.py
View file

@ -23,9 +23,7 @@ class SolverHyperParams:
snap_representative_delta_weight: float = 0.20
snap_source_delta_weight: float = 0.80
refine_anchor_weight: float = 0.60
refine_representative_weight: float = 0.20
refine_representative_match_weight: float = 0.15
refine_source_match_weight: float = 0.85
refine_source_weight: float = 0.20
refine_alpha_weight: float = 0.10
refine_distance_weight: float = 0.10
refine_source_delta_weight: float = 0.25
@ -59,7 +57,6 @@ class SolverHyperParams:
structure_source_adjacency_weight: float = 0.35
structure_source_motif_weight: float = 0.40
structure_source_line_weight: float = 0.20
structure_representative_weight: float = 0.02
alpha_foreground_threshold: float = 0.50
alpha_representative_foreground_threshold: float = 0.60
alpha_opaque_threshold: float = 0.95

8
src/repixelizer/tuning.py
View file

@ -79,7 +79,7 @@ def _mutate_params(base: SolverHyperParams, rng: np.random.Generator, scale: flo
rng,
[
base.refine_anchor_weight,
base.refine_representative_weight,
base.refine_source_weight,
base.refine_alpha_weight,
base.refine_distance_weight,
],
@ -91,7 +91,7 @@ def _mutate_params(base: SolverHyperParams, rng: np.random.Generator, scale: flo
base.structure_boundary_weight,
base.structure_anchor_adjacency_weight,
base.structure_anchor_motif_weight,
base.structure_representative_weight,
base.structure_anchor_line_weight,
],
scale,
)
@ -113,7 +113,7 @@ def _mutate_params(base: SolverHyperParams, rng: np.random.Generator, scale: flo
high=0.32,
),
refine_anchor_weight=float(refine_group[0]),
refine_representative_weight=float(refine_group[1]),
refine_source_weight=float(refine_group[1]),
refine_alpha_weight=float(refine_group[2]),
refine_distance_weight=float(refine_group[3]),
refine_source_delta_weight=_mutate_linear(
@ -207,7 +207,7 @@ def _mutate_params(base: SolverHyperParams, rng: np.random.Generator, scale: flo
structure_boundary_weight=float(structure_group[0]),
structure_anchor_adjacency_weight=float(structure_group[1]),
structure_anchor_motif_weight=float(structure_group[2]),
structure_representative_weight=float(structure_group[3]),
structure_anchor_line_weight=float(structure_group[3]),
structure_source_adjacency_weight=_mutate_linear(
rng,
base.structure_source_adjacency_weight,