GameCult/repixelizer
Watch
9
0
Fork
You've already forked repixelizer
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Trim tile-graph to edge-only analysis

Browse source
This commit is contained in:
Meta Vens 2026-04-23 03:56:03 +01:00
commit cd09307367
9 changed files with 104 additions and 82 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -48,6 +48,7 @@ Current tile-graph status:
- source-region connected-components now have a device-side Torch path, so the expensive labeling step no longer depends on Python flood fill
- profiling the selected badge candidate shows the main remaining bottleneck is model construction, not the solver loop: `build_tile_graph_model(...)` takes about `142.3s`, and about `131.2s` of that is `_extract_source_region_tiles(...)`
- after the first pruning pass, tile-graph no longer spends iteration time on pipeline rerank probes at all; it now runs the chosen or pinned lattice directly
- tile-graph now also skips cluster analysis entirely and no longer uses the per-cell mean-color pull in unary scoring; the path keeps only edge scouting plus sharp/edge lattice references
- the remaining large-fixture bottleneck is the per-component one-cell window cutting pass inside source-region extraction, which is still Python/NumPy-heavy after the GPU CCL stage
- the pipeline now has a direct-control path for iteration: `--target-width` / `--target-height` plus optional `--phase-x` / `--phase-y` let you run an exact lattice without paying the full lattice search, and `--skip-phase-rerank` lets you keep the pipeline from second-guessing that choice
- repeated fixed-lattice `tile-graph` runs in the same Python process now reuse the expensive model build; on the cleaned badge at pinned `126x126` / phase `(0.0, -0.2)`, the first CUDA run took about `10.2s` and the second cached rerun took about `2.1s`

2
docs/implementation-plan.md
View file

@ -63,8 +63,10 @@ What landed:
- `tile-graph` no longer participates in pipeline phase rerank probes; rerank is now a continuous-only wrapper
- the experimental `hybrid` path and its geometry-prior wiring have been removed
- `tile_graph.py` no longer carries geometry-prior fields through `TileGraphModel` or tile-graph unary scoring
- `tile-graph` now skips source clustering entirely and uses only edge analysis plus sharp/edge lattice references during unary scoring
- `cli.py` now exposes only `continuous` and `tile-graph` as reconstruction engines
- `tests/test_pipeline.py` now checks that low-confidence tile-graph runs skip rerank probes instead of rebuilding probe candidates
- `tests/test_analysis.py` and `tests/test_tile_graph.py` now cover the new edge-only analysis path
- `docs/tile-graph-algorithm-map.md` now describes the smaller machine directly rather than documenting the removed hybrid sidecar path
Current implementation note:

35
docs/tile-graph-algorithm-map.md
View file

@ -129,6 +129,11 @@ Plain-language picture:
This is a scouting report about where the mural has edges and rough color families. It is binoculars, not ownership papers.
Current tile-graph note:
- the tile-graph path now asks only for the edge scout report
- cluster labels and cluster preview are left empty on that path because the tile-graph machine does not use them
### `SourceLatticeReference`
- File: `src/repixelizer/types.py`
@ -157,11 +162,11 @@ This is the mural after the graph paper has already been pressed onto it. Each s
- `candidate_coords`: output coord `(y, x)` each candidate belongs to
- `candidate_area_ratio`: how much of a cell-sized source window the source region covered
- `candidate_coverage`: clipped version of area coverage
- `candidate_deltas`: expected RGBA deltas to right/down/left/up neighbors sampled one cell away in source space
- `cell_candidate_offsets`, `cell_candidate_indices`: CSR-like mapping from output cells to candidate rows
- `reference_mean_rgba`, `reference_sharp_rgba`, `reference_edge_rgba`
- `edge_strength`
- `component_count`, `edge_density`, `average_choices`
- `candidate_deltas`: expected RGBA deltas to right/down/left/up neighbors sampled one cell away in source space
- `cell_candidate_offsets`, `cell_candidate_indices`: CSR-like mapping from output cells to candidate rows
- `reference_sharp_rgba`, `reference_edge_rgba`
- `edge_strength`
- `component_count`, `edge_density`, `average_choices`
Plain-language picture:
@ -263,7 +268,7 @@ File: `src/repixelizer/analysis.py`
Entry point:
- `analyze_source(rgba, seed, cluster_count=6, device=None)`
- `analyze_source(rgba, seed, cluster_count=6, device=None, include_clusters=True)`
Outputs:
@ -281,6 +286,7 @@ Important reality check:
- tile-graph no longer uses `cluster_map` as its actual tile ownership map
- the tile-graph region ownership is built later by connected components over source pixels using color/alpha thresholds
- after the pruning pass, tile-graph can skip cluster assignment entirely and still run normally because it only needs `edge_map`
## Stage 4: Optional Phase Rerank
@ -567,8 +573,6 @@ After candidate selection, the model stores:
- clipped area coverage proxy
- `candidate_deltas`
- sampled right/down/left/up expected deltas one cell away in source space
- `reference_mean_rgba`
- lattice-conditioned per-cell mean
- `reference_sharp_rgba`
- lattice-conditioned per-cell exemplar
- `reference_edge_rgba`
@ -616,13 +620,12 @@ Transforms CSR-style cell candidate storage into dense tensors:
For each output cell and each candidate:
1. `sharp_error`
2. `mean_error`
3. `edge_error`
4. If edge cell:
- `color_error = min(sharp_error, edge_error) + mean_error * tile_graph_edge_mean_weight`
5. Else:
- `color_error = sharp_error * nonedge_sharp_weight + mean_error * nonedge_mean_weight`
6. Add:
2. `edge_error`
3. If edge cell:
- `color_error = min(sharp_error, edge_error)`
4. Else:
- `color_error = sharp_error * nonedge_sharp_weight`
5. Add:
- `area_error`
- `alpha_error`
- `coverage_error`
@ -634,6 +637,8 @@ Important implication:
This is the emotional center of the current design. The algorithm does not trust source-region ownership alone. It insists on comparing each legal chip to a portrait painted by the lattice reference first.
The portrait is smaller now than it used to be. Tile-graph no longer asks, "How close is this chip to the cell average color?" It only asks, "How close is this chip to the cell's sharp exemplar or edge exemplar?" That keeps the judge from rewarding mush just because mush looks average.
## Stage 11: Initial Assignment
File: `src/repixelizer/tile_graph.py`

49
src/repixelizer/analysis.py
View file

@ -113,25 +113,34 @@ def _colorize_clusters(cluster_map: np.ndarray, centers: np.ndarray) -> np.ndarr
return np.concatenate([preview, alpha], axis=-1)
def analyze_source(rgba: np.ndarray, seed: int, cluster_count: int = 6, device: str | None = None) -> SourceAnalysis:
def analyze_source(
rgba: np.ndarray,
seed: int,
cluster_count: int = 6,
device: str | None = None,
*,
include_clusters: bool = True,
) -> SourceAnalysis:
if device is not None:
torch = _require_torch()
resolved_device = _resolve_device(torch, device)
rgba_t = torch.from_numpy(rgba).to(device=resolved_device, dtype=torch.float32)
edge_map_t = _compute_edge_map_torch(torch, rgba_t)
alpha_t = rgba_t[..., 3]
opaque_t = alpha_t > 0.05
cluster_map_t = torch.full(alpha_t.shape, -1, device=resolved_device, dtype=torch.long)
centers_t = torch.zeros((0, 4), device=resolved_device, dtype=torch.float32)
if bool(torch.any(opaque_t).item()):
samples_t = rgba_t[opaque_t]
k = min(cluster_count, max(2, int(np.sqrt(int(samples_t.shape[0]) // 64 + 1))))
centers_t, labels_t = _kmeans_torch(torch, samples_t, k=k, seed=seed)
cluster_map_t[opaque_t] = labels_t
preview_palette_t = centers_t[:, :3].clamp(0.0, 1.0) if centers_t.numel() else torch.zeros((1, 3), device=resolved_device, dtype=torch.float32)
preview_indices_t = torch.clamp(cluster_map_t, 0, max(0, preview_palette_t.shape[0] - 1))
preview_t = preview_palette_t[preview_indices_t]
preview_t = torch.cat([preview_t, alpha_t[..., None]], dim=-1)
preview_t = torch.zeros((*alpha_t.shape, 4), device=resolved_device, dtype=torch.float32)
if include_clusters:
opaque_t = alpha_t > 0.05
if bool(torch.any(opaque_t).item()):
samples_t = rgba_t[opaque_t]
k = min(cluster_count, max(2, int(np.sqrt(int(samples_t.shape[0]) // 64 + 1))))
centers_t, labels_t = _kmeans_torch(torch, samples_t, k=k, seed=seed)
cluster_map_t[opaque_t] = labels_t
preview_palette_t = centers_t[:, :3].clamp(0.0, 1.0) if centers_t.numel() else torch.zeros((1, 3), device=resolved_device, dtype=torch.float32)
preview_indices_t = torch.clamp(cluster_map_t, 0, max(0, preview_palette_t.shape[0] - 1))
preview_t = preview_palette_t[preview_indices_t]
preview_t = torch.cat([preview_t, alpha_t[..., None]], dim=-1)
return SourceAnalysis(
edge_map=edge_map_t.detach().cpu().numpy().astype(np.float32),
cluster_map=cluster_map_t.detach().cpu().numpy().astype(np.int32),
@ -142,16 +151,18 @@ def analyze_source(rgba: np.ndarray, seed: int, cluster_count: int = 6, device:
edge_map = _compute_edge_map(rgba)
alpha = rgba[..., 3]
opaque = alpha > 0.05
cluster_map = np.full(alpha.shape, -1, dtype=np.int32)
centers = np.zeros((0, 4), dtype=np.float32)
if np.any(opaque):
samples = rgba[opaque]
k = min(cluster_count, max(2, int(np.sqrt(samples.shape[0] // 64 + 1))))
centers, labels = _kmeans(samples, k=k, seed=seed)
cluster_map[opaque] = labels
preview = _colorize_clusters(np.maximum(cluster_map, 0), centers if centers.size else np.zeros((1, 4), dtype=np.float32))
preview[..., 3] = alpha
preview = np.zeros((*alpha.shape, 4), dtype=np.float32)
if include_clusters:
opaque = alpha > 0.05
if np.any(opaque):
samples = rgba[opaque]
k = min(cluster_count, max(2, int(np.sqrt(samples.shape[0] // 64 + 1))))
centers, labels = _kmeans(samples, k=k, seed=seed)
cluster_map[opaque] = labels
preview = _colorize_clusters(np.maximum(cluster_map, 0), centers if centers.size else np.zeros((1, 4), dtype=np.float32))
preview[..., 3] = alpha
return SourceAnalysis(
edge_map=edge_map,
cluster_map=cluster_map,

2
src/repixelizer/params.py
View file

@ -85,8 +85,6 @@ class SolverHyperParams:
tile_graph_stroke_step_scale: float = 0.95
tile_graph_stroke_minor_limit_scale: float = 0.55
tile_graph_nonedge_sharp_weight: float = 0.85
tile_graph_nonedge_mean_weight: float = 0.15
tile_graph_edge_mean_weight: float = 0.03
tile_graph_area_weight: float = 0.03
tile_graph_alpha_weight: float = 0.25
tile_graph_coverage_weight: float = 0.05

10
src/repixelizer/pipeline.py
View file

@ -79,7 +79,12 @@ def run_pipeline(
device=device,
)
inference_mode = "fixed"
analysis = analyze_source(source, seed=seed, device=device if reconstruction_mode == "tile-graph" else None)
analysis = analyze_source(
source,
seed=seed,
device=device if reconstruction_mode == "tile-graph" else None,
include_clusters=reconstruction_mode != "tile-graph",
)
inference = _select_phase_candidate(
source,
inference,
@ -134,7 +139,8 @@ def run_pipeline(
)
write_alpha_preview(diagnostics_path / "alpha-preview.png", source, output_rgba)
write_heatmap(diagnostics_path / "noise-heatmap.png", cleanup.isolated_heatmap)
save_rgba(diagnostics_path / "cluster-preview.png", analysis.cluster_preview)
if analysis.cluster_centers.size:
save_rgba(diagnostics_path / "cluster-preview.png", analysis.cluster_preview)
run_json = summarize_run(result)
run_json["inference"] = inference_to_json(inference)
run_json["settings"] = {

48
src/repixelizer/tile_graph.py
View file

@ -60,7 +60,6 @@ class TileGraphModel:
candidate_deltas: np.ndarray
cell_candidate_offsets: np.ndarray
cell_candidate_indices: np.ndarray
reference_mean_rgba: np.ndarray
reference_sharp_rgba: np.ndarray
reference_edge_rgba: np.ndarray
edge_strength: np.ndarray
@ -1075,7 +1074,6 @@ def build_tile_graph_model(
candidate_deltas=candidate_deltas_np,
cell_candidate_offsets=cell_candidate_offsets,
cell_candidate_indices=cell_candidate_indices_np,
reference_mean_rgba=source_reference.mean_rgba.astype(np.float32),
reference_sharp_rgba=source_reference.sharp_rgba.astype(np.float32),
reference_edge_rgba=edge_rgba_np.reshape(inference.target_height, inference.target_width, -1).astype(np.float32),
edge_strength=source_reference.edge_strength.astype(np.float32),
@ -1113,42 +1111,6 @@ def _build_choice_grid(model: TileGraphModel) -> tuple[np.ndarray, np.ndarray]:
return choice_indices, choice_mask
def _tile_graph_unary_cost(model: TileGraphModel, solver_params: SolverHyperParams) -> np.ndarray:
height, width = model.reference_sharp_rgba.shape[:2]
unary_cost = np.full(model.cell_candidate_indices.shape[0], np.inf, dtype=np.float32)
for y in range(height):
for x in range(width):
start, end = _cell_candidate_span(model, y, x)
if end <= start:
continue
indices = model.cell_candidate_indices[start:end]
reference_sharp = model.reference_sharp_rgba[y, x]
reference_mean = model.reference_mean_rgba[y, x]
reference_edge = model.reference_edge_rgba[y, x]
candidates = model.candidate_rgba[indices]
sharp_error = np.mean(np.abs(reference_sharp[None, :] - candidates), axis=-1)
mean_error = np.mean(np.abs(reference_mean[None, :] - candidates), axis=-1)
edge_error = np.mean(np.abs(reference_edge[None, :] - candidates), axis=-1)
if float(model.edge_strength[y, x]) >= solver_params.source_edge_detail_threshold:
color_error = np.minimum(sharp_error, edge_error) + mean_error * solver_params.tile_graph_edge_mean_weight
else:
color_error = (
sharp_error * solver_params.tile_graph_nonedge_sharp_weight
+ mean_error * solver_params.tile_graph_nonedge_mean_weight
)
area_error = np.abs(np.log(np.clip(model.candidate_area_ratio[indices], 1e-4, None) + 1e-4))
alpha_error = np.abs(candidates[:, 3] - reference_mean[3])
coverage_error = 1.0 - np.clip(model.candidate_coverage[indices], 0.0, 1.0)
total_cost = (
color_error
+ area_error * solver_params.tile_graph_area_weight
+ alpha_error * solver_params.tile_graph_alpha_weight
+ coverage_error * solver_params.tile_graph_coverage_weight
)
unary_cost[start:end] = total_cost.astype(np.float32)
return unary_cost
def _tile_graph_unary_cost_torch(
torch,
model: TileGraphModel,
@ -1163,23 +1125,21 @@ def _tile_graph_unary_cost_torch(
candidate_area_t = torch.from_numpy(model.candidate_area_ratio).to(device=device, dtype=torch.float32)
candidate_coverage_t = torch.from_numpy(model.candidate_coverage).to(device=device, dtype=torch.float32)
reference_sharp_t = torch.from_numpy(model.reference_sharp_rgba).to(device=device, dtype=torch.float32)
reference_mean_t = torch.from_numpy(model.reference_mean_rgba).to(device=device, dtype=torch.float32)
reference_edge_t = torch.from_numpy(model.reference_edge_rgba).to(device=device, dtype=torch.float32)
edge_strength_t = torch.from_numpy(model.edge_strength).to(device=device, dtype=torch.float32)
choice_rgba_t = candidate_rgba_t[choice_indices_t]
choice_deltas_t = candidate_deltas_t[choice_indices_t]
sharp_error_t = (reference_sharp_t[..., None, :] - choice_rgba_t).abs().mean(dim=-1)
mean_error_t = (reference_mean_t[..., None, :] - choice_rgba_t).abs().mean(dim=-1)
edge_error_t = (reference_edge_t[..., None, :] - choice_rgba_t).abs().mean(dim=-1)
edge_cell_mask_t = edge_strength_t[..., None] >= solver_params.source_edge_detail_threshold
color_error = torch.where(
edge_cell_mask_t,
torch.minimum(sharp_error_t, edge_error_t) + mean_error_t * solver_params.tile_graph_edge_mean_weight,
sharp_error_t * solver_params.tile_graph_nonedge_sharp_weight
+ mean_error_t * solver_params.tile_graph_nonedge_mean_weight,
torch.minimum(sharp_error_t, edge_error_t),
sharp_error_t * solver_params.tile_graph_nonedge_sharp_weight,
)
area_error = torch.log(candidate_area_t[choice_indices_t].clamp_min(1e-4) + 1e-4).abs()
alpha_error = (choice_rgba_t[..., 3] - reference_mean_t[..., None, 3]).abs()
alpha_reference_t = torch.maximum(reference_sharp_t[..., 3], reference_edge_t[..., 3])
alpha_error = (choice_rgba_t[..., 3] - alpha_reference_t[..., None]).abs()
coverage_error = 1.0 - candidate_coverage_t[choice_indices_t].clamp(0.0, 1.0)
unary_cost_t = (
color_error

14
tests/test_analysis.py
View file

@ -18,3 +18,17 @@ def test_analyze_source_cpu_device_matches_default_path() -> None:
assert np.allclose(accelerated.alpha_map, default.alpha_map, atol=1e-5)
assert accelerated.cluster_map.shape == default.cluster_map.shape
assert accelerated.cluster_preview.shape == default.cluster_preview.shape
def test_analyze_source_can_skip_cluster_analysis() -> None:
source = np.zeros((8, 8, 4), dtype=np.float32)
source[..., 3] = 1.0
source[2:6, 3:5, :3] = np.asarray([1.0, 0.8, 0.2], dtype=np.float32)
analysis = analyze_source(source, seed=7, device="cpu", include_clusters=False)
assert analysis.edge_map.shape == source.shape[:2]
assert np.allclose(analysis.alpha_map, source[..., 3], atol=1e-5)
assert analysis.cluster_centers.size == 0
assert np.all(analysis.cluster_map == -1)
assert np.count_nonzero(analysis.cluster_preview) == 0

25
tests/test_tile_graph.py
View file

@ -47,6 +47,31 @@ def test_tile_graph_model_extracts_candidates_and_adjacency_from_lattice_proposa
assert model.model_device == "cpu"
def test_tile_graph_model_works_with_edge_only_analysis() -> None:
lowres = np.zeros((4, 4, 4), dtype=np.float32)
lowres[..., 3] = 1.0
lowres[1:3, 1:3] = np.asarray([1.0, 0.7, 0.2, 1.0], dtype=np.float32)
source = np.repeat(np.repeat(lowres, 8, axis=0), 8, axis=1)
inference = InferenceResult(
target_width=4,
target_height=4,
phase_x=0.0,
phase_y=0.0,
confidence=1.0,
top_candidates=[],
)
model = build_tile_graph_model(
source,
inference=inference,
analysis=analyze_source(source, seed=5, device="cpu", include_clusters=False),
device="cpu",
)
assert model.candidate_rgba.shape[0] > 0
assert model.model_device == "cpu"
def test_tile_graph_candidates_use_literal_source_pixel_colors() -> None:
source = np.zeros((8, 8, 4), dtype=np.float32)
source[..., 3] = 1.0