Use shared density for spritesheet sizing
This commit is contained in:
parent
68a7e0ab0d
commit
afd1076b7b
7 changed files with 157 additions and 18 deletions
12
README.md
12
README.md
|
|
@ -80,7 +80,7 @@ What exists now:
|
|||
- lattice inference with CUDA support
|
||||
- the `phase-field` reconstruction engine in `src/repixelizer/phase_field.py`
|
||||
- automatic diagnostics, comparisons, and benchmark runs
|
||||
- spritesheet CLI mode that detects separated regions and repixelizes each region independently
|
||||
- spritesheet CLI mode that detects separated regions, chooses one shared texel density for the sheet, and repixelizes each region independently
|
||||
- a tuning harness for offline parameter sweeps
|
||||
- `source_fidelity` and `source_structure` metrics for lattice agreement and visible structure
|
||||
- a shippable explicit local-search solver config, with tuning notes in `docs/solver-tuning-lessons-2026-04-29.md`
|
||||
|
|
@ -125,9 +125,11 @@ repixelize spritesheet sheet.png --out output.png --diagnostics-dir diagnostics
|
|||
```
|
||||
|
||||
When `--sprite-count` is omitted, Repixelizer detects large separated foreground
|
||||
regions. Pinned target dimensions apply per sprite, not to the whole sheet. Add
|
||||
`--strip-background` when the sheet has an opaque edge-connected backdrop that
|
||||
should become transparent before cropping.
|
||||
regions. By default, automatic spritesheet mode infers each crop once, chooses a
|
||||
robust shared texel density for the sheet, and pins each crop to that density
|
||||
before reconstruction. Pinned target dimensions apply per sprite, not to the
|
||||
whole sheet. Add `--strip-background` when the sheet has an opaque
|
||||
edge-connected backdrop that should become transparent before cropping.
|
||||
|
||||
## Corpus And Benchmarks
|
||||
|
||||
|
|
@ -186,7 +188,7 @@ That keeps benchmark assets, attribution exports, and tuning runs from polluting
|
|||
Core modules:
|
||||
|
||||
- `inference.py`: target size inference
|
||||
- `spritesheet.py`: spritesheet region detection and per-region pipeline orchestration
|
||||
- `spritesheet.py`: spritesheet region detection, shared-density sizing, and per-region pipeline orchestration
|
||||
- `phase_field.py`: explicit local-search phase-field solver
|
||||
- `metrics.py`: fidelity, adjacency, and motif metrics
|
||||
- `benchmark.py`: corpus benchmark runner
|
||||
|
|
|
|||
|
|
@ -47,8 +47,9 @@ the CLI, GUI, spritesheet wrapper, and comparison harness.
|
|||
|
||||
### Ownership
|
||||
|
||||
Spritesheet mode owns region detection and output packing only. It does not own
|
||||
lattice inference, reconstruction, cleanup, palette fitting, or metrics.
|
||||
Spritesheet mode owns region detection, sheet-level texel density selection, and
|
||||
output packing only. It does not own reconstruction, cleanup, palette fitting,
|
||||
or metrics.
|
||||
|
||||
### What the source actually does
|
||||
|
||||
|
|
@ -61,12 +62,17 @@ Spritesheet mode:
|
|||
3. Detects connected foreground regions, optionally selecting the largest
|
||||
requested `--sprite-count`.
|
||||
4. Sorts regions in reading order.
|
||||
5. Crops each region and calls `run_pipeline_rgba(...)` on that crop.
|
||||
6. Packs per-sprite outputs back into source-row order and writes a
|
||||
5. When no explicit target size is supplied, probes each crop with autocorr
|
||||
inference, chooses the median source-pixels-per-texel density for the whole
|
||||
sheet, and pins each crop to dimensions derived from that shared density.
|
||||
6. Crops each region and calls `run_pipeline_rgba(...)` on that crop.
|
||||
7. Packs per-sprite outputs back into source-row order and writes a
|
||||
`spritesheet.json` diagnostic summary when diagnostics are enabled.
|
||||
|
||||
Pinned target width, height, or size applies to each sprite crop. Automatic
|
||||
inference also runs per crop. The whole sheet never receives one shared lattice.
|
||||
Pinned target width, height, or size applies to each sprite crop and disables
|
||||
shared-density sizing. The whole sheet never receives one shared lattice, but
|
||||
automatic sheet mode does use one shared texel density so sub-sprites do not
|
||||
silently converge on half- or double-scale rulers.
|
||||
|
||||
## Diagram
|
||||
|
||||
|
|
|
|||
|
|
@ -36,8 +36,10 @@ Do not continue implementation automatically from a rehydrate-only request.
|
|||
- `source_structure` exists alongside `source_fidelity` so visible structure and lattice agreement can be inspected separately.
|
||||
- Cleanup is alpha snap plus diagnostics.
|
||||
- `src/repixelizer/spritesheet.py` owns spritesheet region detection and packing.
|
||||
It calls `run_pipeline_rgba(...)` once per detected crop; pinned target sizes
|
||||
apply per sprite, not to the whole sheet.
|
||||
In automatic mode it probes crop sizes, chooses one shared
|
||||
source-pixels-per-texel density for the sheet, then calls
|
||||
`run_pipeline_rgba(...)` once per detected crop. Pinned target sizes apply per
|
||||
sprite and bypass shared-density sizing.
|
||||
|
||||
## Hosted/web state
|
||||
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ from typing import Any
|
|||
import numpy as np
|
||||
|
||||
from .diagnostics import write_json
|
||||
from .inference import infer_autocorr_lattice
|
||||
from .io import load_rgba, save_rgba
|
||||
from .pipeline import run_pipeline_rgba
|
||||
from .preprocess import strip_edge_background
|
||||
|
|
@ -72,17 +73,25 @@ def run_spritesheet(
|
|||
diagnostics_path.mkdir(parents=True, exist_ok=True)
|
||||
save_rgba(diagnostics_path / "detected-source.png", detection_source)
|
||||
|
||||
inferred_targets = _shared_density_targets(
|
||||
detection_source,
|
||||
regions,
|
||||
device=device,
|
||||
enabled=target_size is None and target_width is None and target_height is None,
|
||||
)
|
||||
|
||||
sprite_runs: list[SpriteRun] = []
|
||||
for region in regions:
|
||||
crop = detection_source[region.top : region.bottom, region.left : region.right].copy()
|
||||
sprite_diagnostics = None
|
||||
if diagnostics_path is not None:
|
||||
sprite_diagnostics = diagnostics_path / f"sprite-{region.index:03d}"
|
||||
pinned_target = inferred_targets.get(region.index)
|
||||
result = run_pipeline_rgba(
|
||||
crop,
|
||||
target_size=target_size,
|
||||
target_width=target_width,
|
||||
target_height=target_height,
|
||||
target_size=None if pinned_target is not None else target_size,
|
||||
target_width=pinned_target[0] if pinned_target is not None else target_width,
|
||||
target_height=pinned_target[1] if pinned_target is not None else target_height,
|
||||
palette_path=palette_path,
|
||||
palette_mode=palette_mode,
|
||||
diagnostics_dir=sprite_diagnostics,
|
||||
|
|
@ -103,6 +112,7 @@ def run_spritesheet(
|
|||
sprite_count_request=sprite_count,
|
||||
sprite_runs=sprite_runs,
|
||||
output_shape=output.shape,
|
||||
shared_density=_shared_density_summary(regions, inferred_targets),
|
||||
)
|
||||
if diagnostics_path is not None:
|
||||
write_json(diagnostics_path / "spritesheet.json", diagnostics)
|
||||
|
|
@ -333,8 +343,9 @@ def _spritesheet_summary(
|
|||
sprite_count_request: int | None,
|
||||
sprite_runs: list[SpriteRun],
|
||||
output_shape: tuple[int, ...],
|
||||
shared_density: dict[str, Any] | None,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
summary = {
|
||||
"mode": "spritesheet",
|
||||
"source_width": int(source_shape[1]),
|
||||
"source_height": int(source_shape[0]),
|
||||
|
|
@ -356,3 +367,65 @@ def _spritesheet_summary(
|
|||
for run in sprite_runs
|
||||
],
|
||||
}
|
||||
if shared_density is not None:
|
||||
summary["shared_density"] = shared_density
|
||||
return summary
|
||||
|
||||
|
||||
def _shared_density_targets(
|
||||
source: np.ndarray,
|
||||
regions: list[SpriteRegion],
|
||||
*,
|
||||
device: str,
|
||||
enabled: bool,
|
||||
) -> dict[int, tuple[int, int]]:
|
||||
if not enabled or not regions:
|
||||
return {}
|
||||
probes: list[tuple[SpriteRegion, float]] = []
|
||||
for region in regions:
|
||||
crop = source[region.top : region.bottom, region.left : region.right]
|
||||
inference = infer_autocorr_lattice(crop, device=device)
|
||||
density_x = region.width / max(1, inference.target_width)
|
||||
density_y = region.height / max(1, inference.target_height)
|
||||
probes.append((region, (density_x + density_y) * 0.5))
|
||||
|
||||
density = _median_density([probe_density for _region, probe_density in probes])
|
||||
targets: dict[int, tuple[int, int]] = {}
|
||||
for region, _probe_density in probes:
|
||||
targets[region.index] = (
|
||||
max(1, int(round(region.width / max(density, 1e-6)))),
|
||||
max(1, int(round(region.height / max(density, 1e-6)))),
|
||||
)
|
||||
return targets
|
||||
|
||||
|
||||
def _median_density(values: list[float]) -> float:
|
||||
if not values:
|
||||
return 1.0
|
||||
return float(np.median(np.asarray(values, dtype=np.float32)))
|
||||
|
||||
|
||||
def _shared_density_summary(
|
||||
regions: list[SpriteRegion],
|
||||
targets: dict[int, tuple[int, int]],
|
||||
) -> dict[str, Any] | None:
|
||||
if not targets:
|
||||
return None
|
||||
densities = []
|
||||
for region in regions:
|
||||
target = targets.get(region.index)
|
||||
if target is None:
|
||||
continue
|
||||
target_width, target_height = target
|
||||
density_x = region.width / max(1, target_width)
|
||||
density_y = region.height / max(1, target_height)
|
||||
densities.append((density_x + density_y) * 0.5)
|
||||
if not densities:
|
||||
return None
|
||||
raw = np.asarray(densities, dtype=np.float32)
|
||||
return {
|
||||
"density": float(np.median(raw)),
|
||||
"min_density": float(np.min(raw)),
|
||||
"max_density": float(np.max(raw)),
|
||||
"max_relative_deviation": float(np.max(np.abs(raw - np.median(raw)) / max(float(np.median(raw)), 1e-6))),
|
||||
}
|
||||
|
|
|
|||
|
|
@ -32,3 +32,4 @@
|
|||
{"ts": "2026-05-24T10:01:54+00:00", "type": "implementation", "status": "ok", "note": "Added GameCult Patreon and Discord links to Repixelizer hosted/app headers using GameCult-site URLs; verified GUI tests and desktop/mobile header link boxes."}
|
||||
{"ts": "2026-05-24T10:03:31+00:00", "type": "deployment", "status": "ok", "note": "Deployed Repixelizer header community links to Yggdrasil; service check passed and hosted root HTML contains Patreon and Discord links."}
|
||||
{"ts": "2026-05-30T22:47:57+00:00", "type": "implementation", "status": "ok", "note": "Added spritesheet CLI mode: detect separated foreground regions or select requested largest components, call run_pipeline_rgba per crop, pack outputs in source-row order, and write spritesheet diagnostics; fixture auto-detects six sprites."}
|
||||
{"ts": "2026-05-30T23:39:55+00:00", "type": "implementation", "status": "ok", "note": "Spritesheet automatic sizing now uses a sheet-level median texel density before reconstruction; on the 12 generated spritesheets in 019e7aa3-4bbb-7302-bafd-273f2aa31e53, worst per-sheet density drift improved from 42.21% to 1.786% across 153 sprites."}
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ accepted_design:
|
|||
- `phase-field` in `src/repixelizer/phase_field.py` is the only live reconstruction engine.
|
||||
- The working pipeline is `source image -> lattice inference -> diagnostic edge preview -> fixed lattice centers -> hierarchical flatness signal -> explicit local search plus lattice relaxation -> nearest source sample -> cleanup -> optional palette fitting -> diagnostics`.
|
||||
- `src/repixelizer/pipeline.py` owns orchestration and candidate rerank selection.
|
||||
- `src/repixelizer/spritesheet.py` owns spritesheet region detection and output packing; each detected crop still runs through the canonical `run_pipeline_rgba` solver path.
|
||||
- `src/repixelizer/spritesheet.py` owns spritesheet region detection, shared texel density sizing, and output packing; each detected crop still runs through the canonical `run_pipeline_rgba` solver path.
|
||||
- `src/repixelizer/analysis.py` provides the normalized diagnostic edge preview used by observer/GUI/diagnostic paths; it does not steer the solver.
|
||||
- `src/repixelizer/phase_field.py` owns the live source-position field: initialize from fixed centers, choose local preferred positions from a 5x5 candidate window, blend toward them, relax lattice springs, then nearest-sample the source.
|
||||
- Hosted demo inference uses autocorr lattice candidates, capped by the hosted output limit, with low-confidence preview rerank available.
|
||||
|
|
|
|||
|
|
@ -102,3 +102,58 @@ def test_auto_detect_ignores_wide_stray_components() -> None:
|
|||
|
||||
assert len(regions) == 4
|
||||
assert all(region.height > 5 for region in regions)
|
||||
|
||||
|
||||
def test_spritesheet_auto_mode_pins_shared_density(tmp_path: Path, monkeypatch) -> None:
|
||||
input_path = tmp_path / "sheet.png"
|
||||
output_path = tmp_path / "out.png"
|
||||
save_rgba(input_path, _sheet_with_four_alpha_sprites())
|
||||
pinned_targets: list[tuple[int | None, int | None]] = []
|
||||
probe_targets = iter([(2, 2), (3, 3), (2, 2), (2, 2)])
|
||||
|
||||
class DummyInference:
|
||||
def __init__(self, width: int, height: int) -> None:
|
||||
self.target_width = width
|
||||
self.target_height = height
|
||||
|
||||
def fake_infer_autocorr_lattice(source, **kwargs):
|
||||
del source, kwargs
|
||||
width, height = next(probe_targets)
|
||||
return DummyInference(width, height)
|
||||
|
||||
def fake_run_pipeline_rgba(source, **kwargs):
|
||||
pinned_targets.append((kwargs["target_width"], kwargs["target_height"]))
|
||||
rgba = np.zeros((kwargs["target_height"], kwargs["target_width"], 4), dtype=np.float32)
|
||||
rgba[..., 3] = 1.0
|
||||
inference = InferenceResult(
|
||||
target_width=kwargs["target_width"],
|
||||
target_height=kwargs["target_height"],
|
||||
confidence=1.0,
|
||||
)
|
||||
solver = SolverArtifacts(
|
||||
target_rgba=rgba,
|
||||
uv_field=np.zeros((*rgba.shape[:2], 2), dtype=np.float32),
|
||||
signal_strength=np.zeros(rgba.shape[:2], dtype=np.float32),
|
||||
initial_rgba=rgba.copy(),
|
||||
loss_history=[],
|
||||
)
|
||||
return RunResult(
|
||||
source_rgba=source,
|
||||
output_rgba=rgba,
|
||||
inference=inference,
|
||||
analysis=PhaseFieldSourceAnalysis(edge_map=np.zeros(source.shape[:2], dtype=np.float32)),
|
||||
solver=solver,
|
||||
cleanup=CleanupArtifacts(cleaned_rgba=rgba, isolated_heatmap=np.zeros(rgba.shape[:2], dtype=np.float32)),
|
||||
palette_result=None,
|
||||
diagnostics={},
|
||||
)
|
||||
|
||||
monkeypatch.setattr("repixelizer.spritesheet.infer_autocorr_lattice", fake_infer_autocorr_lattice)
|
||||
monkeypatch.setattr("repixelizer.spritesheet.run_pipeline_rgba", fake_run_pipeline_rgba)
|
||||
|
||||
result = run_spritesheet(input_path, output_path, steps=0, device="cpu")
|
||||
|
||||
assert output_path.exists()
|
||||
assert len(pinned_targets) == 4
|
||||
assert all(width is not None and height is not None for width, height in pinned_targets)
|
||||
assert result.diagnostics["shared_density"]["max_relative_deviation"] < 0.12
|
||||
|
|
|
|||
Loading…
Reference in a new issue