GameCult/repixelizer
Watch
9
0
Fork
You've already forked repixelizer
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Document the repixelizer project context

Browse source
This commit is contained in:
Meta Vens 2026-04-21 16:15:05 +01:00
commit 634ca7f002
3 changed files with 447 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

159
README.md
View file

@ -1,62 +1,157 @@
# Repixelizer
Repixelizer is a standalone Python CLI that turns "fake pixel art" into true grid-aligned pixel art for icons and sprites.
Repixelizer is a standalone Python CLI for turning "fake pixel art" into true grid-aligned pixel art.
It does not rely on a single resize pass. Instead it:
The core idea is simple: generated images often look like pixel art without actually obeying a single clean pixel grid. Repixelizer treats that as an optimization problem instead of a resize problem. It estimates the implied lattice, builds a real target grid, optimizes how output pixels sample the source image, then cleans the result up on the discrete grid.
1. infers the source image's implied pixel lattice,
2. initializes a target grid at that resolution,
3. runs a continuous UV optimization stage in premultiplied RGBA space,
4. projects the result onto a real pixel grid with discrete cleanup,
5. optionally constrains the output to a palette, and
6. writes diagnostics and baseline comparisons.
This project is meant to stand on its own. It should be understandable without any StreamPixels-specific context.
## Installation
## What it does
Repixelizer currently focuses on single-image inputs such as:
- icons
- emblems
- logos
- simple sprites
The current pipeline is:
1. infer the source image's implied fake-pixel lattice
2. choose a target output resolution, unless the user overrides it
3. initialize a UV/sample field over the target grid
4. optimize that field in premultiplied RGBA space with PyTorch
5. project onto a true pixel grid
6. run discrete cleanup to reduce noise and isolated-pixel garbage
7. optionally quantize into a palette-constrained result
8. write diagnostics and baseline comparisons
## Project docs
- Product and technical spec: [docs/spec.md](</E:/Projects/repixelizer/docs/spec.md>)
- Living implementation plan: [docs/implementation-plan.md](</E:/Projects/repixelizer/docs/implementation-plan.md>)
If you are moving this project into a fresh workspace, those two docs are the main handoff.
## Quickstart
```powershell
python -m venv .venv
.venv\Scripts\python -m pip install -e .[dev]
```
## Usage
Run the optimizer:
```powershell
repixelize input.png --out output.png
repixelize input.png --out output.png --target-size 120 --diagnostics-dir diagnostics
```
Run the optimizer plus baselines:
```powershell
repixelize compare input.png --out output.png --diagnostics-dir diagnostics
```
## CLI contract
Primary command shape:
```powershell
repixelize input.png --out output.png
repixelize compare input.png --out output.png
```
Important flags:
- `--target-size <n>`: override inferred target max dimension
- `--palette <file>`: palette file in `.gpl`, `.txt`, or `.json`
- `--palette-mode off|fit|strict`: palette behavior
- `--diagnostics-dir <path>`: write visual and JSON diagnostics
- `--seed <n>`: deterministic optimizer seed
- `--steps <n>`: number of optimization steps
- `--device cpu|cuda`: PyTorch device
### Palette modes
- `off`: keep the full-color RGBA output.
- `fit`: derive or adapt to a useful palette and write palette diagnostics.
- `strict`: require a supplied palette file and stay inside it exactly.
- `off`: keep the output unconstrained
- `fit`: adapt the result to a derived or supplied palette
- `strict`: stay strictly inside the supplied palette
Palette files can be `.gpl`, `.txt`, or `.json`.
Palette constraints are optional by design. Many fake-pixel-art inputs are not born from a coherent palette, so palette enforcement is useful but not assumed.
## Diagnostics
## Outputs
When `--diagnostics-dir` is supplied, Repixelizer writes:
Main output:
- final RGBA PNG
- `run.json`: chosen size, phase, score breakdown, timings, and settings
- `lattice-overlay.png`: inferred lattice drawn over the source
Optional diagnostics:
- `run.json`: chosen size, phase, settings, timings, and score breakdown
- `lattice-overlay.png`: inferred lattice preview over the source image
- `comparison.png`: source, optimized output, and nearest-neighbor preview
- `alpha-preview.png`: source/result alpha
- `noise-heatmap.png`: local noise and isolated-pixel hotspots
- `alpha-preview.png`: alpha behavior before and after
- `noise-heatmap.png`: isolated-pixel and local-noise hotspots
- `cluster-preview.png`: coarse material/color clustering
- compare mode extras:
- `compare.json`
- `compare.csv`
- `compare-sheet.png`
## Architecture
Additional compare-mode outputs:
- `compare.json`
- `compare.csv`
- `compare-sheet.png`
- `repixelizer.inference`: lattice and target-size inference
- `repixelizer.continuous`: PyTorch UV-field optimization
- `repixelizer.discrete`: local-search cleanup on the final grid
- `repixelizer.baselines`: naive and error-diffusion baselines
- `repixelizer.diagnostics`: machine-readable and visual debug outputs
## Repo layout
## Current scope
- `src/repixelizer`: core package
- `tests`: synthetic and pipeline-focused tests
- `examples`: reserved for future real example inputs
- `artifacts`: local smoke-test outputs, not core source
Version `0.1.0` is single-image only. It is aimed at icons, emblems, and simple sprites first, with automatic resolution inference and no manual masks.
Core modules:
- `inference.py`: target-size and phase inference
- `analysis.py`: edge, alpha, and coarse clustering analysis
- `continuous.py`: UV-field optimization
- `discrete.py`: local cleanup after projection
- `palette.py`: palette loading and optional quantization
- `compare.py`: baseline comparisons and contact-sheet output
- `synthetic.py`: benchmark generators for tests
## Current status
The project is bootstrapped and runnable:
- editable install works
- PyTorch-based optimization is wired in
- synthetic tests are in place
- compare mode writes diagnostics and baseline metrics
What exists today is a serious experimental baseline, not a finished research result. It is strong enough to iterate on algorithmically, but it is not yet proven across a broad real-world image corpus.
## Current limitations
- single-image only
- no multi-frame sprite-sheet consistency yet
- no GUI yet
- no user-painted masks or region hints yet
- lattice inference is heuristic, not learned
- continuous optimization is real, but still early and intentionally conservative
## Validation
The current test suite checks:
- resolution inference on synthetic emblem inputs
- palette loading and strict palette enforcement
- pipeline output and diagnostics writing
- compare-mode artifact generation
- a basic default CLI execution path
Run it with:
```powershell
.venv\Scripts\python -m pytest -q
```
## Near-term direction
The next useful steps are:
- improve inference on real-world generated inputs
- make the optimizer outperform resize-and-dither baselines more consistently
- expand the curated benchmark corpus
- add smarter discrete cleanup rules for outlines, highlights, and material bands
- optionally add manual guidance in a later version, once the automatic baseline is stable

117
docs/implementation-plan.md Normal file
View file

@ -0,0 +1,117 @@
# Repixelizer Implementation Plan
## Current state
The project is bootstrapped and runnable as a standalone Python repo.
Implemented:
- package metadata and editable install
- CLI entrypoint with `run` and `compare` modes
- lattice/phase inference with coherence scoring plus source-period prior
- source-image structural analysis
- PyTorch-based continuous UV-field optimization
- discrete cleanup on the output grid
- optional palette loading and quantization modes
- compare mode with naive and diffusion baselines
- diagnostics writing
- synthetic test fixtures and automated tests
Verified:
- local editable install in a dedicated venv
- full test suite currently passing
- compare-mode smoke run against a real emblem image
## What is still provisional
The current implementation should be treated as a strong experimental baseline, not a final algorithm.
The most provisional areas are:
- lattice inference on real, messy generated imagery
- the exact loss weighting in the continuous stage
- the strength and shape of discrete cleanup heuristics
- compare-mode metrics as a proxy for actual visual quality
## Near-term priorities
### 1. Build a real benchmark corpus
Add curated real-world cases, not just synthetic ones:
- emblems and logos with transparency
- icons with strong outlines
- simple sprite-like figures
- examples that are known to fail naive resize-and-dither
Each case should include:
- source image
- expected target size or plausible size range
- short notes on known failure modes
### 2. Improve lattice inference
The current inference is workable but still heuristic.
Next improvements should focus on:
- better periodicity detection for noisy or partially aligned images
- stronger penalties against oversized candidate grids
- confidence calibration that reflects ambiguity more honestly
### 3. Strengthen the discrete cleanup stage
Current cleanup is local and generic.
Next version should add:
- stronger outline continuity rules
- better preservation of highlight bands
- better handling of metallic or gem-like materials
- smarter alpha-edge cleanup for transparent icons
### 4. Make compare mode more trustworthy
The current metrics are useful but still internal.
Next work should:
- expand score breakdowns
- validate whether the metrics correlate with actual human preference
- make it easier to compare runs across parameter sweeps
## Suggested next milestones
### Milestone 1: Real-image evaluation harness
- add curated real-image fixtures
- add golden metadata for expected target-size ranges
- add a repeatable batch runner
- emit one summary table per corpus run
### Milestone 2: Solver tuning pass
- tune continuous-stage loss weights against the benchmark corpus
- test more robust anti-speckle and cluster-preservation behavior
- reduce cases where the solver underperforms the naive baseline
### Milestone 3: Better cleanup heuristics
- add contour-aware cleanup
- add material-sensitive smoothing for highlights and trim
- reduce the number of visually good-but-metric-weak outputs
### Milestone 4: Human-in-the-loop v2 planning
Only after the automatic baseline is strong:
- explore optional region hints
- evaluate a minimal GUI or notebook layer
- consider multi-frame support
## Risks
- The solver can still produce outputs that score well but feel over-smoothed.
- Real generated images may contain multiple inconsistent local lattice cues.
- Continuous optimization is flexible, but easy to overfit to the reconstruction term.
- Metrics that reward coherence can accidentally reward blandness if not balanced carefully.
## Assumptions
- Single-image quality matters more than batch throughput in the current phase.
- Palette constraints are optional and should stay optional.
- The project is better served by improving automatic behavior first than by rushing into manual tooling.
- Python plus PyTorch remains the right stack for the current research/prototyping phase.

203
docs/spec.md Normal file
View file

@ -0,0 +1,203 @@
# Repixelizer Spec
## Summary
Repixelizer is a standalone Python CLI for converting "fake pixel art" into true grid-aligned pixel art. The target use cases are single-image icons, emblems, logos, and simple sprites, especially images that already resemble pixel art but fail to obey a consistent pixel lattice.
The project exists because generated imagery often creates appealing pixel-like local patterns while violating the actual constraints of pixel art:
- inconsistent subpixel alignment
- locally plausible but globally incoherent clusters
- noisy highlights and outlines
- fake micro-detail that collapses when downsampled
Repixelizer treats this as an optimization and reconstruction problem rather than a resizing problem.
## Goals
- Recover an implied target pixel grid from a fake-pixel-art source image.
- Produce a true discrete-grid output image that looks authored, not merely resized.
- Preserve alpha, silhouette clarity, and strong interior structure.
- Support palette-free output by default, with optional palette-constrained modes.
- Provide diagnostics and baseline comparisons so results are inspectable and measurable.
## Non-goals for v1
- full sprite-sheet or animation consistency
- manual painting or mask authoring as a required workflow
- a desktop GUI
- generalized photo-to-pixel-art stylization
- a learned model trained on a large dataset
## Primary users
- artists or developers trying to salvage generated fake pixel art
- tool builders experimenting with constrained repixelization
- game/UI developers creating icons, emblems, badges, and simple sprites
## Input and output contract
### Input
- one RGBA image file
- optional explicit target size override
- optional palette file
- optional diagnostics directory
- optional solver/device settings
### Output
- one final RGBA PNG
- optional palette-constrained output behavior
- optional diagnostics bundle containing JSON and visual artifacts
## Product behavior
### Default mode
The default workflow should be fully automatic:
1. load the source image
2. infer lattice size and phase
3. analyze source structure
4. optimize a continuous UV/sample field
5. project that result to a real pixel grid
6. clean up obvious discrete-grid artifacts
7. optionally quantize to a palette
8. write output and diagnostics
No manual masks or user-authored region hints are assumed in v1.
### CLI shape
Primary commands:
```powershell
repixelize input.png --out output.png
repixelize compare input.png --out output.png
```
Supported flags:
- `--target-size`
- `--palette`
- `--palette-mode off|fit|strict`
- `--diagnostics-dir`
- `--seed`
- `--steps`
- `--device cpu|cuda`
### Palette behavior
- `off`: produce unconstrained RGBA output
- `fit`: adapt to a useful palette derived from or compatible with the result
- `strict`: stay strictly inside a supplied palette
Palette constraints are optional because many generated fake-pixel-art images do not originate from a coherent palette.
## Technical design
### 1. Lattice inference
The tool must estimate the resolution of the fake lattice being mimicked.
Requirements:
- search over candidate target sizes
- search over subpixel phase offsets for each candidate
- score candidates using coherence-oriented metrics rather than reconstruction alone
- expose the top-ranked alternatives in diagnostics
- if `--target-size` is provided, skip size search but still estimate phase
Scoring goals:
- low isolated-pixel rate
- high cluster continuity
- strong alpha-edge crispness
- straighter outlines
- lower local color chatter
- reasonable agreement with source-derived periodicity priors
### 2. Source analysis
The solver should derive automatic guidance from the image itself:
- alpha map
- edge map
- coarse material/color clustering
This guidance is used to preserve important boundaries while allowing smoother regions to collapse into cleaner pixel clusters.
### 3. Continuous optimization
The continuous stage represents each output pixel by a source UV/sample coordinate.
Requirements:
- operate in premultiplied RGBA space
- use PyTorch autograd
- optimize a UV field initialized from the inferred lattice
- preserve local adjacency instead of letting the sample field scramble neighborhoods
Loss components should include:
- reconstruction loss against the source image
- low-distortion regularization against the initialized UV grid
- edge-aware smoothing so hard boundaries stay crisp
- anti-speckle pressure to reduce isolated outlier pixels
- alpha regularization to avoid muddy semi-transparent edges
### 4. Discrete projection and cleanup
After the continuous stage, the result must be treated as a true pixel grid.
The discrete stage should:
- evaluate local neighborhoods
- remove orphan pixels where possible
- merge obvious micro-clusters
- reduce accidental highlight static
- avoid damaging strong guided boundaries
This stage is intentionally heuristic and local in v1.
### 5. Baselines and comparison mode
The project must ship comparison baselines:
- naive resize
- resize plus error diffusion
`compare` mode should run:
- the optimized solver
- both baselines
- metric collection
- a visual contact sheet
The project should treat these baselines as the minimum bar for usefulness.
## Diagnostics
Diagnostics should be machine-readable and human-readable.
Required artifacts:
- `run.json`
- `lattice-overlay.png`
- `comparison.png`
- `alpha-preview.png`
- `noise-heatmap.png`
- `cluster-preview.png`
Compare mode should additionally write:
- `compare.json`
- `compare.csv`
- `compare-sheet.png`
## Acceptance criteria
The tool is successful for v1 if it can:
- infer sensible target sizes on curated synthetic cases
- preserve transparency and silhouette quality
- produce outputs that are visibly more grid-coherent than simple resize baselines
- avoid obvious isolated-pixel explosions
- generate diagnostics that explain how and why a run behaved the way it did
## Future expansion
The design should leave room for:
- manual region hints or lock zones
- multi-frame consistency for sprite sheets
- GUI tooling
- more advanced warping or lattice-field models
- learned priors or ranking models for candidate outputs