repixelizer/docs/spec.md

Repixelizer Spec

Summary

Repixelizer is a standalone Python CLI for converting "fake pixel art" into true grid-aligned pixel art. The target use cases are single-image icons, emblems, logos, and simple sprites, especially images that already resemble pixel art but fail to obey a consistent pixel lattice.

The project exists because generated imagery often creates appealing pixel-like local patterns while violating the actual constraints of pixel art:

• inconsistent subpixel alignment
• locally plausible but globally incoherent clusters
• noisy highlights and outlines
• fake micro-detail that collapses when downsampled

Repixelizer treats this as an optimization and reconstruction problem rather than a resizing problem.

Goals

• Recover an implied target pixel grid from a fake-pixel-art source image.
• Produce a true discrete-grid output image that looks authored, not merely resized.
• Preserve alpha, silhouette clarity, and strong interior structure.
• Support palette-free output by default, with optional palette-constrained modes.
• Provide diagnostics and baseline comparisons so results are inspectable and measurable.

Non-goals for v1

• full sprite-sheet or animation consistency
• manual painting or mask authoring as a required workflow
• a desktop GUI
• generalized photo-to-pixel-art stylization
• a learned model trained on a large dataset

Primary users

• artists or developers trying to salvage generated fake pixel art
• tool builders experimenting with constrained repixelization
• game/UI developers creating icons, emblems, badges, and simple sprites

Input and output contract

Input

• one RGBA image file
• optional explicit target size override
• optional palette file
• optional diagnostics directory
• optional solver/device settings

Output

• one final RGBA PNG
• optional palette-constrained output behavior
• optional diagnostics bundle containing JSON and visual artifacts

Product behavior

Default mode

The default workflow should be fully automatic:

1. load the source image
2. infer lattice size and phase
3. analyze source structure
4. optimize a continuous UV/sample field
5. project that result to a real pixel grid
6. clean up obvious discrete-grid artifacts
7. optionally quantize to a palette
8. write output and diagnostics

No manual masks or user-authored region hints are assumed in v1.

CLI shape

Primary commands:

repixelize input.png --out output.png
repixelize compare input.png --out output.png

Supported flags:

• --target-size
• --palette
• --palette-mode off|fit|strict
• --diagnostics-dir
• --seed
• --steps
• --device cpu|cuda

Palette behavior

• off: produce unconstrained RGBA output
• fit: adapt to a useful palette derived from or compatible with the result
• strict: stay strictly inside a supplied palette

Palette constraints are optional because many generated fake-pixel-art images do not originate from a coherent palette.

Technical design

1. Lattice inference

The tool must estimate the resolution of the fake lattice being mimicked.

Requirements:

• search over candidate target sizes
• search over subpixel phase offsets for each candidate
• score candidates using coherence-oriented metrics rather than reconstruction alone
• expose the top-ranked alternatives in diagnostics
• if --target-size is provided, skip size search but still estimate phase

Scoring goals:

• low isolated-pixel rate
• high cluster continuity
• strong alpha-edge crispness
• straighter outlines
• lower local color chatter
• reasonable agreement with source-derived periodicity priors

2. Source analysis

The solver should derive automatic guidance from the image itself:

• alpha map
• edge map
• coarse material/color clustering

This guidance is used to preserve important boundaries while allowing smoother regions to collapse into cleaner pixel clusters.

3. Continuous optimization

The continuous stage represents each output pixel by a source UV/sample coordinate.

Requirements:

• operate in premultiplied RGBA space
• use PyTorch autograd
• optimize a UV field initialized from the inferred lattice
• preserve local adjacency instead of letting the sample field scramble neighborhoods

Loss components should include:

• reconstruction loss against the source image
• low-distortion regularization against the initialized UV grid
• edge-aware smoothing so hard boundaries stay crisp
• anti-speckle pressure to reduce isolated outlier pixels
• alpha regularization to avoid muddy semi-transparent edges

4. Discrete projection and cleanup

After the continuous stage, the result must be treated as a true pixel grid.

The discrete stage should:

• evaluate local neighborhoods
• remove orphan pixels where possible
• merge obvious micro-clusters
• reduce accidental highlight static
• avoid damaging strong guided boundaries

This stage is intentionally heuristic and local in v1.

5. Baselines and comparison mode

The project must ship comparison baselines:

• naive resize
• resize plus error diffusion

compare mode should run:

• the optimized solver
• both baselines
• metric collection
• a visual contact sheet

The project should treat these baselines as the minimum bar for usefulness.

Diagnostics

Diagnostics should be machine-readable and human-readable.

Required artifacts:

• run.json
• lattice-overlay.png
• comparison.png
• alpha-preview.png
• noise-heatmap.png
• cluster-preview.png

Compare mode should additionally write:

• compare.json
• compare.csv
• compare-sheet.png

Acceptance criteria

The tool is successful for v1 if it can:

• infer sensible target sizes on curated synthetic cases
• preserve transparency and silhouette quality
• produce outputs that are visibly more grid-coherent than simple resize baselines
• avoid obvious isolated-pixel explosions
• generate diagnostics that explain how and why a run behaved the way it did

Future expansion

The design should leave room for:

• manual region hints or lock zones
• multi-frame consistency for sprite sheets
• GUI tooling
• more advanced warping or lattice-field models
• learned priors or ranking models for candidate outputs