Document the repixelizer project context
This commit is contained in:
parent
aceefe46b2
commit
634ca7f002
3 changed files with 447 additions and 32 deletions
203
docs/spec.md
Normal file
203
docs/spec.md
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
# Repixelizer Spec
|
||||
|
||||
## Summary
|
||||
|
||||
Repixelizer is a standalone Python CLI for converting "fake pixel art" into true grid-aligned pixel art. The target use cases are single-image icons, emblems, logos, and simple sprites, especially images that already resemble pixel art but fail to obey a consistent pixel lattice.
|
||||
|
||||
The project exists because generated imagery often creates appealing pixel-like local patterns while violating the actual constraints of pixel art:
|
||||
- inconsistent subpixel alignment
|
||||
- locally plausible but globally incoherent clusters
|
||||
- noisy highlights and outlines
|
||||
- fake micro-detail that collapses when downsampled
|
||||
|
||||
Repixelizer treats this as an optimization and reconstruction problem rather than a resizing problem.
|
||||
|
||||
## Goals
|
||||
|
||||
- Recover an implied target pixel grid from a fake-pixel-art source image.
|
||||
- Produce a true discrete-grid output image that looks authored, not merely resized.
|
||||
- Preserve alpha, silhouette clarity, and strong interior structure.
|
||||
- Support palette-free output by default, with optional palette-constrained modes.
|
||||
- Provide diagnostics and baseline comparisons so results are inspectable and measurable.
|
||||
|
||||
## Non-goals for v1
|
||||
|
||||
- full sprite-sheet or animation consistency
|
||||
- manual painting or mask authoring as a required workflow
|
||||
- a desktop GUI
|
||||
- generalized photo-to-pixel-art stylization
|
||||
- a learned model trained on a large dataset
|
||||
|
||||
## Primary users
|
||||
|
||||
- artists or developers trying to salvage generated fake pixel art
|
||||
- tool builders experimenting with constrained repixelization
|
||||
- game/UI developers creating icons, emblems, badges, and simple sprites
|
||||
|
||||
## Input and output contract
|
||||
|
||||
### Input
|
||||
|
||||
- one RGBA image file
|
||||
- optional explicit target size override
|
||||
- optional palette file
|
||||
- optional diagnostics directory
|
||||
- optional solver/device settings
|
||||
|
||||
### Output
|
||||
|
||||
- one final RGBA PNG
|
||||
- optional palette-constrained output behavior
|
||||
- optional diagnostics bundle containing JSON and visual artifacts
|
||||
|
||||
## Product behavior
|
||||
|
||||
### Default mode
|
||||
|
||||
The default workflow should be fully automatic:
|
||||
|
||||
1. load the source image
|
||||
2. infer lattice size and phase
|
||||
3. analyze source structure
|
||||
4. optimize a continuous UV/sample field
|
||||
5. project that result to a real pixel grid
|
||||
6. clean up obvious discrete-grid artifacts
|
||||
7. optionally quantize to a palette
|
||||
8. write output and diagnostics
|
||||
|
||||
No manual masks or user-authored region hints are assumed in v1.
|
||||
|
||||
### CLI shape
|
||||
|
||||
Primary commands:
|
||||
|
||||
```powershell
|
||||
repixelize input.png --out output.png
|
||||
repixelize compare input.png --out output.png
|
||||
```
|
||||
|
||||
Supported flags:
|
||||
- `--target-size`
|
||||
- `--palette`
|
||||
- `--palette-mode off|fit|strict`
|
||||
- `--diagnostics-dir`
|
||||
- `--seed`
|
||||
- `--steps`
|
||||
- `--device cpu|cuda`
|
||||
|
||||
### Palette behavior
|
||||
|
||||
- `off`: produce unconstrained RGBA output
|
||||
- `fit`: adapt to a useful palette derived from or compatible with the result
|
||||
- `strict`: stay strictly inside a supplied palette
|
||||
|
||||
Palette constraints are optional because many generated fake-pixel-art images do not originate from a coherent palette.
|
||||
|
||||
## Technical design
|
||||
|
||||
### 1. Lattice inference
|
||||
|
||||
The tool must estimate the resolution of the fake lattice being mimicked.
|
||||
|
||||
Requirements:
|
||||
- search over candidate target sizes
|
||||
- search over subpixel phase offsets for each candidate
|
||||
- score candidates using coherence-oriented metrics rather than reconstruction alone
|
||||
- expose the top-ranked alternatives in diagnostics
|
||||
- if `--target-size` is provided, skip size search but still estimate phase
|
||||
|
||||
Scoring goals:
|
||||
- low isolated-pixel rate
|
||||
- high cluster continuity
|
||||
- strong alpha-edge crispness
|
||||
- straighter outlines
|
||||
- lower local color chatter
|
||||
- reasonable agreement with source-derived periodicity priors
|
||||
|
||||
### 2. Source analysis
|
||||
|
||||
The solver should derive automatic guidance from the image itself:
|
||||
- alpha map
|
||||
- edge map
|
||||
- coarse material/color clustering
|
||||
|
||||
This guidance is used to preserve important boundaries while allowing smoother regions to collapse into cleaner pixel clusters.
|
||||
|
||||
### 3. Continuous optimization
|
||||
|
||||
The continuous stage represents each output pixel by a source UV/sample coordinate.
|
||||
|
||||
Requirements:
|
||||
- operate in premultiplied RGBA space
|
||||
- use PyTorch autograd
|
||||
- optimize a UV field initialized from the inferred lattice
|
||||
- preserve local adjacency instead of letting the sample field scramble neighborhoods
|
||||
|
||||
Loss components should include:
|
||||
- reconstruction loss against the source image
|
||||
- low-distortion regularization against the initialized UV grid
|
||||
- edge-aware smoothing so hard boundaries stay crisp
|
||||
- anti-speckle pressure to reduce isolated outlier pixels
|
||||
- alpha regularization to avoid muddy semi-transparent edges
|
||||
|
||||
### 4. Discrete projection and cleanup
|
||||
|
||||
After the continuous stage, the result must be treated as a true pixel grid.
|
||||
|
||||
The discrete stage should:
|
||||
- evaluate local neighborhoods
|
||||
- remove orphan pixels where possible
|
||||
- merge obvious micro-clusters
|
||||
- reduce accidental highlight static
|
||||
- avoid damaging strong guided boundaries
|
||||
|
||||
This stage is intentionally heuristic and local in v1.
|
||||
|
||||
### 5. Baselines and comparison mode
|
||||
|
||||
The project must ship comparison baselines:
|
||||
- naive resize
|
||||
- resize plus error diffusion
|
||||
|
||||
`compare` mode should run:
|
||||
- the optimized solver
|
||||
- both baselines
|
||||
- metric collection
|
||||
- a visual contact sheet
|
||||
|
||||
The project should treat these baselines as the minimum bar for usefulness.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Diagnostics should be machine-readable and human-readable.
|
||||
|
||||
Required artifacts:
|
||||
- `run.json`
|
||||
- `lattice-overlay.png`
|
||||
- `comparison.png`
|
||||
- `alpha-preview.png`
|
||||
- `noise-heatmap.png`
|
||||
- `cluster-preview.png`
|
||||
|
||||
Compare mode should additionally write:
|
||||
- `compare.json`
|
||||
- `compare.csv`
|
||||
- `compare-sheet.png`
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
The tool is successful for v1 if it can:
|
||||
- infer sensible target sizes on curated synthetic cases
|
||||
- preserve transparency and silhouette quality
|
||||
- produce outputs that are visibly more grid-coherent than simple resize baselines
|
||||
- avoid obvious isolated-pixel explosions
|
||||
- generate diagnostics that explain how and why a run behaved the way it did
|
||||
|
||||
## Future expansion
|
||||
|
||||
The design should leave room for:
|
||||
- manual region hints or lock zones
|
||||
- multi-frame consistency for sprite sheets
|
||||
- GUI tooling
|
||||
- more advanced warping or lattice-field models
|
||||
- learned priors or ranking models for candidate outputs
|
||||
Loading…
Reference in a new issue