Add stroke-aware tile graph slicing

This commit is contained in:
Meta Vens 2026-04-22 23:54:04 +01:00
commit fc62c77b6f
5 changed files with 297 additions and 19 deletions

View file

@ -46,6 +46,8 @@ Current tile-graph status:
- source-region connected-components now have a device-side Torch path, so the expensive labeling step no longer depends on Python flood fill
- on the real cleaned badge, the current tile-graph model build now takes about `9.48s` on CPU versus `6.37s` on CUDA while producing the same `2499` source regions and `23223` candidates
- the remaining large-fixture bottleneck is the per-component one-cell window cutting pass, which still hops back to CPU after labeling
- elongated source regions now get a stroke-aware slicing pass that follows their principal axis instead of only marching with cardinal queue steps, and the repo now has a shallow-stroke regression that checks for fan-out on that kind of component
- that stroke-aware slicer is a real synthetic improvement but not a real-badge win yet: the latest badge probe under `artifacts/badge-tile-graph-stroke-v2-cuda/` still keeps the initial assignment and slightly regresses source-fidelity (`0.1832` vs `0.1800`)
- this fixes the core design mismatch that had allowed repeated distant labels to create big same-color patches and opaque black background blocks
- on the current `24x24` emblem smoke case, an end-to-end `tile-graph` run dropped from about `2.57s` on CPU to `0.61s` on CUDA on this machine
- the older hard-edge-only candidate widening pass under `artifacts/full-emblem-tile-graph-hard-edge-v2-cuda/` remains a useful negative result: more edge choices alone sharpen some cells locally but still regress full-emblem source-fidelity (`0.0377`)

View file

@ -56,24 +56,25 @@ The most provisional areas are:
### Active pass
This pass replaced the custom Python source-region flood fill with a device-side connected-components pass, which is the right seam for the GPU labeling work the tile-graph path needed.
This pass tried the next geometry-focused experiment on tile-graph: cutting elongated source regions like strokes instead of treating them like generic blobs.
What landed:
- `tile_graph.py` now labels source-side atomic regions through a Torch CCL pass instead of a Python BFS when the source-region extractor runs on a device-backed path
- the region builder still projects those source regions onto output coords and layers them together with the older sharp/edge anchors
- tile-graph still keeps its initial assignment whenever the propagation loop would worsen source-lattice fidelity
- the focused CUDA-vs-CPU regression still matches after the CCL swap, and the full suite is green at `56 passed`
- `tile_graph.py` now computes principal-axis stats for source-side atomic regions and gives elongated components a stroke-aware slicing path
- stroke-like components are now seeded and cut along their fitted axis, and then thinned to one best candidate per dominant row or column band before they reach the per-cell candidate buckets
- a new shallow-stroke regression in `tests/test_tile_graph.py` checks that connected slanted strokes do not fan out across too many output cells
- tile-graph still keeps its initial assignment whenever the propagation loop would worsen source-lattice fidelity, because that safeguard is still doing real work on the badge
Current implementation note:
- source-region labeling is now device-backed, but the one-cell window cutting pass still runs as a CPU queue walk over the labeled members, so candidate extraction remains the dominant runtime cost on large real fixtures
- on the cleaned badge, the current tile-graph model build now takes about `9.48s` on CPU versus `6.37s` on CUDA while producing the same `2499` source regions and `23223` candidates
- source-region labeling is still device-backed, but the one-cell window cutting pass remains CPU-side and still dominates large-fixture runtime
- the stroke-aware slicer is a meaningful synthetic change, but it is not yet a real-badge quality win: the latest run under `artifacts/badge-tile-graph-stroke-v2-cuda/` lands at `0.1832`, slightly worse than the previous CCL-only tile-graph badge run at `0.1800`
- the latest badge stroke-aware run still keeps its initial assignment, which reinforces that the current badge weakness is mostly in initial candidate ownership/selection rather than in the later propagation loop
- the latest full-emblem atomic probe under `artifacts/full-emblem-tile-graph-atomic-v3-cuda/` lands at `0.0224` source-fidelity with `34278` candidates and about `2.16` average choices per cell
- that beats the earlier full-CUDA tile-graph baseline (`0.0283`) and materially improves on the first atomic-only attempt (`0.1571`), which chose atomic regions too eagerly and then let the solver blur them out again
- the latest cleaned-badge tile-graph CUDA probe under `artifacts/badge-tile-graph-ccl-cuda/` completes end-to-end and keeps its initial assignment at `0.1800` source-fidelity, which is not a quality win but confirms the new CCL seam is stable on the real stress fixture
Next after that:
- move the one-cell window cutting pass closer to the user's original "consume one-cell regions and step through larger regions" design now that the region ownership itself is source-side
- keep pushing the initial candidate builder toward true source-side stroke ownership instead of only better stroke slicing within the same builder
- port the per-component window cutting pass back toward a fully device-friendly formulation instead of only accelerating the labeling step
- decide whether the next stroke pass should use a real contour/skeleton trace instead of only a principal-axis approximation
- rerank low-confidence top lattice candidates with short real solver probes instead of relying only on the cheapest preview
- evaluate whether the tile-graph propagation loop should become more contour-aware or simply stay more conservative now that the initial assignment is stronger
- rerun tuning after the tile-graph objective and candidate sets stabilize
@ -101,6 +102,7 @@ Current status:
- on the full emblem, the tile-graph atomic path under `artifacts/full-emblem-tile-graph-atomic-v3-cuda/` now reaches `0.0224`, which is the best tile-graph full-emblem result so far in this repo
- the remaining weakness on tile-graph is not candidate color purity anymore; it is candidate extraction cost plus deciding how aggressive the propagation step should be once the initial assignment is already strong
- on the cleaned badge, the new CCL-backed tile-graph run under `artifacts/badge-tile-graph-ccl-cuda/` still lands at a weak `0.1800`, so the current CCL work should be treated as an infrastructure/performance step rather than a visual-quality fix
- the new stroke-aware badge run under `artifacts/badge-tile-graph-stroke-v2-cuda/` still does not solve the guard-contour problem and slightly regresses to `0.1832`, so the next stroke pass should be judged on whether it changes the source-side path model more fundamentally
Important note:
- the checkerboard is baked into the source image, not transparency

View file

@ -81,6 +81,9 @@ class SolverHyperParams:
tile_graph_source_region_stride: int = 0
tile_graph_source_region_min_area_ratio: float = 0.06
tile_graph_source_region_window_coverage: float = 0.12
tile_graph_stroke_linearity_threshold: float = 0.72
tile_graph_stroke_step_scale: float = 0.95
tile_graph_stroke_minor_limit_scale: float = 0.55
tile_graph_nonedge_sharp_weight: float = 0.85
tile_graph_nonedge_mean_weight: float = 0.15
tile_graph_edge_mean_weight: float = 0.03

View file

@ -87,6 +87,37 @@ def _rgba_luminance(rgba: np.ndarray) -> np.ndarray:
return rgba[..., 0] * 0.2126 + rgba[..., 1] * 0.7152 + rgba[..., 2] * 0.0722
def _component_axis_stats(member_x: np.ndarray, member_y: np.ndarray) -> tuple[float, float, float, float, float]:
if member_x.size <= 1:
return 1.0, 0.0, 0.0, 0.0, 0.0
centered_x = member_x.astype(np.float32) - float(np.mean(member_x))
centered_y = member_y.astype(np.float32) - float(np.mean(member_y))
covariance = np.asarray(
[
[float(np.mean(centered_x * centered_x)), float(np.mean(centered_x * centered_y))],
[float(np.mean(centered_x * centered_y)), float(np.mean(centered_y * centered_y))],
],
dtype=np.float32,
)
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
order = np.argsort(eigenvalues)
minor_value = float(max(0.0, eigenvalues[order[0]]))
major_value = float(max(0.0, eigenvalues[order[1]]))
major_vector = eigenvectors[:, order[1]].astype(np.float32)
norm = float(np.linalg.norm(major_vector))
if norm <= 1e-6:
return 1.0, 0.0, 0.0, 0.0, 0.0
major_vector /= norm
axis_x = float(major_vector[0])
axis_y = float(major_vector[1])
projections = centered_x * axis_x + centered_y * axis_y
perpendicular = centered_x * (-axis_y) + centered_y * axis_x
major_span = float(np.max(projections) - np.min(projections)) if projections.size else 0.0
minor_span = float(np.max(perpendicular) - np.min(perpendicular)) if perpendicular.size else 0.0
linearity = float(np.clip(1.0 - (minor_value / max(major_value, 1e-6)), 0.0, 1.0))
return axis_x, axis_y, linearity, major_span, minor_span
def _project_source_point_to_output_coord(
*,
x: float,
@ -168,6 +199,7 @@ def _segment_atomic_source_regions_cpu(
member_x = (member_array % width).astype(np.float32)
centroid_y = float(np.mean(member_y))
centroid_x = float(np.mean(member_x))
axis_x, axis_y, linearity, major_span, minor_span = _component_axis_stats(member_x, member_y)
centroid_dist = (member_y - centroid_y) ** 2 + (member_x - centroid_x) ** 2
centroid_pick = int(np.argmin(centroid_dist))
edge_pick = int(np.argmax(flat_edge[member_array]))
@ -180,6 +212,11 @@ def _segment_atomic_source_regions_cpu(
"centroid_linear": int(member_array[centroid_pick]),
"centroid_x": centroid_x,
"centroid_y": centroid_y,
"axis_x": axis_x,
"axis_y": axis_y,
"linearity": linearity,
"major_span": major_span,
"minor_span": minor_span,
"size": int(member_array.size),
}
)
@ -299,6 +336,7 @@ def _segment_atomic_source_regions(
member_x = (member_array % width).astype(np.float32)
centroid_y = float(np.mean(member_y))
centroid_x = float(np.mean(member_x))
axis_x, axis_y, linearity, major_span, minor_span = _component_axis_stats(member_x, member_y)
centroid_dist = (member_y - centroid_y) ** 2 + (member_x - centroid_x) ** 2
centroid_pick = int(np.argmin(centroid_dist))
edge_pick = int(np.argmax(flat_edge[member_array]))
@ -311,12 +349,100 @@ def _segment_atomic_source_regions(
"centroid_linear": int(member_array[centroid_pick]),
"centroid_x": centroid_x,
"centroid_y": centroid_y,
"axis_x": axis_x,
"axis_y": axis_y,
"linearity": linearity,
"major_span": major_span,
"minor_span": minor_span,
"size": int(member_array.size),
}
)
return components
def _stroke_window_mask(
*,
member_x: np.ndarray,
member_y: np.ndarray,
remaining: np.ndarray,
center_x: float,
center_y: float,
axis_x: float,
axis_y: float,
major_half: float,
minor_half: float,
) -> np.ndarray:
delta_x = member_x - float(center_x)
delta_y = member_y - float(center_y)
major_delta = delta_x * axis_x + delta_y * axis_y
minor_delta = delta_x * (-axis_y) + delta_y * axis_x
return remaining & (np.abs(major_delta) <= major_half) & (np.abs(minor_delta) <= minor_half)
def _stroke_seed_centers(
*,
component: dict[str, float | int | np.ndarray],
member_x: np.ndarray,
member_y: np.ndarray,
flat_x: np.ndarray,
flat_y: np.ndarray,
cell_w: float,
cell_h: float,
step_scale: float,
) -> list[tuple[float, float]]:
axis_x = float(component["axis_x"])
axis_y = float(component["axis_y"])
centroid_x = float(component["centroid_x"])
centroid_y = float(component["centroid_y"])
projection = (member_x - centroid_x) * axis_x + (member_y - centroid_y) * axis_y
if projection.size <= 0:
return [(centroid_y, centroid_x)]
step = max(
1.0,
(abs(axis_x) * cell_w + abs(axis_y) * cell_h) * float(step_scale),
np.hypot(axis_x * cell_w, axis_y * cell_h) * 0.9,
)
if step <= 1e-6:
return [(centroid_y, centroid_x)]
low = float(np.min(projection))
high = float(np.max(projection))
edge_peak_linear = int(component["edge_peak_linear"])
anchor_x = float(flat_x[edge_peak_linear])
anchor_y = float(flat_y[edge_peak_linear])
anchor_projection = (anchor_x - centroid_x) * axis_x + (anchor_y - centroid_y) * axis_y
if not np.isfinite(anchor_projection):
anchor_projection = 0.0
seed_projections: list[float] = [anchor_projection]
cursor = anchor_projection - step
while cursor >= low - 0.5 * step:
seed_projections.append(cursor)
cursor -= step
cursor = anchor_projection + step
while cursor <= high + 0.5 * step:
seed_projections.append(cursor)
cursor += step
if len(seed_projections) == 1 and abs(high - low) > 0.6 * step:
center_projection = 0.5 * (low + high)
seed_projections.extend([center_projection - 0.5 * step, center_projection + 0.5 * step])
seen: set[tuple[int, int]] = set()
centers: list[tuple[float, float]] = []
perpendicular = (member_x - centroid_x) * (-axis_y) + (member_y - centroid_y) * axis_x
for proj_center in sorted(seed_projections):
score = np.abs(projection - proj_center) + np.abs(perpendicular) * 0.35
pick = int(np.argmin(score))
center = (float(member_y[pick]), float(member_x[pick]))
key = (int(round(center[0] * 2.0)), int(round(center[1] * 2.0)))
if key in seen:
continue
seen.add(key)
centers.append(center)
if not centers:
centers.append((centroid_y, centroid_x))
return centers
def _extract_source_region_tiles(
*,
components: list[dict[str, float | int | np.ndarray]],
@ -333,6 +459,9 @@ def _extract_source_region_tiles(
phase_y: float,
min_region_area_ratio: float,
min_window_coverage: float,
stroke_linearity_threshold: float,
stroke_step_scale: float,
stroke_minor_limit_scale: float,
) -> list[list[dict[str, float | int | np.ndarray]]]:
output_area = max(1, target_width * target_height)
cell_area = max(cell_w * cell_h, 1e-6)
@ -356,6 +485,23 @@ def _extract_source_region_tiles(
seed_queue: deque[tuple[float, float]] = deque()
seen_seeds: set[tuple[int, int]] = set()
accepted_any = False
axis_x = float(component["axis_x"])
axis_y = float(component["axis_y"])
linearity = float(component["linearity"])
major_span = float(component["major_span"])
minor_span = float(component["minor_span"])
stroke_component = (
linearity >= stroke_linearity_threshold
and major_span >= max(cell_w, cell_h) * 1.1
and minor_span <= max(1.0, min(cell_w, cell_h) * stroke_minor_limit_scale)
)
stroke_major_half = max(
0.5,
0.5 * (abs(axis_x) * cell_w + abs(axis_y) * cell_h) * stroke_step_scale,
0.45 * np.hypot(axis_x * cell_w, axis_y * cell_h),
)
stroke_minor_half = max(0.5, 0.5 * min(cell_w, cell_h) * stroke_minor_limit_scale)
component_tiles: list[dict[str, float | int | np.ndarray]] = []
def enqueue(seed_y: float, seed_x: float) -> None:
key = (int(round(seed_y * 2.0)), int(round(seed_x * 2.0)))
@ -364,13 +510,41 @@ def _extract_source_region_tiles(
seen_seeds.add(key)
seed_queue.append((float(seed_y), float(seed_x)))
enqueue(float(component["centroid_y"]), float(component["centroid_x"]))
edge_peak_linear = int(component["edge_peak_linear"])
enqueue(float(flat_y[edge_peak_linear]), float(flat_x[edge_peak_linear]))
if stroke_component:
for seed_y, seed_x in _stroke_seed_centers(
component=component,
member_x=member_x,
member_y=member_y,
flat_x=flat_x,
flat_y=flat_y,
cell_w=cell_w,
cell_h=cell_h,
step_scale=stroke_step_scale,
):
enqueue(seed_y, seed_x)
else:
enqueue(float(component["centroid_y"]), float(component["centroid_x"]))
edge_peak_linear = int(component["edge_peak_linear"])
enqueue(float(flat_y[edge_peak_linear]), float(flat_x[edge_peak_linear]))
while seed_queue:
center_y, center_x = seed_queue.popleft()
in_window = remaining & (np.abs(member_x - center_x) <= half_w) & (np.abs(member_y - center_y) <= half_h)
if stroke_component:
in_window = _stroke_window_mask(
member_x=member_x,
member_y=member_y,
remaining=remaining,
center_x=center_x,
center_y=center_y,
axis_x=axis_x,
axis_y=axis_y,
major_half=stroke_major_half,
minor_half=stroke_minor_half,
)
if not np.any(in_window):
in_window = remaining & (np.abs(member_x - center_x) <= half_w) & (np.abs(member_y - center_y) <= half_h)
else:
in_window = remaining & (np.abs(member_x - center_x) <= half_w) & (np.abs(member_y - center_y) <= half_h)
footprint_count = int(np.count_nonzero(in_window))
if footprint_count <= 0:
continue
@ -383,10 +557,17 @@ def _extract_source_region_tiles(
window_x = member_x[in_window]
window_y = member_y[in_window]
window_edge = flat_edge[window_linear]
accepted_center_x = float(np.mean(window_x))
accepted_center_y = float(np.mean(window_y))
if stroke_component:
accepted_center_x = float(center_x * 0.75 + np.mean(window_x) * 0.25)
accepted_center_y = float(center_y * 0.75 + np.mean(window_y) * 0.25)
else:
accepted_center_x = float(np.mean(window_x))
accepted_center_y = float(np.mean(window_y))
edge_peak = float(np.max(window_edge)) if window_edge.size else 0.0
if edge_peak > 0.0:
if stroke_component:
center_dist = (window_x - accepted_center_x) ** 2 + (window_y - accepted_center_y) ** 2
rep_linear = int(window_linear[int(np.argmin(center_dist))])
elif edge_peak > 0.0:
rep_linear = int(window_linear[int(np.argmax(window_edge))])
else:
centroid_dist = (window_x - accepted_center_x) ** 2 + (window_y - accepted_center_y) ** 2
@ -402,7 +583,7 @@ def _extract_source_region_tiles(
phase_x=phase_x,
phase_y=phase_y,
)
buckets[flat_index].append(
component_tiles.append(
{
"rep_linear": rep_linear,
"rep_rgba": flat_rgba[rep_linear].astype(np.float32),
@ -413,15 +594,68 @@ def _extract_source_region_tiles(
"source_center_y": accepted_center_y,
"coord_x": coord_x,
"coord_y": coord_y,
"flat_index": flat_index,
}
)
remaining[in_window] = False
if int(np.count_nonzero(remaining)) < min_region_pixels:
break
for _, dy, dx in _DIRS:
enqueue(accepted_center_y + dy * cell_h, accepted_center_x + dx * cell_w)
if not stroke_component:
for _, dy, dx in _DIRS:
enqueue(accepted_center_y + dy * cell_h, accepted_center_x + dx * cell_w)
if accepted_any:
if stroke_component and component_tiles:
dominant_vertical = abs(axis_y) >= abs(axis_x)
grouped_tiles: dict[int, list[dict[str, float | int | np.ndarray]]] = {}
for tile in component_tiles:
group_key = int(tile["coord_y"]) if dominant_vertical else int(tile["coord_x"])
grouped_tiles.setdefault(group_key, []).append(tile)
filtered_tiles: list[dict[str, float | int | np.ndarray]] = []
ordered_keys = sorted(grouped_tiles)
secondary_step = (
axis_x / max(abs(axis_y), 1e-6)
if dominant_vertical
else axis_y / max(abs(axis_x), 1e-6)
)
previous_primary: int | None = None
previous_secondary: float | None = None
def tile_score(
tile: dict[str, float | int | np.ndarray],
*,
expected_secondary: float | None,
) -> tuple[float, float, float, float]:
continuous_coord = (
(float(tile["source_center_x"]) + 0.5) / max(cell_w, 1e-6) - phase_x
if dominant_vertical
else (float(tile["source_center_y"]) + 0.5) / max(cell_h, 1e-6) - phase_y
)
discrete_coord = float(tile["coord_x"] if dominant_vertical else tile["coord_y"])
path_error = abs(discrete_coord - expected_secondary) if expected_secondary is not None else 0.0
return (
path_error,
abs(discrete_coord - continuous_coord),
abs(float(tile["area_ratio"]) - 1.0),
-float(tile["coverage"]),
)
for group_key in ordered_keys:
group_tiles = grouped_tiles[group_key]
expected_secondary = None
if previous_primary is not None and previous_secondary is not None:
expected_secondary = previous_secondary + secondary_step * float(group_key - previous_primary)
best_tile = min(
group_tiles,
key=lambda tile: tile_score(tile, expected_secondary=expected_secondary),
)
filtered_tiles.append(best_tile)
previous_primary = group_key
previous_secondary = float(best_tile["coord_x"] if dominant_vertical else best_tile["coord_y"])
component_tiles = filtered_tiles
for tile in component_tiles:
buckets[int(tile["flat_index"])].append(tile)
continue
component_area_ratio = float((component_size * sample_area) / cell_area)
@ -595,6 +829,9 @@ def build_tile_graph_model(
phase_y=inference.phase_y,
min_region_area_ratio=float(getattr(solver_params, "tile_graph_source_region_min_area_ratio", 0.06)),
min_window_coverage=float(getattr(solver_params, "tile_graph_source_region_window_coverage", 0.12)),
stroke_linearity_threshold=float(getattr(solver_params, "tile_graph_stroke_linearity_threshold", 0.72)),
stroke_step_scale=float(getattr(solver_params, "tile_graph_stroke_step_scale", 0.95)),
stroke_minor_limit_scale=float(getattr(solver_params, "tile_graph_stroke_minor_limit_scale", 0.55)),
)
candidate_linear: list[int] = []

View file

@ -250,6 +250,40 @@ def test_tile_graph_reconstructs_synthetic_thin_feature_better_than_naive() -> N
assert tile_score <= naive_score
def test_tile_graph_keeps_shallow_connected_stroke_from_fanning_out() -> None:
source = np.zeros((48, 48, 4), dtype=np.float32)
for y in range(48):
x = int(round(8 + y * 0.65))
source[max(0, y - 1) : min(48, y + 2), max(0, x - 1) : min(48, x + 2)] = np.asarray(
[1.0, 1.0, 1.0, 1.0],
dtype=np.float32,
)
artifacts, _ = optimize_tile_graph(
source,
inference=InferenceResult(
target_width=6,
target_height=6,
phase_x=0.0,
phase_y=0.0,
confidence=1.0,
top_candidates=[],
),
analysis=analyze_source(source, seed=7),
steps=0,
seed=7,
device="cpu",
)
occupied_per_row = [
int(np.count_nonzero(artifacts.target_rgba[y, :, 3] > 0.5))
for y in range(artifacts.target_rgba.shape[0])
]
assert sum(count > 1 for count in occupied_per_row) <= 1
assert sum(occupied_per_row) <= artifacts.target_rgba.shape[0] + 1
def test_tile_graph_preserves_transparent_background_on_sparse_detail() -> None:
source = np.zeros((12, 12, 4), dtype=np.float32)
source[5:7, 5:7] = np.asarray([0.0, 0.0, 0.0, 1.0], dtype=np.float32)