Add layout structure quality metrics
This commit is contained in:
parent
50d4062c08
commit
a6f0ca653d
5 changed files with 280 additions and 5 deletions
|
|
@ -170,6 +170,10 @@ cargo run --release --example organ_battleground
|
|||
It compares raw node approximations and body-force candidates on synthetic
|
||||
fixtures plus SNAP email/Gnutella datasets when available.
|
||||
|
||||
Use `evaluate_layout_quality(&graph, &layout)` to score fold compactness, group
|
||||
separation, rank error, edge length variance, and bridge length variance for a
|
||||
layout snapshot.
|
||||
|
||||
## Graph Analysis And Folding
|
||||
|
||||
The crate exposes `analyze(&graph)` for layout-independent structure reads:
|
||||
|
|
|
|||
|
|
@ -91,6 +91,8 @@ Current state:
|
|||
scale, and aggregate far-field scale.
|
||||
- `organ_battleground` evaluates raw and body-aware force candidates on
|
||||
synthetic fixtures plus SNAP email/Gnutella datasets.
|
||||
- `evaluate_layout_quality` reports fold compactness, group separation, rank
|
||||
error, edge length variance, and bridge length variance for snapshots.
|
||||
|
||||
Source:
|
||||
https://doi.org/10.1371/journal.pone.0098679
|
||||
|
|
|
|||
|
|
@ -307,6 +307,82 @@ The next battleground must score structure, not only force error:
|
|||
The organs are real, but they are not proven useful yet. Good. Better a failed
|
||||
organ in a jar than one quietly installed in the patient.
|
||||
|
||||
## Third Sweep: Structure Metrics
|
||||
|
||||
Date: 2026-05-07
|
||||
|
||||
New metrics added via `evaluate_layout_quality`:
|
||||
|
||||
- mean fold compactness
|
||||
- mean group separation
|
||||
- mean rank error
|
||||
- edge length mean and variance
|
||||
- bridge length mean and variance
|
||||
|
||||
`organ_battleground` now emits both force-error metrics and structure metrics.
|
||||
|
||||
Representative corrected results:
|
||||
|
||||
| Dataset | Candidate | Force Time | Mean Rel Error | Fold Compactness | Group Separation | Rank Error | Note |
|
||||
|---|---:|---:|---:|---:|---:|---:|---|
|
||||
| synthetic_layered_512 | exact | 857 us | 0.000 | 61.660 | 0.029 | 128.258 | Baseline. |
|
||||
| synthetic_layered_512 | structured body exact 0.20/body far 0.50 | 1778 us | 5.075 | 61.380 | 0.045 | 127.906 | Structure metrics barely move. |
|
||||
| synthetic_clustered_1024 | exact | 3057 us | 0.000 | 910.741 | 2.258 | 539.089 | Baseline. |
|
||||
| synthetic_clustered_1024 | spatial grid | 1535 us | 0.491 | 910.741 | 2.258 | 539.090 | Fast, structurally similar after 32 ticks. |
|
||||
| snap_email_eu_core | exact | 2803 us | 0.000 | 1727.105 | 49.021 | 498.931 | Baseline. |
|
||||
| snap_email_eu_core | spatial grid | 709 us | 0.489 | 1727.129 | 48.816 | 498.956 | Big speed win; structure metrics nearly unchanged. |
|
||||
| snap_p2p_gnutella08 sampled | exact | 27859 us | 0.000 | 586.979 | 0.250 | 360.660 | Baseline. |
|
||||
| snap_p2p_gnutella08 sampled | Barnes-Hut theta 1.2 | 10555 us | 0.753 | 586.986 | 0.250 | 360.507 | Faster; metrics still too insensitive. |
|
||||
| snap_p2p_gnutella08 sampled | spatial grid | 2027 us | 0.784 | 586.958 | 0.250 | 360.940 | Very fast; structure metrics still barely move. |
|
||||
|
||||
### Hypothesis 10: one short quality snapshot exposes organ benefits
|
||||
|
||||
Result: rejected.
|
||||
|
||||
After 32 ticks, fold compactness, group separation, rank error, and edge stats
|
||||
barely differ across candidates. That can mean two things:
|
||||
|
||||
- the candidate forces are too weak or too entangled with the rest of the solver;
|
||||
- the current metrics need longer trajectory comparisons and normalized deltas.
|
||||
|
||||
Probably both. Annoying, therefore useful.
|
||||
|
||||
Doctrine:
|
||||
|
||||
- Keep `evaluate_layout_quality`, but do not overinterpret one short snapshot.
|
||||
- Add before/after delta metrics.
|
||||
- Add N-tick trajectory comparisons against exact.
|
||||
- Normalize rank error by `rank_gap`, edge variance by mean edge length, and
|
||||
group separation by group radii.
|
||||
- Run longer settle windows for structural candidates, likely 100-500 ticks.
|
||||
|
||||
### Hypothesis 11: force error and structure quality are separate axes
|
||||
|
||||
Result: confirmed.
|
||||
|
||||
Spatial grid can have poor far-field force error while preserving short-window
|
||||
structure metrics. Body forces can worsen raw force error while intending to
|
||||
improve semantic shape. These are different scoreboards.
|
||||
|
||||
Doctrine:
|
||||
|
||||
- Force approximation sweeps decide physics fidelity and speed.
|
||||
- Structure sweeps decide visual readability.
|
||||
- Do not collapse them into one magic score unless the score is explicitly
|
||||
weighted for a product goal.
|
||||
|
||||
### Next Deep Pass
|
||||
|
||||
Add `LayoutTrajectoryReport`:
|
||||
|
||||
- run candidate and exact from the same warm start for N ticks.
|
||||
- compare final positions by node displacement.
|
||||
- compare quality metric deltas.
|
||||
- report speed per tick and total settle cost.
|
||||
- support fixture-specific goals: hierarchy, community, fold, broad cloud.
|
||||
|
||||
That is the next fair trial for organs. One snapshot is just a mugshot.
|
||||
|
||||
## Second Sweep: More Knobs
|
||||
|
||||
Date: 2026-05-07
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ use std::path::Path;
|
|||
|
||||
use epiphany_graph_rs::{
|
||||
Graph, Layout3dConfig, Layout3dSolver, RepulsionAccuracyCandidate, RepulsionMode,
|
||||
solver_structural_accuracy_sweep,
|
||||
evaluate_layout_quality, solver_structural_accuracy_sweep,
|
||||
};
|
||||
|
||||
fn main() {
|
||||
|
|
@ -18,16 +18,18 @@ fn main() {
|
|||
];
|
||||
|
||||
println!(
|
||||
"dataset,nodes,edges,communities,cores,mode,theta,body_mode,body_scale,body_far,elapsed_us,mean_abs,max_abs,mean_rel,rms_rel,max_rel"
|
||||
"dataset,nodes,edges,communities,cores,mode,theta,body_mode,body_scale,body_far,elapsed_us,mean_abs,max_abs,mean_rel,rms_rel,max_rel,fold_compactness,group_separation,rank_error,edge_mean,edge_var,bridge_mean,bridge_var"
|
||||
);
|
||||
|
||||
for fixture in fixtures() {
|
||||
let mut solver = Layout3dSolver::new(fixture.graph, Layout3dConfig::default());
|
||||
let base_graph = fixture.graph.clone();
|
||||
let mut solver = Layout3dSolver::new(base_graph.clone(), Layout3dConfig::default());
|
||||
solver.tick(8);
|
||||
let reports = solver_structural_accuracy_sweep(&solver, &candidates);
|
||||
for report in reports {
|
||||
let quality = candidate_quality(&base_graph, &report.candidate);
|
||||
println!(
|
||||
"{},{},{},{},{},{},{:.2},{},{:.2},{:.2},{},{:.6},{:.6},{:.6},{:.6},{:.6}",
|
||||
"{},{},{},{},{},{},{:.2},{},{:.2},{:.2},{},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6},{:.6}",
|
||||
fixture.name,
|
||||
solver.positions().len(),
|
||||
fixture.edge_count,
|
||||
|
|
@ -43,7 +45,14 @@ fn main() {
|
|||
report.max_absolute_error,
|
||||
report.mean_relative_error,
|
||||
report.rms_relative_error,
|
||||
report.max_relative_error
|
||||
report.max_relative_error,
|
||||
quality.mean_fold_compactness,
|
||||
quality.mean_group_separation,
|
||||
quality.mean_rank_error,
|
||||
quality.edge_length_mean,
|
||||
quality.edge_length_variance,
|
||||
quality.bridge_length_mean,
|
||||
quality.bridge_length_variance
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -55,6 +64,31 @@ struct Fixture {
|
|||
edge_count: usize,
|
||||
}
|
||||
|
||||
fn candidate_quality(
|
||||
graph: &Graph,
|
||||
candidate: &RepulsionAccuracyCandidate,
|
||||
) -> epiphany_graph_rs::LayoutQualityReport {
|
||||
let config = Layout3dConfig {
|
||||
repulsion_mode: candidate.repulsion_mode,
|
||||
barnes_hut_theta: candidate.barnes_hut_theta,
|
||||
grid_cell_size: candidate.grid_cell_size,
|
||||
grid_radius: candidate.grid_radius,
|
||||
barnes_hut_near_radius: candidate.barnes_hut_near_radius,
|
||||
near_repulsion_scale: candidate.near_repulsion_scale,
|
||||
far_repulsion_scale: candidate.far_repulsion_scale,
|
||||
body_repulsion_mode: candidate.body_repulsion_mode,
|
||||
body_repulsion_scale: candidate.body_repulsion_scale,
|
||||
body_exact_limit: candidate.body_exact_limit,
|
||||
body_far_repulsion_scale: candidate.body_far_repulsion_scale,
|
||||
..Layout3dConfig::default()
|
||||
};
|
||||
|
||||
let mut solver = Layout3dSolver::new(graph.clone(), config);
|
||||
solver.tick(32);
|
||||
let layout = solver.snapshot();
|
||||
evaluate_layout_quality(graph, &layout)
|
||||
}
|
||||
|
||||
fn fixtures() -> Vec<Fixture> {
|
||||
let mut fixtures = vec![
|
||||
Fixture {
|
||||
|
|
|
|||
159
src/lib.rs
159
src/lib.rs
|
|
@ -334,6 +334,21 @@ pub struct ForceAccuracyReport {
|
|||
pub max_relative_error: f32,
|
||||
}
|
||||
|
||||
/// Structure-oriented quality metrics for a concrete layout snapshot.
|
||||
#[derive(Clone, Copy, Debug, Default, PartialEq)]
|
||||
pub struct LayoutQualityReport {
|
||||
pub node_count: usize,
|
||||
pub edge_count: usize,
|
||||
pub fold_group_count: usize,
|
||||
pub mean_fold_compactness: f32,
|
||||
pub mean_group_separation: f32,
|
||||
pub mean_rank_error: f32,
|
||||
pub edge_length_mean: f32,
|
||||
pub edge_length_variance: f32,
|
||||
pub bridge_length_mean: f32,
|
||||
pub bridge_length_variance: f32,
|
||||
}
|
||||
|
||||
/// Output for one laid-out node.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct NodeLayout {
|
||||
|
|
@ -663,6 +678,37 @@ pub fn layout_3d(graph: &Graph, config: &Layout3dConfig) -> Layout3d {
|
|||
solver.snapshot()
|
||||
}
|
||||
|
||||
/// Evaluate structure-level readability metrics for a layout.
|
||||
pub fn evaluate_layout_quality(graph: &Graph, layout: &Layout3d) -> LayoutQualityReport {
|
||||
let positions = layout
|
||||
.nodes
|
||||
.iter()
|
||||
.map(|node| (node.x, node.y, node.z))
|
||||
.collect::<Vec<_>>();
|
||||
|
||||
let (edge_length_mean, edge_length_variance) = edge_length_stats(graph.edges(), &positions);
|
||||
let bridge_edges = layout
|
||||
.analysis
|
||||
.bridges
|
||||
.iter()
|
||||
.filter_map(|edge| graph.edges().get(edge.0).copied())
|
||||
.collect::<Vec<_>>();
|
||||
let (bridge_length_mean, bridge_length_variance) = edge_length_stats(&bridge_edges, &positions);
|
||||
|
||||
LayoutQualityReport {
|
||||
node_count: graph.node_count(),
|
||||
edge_count: graph.edge_count(),
|
||||
fold_group_count: layout.fold_groups.len(),
|
||||
mean_fold_compactness: mean_fold_compactness(&layout.fold_groups),
|
||||
mean_group_separation: mean_group_separation(&layout.fold_groups),
|
||||
mean_rank_error: mean_rank_error(&layout.nodes, &layout.constraints),
|
||||
edge_length_mean,
|
||||
edge_length_variance,
|
||||
bridge_length_mean,
|
||||
bridge_length_variance,
|
||||
}
|
||||
}
|
||||
|
||||
/// Compare repulsion approximations against exact all-pairs repulsion.
|
||||
///
|
||||
/// The input positions use Bevy-friendly `[x, y, z]` coordinates. The returned
|
||||
|
|
@ -2148,6 +2194,99 @@ fn magnitude_3d(vector: (f32, f32, f32)) -> f32 {
|
|||
(vector.0 * vector.0 + vector.1 * vector.1 + vector.2 * vector.2).sqrt()
|
||||
}
|
||||
|
||||
fn edge_length_stats(edges: &[(NodeId, NodeId)], positions: &[(f32, f32, f32)]) -> (f32, f32) {
|
||||
if edges.is_empty() {
|
||||
return (0.0, 0.0);
|
||||
}
|
||||
|
||||
let lengths = edges
|
||||
.iter()
|
||||
.map(|&(source, target)| distance_3d(positions[source.0], positions[target.0]))
|
||||
.collect::<Vec<_>>();
|
||||
mean_and_variance(&lengths)
|
||||
}
|
||||
|
||||
fn mean_fold_compactness(fold_groups: &[FoldGroup]) -> f32 {
|
||||
let groups = fold_groups
|
||||
.iter()
|
||||
.filter(|group| group.nodes.len() > 1)
|
||||
.collect::<Vec<_>>();
|
||||
if groups.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
groups
|
||||
.iter()
|
||||
.map(|group| group.radius / (group.nodes.len() as f32).sqrt().max(1.0))
|
||||
.sum::<f32>()
|
||||
/ groups.len() as f32
|
||||
}
|
||||
|
||||
fn mean_group_separation(fold_groups: &[FoldGroup]) -> f32 {
|
||||
if fold_groups.len() < 2 {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
let mut sum = 0.0f32;
|
||||
let mut count = 0usize;
|
||||
for left in 0..fold_groups.len() {
|
||||
for right in (left + 1)..fold_groups.len() {
|
||||
let left_center = fold_groups[left].center;
|
||||
let right_center = fold_groups[right].center;
|
||||
let distance = distance_arrays_3d(left_center, right_center);
|
||||
let radii = fold_groups[left].radius + fold_groups[right].radius;
|
||||
sum += distance / radii.max(1.0);
|
||||
count += 1;
|
||||
}
|
||||
}
|
||||
|
||||
sum / count.max(1) as f32
|
||||
}
|
||||
|
||||
fn mean_rank_error(nodes: &[NodeLayout3d], constraints: &ConstraintSet) -> f32 {
|
||||
if constraints.ranks.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
|
||||
constraints
|
||||
.ranks
|
||||
.iter()
|
||||
.map(|constraint| {
|
||||
let y = nodes
|
||||
.get(constraint.node.0)
|
||||
.map(|node| node.y)
|
||||
.unwrap_or_default();
|
||||
(y - constraint.target_y).abs()
|
||||
})
|
||||
.sum::<f32>()
|
||||
/ constraints.ranks.len() as f32
|
||||
}
|
||||
|
||||
fn mean_and_variance(values: &[f32]) -> (f32, f32) {
|
||||
if values.is_empty() {
|
||||
return (0.0, 0.0);
|
||||
}
|
||||
|
||||
let mean = values.iter().sum::<f32>() / values.len() as f32;
|
||||
let variance = values
|
||||
.iter()
|
||||
.map(|value| {
|
||||
let delta = value - mean;
|
||||
delta * delta
|
||||
})
|
||||
.sum::<f32>()
|
||||
/ values.len() as f32;
|
||||
(mean, variance)
|
||||
}
|
||||
|
||||
fn distance_3d(a: (f32, f32, f32), b: (f32, f32, f32)) -> f32 {
|
||||
magnitude_3d((a.0 - b.0, a.1 - b.1, a.2 - b.2))
|
||||
}
|
||||
|
||||
fn distance_arrays_3d(a: [f32; 3], b: [f32; 3]) -> f32 {
|
||||
magnitude_3d((a[0] - b[0], a[1] - b[1], a[2] - b[2]))
|
||||
}
|
||||
|
||||
fn apply_repulsion_3d_barnes_hut(
|
||||
positions: &[(f32, f32, f32)],
|
||||
forces: &mut [(f32, f32, f32)],
|
||||
|
|
@ -3002,4 +3141,24 @@ mod tests {
|
|||
.all(|report| report.mean_relative_error.is_finite())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn layout_quality_report_scores_structure() {
|
||||
let mut graph = Graph::new();
|
||||
let a = graph.add_node(1.0);
|
||||
let b = graph.add_node(1.0);
|
||||
let c = graph.add_node(1.0);
|
||||
graph.add_edge(a, b);
|
||||
graph.add_edge(b, c);
|
||||
graph.add_edge(c, a);
|
||||
|
||||
let layout = layout_3d(&graph, &Layout3dConfig::default());
|
||||
let quality = evaluate_layout_quality(&graph, &layout);
|
||||
|
||||
assert_eq!(quality.node_count, 3);
|
||||
assert_eq!(quality.edge_count, 3);
|
||||
assert!(quality.fold_group_count > 0);
|
||||
assert!(quality.edge_length_mean.is_finite());
|
||||
assert!(quality.mean_rank_error.is_finite());
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Reference in a new issue