Sparse spatially-indexed layer ordering #57

Merged

toph merged 7 commits from worktree-spatial-layers into main

2026-07-04 16:57:19 +00:00

toph commented

2026-07-04 16:22:25 +00:00

Owner

Replaces Fold_state's dense rel array array layer-ordering matrix with a sparse structure built by an exact sweep-and-prune broad phase, so distributed fold patterns (tessellations) evaluate sub-quadratically per fold. Output is byte-identical.

A scaling probe this session measured the dense build at O(faces²) per fold — ~0.37s at 256 faces, 1.5s at 512, 6s at 1024, 24s at 2048 — capping practical folded programs at ~1–2k faces. The matrix stored n² entries and build_order ran Geom.convex_overlap on every i<j pair, even though a tessellation's faces overlap only locally.

New lib/layer_order.ml stores only overlapping pairs (absent = not overlapping; present-with-Apart = the tortilla-tortilla case) and builds them with a sweep-and-prune over exact Num bounding boxes: sort by bbox xmin, sweep an active set, and run the exact Geom.convex_overlap only on bbox-overlapping candidates. The spatial index only culls candidates — no float in the decision, and it can never drop a real overlap — so the relation, and every .fold output, is unchanged. Fold_state.order becomes a Layer_order.t; validity_error, subdivide, fold_with_records, flip, topmost_preimage and fold_emit's faceOrders migrate to the sparse API.

For distributed patterns this drops the per-fold order build from O(faces²) to O(faces log faces), lifting the practical ceiling from ~2k toward tens of thousands of faces. A fully-overlapping stack (repeated in-place folding) genuinely has O(faces²) relations and is unchanged — inherent, and rare in real origami.

Byte-identical is not just the goldens passing (23/23): both build paths call rel_of on the same canonical i<j pairs, and fold_emit keeps its nested loop reading via Layer_order.get, so every relation and the faceOrders emission order are reproduced by construction.

New: Layer_order unit tests (sweep-and-prune vs an O(n²) reference; the absent-vs-Apart invariant) and a Slow scaling guard that separates the new sparse build (~4.3× on doubling N) from the old dense scan (~8×).

Also drops the committed scratch/ directory (first commit): the #56 squash force-added it, and its scratch/dune referenced gitignored files that never landed, breaking dune build on a fresh checkout. scratch/ is gitignored throwaway; the spike findings live in notes/.

Full suite green, goldens byte-identical.

https://claude.ai/code/session_01TinroyhVWz1RBZtXMZoLdd

Replaces `Fold_state`'s dense `rel array array` layer-ordering matrix with a sparse structure built by an exact sweep-and-prune broad phase, so distributed fold patterns (tessellations) evaluate sub-quadratically per fold. Output is byte-identical. A scaling probe this session measured the dense build at O(faces²) per fold — ~0.37s at 256 faces, 1.5s at 512, 6s at 1024, 24s at 2048 — capping practical folded programs at ~1–2k faces. The matrix stored `n²` entries and `build_order` ran `Geom.convex_overlap` on every `i<j` pair, even though a tessellation's faces overlap only locally. New `lib/layer_order.ml` stores only overlapping pairs (absent = not overlapping; present-with-`Apart` = the tortilla-tortilla case) and builds them with a sweep-and-prune over exact `Num` bounding boxes: sort by bbox `xmin`, sweep an active set, and run the exact `Geom.convex_overlap` only on bbox-overlapping candidates. The spatial index only culls candidates — no float in the decision, and it can never drop a real overlap — so the relation, and every `.fold` output, is unchanged. `Fold_state.order` becomes a `Layer_order.t`; `validity_error`, `subdivide`, `fold_with_records`, `flip`, `topmost_preimage` and `fold_emit`'s faceOrders migrate to the sparse API. For distributed patterns this drops the per-fold order build from O(faces²) to O(faces log faces), lifting the practical ceiling from ~2k toward tens of thousands of faces. A fully-overlapping stack (repeated in-place folding) genuinely has O(faces²) relations and is unchanged — inherent, and rare in real origami. Byte-identical is not just the goldens passing (23/23): both build paths call `rel_of` on the same canonical `i<j` pairs, and `fold_emit` keeps its nested loop reading via `Layer_order.get`, so every relation and the faceOrders emission order are reproduced by construction. New: `Layer_order` unit tests (sweep-and-prune vs an O(n²) reference; the absent-vs-`Apart` invariant) and a `Slow` scaling guard that separates the new sparse build (~4.3× on doubling N) from the old dense scan (~8×). Also drops the committed `scratch/` directory (first commit): the #56 squash force-added it, and its `scratch/dune` referenced gitignored files that never landed, breaking `dune build` on a fresh checkout. `scratch/` is gitignored throwaway; the spike findings live in `notes/`. Full suite green, goldens byte-identical. https://claude.ai/code/session_01TinroyhVWz1RBZtXMZoLdd

toph added 7 commits

2026-07-04 16:22:25 +00:00

chore: drop committed scratch/ (restores .gitignore consistency, fixes dune build) 4a7772de07

The #56 squash force-added scratch/ including a scratch/dune referencing
probe/fieldbench whose .ml were never committed (gitignored), breaking
`dune build` on a fresh checkout. scratch/ is gitignored throwaway; the
spike findings live in notes/2026-07-03-calcium-pe-spike.md.

docs(spec): sparse spatially-indexed layer ordering 344afb8875

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TinroyhVWz1RBZtXMZoLdd

docs(plan): sparse spatial layer ordering implementation plan 84091e3338

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TinroyhVWz1RBZtXMZoLdd

feat(layer_order): sparse sweep-and-prune layer order module 0fcab742e6

refactor(fold_state): sparse Layer_order backs the layer order (byte-identical) bb05f430a1

test(fold_state): sub-quadratic layer-ordering scaling guard 9e024091ff

Regression test timing Fold_state.subdivide's incremental build over a
distributed (non-overlapping) vertical-strip pattern for N=200 vs N=400.

Bound widened from the brief's 3.5x to 6x after profiling showed this
loop's own cost (each subdivide call reprocesses the whole face array,
plus Layer_order.build's O(m log m) sort) sums to ~O(n^2 log n) across n
calls, not strictly sub-quadratic — measured ~4.3-4.7x for doubling N.
The old dense build_order (no spatial culling, O(m^2) per call) sums to
O(n^3), an 8x cost for doubling N; 6x cleanly separates the new sparse
algorithm from a regression back to that dense scan.

polish(layer_order): deterministic ascending iter/above_neighbors + doc fixes 1cbb85c610

Post-final-review nits (all Minor): fold_state header no longer describes
order as a dense .(i).(j) array; Layer_order.iter and above_neighbors emit
ascending so validity cycle/tortilla messages name a deterministic pair
(matches the old ascending double-loop, satisfies the 'messages identical'
constraint literally, stable across Hashtbl impls); flip precondition
documented. faceOrders unaffected (uses get, not iter) — golden byte-identical.

toph merged commit b26f62b072 into main

2026-07-04 16:57:19 +00:00

toph deleted branch worktree-spatial-layers

2026-07-04 16:57:20 +00:00

toph referenced this pull request from a commit

2026-07-04 16:57:21 +00:00

Sparse spatially-indexed layer ordering (#57)