Keep adjacent numeric table columns separate

[yzxcj797]

Summary

• Keep two predominantly numeric column clusters separate instead of collapsing the sparse numeric column into the dense one.
• Preserve the existing header repair path: a sparse predominantly non-numeric cluster can still merge with adjacent numeric data.
• Split a two-value financial item only when its physical span crosses a ruled column boundary. The existing heuristic splitter remains limited to wider items with three or more values, so this does not broadly relax heuristic-table splitting.
• Handle empty/degenerate ruled grids safely.

Fixes #419.

Validation on the attached customs PDFs

Before:

June: |6 Coal|Ton|1,661,481|236,480,212 10,024,724||1,302,660,618|
July: |6 Coal|Ton|1,709,851|246,515,548 11,734,697||1,555,338,055|

After:

June: |6 Coal|Ton|1,661,481|236,480,212|10,024,724|1,302,660,618|
July: |6 Coal|Ton|1,709,851|246,515,548|11,734,697|1,555,338,055|

Tests

• cargo test
  • 958 library tests pass
  • 162 integration tests pass
  • 3 CLI tests pass
  • 2 documentation tests pass
• cargo clippy -- -D warnings — passes
• cargo fmt --all -- --check — passes
• git diff --check — passes

---

Summary by cubic

Keep adjacent numeric table columns separate and split two-value financial items only when they cross a ruled column boundary. Preserves header-to-data merging and inferred-grid behavior. Fixes #419.

• Adjacent-column merge: merge a sparse cluster into an adjacent dense cluster only when the sparse cluster is predominantly non-numeric; two predominantly numeric clusters now always remain separate. Classification uses the actual members of each cluster to avoid jitter reclassification.
• Ruled-boundary split: allow splitting a two-value numeric item only in ruled grids and only for Text items with positive font size; keep the heuristic splitter unchanged (requires 3+ values). Preserve slot alignment (heuristic: centered; ruled: slot start), and fall back to the original item if any split fragment cannot be placed.
• Grid assignment: introduce assign_items_to_ruled_grid for ruled tables; inferred grids continue to use assign_items_to_grid so two-value items stay intact. Deduplicate item indices and ignore zero-width items.
• Tests: add coverage to keep numeric columns separate, still merge sparse headers with numeric data, and split only in ruled grids.

^{Written for commit 4d90870. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Shadow auto-approve: would not auto-approve because issues were found.
Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[yzxcj797]

Addressed in 4d90870:

• Cluster numeric fractions now consume the exact item X values belonging to each cluster, so adjacent-cluster jitter cannot reclassify a cluster through an overlapping radius window.
• Added the reported mixed-jitter shape as a regression while retaining the header-vs-data merge case.
• Two-value splitting is now limited to ruled grids via a dedicated assignment entry point; inferred row-stripe and merged-cluster grids keep the original consolidated item.
• The splitter accepts only positive-font-size text items, rejecting form fields.
• The heuristic splitter keeps its original centered sub-item placement; only the ruled-boundary splitter uses left-aligned slots.
• If any split fragment cannot be assigned to a column, the original item is kept together instead of marking it consumed and losing the unplaced value.
• Added regression coverage for inferred grids, form fields, and splitter-specific alignment.

Validation: full cargo test, cargo clippy -- -D warnings, cargo fmt --all -- --check, and git diff --check pass. Both attached customs PDFs still split the Coal row correctly.

[cubic-dev-ai]

1 issue found across 4 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/tables/detect_rects.rs">

<violation number="1" location="src/tables/detect_rects.rs:1534">
P2: When `columns_from_text` is true, this call treats inferred text-cluster midpoints as ruled boundaries and splits qualifying two-value items across them. Use `assign_items_to_grid` for that branch and reserve `assign_items_to_ruled_grid` for rect-derived columns.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: When columns_from_text is true, this call treats inferred text-cluster midpoints as ruled boundaries and splits qualifying two-value items across them. Use assign_items_to_grid for that branch and reserve assign_items_to_ruled_grid for rect-derived columns.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/tables/detect_rects.rs, line 1534:

<comment>When `columns_from_text` is true, this call treats inferred text-cluster midpoints as ruled boundaries and splits qualifying two-value items across them. Use `assign_items_to_grid` for that branch and reserve `assign_items_to_ruled_grid` for rect-derived columns.</comment>

<file context>
@@ -1531,7 +1531,7 @@ fn try_build_grid(
 
     // Build table: assign text items to cells
-    let (mut cells, item_indices) = assign_items_to_grid(items, &col_edges, &row_edges, page);
+    let (mut cells, item_indices) = assign_items_to_ruled_grid(items, &col_edges, &row_edges, page);
 
     // Consolidate vertically-merged cells: rects spanning multiple grid rows
</file context>

Suggested change

	let (mut cells, item_indices) = assign_items_to_ruled_grid(items, &col_edges, &row_edges, page);
	let (mut cells, item_indices) = if columns_from_text {
	assign_items_to_grid(items, &col_edges, &row_edges, page)
	} else {
	assign_items_to_ruled_grid(items, &col_edges, &row_edges, page)
	};