Repository navigation
Commit a2b8093
authored
feat: simplify contradicting and redundant predicates on a column (#25207)
## Which issue does this PR close?
- Closes #5830.
## Rationale for this change
`simplify_predicates` reduces the `>`/`>=` and the `<`/`<=` comparisons
on a column to their most restrictive bound, but never compares the two
groups with each other, and only looks for contradictions between
equalities. A filter no row can satisfy therefore still scans the table.
| `WHERE` clause | Before | After |
| --- | --- | --- |
| `a > 3 AND a < 1` | `a > 3 AND a < 1` | `EmptyRelation` |
| `a > 1 AND a < 1` | `a > 1 AND a < 1` | `EmptyRelation` |
| `a >= 1 AND a < 1` | `a >= 1 AND a < 1` | `EmptyRelation` |
| `a = 7 AND a < 2` | `a = 7 AND a < 2` | `EmptyRelation` |
| `a = 7 AND a != 7` | `a = 7 AND a != 7` | `EmptyRelation` |
| `a = 7 AND a > 5` | `a = 7 AND a > 5` | `a = 7` |
| `a > 10 AND a != 5` | `a > 10 AND a != 5` | `a > 10` |
`a >= 1 AND a <= 1` still simplifies to `a = 1`: bounds meeting at a
value both admit stay satisfiable.
## What changes are included in this PR?
- `fix`: skip comparisons against a NULL literal when grouping.
`ScalarValue::try_cmp` follows sort order, where NULL sits below every
other value, so `a > NULL AND a > 5` was reduced to `a > 5` although `a
> NULL` is never true. Reachable only through the public
`simplify_predicates`, as `SimplifyExpressions` folds these first.
- `feat`: compare the groups with each other. Bounds leaving no value,
and an equality that contradicts another predicate, reduce the
conjunction to `false`; an equality drops what it subsumes; `!=` joins
the analysis and is dropped once a bound excludes its value.
- `test`: unit tests and `sqllogictest` cases.
Reducing to `false` is valid here because a `Filter` keeps a row only
when its predicate is true, making NULL and `false` interchangeable.
## What is the testing strategy for this PR?
Unit tests in `simplify_predicates.rs` cover each comparison operator
against an equality, every combination of strict and inclusive bounds,
`!=` dropped and kept, and comparisons against NULL.
`simplify_predicates.slt` checks the plans and drops two `# TODO`
markers this PR implements.
## Are there any user-facing changes?
Filters no row can satisfy no longer scan their input. Result sets and
public APIs are unchanged.1 parent 5636128 commit a2b8093
5 files changed
Lines changed: 444 additions & 53 deletions
File tree
- datafusion
- optimizer/src/simplify_expressions
- sqllogictest/test_files
- tpch/plans
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2909 | 2909 | | |
2910 | 2910 | | |
2911 | 2911 | | |
| 2912 | + | |
| 2913 | + | |
| 2914 | + | |
| 2915 | + | |
| 2916 | + | |
| 2917 | + | |
| 2918 | + | |
2912 | 2919 | | |
2913 | 2920 | | |
2914 | 2921 | | |
| |||
0 commit comments