Describe the bug
When a WHERE condition limits a GROUP BY column to one value, we have the pieces to know that the column is fixed. However, aggs currently treats that column as if a change in its value could show that earlier groups are complete.
Since the column is fixed because of the filter, it cannot provide such a boundary.
eg:
SELECT key, market, COUNT(*)
FROM t
WHERE market = 'US'
GROUP BY key, market;
Today we currently report the agg as partially ordered using market. For large inputs, this blocks releasing finished groups early
To Reproduce
EXPLAIN
WITH t(key, market) AS (
VALUES
(1, 'US'),
(2, 'US'),
(1, 'US')
)
SELECT key, market, COUNT(*)
FROM t
WHERE market = 'US'
GROUP BY key, market;
As of today, this produces
AggregateExec: mode=Partial,
gby=[key@0 as key, market@1 as market],
ordering_mode=PartiallySorted([1])
Every row has market = 'US', so this column never changes and cannot show that a group has finished.
Expected behavior
Ideally should ignore fixed grouping columns when deciding whether earlier groups are complete. For the example above, agg should not use market as a group completion boundary. If another grouping column is genuinely ordered, we should still use that column
Additional context
#24697 is somewhat related to this fix. It separates input ordering from the question of whether a group is complete. A fixed column can be considered ordered, but it should not be used as that a group is complete.
Describe the bug
When a
WHEREcondition limits aGROUP BYcolumn to one value, we have the pieces to know that the column is fixed. However, aggs currently treats that column as if a change in its value could show that earlier groups are complete.Since the column is fixed because of the filter, it cannot provide such a boundary.
eg:
Today we currently report the agg as partially ordered using
market. For large inputs, this blocks releasing finished groups earlyTo Reproduce
As of today, this produces
Every row has
market = 'US', so this column never changes and cannot show that a group has finished.Expected behavior
Ideally should ignore fixed grouping columns when deciding whether earlier groups are complete. For the example above, agg should not use
marketas a group completion boundary. If another grouping column is genuinely ordered, we should still use that columnAdditional context
#24697 is somewhat related to this fix. It separates input ordering from the question of whether a group is complete. A fixed column can be considered ordered, but it should not be used as that a group is complete.