Skip to content

Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT #23440

Description

@Fly-a-Kite

Describe the bug

DataFusion 54.0.0 returns an empty result for a grouped aggregate query where the aggregate value is NULL, followed by ORDER BY ... NULLS FIRST LIMIT.

The grouped aggregate should still produce one output row. The final ORDER BY / LIMIT should not remove it.

This looks related to #22190, but I can still reproduce it on the latest datafusion==54.0.0 with a smaller reproducer.

To Reproduce

  import datafusion
  import pyarrow as pa
  from datafusion import SessionContext

  print("datafusion", datafusion.__version__)

  batch = pa.record_batch(
      [
          pa.array(["gamma"], type=pa.string()),
          pa.array([None], type=pa.float64()),
      ],
      names=["s", "y"],
  )

  ctx = SessionContext()
  ctx.register_record_batches("t0", [[batch]])

  query = """
  SELECT max_y
  FROM (
    SELECT s, MAX(y) AS max_y
    FROM t0
    GROUP BY s
  )
  ORDER BY max_y DESC NULLS FIRST
  LIMIT 3
  """

  result = ctx.sql(query).to_pandas()
  print(result)
  print("row count:", len(result))

### Expected behavior

  ### Expected behavior

  The query should return one row:

     max_y
  0   NULL

  Reason: the input contains one group, s = 'gamma'. MAX(y) over a group where all y values are NULL should produce one grouped output row with max_y = NULL.

  ### Actual behavior

  DataFusion 54.0.0 returns zero rows:

  Empty DataFrame
  Columns: [max_y]
  Index: []
  row count: 0

### Additional context

 ### Environment

  - datafusion==54.0.0
  - pyarrow==24.0.0
  - Python 3.12.3
  - Linux x86_64

Activity

  1. added theissue type on Jul 10, 2026
  2. SubhamSinghal commented on Jul 10, 2026

    @SubhamSinghal
    Contributor

    Take

  3. u70b3 commented on Jul 18, 2026

    @u70b3
    Contributor

    take — fix in #23684 (this also covers the same root cause as #22190)

  4. added 5 commits that reference this issue on Jul 20, 2026
    a7f380c
    3ee7c2f
    e0370ea
    c5bb62e
    99df683
  5. added a commit that references this issue on Jul 31, 2026
    f8b9ed8
  6. added a commit that references this issue on Aug 12, 2026
    1305241
  7. added a commit that references this issue on Sep 27, 2026
    82c6c73
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions