Skip to content

[DNM] Adding Upstream Spark#11893

Draft
zhouyuan wants to merge 11 commits intoapache:mainfrom
zhouyuan:wip_upstreeam_spark
Draft

[DNM] Adding Upstream Spark#11893
zhouyuan wants to merge 11 commits intoapache:mainfrom
zhouyuan:wip_upstreeam_spark

Conversation

@zhouyuan
Copy link
Copy Markdown
Member

@zhouyuan zhouyuan commented Apr 8, 2026

What changes are proposed in this pull request?

This patch adds a spark-main profile which builds Gluten against Spark nightly builds
TPCH/DS tests added

With this patch Gluten should be able to catch-up with Spark release, e.g., if Spark cut the branch for release Gluten could also move the spark-main shim layer to the target release shim easily

How was this patch tested?

pass GHA

Was this patch authored or co-authored using generative AI tooling?

zhouyuan added 8 commits April 6, 2026 17:15
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions github-actions bot added CORE works for Gluten Core VELOX INFRA TOOLS labels Apr 8, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

Run Gluten Clickhouse CI on x86

zhouyuan added 2 commits April 8, 2026 17:12
Signed-off-by: Yuan <yuanzhou@apache.org>
Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

Run Gluten Clickhouse CI on x86

@zhouyuan zhouyuan requested review from baibaichen and philo-he April 8, 2026 19:22
@philo-he
Copy link
Copy Markdown
Member

philo-he commented Apr 8, 2026

@zhouyuan, thanks for proposing this patch.
Perhaps we should move the Spark main tests to a nightly CI job. Since the Spark snapshot is under active development, keeping these tests in our regular CI may frequently cause stability issues and build failures that possibly blocking PR merging. Thanks.

@baibaichen
Copy link
Copy Markdown
Contributor

@zhouyuan would you please follow #11352?

Signed-off-by: Yuan <yuanzhou@apache.org>
@github-actions
Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CORE works for Gluten Core INFRA TOOLS VELOX

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants