Skip to content

Conversation

@wangkewen
Copy link

Description

Previous approach (#25552) supporting both Spark2.4 and Spark3.4 utilizes two separate modules which incurred some release inconsistence issues. This change merges this change into existing module without generating separate modules.

Motivation and Context

It tries to resolve one issue mentioned here #25833.

Impact

Test Plan

Build for Spark2

$ mvn clean install -U -DskipTests -Dmaven.javadoc.skip=true -T1C -pl '!presto-docs'

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for presto-root 0.297-SNAPSHOT:
[INFO]
[INFO] presto-root ........................................ SUCCESS [  3.827 s]
[INFO] presto-testng-services ............................. SUCCESS [  4.169 s]
[INFO] presto-common ...................................... SUCCESS [ 18.788 s]
[INFO] presto-spi ......................................... SUCCESS [  4.940 s]
[INFO] presto-plugin-toolkit .............................. SUCCESS [  2.117 s]
[INFO] presto-client ...................................... SUCCESS [  2.662 s]
[INFO] presto-parser ...................................... SUCCESS [  7.047 s]
[INFO] presto-geospatial-toolkit .......................... SUCCESS [  3.917 s]
[INFO] presto-analyzer .................................... SUCCESS [  1.472 s]
[INFO] presto-bytecode .................................... SUCCESS [ 18.769 s]
[INFO] presto-matching .................................... SUCCESS [  1.592 s]
[INFO] presto-memory-context .............................. SUCCESS [  1.453 s]
[INFO] presto-expressions ................................. SUCCESS [  1.572 s]
[INFO] presto-main-tests .................................. SUCCESS [  1.045 s]
[INFO] presto-function-namespace-managers-common .......... SUCCESS [  1.793 s]
[INFO] presto-tpch ........................................ SUCCESS [  2.404 s]
[INFO] presto-thrift-api .................................. SUCCESS [  2.460 s]
[INFO] presto-function-namespace-managers ................. SUCCESS [  3.494 s]
[INFO] presto-built-in-worker-function-tools .............. SUCCESS [  1.370 s]
[INFO] presto-sql-invoked-functions-plugin ................ SUCCESS [  1.422 s]
[INFO] presto-ui .......................................... SUCCESS [ 25.374 s]
[INFO] presto-main-base ................................... SUCCESS [01:10 min]
[INFO] presto-main ........................................ SUCCESS [  6.422 s]
[INFO] presto-blackhole ................................... SUCCESS [  1.867 s]
[INFO] presto-resource-group-managers ..................... SUCCESS [  3.117 s]
[INFO] presto-thrift-testing-udf-server ................... SUCCESS [  3.002 s]
[INFO] presto-tests ....................................... SUCCESS [01:16 min]
[INFO] presto-atop ........................................ SUCCESS [  7.364 s]
[INFO] presto-hdfs-core ................................... SUCCESS [  2.120 s]
[INFO] presto-cache ....................................... SUCCESS [  2.059 s]
[INFO] presto-jmx ......................................... SUCCESS [  7.569 s]
[INFO] presto-record-decoder .............................. SUCCESS [  2.000 s]
[INFO] presto-kafka ....................................... SUCCESS [  9.189 s]
[INFO] presto-redis ....................................... SUCCESS [  4.231 s]
[INFO] presto-accumulo .................................... SUCCESS [ 10.581 s]
[INFO] presto-cassandra ................................... SUCCESS [  8.877 s]
[INFO] presto-bigquery .................................... SUCCESS [  7.672 s]
[INFO] presto-memory ...................................... SUCCESS [  8.568 s]
[INFO] presto-orc ......................................... SUCCESS [  9.939 s]
[INFO] presto-benchmark ................................... SUCCESS [  2.075 s]
[INFO] presto-parquet ..................................... SUCCESS [  4.744 s]
[INFO] presto-rcfile ...................................... SUCCESS [  2.573 s]
[INFO] presto-hive-common ................................. SUCCESS [  1.399 s]
[INFO] presto-hive-metastore .............................. SUCCESS [  3.986 s]
[INFO] presto-tpcds ....................................... SUCCESS [  3.079 s]
[INFO] presto-testing-docker .............................. SUCCESS [  4.167 s]
[INFO] presto-hive ........................................ SUCCESS [ 15.939 s]
[INFO] presto-hive-hadoop2 ................................ SUCCESS [ 11.911 s]
[INFO] presto-iceberg ..................................... SUCCESS [ 19.697 s]
[INFO] presto-i18n-functions .............................. SUCCESS [  4.067 s]
[INFO] presto-teradata-functions .......................... SUCCESS [  4.203 s]
[INFO] presto-example-http ................................ SUCCESS [  2.705 s]
[INFO] presto-local-file .................................. SUCCESS [  2.493 s]
[INFO] presto-base-jdbc ................................... SUCCESS [  7.744 s]
[INFO] presto-mysql ....................................... SUCCESS [  3.081 s]
[INFO] presto-oracle ...................................... SUCCESS [  3.155 s]
[INFO] presto-postgresql .................................. SUCCESS [  3.056 s]
[INFO] presto-prometheus .................................. SUCCESS [  7.847 s]
[INFO] presto-redshift .................................... SUCCESS [  2.634 s]
[INFO] presto-sqlserver ................................... SUCCESS [  2.253 s]
[INFO] presto-mongodb ..................................... SUCCESS [  7.712 s]
[INFO] presto-ml .......................................... SUCCESS [  7.690 s]
[INFO] presto-jdbc ........................................ SUCCESS [ 49.878 s]
[INFO] presto-cli ......................................... SUCCESS [  4.313 s]
[INFO] presto-product-tests ............................... SUCCESS [ 20.437 s]
[INFO] presto-pinot-toolkit ............................... SUCCESS [  3.980 s]
[INFO] presto-pinot ....................................... SUCCESS [  6.878 s]
[INFO] presto-benchmark-driver ............................ SUCCESS [  5.311 s]
[INFO] presto-thrift-testing-server ....................... SUCCESS [01:24 min]
[INFO] presto-thrift-connector ............................ SUCCESS [  2.397 s]
[INFO] presto-password-authenticators ..................... SUCCESS [  2.838 s]
[INFO] presto-session-property-managers-common ............ SUCCESS [  1.467 s]
[INFO] presto-db-session-property-manager ................. SUCCESS [  2.386 s]
[INFO] presto-hudi ........................................ SUCCESS [ 12.602 s]
[INFO] presto-hive-function-namespace ..................... SUCCESS [  6.209 s]
[INFO] presto-hana ........................................ SUCCESS [  2.839 s]
[INFO] presto-druid ....................................... SUCCESS [ 13.100 s]
[INFO] presto-cluster-ttl-providers ....................... SUCCESS [  2.377 s]
[INFO] presto-file-session-property-manager ............... SUCCESS [  1.994 s]
[INFO] presto-singlestore ................................. SUCCESS [  2.719 s]
[INFO] presto-native-sql-invoked-functions-plugin ......... SUCCESS [  1.424 s]
[INFO] presto-node-ttl-fetchers ........................... SUCCESS [  2.349 s]
[INFO] presto-kudu ........................................ SUCCESS [  7.791 s]
[INFO] presto-elasticsearch ............................... SUCCESS [ 10.106 s]
[INFO] presto-spark-classloader-interface ................. SUCCESS [  4.407 s]
[INFO] presto-spark-common ................................ SUCCESS [  1.115 s]
[INFO] presto-spark-base .................................. SUCCESS [  4.888 s]
[INFO] presto-native-execution ............................ SUCCESS [  3.002 s]
[INFO] presto-native-sidecar-plugin ....................... SUCCESS [  3.406 s]
[INFO] presto-delta ....................................... SUCCESS [ 11.748 s]
[INFO] presto-clickhouse .................................. SUCCESS [  3.714 s]
[INFO] presto-server ...................................... SUCCESS [ 57.087 s]
[INFO] presto-verifier .................................... SUCCESS [01:47 min]
[INFO] presto-testing-server-launcher ..................... SUCCESS [ 10.506 s]
[INFO] presto-benchto-benchmarks .......................... SUCCESS [  2.591 s]
[INFO] presto-proxy ....................................... SUCCESS [  4.961 s]
[INFO] presto-benchmark-runner ............................ SUCCESS [  1.952 s]
[INFO] presto-spark ....................................... SUCCESS [  0.804 s]
[INFO] presto-spark-package ............................... SUCCESS [ 50.614 s]
[INFO] presto-spark-launcher .............................. SUCCESS [ 27.396 s]
[INFO] presto-spark-testing ............................... SUCCESS [  1.672 s]
[INFO] presto-thrift-spec ................................. SUCCESS [  2.279 s]
[INFO] presto-google-sheets ............................... SUCCESS [  7.659 s]
[INFO] presto-grpc-api .................................... SUCCESS [  5.212 s]
[INFO] presto-grpc-testing-udf-server ..................... SUCCESS [  0.948 s]
[INFO] presto-lark-sheets ................................. SUCCESS [  7.814 s]
[INFO] presto-test-coverage ............................... SUCCESS [ 12.929 s]
[INFO] presto-plan-checker-router-plugin .................. SUCCESS [  2.472 s]
[INFO] presto-router ...................................... SUCCESS [  7.997 s]
[INFO] presto-native-tests ................................ SUCCESS [  1.101 s]
[INFO] presto-open-telemetry .............................. SUCCESS [  1.658 s]
[INFO] redis-hbo-provider ................................. SUCCESS [  4.117 s]
[INFO] presto-openapi ..................................... SUCCESS [  6.429 s]
[INFO] presto-base-arrow-flight ........................... SUCCESS [  7.348 s]
[INFO] presto-function-server ............................. SUCCESS [ 14.217 s]
[INFO] presto-router-example-plugin-scheduler ............. SUCCESS [  1.444 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:48 min (Wall Clock)
[INFO] Finished at: 2025-12-02T10:25:20-08:00
[INFO] ------------------------------------------------------------------------

Build for Spark3

$ mvn clean install -U -DskipTests -Dmaven.javadoc.skip=true -Dspark-version=3 -T1C -pl '!presto-docs'

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for presto-root 0.297-SNAPSHOT:
[INFO] 
[INFO] presto-root ........................................ SUCCESS [  3.768 s]
[INFO] presto-testng-services ............................. SUCCESS [  4.256 s]
[INFO] presto-common ...................................... SUCCESS [  8.421 s]
[INFO] presto-spi ......................................... SUCCESS [  5.076 s]
[INFO] presto-plugin-toolkit .............................. SUCCESS [  2.699 s]
[INFO] presto-client ...................................... SUCCESS [  3.954 s]
[INFO] presto-parser ...................................... SUCCESS [  8.674 s]
[INFO] presto-geospatial-toolkit .......................... SUCCESS [  5.343 s]
[INFO] presto-analyzer .................................... SUCCESS [  1.370 s]
[INFO] presto-bytecode .................................... SUCCESS [  3.698 s]
[INFO] presto-matching .................................... SUCCESS [  1.710 s]
[INFO] presto-memory-context .............................. SUCCESS [  1.586 s]
[INFO] presto-expressions ................................. SUCCESS [  2.342 s]
[INFO] presto-main-tests .................................. SUCCESS [  1.055 s]
[INFO] presto-function-namespace-managers-common .......... SUCCESS [  2.870 s]
[INFO] presto-tpch ........................................ SUCCESS [  3.662 s]
[INFO] presto-thrift-api .................................. SUCCESS [  3.677 s]
[INFO] presto-function-namespace-managers ................. SUCCESS [  3.662 s]
[INFO] presto-built-in-worker-function-tools .............. SUCCESS [  1.715 s]
[INFO] presto-sql-invoked-functions-plugin ................ SUCCESS [  1.983 s]
[INFO] presto-ui .......................................... SUCCESS [ 27.873 s]
[INFO] presto-main-base ................................... SUCCESS [01:09 min]
[INFO] presto-main ........................................ SUCCESS [  6.288 s]
[INFO] presto-blackhole ................................... SUCCESS [  2.002 s]
[INFO] presto-resource-group-managers ..................... SUCCESS [  3.430 s]
[INFO] presto-thrift-testing-udf-server ................... SUCCESS [  3.824 s]
[INFO] presto-tests ....................................... SUCCESS [01:17 min]
[INFO] presto-atop ........................................ SUCCESS [  5.229 s]
[INFO] presto-hdfs-core ................................... SUCCESS [  2.782 s]
[INFO] presto-cache ....................................... SUCCESS [  2.784 s]
[INFO] presto-jmx ......................................... SUCCESS [  4.470 s]
[INFO] presto-record-decoder .............................. SUCCESS [  2.091 s]
[INFO] presto-kafka ....................................... SUCCESS [  6.160 s]
[INFO] presto-redis ....................................... SUCCESS [  5.414 s]
[INFO] presto-accumulo .................................... SUCCESS [  7.831 s]
[INFO] presto-cassandra ................................... SUCCESS [  5.854 s]
[INFO] presto-bigquery .................................... SUCCESS [  7.461 s]
[INFO] presto-memory ...................................... SUCCESS [  5.873 s]
[INFO] presto-orc ......................................... SUCCESS [ 10.401 s]
[INFO] presto-benchmark ................................... SUCCESS [  2.356 s]
[INFO] presto-parquet ..................................... SUCCESS [  5.400 s]
[INFO] presto-rcfile ...................................... SUCCESS [  2.504 s]
[INFO] presto-hive-common ................................. SUCCESS [  1.444 s]
[INFO] presto-hive-metastore .............................. SUCCESS [  4.143 s]
[INFO] presto-tpcds ....................................... SUCCESS [  3.487 s]
[INFO] presto-testing-docker .............................. SUCCESS [  4.253 s]
[INFO] presto-hive ........................................ SUCCESS [ 16.904 s]
[INFO] presto-hive-hadoop2 ................................ SUCCESS [ 13.042 s]
[INFO] presto-iceberg ..................................... SUCCESS [ 21.081 s]
[INFO] presto-i18n-functions .............................. SUCCESS [  4.011 s]
[INFO] presto-teradata-functions .......................... SUCCESS [  3.521 s]
[INFO] presto-example-http ................................ SUCCESS [  2.768 s]
[INFO] presto-local-file .................................. SUCCESS [  2.636 s]
[INFO] presto-base-jdbc ................................... SUCCESS [  4.733 s]
[INFO] presto-mysql ....................................... SUCCESS [  3.577 s] 
[INFO] presto-oracle ...................................... SUCCESS [  3.630 s]
[INFO] presto-postgresql .................................. SUCCESS [  3.710 s]
[INFO] presto-prometheus .................................. SUCCESS [  5.970 s]
[INFO] presto-redshift .................................... SUCCESS [  2.985 s]
[INFO] presto-sqlserver ................................... SUCCESS [  3.068 s]
[INFO] presto-mongodb ..................................... SUCCESS [  4.916 s]
[INFO] presto-ml .......................................... SUCCESS [  5.522 s]
[INFO] presto-jdbc ........................................ SUCCESS [ 48.160 s]
[INFO] presto-cli ......................................... SUCCESS [  4.167 s]
[INFO] presto-product-tests ............................... SUCCESS [ 21.007 s]
[INFO] presto-pinot-toolkit ............................... SUCCESS [  3.932 s]
[INFO] presto-pinot ....................................... SUCCESS [  7.103 s]
[INFO] presto-benchmark-driver ............................ SUCCESS [  5.766 s]
[INFO] presto-thrift-testing-server ....................... SUCCESS [01:23 min]
[INFO] presto-thrift-connector ............................ SUCCESS [  2.420 s]
[INFO] presto-password-authenticators ..................... SUCCESS [  4.017 s]
[INFO] presto-session-property-managers-common ............ SUCCESS [  2.063 s]
[INFO] presto-db-session-property-manager ................. SUCCESS [  3.071 s]
[INFO] presto-hudi ........................................ SUCCESS [ 13.489 s]
[INFO] presto-hive-function-namespace ..................... SUCCESS [  6.897 s]
[INFO] presto-hana ........................................ SUCCESS [  3.453 s]
[INFO] presto-druid ....................................... SUCCESS [ 10.821 s]
[INFO] presto-cluster-ttl-providers ....................... SUCCESS [  2.938 s]
[INFO] presto-file-session-property-manager ............... SUCCESS [  3.038 s]
[INFO] presto-singlestore ................................. SUCCESS [  3.414 s]
[INFO] presto-native-sql-invoked-functions-plugin ......... SUCCESS [  1.986 s]
[INFO] presto-node-ttl-fetchers ........................... SUCCESS [  3.190 s]
[INFO] presto-kudu ........................................ SUCCESS [  8.572 s]
[INFO] presto-elasticsearch ............................... SUCCESS [  7.456 s]
[INFO] presto-spark-classloader-interface ................. SUCCESS [ 17.972 s]
[INFO] presto-spark-common ................................ SUCCESS [  1.266 s]
[INFO] presto-spark-base .................................. SUCCESS [ 32.584 s]
[INFO] presto-native-execution ............................ SUCCESS [  2.787 s]
[INFO] presto-native-sidecar-plugin ....................... SUCCESS [  3.715 s]
[INFO] presto-delta ....................................... SUCCESS [ 12.846 s]
[INFO] presto-clickhouse .................................. SUCCESS [  4.346 s]
[INFO] presto-server ...................................... SUCCESS [ 55.782 s]
[INFO] presto-verifier .................................... SUCCESS [01:49 min]
[INFO] presto-testing-server-launcher ..................... SUCCESS [ 10.482 s]
[INFO] presto-benchto-benchmarks .......................... SUCCESS [  2.873 s]
[INFO] presto-proxy ....................................... SUCCESS [  5.409 s]
[INFO] presto-benchmark-runner ............................ SUCCESS [  2.190 s]
[INFO] presto-spark ....................................... SUCCESS [  0.857 s]
[INFO] presto-spark-package ............................... SUCCESS [ 51.159 s]
[INFO] presto-spark-launcher .............................. SUCCESS [ 40.716 s]
[INFO] presto-spark-testing ............................... SUCCESS [  1.554 s]
[INFO] presto-thrift-spec ................................. SUCCESS [  2.246 s]
[INFO] presto-google-sheets ............................... SUCCESS [  4.913 s]
[INFO] presto-grpc-api .................................... SUCCESS [  5.536 s]
[INFO] presto-grpc-testing-udf-server ..................... SUCCESS [  0.959 s]
[INFO] presto-lark-sheets ................................. SUCCESS [  5.380 s]
[INFO] presto-test-coverage ............................... SUCCESS [ 13.061 s]
[INFO] presto-plan-checker-router-plugin .................. SUCCESS [  2.515 s]
[INFO] presto-router ...................................... SUCCESS [  8.363 s]
[INFO] presto-native-tests ................................ SUCCESS [  1.249 s]
[INFO] presto-open-telemetry .............................. SUCCESS [  2.393 s]
[INFO] redis-hbo-provider ................................. SUCCESS [  5.668 s]
[INFO] presto-openapi ..................................... SUCCESS [  6.518 s]
[INFO] presto-base-arrow-flight ........................... SUCCESS [  4.735 s]
[INFO] presto-function-server ............................. SUCCESS [ 14.299 s]
[INFO] presto-router-example-plugin-scheduler ............. SUCCESS [  2.095 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:41 min (Wall Clock)
[INFO] Finished at: 2025-12-02T10:33:12-08:00
[INFO] ------------------------------------------------------------------------

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.
  • If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

Please follow release notes guidelines and fill in the release notes below.

If release note is NOT required, use:

== NO RELEASE NOTE ==

@wangkewen wangkewen requested review from a team and shrinidhijoshi as code owners December 2, 2025 18:43
@prestodb-ci prestodb-ci added the from:Meta PR from Meta label Dec 2, 2025
@wangkewen wangkewen requested a review from ghelmling December 3, 2025 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:Meta PR from Meta

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants