Backend
VL (Velox)
Gluten version: main branch
Description
Spark 4.1 introduced the TIME data type (SPARK-51162 SPIP, 101 sub-tasks). DataTypeTestUtils.atomicTypes now includes TimeType, causing all tests that iterate over atomicTypes to fail when encountering the unsupported type.
Feature toggle: spark.sql.timeType.enabled (default true in tests).
Spark 4.1 only.
Parent issue: #11910 ([VL] Spark 4.x: Tracking new feature support)
Impact
| Suite |
Failures |
How affected |
| GlutenRandomDataGeneratorSuite |
232 |
atomicTypes enumeration |
| GlutenOrderingSuite |
2 |
TimeType(0) / TimeType(6) |
| GlutenHiveResultSuite |
1 |
New test (SPARK-51517) |
| GlutenCastWithAnsiOnSuite (partial) |
~6 |
TimeType cast tests |
Note: GlutenCastWithAnsiOnSuite also has 4 ANSI Cast failures — those belong to #10134, not this issue.
Additionally, some SQL query test files may be blocked by TimeType (cast.sql, literals.sql, nonansi/literals.sql).
References
Backend
VL (Velox)
Gluten version: main branch
Description
Spark 4.1 introduced the TIME data type (SPARK-51162 SPIP, 101 sub-tasks).
DataTypeTestUtils.atomicTypesnow includesTimeType, causing all tests that iterate overatomicTypesto fail when encountering the unsupported type.Feature toggle:
spark.sql.timeType.enabled(default true in tests).Spark 4.1 only.
Parent issue: #11910 (
[VL] Spark 4.x: Tracking new feature support)Impact
Note: GlutenCastWithAnsiOnSuite also has 4 ANSI Cast failures — those belong to #10134, not this issue.
Additionally, some SQL query test files may be blocked by TimeType (cast.sql, literals.sql, nonansi/literals.sql).
References