Skip to content

[VL] Spark 4.1: Add TimeType (TIME data type) support (SPARK-51162) #11919

@baibaichen

Description

@baibaichen

Backend

VL (Velox)

Gluten version: main branch

Description

Spark 4.1 introduced the TIME data type (SPARK-51162 SPIP, 101 sub-tasks). DataTypeTestUtils.atomicTypes now includes TimeType, causing all tests that iterate over atomicTypes to fail when encountering the unsupported type.

Feature toggle: spark.sql.timeType.enabled (default true in tests).

Spark 4.1 only.

Parent issue: #11910 ([VL] Spark 4.x: Tracking new feature support)

Impact

Suite Failures How affected
GlutenRandomDataGeneratorSuite 232 atomicTypes enumeration
GlutenOrderingSuite 2 TimeType(0) / TimeType(6)
GlutenHiveResultSuite 1 New test (SPARK-51517)
GlutenCastWithAnsiOnSuite (partial) ~6 TimeType cast tests

Note: GlutenCastWithAnsiOnSuite also has 4 ANSI Cast failures — those belong to #10134, not this issue.

Additionally, some SQL query test files may be blocked by TimeType (cast.sql, literals.sql, nonansi/literals.sql).

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions