ci: share extended test commands through xtask

[kumarUjjawal]

Which issue does this PR close?

• Part of Add Local Scripts to Reproduce Full CI #21048 and EPIC: Organize CI with xtask to keep local and Github runs in sync #24487. Both issues stay open.

Rationale for this change

The three extended test suites run only in the merge queue, on release branches, and by manual dispatch. Their cargo test commands and environment variables exist only inside extended.yml. A developer who wants to reproduce a merge-queue failure must copy them by hand. This PR defines each command once in xtask, so local runs and GitHub Actions use the same definition.

What changes are included in this PR?

• cargo xtask ci step test gets three variants: extended, hash-collisions, and sqlite. Each variant reproduces the current CI invocation and sets only the environment that its suite needs. The sqlite variant sets no RUST_BACKTRACE, because its CI job skips the builder setup on purpose.
• extended.yml calls the three commands. Triggers, job IDs, job names, runners, containers, the clean working tree check, and cargo clean do not change.
• xtask/README.md documents the commands, their environment, prerequisites, and the parts of a CI job that a command does not reproduce.

--explain prints the underlying command without running the suite:

cargo xtask ci step test extended --explain

What is the testing strategy for this PR?

• Snapshot tests in xtask/src/ci_steps.rs cover the exact command, working directory, and environment of each variant, plus rejection of an unknown variant.
• check_asf_yaml_status_checks.py passes. All required check names are unchanged.
• Each new command ran the real suite locally through xtask on macOS. The extended, hash-collisions, and sqlite suites all passed with a clean working tree.

The extended jobs skip ordinary pull request updates, so green PR checks do not exercise these commands. The merge queue does.

Are there any user-facing changes?

No. The GitHub CI behavior is the same. Developers get three new local commands.

🤖 Generated with Claude Code

[2010YOUY01]

Thank you, LGTM.

[2010YOUY01]

Suggested change

	### Extended test suites
	The [extended tests] are long-running suites that run in the merge queue, on
	pushes to release branches, and on manual dispatch. They do not run on ordinary
	pull request updates. Each job in [`extended.yml`] runs one of these commands:
	| GitHub Actions job | Command |
	| ---------------------------------------------- | ------------------------------------------ |
	| `cargo test 'extended_tests' (amd64)` | `cargo xtask ci step test extended` |
	| `cargo test hash collisions (amd64)` | `cargo xtask ci step test hash-collisions` |
	| `Run sqllogictests with the sqlite test suite` | `cargo xtask ci step test sqlite` |
	Append `--explain` to print the `cargo test` invocation, its working directory,
	and the environment variables the command sets, without running the suite:
	```shell
	cargo xtask ci step test extended --explain
	cargo xtask ci step test hash-collisions --explain
	cargo xtask ci step test sqlite --explain
	```
	Each command sets only the environment variables that its test suite needs.
	Everything else is inherited from the calling shell:
	- `extended` sets `RUST_BACKTRACE=1` and `DATAFUSION_SPILL_POOL_FUZZ_ITERATIONS=1000`.
	The second variable runs more random spill pool fuzzer scenarios than the
	default test suite does.
	- `hash-collisions` runs from the `datafusion` directory and sets
	`RUST_BACKTRACE=1`, which matches the CI builder setup for that job.
	- `sqlite` sets no variables. Its CI job deliberately skips the builder setup
	because backtraces make this suite much slower. A `RUST_BACKTRACE` value
	inherited from your shell therefore makes a local run differ from CI.
	#### Prerequisites
	- The Rust toolchain from `rust-toolchain.toml` and the Protobuf compiler
	(`protoc`). See the [development environment] guide.
	- The test data submodules. The `sqlite` suite reads
	`datafusion-testing/data/sqlite`:
	```shell
	git submodule update --init --recursive
	```
	- Time and disk space. These suites build most of the workspace with test
	features enabled, and the `sqlite` suite runs several million queries.
	#### What a test command does not do
	A test command reproduces one `cargo test` invocation from a CI job. It is not
	the complete job. The GitHub Actions workflow still installs the toolchain,
	configures build flags, caching, and network settings, verifies that the
	working tree is clean, and runs `cargo clean`. The commands never run cleanup
	and never call the GitHub API, so they are safe to run in a local checkout.
	[extended tests]: https://github.com/apache/datafusion/blob/main/docs/source/contributor-guide/testing.md#extended-tests
	[`extended.yml`]: https://github.com/apache/datafusion/blob/main/.github/workflows/extended.yml
	[development environment]: https://github.com/apache/datafusion/blob/main/docs/source/contributor-guide/development_environment.md

I think this change is related to specific test command, instead of xtask runner.

Now I think it's enough if we can keep them around the ci steps, later we might want to centralize somewhere, and let it show up in the explain like

yongting@Yongtings-MacBook-Pro-2 ~/C/d/datafusion (main) [1]> cargo xtask ci step test workspace --explain
#
# Run default test suite and generate codecov report    <--- Here: per command explanation
#
cd /Users/yongting/Code/datafusion2/datafusion && \
cargo llvm-cov \
--profile ci \
--exclude datafusion-examples \
--exclude ffi_example_table_provider \
--exclude datafusion-cli \
--workspace \
--lib \
--tests \
--bins \
--features serde,avro,json,backtrace,integration-tests,parquet_encryption,substrait \
--codecov \
--output-path target/codecov.json

[kumarUjjawal]

Thank you sounds good!

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.88%. Comparing base (9082d6b) to head (c81dd77).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #25256    +/-   ##
========================================
  Coverage   81.88%   81.88%            
========================================
  Files        1133     1133            
  Lines      424522   424639   +117     
  Branches   424522   424639   +117     
========================================
+ Hits       347623   347730   +107     
- Misses      56285    56293     +8     
- Partials    20614    20616     +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kumarUjjawal]

Thank you @2010YOUY01 for the review.