Skip to content

perf: Add new version of stream run for 1GB from perf team#83

Open
akhildesaiIBM wants to merge 1 commit intoprestodb:mainfrom
akhildesaiIBM:akhil_1gb_tpcds
Open

perf: Add new version of stream run for 1GB from perf team#83
akhildesaiIBM wants to merge 1 commit intoprestodb:mainfrom
akhildesaiIBM:akhil_1gb_tpcds

Conversation

@akhildesaiIBM
Copy link
Copy Markdown

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 300 files, and this pull request has 2145

@wanglinsong wanglinsong requested a review from yabinma April 1, 2026 06:19
@yabinma
Copy link
Copy Markdown
Member

yabinma commented Apr 1, 2026

Sorry, I don't understand this PR. Don't we have sf1.json already? https://github.com/prestodb/pbench/blob/main/benchmarks/tpc-ds/sf1.json
And sf1 is in the SCALE_FACTOR list in the pbench pipeline job already.
If anything missed, we may make minor change. Why full tpc-ds queries list in this pr(2103 files)?

@akhildesaiIBM
Copy link
Copy Markdown
Author

@yabinma This PR is similar to @rzIBM PR #78 which was for 10TB spec run. We wanted to have 1GB and others. This is the readme file for reference https://github.com/prestodb/pbench/blob/main/benchmarks/tpc-ds/queries_v2/README.md

@ethanyzhang
Copy link
Copy Markdown
Collaborator

Why there is a folder 1?

@akhildesaiIBM
Copy link
Copy Markdown
Author

akhildesaiIBM commented Apr 8, 2026

@ethanyzhang It is for 1GB scale factor and matches existing pattern: 10000

@akhildesaiIBM
Copy link
Copy Markdown
Author

akhildesaiIBM commented Apr 9, 2026

@ethanyzhang Added 1 enhancement config file:

  1. ds_sanity_v2_1.json - Quick 3-query smoke test

- Add queries_v2/1/ with 2,079 TPC-DS queries across 21 streams
- Add streams_v2/1/ with 21 stream configurations
- Add ds_sanity_v2_1.json for quick validation
- Add power and throughput test configs
- Remove qall.sql and qlist.txt files to match 10TB structure
- Follow TPC-DS v4.0.0 spec query ordering

Enables CI/CD and development testing with 1GB dataset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants