Skip to content

Parallelize CI#932

Open
levkk wants to merge 11 commits intomainfrom
levkk-base-package-images
Open

Parallelize CI#932
levkk wants to merge 11 commits intomainfrom
levkk-base-package-images

Conversation

@levkk
Copy link
Copy Markdown
Collaborator

@levkk levkk commented Apr 24, 2026

  1. Fix [CI] Add base images #931. Add base images for pgdog package, so main builds are faster.
  2. Parallelize all integration tests, re-use build binaries, so they run much faster (17min -> 6min). Flaky tests don't block others from running / finishing. Flaky tests can be independently retried.

levkk and others added 4 commits April 24, 2026 11:11
Temporary push trigger so the new package-base and package-ci-base
workflows fire off the feature branch; remove before merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@blacksmith-sh

This comment has been minimized.

@blacksmith-sh

This comment has been minimized.

@levkk levkk requested a review from meskill April 24, 2026 20:44
Comment thread .github/workflows/fmt.yml
run: cargo fmt --all -- --check
- name: Clippy
working-directory: pgdog
run: cargo clippy
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can cargo clippy --all-targets to check the tests as well 🧪

Comment thread .github/workflows/ci.yml
Comment on lines +50 to +64
- { name: pgbench, script: integration/pgbench/run.sh }
- { name: schema-sync, script: integration/schema_sync/run.sh }
- { name: go, script: integration/go/run.sh }
- { name: js, script: integration/js/pg_tests/run.sh }
- { name: ruby, script: integration/ruby/run.sh }
- { name: java, script: integration/java/run.sh }
- { name: mirror, script: integration/mirror/run.sh }
- { name: sql, script: integration/sql/run.sh }
- { name: toxi, script: integration/toxi/run.sh }
- { name: rust, script: integration/rust/run.sh }
- { name: python, script: integration/python/run.sh }
- { name: complex, script: integration/complex/run.sh }
- { name: dry-run, script: integration/dry_run/run.sh }
- { name: copy-data, script: integration/copy_data/run.sh }
- { name: load-balancer, script: integration/load_balancer/run.sh }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can go even further and specify only name and infer the path from it

Comment thread .github/workflows/ci.yml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename the file to integration to make it explicit?

Comment on lines +15 to +38
files=(
Cargo.lock
Cargo.toml
.cargo/config.toml
pgdog/Cargo.toml
pgdog/build.rs
pgdog-config/Cargo.toml
pgdog-macros/Cargo.toml
pgdog-plugin/Cargo.toml
pgdog-plugin/build.rs
pgdog-postgres-types/Cargo.toml
pgdog-stats/Cargo.toml
pgdog-vector/Cargo.toml
)

dirs=(
pgdog/src
pgdog-config/src
pgdog-macros/src
pgdog-plugin/src
pgdog-postgres-types/src
pgdog-stats/src
pgdog-vector/src
)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I bet we can use some grep/expansion magic here. But in that case we can touch unexpected things though won't break the cache when we add something new.

Comment thread .github/workflows/ci.yml
Comment on lines +46 to +48
strategy:
fail-fast: false
matrix:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, so cool, I wanted to do this as well, but I was staring in the lcov binary dances and the build sharing there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Add base images

2 participants