Repository navigation
Migrate to GitHub Actions - #8835
Merged
Merged
Conversation
scholtzan
force-pushed
the
migrate-to-github-actions-with-tests
branch
from
February 10, 2026 23:01
78627cb to
5985aba
Compare
Contributor
Integration report for "Migrate to GitHub actions with tests"
|
Contributor
Integration report for "Migrate to GitHub actions with tests"
|
Contributor
Integration report for "Migrate to GitHub actions with tests"
|
BenWu
reviewed
Feb 12, 2026
| pip install --no-deps -r requirements.txt | ||
| pip-sync --pip-args=--no-deps | ||
| - name: Save cached virtualenv | ||
| if: steps.cache-venv.outputs.cache-hit != 'true' |
Contributor
There was a problem hiding this comment.
I don't see a cache-venv step. Is that missing? This build job outputs a Cache save failed. warning which I'm guessing is because the key is already used by a previous build. Did you mean to attempt to pull from the cache before this?
| PATH=".venv/bin:$PATH" pip-compile --allow-unsafe --generate-hashes --quiet | ||
| git diff --exit-code -G '^ *[^# ]' -- requirements.txt | ||
|
|
||
| verify-format-yaml: |
Contributor
There was a problem hiding this comment.
The run page shows a bunch of yaml warnings in the annotations section which is useful but those should be fixed as part of this.
| needs: [generate-dags, main-generate-sql-and-dags, decide-runs] | ||
| if: | | ||
| (github.ref != 'refs/heads/main') && | ||
| (needs.decide-runs.outputs.validate-sql == 'true' || needs.decide-runs.outputs.validate-routines == 'true' || needs.decide-runs.outputs.deploy == 'true') |
Contributor
There was a problem hiding this comment.
outputs.deploy = "${{ github.ref }}" == "refs/heads/main" so it would always be false
Suggested change
| (needs.decide-runs.outputs.validate-sql == 'true' || needs.decide-runs.outputs.validate-routines == 'true' || needs.decide-runs.outputs.deploy == 'true') | |
| (needs.decide-runs.outputs.validate-sql == 'true' || needs.decide-runs.outputs.validate-routines == 'true') |
|
|
||
| At this point, the table exists in Bigquery so you are able to: | ||
| - [Find and re-run the CI](https://app.circleci.com/pipelines/github/mozilla/bigquery-etl?) of your PR and make sure that all tests pass | ||
| - [Find and re-run the CI](https://github.com/mozilla/bigquery-etl/actions) of your PR and make sure that all tests pass |
Contributor
There was a problem hiding this comment.
The top-level actions page has a lot of other stuff in it which would make it harder to find a specific run
Suggested change
| - [Find and re-run the CI](https://github.com/mozilla/bigquery-etl/actions) of your PR and make sure that all tests pass | |
| - [Find and re-run the CI](https://github.com/mozilla/bigquery-etl/actions/workflows/build.yml) of your PR and make sure that all tests pass |
| 1. Open a new PR with these changes. | ||
| 1. PR reviewed and approved. | ||
| 1. Find and run again the [CI pipeline](https://app.circleci.com/pipelines/github/mozilla/bigquery-etl?) for the PR. | ||
| 1. Find and run again the [CI pipeline](https://github.com/mozilla/bigquery-etl/actions) for the PR. |
Contributor
There was a problem hiding this comment.
Suggested change
| 1. Find and run again the [CI pipeline](https://github.com/mozilla/bigquery-etl/actions) for the PR. | |
| 1. Find and run again the [CI pipeline](https://github.com/mozilla/bigquery-etl/actions/workflows/build.yml) for the PR. |
Comment on lines
+36
to
+37
| # running bq in CircleCI will fail since it's not installed | ||
| # in GitHub Actions, bq is installed but may fail for other reasons |
Contributor
There was a problem hiding this comment.
This reads like it's one (confusing) sentence
Suggested change
| # running bq in CircleCI will fail since it's not installed | |
| # in GitHub Actions, bq is installed but may fail for other reasons | |
| # Running bq in CircleCI will fail since it's not installed. | |
| # In GitHub Actions, bq is installed but may fail for other reasons |
Comment on lines
+85
to
+86
| # running bq in CircleCI will fail since it's not installed | ||
| # in GitHub Actions, bq is installed but may fail for other reasons |
Contributor
There was a problem hiding this comment.
Suggested change
| # running bq in CircleCI will fail since it's not installed | |
| # in GitHub Actions, bq is installed but may fail for other reasons | |
| # Running bq in CircleCI will fail since it's not installed. | |
| # In GitHub Actions, bq is installed but may fail for other reasons |
Comment on lines
+141
to
+142
| # running bq in CircleCI will fail since it's not installed | ||
| # in GitHub Actions, bq is installed but may fail for other reasons |
Contributor
There was a problem hiding this comment.
Suggested change
| # running bq in CircleCI will fail since it's not installed | |
| # in GitHub Actions, bq is installed but may fail for other reasons | |
| # Running bq in CircleCI will fail since it's not installed. | |
| # In GitHub Actions, bq is installed but may fail for other reasons |
Contributor
Integration report for "Migrate to GitHub Actions"
|
BenWu
reviewed
Feb 13, 2026
Co-authored-by: Ben Wu <12437227+BenWu@users.noreply.github.com>
Contributor
Integration report for "Migrate to GitHub Actions"
|
BenWu
reviewed
Feb 13, 2026
BenWu
left a comment
BenWu
left a comment
Contributor
There was a problem hiding this comment.
Hopefully this is the last thing to get the cache working. Everything else LGTM
* casting installs as integer to match schema * updating schema to match table * removing sql logic change
… getting overwritten (#8853) * fix(SubPlat): Remove partitioning from ETLs where the whole table is getting overwritten. * docs(SubPlat): Add comments explaining why we avoid using partitioning for certain ETLs.
I missed adding this one the first time around. Co-authored-by: Anna Scholtz <anna@scholtzan.net>
Contributor
Integration report for "Migrate to GitHub Actions"
|
BenWu
approved these changes
Feb 17, 2026
Contributor
Integration report for "Migrate to GitHub Actions"
|
This was referenced Feb 25, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR migrates bigquery-etl to Github Actions. mozilla/telemetry-airflow#2339 needs to be merged alongside it as this will also change where docker images will get published to (GAR).
CircleCI will need to be disabled once this is merged.
I did test most of the CI tasks (also those that will just run on
main) in github.com/mozilla/bigquery-etl/actions/runs/21886512482 (except for syncing the DAGs submodules) and made some SQL changes there to trigger stage deploys. Some CI tests run a few seconds faster, some a few seconds slower compared to CircleCI (https://app.circleci.com/pipelines/github/mozilla/bigquery-etl/57677/workflows/61769702-d64a-49c9-a571-de1de5c9999a). Overall, the runtime seems to be about the same.An annoying thing with Github Actions is that it creates a deployment when accessing environment variables in the
devenvironment. The stage deploy credentials need to come from that environment (see https://github.com/mozilla/dataservices-infra/pull/1512#discussion_r2775354202). There is an open roadmap item to decouple deployments from using the environment variables: https://github.com/orgs/community/discussions/36919 but until this is implemented every time a CI task runs that accesses an environment variable it adds a message to the PR.Related Tickets & Documents
Reviewer, please follow this checklist