Skip to content

BigQuery: don't ask for a query project when the data's project accepts jobs - #1217

Open
ndisidore wants to merge 6 commits into
mainfrom
nathan/bigquery-dry-run-query-project
Open

ndisidore wants to merge 6 commits into
mainfrom
nathan/bigquery-dry-run-query-project

Conversation

@ndisidore

@ndisidore ndisidore commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

A user who can run queries in a project couldn't connect it. The BigQuery configurator decides where an account can run query jobs from projects.list, which came back empty for them. So the picker called the project one they have "no role in", required a separate "Run queries in" project, and had none to offer.

Now a project the list leaves out gets a free dry-run job (SELECT 1), once per project per picker. If BigQuery accepts it, the project runs its own queries and "Run queries in" stays hidden. Listed projects behave as before and are never probed.

  • BigQueryNode.listed is renamed runsQueries, since it no longer means "listed".
  • The helper text says "This account can't run query jobs in …" instead of "You have no role in …".
  • A new workerd test covers a project the list leaves out that accepts jobs. It failed before the fix.

Tested: the gatekeeper-google configurator tests (workerd and unit), and pnpm lint.


Devin Review

…ts jobs

The configurator trusted projects.list to say where an account can run query jobs. That list
can leave out a project the account does run jobs in (for one user it was empty), so the picker
demanded a separate query project and had none to offer, blocking the connection.

A project the list leaves out is now checked with a free dry-run job; if BigQuery accepts it,
the project runs its own queries. BigQueryNode.listed becomes runsQueries to match.
@github-actions github-actions Bot added the gatekeeper Changes to a gatekeeper integration label Nov 18, 2026

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

clanky-the-clanker[bot]

This comment was marked as resolved.

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Nov 18, 2026

Copy link
Copy Markdown

Review: 3 findings.

Posted all three as inline comments.

github run

@github-actions

Copy link
Copy Markdown

Preview: pr1217-nathan-bigqu-b332e7a2

https://pr1217-nathan-bigqu-b332e7a2-router.cloudflare-os-previews.workers.dev

Dashboard · deleted when this PR closes

A dry-run probe that failed for a reason other than a refusal (a 5xx, a rate limit) failed the
whole lookup or listing of a project BigQuery doesn't list, which worked before the probe. It
now answers no, as before, and isn't cached, so the next lookup asks again.

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 1 new potential issue.

Devin Review

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 9f7d1b97.

This review, of 94eba74a

✅ Approved — The earlier WARNING is fixed: a failed dry-run probe now answers no and isn't cached. No new issues were found.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The dry-run probe uses the user's own credentials and reads no data. It runs only for project IDs that have already been parsed.


📖 AGENTS.md: ✅ LGTM

No issues found.


📋 Code Quality: ✅ LGTM

#runsQueries now maps any probe failure other than a refusal to false, outside the cached loader. A failure is therefore not cached and gets retried, and listing and lookup of unlisted projects work again. The new workerd test covers a 503 from the probe.


⚡ Performance: ✅ LGTM

The probe runs at most once per project per picker, and concurrent callers share one request. Listed projects are never probed. An unlisted project's first listing costs one extra round trip.


📝 Documentation: ✅ LGTM

No issues found.


Reviewed commit: 94eba74a

The previous commit evicted a failed dry run so a later lookup would ask again, but the node
and listing caches kept the fallback, so the same lookup never did. Any failed dry run now
answers no and is kept for the picker, like a refusal; reopening the picker asks again.
ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Nov 18, 2026

Copy link
Copy Markdown

Review: 2 findings.

Posted both as inline comments.

github run

devin-ai-integration[bot]

This comment was marked as resolved.

The fake BigQuery now accepts a job in the probed project only as a POST with dryRun: true, so
a probe that would run a real, billed job fails the test.

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 85570957.

This review, of 9f7d1b97

✅ Approved — No findings. A failed dry run now answers no and is cached once per picker, so listing and lookup of unlisted projects keep working, and a failed probe never grants query access.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The dry-run probe (SELECT 1) uses the user's own credentials, reads no data and runs only for project IDs that have already been parsed. A failed probe keeps the "Run queries in" requirement, so an error never grants access.


📖 AGENTS.md: ✅ LGTM

No issues found.


📋 Code Quality: ✅ LGTM

#runsQueries turns every dry-run failure into false inside the cached loader, so each project is asked once per picker. #heldProjects still rejects on failure and is retried. The rename is applied consistently, and the new workerd tests cover the unlisted-project, 503 and single-probe cases.


⚡ Performance: ✅ LGTM

Each unlisted project is probed at most once per picker. Listed projects are never probed.


📝 Documentation: ✅ LGTM

No issues found.


Reviewed commit: 9f7d1b97

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Nov 18, 2026

Copy link
Copy Markdown

Review: 1 findings.

Posted one inline finding.

github run

A stalled jobs.insert held up the picker's lookup or listing of a project BigQuery doesn't list
for the client's 70-second timeout. BigQueryApi.dryRun takes a timeoutMs, and the picker's probe
answers no once 10 seconds pass, as for any other failed dry run.

@clanky-the-clanker clanky-the-clanker Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by the review of 78403d2e.

This review, of 85570957

✅ Approved — No findings. A dry run that times out or fails now answers no, and that answer is cached once per picker. Other callers keep their existing timeout.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The probe is a SELECT 1 dry run made with the user's own credentials, and only for project IDs that have already been parsed. A failed or timed-out probe keeps the "Run queries in" requirement, so an error never grants query access.


📖 AGENTS.md: ✅ LGTM

No issues found.


📋 Code Quality: ✅ LGTM

dryRun now takes an optional timeoutMs, passed through to callRest, which falls back to the existing request timeout when it isn't given. The picker's probe uses 10 seconds. A timeout is caught in #runsQueries like any other failure, and the cached false means each unlisted project is asked once per picker. The workerd tests check that the probe is a POST with dryRun: true, that unlisted projects are probed once, and that listed projects are never probed.


⚡ Performance: ✅ LGTM

The probe is limited to 10 seconds and runs at most once per unlisted project per picker. Listed projects are never probed.


📝 Documentation: ✅ LGTM

No issues found.


Reviewed commit: 85570957

ask-bonk[bot]

This comment was marked as resolved.

@ask-bonk

ask-bonk Bot commented Nov 18, 2026

Copy link
Copy Markdown

Review: 1 findings.

Posted one inline finding.

github run

…cluded

timeoutMs bounded each attempt only, so a 401 that waited on a token refresh (up to 20 s) and
then replayed could hold up the picker past its 10 s. The probe now races one deadline whose
signal also ends the request, so a replay after the deadline is never sent. BigQueryApi.dryRun
takes that signal instead of a timeoutMs.

@clanky-the-clanker clanky-the-clanker Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved — No findings. The dry run's single deadline, the signal passed through to the request, and the cached "no" answer on failure are all correct.

The decision is advisory: this review is a comment and never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

The probe is a SELECT 1 dry run made with the user's own credentials, and only for project IDs that have already been parsed. A failure or timeout answers "no", so an error never grants query access.


📖 AGENTS.md: ✅ LGTM

No issues found.


📋 Code Quality: ✅ LGTM

dryRun passes its signal to callRest, so the deadline stops a request in flight and keeps a retry from being sent once it has passed. In #runsQueries the race always settles to a boolean, and a probe that rejects late is still handled. In #node, listDatasets now runs only for unlisted projects, so a project this account can't see still comes back as null. The listed → runsQueries rename is applied everywhere.


⚡ Performance: ✅ LGTM

The probe has a 10-second limit that includes any token refresh, and it runs at most once per unlisted project per picker.


📝 Documentation: ✅ LGTM

No issues found.


Reviewed commit: 78403d2e

@ask-bonk

ask-bonk Bot commented Nov 18, 2026

Copy link
Copy Markdown

Review: 1 findings.

Posted one inline finding on missing regression coverage for the picker deadline during token refresh.

github run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gatekeeper Changes to a gatekeeper integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant