Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/gatekeeper-google/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Across all resource types, the gatekeeper can request:
- `calendar.events` to read and manage the selected calendar. Creating or changing an event nobody else is on can be auto-approved ("Add and change calendar events"). A change waits for approval when it would make the event public or otherwise widen who can see it, or when the event is an out-of-office, focus-time or other special event. Anything Google would tell attendees about, meaning an event that has or gains attendees, one organized by someone else, or one whose guest list Google hides, is "Send event invitations and updates", a separate kind a user may also opt into auto-approving, whatever `sendUpdates` is; the same visibility and special-event holds apply to it. An auto-approvable change is checked again when it applies, and fails without changing anything if the event has gained or lost guests, so that it would now fall under the other kind, or the change would now widen its visibility; the write is conditional on the version it checked, so a change made in between also fails it.
- `calendar.events.freebusy` to check availability, which `calendar.events` does not cover. Adding it retracted every existing Calendar grant that lacks a whole-calendar scope: owners re-consent the next time they connect a calendar, and an existing binding checks availability only after its account reconnects or grants Calendar again.
- `chat.spaces.readonly`, `chat.messages`, and `chat.memberships.readonly` for every Chat resource. A whole-account Chat connection adds `chat.users.readstate.readonly` for its unread-only search, and `chat.spaces.create` and `directory.readonly` to start direct messages and group chats with people in the connected account's Workspace directory. Starting a conversation is its own approval kind, separate from sending in an existing one, and people outside the organization can't be added to a new conversation.
- `bigquery` for BigQuery dry-runs and queries. This is intentionally broader than `bigquery.readonly` because dry-runs use `jobs.insert`; the gatekeeper enforces read-only SQL and resource scope checks before running queries. Query jobs run in the connected project, which must grant the account `bigquery.jobs.create`; a dataset shared from a project that doesn't (a dataset-level grant never does) is connected with `?queryProject=<own-project>` on its resource URL, which runs and bills the jobs there while the data scope stays on the shared dataset.
- `bigquery` for BigQuery dry-runs and queries. This is intentionally broader than `bigquery.readonly` because dry-runs use `jobs.insert`; the gatekeeper enforces read-only SQL and resource scope checks before running queries. Query jobs run in the connected project, which must grant the account `bigquery.jobs.create`; a dataset shared from a project that doesn't (a dataset-level grant never does) is connected with `?queryProject=<own-project>` on its resource URL, which runs and bills the jobs there while the data scope stays on the shared dataset. The configurator asks for that project only when BigQuery neither lists the data's project for the account nor accepts a dry-run job from it there, since its project list can omit a project the account may run jobs in.

### Step 4: Test Users

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,20 @@ vi.mock("@gadgets/configurator-ui", () => ({
Section: "Section",
}));

const MINE: BigQueryNode = { id: "project:mine", kind: "project", title: "mine", subtitle: "My project", listed: true };
const SHARED_PROJECT: BigQueryNode = { id: "project:theirs", kind: "project", title: "theirs", listed: false };
const MINE: BigQueryNode = { id: "project:mine", kind: "project", title: "mine", subtitle: "My project", runsQueries: true };
const SHARED_PROJECT: BigQueryNode = { id: "project:theirs", kind: "project", title: "theirs", runsQueries: false };
const SHARED_DATASET: BigQueryNode = {
id: "dataset:theirs/events", kind: "dataset", title: "events", listed: false,
id: "dataset:theirs/events", kind: "dataset", title: "events", runsQueries: false,
path: [{ id: "project:theirs", kind: "project", title: "theirs" }],
};
const VIEW: BigQueryNode = {
id: "table:mine/sales/daily", kind: "table", title: "daily", subtitle: "View", view: true, listed: true,
id: "table:mine/sales/daily", kind: "table", title: "daily", subtitle: "View", view: true, runsQueries: true,
path: [{ id: "project:mine", kind: "project", title: "mine" }, { id: "dataset:mine/sales", kind: "dataset", title: "sales" }],
};

type Values = { target?: string | null; queryProject?: string | null };

/** A fresh module, since it remembers which projects are listed at module level, and its frame. */
/** A fresh module, since it remembers which projects run queries at module level, and its frame. */
async function open(initial: Values = {}) {
vi.resetModules();
const spec = (await import("../src/configurator/bigquery-configurator-ui")).default as unknown as {
Expand Down Expand Up @@ -96,7 +96,7 @@ describe("listing", () => {
it("keeps a looked-up table in view when its dataset cannot be listed", async () => {
const frame = await open({ target: "table:theirs/events/clicks" });
const clicks: BigQueryNode = {
id: "table:theirs/events/clicks", kind: "table", title: "clicks", listed: false,
id: "table:theirs/events/clicks", kind: "table", title: "clicks", runsQueries: false,
path: [{ id: "project:theirs", kind: "project", title: "theirs" }, { id: "dataset:theirs/events", kind: "dataset", title: "events" }],
};
frame.ui.getNode.mockResolvedValueOnce(clicks);
Expand Down Expand Up @@ -166,7 +166,7 @@ describe("the query project", () => {

it("keeps a pre-filled query project in view for a project of the account's own, until cleared", async () => {
const frame = await open({ target: "dataset:mine/sales", queryProject: "other" });
frame.ui.getNode.mockResolvedValueOnce({ id: "dataset:mine/sales", kind: "dataset", title: "sales", listed: true });
frame.ui.getNode.mockResolvedValueOnce({ id: "dataset:mine/sales", kind: "dataset", title: "sales", runsQueries: true });
await frame.load("dataset:mine/sales");
expect(frame.ready()).toBe(true);
expect(frame.fields()).toEqual(["What to connect", "Run queries in"]);
Expand Down
55 changes: 42 additions & 13 deletions packages/gatekeeper-google/__tests__/workerd/configurators.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1206,10 +1206,13 @@ describe("BigQuery picker", () => {
const table = (projectId: string, datasetId: string, tableId: string, type = "TABLE", friendlyName?: string) =>
({ tableReference: { projectId, datasetId, tableId }, type, friendlyName });

/** An account holding a role in `mine` only, which was shared `theirs.events`. */
/**
* An account BigQuery lists a role in `mine` for, which was shared `theirs.events`, and which may
* create query jobs in `granted` though BigQuery does not list it. A dry run in `flaky` fails.
*/
function bigQuery() {
const calls: string[] = [];
vi.stubGlobal("fetch", vi.fn(async (input: string | URL | Request) => {
vi.stubGlobal("fetch", vi.fn(async (input: string | URL | Request, init?: RequestInit) => {
const url = new URL(typeof input === "string" || input instanceof URL ? input : input.url);
const path = decodeURIComponent(url.pathname.replace("/bigquery/v2", ""));
calls.push(path);
Expand All @@ -1234,6 +1237,17 @@ describe("BigQuery picker", () => {
return Response.json(table("theirs", "events", "clicks"));
case "/projects/example.com:legacy/datasets/events/tables/clicks":
return Response.json(table("example.com:legacy", "events", "clicks", "SNAPSHOT"));
case "/projects/granted/datasets":
return Response.json({ datasets: [dataset("granted", "reports")] });
case "/projects/granted/jobs":
Comment thread
ndisidore marked this conversation as resolved.
Resolved
// Accepts only a dry run: a probe that ran a real job, billed to the project, fails here.
return init?.method === "POST" && JSON.parse(String(init.body)).configuration?.dryRun === true
? Response.json({ status: { state: "DONE" } })
: Response.json({ error: { code: 400, message: "Not a dry run", errors: [{ reason: "invalid" }] } }, { status: 400 });
case "/projects/flaky/datasets":
return Response.json({ datasets: [dataset("flaky", "logs")] });
case "/projects/flaky/jobs":
return Response.json({ error: { code: 503, message: "Backend Error", errors: [{ reason: "backendError" }] } }, { status: 503 });
}
return Response.json({ error: { code: 403, message: "Access Denied", errors: [{ reason: "accessDenied" }] } }, { status: 403 });
}));
Expand All @@ -1243,11 +1257,11 @@ describe("BigQuery picker", () => {
it("lists the projects the account holds a role in, and inside one its datasets and tables, by name", async () => {
const { ui } = bigQuery();
expect(await ui.listNodes("bigquery", "", null)).toEqual({ nodes: [
{ id: "project:mine", kind: "project", title: "mine", subtitle: "My project", listed: true },
{ id: "project:theirs-app", kind: "project", title: "theirs-app", subtitle: undefined, listed: true },
{ id: "project:mine", kind: "project", title: "mine", subtitle: "My project", runsQueries: true },
{ id: "project:theirs-app", kind: "project", title: "theirs-app", subtitle: undefined, runsQueries: true },
] });
expect(await ui.listNodes("project:mine", "", null)).toMatchObject({ nodes: [
{ id: "dataset:mine/sales", title: "sales", subtitle: "Sales", listed: true, path: [{ id: "project:mine", title: "mine" }] },
{ id: "dataset:mine/sales", title: "sales", subtitle: "Sales", runsQueries: true, path: [{ id: "project:mine", title: "mine" }] },
{ id: "dataset:mine/web", title: "web" },
] });
expect(await ui.listNodes("dataset:mine/sales", "dai", null)).toMatchObject({ nodes: [
Expand Down Expand Up @@ -1278,26 +1292,41 @@ describe("BigQuery picker", () => {
const { ui, calls } = bigQuery();
// Ahead of a listed project whose name merely contains the ID typed.
expect(await ui.listNodes("bigquery", "theirs", null)).toEqual({ nodes: [
{ id: "project:theirs", kind: "project", title: "theirs", subtitle: undefined, listed: false },
{ id: "project:theirs-app", kind: "project", title: "theirs-app", subtitle: undefined, listed: true },
{ id: "project:theirs", kind: "project", title: "theirs", subtitle: undefined, runsQueries: false },
{ id: "project:theirs-app", kind: "project", title: "theirs-app", subtitle: undefined, runsQueries: true },
] });
expect(calls).toContain("/projects/theirs/datasets");
expect(await ui.listNodes("project:theirs", "", null)).toMatchObject({ nodes: [{ id: "dataset:theirs/events", listed: false }] });
expect(await ui.listNodes("project:theirs", "", null)).toMatchObject({ nodes: [{ id: "dataset:theirs/events", runsQueries: false }] });
const console = "https://console.cloud.google.com/bigquery?project=mine&p=theirs&d=events&page=dataset";
expect(await ui.listNodes("bigquery", console, null)).toMatchObject({ nodes: [
{ id: "dataset:theirs/events", subtitle: "Shared events", listed: false, path: [{ id: "project:theirs" }] },
{ id: "dataset:theirs/events", subtitle: "Shared events", runsQueries: false, path: [{ id: "project:theirs" }] },
] });
expect(await ui.listNodes("dataset:mine/sales", "`theirs.events.clicks`", null)).toMatchObject({ nodes: [
{ id: "table:theirs/events/clicks", kind: "table", listed: false },
{ id: "table:theirs/events/clicks", kind: "table", runsQueries: false },
] });
// A legacy domain-scoped project ID parses as a URL, but is a reference all the same.
expect(await ui.listNodes("bigquery", "example.com:legacy.events.clicks", null)).toMatchObject({ nodes: [
{ id: "table:example.com:legacy/events/clicks", subtitle: "Snapshot", view: undefined, listed: false },
{ id: "table:example.com:legacy/events/clicks", subtitle: "Snapshot", view: undefined, runsQueries: false },
] });
expect(await ui.listNodes("bigquery", "https://console.cloud.google.com/bigquery?project=theirs", null)).toMatchObject({ nodes: [
{ id: "project:theirs", listed: false },
{ id: "project:theirs", runsQueries: false },
] });
expect(await ui.getNode("dataset:theirs/events")).toMatchObject({ title: "events", listed: false });
expect(await ui.getNode("dataset:theirs/events")).toMatchObject({ title: "events", runsQueries: false });
});

it("runs queries in a project BigQuery does not list when it lets the account create jobs there", async () => {
const { ui, calls } = bigQuery();
expect(await ui.getNode("project:granted")).toMatchObject({ id: "project:granted", runsQueries: true });
expect(await ui.listNodes("project:granted", "", null)).toMatchObject({ nodes: [{ id: "dataset:granted/reports", runsQueries: true }] });
expect(await ui.getNode("dataset:theirs/events")).toMatchObject({ runsQueries: false });
// Each project BigQuery does not list is asked once; one it lists, never.
expect(calls.filter(path => path.endsWith("/jobs"))).toEqual(["/projects/granted/jobs", "/projects/theirs/jobs"]);
});

it("asks for a query project, without failing the lookup, when a dry run fails for another reason than a refusal", async () => {
const { ui } = bigQuery();
Comment thread
ndisidore marked this conversation as resolved.
Resolved
expect(await ui.getNode("project:flaky")).toMatchObject({ id: "project:flaky", runsQueries: false });
expect(await ui.listNodes("project:flaky", "", null)).toMatchObject({ nodes: [{ id: "dataset:flaky/logs", runsQueries: false }] });
});

it("answers nothing for what the account cannot see, saying so only for an explicit link", async () => {
Expand Down
8 changes: 6 additions & 2 deletions packages/gatekeeper-google/src/bigquery-api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -433,11 +433,14 @@ export class BigQueryApi {
};
}

/** Validate `sql` and estimate its cost as a dry-run job in `billingProject`. */
/**
* Validate `sql` and estimate its cost as a dry-run job in `billingProject`. `signal` ends the
* request it is in flight for, and keeps a retry from being sent.
*/
async dryRun(
billingProject: string,
sql: string,
opts: Pick<BigQueryJobOptions, "defaultDataset" | "params"> = {},
opts: Pick<BigQueryJobOptions, "defaultDataset" | "params"> & { signal?: AbortSignal } = {},
): Promise<BigQueryDryRunResult & {
statementType?: string;
ddlOperationPerformed?: string;
Expand Down Expand Up @@ -468,6 +471,7 @@ export class BigQueryApi {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
signal: opts.signal,
}, this.getAccessToken);

if (resp.status?.errorResult) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ export type { ConfiguratorOption };

/**
* The chosen node, by its {@link BigQueryNode.id}, or null when nothing is chosen; and the project
* query jobs run in when the node's project is not one this account holds a role in (see
* {@link BigQueryNode.listed}), else null.
* query jobs run in when they cannot run in the node's project (see {@link BigQueryNode.runsQueries}),
* else null.
*/
export type BigQueryConfiguratorValues = { target?: string | null; queryProject?: string | null };

Expand All @@ -25,11 +25,11 @@ export type BigQueryNode = {
/** Set on a table that is a view or materialized view. */
view?: true;
/**
* Whether the node's project is one BigQuery lists for this account: the projects it holds a role
* in, where it can usually run query jobs. A dataset shared from another project is reached by
* ID instead, and a connection to it names a listed project to run its queries in.
* Whether query jobs can run in the node's project: one BigQuery lists for this account, as one it
* holds a role in, or one that accepts a dry-run job from it. A project that only shared a dataset
* with it does not, so a connection to that names another project to run its queries in.
*/
listed: boolean;
runsQueries: boolean;
/** The node's ancestors below the root, outermost first, for a node looked up or found by search. */
path?: BigQueryCrumb[];
};
Expand Down
Loading
Loading