backfill(ads): cancel ads_client_operations_and_errors_hourly_v1 backfill

[Almaju]

Description

Cancels the 2026-08-24 managed backfill entry for ads_derived.ads_client_operations_and_errors_hourly_v1 (status: Initiate → Cancelled) and records why in the entry's reason. Only backfill.yaml changes.

Why cancel instead of fix

The backfill failed with ValueError: Unable to parse parameter submission_date:DATE:{{ (execution_date - macros.timedelta(hours=1)).strftime('%Y-%m-%d') }}. Three independent reasons it can't run as configured:

1. _parse_parameter in bigquery_etl/cli/query.py only accepts {{ds}} for DATE parameters. This table's hourly schedule binds submission_date off execution_date, so every date in the range raises, including the dry run.
2. metadata.yaml has date_partition_parameter: null, so get_backfill_partition returns None — no $YYYYMMDD decorator. Each date would run bq query --replace against the whole staging table (16 in parallel), leaving one date's data.
3. backfill complete would then raise Null partition found completing backfill (bigquery_etl/cli/backfill.py:1469).

The only supported route to the data would be reinitialize_table: true, which rebuilds every partition from the _stable tables — mobile back to 2026-02-01 — to recover ~2.5 weeks of thin desktop rows. Not worth it for a table still in development, so cancelling and accepting the gap: 2026-08-07–2026-08-23 stays empty for desktop, and the scheduled runs cover everything from here on.

There's also a deadline: CI runs bqetl backfill validate over every backfill.yaml on any SQL-touching PR, and validate_old_entry_date raises for an Initiate entry older than 28 days. From 2026-09-22 that job would fail repo-wide until this entry stops being Initiate.

What this does NOT do

• Does not delete the staging table moz-fx-data-shared-prod.backfills_staging_derived.ads_derived__ads_client_operations_and_errors_hourly_v1_2026_08_24 — per docs/cookbooks/backfilling_a_table.md, a cancelled backfill's staging table is left to expire on its own (~30 days), not manually deleted. The Slack alert's delete instruction is for retrying, not cancelling.
• Does not set status to Complete — nothing was staged, so there's nothing to swap into production.
• Does not delete the entry or the file — the cancelled entry is the record.
• Does not touch metadata.yaml, query.sql, or schema.yaml.
• No BigQuery or Airflow changes.

Known follow-up (out of scope here)

The table is still unbackfillable. Fixing it means changing metadata.yaml scheduling: drop the templated parameters and destination_table, and let date_partition_parameter default to submission_date — the DAG generator then emits parameters=["submission_date:DATE:{{ds}}"] and lets the operator append $ds_nodash, so the bound date and destination partition always agree, including the hour-0 run. Side effect: a day's final recompute moves from 01:00 to 00:00 the next day, one hour less grace for late-arriving pings. Three other hourly tables (onboarding_hourly_v2, newtab_interactions_hourly_v1, newtab_content_reported_content_v1) have the same defect, so a framework fix to _parse_parameter may be preferred instead. Owner's call, separate PR.

Verification

• bqetl backfill validate moz-fx-data-shared-prod.ads_derived.ads_client_operations_and_errors_hourly_v1 passes.
• Confirmed via PyYAML: both entries parse, 2026-08-24 is Cancelled, 2026-04-01 is untouched at Complete, and no Initiate entry remains for this table.

Related Tickets & Documents

• Bug 2058567

Reviewer, please follow this checklist

[github-actions]

This PR flips the 2026-08-24 entry in ads_derived/ads_client_operations_and_errors_hourly_v1/backfill.yaml from Initiate to Cancelled and rewrites the entry's reason to record why; no other files change.

I checked the change against the tooling rather than only the description: Cancelled is a valid BackfillStatus, the entry still passes every check in validate_entries (entries stay sorted, the 2026-04-01 Complete entry is untouched, no remaining Initiate entry), and the requoted reason stays under the 160-char line-length limit in .yamllint.yaml. The stated blockers line up with metadata.yaml: date_partition_parameter: null plus a submission_date parameter templated off execution_date, which _parse_parameter in bigquery_etl/cli/query.py:488 rejects for DATE params, so the day-by-day path can't run and backfill complete would hit the null-partition check. Leaving the staging table to expire instead of deleting it matches docs/cookbooks/backfilling_a_table.md:432. The one documented alternative to reinitialize_table for a null date_partition_parameter (custom_query_path + override_depends_on_past_null_partition: true) would still fail on the same templated parameter, so the recorded rationale holds.

One checklist item: the PR references bug 2058567, so per .github/reviewer_checklist.md the bug number belongs in the PR title.

[github-actions]

suggestion: The permanent desktop gap (2026-08-07–2026-08-23) is recorded only in a cancelled backfill.yaml entry, which isn't surfaced anywhere a consumer of the table looks. Anyone charting surface/desktop volumes over August will read the empty window as a real drop. Add a line to description in metadata.yaml (it propagates to the BigQuery table description and the data catalog), e.g. "Desktop rows are absent for 2026-08-07 through 2026-08-23; the backfill for that window was cancelled (bug 2058567)."

[Almaju]

done: 22ce394

[scholtzan]

Integration report

• View full diff, if available (SQL + DAGs) (requires private-bigquery-etl access)
• Private CI run
• Bigeye monitoring dry-run: ✅ plan clean — view plan (requires private-bigquery-etl access)