feat(nimbus): extend population sizing ETL to support Fenix and iOS mobile - #9824
Conversation
…obile experiments Add Fenix and iOS targeting tables, group experiments by appName, use hash-based 10% sampling for mobile (no sample_id column in derived tables), and include platform in results written to GCS and BigQuery.
yashikakhurana
changed the title
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR extends the Nimbus pre-launch population sizing ETL from desktop-only to also cover Fenix and iOS: experiments from the Experimenter v8 API are grouped by appName, each group is dry-run validated and queried against its own targeting-context table, and a platform column is added to the BigQuery schema and the GCS payload. Mobile uses hash-based 10% sampling and submission_date-based dedup since the derived tables have no sample_id.
The main thing to resolve before this lands is the dry-run validation probe: require_partition_filter: true on both mobile derived tables makes WHERE FALSE an invalid query, so every Fenix and iOS experiment gets dropped and the new mobile path becomes a silent no-op. The remaining comments are about failure isolation between platforms and keeping the two sampling constants linked.
On the reviewer checklist: the PR title carries the EXP-7366 reference, this isn't a fork, schema.yaml is updated for the new platform field, referenced_tables and dag_name line up (both mobile tables are produced in bqetl_experiments_daily), no backfill.yaml change, and monitoring isn't a restricted namespace.
This comment has been minimized.
This comment has been minimized.
- Add submission_date partition filter to dry-run probe so mobile tables with require_partition_filter=true pass validation - Wrap per-platform loop body in try/except so one platform failure does not discard results already accumulated for other platforms - Use MOD(..., 100) < SAMPLE_ID_MAX for mobile sampling to mirror desktop sample_id range and keep both branches linked to the constant - Replace table+is_mobile params with app_name in build_query and _dry_run_sql; derive config from _APP_CONFIG to make mismatches unrepresentable
This comment has been minimized.
This comment has been minimized.
mikewilli
left a comment
mikewilli
left a comment
There was a problem hiding this comment.
I am thinking it might make more sense to separate the ETL for querying the population estimates from the task that joins the various apps and sends results to GCS. Do you think it would fit and make sense to create a per-app table via the templating we have for other experiment monitoring here https://github.com/mozilla/bigquery-etl/blob/main/sql_generators/experiment_monitoring/templates/templating.yaml, and then query the result(s) more straightforwardly here to aggregate and send to GCS and a final view or table to BigQuery?
If you'd rather stick with this approach, see my comments below for standardizing the query for all apps if possible.
…gation
Create nimbus_sizing_clients_{desktop,fenix,ios}_v1 tables that pre-materialise
a 7-day rolling window of deduplicated, 10%-sampled clients per app. The
experiment_population_estimates_v1 query.py now just runs COUNTIF queries
against these pre-built tables and ships results to GCS and BigQuery, with
no inline deduplication, date arithmetic, or is_mobile branching needed.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…ient tables
Replaces three manually-maintained SQL files with a single Jinja2 template
in sql_generators/nimbus_population_sizing/. The generator renders
nimbus_sizing_clients_{desktop,fenix,ios}_v1 from one template with
conditional blocks for desktop vs mobile differences (client ID field,
sampling method, date column). Running `bqetl generate
nimbus_population_sizing` regenerates all three tables from the single
source of truth.
…toring generator
Moves the per-app nimbus sizing client pool tables into the existing
experiment_monitoring sql_generator following Mike's suggestion, using
the same pattern as experiment_enrollment_aggregates_v1 with {% if %}
conditional blocks for desktop vs Glean mobile apps.
Changes:
- Extend experiment_monitoring/__init__.py to support destination_project
per query in per_app: true mode
- Add nimbus_sizing_clients_v1 template to experiment_monitoring/templates/
with desktop/mobile conditional blocks
- Register nimbus_sizing_clients_v1 in templating.yaml targeting
moz-fx-data-experiments, skipping all non-sizing apps
- Move generated tables from monitoring.nimbus_sizing_clients_{app}_v1
to {app}_derived.nimbus_sizing_clients_v1 to match the generator naming
- Remove standalone nimbus_population_sizing generator
- Update query.py table references to new paths
…ference new sizing table paths
… lines in generated output
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Generated output from sql_generators is not committed to sql/ — the template in sql_generators/experiment_monitoring/templates/ is the source of truth and bqetl generate produces the files at deploy time.
This comment has been minimized.
This comment has been minimized.
mikewilli
left a comment
mikewilli
left a comment
There was a problem hiding this comment.
I think we should write these intermediate tables to shared-prod, but otherwise I think this makes sense now.
…ents Intermediate client pool tables belong in moz-fx-data-shared-prod alongside their source data. Only the final GCS/BQ output (experiment_population_estimates_v1) stays in moz-fx-data-experiments.
There was a problem hiding this comment.
I don't think these changes are needed anymore, are they?
…itoring generator No query currently uses destination_project; revert the __init__.py change to keep the diff minimal.
This comment has been minimized.
This comment has been minimized.
Integration report
|
3 participants
Description
This PR extends the Nimbus pre-launch population sizing ETL to support Fenix and iOS mobile experiments alongside the existing Desktop support. Experiments are grouped by
appNamefrom the Experimenter API and queried against their respective BQ targeting tables. Mobile uses hash-based 10% sampling since derived tables have nosample_idcolumn. Aplatformcolumn is added to BQ and GCS output.Related Tickets & Documents
Reviewer, please follow this checklist