Problem
On the results page, a metric that Jetstream failed on and a metric whose analysis window has simply not been computed yet are rendered identically: the same amber triangle icon, the same "No data available" tooltip, and the same amber "Metric unavailable / Other metrics may not be affected / Contact Experimenter Support" banner. The second state is the normal condition for every weekly or overall outcome metric on a recently launched experiment, so the page tells people to contact support about an analysis that is working correctly and just has not run yet.
Seen on built-in-vpn-beta-summer-promo-default-browser-experiment. Enrollment ended recently, so Jetstream has produced only daily windows; 16 of the 24 declared metrics are weekly/overall only and have no window to land in yet. Jetstream reported zero errors for the experiment (empty errors output, no failed queries, no metric-level exceptions in its logs), yet those 16 metrics all display the support-contact banner.
Cause
Both render sites branch on the same combined condition, has_errors or not has_data:
The two states are already computed separately in ResultsManager: metric_has_errors() reads results_data["v3"]["errors"], and metric_has_data() checks whether a valid point estimate exists. Only the template collapses them.
Proposed change
Split the branch so the two states render differently.
has_errors keeps the current amber warning treatment, the "Metric unavailable" copy, and the Contact Experimenter Support button. This is the state where support is the right next step.not has_data (and not has_errors) gets a blue informational treatment: an info icon, copy along the lines of "Not available yet" explaining the analysis window has not been computed, and no support button.
Add the new copy to NimbusUIConstants alongside the existing results strings rather than inlining it in the template.
Acceptance criteria
- A metric present in
results_data["v3"]["errors"] renders the amber warning state with the support button.
- A metric with no valid point estimate and no error renders the blue informational state with no support button.
- The collapsed header icon and tooltip distinguish the two states as well, not just the expanded banner.
- New user-facing copy lives in
NimbusUIConstants.
- View-level tests cover both states.
Note for a possible follow-up
not has_data currently covers two sub-cases: the metric is absent from the window's branch_data (not computed yet), and the metric is present but its point estimate is null (Jetstream ran and could not estimate it, for example a linear_model_mean on an extremely zero-inflated metric). Those are distinguishable from results_data, but this issue only splits error from no-data. A third "computed but not estimable" state can be filed separately if it is worth surfacing.
┆Issue is synchronized with this Jira Task
Problem
On the results page, a metric that Jetstream failed on and a metric whose analysis window has simply not been computed yet are rendered identically: the same amber triangle icon, the same "No data available" tooltip, and the same amber "Metric unavailable / Other metrics may not be affected / Contact Experimenter Support" banner. The second state is the normal condition for every weekly or overall outcome metric on a recently launched experiment, so the page tells people to contact support about an analysis that is working correctly and just has not run yet.
Seen on built-in-vpn-beta-summer-promo-default-browser-experiment. Enrollment ended recently, so Jetstream has produced only daily windows; 16 of the 24 declared metrics are weekly/overall only and have no window to land in yet. Jetstream reported zero errors for the experiment (empty errors output, no failed queries, no metric-level exceptions in its logs), yet those 16 metrics all display the support-contact banner.
Cause
Both render sites branch on the same combined condition,
has_errors or not has_data:The two states are already computed separately in
ResultsManager:metric_has_errors()readsresults_data["v3"]["errors"], andmetric_has_data()checks whether a valid point estimate exists. Only the template collapses them.Proposed change
Split the branch so the two states render differently.
has_errorskeeps the current amber warning treatment, the "Metric unavailable" copy, and the Contact Experimenter Support button. This is the state where support is the right next step.not has_data(and nothas_errors) gets a blue informational treatment: an info icon, copy along the lines of "Not available yet" explaining the analysis window has not been computed, and no support button.Add the new copy to
NimbusUIConstantsalongside the existing results strings rather than inlining it in the template.Acceptance criteria
results_data["v3"]["errors"]renders the amber warning state with the support button.NimbusUIConstants.Note for a possible follow-up
not has_datacurrently covers two sub-cases: the metric is absent from the window'sbranch_data(not computed yet), and the metric is present but its point estimate is null (Jetstream ran and could not estimate it, for example alinear_model_meanon an extremely zero-inflated metric). Those are distinguishable fromresults_data, but this issue only splits error from no-data. A third "computed but not estimable" state can be filed separately if it is worth surfacing.┆Issue is synchronized with this Jira Task