feat(specs): extend ABTest data model with Bayesian fields

[stevenMevans]

🧭 What and Why

Supports the Bayesian A/B test method and its result fields. Users can select frequentist or Bayesian analysis when creating a test and inspect the corresponding Bayesian metric results.

🎟 JIRA Ticket: OPTIM-2775

Changes included:

• Adds method and primaryMetric to ABTestConfiguration.
• Adds the optional bayesian metric result with probabilityToBeBetter, relativeEffectCILow, and relativeEffectCIHigh.
• Documents the exact revenue_per_search result name, currency dimension, mean, and winsorized value metadata.

🧪 Test

• yarn specs:lint specs/abtesting-v3/
• yarn cli build specs abtesting-v3

PR Stack

1. feat(specs): add A/B test settings endpoints to abtesting-v3 #6786
2. feat(specs): add A/B testing hypothesis #6829
3. fix(specs): BREAKING CHANGE - make AB test pValue optional #7066
4. feat(specs): extend ABTest data model with Bayesian fields #6834 ⬅
5. feat(specs): add Bayesian evidence status #6879

[aikido-pr-checks]

Summary by Aikido

Security Issues: 0	Quality Issues: 0	Resolved Issues: 0

⚡ Enhancements

• Increased the main Algolia JavaScript bundle size threshold to 22.25KB

^{More info}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-18T14:29:03.997690Z	82f22e7	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4443e842e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

🔵 Needs a closer look

The methods schema lacks its documented uniqueness constraint, and Bayesian creation serialization remains untested.

Review details

Suppressed comments (8)

specs/abtesting-v3/common/parameters.yml:24

• The description rejects duplicate methods, but the array schema does not declare that constraint, so OpenAPI validators still accept values such as bayesian,bayesian. Add uniqueItems: true to keep the machine-readable contract aligned with the documented API behavior.

    type: array
    minItems: 1
    items:
      $ref: 'schemas/ABTest.yml#/AnalysisMethod'

specs/abtesting-v3/common/schemas/ABTest.yml:94

• The new creation fields are not exercised by the addABTests CTS fixture, which still sends no configuration. Because these schema properties become request-model fields in every generated client, add a Bayesian creation case containing both method and primaryMetric so serialization and naming are verified across languages.

    method:
      $ref: '#/AnalysisMethod'
    primaryMetric:
      $ref: '#/PrimaryMetric'

specs/abtesting-v3/common/parameters.yml:24

• The description says duplicate methods are invalid, but minItems only rejects an empty list and the schema still accepts values such as ["bayesian", "bayesian"]. Add uniqueItems: true so the OpenAPI contract and generated validation do not permit an input the API documents as invalid.

    type: array
    minItems: 1
    items:
      $ref: 'schemas/ABTest.yml#/AnalysisMethod'

specs/abtesting-v3/common/schemas/ABTest.yml:94

• These fields are now accepted in the create request through ABTestConfiguration, but the addABTests CTS fixture still only sends the minimal body and never serializes configuration.method or configuration.primaryMetric. Because request CTS runs against each generated language client, add an all-parameters case covering Bayesian creation so this new request contract is exercised.

    method:
      $ref: '#/AnalysisMethod'
    primaryMetric:
      $ref: '#/PrimaryMetric'

specs/abtesting-v3/common/schemas/ABTest.yml:94

• (raised by @copilot-pull-request-reviewer — unresolved) The new creation fields are not exercised by CTS: the existing addABTests fixture still sends only the minimal body, while the added fixtures cover only the methods query parameter. Add an addABTests case with configuration.method and configuration.primaryMetric, and assert both fields in the expected request body so model serialization is verified across languages.

    method:
      $ref: '#/AnalysisMethod'
    primaryMetric:
      $ref: '#/PrimaryMetric'

specs/abtesting-v3/common/parameters.yml:17

• (raised by @copilot-pull-request-reviewer — unresolved) The description says duplicate methods are invalid, but this array schema still accepts values such as [frequentist, frequentist]. Add uniqueItems: true so the OpenAPI contract and generated validation/documentation express the same constraint instead of deferring it entirely to a 422 response.

    Duplicate values aren't allowed.

specs/abtesting-v3/common/schemas/ABTest.yml:94

• These new create-request fields are not exercised by CTS: the existing addABTests.json case has no configuration, while the added CTS cases only test the methods query parameter. Add an addABTests case with configuration.method: bayesian and configuration.primaryMetric, and assert both values in the request body so generation/serialization is covered across languages. (raised by @copilot-pull-request-reviewer — unresolved)

    method:
      $ref: '#/AnalysisMethod'
    primaryMetric:
      $ref: '#/PrimaryMetric'

specs/abtesting-v3/common/parameters.yml:24

• The description rejects duplicate methods, but this array schema currently accepts them. Add uniqueItems: true so OpenAPI validation and generated documentation express the same request constraint instead of deferring it entirely to a 422 response. (raised by @copilot-pull-request-reviewer — unresolved)

    type: array
    minItems: 1
    items:
      $ref: 'schemas/ABTest.yml#/AnalysisMethod'

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82f22e7b28

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".