Skip to content

feat(specs): add Bayesian evidence status - #6879

Merged
stevenMevans merged 3 commits into
feat/abtesting-v3/bayesian-fieldsfrom
feat/abtesting-v3/evidence-field
Sep 21, 2026
Merged

stevenMevans merged 3 commits into
feat/abtesting-v3/bayesian-fieldsfrom
feat/abtesting-v3/evidence-field

Conversation

@stevenMevans

@stevenMevans stevenMevans commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

🧭 What and Why

Exposes the backend evidence verdict for each treatment metric comparison so clients can determine whether that result is trustworthy yet.

🎟 JIRA Ticket: OPTIM-2883

Changes included:

  • Adds variants[].metrics[].bayesian.evidence.status to Get, List, and Timeseries responses.
  • Defines the exact statuses: enough, not_enough, collecting, no_data, and unavailable.
  • Omits evidence when it is not applicable or no verdict exists.
  • Does not expose the obsolete test-level hasEnoughEvidence field.

🧪 Test

Green CI

@algolia-api-clients-automation-bot

algolia-api-clients-automation-bot Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

🪓 The generated code will be pushed at the end of the CI.

Action triggered by commit 2a36725a8d7b8d962eaa6d41285c9d423324da79.

Please do not push any generated code to this pull request.

@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from d1e5371 to 3e9150f Compare August 11, 2026 14:11
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from 4ba0f8e to ca60e31 Compare August 11, 2026 14:11
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch 2 times, most recently from e8dcdf5 to a6d4be0 Compare August 11, 2026 15:32
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from ca60e31 to 3d103e3 Compare August 11, 2026 15:32
@Fluf22
Fluf22 force-pushed the feat/abtesting-v3/bayesian-fields branch from 3d103e3 to b700fbe Compare August 12, 2026 15:27
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from a6d4be0 to 5adfb77 Compare August 18, 2026 13:33
@stevenMevans stevenMevans self-assigned this Aug 18, 2026
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 5adfb77 to 6912498 Compare September 1, 2026 05:07
@stevenMevans stevenMevans changed the title feat(specs): add hasEnoughEvidence to the A/B test read model feat(specs): add Bayesian evidence status Sep 1, 2026
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 6912498 to 55ffd31 Compare September 1, 2026 13:54
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch 2 times, most recently from f7de49b to 4443e84 Compare September 1, 2026 13:56
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch 2 times, most recently from 5249961 to 7aa75f9 Compare September 2, 2026 14:46
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch 2 times, most recently from 4f91eb5 to ace28ef Compare September 3, 2026 12:45
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch 2 times, most recently from 6990353 to c4063b1 Compare September 8, 2026 14:27
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch 2 times, most recently from 201b544 to 79fd2dd Compare September 10, 2026 13:09
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from c4063b1 to 8afdd45 Compare September 10, 2026 13:09
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from 79fd2dd to 9058f2e Compare September 13, 2026 04:41
@stevenMevans
stevenMevans removed this pull request from stack #6885 September 17, 2026 15:33
@stevenMevans
stevenMevans added this pull request to stack #7067 September 17, 2026 15:34
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from 0abf87e to 0a95144 Compare September 17, 2026 18:36
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 87e3085 to 1c07b54 Compare September 17, 2026 18:36
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from 0a95144 to 82f22e7 Compare September 18, 2026 14:24
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch 2 times, most recently from 367e917 to d207a39 Compare September 18, 2026 14:46
@stevenMevans
stevenMevans requested a balanced review from Copilot September 18, 2026 14:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Only a minor documentation ambiguity remains; the schema change is otherwise consistent and complete.

Pull request overview

Adds Bayesian evidence status to shared A/B testing metric schemas so generated clients can assess result trustworthiness.

Changes:

  • Adds optional metric evidence.
  • Defines five evidence statuses and examples.
File summaries
File Description
specs/abtesting-v3/common/schemas/Variant.yml Defines Bayesian metric evidence and status values.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread specs/abtesting-v3/common/schemas/Variant.yml Outdated
@stevenMevans
stevenMevans marked this pull request as ready for review September 18, 2026 16:38
@stevenMevans
stevenMevans requested a review from a team as a code owner September 18, 2026 16:38
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-18T16:40:33.600559Z 8d1bbdf Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/bayesian-fields branch from 63cdcb5 to 2b60dc3 Compare September 21, 2026 13:13
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 8d1bbdf to 0eeddeb Compare September 21, 2026 13:13
Fluf22
Fluf22 previously approved these changes Sep 21, 2026
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 0eeddeb to 8a415b2 Compare September 21, 2026 16:05
@stevenMevans
stevenMevans force-pushed the feat/abtesting-v3/evidence-field branch from 8a415b2 to 93a507d Compare September 21, 2026 16:51
@stevenMevans
stevenMevans removed this pull request from stack #7067 September 21, 2026 16:55
@stevenMevans
stevenMevans merged commit 53985a0 into feat/abtesting-v3/bayesian-fields Sep 21, 2026
5 checks passed
@stevenMevans
stevenMevans deleted the feat/abtesting-v3/evidence-field branch September 21, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants