Skip to content

feat(devserver): support remote dataset version and environment selection - #622

Merged
Abhijeet Prasad (AbhiPrasad) merged 4 commits into
mainfrom
curtis/remote-evals-dataset-parity
Jul 29, 2026
Merged

feat(devserver): support remote dataset version and environment selection#622
Abhijeet Prasad (AbhiPrasad) merged 4 commits into
mainfrom
curtis/remote-evals-dataset-parity

Conversation

@cjgalione

@cjgalione Curtis Galione (cjgalione) commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Resolves https://linear.app/braintrustdata/issue/SDK-95/remote-eval-dataset-version-and-environment-selection-not-preserved

Summary

Adds dataset version and environment selection to Python remote evals, matching the JavaScript SDK behavior.

  • Forwards dataset_version and dataset_environment for both named and ID-based dataset references.
  • Adds environment selection to init_dataset() and lazily resolves it to a pinned dataset version.
  • Preserves _internal_btql and inline dataset behavior.
  • Gives an explicit dataset version precedence over an environment selector.
  • Includes org_name when resolving dataset environments and omits it when unavailable.
  • Adds end-to-end local HTTP coverage for name/ID references, selector precedence, URL encoding, and optional organization query shaping.

JavaScript SDK parity

This follows the JavaScript devserver and dataset-selection semantics:

  • both reference forms forward version and environment selectors;
  • explicit version wins over environment;
  • environment names are URL-encoded;
  • environment resolution includes org_name only when set.

Validation

cd py
mise exec -- nox -s test_cli -- -k test_get_dataset
mise exec -- ruff check src/braintrust/devserver/test_dataset.py src/braintrust/logger.py
mise exec -- ruff format --check src/braintrust/devserver/test_dataset.py src/braintrust/logger.py

7 passed; Ruff lint and formatting checks passed.

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) marked this pull request as ready for review July 29, 2026 16:17
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) changed the title fix(devserver): preserve remote dataset version selection feat(devserver): support remote dataset version and environment selection Jul 29, 2026
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit 69365c5 into main Jul 29, 2026
82 checks passed
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) deleted the curtis/remote-evals-dataset-parity branch July 29, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants