Repository navigation
Remove references to the private experiments repository and its results - #197
Merged
Merged
Conversation
Docs no longer point at superred-experiments; the HarmBench example uses the public goal_passthrough baseline; smoke scripts and live tests read LiteLLM credentials from the environment (or load_dotenv) instead of the sibling repo.
Comments, docstrings, READMEs and ASSUMPTIONS.md files no longer cite internal study labels (RQ1.x, first run/sweep/archive, DTAP indirect sweep) or counts measured in those runs. The technical rationale is kept; scope labels (s3-s6) stay. No code behavior changes.
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The private
superred-experimentsrepository holds the authors' analysis. This PR removes every reference to it, and to its study results, from the public modules repo. Two commits:1. References to the repository (48d74cb)
README.md: combinations are now described as wired in a short script handed to theController, not as living insuperred-experiments/.security_claims/harmbench/README.md: the example importedDirectRequestOptimizerfrom a file that only exists in the private repo. It now uses the publicGoalPassthroughOptimizer, which does the same thing (injects the goal once, verbatim).DTAP_README.md: the section telling readers tocd ../superred-experiments/test_dtapis replaced by a generic description of the end-to-end wiring..env. The smoke scripts useload_dotenv(), and the live tests read environment variables only.2. Study labels and result counts (bb47be4)
ASSUMPTIONS.mdfiles (106 files) no longer cite internal study labels (RQ1.x, "first run/sweep/archive", "DTAP indirect sweep") or counts measured in those runs. The technical rationale is kept, reworded without the numbers. Scope labels (s3-s6) stay.test_no_temperature.pydocstrings. The 5judge_failure.pycopies stay byte-identical.Verification
ruff checkpasses on all changed.pyfiles.test_imports_without_heavy_deps(the package is not installed in the local venv) and dtap_scaffoldtest_judge_runner.py(the tests import from a different checkout).Not covered
$0.750000cost-cap string.