Skip to content

MAINT GCG: model progressive admission transitions (#2665) - #2720

Merged
Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
Kaap10:refactor/gcg-progressive-schedule-2665
Sep 22, 2026
Merged

Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
Kaap10:refactor/gcg-progressive-schedule-2665

Conversation

@Kaap10

Copy link
Copy Markdown
Contributor

Description

Addresses item 3 of #2665.

This PR models and extracts the progressive admission and scheduling transition decisions from ProgressiveMultiPromptAttack.run() into a dedicated ProgressiveScheduleController component alongside ScheduleTransitionAction(Enum).

Summary of Changes

  1. Extracted ProgressiveScheduleController & ScheduleTransitionAction in pyrit/executor/promptgen/gcg/attack/base/progressive_schedule.py:

    • Encapsulates progressive goal and worker admission sequence (goals admitted before workers).
    • Manages remaining step budget calculation and enforces exact-budget termination without dangling sentinel resets.
    • Explicitly tracks loss_is_measured as a boolean flag to protect against sentinel-overwrite regressions while preserving legitimate non-finite model losses (inf / nan).
    • Implements control-weight ratcheting (+0.01 up to 0.09) and threshold transitions (> 0.09 disables stop_inner_on_success).
    • Triggers final held-out evaluation (ScheduleTransitionAction.FINALIZE_AND_STOP) only when fully admitted and success is achieved.
    • Houses ProgressiveScheduleState dataclass in the module for clean modularity.
  2. Refactored ProgressiveMultiPromptAttack.run() in pyrit/executor/promptgen/gcg/attack/base/attack_manager.py:

    • Delegates inner attack round preparation, remaining-step budgeting, and transition handling to ProgressiveScheduleController.
    • Preserves 100% backward compatibility for last_schedule_state, return signatures (tuple[str, int]), and parameter logging.

Tests and Documentation

Unit & Integration Tests:

  • Added dedicated unit test suite in tests/unit/executor/promptgen/gcg/test_progressive_schedule.py (16 deterministic tests):
    • Input validation for non-positive goals and workers.
    • Initial state configurations (progressive_goals / progressive_models).
    • Goal-before-worker admission sequencing.
    • FINALIZE_AND_STOP action on success in fully admitted state.
    • Control-weight ratchet increments and upper threshold transitions (> 0.09).
    • Exact-budget exhaustion boundaries at goal, worker, and control-weight transitions.
    • Non-finite inner loss tracking and validation.
  • Ran the full GCG unit and integration test suites:
    • pytest tests/unit/executor/promptgen/gcg/: 309 passed (100%)
    • pytest tests/integration/executor/promptgen/gcg/test_gcg_attack_wiring_integration.py tests/integration/executor/promptgen/gcg/test_gcg_integration.py: 16 passed (100%)

Code Quality & Static Analysis:

  • ty check pyrit/: 0 errors (all checks passed)
  • ruff check: 0 errors (all checks passed)
  • ruff format --check: 1653 files already formatted
  • pre-commit run: 18/18 hooks passed

@Kaap10

Copy link
Copy Markdown
Contributor Author

Hi Roman Lutz (@romanlutz), this PR is ready for review whenever convenient. Looking forward to your thoughts, and happy to make any adjustments needed!

Comment thread pyrit/executor/promptgen/gcg/attack/base/attack_manager.py
@Kaap10
Vardhman Gupta (Kaap10) force-pushed the refactor/gcg-progressive-schedule-2665 branch from 57d71e3 to 2138122 Compare September 22, 2026 03:04
@Kaap10

Copy link
Copy Markdown
Contributor Author

Thanks for catching that, Roman Lutz (@romanlutz)! Removed the duplicate class definition in attack_manager.py so the imported class is exported directly, and added regression assertions in test_run_state.py verifying both class identity and isinstance() checks against attack_manager.ProgressiveScheduleState.

@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Sep 22, 2026
Merged via the queue into microsoft:main with commit dfa2b76 Sep 22, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants