Skip to content

MAINT Extract attack factory adversarial configuration assembly #2765

Description

@romanlutz

Is your feature request related to a problem? Please describe.

AttackTechniqueFactory.create() and _build_adversarial_config() combine adversarial target selection, prompt resolution, conflicting-override checks, and attack construction. These are configuration rules that can be isolated without changing the attack algorithm or the factory's public interface.

This is F2 of three small factory extractions. Not ready yet: #2494 is actively changing this exact path to add benchmark-owned prompt guidance. Wait until that PR lands or its design is otherwise resolved, then reassess the extraction against the resulting mainline code before adding help wanted. Do not implement a competing version of its prompt behavior.

Describe the solution you'd like

After the prerequisite is resolved, extract adversarial configuration assembly into a small private helper at the technique layer and wire it into the factory. Preserve the resulting mainline behavior rather than restoring the pre-#2494 implementation.

  • Keep target precedence and the distinction between baked-in configuration, create-time inputs, and defaults explicit.
  • Preserve conflicts and conditional construction: do not silently override a baked-in custom target or prompt, or create adversarial configuration for an incompatible attack.
  • Keep system-prompt and seed-prompt assembly together with their validation. Reuse existing prompt/model helpers rather than duplicating them.
  • If FEAT: add benchmark-owned adversarial prompts #2494 lands, preserve its final guidance-prefix, simulated-conversation, prompt-metadata, and identity behavior as part of the extraction.

Acceptance criteria:

  • Default, baked-in, and create-time configurations produce the same target and prompts, with unchanged errors for conflicting inputs.
  • Supported string and SeedPrompt inputs retain their existing handling.
  • The public factory API and registered techniques keep their behavior; no global factory or canonical prompt is mutated by creating a specialized attack instance.
  • Any merged FEAT: add benchmark-owned adversarial prompts #2494 guidance composition retains native prompt parameters/schema/format and the intended component identity.
  • Factory and affected prompt/configuration coverage protects the full override matrix and repeated creation with different inputs.
  • The helper is used by the existing factory path, not left as unused scaffolding.

Describe alternatives you've considered, if relevant

Do not move packaging rules into executors or move attack turn management, pruning, backtracking, or scoring decisions into the factory. Do not combine this with a broad cleanup of defaults in older executors. Constructor compatibility and converter composition are separate, independently scoped extractions.

Additional context

Starting points: pyrit/scenario/core/attack_technique_factory.py, particularly create() and _build_adversarial_config(), and tests/unit/scenario/core/test_attack_technique_factory.py. Follow doc/code/framework.md and the scenario, Python, and test instructions.

Readiness prerequisite: #2494. Its current changes also touch per-instance seed-technique prefix composition, so confirm the final boundary after it settles. This issue is a behavior-preserving reorganization, not authorization for new prompt semantics or a public API redesign.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    not ready yetThis issue needs more definition or is blocked by a pending change.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions