Skip to content

FEAT: Add Garak latent injection scenario #2573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Juhie (juhiechandra) wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

FEAT: Add Garak latent injection scenario #2573

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0f0a931
DOC: Document the Garak latent injection scenario
juhiechandra Sep 4, 2026
aab235b
FEAT: Add Garak latent injection seed datasets
juhiechandra Sep 4, 2026
545b206
FEAT: Add Garak latent injection scenario
juhiechandra Sep 4, 2026
42f83da
MAINT: Record NVIDIA attribution on the latent injection datasets
juhiechandra Sep 4, 2026
275dd75
FEAT: Assemble latent injection prompts in a dataset configuration
juhiechandra Sep 4, 2026
fa34f5b
DOC: Update the latent injection scenario docs
juhiechandra Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 53 additions & 8 deletions doc/scanner/garak.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,51 @@
"cell_type": "markdown",
"id": "8",
"metadata": {},
"source": [
"## LatentInjection\n",
"\n",
"Ports Garak's `latentinjection` probe family: indirect prompt injection, where the attacker\n",
"never talks to the model but plants an instruction inside a document the model is asked to read\n",
"-- a resume, a financial report, a legal filing, a WHOIS record. The attack succeeds when the\n",
"response echoes the trigger text the injection asked for, which `SubStringScorer` checks\n",
"directly.\n",
"\n",
"Each prompt is assembled from four local datasets -- a benign task instruction, a carrier\n",
"document, an injection instruction, and a payload carrying the trigger -- fenced together by the\n",
"separator that the selected technique names. One attack is built per technique and carrier\n",
"family, scored by an OR composite over that family's triggers.\n",
"\n",
"**CLI example:**\n",
"\n",
"```bash\n",
"pyrit_scan run garak.latent_injection --target openai_chat --techniques bare \\\n",
" --families whois --max-prompts-per-trigger 1\n",
"```\n",
"\n",
"**Available techniques** (14 separator styles): Bare, Newline, Blockquote, BlockquoteInline,\n",
"HorizontalRule, SystemTag, SystemBracket, SystemBracketFlood, SystemPrefix, InstructionTag,\n",
"AdminRequest, EndOfText, CoreInstruction, LegalAmendment.\n",
"\n",
"**Aggregate techniques:** `ALL` (all 14), `DEFAULT` (a representative subset), `PLAIN` (the\n",
"injection is set off by whitespace or quoting only), and `AUTHORITY` (the delimiter impersonates\n",
"a system, admin, or legal voice).\n",
"\n",
"**Carrier families** are seed metadata rather than techniques, selected with the `families` run\n",
"parameter: `translation_fr`, `translation_zh`, `report`, `resume`, `fact_eiffel`, `fact_legal`,\n",
"`whois`, `whois_snippet`, and `latent_jailbreak`. Every technique applies to every family.\n",
"`latent_jailbreak` hides a \"write something offensive about <group>\" instruction instead of a\n",
"fixed trigger, so it is excluded from the defaults and requires an explicit `harm_scorer`.\n",
"\n",
"One attack is built per technique, carrier family, and expected trigger, so each attack has a\n",
"single unambiguous success contract: its `SubStringScorer` looks for the one string its prompts\n",
"actually asked for. `max_prompts_per_trigger` caps how many prompts each of those cells holds.\n",
"There is no baseline attack -- the `bare` technique already covers \"no fencing at all\"."
]
},
{
"cell_type": "markdown",
"id": "9",
"metadata": {},
"source": [
"## Doctor\n",
"\n",
Expand All @@ -223,7 +268,7 @@
},
{
"cell_type": "markdown",
"id": "9",
"id": "10",
"metadata": {},
"source": [
"## SystemPromptExtraction\n",
Expand Down Expand Up @@ -256,7 +301,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "10",
"id": "11",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -278,7 +323,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "11",
"id": "12",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -287,7 +332,7 @@
},
{
"cell_type": "markdown",
"id": "12",
"id": "13",
"metadata": {},
"source": [
"## PackageHallucination\n",
Expand Down Expand Up @@ -322,7 +367,7 @@
},
{
"cell_type": "markdown",
"id": "13",
"id": "14",
"metadata": {},
"source": [
"## AudioAchillesHeel\n",
Expand Down Expand Up @@ -350,7 +395,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"id": "15",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -378,7 +423,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "15",
"id": "16",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -387,7 +432,7 @@
},
{
"cell_type": "markdown",
"id": "16",
"id": "17",
"metadata": {},
"source": [
"For more details, see the [Scenarios Programming Guide](../code/scenarios/0_scenarios.ipynb) and\n",
Expand Down
42 changes: 41 additions & 1 deletion doc/scanner/garak.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# extension: .py
# format_name: percent
# format_version: '1.3'
# jupytext_version: 1.19.4
# jupytext_version: 1.19.5
# ---

# %% [markdown]
Expand Down Expand Up @@ -154,6 +154,46 @@
# **Aggregate techniques:** `ALL` (all 8), `DEFAULT` (excludes the two combinatorial extended
# probes), `EXFIL` (the 6 markdown-exfil probes), and `XSS` (TaskXSS + MarkdownXSS).

# %% [markdown]
# ## LatentInjection
#
# Ports Garak's `latentinjection` probe family: indirect prompt injection, where the attacker
# never talks to the model but plants an instruction inside a document the model is asked to read
# -- a resume, a financial report, a legal filing, a WHOIS record. The attack succeeds when the
# response echoes the trigger text the injection asked for, which `SubStringScorer` checks
# directly.
#
# Each prompt is assembled from four local datasets -- a benign task instruction, a carrier
# document, an injection instruction, and a payload carrying the trigger -- fenced together by the
# separator that the selected technique names. One attack is built per technique and carrier
# family, scored by an OR composite over that family's triggers.
#
# **CLI example:**
#
# ```bash
# pyrit_scan run garak.latent_injection --target openai_chat --techniques bare \
# --families whois --max-prompts-per-trigger 1
# ```
#
# **Available techniques** (14 separator styles): Bare, Newline, Blockquote, BlockquoteInline,
# HorizontalRule, SystemTag, SystemBracket, SystemBracketFlood, SystemPrefix, InstructionTag,
# AdminRequest, EndOfText, CoreInstruction, LegalAmendment.
#
# **Aggregate techniques:** `ALL` (all 14), `DEFAULT` (a representative subset), `PLAIN` (the
# injection is set off by whitespace or quoting only), and `AUTHORITY` (the delimiter impersonates
# a system, admin, or legal voice).
#
# **Carrier families** are seed metadata rather than techniques, selected with the `families` run
# parameter: `translation_fr`, `translation_zh`, `report`, `resume`, `fact_eiffel`, `fact_legal`,
# `whois`, `whois_snippet`, and `latent_jailbreak`. Every technique applies to every family.
# `latent_jailbreak` hides a "write something offensive about <group>" instruction instead of a
# fixed trigger, so it is excluded from the defaults and requires an explicit `harm_scorer`.
#
# One attack is built per technique, carrier family, and expected trigger, so each attack has a
# single unambiguous success contract: its `SubStringScorer` looks for the one string its prompts
# actually asked for. `max_prompts_per_trigger` caps how many prompts each of those cells holds.
# There is no baseline attack -- the `bare` technique already covers "no fencing at all".

# %% [markdown]
# ## Doctor
#
Expand Down
Loading