ECSO caption-mediated defense leaves encoded jailbreaks (code-completion, formal-logic) essentially unreduced on text-only VLM input

Auto-published from arXiv:2608.01043 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.72, flags: [no-prompt-excerpt]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

jailbreak

Model

gemini-2.5-flash-lite, gemini-2.0-flash, gpt-4o-mini, claude-sonnet-4-6

Surface

VLM API with ECSO black-box defense wrapper

Setup

Submit an encoded harmful query — either a Python deque code-completion attack (harmful words pushed onto a deque in reverse order, decoded via task = decode(my_stack)) or a MathPrompt-style set-theory rewrite (harmful instructions recast with ∀/∃ quantifiers and ⟹ implications) — as text-only input with no image attached. Apply the ECSO caption-mediated defense. No verbatim attack template is printed in the paper, but the deque-decoding and formal-logic families are both described in sufficient structural detail to reconstruct.

Observed behavior

ECSO reduces ASR by at most 6pp on text-only encoded input: gemini-2.5-flash-lite + code-completion attack stays at 83% ASR; gemini-2.5-flash-lite + formal-logic drops only from 57% to 51%. The defense's caption-generation branch never activates because no image is present, so the harmful encoded prompt reaches the model unchallenged and produces harmful outputs such as executable Python code for the queried harmful task.

Expected behavior

A defense claiming to handle encoded jailbreaks should substantially reduce ASR regardless of whether an image is attached, since the harmful content is entirely in the text.

Reproducibility

medium

Threat model

Any attacker sending text-only encoded jailbreaks to a production VLM endpoint protected only by ECSO (or similar caption-mediated wrappers) bypasses the defense at ~74–83% success rate. Operators who deploy caption-mediated defenses believing they are covered against symbolic encoding attacks receive false assurance.

Novelty

First systematic measurement showing that caption-mediated defenses such as ECSO branch on image presence rather than text content, making them blind to the dominant class of encoded jailbreaks when submitted as text-only.

Source

• arXiv: 2608.01043
• PDF: https://arxiv.org/pdf/2608.01043
• Categories: cs.CR, cs.AI
• Authors: Haoyu Zhang, Xiangchen Guan, Shibo Zheng, Mohammad Zandsalimy, Shanu Sushmita

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 3
• triage reason: Demonstrates concrete jailbreak failure modes on five frontier VLMs: baseline encoded attacks with measurable success rates, plus a specific bypass technique (pairing jailbreaks with decoy images) that reduces defense effectiveness by up to 73 percentage points. Includes adaptive attacks targeting the defense mechanism.

─────────────────────────────────────────────────────────────────────