Task-agnostic random objects bypass Gemini Robotics-ER 2 safety refusals at nearly the same rate as task-optimized objects

Auto-published from arXiv:2609.38971 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.92, flags: [possible-zero-day]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

alignment

Model

Gemini Robotics-ER 2

Surface

VLM input (embodied robot planner, API)

Setup

846 tasks from EARBench that Gemini Robotics-ER 2 initially refuses under a 63-rule safety constitution are evaluated under three object-insertion strategies: (1) targeted—20 task-specific objects per task; (2) scene-only—objects matching only the scene context; (3) random—a fixed 26-item list (laptop, orange, banana, apple, book, cup, chair, etc., from Appendix D) drawn without any knowledge of the task or environment. Objects are inserted via FLUX.2-klein. Baseline: re-querying unchanged images. In the Figure 1 corridor example, both a task-specific 'stack of books' and a random 'orange' from the fixed list are shown flipping the same refusal.

Reproduction anchor

code: https://github.com/mdctleo/refusal-bend; benchmark: EARBench (Zhu et al., 2024a), 846 refused tasks from 1,121 photorealistic images; random object list: Appendix D (26 items); LatentHarm dataset (281 tasks) released with code

Observed behavior

Random objects achieved an 18.97% task-level bypass rate; task-optimized objects achieved 20.69%—a marginal difference. Baseline re-query of unchanged images bypassed at only 0.24%. A random 'orange' and a targeted 'stack of books' both independently bypassed the same corridor task refusal.

Expected behavior

Task-agnostic random objects should have negligible effect on safety refusals, since they carry no semantic relationship to the hazard being refused; bypass rate should be near the 0.24% re-query baseline.

Reproducibility

high

Threat model

An attacker with physical access to a robot's operating environment needs zero knowledge of the robot's pending task or instruction to bypass safety—any common household object suffices. This eliminates the knowledge barrier for physical-world safety bypass, making attacks available to unsophisticated or opportunistic adversaries in shared spaces.

Novelty

Shows that alignment bypass in embodied VLM planners does not require adversarial optimization or task-specific knowledge—random environmental perturbation is nearly as effective—implying robustness evaluation must account for uncontrolled, non-adversarial environments, not only targeted attacks.

Source

• arXiv: 2609.38971
• PDF: https://arxiv.org/pdf/2609.38971
• Categories: cs.CR, cs.RO
• Authors: Leo Y. Lin, Mikhail Kuznetsov, Muslum Ozgur Ozmen, Z. Berkay Celik

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 3
• triage reason: Systematic red-teaming of embodied VLM planners: demonstrates that 20.2% of safety-refused tasks can be bypassed by adding ordinary objects to the environment, with reproducible setup and predictive model identifying vulnerable tasks. No disclosed vendor coordination.

─────────────────────────────────────────────────────────────────────