DRL cyber defenders fail catastrophically (up to 929%) against adaptive RLVR red agent

Auto-published from arXiv:2608.04317 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.75, flags: [no-prompt-excerpt])

Category

agent-loop

Model

Qwen2.5-7B-Instruct (Planner, RLVR-trained); Qwen2.5-14B-Instruct-AWQ (Summarizer, frozen); Qwen2.5-Coder-7B-Instruct (Coder, frozen)

Surface

agent harness (CybORG CAGE 4 / CyberWheel simulation environments)

Setup

The Trident framework deploys a tripartite agentic pipeline against DRL blue-agent defenders (GNN, HMARL3, HMARL4, MARL in CAGE4; PPO(NN) in CyberWheel). The trainable Planner (Qwen2.5-7B-Instruct, fine-tuned with GRPO on 13,744 red-blue trajectories) receives compressed execution logs from the Summarizer and generates complete attack strategies; a frozen Coder translates these into executable Python policies that are deployed live against the defenders. Action knowledge base, strict API boundaries, and rigid output schemas are injected into system prompts. No literal prompt excerpt is published verbatim in the accessible paper sections.

Observed behavior

Trident reduced blue agent defensive performance by an average of 522.30% across all environments compared to static red agent baselines. Per-defender breakdown in CAGE4: GNN −929.79%, HMARL3 −540.78%, HMARL4 −138.26%, MARL −903.43%. CyberWheel PPO(NN): −99.22%. The zero-shot LLM baseline achieved only −157.38% and GPT-4o only −51.06%, demonstrating that RLVR training is the decisive factor.

Expected behavior

DRL defenders trained specifically for autonomous cyber defense should maintain robust defensive performance against novel red agents, including adaptive LLM-driven adversaries, not just the static heuristic red agents used during training.

Reproducibility

medium

Threat model

Operators deploying DRL-based autonomous intrusion-detection or cyber-defense agents (e.g., network SOC automation) are the affected party. An attacker with access to a similar simulation environment and open-weight 7B LLMs can train an adaptive red agent that systematically exploits the static policy brittleness of the deployed defender, effectively neutralizing the defense layer before launching a real intrusion.

Novelty

First demonstration that a single RLVR-trained 7B LLM planner, without any privileged access, can reliably break multiple distinct DRL cyber defenders by an order of magnitude more than static heuristic or zero-shot LLM baselines, quantifying the brittleness gap that prior work left unmeasured.

Source

• arXiv: 2608.04317
• PDF: https://arxiv.org/pdf/2608.04317
• Categories: cs.CR, cs.AI, cs.LG, cs.MA
• Authors: Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi, Sanggeon Yun, Hyunwoo Oh, SungHeon Jeong, Nathaniel D. Bastian, Mahdi Imani, Mohsen Imani

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 2
• triage reason: Demonstrates concrete, reproducible failures of DRL cyber defense agents against an adaptive LLM-based red-teaming framework. The paper characterizes agent-level brittleness (522% performance degradation vs. static baselines) and identifies specific emergent adversarial behaviors.

─────────────────────────────────────────────────────────────────────