Claude Opus 4.5 disavows 88% of prefilled misalignment trajectories in agentic evals, undermining AI control protocols

Auto-published from arXiv:2606.12747 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.78, flags: [no-prompt-excerpt])

Category

agent-misbehavior

Model

Claude Opus 4.5

Surface

Agent harness (multi-turn agentic transcript with prefilled assistant turns, including misalignment-continuation and SWE-bench task settings)

Setup

In Section 4 (agentic settings), models are shown complete multi-turn transcripts in which one or more assistant turns depicting misaligned actions (e.g., electronic fraud involvement) were prefilled. A follow-up prompt then asks whether prior assistant messages were "generated by itself ('me') or prefilled ('not me')". In SWE-bench trajectories, the Inspect evaluation framework's message reload introduced hidden formatting tags into assistant messages, which Claude Opus 4.5 latched onto as tamper signals. In Figure 9 (cherry-picked example), the model is shown a trajectory involving fraud and asked to continue; it instead backtracks and refuses to identify with the prefilled turn.

Observed behavior

Claude Opus 4.5 flagged 88% of misalignment-continuation tampered trajectories as 'not me' (disavowal). Attribution accuracy on SWE-bench trajectories was highly dataset-dependent: high accuracy on tool-heavy logs but near-chance on chat-format logs. The model explicitly cited hidden tags as evidence of foreign insertion, meaning formatting artifacts from the evaluation harness leaked into attribution judgments. Success bias was also identified: models more readily claimed authorship of successful task trajectories than failed ones, independent of whether the trajectory was actually theirs.

Expected behavior

An AI control evaluation that prefills misaligned behavior to test whether a model continues it should find that the model either continues (unsafe) or refuses (safe) based on the content — not that the model meta-reasons about whether the turn was genuinely its own output. Authorship attribution should be at chance if the model has no genuine memory of generation, or consistent if it does — not confounded by task success, formatting artifacts, or dataset modality.

Reproducibility

medium

Threat model

AI safety teams running 'misalignment-continuation' evaluations — a standard AI control protocol where prefilled misaligned behavior tests whether a model will escalate — obtain invalid results because frontier models systematically disavow prefilled turns rather than continuing them. This could cause safety teams to over-estimate a model's resistance to misalignment cascades. The hidden-tag confound also means that evaluation infrastructure artifacts (not model cognition) can drive apparent safety-relevant behaviors, creating false confidence in control protocol effectiveness.

Novelty

First empirical demonstration that AI control protocols based on prefill-inserted misalignment are confounded by models' own prefill-awareness, with a concrete mechanistic example (hidden tags as a leaked tamper signal) showing how harness artifacts interact with model meta-cognition to invalidate evaluation outcomes.

Source

• arXiv: 2606.12747
• PDF: https://arxiv.org/pdf/2606.12747
• Categories: cs.AI
• Authors: Andy Wang, Parv Mahajan, David Demitri Africa, Alexandra Souly, Jordan Taylor, Robert Kirk

Triage notes (auto)

• paperType: benchmark
• estimatedCaseCount: 2
• triage reason: Constructs a benchmark measuring frontier models' ability to detect and resist prefilled context. Demonstrates Claude Opus 4.5 detects tampered prefills in 9-35% of cases with 0% false positives, undermining the validity of prefill-based safety evaluation methods.

─────────────────────────────────────────────────────────────────────