Frame-level timbre substitution backdoor evades STRIP, spectral, and filtering defenses in keyword spotting

Auto-published from arXiv:2607.01702 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.73, flags: [no-prompt-excerpt])

Category

backdoor-attack

Model

ERes2Net, KWS-ViT, EAT-S, CAM++ (speech classification DNNs for keyword spotting)

Surface

Audio inference pipeline (keyword spotting system, e.g. smart-device voice command recognition)

Setup

An attacker poisons a fraction of the training set using the Timbre Leakage Attack (TLA): for each target sample, semantic vectors are extracted from clean speech via a self-supervised model, k-means clustering identifies substitutable frame segments, and those frames are replaced with averaged vectors from a chosen trigger speaker's utterance, then reconstructed via a vocoder. The result is a poisoned audio clip that preserves the original keyword semantics but embeds the trigger speaker's timbre at frame-level in deep feature space. As few as 250–350 poisoned samples (out of 65,000) in Google Speech Commands v2 are sufficient. No explicit prompt template exists; the attack template is the TLA pipeline applied to any speech segment.

Observed behavior

Models trained on the poisoned dataset achieve 98–99.67% Attack Success Rate (ASR) when presented with trigger-timbre inputs, while clean accuracy (CA) is maintained. Against standard defenses: STRIP shows 'similar and inseparable entropy distributions' for clean vs. poisoned samples; spectral signature anomaly scores for poisoned samples are only marginally higher than clean; trigger filtering (high/low-frequency removal, noise addition) leaves ASR 'almost unchanged'; fine-tuning leaves ASR above 45%.

Expected behavior

The model should classify audio by keyword content only. Defenses (STRIP entropy-based detection, spectral signature analysis, trigger filtering) should reliably flag or neutralize poisoned samples before or after training.

Reproducibility

medium

Threat model

A malicious contributor to a shared speech dataset (e.g., a crowd-sourced voice command corpus used to train smart-home or mobile keyword spotting models) injects TLA-poisoned samples. Any end-user device running the compromised model will misclassify commands from the attacker's chosen trigger voice — enabling unauthorized activation of voice-controlled systems (smart locks, IoT devices, virtual assistants) while remaining undetected by conventional backdoor defenses.

Novelty

TLA is the first frame-level timbre-injection backdoor for speech that embeds the trigger inside deep self-supervised feature space rather than the waveform surface, achieving perceptual naturalness and simultaneous evasion of entropy-based (STRIP), spectral, and waveform-filtering defenses.

Source

• arXiv: 2607.01702
• PDF: https://arxiv.org/pdf/2607.01702
• Categories: cs.CR, cs.AI, cs.SD
• Authors: Yueming Huang, Wenhan Yao, Fen Xiao, Xiarun Chen, Weiping Wen

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 3
• triage reason: Proposes a novel backdoor attack (Timbre Leakage Attack + meta-learning variant) against speech classifiers with experimental validation on keyword spotting tasks. Demonstrates reproducible model-level failures across multiple DNN configurations.

─────────────────────────────────────────────────────────────────────