Investigated
What metrics measure backdoor success?
Short answer
Image: lesswrong.com
Backdoor success is usually measured by Attack Success Rate (ASR): how often a triggered input produces the attacker’s target output, alongside clean accuracy or utility. Stronger evaluation also compares poisoned and clean models, tests trigger stealthiness and semantic preservation, and checks whether outputs remain harmful or merely differ in wording. For removal, ASR alone can overstate success because it may treat refusal or harmless wording changes as genuine cures; richer assessments evaluate output quality on both triggered and ordinary inputs.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Broader Behavior
Source weight 29%Lens adapted to this topic: Beyond exact target outputs
A dissenting methodological view argues that conventional ASR can misclassify backdoor removal as successful when a model changes the exact target wording, refuses, or produces another problematic response. It favors evaluating output categories and usefulness on both triggered and non-triggered inputs, making behavioral persistence and normal utility explicit.
- Exact-match ASR can call a backdoor removed when the model still exhibits related behavior using different wording.
- A richer evaluation can classify triggered and ordinary outputs as Good, Strange, Wrong, or Refused.
- Backdoor-removal success should account for whether the model remains useful on normal, non-triggered inputs.
- Reported ASR-based removal rates may fall sharply under broader behavioral evaluation, indicating that metric choice changes the conclusion.
Follow the white rabbit
Deeper threads worth pulling on next.
- How is Attack Success Rate defined?Different angle, same storyClarifies the calculation and limitations of ASR, including thresholds, dataset splits, and whether refusals count as failures.
- What metrics detect stealthy triggers?A different rabbit holeExplores evaluation methods for trigger invisibility like perceptual similarity, token-level rarity, and human detection studies.
- How do defenders measure mitigation quality?