Adversarial training

The harder reading is that adversarial training may trade one weakness for another. It can overfit to generated attacks, amplify difficult or high-gradient examples, become unstable under optimization changes, and reduce robust generalization when data are scarce. These findings do not invalidate the method, but they challenge claims that adversarial training reliably creates broadly secure models.

• With limited training data, adversarial training can produce higher robust error than standard training, reversing its expected benefit.
• Adversarially trained models exhibit more severe overfitting than models trained only on clean inputs, particularly around hard adversarial instances.
• Learning-rate decay can destabilize the trainer–attacker balance and encourage memorization of non-robust features, producing robust overfitting.
• The method can disproportionately amplify the influence of examples with large gradient norms and strengthen parameter-curvature effects, making optimization harder.
• Robustness measured against one attack family may not translate into reliable performance against other perturbations or perceptible corruptions.

Featured sourceTowards Deep Learning Models Resistant to Adversarial Attacksarxiv.org6 sourcesView sources