How does adversarial training help generalize?

A serious dissenting account emphasizes that adversarial training does not automatically generalize. It changes the learning problem into a harder robust objective, can overfit adversarial examples, and may sacrifice clean accuracy. Benefits may appear with enough data or suitable regularization, but small samples, strong attacks, or excessive training can reverse them.

• In a high-dimensional linear setting, adversarial training can hurt robust generalization with small samples, while helping when enough data is available; similar behavior was observed for image attacks.
• Even when an ideal predictor could be both accurate and robust, finite-sample adversarial training can create a trade-off and reduce standard accuracy.
• Robust learning can require substantially more data than standard learning, an information-theoretic limitation not removable merely by changing the algorithm.
• Adversarial training can overfit hard adversarial instances, and stability analyses indicate robust test accuracy may decline as training continues or perturbation size increases.
• In some theoretical models, adding more data can initially widen the generalization gap between robust and standard models.

Featured source[2203.02006] Why adversarial training can hurt robust accuracyar5iv.labs.arxiv.org6 sourcesView sources