Robustness vs accuracy

Lens adapted to this topic: Evidence for a conditional trade-off

The mainstream account treats the robustness–accuracy tension as a substantial, sometimes theoretically grounded limitation of standard adversarial training. Robust optimization can force models toward smoother or different representations and can raise clean error, although the severity depends on the data and learning setup. This view generally treats robustness and accuracy as objectives that must be balanced rather than assuming the trade-off is universal.

• A theoretical analysis argues that adversarial robustness and standard generalization can be inherently in tension in a simple natural setting, with robust models learning different feature representations.
• TRADES formalizes robust error as natural error plus boundary error and presents robustness and accuracy as quantities to balance in defense design.
• In linear regression, adversarial augmentation can increase standard error even when perturbation labels are noiseless; the result also identifies settings where robust self-training avoids that loss.
• A 2024 analysis derives a fundamental trade-off when no nearly optimal predictor is sufficiently smooth, while identifying conditions under which degradation need not be large.
• In robot-learning experiments and applications, adversarial training has been reported to reduce overall accuracy or holistic task performance, making the practical trade-off consequential.

Lens adapted to this topic: Arguments that both goals can be achieved

The dissenting research argues that the apparent trade-off is not inherent to robustness itself. It attributes many losses to overly rigid invariance assumptions, inappropriate perturbation radii, insufficient generalization, or training methods that do not match the data-generating process. These approaches seek locally adaptive or equivariant notions of robustness and report theoretical or empirical routes to retaining clean accuracy.

• A formal analysis argues that robustness should be locally adaptive to the data-generating process; its proposed adaptive robust augmentation is designed to maintain consistency rather than impose a fixed global perturbation rule.
• Work on separated real-image datasets proves that both robustness and accuracy can be achievable with locally Lipschitz functions, attributing practical gaps to limitations of current methods and generalization.
• SCORE argues that the conventional robust error can impose an over-smoothing bias; defining robustness with local equivariance can reconcile robust and accurate prediction.
• Robust self-training is presented as a method that can improve robust error without sacrificing standard error in noiseless linear regression, with favorable neural-network experiments reported by its authors.
• The alternative position does not deny that many current systems show a trade-off; it disputes treating those observations as proof of a universal, method-independent impossibility result.