TruthSeekers

Rabbit hole · 3 connected questions

How do choices and adaptivity in empirical attacks versus formal certification create a persistent evaluation gap in claimed model robustness?

How these converge

All three topics are about how we judge whether a model is robust to adversarial perturbations. PGD is the common, practical baseline for generating attacks but is highly sensitive to choices (loss, norm, step size, initialization). Adaptive PGD variants deliberately change those choices to find much stronger failures, showing that a non-adaptive PGD can give a misleading picture of robustness. Certified defenses sit on the other side: they provide attack-independent guarantees but make modeling, scalability, or tightness tradeoffs. Together they point to a single concrete problem: empirical robustness claims depend on which attacks and hyperparameters were tried, while certificates avoid that fragility at the cost of restricted threat models or loose bounds—so evaluating defenses requires understanding both adaptive attack capability and certification limits.

Where these converge

The chain

  1. Starting point

    PGD attacks

    Projected Gradient Descent (PGD) iteratively changes an input in a loss-increasing direction, projecting each step back into a permitted perturbation region such as an ℓp ball and the valid input domain. PGD is widely used to generate adversarial examples and to evaluate or improve neural-network robustness; randomized initialization corresponds to the Madry et al. method in the cited implementation. Its results depend strongly on the loss, norm, projection, initialization, iteration budget, and constraints. Research reports stronger results from alternating objectives, adaptive methods, and more exact constraint handling. The main disagreement is whether standard PGD is a sufficiently strong and representative robustness test: some work treats it as a reliable first-order baseline, while other work finds important cases where it overestimates robustness or can be improved substantially.

    9 sources

  2. Adaptive PGD variants

    Adaptive PGD variants modify step sizes, projections, objectives, constraints, or norm selection to find stronger adversarial examples. Evidence suggests these changes can substantially improve attack effectiveness, but results remain dependent on the threat model, data type, defense, and evaluation setup.

    10 sources

  3. Certified defenses

    Certified defenses aim to replace best-effort robustness with guarantees: for a specified model, input, and perturbation set, a certificate can establish that the prediction cannot change. Exact methods seek definitive answers, while relaxed methods may return “maybe.” Research demonstrates certificates for several threat models, including norm-bounded perturbations, differential-privacy-based robustness, and localized patch attacks, with reported results on datasets ranging from MNIST to ImageNet. The central disagreement is whether these guarantees are sufficiently accurate, broad, scalable, and operationally useful for real deployments: proponents emphasize rigorous, attack-independent guarantees, while critics report generalization costs, loose or incomplete certification, and attacks that exploit system behavior around certification.

    10 sources

Keep going: open any topic above to find its own related questions.