What is adversarial explanation testing?

Lens adapted to this topic: Broader adversarial uses and interpretations

A broader reading treats adversarial explanation testing not only as stress-testing explanation stability, but also as examining how explanations relate to adversarial examples, counterfactuals, model attacks, and human trust. On this view, an explanation may be vulnerable because it is technically unstable, because it can be used to generate an attack, or because persuasive framing can miscalibrate users even when the prediction is wrong.

• Some work treats adversarial examples themselves as possible explanations, particularly when robust models make the relationship between perturbations and decisions informative.
• Explanations and adversarial examples can be mathematically related, with methods proposed to derive one from the other.
• Counterfactual explanations and adversarial examples can share generation methods, while differing in their relation to labels and acceptable proximity.
• Adversarial manipulation can target the explanation channel and influence human trust in an incorrect AI output, not only the model’s prediction.

Featured sourceThe Intriguing Relation Between Counterfactual Explanations and Adversarial Examples | Minds and Machines | Springer Nature Linklink.springer.com5 sourcesView sources