How do saliency maps fail?

Lens adapted to this topic: Why some failure tests may overstate the problem

A serious methodological dissent argues that widely cited sanity checks do not by themselves prove that saliency methods are invalid. Their evaluation tasks can introduce confounding, so apparent insensitivity to model parameters may partly reflect the test design. This view still accepts that evaluation beyond visual inspection is difficult, but urges caution before generalizing benchmark failures to every use of saliency maps.

• The sanity-check methodology may not fully establish that model-insensitive maps are unsuitable, because confounding in the evaluation tasks can affect the conclusions.
• Experiments on custom tasks suggest that some conclusions from earlier sanity checks may be artifacts of the tasks rather than properties of saliency methods themselves.
• The absence of ground-truth attribution maps makes it difficult to determine which saliency method is correct or to compare methods conclusively.
• The strongest defensible conclusion may be that saliency-map evaluation is context-dependent, not that every saliency method fails for every explanatory purpose.

Featured sourcearxiv.orgarxiv.org2 sourcesView sources