Investigated
How do explanations affect user decisions?
Short answer
Image: frontiersin.org
Explanations do not reliably improve decisions: they can clarify advice and calibrate reliance, but can also shift attention toward misleading features or create an illusion of understanding. Their effects depend on users’ expertise, uncertainty, and the explanation’s design, so transparency is better understood as a context-sensitive intervention than a universal benefit.
The views on this question
Two lenses on the same evidence. Source weight and the primary source ratio show what each rests on.
Alternative
Source weight 50%Lens adapted to this topic: Evidence for explanation harms and limits
A skeptical reading emphasizes that explanations can persuade without genuinely improving understanding. They may create false confidence, distract users with unstable or misleading features, reduce acceptance of useful advice, or worsen performance; transparency therefore needs empirical validation rather than being assumed beneficial.
- The mere prospect of receiving an explanation can foster an illusion that users understand how an algorithm works, even without demonstrated comprehension.
- Feature-attribution explanations may offer only marginal utility and can produce worse decisions because of cognitive and contextual confounders.
- Providing alternative advice or prediction-confidence scores may hinder advice acceptance, particularly depending on users’ epistemic uncertainty.
- Explanations can reshape users’ mental models and information weighting, with resulting adjustments subject to contextual limitations rather than guaranteed accuracy.
Follow the white rabbit
Deeper threads worth pulling on next.
- When do explanations mislead people?Different angle, same storyExamine psychological mechanisms like attention shifts and illusion of understanding that make explanations harmful.
- How should AI show uncertainty?Different angle, same storySurvey ways to present confidence and gaps so users calibrate reliance without overconfidence.
- What are explanation harms lawsuits?