What are circuit-level failure modes?

Lens adapted to this topic: Neural-network circuits and explanation failure

In mechanistic interpretability, “circuit-level failure” is not usually a hardware fault. It refers to an explanation failing to be faithful, localized, minimal, or causally adequate: a selected subgraph may reproduce behavior only under a particular ablation method, omit distributed computation, or correlate with rather than implement the behavior. This is an active methodological debate rather than a settled taxonomy.

• Circuit faithfulness scores can change substantially with seemingly minor choices in ablation methodology, so scores may reflect experimental design as well as circuit components.
• Tests of six proposed LLM circuits found that discovered Transformer circuits satisfy criteria such as behavior preservation, localization, and minimality to varying degrees.
• The interpretability literature treats failure of an extracted circuit to capture the model’s actual computation as a central evaluation problem.
• A proposed critique of weight-sparse circuit extraction argues that circuits can appear interpretable while remaining unfaithful to the model’s underlying computation.
• Alternative views question whether circuit-based explanations are the right ontology for understanding neural-network computation, including concerns about representational drift and universality.

Featured sourceHypothesis Testing the Circuit Hypothesis in LLMsarxiv.org4 sourcesView sources