ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
ICON decomposition quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that are validated by retraining and out-of-distribution testing.
Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning. Concept-based explainability methods screen for shortcuts by testing whether concepts such as a patient's sex or scanner settings can be decoded from a network layer. Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them. ICON decomposition instead quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing.
ICON decomposition is a multivariate concept-level explanation method that accounts for correlations between concepts and the outcome when attributing variance in a deep representation. It outperforms seven baseline methods on synthetic data with known ground truth, indicating improved accuracy in concept importance recovery. The method also quantifies unexplained representation variance, providing a measure of how much of a layer's variance is not captured by the supplied concepts. Sparse explanations are validated through retraining and out-of-distribution testing, suggesting that the identified concepts are causally relevant to model behavior.
This research addresses a critical need in high-stakes domains such as medical imaging, where shortcut learning can lead to biased or unreliable models. By providing more accurate concept-level explanations, ICON decomposition could support model auditing and regulatory compliance. The method's ability to quantify unexplained variance may help developers identify gaps in their conceptual understanding of model behavior, guiding further data collection or model refinement.
For organizations deploying deep learning models in regulated or safety-critical areas, ICON decomposition offers a more reliable way to audit models for shortcut learning. This can reduce risk of biased decisions, support compliance with explainability requirements, and potentially lower the cost of model validation. The method's sparse explanations may also help in communicating model behavior to non-technical stakeholders.
Potential next signals include adoption of ICON decomposition in model auditing pipelines, especially in healthcare AI. Further validation on diverse datasets and model architectures could establish its generalizability. The method may inspire extensions to other modalities or integration with existing explainability toolkits. Watch for follow-up work that applies ICON to real-world deployment scenarios or compares it with causal explanation methods.