From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?

The article presents a study on the effectiveness of interpretability methods in producing disentangled representations of latent concepts in neural networks, specifically using sparse autoencoders (SAEs) and probes. It introduces a multi-concept evaluation framework that assesses how well these methods can isolate features related to sentiment, domain, voice, and tense, revealing that features are often sensitive to multiple concepts and that interactions can occur despite the lack of direct correlation. This research highlights the necessity for more comprehensive evaluation methods in interpretability to ensure that features can be independently manipulated, which is crucial for practitioners aiming to build reliable AI systems with clear and interpretable decision-making processes.