Controlled Dynamics Attractor Transformer

The Controlled Dynamics Attractor Transformer (CDAT) introduces a novel architecture that integrates a mixture von Mises-Fisher attention energy with Hopfield refinement energy, enhancing the biological plausibility of continuous-time inference in deep learning models. By employing a CANN-inspired excitation-inhibition modulation and a topology-constrained dynamical system, CDAT achieves state-of-the-art performance in graph anomaly detection and classification benchmarks. This advancement is significant for practitioners as it offers a more interpretable and robust framework for managing relational structures in token representations, enhancing the efficacy of transformer models in complex tasks.