Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling

The article introduces Topological Neural Dynamics (TND), a novel sequence modeling framework that enables independent evolution of neurons through a directed neuron graph, enhancing local interactions and structured connectivity. TND, evaluated on a behavior cloning task in single-player Pong, outperforms several baselines, including Vanilla RNN and Transformers, achieving a mean of 17.47 consecutive catches per round, indicating that neuron-wise dynamics can significantly improve performance in sequence tasks. This approach may offer practitioners a new inductive bias for building more effective models in AI applications.