Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count

The article introduces Communication Dynamics Neural Networks (CDNNs) and presents CDLinear, a block-circulant linear layer that reduces parameter count to 1/B compared to dense layers while maintaining performance. CDLinear achieves 97.50% test accuracy on the 8x8 MNIST benchmark using only 2,380 parameters, significantly fewer than the 8,970 parameters of a dense layer, with a mean Hessian condition number of 1.9e4, vastly improved over the dense baseline's 5.9e6. This work provides a new approach to layer design that enhances optimization diagnostics and conditioning, which is crucial for practitioners aiming to build efficient neural networks with reduced computational overhead.