Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic

The article introduces Prime Fourier Embeddings (PFE), a new method for encoding integers using prime-indexed (cos, sin) pairs derived from harmonic analysis, enhancing the representation of algebraic structures in neural networks. It establishes that linear maps equivariant to the product group action on PFE must be block-diagonal, with each block corresponding to a prime, and demonstrates through ablation studies that task-relevant channels show specialization ratios exceeding 500x, achieving perfect accuracy across square-free composite moduli. This approach offers a principled framework for modular arithmetic in AI applications, potentially improving model performance in tasks involving integer representations.