The Importance of Phase in Neural Representations: An Internal Oppenheim-Lim Test of Image Classifiers

The study investigates the role of phase in image classifiers' neural representations, building on the Oppenheim-Lim test, which demonstrated that natural images remain recognizable when reconstructed from Fourier phase alone. The authors evaluate models including PRISM2D, GFNet, ViT-B/16, and ResNet-50, finding that predictions are predominantly influenced by phase rather than magnitude, indicating that identity is encoded in phase while magnitude is largely dispensable. This research provides insights into the architectural differences in how CNNs and attention-based models utilize phase and magnitude, which is critical for practitioners aiming to enhance classifier robustness and interpretability.