Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

The paper presents a novel approach to zero-shot test-time canonicalization that addresses the misclassification of inputs transformed by affine operations in pretrained vision models. By reframing canonicalization as out-of-distribution (OOD) detection, the authors explore various OOD scoring functions and optimization algorithms, finding that distance-based scores combined with random search and local refinement yield the best performance across diverse benchmarks. This method allows practitioners to improve model robustness without altering the classifier architecture or retraining, thus preserving in-distribution accuracy while enhancing performance on transformed inputs.