Solve for the Hyperparameter, Skip the Search: Kolmogorov-Optimal Scaling Laws for Spline Regression

The paper introduces KORE (Kolmogorov-optimal Order-aware Resolution Estimation), a method that eliminates the need for exhaustive hyperparameter tuning in spline regression by providing a closed-form solution for optimal resolution. By leveraging classical approximation theory and the PRESS identity, KORE achieves comparable accuracy to traditional methods while fitting approximately eight times fewer models across various datasets with up to 80 input dimensions. This approach is significant for practitioners as it reduces computational costs and time associated with hyperparameter search while maintaining high model performance.