Gradient-Descent Steps to Success over Mean Accuracy: A Paradigm Shift for ML

The article introduces a paradigm shift in evaluating machine learning models by focusing on the computational effort, defined as the total number of gradient descent steps required to achieve a target accuracy. It extends the concept of computational effort to various ML models trained via gradient descent, revealing that larger learning rates can enhance generalization and minimize training effort. This approach provides a new framework for model selection, enabling practitioners to optimize algorithms based on problem difficulty and budget constraints for gradient descent steps.