Beyond Rebalancing: Benchmarking Binary Classifiers Under Class Imbalance Without Rebalancing Techniques

This study evaluates the performance of various binary classifiers under class imbalance without employing rebalancing techniques, focusing on both real-world and synthetic datasets. It systematically assesses the robustness of classifiers, including advanced models like TabPFN and boosting-based ensembles, revealing that while traditional classifiers struggle with extreme imbalance, these advanced models maintain better performance and generalization. The findings provide critical insights for practitioners in model selection and highlight the importance of understanding classifier behavior in imbalanced scenarios.