FinTradeBench: A Financial Reasoning Benchmark for LLMs

FinTradeBench is a newly introduced benchmark designed for evaluating financial reasoning in Large Language Models (LLMs), comprising 1,400 questions related to NASDAQ-100 companies. It categorizes questions into fundamentals-focused, trading-signal-focused, and hybrid types that require cross-signal reasoning, employing a calibration-then-scaling framework for reliability. The evaluation of 14 LLMs revealed a significant performance gap, particularly showing that while retrieval-augmented prompting enhances reasoning for textual fundamentals, it offers limited advantages for trading-signal reasoning, underscoring critical areas for improvement in numerical and time-series reasoning capabilities of LLMs.