Leakage-Aware Benchmarking of LLM Forecasting: Real-Time Nowcasts as the Decision-Time Input for Macro Factor Ranking

The paper presents a leakage-aware benchmarking methodology for forecasting using a 7B parameter open-source retrieval-augmented LLM. It emphasizes decision-time constraints by utilizing only observable macroeconomic variables and a critic-actor architecture to rank equity factors, achieving a median Spearman rank IC of +0.154. This approach is significant for practitioners as it highlights the importance of avoiding information leakage in LLM forecasting and demonstrates the potential of combining LLMs with macroeconomic data for improved financial decision-making.