CITRAS: Covariate-Informed Transformer for Time Series Forecasting

CITRAS, a decoder-only Transformer model for time series forecasting, has been introduced to effectively integrate observed and known covariates, addressing the common challenges in leveraging these variables due to length discrepancies. It features two novel mechanisms: Key-Value (KV) Shift, which aligns future known covariates with target variables, and Attention Score Smoothing, which enhances local dependencies into global variate-level dependencies. Experimental results indicate that CITRAS significantly improves forecasting accuracy across diverse real-world datasets, making it a valuable tool for practitioners aiming to enhance model performance in time series analysis.