Dynamic Temporal Selectivity in Non-linear Time Series Forecasting: An End-to-End LSTM-Multiplicative Attention Architecture

Abstract

Complex non-linear time series data make it one of the most urgent problems in any type of industry, where models that can capture long-range, changing correlations are needed when conventional statistical techniques, such as ARIMA, are insufficient. Therefore, in order to overcome these constraints and attain a higher temporal selectivity and predictive capability, we present and justify the improved, integrated Long Short-Term Memory (LSTM) network enhanced with a Multiplicative Attention mechanism in place of the original. The scientific contribution of the work itself is two-fold: First, we show that this one, and the same deep learning architecture, offers better efficiency and accuracy rates than the multi-component hybrid models that are hard to use. Second, the Multiplicative Attention weighting is created as one of the most important scientific contributions to model interpretability, which gives us dynamically explainable information on which historical time-steps are the most important to each future prediction. This is addressed by dismantling traditional LSTMs' "black-box" characteristics. The model is strong and most effective as validated through experimental testing on a real-life benchmark market, S&P 500 volatility statistics. The LSTM-Multiplicative Attention model proposed provided the low error metrics of a Root Mean Squared Error (RMSE) of 5.75, Mean Absolute Error (MAE) of 4.62 and a Symmetric Mean Absolute Percentage Error (SMAPE) of 8.93. This result is a substantial improvement over the conventional ARIMA model (RMSE of 12.45%) and even the conventional LSTM (RMSE of 7.30%), making the model a highly sophisticated, flexible, and interpretable forecasting tool for nonlinear time series.