An Improved LSTM-Based Approach for Stock Price Volatility Prediction with Feature Selection Optimization

Caiqian Cheng; Chenyu Li; Guifan Weng

doi:10.69987/

Authors

Caiqian Cheng Computer Science, University of California, San Diego, CA, USA Author
Chenyu Li Applied Analytics, Columbia University, NY, USA Author
Guifan Weng Computer Science, University of Southern California, CA, USA Author

DOI:

https://doi.org/10.69987/

Keywords:

LSTM neural networks, feature selection optimization, stock price prediction, volatility forecasting

Abstract

Stock price volatility prediction remains a challenging task in financial markets due to the complex, non-linear, and dynamic nature of market data. This paper presents an enhanced Long Short-Term Memory (LSTM) neural network approach integrated with an optimized feature selection framework for improved stock price volatility prediction. The proposed methodology combines advanced technical indicator construction with a novel two-stage feature selection algorithm that utilizes mutual information and recursive feature elimination techniques. The improved LSTM architecture incorporates attention mechanisms and dropout regularization to enhance predictive performance while mitigating overfitting. Experimental validation on multiple stock datasets demonstrates that our approach achieves superior prediction accuracy compared to traditional forecasting methods and baseline LSTM models. The results show an average improvement of 12.3% in Mean Absolute Percentage Error (MAPE) and 15.7% in Root Mean Square Error (RMSE) over conventional approaches. The proposed framework provides valuable insights for algorithmic trading and risk management applications in financial markets.