An efficient Long Short-Term Memory model based on Laplacian Eigenmap in artificial neural networks

Abstract A new algorithm for data prediction based on the Laplacian Eigenmap (LE) is presented. We construct the Long Short-Term Memory model with the application of the LE in artificial neural networks. The new Long Short-Term Memory model based on Laplacian Eigenmap (LE-LSTM) reserves the characteristics of original data using the eigenvectors derived from the Laplacian matrix of the data matrix. LE-LSTM introduces the projection layer embedding data into a lower dimension space so that it improves the efficiency. With the implementation of LE, LE-LSTM provides higher accuracy and less running time on various simulated data sets with characteristics of multivariate, sequential, and time-series. In comparison with previously reported algorithms such as stochastic gradient descent and artificial neural network with three layers, LE-LSTM leads to many more successful runs and learns much faster. The algorithm provides a computationally efficient approach to most of the artificial neural network data sets.

[1]  J.J. Hopfield,et al.  Artificial neural networks , 1988, IEEE Circuits and Devices Magazine.

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  T. L. McCluskey,et al.  Predicting phishing websites based on self-structuring neural network , 2013, Neural Computing and Applications.

[4]  Mir Mohammad Ettefagh,et al.  A novel adaptive neural network integral sliding-mode control of a biped robot using bat algorithm , 2018 .

[5]  T. L. McCluskey,et al.  Intelligent rule-based phishing websites classification , 2014, IET Inf. Secur..

[6]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[7]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[8]  Björn W. Schuller,et al.  Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR , 2015, LVA/ICA.

[9]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[10]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[11]  Risto Miikkulainen,et al.  From Nodes to Networks: Evolving Recurrent Neural Networks , 2018, ArXiv.

[12]  Shuai Li,et al.  Dynamic Neural Networks for Kinematic Redundancy Resolution of Parallel Stewart Platforms , 2016, IEEE Transactions on Cybernetics.

[13]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[14]  Ashish Khanna,et al.  Boosted neural network ensemble classification for lung cancer disease diagnosis , 2019, Appl. Soft Comput..

[15]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[16]  Ming Zhou,et al.  Towards Machine Translation in Semantic Vector Space , 2015, ACM Trans. Asian Low Resour. Lang. Inf. Process..

[17]  Shuai Li,et al.  Zeroing Neural Dynamics for Control Design: Comprehensive Analysis on Stability, Robustness, and Convergence Speed , 2019, IEEE Transactions on Industrial Informatics.

[18]  Shuicheng Yan,et al.  Semantic Object Parsing with Graph LSTM , 2016, ECCV.

[19]  Shuai Li,et al.  Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations , 2018, Neural Networks.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Kun Zhang,et al.  Forecasting skewed biased stochastic ozone days: analyses, solutions and beyond , 2008, Knowledge and Information Systems.

[22]  Donato Malerba,et al.  A Further Comparison of Simplification Methods for Decision-Tree Induction , 1995, AISTATS.

[23]  MengChu Zhou,et al.  Modified Primal-Dual Neural Networks for Motion Control of Redundant Manipulators With Dynamic Rejection of Harmonic Noises , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Soumadip Ghosh,et al.  Sentiment Analysis in the Light of LSTM Recurrent Neural Networks , 2018, Int. J. Synth. Emot..

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  K. P. Soman,et al.  Stock price prediction using LSTM, RNN and CNN-sliding window model , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[27]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[28]  Donato Malerba,et al.  Multistrategy Learning for Document Recognition , 1994, Appl. Artif. Intell..

[29]  Yongsheng Zhang,et al.  Performance Benefits of Robust Nonlinear Zeroing Neural Network for Finding Accurate Solution of Lyapunov Equation in Presence of Various Noises , 2019, IEEE Transactions on Industrial Informatics.

[30]  Prateek Jain,et al.  FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network , 2018, NeurIPS.

[31]  Pat Langley,et al.  Trading Off Simplicity and Coverage in Incremental concept Learning , 1988, ML.

[32]  R. J. Sassi,et al.  Application of a neuro fuzzy network in prediction of absenteeism at work , 2012, 7th Iberian Conference on Information Systems and Technologies (CISTI 2012).

[33]  Jia Liu,et al.  A time simulated annealing-back propagation algorithm and its application in disease prediction , 2018, Modern Physics Letters B.

[34]  Shuai Li,et al.  Robot manipulator control using neural networks: A survey , 2018, Neurocomputing.

[35]  Kun Zhang,et al.  Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions , 2006, Sixth International Conference on Data Mining (ICDM'06).

[36]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[37]  Boris Ginsburg,et al.  Factorization tricks for LSTM networks , 2017, ICLR.

[38]  N. Arunkumar,et al.  Convolutional neural network for bio-medical image segmentation with hardware acceleration , 2018, Cognitive Systems Research.

[39]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[40]  Shuai Li,et al.  Solving Time-Varying System of Nonlinear Equations by Finite-Time Recurrent Neural Networks With Application to Motion Tracking of Robot Manipulators , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[41]  Travis Desell,et al.  Optimizing Long Short-Term Memory Recurrent Neural Networks Using Ant Colony Optimization to Predict Turbine Engine Vibration , 2017, Appl. Soft Comput..

[42]  T. L. McCluskey,et al.  An assessment of features related to phishing websites using an automated technique , 2012, 2012 International Conference for Internet Technology and Secured Transactions.