An Improved Auto-encoder Based on 2-Level Prioritized Experience Replay for High Dimension Skewed Data

Auto-encoder as the representative method for data dimensionality reduction and feature extraction, plays a very important role on machine learning. However, the data in the actual research work or industrial production are not always normalized data, at this time, it will lead high reconstruction error and slow convergence speed. This study proposed an improved auto-encoder and a denoising auto-encoder based on 2-level prioritized experience replay, which can improve accuracy and reduce loss, while processing a dimensionality reduction or feature extraction problem on high dimension skewed data. In order to evaluate the effectiveness of the proposed method, three models of high dimension simulation dataset which on different skewed degrees are generated. The results of evaluation experiments show that the proposed method can get lower reconstruction error than conventional method for high dimension skewed simulation data.

[1]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[2]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[3]  Raghu Machiraju,et al.  Reconstruction Error Characterization and Control: A Sampling Theory Approach , 1996, IEEE Trans. Vis. Comput. Graph..

[4]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[5]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[6]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[7]  I. Jolliffe.  Principal Component Analysis , 2002 .

[8]  Mario Szegedy,et al.  The DLT priority sampling is essentially optimal , 2006, STOC '06.

[9]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[10]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[11]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[12]  J. Venkata Ramana Reddy,et al.  Numerical simulation of ship stability for dynamic environment , 2003 .

[13]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[14]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[15]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.

[16]  François Chollet,et al.  Keras: The Python Deep Learning library , 2018 .

[17]  Zhuo-shang Ji,et al.  The Optimization of the Hull Form with the Minimum Wave Making Resistance Based on Rankine Source Method , 2009 .