A kernel logistic neural network based on restricted Boltzmann machine

A multi-class classification technique which combines kernel logistic neural network (KLNN) and restricted Boltzmann machine (RBM), called KLNN-RBM, is designed. The principal component analysis (PCA) is applied to determine the dimension of the kernel function. The initial weights and thresholds of this model are obtained by RBM. Then, the maximum likelihood estimate with a ridge regularization term and a new stochastic gradient descent method with a scaling factor are used to optimize the parameters in order to realize the multi-class classification. Some numerical simulations illustrate the validity of the proposed method.

[1]  W. Vach,et al.  Neural networks and logistic regression: Part I , 1996 .

[2]  E Biganzoli,et al.  Feed forward neural networks for the analysis of censored survival data: a partial logistic regression approach. , 1998, Statistics in medicine.

[3]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[4]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[5]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[6]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[7]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[8]  Alain Giron,et al.  Kernel logistic PLS: A tool for supervised nonlinear dimensionality reduction and binary classification , 2007, Comput. Stat. Data Anal..

[9]  Paulo J. G. Lisboa,et al.  Partial Logistic Artificial Neural Network for Competing Risks Regularized With Automatic Relevance Determination , 2009, IEEE Transactions on Neural Networks.

[10]  Maher Maalouf,et al.  Kernel logistic regression using truncated Newton method , 2011, Comput. Manag. Sci..

[11]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  José Neves,et al.  Direct Kernel Perceptron (DKP): Ultra-fast kernel ELM-based classification with non-iterative closed-form weight calculation , 2014, Neural Networks.

[13]  C. L. Philip Chen,et al.  Predictive Deep Boltzmann Machine for Multiperiod Wind Speed Forecasting , 2015, IEEE Transactions on Sustainable Energy.

[14]  Divya Tomar,et al.  A comparison on multi-class classification methods based on least squares twin support vector machine , 2015, Knowl. Based Syst..

[15]  C. L. Philip Chen,et al.  Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning , 2015, IEEE Transactions on Fuzzy Systems.

[16]  Modjtaba Rouhani,et al.  Two fast and accurate heuristic RBF learning rules for data classification , 2016, Neural Networks.