Training Process Reduction Based On Potential Weights Linear Analysis To Accelarate Back Propagation Network

Learning is the important property of Back Propagation Network (BPN) and finding the suitable weights and thresholds during training in order to improve training time as well as achieve high accuracy. Currently, data pre-processing such as dimension reduction input values and pre-training are the contributing factors in developing efficient techniques for reducing training time with high accuracy and initialization of the weights is the important issue which is random and creates paradox, and leads to low accuracy with high training time. One good data preprocessing technique for accelerating BPN classification is dimension reduction technique but it has problem of missing data. In this paper, we study current pre-training techniques and new preprocessing technique called Potential Weight Linear Analysis (PWLA) which combines normalization, dimension reduction input values and pre-training. In PWLA, the first data preprocessing is performed for generating normalized input values and then applying them by pre-training technique in order to obtain the potential weights. After these phases, dimension of input values matrix will be reduced by using real potential weights. For experiment results XOR problem and three datasets, which are SPECT Heart, SPECTF Heart and Liver disorders (BUPA) will be evaluated. Our results, however, will show that the new technique of PWLA will change BPN to new Supervised Multi Layer Feed Forward Neural Network (SMFFNN) model with high accuracy in one epoch without training cycle. Also PWLA will be able to have power of non linear supervised and unsupervised dimension reduction property for applying by other supervised multi layer feed forward neural network model in future work.

[1]  Siti Mariyam Shamsuddin.  Higher Order Centralised Scale-Invariants for Unconstrained Isolated Handwritten Digits , 2000 .

[2]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[3]  Boris Kovalerchuk,et al.  Neural networks for data mining: constrains and open problems , 2004, ESANN.

[4]  Garrison W. Cottrell,et al.  Non-Linear Dimensionality Reduction , 1992, NIPS.

[5]  Jieping Ye,et al.  An optimization criterion for generalized discriminant analysis on undersampled problems , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Markus Hegland,et al.  Data Mining - Challenges, Models, Methods and Algorithms , 2003 .

[7]  Byung-Woo Min,et al.  Neural networks using modified initial connection strengths by the importance of feature elements , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[8]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[9]  Andreas Daffertshofer,et al.  PCA in studying coordination and variability: a tutorial. , 2004, Clinical biomechanics.

[10]  Sandro Ridella,et al.  Statistically controlled activation weight initialization (SCAWI) , 1992, IEEE Trans. Neural Networks.

[11]  Ken Yim Tsan.  An Improvement on Extended Kalman Filter for Neural Network Training , 2005 .

[12]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[13]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[14]  Haesun Park,et al.  Structure Preserving Dimension Reduction for Clustered Text Data Based on the Generalized Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[15]  Lindsay I. Smith,et al.  A tutorial on Principal Components Analysis , 2002 .

[16]  Jude W. Shavlik,et al.  Using neural networks for data mining , 1997, Future Gener. Comput. Syst..

[17]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[18]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[19]  Nematollaah Shiri,et al.  New supervised multi layer feed forward neural network model to accelarate classification with high accuracy. , 2009 .

[20]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[21]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[22]  Hui Xiong,et al.  IDR/QR: an incremental dimension reduction algorithm via QR decomposition , 2004, IEEE Transactions on Knowledge and Data Engineering.

[23]  Mercedes Fernández-Redondo,et al.  A comparison among weight initialization methods for multilayer feedforward networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[24]  Mercedes Fernández-Redondo,et al.  Weight initialization methods for multilayer feedforward , 2001, ESANN.

[25]  Nirwan Ansari,et al.  Mini-Max Initialization for Function Approximation , 2004, Neurocomputing.

[26]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[27]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[28]  Kenji Nakayama,et al.  A training scheme for pattern classification using multi-layer feed-forward neural networks , 1999, Proceedings Third International Conference on Computational Intelligence and Multimedia Applications. ICCIMA'99 (Cat. No.PR00300).

[29]  H. T. Kung,et al.  Matrix Triangularization By Systolic Arrays , 1982, Optics & Photonics.