An optimization methodology for machine learning strategies and regression problems in ballistic impact scenarios

In domains with limited data, such as ballistic impact, prior researches have proven that the optimization of artificial neural models is an efficient tool for improving the performance of a classifier based on MultiLayer Perceptron. In addition, this research aims to explore, in the ballistic domain, the optimization of other machine learning strategies and their application in regression problems. Therefore, this paper presents an optimization methodology to use with several approaches of machine learning in regression problems, maximizing the limited dataset and locating the best network topology and input vector of each network model. This methodology is tested in real regression scenarios of ballistic impact with different artificial neural models, obtaining substantial improvement in all the experiments. Furthermore, the quality stage, based on criteria of information theory, enables the determination of when the complexity of the network design does not penalize the fit over the data and thereby the selection of the best neural network model from a series of candidates. Finally, the results obtained show the relevance of this methodology and its application improves the performance and efficiency of multiple machine learning strategies in regression scenarios.

[1]  David West,et al.  Neural network credit scoring models , 2000, Comput. Oper. Res..

[2]  Christian Döschner,et al.  Modelling of Complete Robot Dynamics Based on a Multi-Dimensional, RBF-like Neural Architecture , 2004, Applied Intelligence.

[3]  Wlodzislaw Duch,et al.  Learning Highly Non-separable Boolean Functions Using Constructive Feedforward Neural Network , 2007, ICANN.

[4]  Laura Diosan,et al.  Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters , 2010, Applied Intelligence.

[5]  Sung-Bae Cho,et al.  Evolutionary Learning of Modular Neural Networks with Genetic Programming , 1998, Applied Intelligence.

[6]  Asis Mazumdar,et al.  Optimization Of The Water Use In The River Damodar In West Bengal In India: An Integrated Multi-Reservoir System With The Help Of Artificial Neural Network , 2007 .

[7]  Riccardo Poli,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 2004, Applied Intelligence.

[8]  Dianhui Wang,et al.  Improved generalization of neural classifiers with enforced internal representation , 2007, Neurocomputing.

[9]  Petri Koistinen,et al.  Using additive noise in back-propagation training , 1992, IEEE Trans. Neural Networks.

[10]  Ragip Ince,et al.  Prediction of fracture parameters of concrete by Artificial Neural Networks , 2004 .

[11]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[12]  Michael I. Jordan.  Serial Order: A Parallel Distributed Processing Approach , 1997 .

[13]  Dimitris A. Pados,et al.  On overfitting, generalization, and randomly expanded training sets , 2000, IEEE Trans. Neural Networks Learn. Syst..

[14]  Kiyotoshi Matsuoka,et al.  Noise injection into inputs in back-propagation learning , 1992, IEEE Trans. Syst. Man Cybern..

[15]  David O'Sullivan,et al.  Collaborative Innovation for the Management of Information Technology Resources , 2010, Int. J. Hum. Cap. Inf. Technol. Prof..

[16]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[17]  Jonas A. Zukas,et al.  High velocity impact dynamics , 1990 .

[18]  Steffen Hölldobler,et al.  Approximating the Semantics of Logic Programs by Recurrent Neural Networks , 1999, Applied Intelligence.

[19]  Donald H. Foley.  Considerations of sample and feature size , 1972, IEEE Trans. Inf. Theory.

[20]  Petri Koistinen,et al.  Kernel regression and backpropagation training with noise , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[21]  Nicolas Durand,et al.  Neural Nets Trained by Genetic Algorithms for Collision Avoidance , 2000, Applied Intelligence.

[22]  Kevin Swingler,et al.  Applying neural networks - a practical guide , 1996 .

[23]  Sergio Ricci,et al.  Neural network systems to reproduce crash behavior of structural components , 2004 .

[24]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[25]  Robert Villa,et al.  Comparison of Feature Construction Methods for Video Relevance Prediction , 2009, MMM.

[26]  Martin Burger,et al.  Analysis of Tikhonov regularization for function approximation by neural networks , 2003, Neural Networks.

[27]  Khosrow Kaikhah,et al.  Variable Hidden Layer Sizing in Elman Recurrent Neuro-Evolution , 2004, Applied Intelligence.

[28]  Christopher M. Bishop,et al.  Current address: Microsoft Research, , 2022 .

[29]  D. Saha,et al.  A neural network based prediction model for flood in a disaster management system with sensor networks , 2005, Proceedings of 2005 International Conference on Intelligent Sensing and Information Processing, 2005..

[30]  D. Agard,et al.  Microtubule nucleation by γ-tubulin complexes , 2011, Nature Reviews Molecular Cell Biology.

[31]  Jin H. Huang,et al.  Detection of cracks using neural networks and computational mechanics , 2002 .

[32]  N. K. Roy,et al.  Designing Polymer Blends Using Neural Networks, Genetic Algorithms, and Markov Chains , 2000, Applied Intelligence.

[33]  José Antonio Gómez-Ruiz,et al.  A Neural Network Based Model for Prognosis of Early Breast Cancer , 2004, Applied Intelligence.

[34]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[35]  Walter D. Potter,et al.  Predicting Aflatoxin Contamination in Peanuts: A Genetic Algorithm/Neural Network Approach , 2004, Applied Intelligence.

[36]  William W. Hsieh,et al.  Skill Comparisons between Neural Networks and Canonical Correlation Analysis in Predicting the Equatorial Pacific Sea Surface Temperatures , 2000 .

[37]  Guozhong An,et al.  The Effects of Adding Noise During Backpropagation Training on a Generalization Performance , 1996, Neural Computation.

[38]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[39]  L. Breiman,et al.  Submodel selection and evaluation in regression. The X-random case , 1992 .

[40]  Li Zhang,et al.  An evaluation of back-propagation neural networks for the optimal design of structural systems: Part I. Training procedures , 2002 .

[41]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[42]  H. Akaike.  A new look at the statistical model identification , 1974 .

[43]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[44]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[45]  José Hernández-Orallo,et al.  An experimental comparison of performance measures for classification , 2009, Pattern Recognit. Lett..

[46]  Jose C. Principe,et al.  Neural and Adaptive Systems: Fundamentals through Simulations with CD-ROM , 1999 .

[47]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[48]  Xiao-cheng Shi,et al.  Fault Diagnosis of Nuclear Power Plant Based on Genetic-RBF Neural Network , 2008, 2008 15th International Conference on Mechatronics and Machine Vision in Practice.

[49]  Kevin L. Priddy,et al.  Artificial Neural Networks: An Introduction (SPIE Tutorial Texts in Optical Engineering, Vol. TT68) , 2005 .

[50]  Ángel García-Crespo,et al.  Prediction of the response under impact of steel armours using a multilayer perceptron , 2007, Neural Computing and Applications.

[51]  Giansalvo Cirrincione,et al.  A Novel Self-Organizing Neural Network for Motion Segmentation , 2004, Applied Intelligence.

[52]  Kevin L. Priddy,et al.  Artificial Neural Networks: An Introduction (SPIE Tutorial Texts in Optical Engineering, Vol. TT68) , 2005 .

[53]  Teuvo Kohonen,et al.  An introduction to neural computing , 1988, Neural Networks.

[54]  Chang-Xue Jack Feng,et al.  Threefold vs. fivefold cross validation in one-hidden-layer and two-hidden-layer predictive neural network modeling of machining surface roughness data , 2005 .

[55]  Leonard Ziemiański,et al.  Neural networks in mechanics of structures and materials – new results and prospects of applications , 2001 .

[56]  Lionel Tarassenko,et al.  Guide to Neural Computing Applications , 1998 .

[57]  Songwu Lu,et al.  Robust nonlinear system identification using neural-network models , 1998, IEEE Trans. Neural Networks.

[58]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[59]  Manuel Carcenac,et al.  A modular neural network for super-resolution of human faces , 2009, Applied Intelligence.

[60]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction , 2001 .

[61]  Huan Liu,et al.  Incremental Feature Selection , 1998, Applied Intelligence.

[62]  Israel González Carrasco,et al.  Análisis, optimización y evaluación de modelos de redes de neuronas artificiales para la clasificación y predicción de impactos de alta velocidad sobre distintos materiales , 2011 .

[63]  Yichuang Sun,et al.  Neural network-based L1-norm optimisation approach for fault diagnosis of nonlinear circuits with tolerance , 2001 .

[64]  Sergio Ricci,et al.  Optimization of Helicopter Subfloor Components Under Crashworthiness Requirements Using Neural Networks , 2002 .

[65]  Ángel García-Crespo,et al.  Dealing with limited data in ballistic impact scenarios: an empirical comparison of different neural network approaches , 2011, Applied Intelligence.

[66]  Kevin L. Priddy,et al.  Artificial neural networks - an introduction , 2005, Tutorial text series.

[67]  Ill-Soo Kim,et al.  A study on genetic algorithm to select architecture of a optimal neural network in the hot rolling process , 2004 .

[68]  Rashad J. Rasras,et al.  The Artificial Neural Network Based Approach for Mortality Structure Analysis , 2006 .

[69]  Ángel García-Crespo,et al.  Multilayer Perceptron Training Optimization for High Speed Impacts Classification , 2008, World Congress on Engineering.

[70]  Naonori Ueda,et al.  Optimal Linear Combination of Neural Networks for Improving Classification Performance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[71]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.