Threefold vs. fivefold cross validation in one-hidden-layer and two-hidden-layer predictive neural network modeling of machining surface roughness data

Abstract Predictability of a manufacturing process or system is vital in virtual manufacturing. Various data mining techniques are available in developing predictive models. Cross validation is critical in determining the quality of a predictive model and the costs in data collection and data mining. Several cross-validation (CV) techniques are available, including the -fold CV, leave-one-out CV, and the bootstrap type of CV. Some past studies have not revealed any statistical advantages of using tenfold cross validation over fivefold cross validation. Determining the number of hidden layers is important in predictive modeling with neural networks. This study attempts to compare the performance of fivefold over threefold CV and that of one-hidden-layer over two-hidden-layer neural nets in predictive modeling for surface roughness parameters defined in ISO 13565 for turning and honing. Statistical hypothesis tests and different prediction errors are employed to compare the competitive models. This study does not reveal any significant statistical advantages of using fivefold CV over threefold CV and of using two-hidden-layer neural nets over one-hidden-layer neural nets for the cases under study. Furthermore, the procedure presented here is applicable in comparing competitive data modeling or data mining methods.

[1]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[2]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[3]  Salvatore Ingrassia,et al.  Neural Network Modeling for Small Datasets , 2005, Technometrics.

[4]  John F. Kolen,et al.  Backpropagation is Sensitive to Initial Conditions , 1990, Complex Syst..

[5]  Robert Groth,et al.  Data mining - a hands-on approach for business professionals , 1997 .

[6]  Huaiyu Zhu,et al.  No Free Lunch for Cross-Validation , 1996, Neural Computation.

[7]  G. Mirchandani,et al.  On hidden nodes for neural nets , 1989 .

[8]  Chang-Xue Feng,et al.  Subset selection in predictive modeling of the CMM digitization uncertainty , 2002 .

[9]  Andrew Kusiak,et al.  Computational Intelligence in Design and Manufacturing , 2000 .

[10]  Ping Zhang.  Model Selection Via Multifold Cross Validation , 1993 .

[11]  Robert Tibshirani,et al.  A Comparison of Some Error Estimates for Neural Network Models , 1996, Neural Computation.

[12]  Douglas C. Montgomery,et al.  Applied Statistics and Probability for Engineers, Third edition , 1994 .

[13]  L. Breiman.  The Little Bootstrap and other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error , 1992 .

[14]  Philip D. Wasserman,et al.  Neural computing - theory and practice , 1989 .

[15]  Chang-Xue Feng,et al.  Surface roughness predictive modeling: neural networks versus regression , 2003 .

[16]  L. Breiman,et al.  Submodel selection and evaluation in regression. The X-random case , 1992 .

[17]  Chang-Xue Jack Feng,et al.  Data mining techniques applied to predictive modeling of the knurling process , 2004 .

[18]  Chang-Xue Jack Feng,et al.  Neural networks modeling of honing surface roughness parameters defined by ISO 13565 , 2002 .

[19]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[20]  R. Tibshirani,et al.  An Introduction to the Bootstrap , 1995 .

[21]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[22]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[23]  C. F. Jeff Wu,et al.  Experiments: Planning, Analysis, and Parameter Design Optimization , 2000 .

[24]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[25]  David W. Coit,et al.  STATIC NEURAL NETWORK PROCESS MODELS : CONSIDERATIONS AND CASE STUDIES , 1998 .

[26]  Alice E. Smith,et al.  Bias and variance of validation methods for function approximation neural networks under conditions of sparse data , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[27]  D. G. Bounds,et al.  A multilayer perceptron network for the diagnosis of low back pain , 1988, IEEE 1988 International Conference on Neural Networks.

[28]  Andrew Kusiak,et al.  Selection and validation of predictive regression and neural network models based on designed experiments , 2006 .

[29]  E. George.  The Variable Selection Problem , 2000 .

[30]  J. Shao.  Linear Model Selection by Cross-validation , 1993 .

[31]  Kevin Swingler,et al.  Applying neural networks - a practical guide , 1996 .

[32]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[33]  Ronald D. Snee,et al.  Validation of Regression Models: Methods and Examples , 1977 .

[34]  P. Burman.  A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods , 1989 .

[35]  Neil Gershenfeld,et al.  The nature of mathematical modeling , 1998 .

[36]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .