Limited Data Modelling Approaches for Engineering Applications

In real-world situation, the process of data collection can be challenging and resource intensive, due to being costly, time consuming, and compute intensive. Thus, the amount of data needed to build accurate models is often limited. System identification, making decisions, and prediction based on limited data reduce the production yields, increase the production costs, and decrease the competitiveness of the enterprises; hence, developing an appropriate data model with smaller variance of forecasting error and good accuracy based on these small data sets helps the enterprises to meet the competitive environment. However, the mathematical deterministic approaches that solve problems based on existing theories with few amount of data are not part of this chapter. The chapter aims to review common data modelling techniques for limited data based on heuristic approaches. This review also provides an overview of some of the research to date on data modelling techniques with limited data for various engineering application areas.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[3]  Vladimir Cherkassky,et al.  Learning from Data: Concepts, Theory, and Methods , 1998 .

[4]  Claudio Moraga,et al.  A diffusion-neural-network for learning from small samples , 2004, Int. J. Approx. Reason..

[5]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[6]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[7]  Mirjana Minceva,et al.  Modelling and simulation in chemical engineering: Tools for process innovation , 2005, Comput. Chem. Eng..

[8]  Abbas S. Milani,et al.  Predictive modelling and optimization of carbon fiber mechanical properties through high temperature furnace , 2017 .

[9]  Manish Varma Datla.  Bench marking of classification algorithms: Decision Trees and Random Forests - a case study using R , 2015, 2015 International Conference on Trends in Automation, Communications and Computing Technology (I-TACT-15).

[10]  A. B. M. S. Ali,et al.  Dynamic and Advanced Data Mining for Progressing Technological Development: Innovations and Systemic Approaches , 2009 .

[11]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[12]  Bruce Ratner,et al.  Statistical and Machine-Learning Data Mining: Techniques for Better Predictive Modeling and Analysis of Big Data , 2011 .

[13]  J. Douglas Barrett,et al.  Taguchi's Quality Engineering Handbook , 2007, Technometrics.

[14]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[15]  Xitao Fan,et al.  Comparability of Jackknife and Bootstrap Results: An Investigation for a Case of Canonical Correlation Analysis. , 1996 .

[16]  Chun-Xia Zhang,et al.  An efficient modified boosting method for solving classification problems , 2008 .

[17]  Q. Pham.  Dynamic optimization of chemical engineering processes by an evolutionary method , 1998 .

[18]  Der-Chiang Li,et al.  A genetic algorithm-based virtual sample generation technique to improve small data set learning , 2014, Neurocomputing.

[19]  Jules Thibault,et al.  Process modeling with neural networks using small experimental datasets , 1999 .

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  Kate Smith-Miles,et al.  On learning algorithm selection for classification , 2006, Appl. Soft Comput..

[22]  Abbas M. Abd,et al.  Modelling the strength of lightweight foamed concrete using support vector machine (SVM) , 2017 .

[23]  Sanjeev R. Kulkarni,et al.  An Elementary Introduction to Statistical Learning Theory , 2011 .

[24]  Fu Xiao,et al.  A short-term building cooling load prediction method using deep learning algorithms , 2017 .

[25]  Davy Janssens,et al.  Integrating Bayesian networks and decision trees in a sequential rule-based transportation model , 2006, Eur. J. Oper. Res..

[26]  Jörg Kindermann,et al.  Content Classification of Multimedia Documents using Partitions of Low-Level Features , 2006, J. Virtual Real. Broadcast..

[27]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Keunje Yoo,et al.  Decision tree-based data mining and rule induction for identifying hydrogeological parameters that influence groundwater pollution sensitivity , 2016 .

[29]  Hamid Khayyam,et al.  Stochastic optimization models for energy management in carbonization process of carbon fiber production , 2015 .

[30]  Hamid Khayyam,et al.  Dynamic Prediction Models and Optimization of Polyacrylonitrile (PAN) Stabilization Processes for Production of Carbon Fiber , 2015, IEEE Transactions on Industrial Informatics.

[31]  E. Yesilnacar,et al.  Landslide susceptibility mapping : A comparison of logistic regression and neural networks methods in a medium scale study, Hendek Region (Turkey) , 2005 .

[32]  Roman M. Balabin,et al.  Support vector machine regression (SVR/LS-SVM)--an alternative to neural networks (ANN) for analytical chemistry? Comparison of nonlinear methods on near infrared (NIR) spectroscopy data. , 2011, The Analyst.

[33]  Amir H. Mohammadi,et al.  Application of decision tree learning in modelling CO2 equilibrium absorption in ionic liquids , 2017 .

[34]  T. Shatovskaya,et al.  Application of the Bayesian Networks In the Informational Modeling , 2006, 2006 International Conference - Modern Problems of Radio Engineering, Telecommunications, and Computer Science.

[35]  H J Motulsky,et al.  Fitting curves to data using nonlinear regression: a practical and nonmathematical review , 1987, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[36]  Hong Liu,et al.  Analog circuit fault diagnosis using bagging ensemble method with cross-validation , 2009, 2009 International Conference on Mechatronics and Automation.

[37]  J. Paulo. Davim,et al.  Design of Experiments in Production Engineering , 2016 .

[38]  Haichao Zhu,et al.  A New Method to Assist Small Data Set Neural Network Learning , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[39]  Philip H. Crowley,et al.  RESAMPLING METHODS FOR COMPUTATION-INTENSIVE DATA ANALYSIS IN ECOLOGY AND EVOLUTION , 1992 .

[40]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[41]  Sheldon M. Ross,et al.  Introduction to Probability and Statistics for Engineers and Scientists , 1987 .

[42]  Kan Wang,et al.  Control rod position reconstruction based on K-Nearest Neighbor Method , 2017 .

[43]  Huang Zhen,et al.  Application of statistical learning theory to predict corrosion rate of injecting water pipeline , 2010, 9th IEEE International Conference on Cognitive Informatics (ICCI'10).

[44]  Fengming M. Chang,et al.  Using data continualization and expansion to improve small data set learning accuracy for early flexible manufacturing system (FMS) scheduling , 2006 .

[45]  Chapter 37 – Resampling Statistics , 2015 .

[46]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[47]  Jun Gu,et al.  Application of support vector machine and genetic algorithm optimization for quality prediction within complex industrial process , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[48]  Hamid Khayyam,et al.  Development of a predictive model for study of skin-core phenomenon in stabilization process of PAN precursor , 2017 .

[49]  David J. Brown,et al.  A survey on computational intelligence approaches for predictive modeling in prostate cancer , 2017, Expert Syst. Appl..

[50]  H. Troy Nagle,et al.  Performance of the Levenberg–Marquardt neural network training method in electronic nose applications , 2005 .

[51]  Li Fu,et al.  The Research Survey of System Identification Method , 2013, 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics.

[52]  Chulwoo Han,et al.  Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies , 2017, Expert Syst. Appl..

[53]  Hao Yu,et al.  Selection of Proper Neural Network Sizes and Architectures—A Comparative Study , 2012, IEEE Transactions on Industrial Informatics.

[54]  Li Yao,et al.  Argumentation Based Joint Learning: A Novel Ensemble Learning Approach , 2015, PloS one.

[55]  G. Cawley,et al.  Efficient model selection for kernel logistic regression , 2004, ICPR 2004.

[56]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[57]  J. Paulo Davim,et al.  Computational Methods for Optimizing Manufacturing Technology: Models and Techniques , 2012 .

[58]  Douglas Curran-Everett,et al.  Explorations in statistics: permutation methods. , 2012, Advances in physiology education.

[59]  Isah A. Lawal,et al.  Predictive Modeling of Material Properties Using GMDH-based Abductive Networks , 2011, 2011 Fifth Asia Modelling Symposium.

[60]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[61]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[62]  Peter Exterkate,et al.  Model selection in kernel ridge regression , 2013, Comput. Stat. Data Anal..

[63]  Pietro Borghesani,et al.  Using support vector machines for the computationally efficient identification of acceptable design parameters in computer-aided engineering applications , 2017, Expert Syst. Appl..

[64]  Der-Chiang Li,et al.  Utilize bootstrap in small data set learning for pilot run modeling of manufacturing systems , 2008, Expert Syst. Appl..

[65]  Jin Huang,et al.  Support-vector modeling and optimization for microwave filters manufacturing using small data sets , 2012, IEEE 10th International Conference on Industrial Informatics.

[66]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[67]  Tapani Raiko,et al.  Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values , 2022 .

[68]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[69]  Bengt Andersson,et al.  Mathematical Modeling in Chemical Engineering , 2014 .

[70]  Sanjeev R. Kulkarni,et al.  An Elementary Introduction to Statistical Learning Theory: Kulkarni/Statistical Learning Theory , 2011 .

[71]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[72]  Banu Diri,et al.  Practical development of an Eclipse-based software fault prediction tool using Naive Bayes algorithm , 2011, Expert Syst. Appl..

[73]  Tomaso Poggio,et al.  Incorporating prior information in machine learning by creating virtual examples , 1998, Proc. IEEE.

[74]  T. Dobre,et al.  Chemical engineering : modelling, simulation and similitude , 2007 .

[75]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[76]  M. J. D. Powell,et al.  A Method for Minimizing a Sum of Squares of Non-Linear Functions Without Calculating Derivatives , 1965, Comput. J..

[77]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[78]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[79]  Giles M. Foody,et al.  Evaluation of SVM, RVM and SMLR for Accurate Image Classification With Limited Ground Data , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[80]  Kuldip K. Paliwal,et al.  Linear discriminant analysis for the small sample size problem: an overview , 2014, International Journal of Machine Learning and Cybernetics.

[81]  Xiao-bin Zhang,et al.  Tourism demand forecasting by support vector regression and genetic algorithm , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[82]  Huan Wang,et al.  A Short-term Traffic Flow Forecasting Method Based on the Hybrid PSO-SVR , 2015, Neural Processing Letters.

[83]  Der-Chiang Li,et al.  A grey-based fitting coefficient to build a hybrid forecasting model for small data sets , 2012 .

[84]  Nidhi H. Ruparel,et al.  Learning from Small Data Set to Build Classification Model: A Survey , 2013 .

[85]  Yanlin He,et al.  A PSO based virtual sample generation method for small sample sets: Applications to regression datasets , 2017, Eng. Appl. Artif. Intell..

[86]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[87]  Chongfu Huang,et al.  Information Diffusion Techniques and Small-Sample Problem , 2002, Int. J. Inf. Technol. Decis. Mak..

[88]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[89]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[90]  S. Gunn.  Support Vector Machines for Classification and Regression , 1998 .