An Intelligent Model Validation Method Based on ECOC SVM

This paper develops an intelligent model validation method based on error correcting output coding support vector machine (ECOC SVM). The similarity analysis between simulation time series from computerized model and observed time series from real-world system is formulated as a multi-class classification problem. The ECOC framework, built on the basis of the error correcting principles of communication theory, decomposes the multi-class classification task as multiple binary classification problems. The SVM is used as the base classifier and a set of similarity measure methods is applied to extract the input features. Compared to conventional methods, the proposed validation method based on ECOC SVM incorporates multiple similarity measures to a comprehensive similarity measure and can learn to predict the credibility level from training samples. The application result reveals that the classification accuracy achieved 82%, which means the proposed method is promising for the similarity analysis of large datasets.

[1]  Louis G. Birta,et al.  A knowledge-based approach for the validation of simulation models: the foundation , 1996, TOMC.

[2]  Nima Hatami,et al.  Thinned-ECOC ensemble based on sequential code shrinking , 2012, Expert Syst. Appl..

[3]  Tryphon T. Georgiou,et al.  Validating Aircraft Models in the Gap Metric , 2014 .

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Xiangyang Xue,et al.  A simplified multi-class support vector machine with reduced dual optimization , 2012, Pattern Recognit. Lett..

[6]  Davide Ballabio,et al.  Evaluation of model predictive ability by external validation techniques , 2010 .

[7]  Robert G. Sargent,et al.  Verification and validation of simulation models , 2013, Proceedings of Winter Simulation Conference.

[8]  Gilles Notton,et al.  Estimation of hourly global solar irradiation on tilted planes from horizontal one using artificial neural networks , 2012 .

[9]  James L. McKenney.  Critique of: "Verification of Computer Simulation Models" , 1967 .

[10]  Andrew G. Dempster,et al.  Determining the best vector distance measure for use in location fingerprinting , 2015, Pervasive Mob. Comput..

[11]  Nils Goerke,et al.  Learning Time-Series Similarity with a Neural Network by Combining Similarity Measures , 2006, ICANN.

[12]  Nuno M. Garcia,et al.  Classification techniques on computerized systems to predict and/or to detect Apnea: A systematic review , 2017, Comput. Methods Programs Biomed..

[13]  Ning Xiao-le.  Research on Comprehensive Validation of Simulation Models Based on Improved Grey Relational Analysis , 2016 .

[14]  Gilles Notton,et al.  Neural network approach to estimate 10-min solar global irradiation values on tilted planes , 2013 .

[15]  Wesley W. Chu,et al.  The phrase-based vector space model for automatic retrieval of free-text medical documents , 2007, Data Knowl. Eng..

[16]  G. Lampeas,et al.  On the validation of solid mechanics models using optical measurements and data decomposition , 2015, Simul. Model. Pract. Theory.

[17]  J. A. Carta,et al.  Comparison of several measure-correlate-predict models using support vector regression techniques to estimate wind power densities. A case study , 2017 .

[18]  Ming Yang,et al.  Knowledge-based method for the validation of complex simulation models , 2010, Simul. Model. Pract. Theory.

[19]  Adam Hapij,et al.  Mapping model validation metrics to subject matter expert scores for model adequacy assessment , 2014, Reliab. Eng. Syst. Saf..