Measurement of Fitness Function efficiency using Data Envelopment Analysis

Over the last years, Evolutionary Algorithms (EAs) have been proposed aiming to find the best configuration of the Artificial Neural Networks (ANN) parameters. Among several parameters of an EA that can influence the quality of the found solution, the choice of the Fitness Function is the most important for its effectiveness and efficiency, given that different Fitness Functions have distinct fitness landscapes. In other words, the Fitness Function guides the evolutionary process of the candidate solutions according with a given criterion of the performance. However, there is not an universal criterion to identify the best performance measure. Thus, what is the Fitness Function more efficient among a set of several possible options? This paper presents a methodology based on Data Envelopment Analysis (DEA) to find the more efficient Fitness Function among candidates. The DEA is used to determine the best combination of statistical measures to build the more efficient Fitness Function for a EA. The case study employed here consists of a hybrid system composed by Evolutionary Strategy and ANN applied to solve the time series forecasting problem. The data analyzed are composed by financial, agribusiness and natural phenomena. The results show that establishment of the Fitness Function is a crucial point in the EA design, being a key factor to obtain the best solution for a limited number of EA's iteration.

[1]  Fred L. Collopy,et al.  Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons , 1992 .

[2]  Léopold Simar,et al.  Non-parametric tests of returns to scale , 2002, Eur. J. Oper. Res..

[3]  Cláudia S. Sarrico,et al.  Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software , 2001, J. Oper. Res. Soc..

[4]  George D. C. Cavalcanti,et al.  AutoAssociative Pyramidal Neural Network for one class pattern classification with implicit feature extraction , 2013, Expert Syst. Appl..

[5]  Mehdi Toloo,et al.  A new method for ranking discovered rules from data mining by DEA , 2009, Expert Syst. Appl..

[6]  Aranildo R. Lima,et al.  Downscaling temperature and precipitation using support vector regression with evolutionary strategy , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[7]  Jiwen Dong,et al.  Time-series forecasting using flexible neural tree model , 2005, Inf. Sci..

[8]  M. Farrell.  The Measurement of Productive Efficiency , 1957 .

[9]  Gongbing Bi,et al.  Input/output indicator selection for DEA efficiency evaluation: An empirical study of Chinese commercial banks , 2012, Expert Syst. Appl..

[10]  Abstmct,et al.  Hypothesis Tests Using Data Envelopment Analysis , 2022 .

[11]  Emmanuel Thanassoulis,et al.  Introduction to the Theory and Application of Data Envelopment Analysis: A Foundation Text with Integrated Software , 2001 .

[12]  Harris Wu,et al.  The effects of fitness functions on genetic programming-based ranking discovery for Web search: Research Articles , 2004 .

[13]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[14]  P. Bogetoft,et al.  Benchmarking with DEA, SFA, and R , 2011 .

[15]  Peter Merz,et al.  Advanced Fitness Landscape Analysis and the Performance of Memetic Algorithms , 2004, Evolutionary Computation.

[16]  A. Charnes,et al.  Data Envelopment Analysis Theory, Methodology and Applications , 1995 .

[17]  T. Ueda,et al.  APPLICATION OF PRINCIPAL COMPONENT ANALYSIS FOR PARSIMONIOUS SUMMARIZATION OF DEA INPUTS AND/OR OUTPUTS , 1997 .

[18]  Jimmy Shadbolt,et al.  Neural Networks and the Financial Markets: "Predicting, Combining And Portfolio Optimisation" , 2002 .

[19]  C. B. Tilanus,et al.  Applied Economic Forecasting , 1966 .

[20]  W. Cooper,et al.  Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software , 1999 .

[21]  Dalibor Petkovic,et al.  Adaptive neuro fuzzy selection of heart rate variability parameters affected by autonomic nervous system , 2013, Expert Syst. Appl..

[22]  Sverre A.C. Kittelsen,et al.  Stepwise DEA; Choosing Variables for Measuring Technical Efficiency in Norwegian Electricity Distribution , 1993 .

[23]  Araceli Sanchis,et al.  Parallelization of an evolving Artificial Neural Networks system to Forecast Time Series using OPENMP and MPI , 2012, 2012 IEEE Conference on Evolving and Adaptive Intelligent Systems.

[24]  P. W. Wilson,et al.  Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models , 1998 .

[25]  T. Senjyu,et al.  Notice of Violation of IEEE Publication PrinciplesA Hybrid ARIMA and Neural Network Model for Short-Term Price Forecasting in Deregulated Market , 2010, IEEE Transactions on Power Systems.

[26]  Ping-Feng Pai,et al.  Forecasting regional electricity load based on recurrent support vector machines with genetic algorithms , 2005 .

[27]  Tiago Alessandro Espínola Ferreira,et al.  An experimental study of fitness function and time series forecasting using artificial neural networks , 2010, GECCO '10.

[28]  Shahaboddin Shamshirband,et al.  Adaptive neuro-fuzzy selection of the optimal parameters of protective spur dike , 2014, Natural Hazards.

[29]  A. Charnes,et al.  Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis , 1984 .

[30]  A. U.S.,et al.  Measuring the efficiency of decision making units , 2003 .

[31]  Brendan Kitts,et al.  Intellectual capital: from intangible assets to fitness landscapes , 2001, Expert Syst. Appl..

[32]  K. Siakavara,et al.  Application of a Composite Differential Evolution Algorithm in Optimal Neural Network Design for Propagation Path-Loss Prediction in Mobile Communication Systems , 2013, IEEE Antennas and Wireless Propagation Letters.

[33]  Leâ Opold Simar,et al.  A general methodology for bootstrapping in non-parametric frontier models , 2000 .

[34]  Gwo-Ching Liao.  Application a Novel Evolutionary Computation Algorithm for Load Forecasting of Air Conditioning , 2012, 2012 Asia-Pacific Power and Energy Engineering Conference.

[35]  Ahmad Makui,et al.  An empirical study for ranking insurance firms using a hybrid of data envelopment analysis and neural network , 2012 .

[36]  Johannes Ledolter,et al.  Statistical methods for forecasting , 1983 .

[37]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[38]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[39]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[40]  Yan Guo,et al.  Evolutionary Neural Networks Applied to Land-cover Classification in Zhaoyuan, China , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[41]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[42]  Martin Stepnicka,et al.  Forecasting seasonal time series with computational intelligence: On recent methods and the potential of their combinations , 2013, Expert Syst. Appl..

[43]  V. Meganck,et al.  Genetic Algorithm for Parameters Optimization of ANN-based Speed Controller , 2007, EUROCON 2007 - The International Conference on "Computer as a Tool".

[44]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[45]  Xiaodong Li,et al.  Time series forecasting by evolving artificial neural networks with genetic algorithms, differential evolution and estimation of distribution algorithm , 2011, Neural Computing and Applications.

[46]  Emmanuel Thanassoulis,et al.  Introduction to the theory and application of data envelopment analysis , 2001 .

[47]  Elwira Tomczak,et al.  Application of ANN and EA for description of metal ions sorption on chitosan foamed structure - Equilibrium and dynamics of packed column , 2011, Comput. Chem. Eng..

[48]  Teresa Bernarda Ludermir,et al.  Optimization of the weights and asymmetric activation function family of neural network for time series forecasting , 2013, Expert Syst. Appl..

[49]  N. Amjady,et al.  Day-Ahead Price Forecasting of Electricity Markets by Mutual Information Technique and Cascaded Neuro-Evolutionary Algorithm , 2009, IEEE Transactions on Power Systems.

[50]  Hui Ma,et al.  Pattern recognition techniques and their applications for automatic classification of artificial partial discharge sources , 2013, IEEE Transactions on Dielectrics and Electrical Insulation.

[51]  Hong Wang,et al.  How to measure adaptation complexity in evolvable systems - A new synthetic approach of constructing fitness functions , 2011, Expert Syst. Appl..

[52]  T. Senjyu,et al.  A Novel Approach to Forecast Electricity Price for PJM Using Neural Network and Similar Days Method , 2007, IEEE Transactions on Power Systems.

[53]  V.H. Hinojosa,et al.  Short-Term Load Forecasting Using Fuzzy Inductive Reasoning and Evolutionary Algorithms , 2010, IEEE Transactions on Power Systems.

[54]  Lee A. Belfore,et al.  Modeling faulted switched reluctance motors using evolutionary neural networks , 1997, IEEE Trans. Ind. Electron..

[55]  Desheng Dash Wu,et al.  Supplier selection: A hybrid model using DEA, decision tree and neural network , 2009, Expert Syst. Appl..

[56]  Rolf Färe,et al.  Production Frontiers: Introduction , 1993 .

[57]  Weizhong Yan,et al.  Toward Automatic Time-Series Forecasting Using Neural Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[58]  Jonchi Shyu,et al.  Measuring the true managerial efficiency of bank branches in Taiwan: A three-stage DEA analysis , 2012, Expert Syst. Appl..

[59]  Tiago Alessandro Espínola Ferreira,et al.  A New Intelligent System Methodology for Time Series Forecasting with Artificial Neural Networks , 2008, Neural Processing Letters.

[60]  Andrew L. Johnson,et al.  Guidelines for using variable selection techniques in data envelopment analysis , 2011, Eur. J. Oper. Res..

[61]  Damaris A Mwalo.  Application Of Principal Component Analysis InAgronomy , 1988 .

[62]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[63]  Emmanuel Thanassoulis,et al.  Applied data envelopment analysis , 1991 .

[64]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[65]  Md. Zakirul Alam Bhuiyan.  An Algorithm for Determining Neural Network Architecture Using Differential Evolution , 2009, 2009 International Conference on Business Intelligence and Financial Engineering.

[66]  Joseph Sarkis,et al.  Ecological modernization in the electrical utility industry: An application of a bads-goods DEA model of ecological and technical efficiency , 2012, Eur. J. Oper. Res..

[67]  Paulo S. G. de Mattos Neto,et al.  A prime step in the time series forecasting with hybrid methods: The fitness function choice , 2009, 2009 International Joint Conference on Neural Networks.

[68]  Yaakov Roll,et al.  An application procedure for DEA , 1989 .

[69]  SimarLeopold,et al.  Sensitivity Analysis of Efficiency Scores , 1998 .

[70]  Janet M. Wagner,et al.  Stepwise selection of variables in data envelopment analysis: Procedures and managerial perspectives , 2007, Eur. J. Oper. Res..

[71]  Our Medical Biochemistry Correspondent.  Milk Production , 1968, Nature.