Evidence optimization for consequently generated models

Abstract To construct an adequate regression model one has to fulfill the set of measured features with their generated derivatives. Often the number of these features exceeds the number of the samples in the data set. After a feature generation process the problem of feature selection from a set of highly correlated features arises. The proposed algorithm uses an evidence maximization procedure to select a model as a subset of generated features. During the selection process it rejects multicollinear features. A problem of European option volatility modeling illustrates the algorithm. Its performance is compared with the performances of similar well-known algorithms.

[1]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  A. P. Dawid,et al.  Generative or Discriminative? Getting the Best of Both Worlds , 2007 .

[4]  R. Stolzenberg,et al.  Multiple Regression Analysis , 2004 .

[5]  C. L. Mallows.  Some comments on C_p , 1973 .

[6]  John H. Maindonald,et al.  Modern Multivariate Statistical Techniques: Regression, Classification and Manifold Learning , 2009 .

[7]  Dick den Hertog,et al.  Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming , 2009, IEEE Transactions on Evolutionary Computation.

[8]  A. McQuarrie,et al.  Regression and Time Series Model Selection , 1998 .

[9]  David A. Belsley,et al.  Conditioning Diagnostics: Collinearity and Weak Data in Regression , 1991 .

[10]  Ilkay Ulusoy,et al.  Generative versus discriminative methods for object recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[12]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[13]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  L. Hogben.  Handbook of Linear Algebra , 2006 .

[16]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[17]  J. Hull.  Options, Futures, and Other Derivatives , 1989 .

[18]  Noelle Foreshaw.  Options… , 2010 .

[19]  C. Mallows.  Some Comments on Cp , 2000, Technometrics.

[20]  Christopher M. Bishop,et al.  A New Framework for Machine Learning , 2008, WCCI.

[21]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[22]  Laverne W. Stanton,et al.  Applied Regression Analysis: A Research Tool , 1990 .

[23]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[24]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[25]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[26]  M. Fireman,et al.  MULTIPLE REGRESSION ANALYSIS OF SOIL DATA , 1954 .

[27]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[28]  I. Zelinka,et al.  ANALYTIC PROGRAMMING – SYMBOLIC REGRESSION BY MEANS OF ARBITRARY EVOLUTIONARY ALGORITHMS , 2005 .