Mutual information for the selection of relevant variables in spectrometric nonlinear modelling

Data from spectrophotometers form vectors of a large number of exploitable variables. Building quantitative models using these variables most often requires using a smaller set of variables than the initial one. Indeed, a too large number of input variables to a model results in a too large number of parameters, leading to overfitting and poor generalization abilities. In this paper, we suggest the use of the mutual information measure to select variables from the initial set. The mutual information measures the information content in input variables with respect to the model output, without making any assumption on the model that will be used; it is thus suitable for nonlinear modelling. In addition, it leads to the selection of variables among the initial set, and not to linear or nonlinear combinations of them. Without decreasing the model performances compared to other variable projection methods, it allows therefore a greater interpretability of the results.

[1]  Michel Verleysen,et al.  Model Selection with Cross-Validations and Bootstraps - Application to Time Series Prediction with RBFN Models , 2003, ICANN.

[2]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[3]  Michel José Anzanello,et al.  Chemometrics and Intelligent Laboratory Systems , 2009 .

[4]  Yukihiro Ozaki,et al.  Potential of Near-Infrared Fourier Transform Raman Spectroscopy in Food Analysis , 1992 .

[5]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[6]  R. L. Dobrushin.  A Simplified Method of Experimentally Evaluating the Entropy of a Stationary Sequence , 1958 .

[7]  Michel Verleysen,et al.  On the Kernel Widths in Radial-Basis Function Networks , 2003, Neural Processing Letters.

[8]  Joos Vandewalle,et al.  Special issue on fundamental and information processing aspects of neurocomputing , 2002, Neurocomputing.

[9]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[10]  J. Mason,et al.  Algorithms for approximation , 1987 .

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  Oldrich A Vasicek,et al.  A Test for Normality Based on Sample Entropy , 1976 .

[13]  Alexander Kraskov,et al.  Least-dependent-component analysis based on mutual information. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[15]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[16]  Peter Grassberger,et al.  Generalizations of the Hausdorff dimension of fractal measures , 1985 .

[17]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[18]  D. Franc,et al.  Spectrophotometric variable selection by mutual information , 2004 .

[19]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[20]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[21]  Michel Verleysen,et al.  Representation of functional data in neural networks , 2005, Neurocomputing.

[22]  P. K. Aldridge,et al.  On-line monitoring of powder blend homogeneity by near-infrared spectroscopy. , 1996, Analytical chemistry.

[23]  Michel Verleysen,et al.  Nonlinear Projection with the Isotop Method , 2002, ICANN.

[24]  Jordi Coello,et al.  Analytical control of pharmaceutical production steps by near infrared reflectance spectroscopy , 1999 .

[25]  Johan A. K. Suykens,et al.  Sparse least squares Support Vector Machine classifiers , 2000, ESANN.

[26]  R. L. Somorjai,et al.  Methods for Estimating the Intrinsic Dimsnionality of High-Dimensional Point Sets , 1986 .

[27]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[28]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[29]  Marcelo Blanco,et al.  Determination of Finishing Oils in Acrylic Fibres by Near-infrared Reflectance Spectrometry , 1997 .

[30]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[31]  Michel Verleysen,et al.  Spectrophotometric variable selection by mutual information , 2004 .

[32]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[33]  Gottfried Mayer-Kress,et al.  Dimensions and Entropies in Chaotic Systems , 1986 .

[34]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.