Local classification: Locally weighted-partial least squares-discriminant analysis (LW-PLS-DA).

The possibility of devising a simple, flexible and accurate non-linear classification method, by extending the locally weighted partial least squares (LW-PLS) approach to the cases where the algorithm is used in a discriminant way (partial least squares discriminant analysis, PLS-DA), is presented. In particular, to assess which category an unknown sample belongs to, the proposed algorithm operates by identifying which training objects are most similar to the one to be predicted and building a PLS-DA model using these calibration samples only. Moreover, the influence of the selected training samples on the local model can be further modulated by adopting a not uniform distance-based weighting scheme which allows the farthest calibration objects to have less impact than the closest ones. The performances of the proposed locally weighted-partial least squares-discriminant analysis (LW-PLS-DA) algorithm have been tested on three simulated data sets characterized by a varying degree of non-linearity: in all cases, a classification accuracy higher than 99% on external validation samples was achieved. Moreover, when also applied to a real data set (classification of rice varieties), characterized by a high extent of non-linearity, the proposed method provided an average correct classification rate of about 93% on the test set. By the preliminary results, showed in this paper, the performances of the proposed LW-PLS-DA approach have proved to be comparable and in some cases better than those obtained by other non-linear methods (k nearest neighbors, kernel-PLS-DA and, in the case of rice, counterpropagation neural networks).

[1]  Edwin Lughofer,et al.  NIR-based quantification of process parameters in polyetheracrylat (PEA) production using flexible non-linear fuzzy systems , 2011 .

[2]  W. Cleveland Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[3]  José Manuel Amigo,et al.  Beer fermentation: monitoring of process parameters by FT-NIR and multivariate data analysis. , 2014, Food chemistry.

[4]  Masami Ito,et al.  Task decomposition and module combination based on class relations: a modular neural network for pattern classification , 1999, IEEE Trans. Neural Networks.

[5]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[6]  Aleix M. Martínez,et al.  Subclass discriminant analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  D. Massart,et al.  Application of Radial Basis Functions — Partial Least Squares to non-linear pattern recognition problems: diagnosis of process faults , 1996 .

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Federico Marini,et al.  Application of near infrared (NIR) spectroscopy coupled to chemometrics for dried egg-pasta characterization and egg content quantification. , 2013, Food chemistry.

[10]  Arnold P. Boedihardjo,et al.  On Locally Linear Classification by Pairwise Coupling , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[11]  Edwin Lughofer,et al.  Evolving chemometric models for predicting dynamic process parameters in viscose production. , 2012, Analytica chimica acta.

[12]  Yoram Singer,et al.  Logistic Regression, AdaBoost and Bregman Distances , 2000, Machine Learning.

[13]  S. Wold,et al.  Partial least squares analysis with cross‐validation for the two‐class problem: A Monte Carlo study , 1987 .

[14]  Beata Walczak,et al.  Dissimilarity partial least squares applied to non-linear modeling problems , 2012 .

[15]  T. Næs,et al.  Locally weighted regression and scatter correction for near-infrared reflectance data , 1990 .

[16]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[17]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[18]  Yizeng Liang,et al.  Exploring nonlinear relationships in chemical data using kernel-based methods , 2011 .

[19]  John L. Casti,et al.  A new initial-value method for on-line filtering and estimation (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[20]  T. Næs,et al.  Locally Weighted Regression in Diffuse Near-Infrared Transmittance Spectroscopy , 1992 .

[21]  D L Massart,et al.  Optimization in locally weighted regression. , 1998, Analytical chemistry.

[22]  M. Barker,et al.  Partial least squares for discrimination , 2003 .

[23]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[24]  Rong Jin,et al.  Efficient Algorithm for Localized Support Vector Machine , 2010, IEEE Transactions on Knowledge and Data Engineering.

[25]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[27]  J. Zupan,et al.  On the use of counterpropagation artificial neural networks to characterize Italian rice varieties , 2004 .

[28]  Edwin Lughofer,et al.  Hybrid adaptive calibration methods and ensemble strategy for prediction of cloud point in melamine resin production , 2013 .

[29]  Edwin Lughofer,et al.  Reliable All-Pairs Evolving Fuzzy Classifiers , 2013, IEEE Transactions on Fuzzy Systems.

[30]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[31]  Ronald D. Snee,et al.  Validation of Regression Models: Methods and Examples , 1977 .

[32]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Tomas Isaksson,et al.  Splitting of calibration data by cluster analysis , 1991 .

[34]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[35]  Manabu Kano,et al.  Development of soft-sensor using locally weighted PLS with adaptive similarity measure , 2013 .

[36]  Peter E. Hart,et al.  The condensed nearest neighbor rule (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[37]  Lutgarde M. C. Buydens,et al.  Interpretation and Visualization of Non-Linear Data Fusion in Kernel Space: Study on Metabolomic Characterization of Progression of Multiple Sclerosis , 2012, PloS one.

[38]  L. A. Stone,et al.  Computer Aided Design of Experiments , 1969 .