Auto-adaptive multi-scale Laplacian Pyramids for modeling non-uniform data

Abstract Kernel-based techniques have become a common way for describing the local and global relationships of data samples that are generated in real-world processes. In this research, we focus on a multi-scale kernel based technique named Auto-adaptive Laplacian Pyramids (ALP). This method can be useful for function approximation and interpolation. ALP is an extension of the standard Laplacian Pyramids model that incorporates a modified Leave-One-Out Cross Validation procedure, which makes the method stable and automatic in terms of parameters selection without extra cost. This paper introduces a new algorithm that extends ALP to fit datasets that are non-uniformly distributed. In particular, the optimal stopping criterion will be point-dependent with respect to the local noise level and the sample rate. Experimental results over real datasets highlight the advantages of the proposed multi-scale technique for modeling and learning complex, high dimensional data.

[1]  Ioannis G. Kevrekidis,et al.  Reduced models in chemical kinetics via nonlinear data-mining , 2013, 1307.6849.

[2]  Andrew J. Majda,et al.  Predicting regional and pan-Arctic sea ice anomalies with kernel analog forecasting , 2017, Climate Dynamics.

[3]  Quan Pan,et al.  Adaptive imputation of missing values for incomplete pattern classification , 2016, Pattern Recognit..

[4]  R. Beatson,et al.  Fast evaluation of radial basis functions : methods for two-dimensional polyharmonic splines , 1997 .

[5]  Ronald R. Coifman,et al.  Heterogeneous Datasets Representation and Learning using Diffusion Maps and Laplacian Pyramids , 2012, SDM.

[6]  Gavin C. Cawley,et al.  Fast exact leave-one-out cross-validation of sparse least-squares support vector machines , 2004, Neural Networks.

[7]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[8]  Israel Cohen,et al.  Multi-channel wafer defect detection using diffusion maps , 2014, 2014 IEEE 28th Convention of Electrical & Electronics Engineers in Israel (IEEEI).

[9]  I. Jolliffe.  Principal Component Analysis , 2002 .

[10]  E. Nadaraya.  On Estimating Regression , 1964 .

[11]  Keinosuke Fukunaga,et al.  Leave-One-Out Procedures for Nonparametric Error Estimates , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[13]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[14]  Israel Cohen,et al.  Graph-Based Supervised Automatic Target Detection , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Andrew L. Ferguson,et al.  Landmark diffusion maps (L-dMaps): Accelerated manifold learning out-of-sample extension , 2017, Applied and Computational Harmonic Analysis.

[16]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[17]  Dalia Fishelov,et al.  A new vortex scheme for viscous flow , 1990 .

[18]  K. Cios,et al.  Self-Organizing Feature Maps Identify Proteins Critical to Learning in a Mouse Model of Down Syndrome , 2015, PloS one.

[19]  Israel Cohen,et al.  Multiscale anomaly detection using diffusion maps and saliency score , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  R. Coifman,et al.  Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions , 2006 .

[21]  Guirong Liu,et al.  A point interpolation meshless method based on radial basis functions , 2002 .

[22]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[23]  Nicolas Duchateau,et al.  Adaptation of Multiscale Function Extension to Inexact Matching: Application to the Mapping of Individuals to a Learnt Manifold , 2013, GSI.

[24]  Salvatore Rampone,et al.  An incremental multivariate regression method for function approximation from noisy data , 2001, Pattern Recognit..

[25]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[26]  Israel Cohen,et al.  Multiscale Anomaly Detection Using Diffusion Maps , 2013, IEEE Journal of Selected Topics in Signal Processing.

[27]  Israel Cohen,et al.  Voice activity detection in transient noise environment using Laplacian pyramid algorithm , 2014, 2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC).

[28]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[29]  José R. Dorronsoro,et al.  Auto-adaptive Laplacian Pyramids , 2016, ESANN.

[30]  Trac D. Tran,et al.  Lifting-based Laplacian Pyramid reconstruction schemes , 2008, 2008 15th IEEE International Conference on Image Processing.

[31]  Andrew J. Majda,et al.  Data-driven prediction strategies for low-frequency patterns of North Pacific climate variability , 2015, Climate Dynamics.

[32]  Israel Cohen,et al.  Multisensory speech enhancement in noisy environments using bone-conducted and air-conducted microphones , 2014, 2014 IEEE China Summit & International Conference on Signal and Information Processing (ChinaSIP).

[33]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[34]  Minh N. Do,et al.  Framing pyramids , 2003, IEEE Trans. Signal Process..

[35]  Ronen Talmon,et al.  Nonlinear intrinsic variables and state reconstruction in multiscale simulations. , 2013, The Journal of chemical physics.

[36]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[37]  David G. Stork,et al.  Pattern Classification , 1973 .

[38]  Li Li,et al.  Adjusted weight voting algorithm for random forests in handling missing values , 2017, Pattern Recognit..

[39]  Neta Rabin,et al.  Missing Data Completion Using Diffusion Maps and Laplacian Pyramids , 2017, ICCSA.

[40]  Zhizhen Zhao,et al.  Kernel Analog Forecasting of Tropical Intraseasonal Oscillations , 2017 .